KR20200000541A - Method for training artificial neurel network - Google Patents

Abstract

Disclosed is an information processing method, which is performed by at least one processor. The information processing method, which is performed by at least one processor, according to an embodiment of the present invention comprises the steps of: receiving first data; generating second data based on the first data by using a first computational model including a plurality of sub-models connected in series, wherein the sub-models include at least one layer for changing a dimension; and generating a cost function based on at least two pieces of the first data, the second data, and a plurality of sub-data generated by the plurality of sub-models.

Description

Learning method of artificial neural network {METHOD FOR TRAINING ARTIFICIAL NEUREL NETWORK}

TECHNICAL FIELD This disclosure relates to artificial neural networks, and more particularly, to a method of training artificial neural networks and data classification through trained neural networks.

Pattern recognition is a field of machine learning, and refers to the science of recognizing the regularity of patterns and data. Pattern recognition techniques include supervised learning and unsupervised learning methods. Supervised learning means a method in which an algorithm learns pattern recognition using data for which the result of pattern recognition has already been determined (called "training" data).

Various methods have been proposed to implement a pattern recognition technique, and US Patent 8,873,856 shows an example of such a pattern recognition technique.

However, regardless of the type and method of the pattern recognition algorithm, the more various and the more the training data, the higher quality learning is performed, so a technique for securing a large amount of training data is required. In particular, when the data are different pattern defects occurring in the manufacturing process, defects that may occur under the same manufacturing process tend to have a similar shape on different patterns, but the defect data may be processed separately for different patterns. There is discomfort that must be obtained through.

Accordingly, there is a need in the art for a technology capable of efficiently detecting defects on various patterns with training data composed of only normal data that is easy to secure.

United States Patent Registration No. US9,141,916B1 United States Patent Publication No. US2008 / 0222135A1

The present disclosure is devised in response to the above-described background, and is to provide a method for training an artificial neural network and data classification through the trained neural network.

In accordance with one embodiment of the present disclosure for solving the above problems, an information processing method performed in one or more processors is disclosed. The information processing method comprises: receiving, by a processor, first data; Generating, by a processor, second data based on the first data using a first computational model comprising a plurality of submodels connected in series, wherein the submodel includes at least one layer for changing dimensions box-; Generating, by a processor, a cost function based on at least two of the first data, the second data, and a plurality of sub-data generated by the plurality of sub-models; It may include.

In an alternative embodiment, generating the second data comprises: reducing, by a processor, the dimension of the first data using a first reduction layer included in a first submodel, Generating first sub data by recovering the reduced dimension of the first data using an included first recovery layer; And reduce, by a processor, the dimension of the first sub data using a second reduction layer included in a second sub-model, and use the first sub data using a second recovery layer included in the second sub-model. Generating the second data by recovering a reduced dimension of the; It may include.

In an alternative embodiment, the information processing method further comprises: training, by a processor, at least one of the plurality of submodels based on the cost function; It may include.

In an alternate embodiment, the cost function is:

And a term based on a relationship between at least two of the first data, the second data, and the plurality of sub-data.

In an alternative embodiment, the cost function may include: a term based on the relationship between the first data and the second data, and a term based on the relationship between the first data and the first sub data.

In an alternative embodiment, the first data may comprise: training data consisting of normal data.

In an alternative embodiment, the first data may comprise: image data.

In alternative embodiments, at least one of the plurality of submodels may comprise neural networks.

In an alternative embodiment, the neural network may comprise an auto-encoder or GAN (Generative Adversarial Networks).

In an alternative embodiment, the information processing method comprises: generating, by a processor, a second computational model that shares at least a portion of the first computational model; It may further include.

In an alternate embodiment, the second computational model may: share at least a portion of the first reduction layer and the second recovery layer.

In accordance with another embodiment of the present disclosure, an information processing method performed in one or more processors is disclosed. The information processing method comprises: receiving, by a processor, first data; Generating, by a processor, second data based on the first data using a first computational model comprising a plurality of submodels connected in series, wherein the submodel includes at least one layer that changes dimensions -; Generating, by a processor, test result information based on the first data and the second data; It may include.

In accordance with another embodiment of the present disclosure, an information processing method performed in one or more processors is disclosed. The information processing method comprises: receiving, by a processor, first data; Generating, by a processor, third data based on the first data using a second computational model that shares at least a portion of the first computational model; Generating, by a processor, test result information based on the first data and the third data; It may include.

In accordance with yet another embodiment of the present disclosure, a computing device is disclosed. The computing device comprises: one or more processors; And a memory storing instructions executable in the one or more processors; Wherein the one or more processors comprise: receiving first data; Generating second data based on the first data using a first computational model comprising a plurality of sub-models connected in series, the sub-model comprising at least one layer that changes dimensions; And generating a cost function based at least in part on the first data, the second data, and the plurality of sub-data generated by the plurality of sub-models; Can be performed.

In accordance with yet another embodiment of the present disclosure, a computing device is disclosed. The computing device comprises: one or more processors; And a memory storing instructions executable in the one or more processors; Wherein the one or more processors comprise: receiving first data; Generating second data based on the first data using a first computational model comprising a plurality of sub-models connected in series, the sub-model comprising at least one layer that changes dimensions; Generating test result information based on the first data and the second data; Can be performed.

In accordance with yet another embodiment of the present disclosure, a computing device is disclosed. The computing device includes: one or more processors; And a memory storing instructions executable in the one or more processors; Wherein the one or more processors comprise: receiving first data; Generating third data based on the first data using a second computational model that shares at least a portion of a first computational model; Generating test result information based on the first data and the third data; Can be performed.

The technical solutions obtained in the present disclosure are not limited to the above-mentioned solutions, and other solutions not mentioned above are clearly described to those skilled in the art from the following description. It can be understood.

The present disclosure devised in response to the above-described background art can provide a training method for artificial neural networks and data classification through trained neural networks.

Various aspects are now described with reference to the drawings, wherein like reference numerals are used to refer to like components throughout. In the following examples, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of one or more aspects. However, it will be apparent that such aspect (s) may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to facilitate describing one or more aspects.
1 illustrates a block diagram of a computing device performing an information processing method according to an embodiment of the present disclosure.
2 is a schematic diagram illustrating a portion of an artificial neural network in accordance with an embodiment of the present disclosure.
3 is a schematic diagram of a computational model for implementing the information processing method of the present invention in accordance with one embodiment of the present disclosure.
FIG. 4 is a flowchart for describing steps of an information processing method for training a first connection model according to an embodiment of the present disclosure.
FIG. 5 is a flowchart for describing steps of an information processing method for performing a test using a first connection model according to an embodiment of the present disclosure.
FIG. 6 is a flowchart for describing steps of an information processing method for performing a test using a second connection model according to an embodiment of the present disclosure.
7 shows a brief general schematic diagram of an example computing environment in which embodiments of the present disclosure may be implemented.

Various embodiments are now described with reference to the drawings. In this specification, various descriptions are presented to provide an understanding of the present disclosure. It is evident, however, that such embodiments may be practiced without these specific details. In other instances, well-known structures and devices are provided in block diagram form in order to facilitate describing the embodiments.

As used herein, the terms “component”, “module”, “system” and the like refer to computer-related entities, hardware, firmware, software, a combination of software and hardware, or the execution of software. For example, a component may be, but is not limited to being, a process running on a processor, a processor, an object, a thread of execution, a program, and / or a computer. For example, both an application running on a computing device and the computing device can be a component. One or more components can reside within a processor and / or thread of execution, and a component can be localized within one computer or distributed between two or more computers. In addition, these components can execute from various computer readable media having various data structures stored thereon. The components may, for example, via a network such as the Internet and other systems via signals with one or more data packets (e.g., data and / or signals from one component interacting with other components in a local system, distributed system). Data transmitted) may be communicated via local and / or remote processes.

In addition, the terms "comprises" and / or "comprising" mean that such features and / or components are present, but exclude the presence or addition of one or more other features, components, and / or groups thereof. It should be understood that it does not. Also, unless otherwise specified or in the context of indicating a singular form, the singular in the specification and claims should generally be interpreted as meaning "one or more."

The description of the presented embodiments is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present disclosure should not be limited to the embodiments presented herein but should be construed in the broadest scope consistent with the principles and novel features presented herein.

1 illustrates a block diagram of a computing device in accordance with one embodiment of the present disclosure.

Computing device 100 according to an embodiment of the present disclosure may include a processor 110 and a memory 120. The block diagram shown in FIG. 1 is a simplified configuration of a computing device, and the present disclosure is not limited thereto and may include additional components required for driving.

The processor 110 may include one or more cores, and may include a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), and a tensor processing unit (TPU) of a computing device. and a processor for data analysis, machine learning, and the like, and the above-described types of processors are merely examples and the present disclosure is not limited thereto. The processor 110 may read a computer program stored in the memory 130 to perform an information processing method according to an embodiment of the present disclosure. According to an embodiment of the present disclosure, the processor 110 may perform calculation for learning a neural network. In this specification, learning or training may be used in the same sense. The processor 110 performs neural network learning such as processing input data for learning in machine learning, feature extraction from input data, error calculation, and weight update of neural networks using backpropagation. To perform the calculation. At least one of the CPU, the GPGPU, and the TPU of the processor 110 may process the learning of the computational model. For example, the CPU and GPGPU can work together to learn network functions and classify data using computational models. In addition, in an embodiment of the present disclosure, a processor of a plurality of computing devices may be used together to process learning of a calculation model and data classification using the calculation model. In addition, the computer program executed in the computing device 100 according to an embodiment of the present disclosure may be a program executable by a CPU, a GPGPU, or a TPU.

In an embodiment of the present disclosure, the computing device 100 may distribute and process a computational model using at least one of a CPU, a GPGPU, and a TPU. In addition, in an embodiment of the present disclosure, the computing device 100 may distribute and process a computational model along with other computing devices.

The memory 110 may store a computer program for performing the information processing method according to an embodiment of the present disclosure, and the stored computer program may be read and driven by the processor 110.

2 is a schematic diagram illustrating a portion of an artificial neural network according to an embodiment of the present disclosure.

Throughout this specification, computational models, neural networks, network functions, neural networks may be used in the same sense. A neural network may consist of a set of interconnected computing units, which may generally be referred to as "nodes." Such "nodes" may be referred to as "neurons". The neural network comprises at least one node. Nodes (or neurons) that make up neural networks may be interconnected by one or more “links”.

Within a neural network, one or more nodes connected via a link may form a relationship of input node and output node relatively. The concept of an input node and an output node is relative; any node in an output node relationship to one node may be in an input node relationship in relation to another node, and vice versa. As mentioned above, the input node to output node relationship can be created around the link. One or more output nodes can be connected to a single input node via a link, and vice versa.

In an input node and output node relationship connected via one link, the output node may be determined based on data input to the input node. Here, the node interconnecting the input node and the output node may have a weight. The weight may be variable, and may be varied by the user or algorithm in order for the neural network to perform the desired function. For example, if one or more input nodes are interconnected by each link to one output node, the output node is set to the values input to the input nodes associated with the output node and to the links corresponding to the respective input nodes. The output node value may be determined based on the weight.

As discussed above, a neural network is formed by interconnecting one or more nodes through one or more links to form an input node and an output node relationship within the neural network. The characteristics of the neural network may be determined according to the number of nodes and links in the neural network, the relationship between the nodes and the links, and the value of the weight assigned to each of the links. For example, if there are the same number of nodes and links, and there are two neural networks with different weight values between the links, the two neural networks may be recognized as different from each other.

The neural network may comprise one or more nodes. Some of the nodes that make up the neural network may construct one layer based on distances from the original input node, for example, a set of nodes with a distance n from the original input node, You can configure n layers. The distance from the original input node may be defined by the minimum number of links that must pass to reach the node from the original input node. However, the definition of such a layer is arbitrary for explanation, and the order of the layer in the neural network may be defined in a manner different from that described above. For example, a layer of nodes may be defined by distance from the final output node.

The initial input node may refer to one or more nodes into which data is directly input without going through a link in relation to other nodes of the neural network. Alternatively, in a neural network network, in a relationship between nodes based on a link, it may mean nodes having no other input nodes connected by a link. Similarly, the final output node may refer to one or more nodes that do not have an output node in relation to other nodes of the nodes in the neural network. Also, the hidden node may refer to nodes constituting a neural network other than the first input node and the last output node. The neural network according to an embodiment of the present disclosure may have the same number of nodes in the input layer as the number of nodes in the output layer, and the number of nodes decreases and then increases again as the node progresses from the input layer to the hidden layer. Can be. In addition, the neural network according to another embodiment of the present disclosure may be a neural network in which the number of nodes in the input layer is smaller than the number of nodes in the output layer and the number of nodes decreases as the input layer progresses from the input layer to the hidden layer. have. In addition, the neural network according to another embodiment of the present disclosure may have a larger number of nodes in the input layer than the number of nodes in the output layer, and the number of nodes increases as the number of nodes increases from the input layer to the hidden layer. Can be. The neural network according to another embodiment of the present disclosure may be a neural network in a combined form of the neural networks described above.

A deep neural network (DNN) may refer to a neural network including a plurality of hidden layers in addition to an input layer and an output layer. Deep neural networks can be used to identify potential structures in your data. In other words, you can identify the potential structures of photos, texts, videos, voices, and music (e.g., what objects are in the picture, what are the content and feelings of the text, what are the contents and feelings of the voice, etc.) . Deep neural networks include convolutional neural networks (CNNs), recurrentneural networks (RNNs), auto encoders, Generic Adversarial Networks (GAN), restricted boltzmann (RBM) machine), deep belief network (DBN), Q network, U network, Siamese network and the like.

The neural network may be trained in at least one of supervised learning, unsupervised learning, and semi supervised learning. Training of neural networks is intended to minimize errors in the output. In the neural network learning, the neural network error is inputted from the output layer of the neural network in order to repeatedly input the training data into the neural network, calculate the error of the neural network output and target for the training data, and reduce the error. The process of updating the weight of each node of the neural network by backpropagation in the direction. In the case of teacher learning, the learning data using the correct answer is labeled in each learning data (that is, the labeled learning data), and in the case of the comparative learning, the correct answer may not be labeled in each learning data. That is, for example, the learning data in the case of teacher learning regarding data classification may be data in which a category is labeled in each of the learning data. The labeled training data is input to the neural network, and an error can be calculated by comparing the label of the training data with the output (category) of the neural network. As another example, in the case of non-comparative learning on data classification, an error may be calculated by comparing learning data as an input with a neural network output. The calculated error is propagated backward in the neural network (ie, the direction from the output layer to the input layer), and the connection weight of each node of each layer of the neural network may be updated according to the reverse propagation. The connection weight of each node to be updated may be determined according to a learning rate. The computation of the neural network for the input data and the backpropagation of the errors can constitute a learning cycle. The learning rate may be applied differently according to the number of repetitions of the learning cycle of the neural network. For example, in the early stages of neural network learning, high learning rates can be used to allow the neural network to quickly achieve a certain level of performance, increasing efficiency, and lower learning rates for later learning.

In the learning of neural networks, training data can generally be a subset of the actual data (ie, the data to be processed using the trained neural network), thus reducing the error for the training data but not for the actual data. There may be an increasing learning cycle. Overfitting is a phenomenon in which errors on the actual data are increased by excessively learning the training data. For example, a neural network that learns a cat by showing a yellow cat may not recognize that the cat is a cat other than a yellow cat. Overfitting can be a cause of increased errors in machine learning algorithms. Various optimization methods can be used to prevent such overfitting. In order to prevent overfitting, a method of increasing learning data, regulating, or dropping out of a node of a network in the course of learning may be applied.

Details of the machine learning and deep learning frameworks are discussed in detail in US Pat. No. US 9,141,916B1 (filed May 14, 2013), which is hereby incorporated by reference in its entirety.

In addition, a description of the details of the method for learning the comparative company is specifically discussed in US Patent Publication No. US2008 / 0222135A1 (filed September 11, 2007), which is hereby incorporated by reference in its entirety.

3 is a schematic diagram of a computational model for implementing the information processing method of the present invention in accordance with one embodiment of the present disclosure.

The information processing method of the present disclosure may be performed by one or more processors 110 of the computing device 100. One or more processors 110 of computing device 100 of the present disclosure may perform a computational process of the computational model of the present disclosure.

All computational processes (ie, learning of computational models, feature extraction, feature comparison, etc.) of the information processing method of the present disclosure may be performed by a processor of the computing device 100. That is, the expression processing data in the computational model may mean a process in which the processor 110 of the computing device 100 processes the data by executing the computational model.

The computational models 200, 300 of the present disclosure can be used for classification of data. For example, the computational models 200 and 300 of the present disclosure can be used for anomaly detection.

The anomaly data may refer to abnormal data that deviates from a normal pattern of data. The data may have an atypical pattern and the anomaly data may refer to data that deviates from this atypical pattern. For example, in a production process, data about an image of a product may have an atypical pattern called a normal product, and anomaly data may be data that deviates from the atypical pattern of a normal product (ie, an image of a bad product, for example). Can be. The description of the normal data, the atypical pattern, and the anomaly data of the present disclosure is merely an example and the present disclosure is not limited thereto.

The processor 110 may receive the first data. For example, the processor 110 may receive first data stored in the memory 120 or another storage medium. The first computational model 200 may use the first data as an input. In other words, the processor 110 may process the first data using the first calculation model. In this case, the processor 110 may generate, as an output, second data based on the first data through the first calculation model.

The first data may be training data of the computational models 200 and 300. In addition, the first data may be data to be classified through the calculation models 200 and 300. For example, the first data may be data that is subject to anomaly determination.

The first data of the present disclosure may include training data consisting of normal data. The processor 110 of the present disclosure may train a computational model using training data consisting of normal data.

Specifically, the submodel included in the first computational model of the present disclosure may include at least one layer (eg, a reduction layer and a recovery layer) for changing the dimension. In this case, an error may occur while the input data is changed in dimension and recovered again. The processor 110 may train the first operation model using errors generated by performing the above-described process a plurality of times with one input data. Specifically, referring to FIG. 2, the processor 110 may use the first sub data extracted from the first data, which is the input data, and the recovered data, and the second data extracted from the first sub data, and the second data recovered. The first computational model can be trained. Thus, the information processing method of the present disclosure can generate a plurality of output data using one input data. In addition, the generated plurality of output data may be used for training as training data, and the information processing method of the present disclosure enables efficient training of a computational model even when the training data is small.

The first computational model 200 of the present disclosure may include a plurality of submodels connected in series. As described above, the submodel included in the first operation may include at least one layer for changing the dimension. Accordingly, the submodel of the present disclosure may generate an error in the process of changing and reconstructing the dimension of the input data, which may be used for training and testing.

The submodel included in the first computational model 200 may include various neural networks. For example, the submodels include convolutional neural networks (CNNs), recurrent neural networks (RNNs), auto encoders, autonomous encoders (GANs), restrictive Boltzmann machines (RBMs). This may include restricted boltzmann machines, deep belief networks (DBNs), Q networks, U networks, and Siamese networks.

Specifically, for example, the submodel may be a model for extracting a feature from an input by the processor 110 and then again generating an output identical to the input from the feature. In one embodiment, the submodel may be an auto encoder including an encoder for reducing the dimension of the input data and a decoder for restoring the dimension of the data again. In another embodiment, the submodel may be GAN (Generative Adversarial Networks) including a generator and a separator. In this case, the generator may play the role of the encoder of the auto encoder and the delimiter may play the role of the decoder of the auto encoder. However, the present invention is not limited thereto, and the first operation model 200 may include various neural networks.

The plurality of submodels may be connected in series with each other. Here, serial may mean that at least a part of the output of the submodel may be an input of another submodel, or a part of the submodel may overlap with another submodel. For example, referring to FIG. 2, the processor 110 may generate the first sub data 600 by processing the first data 100 as an input. The second sub-models 230 and 240 may output the second sub-data by processing the first sub-data 600 that is the output of the first sub-models 210 and 220 as an input. In this case, since the second sub-model is the last stage in the first computational model serial structure, the second sub-data may be the second data that is the output of the first computational model 200. However, the present disclosure is not limited thereto, and the first operation model 200 may include various numbers of submodels and may be connected in various ways.

The submodel of the present disclosure may include at least one layer for changing the dimension. For example, the submodel may include at least one of a reduction layer and a recovery layer.

The processor 110 of the present disclosure may reduce the dimension of the input data by using reduction layers. For example, referring to FIG. 2, the processor 110 may reduce the dimension of the first data by using the first reduction layer included in the first submodel 200. In addition, the processor 110 may reduce the dimension of the first sub data by using the second reduction layer included in the second sub model 300. The reduction layer may consist of a single layer or may consist of a plurality of layers.

The processor 110 of the present disclosure may recover reduced dimensions of the input data using reduction layers. For example, referring to FIG. 2, the processor 110 may restore the reduced dimension of the first data using the first recovery layer included in the first submodel 200. In addition, the processor 110 may reduce the dimension of the first sub data using the second recovery layer included in the second sub model 300. The recovery layer may consist of a single layer or may consist of a plurality of layers.

The processor 110 of the present disclosure may generate a cost function based at least in part on the first data, the second data, and the plurality of sub data generated by the plurality of sub models.

The processor 110 may generate a cost function to train the first computational model 200 in a direction in which the value of the cost function decreases. Throughout this specification, a cost function may be used in the same sense as a loss function.

The processor 110 may use various kinds of cost functions. For example, the processor 110 may use at least one of mean squared error (MSE), cross entropy error (CEE), and intersection over union (IOU) as a cost function. However, the present invention is not limited thereto, and the processor 110 may use various types of cost functions. For example, the processor 110 may use the average value of the errors as a cost function, or use the sum of the errors as the cost function.

The processor 110 may generate a cost function using a relationship between various data. Here, the relationship between the data may mean an error (or similarity) between the data. For example, the processor 110 may compare the features of the plurality of data (eg, images) to generate an error value (or similar value). Here, the processor 110 may calculate a function for comparing the mathematical distance between two features, or may be configured as a neural network to use a configuration for determining similarity between the two features. The processor 110 may generate an error value by comparing the features extracted from the data with any data comparison algorithm, and the present disclosure is not limited to the comparison algorithm. The processor 110 may use a deep neural network structure, and among them, may use a deconvolutional neural network structure. However, the present invention is not limited thereto, and the relationship between data may include various meanings such as a difference, an average, and a sum between data.

For example, the processor 110 may generate a cost function using a relationship between the first data 100 that is input data and the second data 400 that is output data of the first operation model 200. In addition, the processor 110 may generate a cost function using the first data and the first sub data which are output data of the first sub models 210 and 220. However, the present invention is not limited thereto.

In addition, the processor 110 may use the sum of the cost functions as the cost function. For example, referring to FIG. 3, the processor 110 may include a term that is a cost function based on the relationship between the first data and the second data, and a term that is a cost function based on the first data and the first sub data. ) Can be used as a cost function.

As the number of sub-models included in the first sub-model increases, the types of terms that may be included in the cost function may increase. For example, when the first computational model 200 includes the first to fifth submodels, the term based on at least two relationships among the first data, the second data, and the first to fourth subdata is a cost function. Can be included. However, the present invention is not limited thereto.

The processor 110 of the present disclosure may set different weights to terms included in the cost function. For example, the processor 110 may multiply a term based on the relationship between the first data and the second data by 2 and a cost function of multiplying the term based on the first data and the first sub data by 1 by the weight. Can be generated. In this case, the effect of the term based on the relationship between the first data and the second data on the value of the cost function can be large. However, the present invention is not limited thereto, and the weight may be variously set.

In addition, the processor 110 may variably set a weight according to a training time point. For example, the processor 110 sets the weights of all terms equally to train the first computational model, and then uses the cost function of multiplying the term based on the relationship between the first data and the second data by the weight to 2 1 You can train a computational model. However, the present invention is not limited thereto, and the processor 110 may variously set weights at various time points.

The processor 110 of the present disclosure may train at least one of the plurality of submodels based on the cost function. For example, the processor 110 may train the first submodel and the second submodel based on the cost function. In addition, the processor 110 may train only the first sub-model or the second sub-model based on the cost function.

In addition, the processor 110 may variably set a submodel to be trained according to a training time point. For example, the processor 110 may train the first submodel and the second submodel, and then additionally train only the second submodel except for the first submodel. In addition, the processor 110 may train only a portion of the submodel. For example, the processor 110 may train only the first reduced layer of the first submodel and the second recovery layer of the second submodel. However, the present invention is not limited thereto, and the processor 110 may train the plurality of submodels in various ways based on the cost function.

The processor 110 of the present disclosure may generate a second computational model 300 that shares at least a portion of the first computational model 200. Here, the sharing may mean using the same (or similar) structure of the neural network (a non-limiting example, the number of layers, the number of nodes, etc.), weights, and the like. For example, referring to FIG. 3, the processor 110 shares a structure and weights of the first reduction layer 210 and the second recovery layer 240 of the first calculation model 200 with the second calculation model 300. ) Can be created. In this case, as shown in FIG. 3, the second computational model 300 is characterized in that the output of the third reduction layer 310, which shares the first reduction layer 210, shares the second recovery layer 240. It may have a structure applied as an input of the third recovery layer 320. However, the present disclosure is not limited thereto, and the processor 110 may generate a second computational model that shares various parts of the first computational model 200.

The processor 110 of the present disclosure may generate the third data 500 based on the first data using the second operation model. In this case, the processor 110 may generate test result information based on the first data and the third data using any algorithm. For example, the processor 110 may perform a test by comparing and scoring first data that is input data of the second operation model and third data 500 that are output data. Herein, the test result information may be information for data classification, and specifically, the test result information may be information for determining whether the first data is anomaly. In addition, the test result information may be information for evaluating the compression efficiency of the input data. The information processing method of the present disclosure may improve test execution speed by using a second computational model that shares at least a portion of the trained first computational model. That is, the information processing method of the present disclosure uses a second computational model including fewer submodels (or layers) than the first computational model to reduce computation time, thereby maintaining test accuracy while maintaining test accuracy. Can improve.

The processor 110 of the present disclosure may generate test result information based on the first data and the second data of the first calculation model. In detail, the processor 110 may perform a test using first data, which is input data of the first operation model, and second data, which is output data. Herein, the test result information may be information for data classification, and specifically, the test result information may be information for determining whether the first data is anomaly. In addition, the test result information may be information for evaluating the compression efficiency of the input data. However, the present invention is not limited thereto, and the test result information may include various information.

In an additional embodiment, the processor 110 of the present disclosure may generate test result information based on at least two of the first data, the second data, the third data, and the plurality of sub data. In detail, the computational model and submodel of the present disclosure may each be trained to produce output data identical to the input data. Thus, the output data of the computational model of the present disclosure and the output data of the submodel can be used to perform the test. For example, the processor 110 may generate a test result based on the first data and output data (eg, the first sub data) of the sub model. As another example, the processor 110 may generate a test result based on output data of at least two submodels. For example, when the first computational model includes three submodels, the processor 110 may generate test result information based on the first sub-data of the first sub-model and the second sub-data of the second sub-model. Can be generated.

In another additional embodiment, the processor 110 of the present disclosure may perform a test by generating a plurality of test result information based on at least two of the first data, the second data, the third data, and the plurality of sub data. . In detail, since the sub-model of the present disclosure can operate as an independent prediction model, the sub-data generated by the sub-model can be utilized for testing with the same qualification as the data generated by the computational model. For example, the processor 110 compares the first data with the first sub data generated by the first sub-model and compares the first data with the second data generated by the first computational model. The anomaly of the first data may be determined using the test result information. As another example, when the first computational model includes four sub-models, by using four test result information generated by comparing the four sub-data generated by the four sub-models with the first data, respectively, Anomaly of the first data may be determined when the test decision information that determines the anomaly is equal to or greater than a predetermined ratio (for example, when 75%, that is, three of the four test result information is result information indicating anomaly, etc.). . In this case, any one of the plurality of test result information may be weighted. For example, a test result generated by comparing the first data and the second data may be weighted twice as much as a test result generated by comparing the first data and the sub data. According to the present disclosure, the computational model of the present disclosure may have an effect of constructing an ensemble model by combining sub-models included in the computational model into a prediction model. However, the present invention is not limited thereto, and the processor 110 of the present disclosure may perform a test in various ways.

The processor 110 may generate test result information by using a relationship between various data. Here, the relationship between the data may mean an error (or similarity) between the data. For example, the processor 110 may generate an error value (or similarity value) by comparing features of a plurality of data (eg, images). In one embodiment, the processor 110 of the present disclosure may generate test result information indicating that the first data is anomalous when an error value (or similarity value) does not satisfy a predetermined criterion (eg, 90%). Can be. In this case, the processor 110 may use a function for comparing the mathematical distance between two features, and may be configured as a neural network to determine the similarity of the two features. The processor 110 may generate an error value (or similarity value) by comparing the features extracted from the data with each other using an arbitrary data comparison algorithm, and the present disclosure is not limited to the comparison algorithm. The processor 110 may use a neural network structure, of which the deconvolutional neural network structure may be used. However, the present invention is not limited thereto.

FIG. 4 is a flowchart for describing steps of an information processing method for training a first connection model according to an embodiment of the present disclosure.

According to an embodiment of the present disclosure, an information processing method for training a first connection model may be implemented by the following steps.

An information processing method performed in one or more processors according to an embodiment of the present disclosure, the information processing method comprising: receiving, by a processor, first data (s410); Generating, by a processor, second data based on the first data using a first computational model comprising a plurality of submodels connected in series (s420), wherein the submodel is at least one for changing dimensions Including a layer; Generating (s430), by a processor, a cost function based on at least two of the first data, the second data, and the plurality of sub-data generated by the plurality of sub-models; Training, by a processor, at least one of the plurality of submodels based on the cost function (s440); It may include.

In an alternative embodiment, the step of generating the second data (s420) comprises: reducing, by a processor, the dimension of the first data using a first reduction layer included in a first submodel, Generating first sub data by recovering the reduced dimension of the first data using the first recovery layer included in the sub model; And reduce, by a processor, the dimension of the first sub data using a second reduction layer included in a second sub-model, and use the first sub data using a second recovery layer included in the second sub-model. Generating the second data by recovering a reduced dimension of the; It may include.

In an alternate embodiment, the cost function is:

And a term based on a relationship between at least two of the first data, the second data, and the plurality of sub-data.

In an alternative embodiment, the cost function may include: a term based on the relationship between the first data and the second data, and a term based on the relationship between the first data and the first sub data.

In an alternative embodiment, the first data may comprise: training data consisting of normal data.

In an alternative embodiment, the first data may comprise: image data.

In alternative embodiments, at least one of the plurality of submodels may comprise neural networks.

In an alternative embodiment, the neural network may comprise an auto-encoder or GAN (Generative Adversarial Networks).

In an alternative embodiment, the information processing method comprises: generating, by a processor, a second computational model that shares at least a portion of the first computational model; It may further include.

In an alternate embodiment, the second computational model may: share at least a portion of the first reduction layer and the second recovery layer.

FIG. 5 is a flowchart for describing steps of an information processing method for performing a test using a first connection model according to an embodiment of the present disclosure.

According to an embodiment of the present disclosure, an information processing method for performing a test using a first connection model may be implemented by the following steps.

According to one embodiment of the present disclosure, an information processing method performed in one or more processors, the information processing method comprising: receiving, by a processor, first data (s510); Generating, by the processor, second data based on the first data using a first operation model including a plurality of sub-models connected in series (s520), wherein the sub-model has at least one layer for changing dimensions Comprising; Generating, by a processor, test result information based on the first data and the second data (s530); It may include.

6 is a flowchart for explaining steps of an information processing method for performing a test using a second connection model according to an embodiment of the present disclosure.

According to an embodiment of the present disclosure, an information processing method for performing a test using a second connection model may be implemented by the following steps.

An information processing method performed in one or more processors, the information processing method comprising: receiving, by a processor, first data (s610); Generating, by a processor, third data based on the first data using a second computational model that shares at least a portion of the first computational model (s620); Generating, by a processor, test result information based on the first data and the third data (s630); It may include.

7 shows a brief, general schematic diagram of an example computing environment in which embodiments of the present disclosure may be implemented.

Although the present disclosure has been described above generally with respect to computer executable instructions that may be executed on one or more computers, those skilled in the art will appreciate that the present disclosure may be implemented in combination with other program modules and / or as a combination of hardware and software. will be.

Generally, program modules include routines, programs, components, data structures, etc. that perform particular tasks or implement particular abstract data types. In addition, those skilled in the art will appreciate that the methods of the present disclosure may include uniprocessor or multiprocessor computer systems, minicomputers, mainframe computers as well as personal computers, handheld computing devices, microprocessor-based or programmable consumer electronics, And other computer system configurations, including one or more associated devices, which may operate in conjunction with one or more associated devices.

The described embodiments of the present disclosure can also be practiced in distributed computing environments where certain tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

Computers typically include a variety of computer readable media. Any medium that can be accessed by a computer can be a computer readable medium, which can be volatile and nonvolatile media, transitory and non-transitory media, removable and non-transitory media. Removable media. By way of example, and not limitation, computer readable media may comprise computer readable storage media and computer readable transmission media. Computer-readable storage media are volatile and nonvolatile media, temporary and non-transitory media, removable and non-removable implemented in any method or technology for storing information such as computer readable instructions, data structures, program modules or other data. Media. Computer storage media may include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROMs, digital video disks or other optical disk storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, Or any other medium that can be accessed by a computer and used to store desired information.

Computer-readable transmission media typically embody computer readable instructions, data structures, program modules or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and the like. Includes all information delivery media. The term modulated data signal means a signal that has one or more of its characteristics set or changed to encode information in the signal. By way of example, and not limitation, computer readable transmission media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, or other wireless media. Combinations of any of the above should also be included within the scope of computer readable transmission media.

An example environment 1000 is illustrated that implements various aspects of the present disclosure, including a computer 1002, which includes a processing device 1004, a system memory 1006, and a system bus 1008. do. System bus 1008 couples system components, including but not limited to system memory 1006, to processing device 1004. Processing apparatus 1004 may be any of a variety of commercial processors. Dual processor and other multiprocessor architectures may also be used as the processing unit 1004.

System bus 1008 may be any of several types of bus structures that may be further interconnected to a memory bus, a peripheral bus, and a local bus using any of a variety of commercial bus architectures. System memory 1006 includes read only memory (ROM) 1010 and random access memory (RAM) 1012. Basic Input / Output System (BIOS) is stored in non-volatile memory 1010, such as ROM, EPROM, EEPROM, etc., and this BIOS is a basic aid to transfer information between components within computer 1002, such as during startup. Contains routines. RAM 1012 may also include high speed RAM, such as static RAM for caching data.

Computer 1002 also includes an internal hard disk drive (HDD) 1014 (e.g., EIDE, SATA) and the internal hard disk drive 1014 may also be configured for external use within a suitable chassis (not shown). ), A magnetic floppy disk drive (FDD) 1016 (eg, for reading from or writing to a removable diskette 1018), and an optical disk drive 1020 (eg, a CD-ROM disk 1022 for reading from or reading from or writing to other high capacity optical media such as DVD). The hard disk drive 1014, the magnetic disk drive 1016, and the optical disk drive 1020 are connected to the system bus 1008 by the hard disk drive interface 1024, the magnetic disk drive interface 1026, and the optical drive interface 1028, respectively. ) Can be connected. The interface 1024 for external drive implementation includes at least one or both of Universal Serial Bus (USB) and IEEE 1394 interface technologies.

These drives and their associated computer readable media provide nonvolatile storage of data, data structures, computer executable instructions, and the like. In the case of computer 1002, drives and media correspond to storing any data in a suitable digital format. Although the above description of computer readable media refers to HDDs, removable magnetic disks, and removable optical media such as CDs or DVDs, those skilled in the art will appreciate zip drives, magnetic cassettes, flash memory cards, cartridges, and the like. Other types of media readable by the computer, etc. may also be used in the exemplary operating environment and it will be appreciated that any such media may include computer executable instructions for performing the methods of the present disclosure.

Multiple program modules may be stored in the drive and RAM 1012, including operating system 1030, one or more application programs 1032, other program modules 1034, and program data 1036. All or a portion of the operating system, applications, modules and / or data may also be cached in RAM 1012. It will be appreciated that the present disclosure may be implemented in various commercially available operating systems or combinations of operating systems.

A user may enter commands and information into the computer 1002 via one or more wired / wireless input devices, such as a keyboard 1038 and a mouse 1040. Other input devices (not shown) may include a microphone, IR remote control, joystick, game pad, stylus pen, touch screen, and the like. These and other input devices are often connected to the processing unit 1004 through an input device interface 1042 that is connected to the system bus 1008, but the parallel port, IEEE 1394 serial port, game port, USB port, IR interface, Etc. can be connected by other interfaces.

A monitor 1044 or other type of display device is also connected to the system bus 1008 via an interface such as a video adapter 1046. In addition to the monitor 1044, the computer generally includes other peripheral output devices (not shown) such as speakers, printers, and the like.

Computer 1002 may operate in a networked environment using logical connections to one or more remote computers, such as remote computer (s) 1048, via wired and / or wireless communications. Remote computer (s) 1048 may be a workstation, computing device computer, router, personal computer, portable computer, microprocessor-based entertainment device, peer device, or other conventional network node, and typically Although many or all of the components described above are included, for simplicity, only memory storage 1050 is shown. The logical connections shown include wired / wireless connections to a local area network (LAN) 1052 and / or a larger network, such as a wide area network (WAN) 1054. Such LAN and WAN networking environments are commonplace in offices and businesses, facilitating enterprise-wide computer networks such as intranets, all of which may be connected to worldwide computer networks, such as the Internet.

When used in a LAN networking environment, the computer 1002 is connected to the local network 1052 via a wired and / or wireless communication network interface or adapter 1056. The adapter 1056 may facilitate wired or wireless communication to the LAN 1052, which also includes a wireless access point installed therein for communicating with the wireless adapter 1056. When used in a WAN networking environment, the computer 1002 may include a modem 1058, connect to a communications computing device on the WAN 1054, or establish communications over the WAN 1054, such as via the Internet. Other means. The modem 1058, which may be an internal or external and wired or wireless device, is connected to the system bus 1008 through the serial port interface 1042. In a networked environment, program modules or portions thereof described with respect to computer 1002 may be stored in remote memory / storage device 1050. It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computers can be used.

Computer 1002 is associated with any wireless device or entity disposed and operating in wireless communication, such as a printer, scanner, desktop and / or portable computer, portable data assistant, communications satellite, wireless detectable tag. Communicate with any equipment or location and telephone. This includes at least Wi-Fi and Bluetooth wireless technology. Thus, the communication can be a predefined structure as in a conventional network or simply an ad hoc communication between at least two devices.

Wireless Fidelity (Wi-Fi) allows you to connect to the Internet without wires. Wi-Fi is a wireless technology such as a cell phone that allows such a device, for example, a computer, to transmit and receive data indoors and outdoors, i. Wi-Fi networks use a wireless technology called IEEE 802.11 (a, b, g, etc.) to provide secure, reliable, high-speed wireless connections. Wi-Fi may be used to connect computers to each other, to the Internet, and to a wired network (using IEEE 802.3 or Ethernet). Wi-Fi networks can operate in unlicensed 2.4 and 5 GHz wireless bands, for example, at 11 Mbps (802.11a) or 54 Mbps (802.11b) data rates, or in products that include both bands (dual band). have.

Those skilled in the art will appreciate that information and signals may be represented using any of a variety of different technologies and techniques. For example, data, instructions, instructions, information, signals, bits, symbols, and chips that may be referenced in the above description may include voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields. Or particles, or any combination thereof.

One of ordinary skill in the art of the disclosure will appreciate that the various illustrative logical blocks, modules, processors, means, circuits and algorithm steps described in connection with the embodiments disclosed herein may be implemented in electronic hardware, It will be appreciated that for purposes of the present invention, various forms of program or design code, or combinations thereof, may be implemented. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. One skilled in the art of the present disclosure may implement the described functionality in various ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

The various embodiments presented herein may be embodied in a method, apparatus, or article of manufacture using standard programming and / or engineering techniques. The term "article of manufacture" includes a computer program, carrier, or media accessible from any computer-readable device. For example, computer-readable media may include magnetic storage devices (eg, hard disks, floppy disks, magnetic strips, etc.), optical discs (eg, CDs, DVDs, etc.), smart cards, and flash memory. Devices, such as, but not limited to, EEPROM, cards, sticks, key drives, and the like. In addition, various storage media presented herein include one or more devices and / or other machine-readable media for storing information.

It is to be understood that the specific order or hierarchy of steps in the processes presented is an example of exemplary approaches. Based upon design priorities, it is understood that the specific order or hierarchy of steps in the processes may be rearranged within the scope of the present disclosure. The accompanying method claims present elements of the various steps in a sample order, but are not meant to be limited to the specific order or hierarchy presented.

The description of the presented embodiments is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present disclosure should not be limited to the embodiments presented herein but should be construed in the broadest scope consistent with the principles and novel features presented herein.

Claims (16)

In the information processing method performed in one or more processors,
Receiving, by the processor, the first data;
Generating, by a processor, second data based on the first data using a first computational model comprising a plurality of submodels connected in series, the submodel including at least one layer for changing dimensions box-;
Generating, by a processor, a cost function based on at least two of the first data, the second data, and a plurality of sub-data generated by the plurality of sub-models;
Including,
Information processing method.
The method of claim 1,
Generating the second data includes:
Reduce, by a processor, the dimension of the first data using a first reduction layer included in a first submodel, and reduce the first data using a first recovery layer included in the first submodel Generating first sub data by recovering the dimension;
Reduce, by a processor, the dimension of the first sub data by using a second reduction layer included in a second sub-model, and by using a second recovery layer included in the second sub model by Generating the second data by recovering the reduced dimension;
Including,
Information processing method.
The method of claim 1,
Training, by a processor, at least one of the plurality of submodels based on the cost function;
Including,
Information processing method
The method of claim 1,
The cost function is:
A term based on a relationship of at least two of the first data, the second data, and the plurality of sub data;
Information processing method
The method of claim 2,
The cost function is:
A term based on the relationship between the first data and the second data, and a term based on the relationship between the first data and the first sub data,
Information processing method.
The method of claim 1,
The first data is:
Including training data consisting of normal data,
Information processing method
The method of claim 1,
The first data is:
Containing image data,
Information processing method
The method of claim 1,
At least one of the plurality of submodels includes neural networks,
Information processing method.
The method of claim 8,
The neural network is:
Including auto-encoder or GAN (Generative Adversarial Networks),
Information processing method.
The method of claim 1,
Generating, by a processor, a second computational model that shares at least a portion of the first computational model;
Further comprising,
Information processing method.
The method of claim 10,
The second computational model is:
Sharing at least a portion of the first reduction layer and the second recovery layer,
Information processing method.
In the information processing method performed in one or more processors,
Receiving, by the processor, the first data;
Generating, by a processor, second data based on the first data using a first computational model comprising a plurality of submodels connected in series, wherein the submodel includes at least one layer that changes dimensions -;
Generating, by a processor, test result information based on the first data and the second data;
Information processing method.
In the information processing method performed in one or more processors
Receiving, by the processor, the first data;
Generating, by a processor, third data based on the first data using a second computational model that shares at least a portion of the first computational model;
Generating, by a processor, test result information based on the first data and the third data;
Including,
Information processing method.
As a computing device,
One or more processors; And
A memory that stores instructions executable by the one or more processors;
Including,
The one or more processors,
Receiving first data;
Generating second data based on the first data using a first computational model comprising a plurality of sub-models connected in series, the sub-model comprising at least one layer that changes dimensions; And
Generating a cost function based at least in part on the first data, the second data, and the plurality of sub-data generated by the plurality of sub-models;
To do,
Computing device.
As a computing device,
One or more processors; And
A memory that stores instructions executable by the one or more processors;
Including,
The one or more processors,
Receiving first data;
Generating second data based on the first data using a first computational model comprising a plurality of sub-models connected in series, the sub-model comprising at least one layer that changes dimensions; And
Generating test result information based on the first data and the second data;
To do,
Computing device.
As a computing device,
One or more processors; And
A memory that stores instructions executable by the one or more processors;
Including,
The one or more processors,
Receiving first data;
Generating third data based on the first data using a second computational model that shares at least a portion of a first computational model; And
Generating test result information based on the first data and the third data;
To do,
Computing device.

Images