KR102183672B1 - A Method of Association Learning for Domain Invariant Human Classifier with Convolutional Neural Networks and the method thereof - Google Patents

Abstract

In order to solve the problem of classifier performance degradation due to domain change, a convolutional learning system and method for domain-invariant human classifiers for convolutional neural networks, which design a loss function so that the classifier can learn the association of objects. A neural network model unit that stores a neural network model (CNN); A data set storage unit for storing a training data set for training a neural network model; A loss function setting unit for setting a loss function of the convolutional neural network model; And a learning unit that trains the convolutional neural network model through the training data set.
With the above-described system and method, a model trained through the corresponding method can operate more robustly to domain changes than conventional methods.

Description

{A Method of Association Learning for Domain Invariant Human Classifier with Convolutional Neural Networks and the method thereof}

The present invention relates to an association learning system and method for a domain-invariant human classifier for a convolutional neural network, in which a loss function is designed so that the classifier can learn the association of objects in order to solve the problem of classifier degradation due to domain change. .

In addition, the present invention trains the classifier to generate embedding vectors of human images acquired in various domains similarly to each other, and makes the distribution of transition probability between the embedding vectors resemble the uniform distribution, so that the classifier is in various domains. It relates to an association learning system and method for a domain-invariant human classifier for a convolutional neural network to operate robustly.

In general, human recognition is a key component technology of surveillance cameras and visitor counting systems. Various methods have been proposed, such as designing a classifier through machine learning for features designed for human recognition. Recently, human recognition methods using a convolutional neural network model showing good results in the field of vision-based object recognition are also being widely studied.

The input image of the surveillance camera or visitor counting system has various domains according to the installation angle or distance of the camera, and the type of lens. However, most of the studies do not deal with the problem of algorithm performance degradation due to domain change or take the form of restricting the input domain.

Therefore, a neural network model that can be classified without being affected by domain changes is needed. In addition, in order to solve this problem, a learning method of a domain-invariant person classifier is required.

Haeusser, Philip, Alexander Mordvintsev, and Daniel Cremers. "Learning by association-a versatile semi-supervised training method for neural networks." (CVPR), 2017. Levi, Dan, and Shai Silberstein. "Tracking and motion cues for rear-view pedestrian detection." Intelligent Transportation Systems (ITSC), 2015. Dollar, Piotr, et al. "Pedestrian detection: An evaluation of the state of the art." IEEE transactions on pattern analysis and machine intelligence, 2012. Szegedy, Christian, et al. "Going deeper with convolutions." (CVPR), 2015. Simonyan, Karen, and Andrew Zisserman. "Very deep convolutional networks for large-scale image recognition." arXiv preprint arXiv: 1409.1556, 2014.

An object of the present invention is to solve the above-described problem, to train a convolutional neural network model by configuring a loss function so that embedding vectors of images acquired in various types of domains are similarly generated, To provide an association learning system and method for a domain-invariant human classifier for a convolutional neural network.

In addition, an object of the present invention is to generate embedding vectors by passing a training batch including images acquired from various domains through a convolutional neural network model, and generate a transition probability by using the similarity of the embedding vectors, A relationship learning system and method for a domain-invariant human classifier for a convolutional neural network, in which a loss function is constructed so that the transition probability between all embedding vectors is the same, and the model trains the association of human images in various domains. To provide.

On the other hand, the similarity of the embedding vectors uses the similarity of vectors such as the similarity matrix defined in [Non-Patent Document 1].

In order to achieve the above object, the present invention relates to an association learning system for a domain-invariant human classifier for a convolutional neural network, comprising: a neural network model unit for storing a convolutional neural network model (CNN); A data set storage unit for storing a training data set for training a neural network model; A loss function setting unit for setting a loss function of the convolutional neural network model; And a learning unit that trains the convolutional neural network model through the training data set.

In addition, in the association learning system for a domain-invariant human classifier for a convolutional neural network, the present invention is characterized in that the training data set includes images of people photographed from at least two or more viewpoints.

In addition, in an association learning system for a domain-invariant human classifier for a convolutional neural network, the present invention is characterized in that the loss function L _total is calculated by Equation 1 below.

[Equation 1]

However, L _{classification} is a classification loss that expresses the _similarity between the predicted result probability of the human classifier model and the actual correct answer probability, and L _similarity is the loss determined by the similarity between the embedding vectors generated in the training batch (B).

In addition, the present invention is characterized in that in the association learning system for a domain-invariant human classifier for a convolutional neural network, the L _{classification} is calculated by the following equation (2).

[Equation 2]

However, CE() means cross-entropy, GT(x) means the actual correct answer for sample x, and H(x) means the prediction result of the human classifier for sample x.

In addition, in the association learning system for a domain-invariant human classifier for a convolutional neural network, the present invention is characterized in that L _similarity is calculated by Equation 3 below.

[Equation 3]

However, CE () is a cross-meaning the entropy (cross-entropy) and, V means the transition probabilities between the uniform distribution of the ^transition character P resemble, P is the embedded ^transition vector.

In addition, in the present invention, in the association learning system for a domain-invariant human classifier for a convolutional neural network, V is a uniform distribution having all probability values of 1/(|B|-1), and |B| is training It is characterized in that it is the number of images included in the batch B.

In addition, in the present invention, in the association learning system for a domain-invariant human classifier for a convolutional neural network, the P ^transition generates a similarity matrix by calculating the dot product of the embedding vectors in the training batch B, and the generated similarity matrix It is characterized in that it is a probability generated by applying a softmax function so that the calculated similarity has a probability distribution.

In addition, the present invention relates to an association learning method for a domain-invariant human classifier for a convolutional neural network, the method comprising: (a) calculating a loss function of a convolutional neural network model; (b) extracting a training batch from the training data set; And (c) training the convolutional neural network model by applying the data of the training batch to the convolutional neural network model set as the loss function.

In addition, in the present invention, in the association learning method for a domain-invariant human classifier for a convolutional neural network, the loss function L _total is calculated by Equation 4 below.

[Equation 4]

However, L _{classification} is a classification loss that expresses the _similarity between the predicted result probability of the human classifier model and the actual correct answer probability, and L _similarity is the loss determined by the similarity between the embedding vectors generated in the training batch (B).

In addition, in the present invention, in the association learning method for a domain invariant human classifier for a convolutional neural network, the L _similarity is calculated by the following equation (5).

[Equation 5]

However, CE () is a cross-meaning the entropy (cross-entropy) and, V means the transition probabilities between the uniform distribution of the ^transition character P resemble, P is the embedded ^transition vector.

Further, the present invention relates to a computer-readable recording medium in which a program for performing an association learning method for a domain-invariant human classifier for a convolutional neural network is recorded.

As described above, according to the association learning system and method for a domain-invariant human classifier for a convolutional neural network according to the present invention, a model trained through the method can operate more robustly to domain changes than conventional methods. A good effect is obtained.

1 is a diagram showing the configuration of an entire system for implementing the present invention.
2 is a diagram schematically showing a problem of performance degradation of a convolutional neural network model according to a domain change.
3 is a block diagram of a configuration of an association learning system for a domain-invariant human classifier for a convolutional neural network according to an embodiment of the present invention.
4 is an exemplary image of a data set including various domains according to an experiment of the present invention.
5 is a table showing experimental results for a data set including various domains according to the experiment of the present invention.
6 is a flowchart illustrating an association learning method for a domain-invariant human classifier for a convolutional neural network according to an embodiment of the present invention.

Hereinafter, specific details for the implementation of the present invention will be described with reference to the drawings.

In addition, in describing the present invention, the same parts are denoted by the same reference numerals, and repeated explanations thereof are omitted.

First, examples of the configuration of an entire system for implementing the present invention will be described with reference to FIG. 1.

As shown in Fig. 1, the entire system for implementing the present invention receives a neural network model 11 and a data set 12, and performs association learning on a convolutional neural network (CNN) model on a computer terminal 20. It can be implemented as a program system. That is, the association learning method or system may be configured as a program and installed and executed in the computer terminal 20. A program installed in the computer terminal 20 may operate like a single program system 30.

At this time, the neural network model 11 is a convolutional neural network (CNN) model.

In addition, the data set 12 is training data for training the neural network model 11. Preferably, the data set 12 is composed of human images or images of people present in various domains.

On the other hand, as another embodiment, the association learning method or system may be implemented as an electronic circuit such as an ASIC (on-demand semiconductor) in addition to being configured as a program and operating on a general-purpose computer. Alternatively, it may be developed as a dedicated terminal 30 that exclusively processes only training the neural network model 11 using the data set 12. This will be referred to as an association learning system. Other possible forms may also be implemented.

Next, before describing the configuration of the present invention, a problem of performance degradation of a convolutional neural network model according to a domain change will be described in more detail.

2 schematically shows the problem of performance degradation of a convolutional neural network model according to a domain change. The image on the left of FIG. 2 shows human images acquired in various domains. The squares on the right side of FIG. 2 represent embedding vectors generated by passing the left images through a convolutional neural network model, and vectors of the same color mean vectors having high similarity to each other.

Embedding vectors generated from human images acquired in the same or similar domains have a high degree of similarity to each other, but the similarity with embedding vectors for images acquired in different domains is measured to be low. This is a phenomenon that occurs because the convolutional neural network model has not been trained on a new image that has not been seen before, and this phenomenon causes performance degradation.

In the method according to the present invention, in order to solve this problem, a convolutional neural network model is trained so that embedding vectors of images acquired in various domains are similarly generated.

Next, a configuration of an association learning system for a domain-invariant human classifier for a convolutional neural network according to an embodiment of the present invention will be described with reference to FIG. 3.

As shown in FIG. 3, the channel extension parameter setting system according to the present invention includes a neural network model unit 31 that stores a convolutional neural network model (CNN), and a data set that stores a training data set for training a neural network model. It consists of a unit 32, a loss function setting unit 33 for setting a loss function, and a learning unit 34 for training a neural network model through a training data set. In addition, it is composed of a storage unit 38 that stores a convolutional neural network model (CNN), a data set, and the like.

First, the neural network model unit 31 stores a convolutional neural network model (CNN). A convolutional neural network model (CNN) may be input and stored, or may be set and stored in advance.

Next, the data set storage unit 32 stores a data set of training data for training the neural network model. The learning data set may be input and stored, or may be set and stored in advance.

4 shows example images of a training data set.

As shown in FIG. 4, preferably, the training data set is a data set for human images including various domains. That is, as images taken from various viewpoints (or domains), they are images (or images) of a person.

Next, the loss function setting unit 33 sets a loss function for learning (or training) the neural network model.

Learning (or training) a neural network model with training data means finding the weight parameter of the neural network model as an optimal value through learning. The index that allows these neural networks to learn is the loss function. That is, the learning process is to find the weight parameter that makes the value of the loss function the least for the training data. This is because the least loss function means that the data and the predicted value become more accurate, and that the least loss function means the optimal weight.

In other words, the loss function is a function that calculates the difference between the value predicted through the model and the value actually desired to occur. By changing the parameters of the human classifier model to be trained, the difference between the predicted value and the actual value is reduced. Several types of functions can exist depending on the purpose of the model to be created.

The loss function (L _total ) is calculated through Equation 1.

[Equation 1]

Here, L _{classification} is a general classification loss in neural network model training, and expresses how similar the predicted result probability of the human classifier model and the actual correct answer probability are. In other words, L _{classification} describes mathematically how well the human classifier model _classifies people, and is calculated by the following equation.

[Equation 2]

Here, GT(x) means the actual correct answer for sample x, and H(x) means the prediction result of the human classifier for sample x.

CE() is cross-entropy. Cross-entropy serves to express numerically how similar the two probability distributions are. The cross-entropy CE(P,Q) for the probability distributions P and Q is calculated as follows.

[Equation 3]

Here, if the probability values of P and Q for sample x are the same, the result of CE(P,Q) is 0. On the other hand, when the probability value of P for sample x is 0 and the probability value of Q is 1, the result of CE(P,Q) becomes infinite. Through this, the difference between the predicted result of the model to be trained and the actual correct answer result can be compared and expressed numerically.

On the other hand, the sample x referred to here means the image in the training batch B. Training batch B applies equally to both L _{classification} and L _similarity . Training arrangement B will be described in more detail below.

Next, L _similarity expresses how similar the embedding vectors of the training batch (B) are to each other. That is, L _similarity is a loss determined by the degree of similarity between embedding vectors generated in the training batch (B) and is calculated through Equation 2.

[Equation 4]

Here, CE() means cross-entropy.

V is a uniform distribution of the ^transition character P resemble, P ^transition means the transition probabilities between the embedded vector. V and P ^transition can be obtained similarly to the visit loss proposed in [Non-Patent Document 1].

In the present invention, since only human objects are trained, the uniform distribution of V is set to 1/(|B|-1). |B| is a scalar value. V uniform distribution is calculated using the number of images in training batch B |B|

Specifically, V is simply a uniform distribution in which all probabilities have a value of 1/(|B|-1). In training batch B containing |B| images, a total of |B| embedding vectors are generated. V uniform distribution is set to 1/(|B|-1) so that one vector and the remaining |B|-1 vectors are trained similarly to each other.

In addition, P ^transition is a probability distribution representing the similarity of the embedding vectors in training arrangement B. A similarity matrix is generated by calculating the dot product of |B| embedding vectors in training batch B. The calculated similarity is made to have a probability distribution by applying a softmax function to the generated similarity matrix. Training is performed so that the calculated P ^transition probability distribution resembles the V probability distribution.

Since batch B includes human images acquired in various domains, the P ^transition value at the initial stage of training is measured high for domains that are similar to each other.

In this case, the loss function is penalized by the L _similarity in Equation 2, and the trained convolutional neural network model operates invariably in the domain as the value of the P ^transition resembles the V uniform distribution as learning progresses. Is done.

For reference, training batch (B) is generated by randomly extracting |B| number of training images from the entire data set. For example, if 10 training images are randomly extracted from 100 data sets and a training batch is generated, a total of 10 training batches (B ₁ ~ B ₁₀ ) are generated. That is, B images included in the data set are randomly extracted and grouped together. This is called one batch, and it is used for model training and is specified as a training batch.

Next, when the loss function of the neural network model is set, the learning unit 34 trains the neural network model through the training data set.

Next, an association learning method for a domain-invariant human classifier for a convolutional neural network according to an embodiment of the present invention will be described with reference to FIG. 6.

As shown in FIG. 6, first, the loss function of the convolutional neural network model is calculated (S10). The loss function is previously calculated by Equations 1 and 2.

Next, a training batch is extracted from the training data set (S20).

Then, the convolutional neural network model is trained by applying the data of the training batch to the convolutional neural network model set as a loss function (S30).

Next, the effects of the present invention will be described through experiments.

That is, in order to show that the method according to the present invention operates invariably in a domain, an experiment was performed by configuring a data set including various domains. The data set used in the experiment consists of GM-ATCI [Non-Patent Literature 2], Caltech-pedestrian [Non-Patent Literature 3] and images obtained by themselves.

4 shows a part of the data set used in the present invention. In the present invention, the convolutional neural network model proposed in [Non-Patent Document 2] was used, and training was performed by setting the batch size to 100.

The table of FIG. 5 shows the results of a human classification experiment for the data set constructed in the present invention. A pedestrian classification model was trained using the same training data and conditions for the model structures of [Non-Patent Document 4] and [Non-Patent Document 5]. The results of the VGG16 model in the table of FIG. 5 show a large difference in performance from the method according to the present invention. In the case of training the VGG16 model, it is the same as training by removing Equation 2 from the loss function Equation 1 when compared to the method according to the present invention. Through this, it can be confirmed that the method according to the present invention operates robustly in various domains.

In the present invention, a method of learning a domain invariant person classifier has been described. A new loss function is proposed to make the embedding vectors of images acquired in several domains similar. Through this, it was shown that the convolutional neural network model can learn the association between human images and thus operates domain-invariably. When using the proposed method through experiments on data sets collected in various environments, it was confirmed that the trained model operates more robustly to domain changes than the existing methods.

In the above, the invention made by the present inventor has been described in detail according to the above embodiment, but the invention is not limited to the above embodiment, and it goes without saying that various modifications can be made without departing from the gist of the invention.

11: neural network model 12: data set
20: computer terminal 30: program system
31: neural network model unit 32: data set storage unit
33: loss function setting unit 34: learning unit
38: storage

Claims (11)

In the association learning system for a domain-invariant human classifier for a convolutional neural network,
A neural network model unit that stores a convolutional neural network model (CNN);
A data set storage unit for storing a training data set for training a neural network model;
A loss function setting unit for setting a loss function of the convolutional neural network model; And,
Including a learning unit for training the convolutional neural network model through the training data set,
The loss function L _total is calculated by the following Equation 1,
In Equation 1, V is a uniform distribution in which all probabilities have a value of 1/(|B|-1), |B| is the number of images in training batch B, and training including |B| In batch B, a total of |B| embedding vectors are created, and one of them and the remaining |B|-1 vectors are trained similarly, so that the V uniform distribution is set to 1/(|B|-1) and,
In Equation 1, P ^transition generates a similarity matrix by calculating the dot product of |B| embedding vectors in training batch B, and by applying a softmax function to the generated similarity matrix, the calculated similarity is An association learning system for a domain-invariant human classifier for a convolutional neural network, characterized in that it is a probability generated by having a probability distribution.
[Equation 1]

,

However, L _{classification} is a classification loss expressing the _similarity between the predicted result probability of the human classifier model and the actual correct answer probability, L _similarity is the loss determined by the similarity between embedding vectors generated in the training batch (B), and CE( ) is cross-meaning the entropy (cross-entropy) and, V means the transition probabilities between the uniform distribution of the ^transition character P resemble, P is the embedded ^transition vector.
The method of claim 1,
The association learning system for a domain-invariant human classifier for a convolutional neural network, wherein the training data set includes images of people photographed from at least two or more viewpoints.
delete The method of claim 1,
The L _{classification} is an association learning system for a domain-invariant human classifier for a convolutional neural network, characterized in that calculated by the following equation (2).
[Equation 2]

However, CE() means cross-entropy, GT(x) means the actual correct answer for sample x, and H(x) means the prediction result of the human classifier for sample x.
delete delete delete In the association learning method for a domain invariant human classifier for a convolutional neural network,
(a) calculating a loss function of the convolutional neural network model;
(b) extracting a training batch from the training data set; And,
(c) training the convolutional neural network model by applying the data of the training batch to the convolutional neural network model set as the loss function,
The loss function L _total is calculated by the following Equation 4,
In Equation 4, V is a uniform distribution with all probability values of 1/(|B|-1), |B| is the number of images included in training batch B, and training including |B| In batch B, a total of |B| embedding vectors are created, and one of them and the remaining |B|-1 vectors are trained similarly, so that the V uniform distribution is set to 1/(|B|-1) and,
In Equation 4, P ^transition generates a similarity matrix by calculating the dot product of |B| embedding vectors in training batch B, and by applying a softmax function to the generated similarity matrix, the calculated similarity is An association learning method for a domain-invariant human classifier for a convolutional neural network, characterized in that it is a probability generated by having a probability distribution.
[Equation 4]

,

However, L _{classification} is a classification loss expressing the _similarity between the predicted result probability of the human classifier model and the actual correct answer probability, L _similarity is the loss determined by the similarity between embedding vectors generated in the training batch (B), and CE( ) is cross-meaning the entropy (cross-entropy) and, V means the transition probabilities between the uniform distribution of the ^transition character P resemble, P is the embedded ^transition vector.
delete delete A recording medium that can be read by a computer on which a program that performs the method of claim 8 is recorded.

Images