JP6846369B2 - Programs, devices and methods for classifying unknown multidimensional vector data groups - Google Patents

Description

The present invention relates to a technique for classifying a multidimensional vector data group.

In recent years, with the miniaturization and high accuracy of sensors for IoT (Internet of Things), a server can collect a large amount of sensor data via a network. Moreover, with the spread of neural networks, such big data can be easily analyzed. Furthermore, various manual monitoring operations have been automated and centrally managed by the system. For example, even in areas where it is difficult for people to come and go (mountains, infrastructure equipment (bridges, steel towers, fences, etc.), wildlife monitoring), remote monitoring is possible by arranging environmental sensors. It's coming.

On the other hand, from the viewpoint of big data analysis, the sensor data itself is not composed in a unified format. Therefore, even if a neural network is used, there is a problem that it is difficult to secure the reliability of the analysis result and it is difficult to promote data sharing. In particular, as the amount of sensor data becomes diverse and large, it becomes difficult to classify multidimensional vector data into a normal state and an abnormal state.

Conventionally, there is a technique of acquiring sensor data from a sensor installed in mechanical equipment and diagnosing the presence or absence of an abnormality sign (see, for example, Patent Document 1). According to this technique, a normal model is learned from the time-series waveform of the sensor data in the normal period, and the sensor data in the operating period is compared with the normal model to diagnose the presence or absence of a sign of abnormality in machinery and equipment. The normal model is composed of clusters generated from sensor data during the normal period by clustering of statistical classification methods. A cluster is a region specified by a center and a radius in a multidimensional vector space.

Japanese Unexamined Patent Publication No. 2017-033470

"Introduction of dimensional compression method using t-SNE", [online], [Search on December 9, 2017], Internet <URL: https://blog.albert2005.co.jp/2015/12/ 02 / tsne />

FIG. 1 is an explanatory diagram showing a problem in analysis of unknown big data.

According to FIG. 1, the technique described in Patent Document 1 also requires prior learning from sensor data in a normal period. Therefore, it has been difficult to classify a wide variety of unknown multidimensional vector data without prior learning.
In addition, although learning is performed by classifying into clusters by clustering, the number of clusters must be specified in advance. Since this number of clusters also affects the accuracy of the learning model, it is extremely difficult to determine the number of clusters in advance for unknown data.

Therefore, an object of the present invention is to provide a program, an apparatus, and a method capable of classifying an unknown multidimensional vector data group without prior learning.

According to the present invention, it is a program for operating a computer mounted on a device for classifying a group of multidimensional vector data.
A cluster number estimation means that estimates the number of clusters k by clustering a multidimensional vector data group,
A class classification means that inputs multidimensional vector data into a neural network whose output layer is the number of clusters k and outputs the classification probability for each k class .
By assigning a label of the class having the highest classification probability by the classification means to the vector data of the plurality of dimensions, the computer functions as a teacher data generation means for generating teacher data. And.

According to other embodiments in the program of the present invention
It is also preferable to make the computer function so that the cluster number estimation means is based on DBSCAN (Density-Based Spatial Clustering of Applications with Noise).

According to other embodiments in the program of the present invention
It is also preferable that the classification means uses a softmax function as an activation function of the output layer in the neural network and causes the computer to output the classification probability for each class having a total probability of 1.

According to other embodiments in the program of the present invention
In front of the cluster number estimation means, it further has a dimensional compression means for dimensionally compressing a multidimensional vector data group.
It is also preferable to make the computer function so as to input the low-dimensional vector data group output from the dimensional compression means to the cluster number estimation means.

According to other embodiments in the program of the present invention
The dimensional compression means is based on t-SNE (t-Distributed Stochastic Neighbor Embedding).
It is also preferable to make the computer function so that the low-dimensional vector data group is a two-dimensional or three-dimensional vector data group.

According to other embodiments in the program of the present invention
It is also preferable to make the computer function so that the multidimensional vector data group is a mixture of vector data output from a plurality of sensors.

According to the present invention, it is an apparatus for classifying a group of multidimensional vector data.
A cluster number estimation means that estimates the number of clusters k by clustering a multidimensional vector data group,
A class classification means that inputs multidimensional vector data into a neural network whose output layer is the number of clusters k and outputs the classification probability for each k class .
It is characterized by having a teacher data generation means for generating teacher data by assigning a label of the class having the highest classification probability by the classification means to the vector data of the plurality of dimensions.

According to the present invention, it is a classification method of a device for inputting a multidimensional vector data group.
The device is
The first step of estimating the number of clusters k by clustering a multidimensional vector data group,
The second step of inputting multidimensional vector data into a neural network whose output layer is the number of clusters k and outputting the classification probabilities for each of k classes ,
By assigning the label of the class having the highest classification probability by the second step to the multidimensional vector data, the third step and the third step of generating the teacher data are executed. ..

According to the program, apparatus and method of the present invention, an unknown multidimensional vector data group can be classified without prior learning.

It is explanatory drawing which shows the problem in the analysis of unknown big data. It is a functional block diagram of the analyzer in this invention. It is explanatory drawing which shows the dimension compression part. It is explanatory drawing which shows the cluster estimation part and the classification part.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 2 is a functional configuration diagram of the analyzer according to the present invention.

The analyzer 1 classifies an unknown multidimensional vector data group. According to FIG. 2, the analyzer 1 includes a vector data group storage unit 100, a dimension compression unit 101, a cluster number estimation unit 11, a classification unit 12, and a teacher data generation unit 13. These functional components are realized by executing a program that makes the computer mounted on the device function. In addition, the processing flow of these functional components can be understood as a method for classifying vector data groups.

[Vector data group storage unit 100]
The vector data group storage unit 100 stores an unknown multidimensional vector data group. "Multiple dimensions" means that vector data output from a plurality of sensors are mixed.
The multidimensional vector data group includes, for example, a moving body sensor (acceleration sensor, geomagnetic sensor, etc.), a biological sensor (heartbeat sensor, etc.), an environmental sensor (temperature / humidity sensor, pressure sensor, pressure sensor, etc.), and an optical sensor (ultrasonic sensor, etc.). And infrared sensors), various vector data such as acoustic sensors can be mixed. According to the present invention, if these vector data groups can be acquired in a time series, it is possible to roughly determine a normal time and an abnormal time.

[Dimension compression unit 101]
The dimensional compression unit 101 is an optional one, which is in front of the cluster number estimation unit 11 and dimensionally compresses a multidimensional vector data group. The dimensional compression unit 101 outputs the dimensionally compressed low-dimensional vector data group to the cluster number estimation unit 11. The low-dimensional vector data group here means a vector data group having about 2 to 3 dimensions.

FIG. 3 is an explanatory diagram showing a dimensional compression unit.

ｐ_ij：ｘ_iからｘ_jの近さを表す同時分布
ｐ_j|i：平均ｘ_iに従うガウス分布についてｘ_jが抽出される確率密度
σ_i ²：平均ｘ_iのガウス分布の分散（異なる2点の類似度のみを表す）
ｐ_i|i＝0

次元削減後も元のデータ構造を完全に再現できていれば、ｐ_ij＝ｑ_ijとなる。そのために、ｔ−ＳＮＥでは、ｐ_ijとｑ_ijとの誤差がなるべく小さくなるように次元削減を目指す。
ｔ−ＳＮＥでは、分布間の距離を測る指標として、カルバック・ライブラー・ダイバージェンス(Kullback-Leibler-divergence)を用いる。

ｔ−ＳＮＥでは、ｐ_jiとｑ_jiとを用いて目的関数Ｃを最小化する。解析的に最小解を求めることができないので、勾配法を用いる。

収束した後のＹ＝｛ｙ1，ｙ2，・・・，ｙn｝が、出力される。次元圧縮により、高次元データを人間が視覚的に把握することができる。 The dimensional compression unit 101 is based on t-SNE (t-Distributed Stochastic Neighbor Embedding) (see, for example, Non-Patent Document 1). This is a method of expressing the "closeness" between two points by a probability distribution.

p _ij : Joint distribution representing the closeness of _{x i} to x _{j p j | i : Probability density at which x j} is extracted for the Gaussian distribution according to the mean x _{i σ i} ² : Variance of the Gaussian distribution of the mean x _{i (different 2)} Represents only the similarity of points)
p _{i | i} = 0

If the original data structure can be completely reproduced even after the dimension reduction, p _ij = q _ij . Therefore, in t-SNE, we aim to reduce the dimension so that the error between _{p ij} and q _{ij is as small as possible.}
In t-SNE, Kullback-Leibler-divergence is used as an index for measuring the distance between distributions.

In t-SNE, the objective function C is minimized by using _{p ji} and q _ji. Since the minimum solution cannot be obtained analytically, the gradient method is used.

Y = {y1, y2, ..., Yn} after convergence is output. Dimensional compression allows humans to visually grasp high-dimensional data.

FIG. 4 is an explanatory diagram showing a cluster estimation unit and a classification unit.

[Cluster number estimation unit 11]
The cluster number estimation unit 11 estimates the number of clusters k by clustering a multidimensional vector data group.
The cluster number estimation unit 11 is based on DBSCAN (Density-Based Spatial Clustering of Applications with Noise). Since it is not necessary to set the number of clusters in advance in DBSCAN, the number of clusters is unknown and the vector data group can be classified.

DBSCAN is a density-based clustering method that uses the following two parameters.
Distance threshold: ε (Eps-neighborhood of a point)
Target number threshold: minPts (a minimum number of points)
Data points are classified into the following three types.
Core points: Points with at least minPts of adjacent points within a radius of ε Reachable points: There are not as many adjacent points as there are minPts within a radius of ε,
Outliers with Core points within radius ε: Points with no adjacent points within radius ε Create a cluster from a collection of core points and assign reachable points to each cluster.
That is, in DBSCAN, when a set of points is given to a certain space, the points in the high density region are grouped together, and the points in the low density region are set as outliers.
Then, the cluster number estimation unit 11 estimates the optimum number of clusters k by DBSCAN and outputs it to the classification unit 12.

[Class classification unit 12]
The class classification unit 12 inputs the multidimensional vector data into the neural network having the output layer as the number of clusters k, and outputs the classification probability for each of k classes. The number of clusters k is estimated by the number of clusters estimation unit 11.

The class classification unit 12 uses a softmax function as an activation function of the output layer in the fully connected neural network, and outputs the classification probability for each class having a total probability of 1.

A neural network is a mathematical model that aims to express the characteristics of a living body in the brain by computer simulation. It refers to a general model in which artificial neurons (units) that form a network by synaptic connection change the synaptic connection strength by learning and have problem-solving ability.

According to FIG. 4, as a forward propagation type convolutional neural network (CNN), there are three layers, an input layer, a hidden layer, and an output layer. It is configured and propagates in one direction from the input layer to the output layer. The intermediate layer may be composed of a plurality of layers in a graph shape. Each layer has multiple units (neurons), and the parameter of the function that connects the unit in the anterior layer to the unit in the posterior layer is called "weight". Learning is to calculate an appropriate "weight" as a parameter of this function.

The neural network of the present invention applies a softmax function as a classification problem (determining which class the data belongs to). As the output layer is k pieces in total, the m-th output y _m, expressed as follows.
_{y m = exp (x m)} / Σ i = 1 k exp (x i)
exp (x): exponential function representing the e ^x (e Napier number of 2.7182 ...)
x _m : Input signal

The output of the softmax function is linked by arrows from all input signals. Each output neuron is affected by _{all input signals x m.}
Further, the total output of the softmax function is 1, and the output of the softmax function can be interpreted as a "probability" due to its nature. From the result of this probability, it can be determined which class it belongs to.
The class determined by the softmax function is an unknown class of multiple sensor data.

However, according to the present invention, a pattern recognition model such as a support vector machine for supervised learning is not used as a classification method. This is because according to the present invention, the learning is classified as unsupervised learning.

[Teacher data generation unit 13]
The teacher data generation unit 13 generates teacher data by assigning the label of the class having the highest classification probability by the classification unit 12 to the vector data of the plurality of dimensions. The teacher data can also be applied to a supervised learning model by accumulating it in the teacher data storage unit.

As described in detail above, according to the program, apparatus and method of the present invention, an unknown multidimensional vector data group can be classified without prior learning.

According to the present invention, it is possible to generally classify a wide variety of large amounts of unknown multidimensional vector data without prior learning. Further, at the time of clustering, it is not necessary to specify in advance the number of clusters k, which also affects the accuracy of the learning model. This point is also suitable for classification of unknown multidimensional vector data.

Further, according to the present invention, unknown multidimensional vector data groups can be classified into classes, so that a wide variety of sensor data can be mixed.
For example, a moving body sensor can be installed in infrastructure equipment (for example, a bridge, a steel tower, a fence, etc.), and the sensor data group can be roughly classified into, for example, a normal state and an abnormal state.
Further, for example, by attaching a moving body sensor or a biological sensor to a person or an animal, the sensor data group can be classified into, for example, grasping the behavior of the person or the animal (running, walking, resting, etc.).
Further, for example, by mounting an environmental sensor, an optical sensor, or an acoustic sensor on an existing machine or equipment, the sensor data group can be roughly classified into, for example, a normal state and an abnormal state.

With respect to the various embodiments of the present invention described above, various changes, modifications and omissions within the scope of the technical idea and viewpoint of the present invention can be easily made by those skilled in the art. The above explanation is just an example and does not attempt to restrict anything. The present invention is limited only to the scope of claims and their equivalents.

1 Analyzer 100 Vector data group storage unit 101 Dimensional compression unit 11 Cluster number estimation unit 12 Class classification unit 13 Teacher data generation unit

Claims (8)

A program that operates a computer installed in a device that classifies a group of multidimensional vector data.
A cluster number estimation means that estimates the number of clusters k by clustering a multidimensional vector data group,
A class classification means that inputs multidimensional vector data into a neural network whose output layer is the number of clusters k and outputs the classification probability for each k class .
By assigning the label of the class having the highest classification probability by the classification means to the vector data of the plurality of dimensions, the computer can function as the teacher data generation means for generating the teacher data. Characteristic program. The program according to claim 1, wherein the cluster number estimation means causes a computer to function so as to be based on DBSCAN (Density-Based Spatial Clustering of Applications with Noise). The class classification means uses a softmax function as an activation function of an output layer in the neural network, and causes a computer to function so as to output a classification probability for each class having a total probability of 1. Item 2. The program according to item 1 or 2. In front of the cluster number estimation means, it further has a dimensional compression means for dimensionally compressing a multidimensional vector data group.
The program according to any one of claims 1 to 3, wherein the computer functions to input the low-dimensional vector data group output from the dimensional compression means to the cluster number estimation means. The dimensional compression means is based on t-SNE (t-Distributed Stochastic Neighbor Embedding).
The program according to claim 4, wherein the low-dimensional vector data group causes a computer to function as if it were a two-dimensional or three-dimensional vector data group. The program according to any one of claims 1 to 5, wherein the multidimensional vector data group causes a computer to function as if vector data output from a plurality of sensors are mixed. A device that classifies a group of multidimensional vector data.
A cluster number estimation means that estimates the number of clusters k by clustering a multidimensional vector data group,
A class classification means that inputs multidimensional vector data into a neural network whose output layer is the number of clusters k and outputs the classification probability for each k class .
An apparatus having a teacher data generation means for generating teacher data by assigning a label of a class having the highest classification probability by the classification means to the vector data of the plurality of dimensions. .. This is a classification method for devices that input multidimensional vector data groups.
The device is
The first step of estimating the number of clusters k by clustering a multidimensional vector data group,
The second step of inputting multidimensional vector data into a neural network whose output layer is the number of clusters k and outputting the classification probabilities for each of k classes ,
By assigning the label of the class having the highest classification probability by the second step to the multidimensional vector data, the third step and the third step of generating the teacher data are executed. How to classify equipment.

Images