JP6892848B2 - Information processing equipment, information processing methods and information processing programs - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and an information processing program.

In recent years, attention has been focused on performing prediction, abnormality detection, automatic control, and the like by utilizing a large amount of IoT (Internet of Things) sensor data in various fields such as factories and automatic driving. The characteristics of such sensor data are that it has a time series in which the value changes from moment to moment and that the sensors interact with each other. A neural network (Deep Learning) is an effective method as a method for analyzing sensor data having these characteristics.

Typical neural network structures generally used in the analysis of time-series data include MLP (Multi-Layer Perceptron), CNN (Convolutional Neural Network), and RNN (Recurrent Neural Networks). These will be described below.

As a structure of a neural network, there is a structure using a multi-layer perceptron (MLP), which is a basic network structure. In this method, as illustrated in FIG. 9, first, sensor data is extracted on the time axis of M dimensions, and then converted into a one-dimensional vector. The operation of converting to a one-dimensional vector is an operation necessary for inputting to a fully connected neural network. Let this be one data set. This operation is advanced by one time step and is performed in the same manner. After repeating this operation to create an input data set, it is input to a general fully connected neural network and the result is output.

It is also conceivable to utilize CNN, which is widely used in the field of image recognition, for time series data analysis. In this neural network, as illustrated in FIG. 10, a matrix operation is performed on each local region using an M × N-dimensional filter (so-called convolution layer), and the feature amount of the data is reduced. Taking time-series data as an example, by repeating the convolution operation in the time-series (column) direction and the sensor data (row) direction, it is possible to output the result considering the time-series of a plurality of sensors. This method is suitable for GPGPU (General-Purpose computing on Graphics Processing Units) parallel computing (relatively easy to speed up). Time series can be considered by the calculation of the convolution layer, and there is an advantage that factor analysis is relatively easy.

RNN is a neural network often used in natural language processing and time series analysis. As illustrated in FIG. 11, the RNN has a structure in which the result obtained by inputting data at a certain time is used again for inputting the next time. By holding the input / output relationship of past data in an RNN cell (for example, GRU (Gated Recurrent Unit), RSTM (Long Short Term Memory), etc.), it is possible to obtain an output considering a long time series state. ..

JP-A-2017-142654

However, the conventional method has a problem that time series data may not be analyzed easily and efficiently. For example, the conventional method using MLP is simple and easy to implement, but it is inefficient because the operation of cutting out M × N-dimensional data and converting it into one dimension is repeated. Further, as the number of dimensions increases, the required memory becomes enormous, and further, one-dimensional conversion is performed, so that there is a problem that the time series is explicitly difficult to handle.

In addition, in the conventional method using CNN, since the convolution operation is also performed in the row direction (sensor direction), there is a strong dependence on the order of the data (sensors). For example, if the order of the sensors is rearranged, the result will be different. There are challenges. Further, a method of convolving only in the column direction (time series) can be considered, but in this case, it is difficult to explicitly consider the dependency between the sensor data.

Further, the conventional method using RNN is not suitable for GPGPU parallel computing, and there is a problem that it is difficult to increase the speed. Further, since the internal state of the RNN cell is complicated, it is difficult to analyze the factors of the input value with respect to the output value of the neural network.

In order to solve the above-mentioned problems and achieve the object, the information processing apparatus of the present invention has a collecting unit that collects data acquired by each sensor installed in the target facility, and a collecting unit that collects the data. A processing unit that inputs data to a fully connected layer trained to cancel the dependency on the order between the data at each time of the data and performs a process of obtaining output data, and an output obtained by the processing unit. A coupling unit that combines data in the time direction to obtain a matrix, a calculation unit that performs a convolution operation on the matrix obtained by the coupling unit, and an output unit that outputs the result calculated by the calculation unit. It is characterized by having.

Further, the information processing method of the present invention is an information processing method executed by an information processing apparatus, and is a collection step of collecting data acquired by each sensor installed in the target facility and a collection step of collecting the data. A processing step of inputting the generated data into a fully connected layer trained to cancel the dependency on the order between the data at each time of the data and obtaining output data, and a processing step obtained by the processing step. A joining step of combining the output data in the time direction to obtain a matrix, a calculation step of performing a convolution operation on the matrix obtained by the joining step, and an output step of outputting the result calculated by the calculation step. It is characterized by including and.

In addition, the information processing program of the present invention collects the data acquired by each sensor installed in the target facility, and collects the data collected by the collection step between the data at each data time. A processing step of inputting to a fully connected layer learned to cancel the dependency on the order of the above and performing a process of obtaining output data, and a combination of combining the output data obtained by the processing step in the time direction to obtain a matrix. It is characterized in that a computer is made to execute a step, a calculation step for performing a convolution operation on the matrix obtained by the combination step, and an output step for outputting the result calculated by the calculation step.

According to the present invention, there is an effect that time series data can be analyzed easily and efficiently.

FIG. 1 is a block diagram showing a configuration example of the analysis system according to the first embodiment. FIG. 2 is a block diagram showing a configuration example of the information processing apparatus according to the first embodiment. FIG. 3 is a diagram showing an example of data stored in the sensor data storage unit. FIG. 4 is a diagram illustrating a data input process to the fully connected layer in the information processing apparatus according to the first embodiment. FIG. 5 is a diagram illustrating a data input process to the fully connected layer in the information processing apparatus according to the first embodiment. FIG. 6 is a diagram illustrating an outline of the entire process in the information processing apparatus according to the first embodiment. FIG. 7 is a flowchart showing an example of a processing flow in the information processing apparatus according to the first embodiment. FIG. 8 is a diagram showing a computer that executes an information processing program. FIG. 9 is a diagram for explaining the prior art. FIG. 10 is a diagram for explaining the prior art. FIG. 11 is a diagram for explaining the prior art.

Hereinafter, the information processing apparatus, the information processing method, and the embodiment of the information processing program according to the present application will be described in detail with reference to the drawings. The information processing apparatus, information processing method, and information processing program according to the present application are not limited by this embodiment.

[First Embodiment]
In the following embodiment, the configuration of the analysis system 100, the configuration of the information processing device 10, and the processing flow of the information processing device 10 according to the first embodiment will be described in order, and finally, the effect of the first embodiment will be described. explain.

[Analysis system configuration]
FIG. 1 is a block diagram showing a configuration example of the analysis system according to the first embodiment. The analysis system 100 according to the first embodiment has an information processing device 10 and a plurality of target equipments 20A to 20C, and the information processing device 10 and the target equipments 20A to 20C are connected to each other via a network 30. .. The configuration shown in FIG. 1 is only an example, and the specific configuration and the number of each device are not particularly limited. Further, when the target equipments 20A to 20C are described without particular distinction, the target equipments 20 are appropriately described.

The information processing device 10 collects data (hereinafter, appropriately referred to as sensor data) acquired by sensors 21 installed in the target equipments 20A to 20C. Then, the information processing device 10 takes the collected sensor data as an input and analyzes it using the trained model, and based on the sensor data, obtains data to be used for product quality prediction, abnormality detection, automatic control, and the like. Obtainable.

A plurality of sensors 21 are installed in each of the target equipments 20A to 20C. The target equipment can be applied to various equipment such as in-plant equipment, reactors, building air conditioners, data center racks, automobiles, and the like, and is not particularly limited.

[Configuration of information processing device]
Next, the configuration of the information processing apparatus 10 will be described with reference to FIG. FIG. 2 is a block diagram showing a configuration example of the information processing apparatus according to the first embodiment. As shown in FIG. 3, the information processing device 10 includes a communication processing unit 11, a control unit 12, and a storage unit 13. The processing of each part of the information processing apparatus 10 will be described below.

The communication processing unit 11 controls communication related to various types of information. For example, the communication processing unit 11 transmits / receives sensor data to / from the target equipment 20.

The storage unit 13 stores data and programs necessary for various processes by the control unit 12, and particularly closely related to the present invention is the sensor data storage unit 13a. For example, the storage unit 13 is a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk.

The sensor data storage unit 13a stores the data of the sensor 21 at the same time collected from each sensor 21 of each target equipment 20 by the collection unit 12a described later. For example, the sensor data storage unit 13a stores data acquired by N sensors 1 to sensors N at each time, as illustrated in FIG. Further, in the example of FIG. 3, the sensor data storage unit 13a has N sensors 1 to every second, such as time “00:01”, “00:02”, “00:03”, and so on. The data of the sensor N is stored. FIG. 3 is a diagram showing an example of data stored in the sensor data storage unit.

The control unit 12 has an internal memory for storing a program that defines various processing procedures and required data, and executes various processing by these. It has a collecting unit 12a, a processing unit 12b, a coupling unit 12c, a calculation unit 12d, and an output unit 12e. Here, the control unit 12 is an electronic circuit such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), or an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

The collection unit 12a collects the data acquired by each sensor 21 installed in the target equipment 20. Specifically, the collecting unit 12a collects the data of each sensor 21 installed in each of the plurality of target equipments 20, and stores the data at the same time in the sensor data storage unit 13a.

For example, the collecting unit 12a receives data periodically (for example, every second) from the sensor 21 installed in the target equipment 20 such as a factory or a plant, and stores the data in the sensor data storage unit 13a. Here, the data acquired by the sensor 21 is, for example, various data such as temperature, pressure, sound, and vibration of the equipment and the reactor in the factory and the plant, and is not limited.

The processing unit 12b inputs the sensor data collected by the collecting unit 12a to the fully connected layer trained to cancel the dependency on the order between the sensor data at each time of the sensor data, and obtains output data. I do.

Here, the process of inputting data to the fully connected layer will be described with reference to FIGS. 4 and 5. 4 and 5 are diagrams for explaining the data input process to the fully connected layer in the information processing apparatus according to the first embodiment. As shown in FIG. 4, the processing unit 12b inputs the sensor data to the fully connected layer at each time. That is, for example, the processing unit 12b inputs N sensor data of the sensors 1 to the sensor N at the time "00:01" into the fully connected layer, and then the sensor 1 to the sensor N at the time "00:02". The process of inputting the N sensor data of the sensors 1 to the fully connected layer and then inputting the N sensor data of the sensors 1 to N at the time "00:03" to the fully connected layer is repeated.

As illustrated in FIG. 5, the fully connected layer is a layer in which all the nodes corresponding to the inputs and outputs are connected to each other. That is, when the processing unit 12b inputs each of the N sensor data at a certain time t, the processing unit 12b obtains N outputs at a certain time t. In addition, each node has a "weight" which means the ease of transmitting a signal. Thanks to this fully connected layer, it is possible to cancel or extract the order dependency between each sensor data. The number of nodes in the fully connected layer may be 2 or more, and any value can be taken. For example, if there is a data string (order between data) that has a physical position (distance) relationship of the sensor itself, apply a neural network trained so that the dependency (correlation) is taken into consideration. You may. That is, in such a case, the fully connected layer cancels the dependency on the order between each data that does not need to be considered, but cancels the dependency (correlation) of the physical position (distance) of each sensor. Instead, the data shall be output.

The connecting unit 12c combines the output data obtained by the processing unit 12b in the time direction to obtain a matrix. For example, the connecting unit 12c joins the rows of values output from the fully connected layer for each time in the time direction (row direction). For example, the coupling unit 12c inputs the sensor data at the time "00:01" to the fully coupled layer, and inputs the row of values output from the fully coupled layer and the sensor data at the time "00:02" to the fully coupled layer. Is input to and the row of values output from the fully connected layer is combined. The joining portion 12c is not limited to the case of joining two rows at a time, and may be made to join three or more rows at a time. In this way, the coupling unit 12c can obtain a new matrix in which the dependency of the order of the sensor data is canceled by combining the rows of the values output from the fully coupled layer for each time.

The calculation unit 12d performs a convolution operation on the matrix obtained by the connection unit 12c. Specifically, the calculation unit 12d performs a convolution operation by inputting the matrix obtained by the connection unit 12c into the convolution layer. Further, the calculation unit 12d performs a convolution calculation using GPGPU. In this way, the calculation unit 12d can consider the time series by performing the convolution operation on the matrix obtained by the connection unit 12c, and further, since the convolution operation is used, the GPGPU can be used. High-speed calculation using is possible. The size of the convolution layer and the number of layers are arbitrary.

The output unit 12e outputs the result calculated by the calculation unit. Specifically, the output unit 12e obtains an output value from the output layer. After that, the output unit 12e may display the output result on the information processing device 10 or an external device, or output the output result to another functional unit or an external device in the information processing device 10 for further processing. You may do so. The output value may be any data, for example, the degree of abnormality for detecting an abnormality in the target equipment 20, or a prediction for predicting the product quality in the target equipment 20. It may be a value.

Here, with reference to FIG. 6, an outline of the entire process in the information processing apparatus 10 according to the first embodiment will be described. FIG. 6 is a diagram illustrating an outline of the entire process in the information processing apparatus according to the first embodiment. As illustrated in FIG. 6, the information processing apparatus 10 first inputs sensor data to the fully connected layer at each time (see (A) in FIG. 6). Subsequently, the information processing apparatus 10 combines the outputs from the fully connected layer in the time direction (see (B) in FIG. 6).

Then, the information processing apparatus 10 performs a convolution operation on the matrix obtained by combining (see (C) of FIG. 6). Finally, the information processing apparatus 10 obtains the output value as a whole from the output layer (see (D) of FIG. 6).

As described above, in the information processing apparatus 10, the dependency relationship between data can be automatically handled by performing the calculation of the fully connected layer with respect to the data direction in the foremost stage in the neural network for time series analysis. Further, in the information processing apparatus 10, it is possible to perform an operation considering time series at high speed by performing a convolution operation on a matrix in which rows of values output from a fully connected layer are combined in the time direction. Is.

In the above example, the frontmost layer is very simple as the fully connected layer. This layer can be easily replaced with a layer in which the sensor data is dropped into the feature space by the AutoEncorder or the Variational Autoencorder.

Further, the neural network including the fully connected layer and the convolution layer that performs the convolution operation is learned by the error backpropagation method or the like. For example, in the process of learning (optimizing) the weight of a neural network, learning is performed based on the existing backpropagation method. After the learning is optimized, the weight of the fully connected layer and the weight of the filter of the convolution layer are automatically optimized, and the dependency between the sensors and the time series can be handled well. The timing of learning the neural network may be performed in advance before the series of analysis processes described in FIG. 6 described above are performed, or may be performed in real time at the same time as the analysis processes are performed. Good.

[Processing procedure of information processing device]
Next, an example of the processing procedure by the information processing apparatus 10 according to the first embodiment will be described with reference to FIG. 7. FIG. 7 is a flowchart showing an example of a processing flow in the information processing apparatus according to the first embodiment. In the example of FIG. 7, a case where a series of analysis processes are performed each time sensor data is collected will be described as an example, but the present invention is not limited to this.

As illustrated in FIG. 7, when the collecting unit 12a collects the data of the sensor 21 in the target equipment 20 (affirmation in step S101), the collecting unit 12a stores the collected data in the sensor data storage unit 13a (step S102).

Then, the processing unit 12b inputs the sensor data of the same time collected by the collecting unit 12a into the fully connected layer learned so as to cancel the dependency relationship regarding the order between the data (step S103). Subsequently, the coupling unit 12c combines the output data obtained by the processing unit 12b in the time direction (step S104). For example, the connecting unit 12c joins the rows of values output from the fully connected layer for each time in the time direction (row direction) to obtain a matrix.

Subsequently, the calculation unit 12d performs a convolution calculation on the matrix obtained by the connection unit 12c (step S105). Specifically, the calculation unit 12d performs a convolution operation by inputting the matrix obtained by the connection unit 12c into the convolution layer. Then, the output unit 12e obtains an output value from the output layer.

(Effect of the first embodiment)
The information processing device 10 according to the first embodiment collects sensor data acquired by each sensor 21 installed in the target facility 20, and collects the collected sensor data between the data at each data time. Input to the fully connected layer learned so as to cancel the dependency on the order, and perform the process of obtaining the output data. Then, the information processing apparatus 10 combines the obtained output data in the time direction to obtain a matrix, performs a convolution operation on the matrix, and outputs the calculation result. Therefore, the information processing apparatus 10 can easily and efficiently analyze the time series data. That is, in the information processing apparatus 10, it is possible to easily analyze the time series data by using a simple neural network in order to perform prediction, abnormality detection, prediction, etc. using a large amount of IoT sensor data.

That is, the neural network used by the information processing apparatus 10 according to the first embodiment is simple and easy to implement, can perform high-speed parallel calculation using GPGPU, and eliminates the dependence on the order of sensor data. , It is possible to efficiently analyze time series data.

In particular, such analysis of time series data is useful in the field of IoT sensor data. This can be utilized in product quality prediction, abnormality detection, and advanced automatic control from various sensor data output from machines, plants, etc. in the manufacturing industry. In addition, the present invention can be used in a wide variety of places, and can be applied to time-series data of social infrastructure such as electric power, water, gas, transportation, and communication.

(System configuration, etc.)
Further, each component of each of the illustrated devices is a functional concept, and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of distribution / integration of each device is not limited to the one shown in the figure, and all or part of the device is functionally or physically distributed / physically in arbitrary units according to various loads and usage conditions. Can be integrated and configured. Further, each processing function performed by each device may be realized by a CPU and a program analyzed and executed by the CPU, or may be realized as hardware by wired logic.

Further, among the processes described in the present embodiment, all or part of the processes described as being automatically performed can be manually performed, or the processes described as being manually performed. It is also possible to automatically perform all or part of the above by a known method. In addition, the processing procedure, control procedure, specific name, and information including various data and parameters shown in the above document and drawings can be arbitrarily changed unless otherwise specified.

(program)
It is also possible to create a program in which the processing executed by the information processing apparatus described in the above embodiment is described in a language that can be executed by a computer. For example, it is possible to create an information processing program in which the processing executed by the information processing apparatus 10 according to the embodiment is described in a language that can be executed by a computer. In this case, when the computer executes the information processing program, the same effect as that of the above embodiment can be obtained. Further, even if the information processing program is recorded on a computer-readable recording medium and the information processing program recorded on the recording medium is read and executed by the computer, the same processing as that of the above embodiment can be realized. Good.

FIG. 8 is a diagram showing a computer that executes an information processing program. As illustrated in FIG. 8, the computer 1000 has, for example, a memory 1010, a CPU 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. However, each of these parts is connected by a bus 1080.

The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012, as illustrated in FIG. The ROM 1011 stores, for example, a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090, as illustrated in FIG. The disk drive interface 1040 is connected to the disk drive 1100 as illustrated in FIG. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to, for example, a mouse 1110 and a keyboard 1120, as illustrated in FIG. The video adapter 1060 is connected, for example, to the display 1130, as illustrated in FIG.

Here, as illustrated in FIG. 8, the hard disk drive 1090 stores, for example, the OS 1091, the application program 1092, the program module 1093, and the program data 1094. That is, the above-mentioned information processing program is stored in, for example, the hard disk drive 1090 as a program module in which instructions executed by the computer 1000 are described.

Further, the various data described in the above embodiment are stored as program data in, for example, a memory 1010 or a hard disk drive 1090. Then, the CPU 1020 reads the program module 1093 and the program data 1094 stored in the memory 1010 and the hard disk drive 1090 into the RAM 1012 as needed, and executes various processing procedures.

The program module 1093 and program data 1094 related to the information processing program are not limited to the case where they are stored in the hard disk drive 1090, for example, are stored in a removable storage medium, and are read out by the CPU 1020 via a disk drive or the like. May be good. Alternatively, the program module 1093 and the program data 1094 related to the information processing program are stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.), and the network interface 1070 is used. It may be read by the CPU 1020 via.

The above-described embodiments and modifications thereof are included in the inventions described in the claims and the equivalent scope thereof, as are included in the technology disclosed in the present application.

10 Information processing device 11 Communication processing unit 12 Control unit 12a Collection unit 12b Processing unit 12c Coupling unit 12d Calculation unit 12e Output unit 13 Storage unit 13a Sensor data storage unit 20A to 20C Target equipment 21 Sensor 100 Analysis system

Claims (4)

A collection unit that collects the data acquired by each sensor installed in the target equipment, and
The data collected by the collecting unit is a fully connected layer whose weight is learned so as to cancel the dependency on the order between the data at each time of the data, and is used as the fully connected layer at the front stage of the neural network. A processing unit that performs processing to input and obtain output data,
A coupling unit that combines the output data obtained by the processing unit in the time direction to obtain a matrix, and
A calculation unit that performs a convolution operation on the matrix obtained by the connection unit, and
Possess an output section for outputting the result calculated by the arithmetic unit,
The neural network including the fully connected layer and the convolution layer that performs the convolution operation is an information processing apparatus that has been learned by an error backpropagation method. The information processing device according to claim 1, wherein the calculation unit performs a convolution calculation using GPGPU. An information processing method executed by an information processing device.
The collection process that collects the data acquired by each sensor installed in the target equipment, and
The data collected by the collection process is a fully connected layer whose weight is learned so as to cancel the dependency on the order between the data at each time of the data, and is used as the fully connected layer at the front stage of the neural network. The processing process of inputting and obtaining output data, and
A joining step of joining the output data obtained by the processing step in the time direction to obtain a matrix, and
A calculation step of performing a convolution operation on the matrix obtained by the combination step, and
Look including an output step of outputting the result calculated by the calculating step,
An information processing method characterized in that the neural network including the fully connected layer and the convolution layer that performs the convolution operation has been learned by an error backpropagation method. A collection step that collects the data acquired by each sensor installed in the target equipment, and
The data collected by the collection step is a fully connected layer whose weight is learned so as to cancel the dependency on the order between the data at each time of the data, and is the fully connected layer at the front stage of the neural network. Processing steps to input and obtain output data,
A combination step of combining the output data obtained by the processing step in the time direction to obtain a matrix, and a combination step.
An operation step that performs a convolution operation on the matrix obtained by the join step, and
Have the computer execute an output step that outputs the result calculated by the calculation step .
An information processing program characterized in that the neural network including the fully connected layer and the convolution layer that performs the convolution operation has been learned by an error backpropagation method.

Images