WO2024004903A1

WO2024004903A1 - Prediction device, prediction method, and prediction program

Info

Publication number: WO2024004903A1
Application number: PCT/JP2023/023474
Authority: WO
Inventors: 慶間島; 憲明八幡; 琢史 ▲柳▼澤; 良平福間; 晴彦貴島; 祥之白石; 吉伸河原; 宙人山下
Original assignee: 国立研究開発法人量子科学技術研究開発機構; 国立大学法人大阪大学
Priority date: 2022-06-29
Filing date: 2023-06-26
Publication date: 2024-01-04
Also published as: JP2024006994A

Abstract

Provided are a prediction device, a prediction method, and a prediction program that can make predictions from multidimensional time series data at high speed and with high accuracy. A prediction device 1, which analyzes multidimensional time series data regarding a predetermined event, and predicts events from the multidimensional time series data, includes a predictive means for acquiring multidimensional time series data, decomposes the multidimensional time series data into matrix data using dynamic mode decomposition, acquires features from the matrix data, acquires predicted values from the features and weights calculated in advance on the basis of the kernel method, and predicts events from the predicted values.

Description

Prediction device, prediction method, and prediction program

The present invention relates to, for example, a prediction device, a prediction method, and a prediction program that analyze multidimensional time series data regarding a predetermined event and predict the event from the multidimensional time series data.

In recent years, technology has been proposed that combines multidimensional time series data with machine learning algorithms to predict events related to multidimensional time series data. As such a prediction method, the output data obtained by applying dynamic mode decomposition to brain wave data as multidimensional time series data is input into a kernel machine using a Grassmann kernel, and the intention of movement (movement) is calculated. A reading method (reading brain information from brain wave data) has been proposed (Non-Patent Document 1).

In the prediction method described above, a characteristic peculiar to kernel machines is that during inference (prediction), it is necessary to repeat similar operations for the number of training data (sample data), and the calculation time required for prediction is proportional to the number of sample data. It becomes long. For this reason, although the prediction accuracy can be improved by increasing the number of sample data, there is a problem in that increasing the number of sample data increases the computation time.

In view of the above problems, the present invention aims to provide a prediction device, a prediction method, and a prediction program that can make predictions from multidimensional time series data at high speed and with high accuracy.

The present invention includes a data acquisition unit that acquires multidimensional time series data regarding a predetermined event, a decomposition unit that decomposes the multidimensional time series data into matrix data by dynamic mode decomposition, and a feature amount that is calculated from the matrix data. a second acquisition means for calculating a predicted value from the feature amount and a weight calculated in advance based on a kernel method; and a prediction means for predicting the event from the predicted value. The present invention is characterized by a prediction device, a prediction method, and a prediction program.

According to the present invention, it is possible to provide a prediction device, a prediction method, and a prediction program that can make predictions from multidimensional time-series data at high speed and with high accuracy.

FIG. 4 is an explanatory diagram of motion type classification in the basic technology of the present invention. Flowchart of the basic technology of the present invention. FIG. 1 is a block diagram showing an example of the configuration of a prediction device according to the present invention. FIG. 2 is a block diagram showing an example of the configuration of a data acquisition section and a result output section of the present embodiment. 5 is a flowchart of prediction processing according to the present invention. A graph showing the relationship between the number of sample data and calculation time in prediction processing. A graph showing the relationship between the number of sample data and calculation time in preprocessing.

An embodiment of the present invention will be described below.
<Basic technology of the present invention>

In recent years, a technology (hereinafter referred to as "basic technology") has been proposed that combines multidimensional time series data regarding a predetermined event with a machine learning algorithm to predict an event from the multidimensional time series data.

Such basic technology can be applied to, for example, BMI (Brain-Machine Interface). BMI predicts or infers (decodes) the movements and instructions that a person conjures up from a person's brain signals, and visualizes brain information, communicates intentions, or operates devices. Specifically, BMI can be measured by computers, robots (e.g., assist robots powered by electric actuators or artificial muscles attached to the human body), or communication devices based on movements (instructions) predicted from brain signals. Robot-operated BMI that controls external devices such as devices that can display messages or output artificial voices, and visual BMI that converts information obtained from brain signals into visualized image data and presents it. There is.

For example, with a robot-operated BMI, even people with physical disabilities can operate external devices by imagining (imagining) body movements in their brains. Specifically, if a robot-operated BMI is applied to a person (patient) who cannot move their hands, the robot-operated BMI acquires brain signals when the patient is recalling hand movements, This brain signal is decoded to obtain brain activity patterns (brain information) that indicate the content of the movement the patient is recalling, and electrical signals based on the brain information are sent to assist robots (such as artificial arms) that can be controlled by artificial nerves. The assist robot then executes the motion that the patient has imagined.

As a specific example of applying basic technology to BMI, dynamic mode decomposition is applied to brain wave data output from multiple electrodes used to detect brain signals (brain waves), and the resulting output data is processed using the Grassmann kernel. A technique has been proposed in which the intention of movement (motion) is read out by inputting it into a kernel machine using

FIG. 1 is an explanatory diagram of motion type classification in the basic technology of the present invention (cited from Non-Patent Document 1). FIG. 2 is a flowchart of the basic technology of the present invention. Hereinafter, before explaining the present invention, basic technology will be explained. In particular, we took electroencephalogram data (electrocorticogram (ECoG) signals) of 11 paralyzed patients (hereinafter referred to as "subjects") who were fitted with subdural electrodes as an example of multidimensional time-series data, and implemented the following. The basic technology and the content of the present invention will be explained while showing examples (experimental examples).

Table 1 shows data such as age, paralysis information, and diagnosis for each of the 11 subjects who participated in the experiment. The age range of the subjects was 13 to 66 years old, and the gender of the subjects was 4 females and 7 males. In addition, five of the subjects had different degrees of motor dysfunction and sensory impairment in the upper limbs due to stroke, but these subjects had damage to the sensorimotor cortex, a history of surgery to the motor cortex, or a history of surgery within the sensorimotor cortex. No brain tumor was detected. Furthermore, among the subjects, no motor dysfunction or sensory disorder was observed in 6 patients who exhibited epilepsy symptoms. All participants or their guardians consented to participate in this study with approval from the Ethics Committee of Osaka University Hospital. Note that 8 of the 11 subjects in this example were the same as those who participated in the past study (Yanagisawa et al. 2012 *1). *1: Yanagisawa T, Hirata M, Saitoh Y, Kishima H, Matsushita K, Goto T, Fukuma R, Yokoi H, Kamitani Y and Yoshimine T 2012 Electrocorticographic control of a prosthetic arm in paralyzed patients Ann. Neurol. 71 353-6 1

Brain wave data (ECoG signal) was recorded at 1 kHz using an EEG-1200 system (manufactured by Nihon Kohden Industries, Ltd.). Subdural electrodes were placed in the sensorimotor cortex based on clinical need. For each subject, 15-60 electrodes (channels) were implanted. All ECoG signals obtained from each subdural electrode were obtained by subtracting the average of the signals of all subdural electrodes at each time point. This is a common method used to calculate average references (Kubanek et al. 2012 *2). *2: Kubanek J, Miller K J, Ojemann J G, Wolpaw J R and Schalk G 2009 Decoding flexion of individual fingers using electrocorticographic signals in humans J. Neural Eng.6 066001

In addition, channels with severe noise were identified from each channel, and channels with severe noise were excluded from the analysis (in this example, three channels of subject 7 and one channel of subject 10 were removed). ).

As shown in FIG. 1, in this example, the subject performed three types of motions M1 to M3, and brain wave data during those motions was measured. Each of the motion types M1 to M3 is one of motions such as grasping the hand, opening the hand, grasping with the hand, and raising the thumb of the hand, and different motions are assigned to each motion type.

In the experiment, the subject was instructed to perform one of three movements M1 to M3 (action instructions), a sound cue was sent three times, and after the third sound cue, the subject performed the instructed movement. did. Movement instructions were communicated using a display placed in front of the subject. The series of steps from receiving this movement instruction to performing the movement and completing it is called one trial. Approximately 30 to 100 trials were performed for each movement M1 to M3.

In addition, in this experiment, the ECoG signal (S _execute ) 0.5 seconds after the third sound cue was executed, and the ECoG signal ( S execute ) 1.0 seconds before the first (first) sound cue. S _norm ) was used. In order to subtract the variation in the baseline value between different trials of the ECoG signal, S _execute was normalized with reference to S _norm in each trial. The normalization method was in accordance with Non-Patent Document 1. Hereinafter, a method for identifying (predicting) the type of exercise performed by a subject using the normalized S _execute as input will be explained in accordance with the basic technology.

As shown in FIG. 2, in the basic technology, when prediction processing is started, brain wave data (ECoG signal) as multidimensional time series data is acquired using the method described above (step S1), and the ECoG signal is converted into a dynamic mode. It is decomposed into matrix data by dynamic mode decomposition (step S2).

Hereinafter, the contents of step S2 (dynamic mode decomposition) will be briefly explained. Dynamic mode decomposition can perform dimension reduction by extracting low-dimensional bases called dynamic modes from high-dimensional time series data. Assuming that the value of the multidimensional ECoG signal at time t is x(t), in dynamic mode decomposition, the signal is approximated as shown in [Equation 1] below.

Here, Y is a complex-valued matrix of size D×P, Ω is a complex-valued diagonal matrix of size P×P, and z is a P-dimensional complex vector. D represents the number of electrodes of the ECoG signal (number of dimensions of multidimensional time series data), and P represents the number of low-dimensional bases (dynamic modes) assumed by the analyst. In dynamic mode decomposition, Y, Ω, and z suitable for approximating the given data are calculated based on the given multidimensional time series data x(t). In this example, a general algorithm using singular value decomposition was adopted for the calculation, and Y, Ω, and z were calculated from the ECoG signal (S _execute ) during exercise movement (Rowley et al. 2009 *3 , Schmid 2010 *4). *3: Rowley C W, Mezic I, Bagheri S, Schlatter P and Henningson D S 2009. 641 115-27 *4: Schmid P 2010 Dynamic mode decomposition of numerical and experimental data J. Fluid Mech. 656 5-28
In this embodiment, the number P of dynamic modes is 300.

As described above, in step S2, the ECoG signal is decomposed into matrix data by dynamic mode decomposition using the method described above. Next, a Grassmann kernel is calculated from the matrix data obtained in step S2 (step S3).

Hereinafter, the contents of step S3 will be explained. One matrix Y can be obtained by applying dynamic mode decomposition to the ECoG signal accompanying one exercise movement (one trial). As preprocessing using the kernel machine in step S4, the similarity between the ECoG signals of two different trials is evaluated by inputting a pair of corresponding matrices Y to the Grassmann kernel and using the calculated value (Hamm et al. . 2016 *5). *5: Hamm J and Lee D D 2008 Grassmann discriminant analysis: a unifying view on subspace-based learning Proc. 25th Int. Conf. on Machine Learning pp 376-83

First, assume that there are two matrices Y ₁ and Y ₂ of size D×P. Each column of the matrices Y ₁ and Y ₂ forms a D-dimensional vector, and represents how much each mode in the dynamic mode decomposition is mixed with which electrode. In the Grassmann kernel, the degree of similarity between matrices Y ₁ and Y ₂ is defined as the degree of overlap between the subspace spanned by the P vectors that make up the matrix Y ₁ and the subspace spanned by the P vectors that make up the matrix Y ₂ . is evaluated using the following formula [Math. 2].

Here, Y ₁ ⁺ and Y ₂ ⁺ are complex conjugate transposes of Y ₁ and Y ₂ , and tr(·) is a function that returns a trace of an input matrix. In addition, because daggerfu cannot be written in the text, it is written as " ⁺ " (the same applies hereafter). The norm on the second side in the equation represents the Frobenius norm for the matrix.

In step S3, the Grassmann kernel is calculated using the method described above. Subsequently, the kernel machine calculates the predicted value of the motion type (step S4), outputs the prediction result (step S5), and ends the prediction process.

Hereinafter, the contents of step S4 and step S5 will be explained. In this example, motion types were classified by support vector machine (SVM) using a Grassmann kernel. Furthermore, in this example, the classifier used was one trained individually for each subject by SVM.

Classification accuracy was evaluated by 10-fold nested cross-validation. The regularization parameters (cost parameters) of the SVM were optimized in the inner loop of nested cross-validation. Classification accuracy was defined by calculating the recall rate for each motor movement type and averaging the values across the three motor movements. The trained kernel machine is given in the form of [Equation 3] below.

Here, y is the predicted value of the motion type returned by the kernel machine, the function f is a function specified by the algorithm of the kernel machine, and N is the number of samples of training data. The real number α _n (n=1,...,N), b is a parameter calculated by the kernel machine training algorithm, the function k (・,・) is the Grassmann kernel, Y _n ∈C ^D×P is the training data The output obtained by subjecting the ECoG signal of the n-th sample to dynamic mode decomposition, Y _test εC ^D×P is the output obtained by subjecting the ECoG signal serving as test data to dynamic mode decomposition.

According to experiments using such basic technology, the prediction accuracy (prediction correct answer rate) when predicting movement types from brain wave data was 79%, and the frequency power feature value using fast Fourier transform (FFT) was found to be 79%. The prediction accuracy was significantly improved (67%). That is, by using the basic technology, it is possible to improve the prediction accuracy when predicting an event (in this embodiment, a motion type) from multidimensional time series data, compared to other conventional methods.

However, calculation of the Grassmann kernel used in the basic technology takes a long time to calculate, and the realistic setting is the number of dimensions of multidimensional time series data (number of electrodes in this example) D = 100, dynamic mode. When the number of decomposition modes P=300, it takes about 0.1 seconds. When making a prediction using the basic technology, this calculation must be repeated N times (as many times as the number of sample data) to obtain the predicted value y, and for example, if N=100, it will take about 10 seconds. In other words, with basic technology, a large number of sample data can improve prediction accuracy, but on the other hand, increasing the number of sample data increases the time required from the start of the calculation to the end of the calculation (computation time). was there. Another issue was that the basic technology assumes a kernel machine, and cannot be combined with machine learning algorithms that are not kernel machines.
<Prediction method of the present invention>

The present inventor conducted extensive research in order to shorten the calculation time to obtain the predicted value y. Then, he invented a calculation method that has an accuracy comparable to or higher than that of the basic technology and can significantly shorten calculation time, and invented a prediction device, a prediction method, and a prediction program using this calculation method.

The prediction method of the present invention will be explained below. The problem with the basic technology described above is that for Grassmann kernel k(Y ₁ , Y ₂ ), a nonlinear mapping φ(Y)(C ^D×P →C ^K (C is a complex number, K is a natural number) satisfies the following property. )), we can solve the problem. This is because by substituting [Math. 4] into [Math. 3] above, [Math. 3] can be transformed into [Math. 5]. Further, "W" in [Math. 5] is expressed as in [Math. 6].

Here, W is a K-dimensional vector and is a linear weight in the feature space after nonlinear transformation in the kernel method. This weight W includes processing equivalent to the repetitive calculation part in the original kernel machine, and can be calculated in advance before the multidimensional time series data (test data) Y _test to be predicted is given. .

Therefore, by calculating the predicted value using [Equation 5], it is possible to omit the step of repeating the Grassmann kernel calculation in [Equation 3] for the number of sample data N, and the amount of calculation can be reduced by the number of sample data N. Regardless of the amount of N, it can be kept constant.

Hereinafter, a method of constructing the nonlinear mapping φ(Y) that satisfies [Equation 4] will be described. In general, past research has shown that such a nonlinear mapping φ(Y) exists for any positive definite kernel if the number of dimensions K of the output vector is allowed to be infinite (Mercer 1909 *6). *6: Mercer, Philosophical Transactions of the Royal Society A, 1909

However, at present, there is no known method for specifically configuring the vector space that satisfies [Equation 4] while suppressing the number of dimensions K of the vector space to which the nonlinear mapping φ(Y) is output to a finite value. The method is given by [Equation 7] below.

Here, vec(YY ⁺ ) is a function that returns a column vector in which the elements of a given matrix are arranged vertically. Further, φ(Y) in [Equation 7] is a feature amount. By this construction method, a finite-dimensional nonlinear mapping φ(Y) specifically satisfying [Equation 4] was obtained. At this time, the value of the number of dimensions K of the vector space to which the nonlinear mapping φ(Y) is output is the square of the number of dimensions D of the multidimensional time series data. If the number of dimensions K of the vector space to which the nonlinear mapping φ(Y) is output is infinite, the calculations in [Equation 4] and [Equation 5] cannot be performed on a computer, and an infinite amount of calculation time is required. However, in the method of the present invention, the value of the number of dimensions K of the vector space to which the nonlinear mapping φ(Y) is output can be suppressed to a finite value.
Next, as an embodiment of the present invention, an example using the above-mentioned calculation method will be described with reference to the drawings.
<Hardware configuration>

FIG. 3 is a block diagram showing an example of the configuration of the prediction device 1. The prediction device 1 is composed of a general-purpose computer (terminal), and is a prediction device (estimation device) for predicting (estimating) an event from multidimensional time series data regarding a predetermined event according to a trained model using AI/machine learning technology. equipment).

As shown in FIG. 3, the prediction device 1 includes an input section 2, a display section 3, a data acquisition section 4, a result output section 5, a control section 6, and an auxiliary storage section 7. Each of the input section 2, display section 3, data acquisition section 4, result output section 5, and auxiliary storage section 7 is connected to the control section 6.

The control unit 6 includes a calculation unit 61 and a main storage unit 62, and executes various calculations and control operations in the prediction device 1. The calculation unit 61 is a calculation processing unit including a CPU, an MPU, or the like. The main storage unit 62 includes RAM (DRAM), ROM, and the like. The RAM is used as a work area and a buffer area for the calculation unit 61. The ROM stores a startup program for the prediction device 1, default values for various information, and the like.

The input unit 2 includes an input member that receives operational input from the user of the prediction device 1, and an input detection circuit interposed between the input member and the control unit 6. The input member is, for example, a touch panel and/or a hardware operation button or operation key. As the touch panel, any type of touch panel can be used, such as a capacitive type, an electromagnetic induction type, a resistive film type, an infrared type, etc. The input detection circuit outputs an operation signal or operation data to the control unit 6 according to the operation of each input member.

The display section 3 includes a display and a display control circuit interposed between the display and the control section 6. As the display, for example, an LCD (liquid crystal display) or an organic EL display can be used. The display control circuit includes a GPU, VRAM, and the like. Under instructions from the control unit 6, the GPU generates display image data in the VRAM for displaying various screens on the display using image generation data stored in the RAM, and displays the generated display image data on the display. Output to.

The data acquisition unit 4 acquires multidimensional time series data (multidimensional series signals) regarding a predetermined event. Multidimensional time series data is time series data with a number of dimensions D of 2 or more, such as data reflecting brain nerve activity such as brain waves (brain signals), joint position data, acoustic data, and behavioral data of collective organisms such as human flow. be.

The result output unit 5 outputs data (output data) corresponding to the prediction result (event) predicted (estimated) from the acquired multidimensional time series data. If the output data is visualized data, it can be output as image data and displayed on the display unit 3 or an external display device. Further, when the output data is operation data for operating an output device such as a computer or a robot (for example, an external device connected to the prediction device 1), it can be output to the output device as the operation data.

Hereinafter, in this embodiment, an example in which the prediction device 1 is applied to BMI will be described. FIG. 4 is a block diagram showing an example of the configuration of the data acquisition section 4 and the result output section 5. As shown in FIG. The example shown in FIG. 4 shows an example of the configuration of the data acquisition section 4 and the result output section 5 when the multidimensional time series data is electroencephalogram data (brain signals). As shown in FIG. 4, a brain signal measurement unit (brain signal measurement device) 41 is connected to the data acquisition unit 4. Furthermore, an output device 51 is connected to the result output section 5 .

The brain signal measurement unit 41 is any device that can measure brain signals indicating the state of human brain activity. As the brain signal measurement unit 41, for example, a scalp electroencephalograph that measures brain waves by attaching a plurality of electrodes to the scalp can be used. Alternatively, an intracranial electroencephalograph may be used in which electrodes are placed directly on the brain surface to measure cortical electroencephalograms. As yet another example, a magnetic resonance imaging device or a near-infrared spectroscopy device may be used as the brain signal measurement unit 41. Further, as the brain signal measuring section 41, a magnetoencephalograph can be used. A magnetoencephalograph is equipped with a plurality of superconducting quantum interference devices (SQUIDs), and measures and acquires brain magnetic field signals at a plurality of positions as brain signals.

The brain signal measurement unit 41 includes a measurement unit 42, an amplification unit 43, and an analog-digital (hereinafter referred to as A/D in the figures) conversion unit (A/D converter) 44.

The measurement unit 42 is a sensor that reads brain signals from a living body, and outputs an analog signal according to the brain signals. As the measurement section 42, an appropriate sensor is used depending on the type of the brain signal measurement section 41 exemplified above. For example, in the case of an electroencephalogram, an electrode can be used as the measurement section 42, and in the case of a magnetoencephalogram, a SQUID can be used as the measurement section 42. The amplification section 43 amplifies the analog signal output by the measurement section 42. For example, it is realized by an amplifier circuit equipped with an operational amplifier. The A/D converter 44 converts the analog signal amplified by the amplifier 43 into a digital signal (digital value). The A/D converter 44 is realized by, for example, an electronic circuit such as an analog-to-digital converter circuit.

The brain signals measured by the brain signal measurement section 41 are output to the data acquisition section 4. Note that, before being output to the data acquisition section 4, predetermined signal processing may be performed in the brain signal measurement section 41 as necessary. For example, noise may be reduced using a noise filter, or may be narrowed down to only signals in a specific frequency band using a bandpass filter.

The output device 51 is an output device (operating device) such as a computer or a robot. Output data (data of the event indicated by the predicted value) output from the result output unit 5 is input to the output device 51, and the output device 51 performs appropriate output (operation in this embodiment) according to the output data. For example, if the output device 51 is a robot, it operates according to output data. Further, if the output device 51 is an intention communication device, it outputs the intention according to the output data (displays a message, outputs an artificial voice, etc.). Furthermore, in the case of visual type BMI, the output device 51 is a display device (the display unit 3 or an external display device, etc.), and an image is displayed on the output device 51 according to output data. Note that the prediction device 1 and the output device 51 may be configured as separate devices, or the prediction device 1 and the output device 51 may be configured as one (such as a robot incorporating the prediction device 1).

Returning to FIG. 3, the auxiliary storage unit 7 is configured with other nonvolatile memories such as HDD, SSD, flash memory, and EEPROM, and is used by the control unit 6 (calculation unit 61) to control the operation of the prediction device 1. Stores programs and various data. Various data stored in the auxiliary storage section 7 are developed (read) in the main storage section 62 as needed.

The auxiliary storage unit 7 includes at least a main processing program 71 for executing various operations of the prediction device 1, a decomposition program 72 for decomposing multidimensional time series data into matrix data by dynamic mode decomposition, and a decomposition program 72 for decomposing the multidimensional time series data into matrix data by dynamic mode decomposition. A feature amount acquisition program 73 that acquires the amount, a predicted value acquisition program 74 that acquires the predicted value from the feature amount and the weight W (weight calculated (operated) in advance based on the kernel method), and sample data for prediction. , a prediction data management program 75 that manages preset weights W, data (prediction data) necessary for prediction processing of various arithmetic expressions, etc., and predicts events related to multidimensional time series data from predicted values. Prediction program 76 and the like are stored.

Further, the auxiliary storage unit 7 stores prediction data 77 necessary for prediction processing, prediction data 78 that is data of an event indicated by a predicted value, and the like. In this embodiment, the prediction data 77 includes data such as calculation formulas and algorithms for acquiring feature quantities from matrix data, and data such as calculation formulas and algorithms for acquiring predicted values from feature quantities and weights W. , data such as arithmetic expressions (including weight W data) and algorithms for predicting events related to multidimensional time series data from predicted values.
<Prediction processing flow>

In the prediction device 1 configured in this manner, the control unit 6 performs prediction processing for predicting events from multidimensional time series data according to each of the programs described above. FIG. 5 is a flowchart of the prediction method of the present invention. As shown in FIG. 5, when the control unit 6 starts the prediction process, it reads the data of the weight W calculated in advance based on the kernel method (step S11), and reads the data of the weight W calculated in advance based on the kernel method (step S11), and reads the data of the weight W calculated in advance based on the kernel method (step S11). signal) (step S12), and decomposes the multidimensional time series data into matrix data by dynamic mode decomposition (step S13). Note that the processing contents of step S12 and step S13 are the same as the processing contents of step S1 and step S2 of the basic technology described above, so a detailed explanation will be omitted.

Next, feature quantities are calculated from the matrix data obtained by dynamic mode decomposition (step S14). In step S14, the feature amount is calculated from the matrix data using the above-mentioned [Equation 7].

Next, a predicted value is calculated from the weight W and the feature amount obtained in step S14 (step S15), an event is predicted from the predicted value by linear discrimination or linear regression (step S16), and the prediction result is output (step S15). S17), the prediction process ends. In step S15, a predicted value is calculated using [Equation 5] described above.

FIG. 6 is a graph showing the relationship between the number N of sample data and calculation time in prediction processing. Note that the basic technology 2 in FIG. 6 is an optimization of the basic technology method described above.

In fact, when we conducted a computer experiment in a realistic situation using the prediction method of the present invention (number of dimensions of multidimensional time series data D = 100, number of modes of dynamic mode decomposition P = 300), we found that the Grassmann kernel The time required for one calculation of (Equation 2) was approximately 0.01 seconds. Furthermore, as described above, the basic technology has a problem in that the calculation time required for prediction increases in proportion to the number N of sample data (about 10 seconds under the same conditions as the computer experiment described above). On the other hand, the prediction method of the present invention requires only one inner product calculation using [Equation 5], so as shown in FIG. can be suppressed to

In addition, as a result of evaluating the prediction accuracy of the prediction method of the present invention when using the same brain wave data as the data used in the basic technology (brain wave data of 11 people), the prediction accuracy rate was the same accuracy as the basic technology (79%). Met. As mentioned above, the basic technology is superior to other methods in terms of prediction accuracy, and the prediction method of the present invention can significantly reduce calculation time while achieving predictions equivalent to the basic technology. It is. That is, by using the prediction method of the present invention, an event can be predicted at high speed (almost in real time) and with high accuracy from multidimensional time series data regarding a predetermined event.

In particular, in the prediction method of the present invention, the predicted value is calculated using [Equation 5] (predicted value calculation condition), which includes the weight W corresponding to the repeated calculation part in the kernel method used in the basic technology. We were able to achieve faster calculations. In addition, while suppressing the number of dimensions K of the vector space that is the output destination of the nonlinear mapping φ(Y) to a finite value, it satisfies [Equation 4] and specifically configures [Equation 7] (feature calculation conditions). The prediction method of the present invention was realized precisely because of the invention.

Furthermore, since the form of [Equation 5] uses φ(Y _test ) as the feature vector, it can be combined with any machine learning algorithm that assumes a single finite-dimensional feature vector as an input. can. For example, as a result of using a machine learning algorithm with sparse regularization (L1-SVM) on the feature vector obtained by [Equation 7], the prediction accuracy rate was 82%, and the prediction accuracy was further improved than the basic technology. . Furthermore, since it can be combined with any machine learning algorithm, the calculation time required for pre-processing (tuning processing, training) can be reduced. FIG. 7 is a graph showing the relationship between the number N of sample data and calculation time in preprocessing using linear SVM (L2-SVM) as a machine learning algorithm. Because of the relationship expressed in [Equation 4], the prediction results output from the machine learning algorithm of this example are equivalent to those of the basic technology. As shown in FIG. 7, it is possible to shorten the training time (calculation time for training the machine learning algorithm) while constructing a prediction model that is equivalent in basic technology and prediction accuracy. Even if the number N of sample data increases, the increase in training time can be suppressed, and training can be performed using a larger amount of training data than in the past within a realistic amount of time. In this way, the present invention has improved versatility compared to the basic technology, and can be combined with various machine learning algorithms even when applied to BMI. Moreover, the present invention is applicable to various fields other than BMI, including the examples described below.

The prediction device of the present invention corresponds to the prediction device 1 of the above embodiment, the data acquisition means corresponds to the data acquisition section 4, and the decomposition means corresponds to the decomposition program 72 and the control section 6 that operates according to the same. However, the first acquisition means corresponds to the feature value acquisition program 73 and the control unit 6 that operates according to this, and the second acquisition means corresponds to the predicted value acquisition program 74 and the control unit 6 that operates according to this, and the prediction means corresponds to the prediction program 76 and the control unit 6 that operates according to the prediction program 76, [Math. 1] corresponds to [Math. 5], and [Math. 2] corresponds to [Math. 7]. However, the present invention is not limited to this, and various other embodiments are possible. Further, the specific configurations mentioned in the above-described embodiments are merely examples, and can be changed as appropriate depending on the actual product.

For example, in the above-described embodiment, the multidimensional time series data is electroencephalogram data, but the multidimensional time series data is not limited to electroencephalogram data. For example, the multidimensional time-series data may be data representing the movement state of a group of organisms, such as the flow of people, such as joint position data, acoustic data, etc.

Specifically, multiple joint position data (position data of shoulders, elbows, knees, etc.) of an exercising person changes over time, so it can be treated as multidimensional time-series data. In this case, the prediction device 1 stores sample data of human postures, decomposes time-varying joint position data into dynamic modes, calculates feature quantities, and calculates predicted values. A human posture can be predicted (estimated) by performing a series of processes (steps S11 to S17) of the present invention including the following. For example, it can be used to analyze human motion during exercise. In this case, the data acquisition section 4 is connected to a device (such as an image analysis device or a three-dimensional coordinate measurement device) that acquires human joint position data, and the result output section 5 is connected to a display device or a computer for motion analysis. etc. are connected.

Furthermore, acoustic data in which the output of multiple different frequency bands changes over time can also be treated as multidimensional time-series data. In this case, the prediction device 1 stores sample data of normal sounds given in advance (normal sounds), and performs short-time Fourier transform on the acquired acoustic data. A series of processes according to the present invention are performed on the spectrogram, and the degree of abnormality can be evaluated by comparison with normal sounds (for example, using one-class SVM). Thereby, it is possible to predict (determine) whether the acquired acoustic data is normal or abnormal. For example, it is possible to detect an abnormality in a machine based on the sound of the machine in operation. In this case, the data acquisition section 4 is connected to a device (for example, a microphone) that acquires acoustic data (converts sound into an electrical signal), and the result output section 5 is connected to a display device, a computer for acoustic analysis, etc. A safety device for a machine in operation (for example, a device for stopping the machine when an abnormality is detected) is connected.

Furthermore, data that records temporal changes in the number of living things such as humans observed in a certain place (the number of specific living things existing in a certain space or area) over a given period of time (data such as human flow data) is also multidimensional. It can be treated as series data. In this case, the prediction device 1 stores sample data during normal times given in advance, performs a series of processes according to the present invention on the obtained data such as the flow of people, and determines the degree of abnormality by comparing with the data during normal times. Evaluation (eg, using a one-class SVM) can be performed. Thereby, it is possible to predict (determine) whether the acquired data such as the flow of people is normal or abnormal. In this case, the data acquisition unit 4 is connected to a device that acquires data such as the flow of people (a camera that photographs the observation target location or a human detection sensor that detects the number of living things such as humans existing at the observation target location), and the result is A display device, a computer for analyzing the flow of people, etc. are connected to the output unit 5.

Furthermore, resting brain activity data of multiple brain regions measured by an MRI (magnetic resonance imaging) device can also be treated as multidimensional time series data. In this case, the prediction device 1 uses pre-given sample data of mental illnesses and sample data for each severity level as training data, performs a series of processes according to the present invention on the acquired resting brain activity data, and uses the training data as training data. and evaluation based on machine learning algorithms (eg, using linear SVM, L1-SVM, logistic regression, forward correlation analysis, neural networks, etc.). This makes it possible to predict (judge) in real time whether a subject has a mental illness and the severity of the mental illness from the acquired resting brain activity data, and in turn, to perform brain information feedback control in real time using the prediction results. can. In this case, the data acquisition unit 4 is connected to an MRI apparatus that acquires resting brain activity data, and the result output unit 5 is connected to a display device or the like.

Furthermore, in the above-described embodiment, each of the feature quantities and the predicted value is obtained using an arithmetic expression when obtaining the feature quantity from the matrix data and when obtaining the predicted value from the feature quantity. Not limited. For example, the conditions for acquiring feature quantities from matrix data (feature quantity acquisition conditions) and the conditions for acquiring predicted values from feature quantities (predicted value acquisition conditions) may be in a table format created from past calculation results, or It can also be configured as an AI technology based on the calculation results. Furthermore, in the above embodiment, vec(YY ⁺ ) in [Equation 7] is a vector in which all the elements of the matrix (YY ⁺ ) are arranged vertically, but some elements of the matrix (YY ⁺ ) You may also use

Further, the present invention is not only provided as a prediction device, but also as a method and program for detecting an event using the prediction device, and a non-temporary (non-transient) tangible storage medium storing the program. can be provided.

This invention can be used in industries where multidimensional time series data regarding a predetermined event is analyzed and events are predicted from the multidimensional time series data.

1...Prediction device 4...Data acquisition unit 5...Result output unit 6...Control unit 61...Calculation unit 62...Main storage unit 7...Auxiliary storage unit 72...Decomposition program 73...Feature amount acquisition program 75...Predicted value acquisition program 76... Prediction program

Claims

a data acquisition means for acquiring multidimensional time series data regarding a predetermined event;
decomposition means for decomposing the multidimensional time series data into matrix data by dynamic mode decomposition;
a first acquisition means for acquiring feature quantities from the matrix data;
a second acquisition means for acquiring a predicted value from the feature amount and a weight calculated in advance based on a kernel method;
A prediction device comprising prediction means for predicting the event from the predicted value.
The prediction device according to claim 1, wherein the weight is a linear weight in a feature space after nonlinear transformation in a kernel method.
The first acquisition means calculates the feature from the matrix data according to a feature calculation condition that constitutes a finite-dimensional nonlinear mapping that satisfies the predicted value calculation condition for calculating the predicted value from the feature in the second acquisition means. The prediction device according to claim 1 or 2, which calculates a quantity.
The multidimensional time series data is brain wave data output from each of a plurality of electrodes used for detecting brain waves,
The prediction device according to claim 3, wherein the prediction means predicts the event in real time based on the predicted value obtained from the brain wave data.
4. The prediction device according to claim 3, wherein the second acquisition means acquires the predicted value by calculation using the following [Equation 1].
6. The prediction device according to claim 5, wherein the first acquisition means acquires the feature amount by calculation using the following [Equation 2].
Obtain multidimensional time series data regarding a predetermined event,
Decomposing the multidimensional time series data into matrix data by dynamic mode decomposition,
Obtain features from the matrix data,
Obtaining a predicted value from the feature amounts and weights calculated in advance based on the kernel method,
A prediction method for predicting the event from the predicted value.
computer,
a data acquisition means for acquiring multidimensional time series data regarding a predetermined event;
decomposition means for decomposing the multidimensional time series data into matrix data by dynamic mode decomposition;
a first acquisition means for acquiring feature quantities from the matrix data;
a second acquisition means for acquiring a predicted value from the feature amount and a weight calculated in advance based on a kernel method;
A prediction program that functions as a prediction means for predicting the event from the predicted value.