WO2019190185A1

WO2019190185A1 - Time series data learning and analyzing method using artificial intelligence

Info

Publication number: WO2019190185A1
Application number: PCT/KR2019/003540
Authority: WO
Inventors: 오규삼; 전은주; 권순환; 손형관; 윤용근; 김민수; 여현주
Original assignee: 삼성에스디에스 주식회사
Priority date: 2018-03-30
Filing date: 2019-03-27
Publication date: 2019-10-03

Abstract

A time series data learning and analyzing method using artificial intelligence is provided. A time series data analyzing method, according to one embodiment of the present invention, is performed by a computing device and can comprise the steps of: inputting a feature of each unit into an intermediate artificial neutral network with respect to each of the plurality of units in which time series data is split at a time axis; obtaining m (m is a natural number of 2 or greater) dimensional intermediate output data from the intermediate artificial neutral network; inputting, into a final artificial neutral network, the intermediate output data of a plurality of units that are temporally immediately adjacent, so as to obtain final output data outputted from the final artificial neutral network; and generating an analysis result of the time series data by using the final output data.

Description

Time series data learning and analysis method using artificial intelligence

The present invention relates to a method for learning and analyzing time series data using artificial intelligence. Specifically, the present invention relates to a method for performing time series data learning and analysis using a plurality of artificial neural networks.

With the digitization of information and the development of data storage technology, a large amount of data is accumulated and machine learning technology is introduced and utilized in various fields. Machine learning is a technique that analyzes a large number of input data to probably classify objects or predict values within a specific range. Machine learning works by empirically analyzing a large number of input data and probabilistically deriving a result rather than deriving a result by a specific rule.

However, in learning and analyzing time series data using artificial intelligence, there is a problem in that time series data may have a very long length compared to image data. Taking ECG data as an example, it is necessary to comprehensively learn and analyze electrocardiogram data measured for 24 hours or more to derive a diagnosis result, which is not only time-consuming during learning and analysis, but also causes the initial input data to be gradually blurred. Dilution can cause problems in accuracy in learning and analyzing long-term data.

The technical problem to be solved by the present invention is to provide a method and apparatus for reducing the amount of computation of time series data learning and analysis using artificial intelligence.

Another technical problem to be solved by the present invention is to provide a method and apparatus that can improve the accuracy of time series data learning and analysis using artificial intelligence.

The technical problem to be solved by the present invention is to provide a method and apparatus for reducing the amount of computation of ECG learning and analysis using artificial intelligence.

The technical problem to be solved by the present invention is to provide a method and apparatus that can improve the accuracy of ECG learning and analysis using artificial intelligence.

The technical problems of the present invention are not limited to the above-mentioned technical problems, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

According to another aspect of the present invention, there is provided a method for analyzing time series data, the method for analyzing time series data performed by a computing device, for each of a plurality of units in which the time series data is split on a time axis. Inputting a feature of each of the units into an intermediate neural network, obtaining intermediate output data of the dimension m (m is a natural number of two or more) from the intermediate neural network, wherein the intermediate of the plurality of units immediately adjacent in time Inputting output data into a final artificial neural network to obtain final output data output from the final artificial neural network, and generating an analysis result of the time series data using the final output data.

In one embodiment, the intermediate neural network and the final artificial neural network may be a Recurrent Neural Network (RNN).

In one embodiment, the neurons of the input layer of the final artificial neural network may be m.

In one embodiment, the step of obtaining the intermediate output data, the first computing device inputs the features of the first unit included in the plurality of units into the intermediate neural network implemented in the first computing device and the intermediate Obtaining m-dimensional intermediate output data from the artificial neural network, and the second computing device inputs the features of the second unit included in the plurality of units into the intermediate neural network implemented in the second computing device and Obtaining m-dimensional intermediate output data from the neural network.

In one embodiment, the step of obtaining final output data comprises: receiving, by a third computing device, intermediate output data of the first unit from the first computing device, and intermediate of the second unit from the second computing device. Output data, the intermediate output data of the first unit and the intermediate output data of the second unit are sequentially input to the final artificial neural network, wherein the first unit and the second unit are immediately adjacent units in time. It may include a step.

In one embodiment, the intermediate artificial neural network is composed of k (k is a natural number of two or more), the inputting to the intermediate artificial neural network, inputting the features of each of the plurality of units into the level 1 intermediate neural network Step, inputting level i output data obtained from the level i (i is one or more natural number) intermediate neural network into the level i + 1 intermediate neural network, and obtaining level i + 1 output data from the level i + 1 intermediate neural network; Including the step, wherein the intermediate output data may be data output from the level k intermediate neural network.

In one embodiment, the time series data may be electrocardiogram data.

In an embodiment, the analysis result may be an electrocardiogram diagnosis result.

In one embodiment, the n units may be beat units split using R-peak.

In one embodiment, the split n bit units compare the similarity between the split bits after splitting the time series data, and replace the bit having a similarity with a specific specific bit above a predetermined value with the feature unit. It may be.

According to another aspect of the present invention, there is provided a method for learning time series data, the method for learning time series data performed by a computing device, for each of a plurality of units in which the time series data is split on a time axis. Inputting a feature of each of the units into an intermediate neural network, obtaining intermediate output data of the dimension m (m is a natural number of two or more) from the intermediate neural network, wherein the intermediate of the plurality of units immediately adjacent in time Inputting the output data to the final artificial neural network to obtain final output data output from the final artificial neural network, and comparing the final output data with a label attached to the time series data.

In one embodiment, the comparing step includes the step of comparing the final output data with a label attached to the time series data to calculate an error, and after the comparing step, back propagating the error to the intermediate The method may further include adjusting a weight of the artificial neural network and a weight of the final artificial neural network.

In one embodiment, the adjusting of the weight may include adjusting the weight of the intermediate neural network by making the learning rate of the intermediate artificial neural network 1 / n times the learning rate of the final artificial neural network. have.

In one embodiment, the time series data may be electrocardiogram data.

In one embodiment, the n units may be beat units split using R-peak.

In accordance with another aspect of the present invention, there is provided an apparatus for analyzing time series data, the network interface receiving time series data, one or more processors, a memory for loading a computer program executed by the processor, and the computer. Storage for storing a program, wherein the computer program is configured to input a feature of each of the units into an intermediate neural network for each of a plurality of units in which the time series data is split on a time axis; an instruction for obtaining intermediate output data in the dimension m (m is a natural number of 2 or more); an instruction for inputting the intermediate output data of a plurality of units immediately adjacent in time to a final neural network, and obtaining the final output data output from the final neural network. , And above By using a species of output data may include instructions to generate analysis results of the time-series data.

According to another aspect of the present invention, there is provided an apparatus for learning time series data, the network interface receiving time series data, one or more processors, a memory for loading a computer program executed by the processor, and the computer. Storage for storing a program, wherein the computer program is configured to input a feature of each of the units into an intermediate neural network for each of a plurality of units in which the time series data is split on a time axis; an instruction for obtaining intermediate output data in the dimension m (m is a natural number of 2 or more); an instruction for inputting the intermediate output data of a plurality of units immediately adjacent in time to a final neural network, and obtaining the final output data output from the final neural network. , And above A bell output data may comprise instructions to compare the label stuck on the time-series data.

According to another aspect of the present invention, there is provided a recording medium for inputting a feature of each unit to an intermediate neural network for each of a plurality of units in which time series data is split on a time axis. Obtaining intermediate output data of the dimension m (m is a natural number of two or more) from the intermediate artificial neural network; inputting the intermediate output data of a plurality of units immediately adjacent in time into a final artificial neural network and outputting the final artificial neural network; A computer readable recording medium having a computer program stored therein for performing the step of obtaining final output data and generating an analysis result of the time series data using the final output data.

According to various embodiments of the present invention described above, when time series data learning and analysis using artificial intelligence is performed, the amount of calculation may be reduced and the accuracy may be improved.

According to various embodiments of the present invention described above, when performing ECG data learning and analysis using artificial intelligence, the amount of calculation may be reduced and accuracy may be improved.

Effects of the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1A and 1B are diagrams illustrating a wearable electrocardiogram measuring apparatus and an attachment method thereof.

2 is a diagram illustrating an electrocardiogram analysis system including an electrocardiogram measuring device, a network 210, and a server.

3A and 3B illustrate a conventional RNN-based electrocardiogram analysis method.

4 is a flowchart illustrating an electrocardiogram analysis method using artificial intelligence according to an embodiment of the present invention.

5 is a diagram illustrating an ECG signal splitting method according to an embodiment of the present invention.

6A through 6C illustrate a pseudo bit processing method according to an embodiment of the present invention.

7A and 7B are diagrams illustrating an intermediate artificial neural network and a final artificial neural network according to an embodiment of the present invention.

8 to 10 are diagrams illustrating a method for analyzing time series data using artificial intelligence according to an embodiment of the present invention.

11 is a flowchart illustrating a method of analyzing time series data using artificial intelligence according to an embodiment of the present invention.

12 is a diagram illustrating a case where an intermediate artificial neural network is implemented at a plurality of levels in an artificial neural network structure according to an embodiment of the present invention.

FIG. 13 is a flowchart illustrating a method of analyzing time series data when an intermediate artificial neural network is implemented at a plurality of levels in a structure of an artificial neural network according to an embodiment of the present invention.

14 is a flowchart illustrating an ECG learning method using artificial intelligence according to an embodiment of the present invention.

15 is a diagram illustrating training data used in an ECG learning method using artificial intelligence according to an embodiment of the present invention.

16 is a diagram illustrating a time series data learning method using artificial intelligence according to an embodiment of the present invention.

17 is a flowchart illustrating a method of learning time series data using artificial intelligence according to an embodiment of the present invention.

18 is a hardware configuration diagram of an exemplary computing device that can implement a time series data analysis method or time series data learning method using artificial intelligence according to embodiments of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Advantages and features of the present invention and methods for achieving them will be apparent with reference to the embodiments described below in detail with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but can be implemented in various different forms, and only the embodiments make the disclosure of the present invention complete, and the general knowledge in the art to which the present invention belongs. It is provided to fully inform the person having the scope of the invention, which is defined only by the scope of the claims. Like reference numerals refer to like elements throughout.

Unless otherwise defined, all terms used in the present specification (including technical and scientific terms) may be used in a sense that can be commonly understood by those skilled in the art. In addition, the terms defined in the commonly used dictionaries are not ideally or excessively interpreted unless they are specifically defined clearly. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase.

As used herein, “comprises” and / or “comprising” refers to a component, step, operation, and / or element that is mentioned in the presence of one or more other components, steps, operations and / or elements. Or does not exclude additions.

Prior to the description herein, some terms used herein will be clarified.

In the technical field to which the present invention belongs, 'neural network' refers to a graph structure composed of a plurality of layers and a plurality of nodes constituting each layer, which are modeled after a neural network of a living organism, especially a human visual / audio cortex. . The artificial neural network comprises one input layer, one or more hidden layers and one output layer.

In the technical field to which the present invention pertains, 'input layer' refers to a layer that receives data to be analyzed / learned in a layer structure of an artificial neural network.

In the technical field to which the present invention belongs, the term 'output layer' refers to a layer in which a result value is output in a layer structure of an artificial neural network.

In the technical field to which the present invention pertains, 'hidden layer' refers to all layers except the input layer and the output layer in the layer structure of the artificial neural network. The neural network consists of a continuous layer of neurons, with neurons in each layer connected to neurons in the next layer. If the input layer and the output layer are directly connected without the hidden layer, each input contributes to the output independently of the other inputs, making it difficult to obtain accurate results. In practice, input data is interdependent and coupled to each other to affect the output in a complex structure, so that hidden layers can be added to capture subtle interactions between inputs where the neurons in the hidden layer affect the final output. In other words, the hidden layer can be thought of as processing a high level of features or attributes in the data.

In the technical field to which the present invention pertains, 'neuron' is a concept in which neurons in biological neural networks correspond to artificial neural networks. Neurons are also called nodes.

In the technical field to which the present invention belongs, 'weight' refers to a concept in which a degree of synaptic connection is strengthened through repeated signal transmission in a biological neural network corresponds to an artificial neural network. Frequent signal transmission from neuron 1 to neuron 2 in biological neural networks enhances the pathway from neuron 1 to neuron 2, the synaptic linkage, for efficient signaling. This is regarded as a kind of learning or memory process, and the artificial neural network describes this as the concept of 'weighting'.

In the technical field to which the present invention belongs, 'back propagation' is a word derived from 'backward propagation of errors', and the back propagation of an error (error) that is a difference between an output value and an actual value is used. This means adjusting the weight of the artificial neural network.

In the technical field to which the present invention belongs, 'gradient descent' is a method of determining the weight of a learning model during backpropagation, and minimizes a difference between an output value of a feedforward function value including a weight and a labeled actual value. It means how to find the weight. For example, when the shape of the loss function is expressed as a parabolic shape, it is possible to calculate the weight by finding the lowest point, and the process of finding the lowest point is called a slope reduction method because it looks like a descending sloped hill.

In the technical field to which the present invention pertains, 'learning rate' means the size of a step for finding the lowest value in the gradient reduction algorithm for weight adjustment of an artificial neural network. Larger learning rates result in faster adjustments but less precision, and smaller learning rates provide higher precision but slower adjustments.

In the technical field to which the present invention belongs, 'Convolutional Neural Networks (CNN)' is a deep learning model that combines an artificial neural network with a filter technology, and the artificial neural network is optimized to acquire characteristics of input data better. . Convolutional neural networks have shown excellent performance in the field of computer vision.

In the technical field to which the present invention pertains, 'Recurrent Neural Network (RNN)' is a deep learning model suitable for learning data that changes over time, such as time series data. It is an artificial neural network constructed by connecting networks at (t) and the next time point (t + 1). The cyclic neural network has a feature that a connection between nodes has a cyclic structure, and this structure enables the neural network to effectively process time-varying features. The cyclic neural network can effectively process input in the form of a sequence and shows good performance in processing time series data having time-varying features such as handwriting recognition and speech recognition.

Artificial intelligence-related technologies and terms described above are described in detail on websites such as Wikipedia and YouTube, and thus detailed descriptions thereof will be omitted. For example, circulatory artificial neural networks are described in detail in https://en.wikipedia.org/wiki/Recurrent_neural_network.

In the present specification, 'time series data' means data listed in chronological order. However, it is not always necessary to be continuous in time, and even discrete data may be 'time series data' as used herein if they are listed in time order.

In the present specification, the term 'unit' means each entity when time series data is split into two or more entities on the time axis.

In the present specification, 'feature' means a feature of time series data or a unit extracted from time series data or a unit.

In the present specification, the "intermediate artificial neural network" means an intermediate neural network that receives input data and outputs intermediate output data. The intermediate output data differs from the output values of ordinary neural networks in that the intermediate output data is not the value to be finally obtained using artificial intelligence.

In the present specification, the "final artificial neural network" refers to an artificial neural network of a final step of receiving intermediate output data from an intermediate artificial neural network and outputting final output data. The final artificial neural network is an artificial neural network located at a terminal of an artificial intelligence system composed of a plurality of artificial neural networks according to an embodiment of the present invention, and the final output data output from the final artificial neural network is finally obtained by using artificial intelligence. The value to get.

The present invention to be described below relates to a method for learning and analyzing time series data using artificial intelligence. Specifically, the present invention discloses a method for learning and analyzing time series data that performs learning and analysis of time series data using a plurality of artificial neural networks.

An electrocardiogram is illustrated as an example of time series data in order to specifically describe the present invention. Specifically, a method of learning and analyzing time series data using artificial intelligence according to an embodiment of the present invention will be described by explaining a process of using a plurality of artificial neural networks according to an embodiment of the present invention in ECG learning and analysis. .

Hereinafter, with reference to the accompanying drawings will be described in detail with respect to the present invention.

Electrocardiogram is an analysis of the heart's electrical activity and recorded in the form of wavelengths. An electrocardiogram is a graph of potentials related to a heartbeat recorded on the surface of the body and is the most widely used test for diagnosing circulatory diseases.

Electrocardiograms are frequently used to diagnose arrhythmias and coronary artery disease. If the arrhythmia occurs intermittently, it is necessary to obtain an electrocardiogram recorded during daily life because the cardiac arrhythmias cannot be diagnosed by only one ECG test. Infrequent arrhythmias may not be detected by short-term ECG recordings, so if you suspect arrhythmias, you should record and record the ECG for a long time to make an accurate diagnosis.

The conventional electrocardiogram measuring device was a method of measuring by attaching a large number of electrodes to the body. For example, Holter ECG (Holter monitor) that is commonly used was a method of measuring the electrocardiogram by attaching 5-7 electrodes to the chest and wearing an inspection device weighing about 500 g on the waist. Conventional electrocardiogram measuring device is large and heavy in size, there is a problem that many electrodes are attached to the body there is a sense of rejection and interfere with free daily life.

In order to solve this problem, a wearable electrocardiogram measuring apparatus that has been recently developed reduces the size and weight of the device and minimizes the number of electrodes. The reason for minimizing the number of electrodes is to consider the convenience of the user's daily life and the battery performance of the ECG device.

FIG. 1A illustrates the wearable electrocardiogram measuring apparatus 100, and as shown in FIG. 1A, the number of electrodes is two.

FIG. 1B illustrates a method of measuring an electrocardiogram by attaching the wearable electrocardiogram measuring apparatus 100 to a body. As shown in FIG. 1B, an electrocardiogram is measured by attaching an electrode near the heart.

The wearable electrocardiogram measuring apparatus 100 is easier to carry than the conventional electrocardiogram measuring apparatus, but the conventional electrocardiogram measuring apparatus analyzes the electrocardiogram signal of a multi-channel by using a plurality of electrodes to measure and measure a relatively accurate electrocardiogram. Compared to the diagnosis, the wearable electrocardiogram measuring apparatus 100 has a problem in that the ECG signal of only one channel is measured and thus the accuracy of ECG measurement and diagnosis is inferior.

In the wearable electrocardiogram measuring apparatus 100, a conventional electrocardiogram measuring apparatus, such as an ECG analysis method used in a Holter ECG, cannot be used as it is, and there is a difficulty in diagnosing arrhythmias using only one channel ECG signal. . Nevertheless, since the wearable ECG measuring apparatus 100 has strengths such as convenience of carrying, there are many studies on a method of diagnosing arrhythmias using only one channel ECG signal measured by the wearable ECG measuring apparatus 100. It's going on. In particular, advances in deep learning techniques in the field of artificial intelligence have shown the possibility of achieving arrhythmia diagnosis with high accuracy with only one signal.

However, the apparatus for measuring the electrocardiogram of the examinee in the present invention is not limited to the wearable electrocardiogram measuring apparatus 100, and may be attached to the body to measure the electrocardiogram. Hereinafter, for convenience of description, the apparatus for measuring an electrocardiogram of a subject in the present invention is unified as an 'electrocardiogram measuring apparatus 100'.

2 is a diagram illustrating an ECG analysis system including an ECG measuring apparatus, a network, and a server.

The electrocardiogram measuring apparatus 100 only measures the electrocardiogram of a subject, and in many cases does not provide an analysis function by itself, the electrocardiogram measuring apparatus 100 performs only the electrocardiogram measurement, and the electrocardiogram analysis is performed at the server 200 stage. It is desirable to make it. Specifically, the electrocardiogram measuring apparatus 100 may measure the electrocardiogram and transmit the electrocardiogram to the server 200, and analyze the electrocardiogram data using artificial intelligence and the like at the server 200.

As shown in FIG. 2, an ECG analysis system according to an exemplary embodiment may include an ECG measuring apparatus 100, a network 210, and a server 200.

The examinee lives daily life with the ECG device 100 attached thereto. The electrocardiogram data measured by the electrocardiogram measuring apparatus 100 is transmitted to the server 200 through the network 210. In this case, the server 200 may be a server of a hospital, a server of a medical service provider, a server of an IT company, or a server of a government agency such as the Ministry of Health and Welfare. Since the ECG data of the individual is sensitive medical information, the server 200 is preferably a server of a company that is allowed to collect and analyze medical information.

The server 200 performs learning and analysis using the electrocardiogram data received from the electrocardiogram measuring apparatus 100. In the server 200 learning the electrocardiogram data, a medical professional, such as a cardiologist, may allow the electrocardiogram data labeled with the electrocardiogram diagnosis result to be learned.

Due to the nature of artificial intelligence, the neural network learning requires a long time and high computing power, while the relatively short time and low computing power are sufficient for the application of the neural network. Therefore, the ECG data is to be performed at the server 200, but the analysis of the ECG data is performed by ECG measuring apparatus 100 itself, an ECG measuring apparatus 100, and a separate ECG analyzing apparatus connected to wire / wireless, ECG measuring apparatus. 100 and can be performed in a smartphone, tablet, notebook computer and the like connected via wired / wireless.

However, in the present specification, a method of learning and analyzing electrocardiogram data according to an embodiment of the present invention will be described on the premise that all of the electrocardiogram data is learned and analyzed in the server 200. Such a description is that the ECG learning and analysis of the ECG data itself, the ECG measuring device 100 and a separate ECG device connected to the wired / wireless, ECG measuring device 100 and a smart phone connected to the wire / wireless It does not preclude it from being performed on tablets, tablets, notebook computers, etc.

The method of learning and analyzing ECG data using AI has been based on CNN. CNN is a deep learning model that combines artificial neural networks with filter technology, and shows excellent performance in computer vision fields such as image analysis. However, since ECG is a time series data having a sequential context unlike an image, there is a limit in analyzing an ECG using a CNN technique specialized for image analysis.

The CNN-based ECG method does not read the sequential context of the entire ECG signal, but rather identifies the features of a part of the ECG signal and combines these features to derive the result of the entire ECG analysis. The CNN-based electrocardiogram analysis method analyzes electrocardiogram data from the microscopic view rather than the macroscopic view, so that when a part of the electrocardiogram signal contains noise, it is difficult to grasp the characteristics of the corresponding electrocardiogram signal discrimination accuracy. Will adversely affect. In addition, when there are a number of conflicting features in a signal, the filtered feature is not considered in the diagnosis, which leads to a problem that the discrimination accuracy is significantly reduced.

In order to overcome this problem, research on the ECG analysis method using RNN is in progress.

3A illustrates an ECG signal measured for a predetermined time in the ECG measuring apparatus 100.

The sharply raised portion of the ECG signal illustrated in FIG. 3A is R-peak. R-peak is explained in detail at https://en.wikipedia.org/wiki/QRS_complex and so the detailed description is omitted.

FIG. 3B illustrates that an ECG signal measured for a predetermined time in the ECG measuring apparatus 100 is input to the RNN in units of three beats.

The ECG signal measured for a certain time is split in beat units using R-peak. The RNN inputs three immediately adjacent bits together in time, because the relationship between the pre-bit, the bit to be discriminated, and the post-bit is important for discriminating arrhythmia from the ECG signal. That is, the prior bit, the bit to be determined, the interval between occurrence of the after bit, the height difference, and the like are important factors for arrhythmia determination.

When an ECG signal is divided into 9 beats and each bit is named A, B, C, D, E, F, G, H, I, RNN has A + B + C, B + C + D, C + D + E, D + E + F, E + F + G, F + G + H and G + H + I are entered.

As shown in Figure 3b, A + B + C, B + C + D, C + D + E, D + E + F, E + F + G, F + G + H and G + H in one RNN + I is input sequentially.

When one bit is sampled at 250 Hz, the ECG signal of FIG. 3A is analyzed using a conventional RNN, and all 7 (the number of bits to be discriminated) Х3 (three adjacent bits are input together) Х250 (each bit Number of Sampling) = 5250 Length input must be processed.

In addition, since arrhythmia does not always appear, it is necessary to collect and analyze an ECG signal of a subject over a long period of time. Healthy adults have a heart rate of 60 to 100 beats per minute, which translates into 86,400 to 144,000 beats per 24 hours. Assuming a heart rate of 80 beats per minute, a 24 hour heart rate is 115,200 beats. This means that 80 bits are measured in 1 minute and 115,200 bits are measured in 24 hours.

To determine each of the 115,200 bits, the pre- and post-bits for each bit must be considered together, so when sampling one bit at 250 Hz, the RNN is approximately 115,200 (the number of bits to be determined) Х3 (neighboring 3 Two bits together) Х250 (number of samples for each bit) = 86,400,000

CNN-based ECG analysis is faster than RNN-based ECG analysis, but the accuracy of analysis is inferior because it cannot read the sequential context of ECG signals. On the other hand, RNN-based ECG analysis can read the sequential context of ECG signals, but it is difficult to guarantee real-time ECG analysis because the length of the input to be processed is too long.

As shown in Figure 4, ECG analysis method using artificial intelligence according to an embodiment of the present invention, ECG signal collection step (S411), ECG signal preprocessing step (S412), R-peak detection step (S413), ECG Signal split step S421, bit significance determination step S422, pseudo bit processing step S430, feature extraction step S440, intermediate neural network analysis step S450, final neural network analysis step S460 And the determination result reporting step (S470).

The electrocardiogram signal collecting step (S411) is a step in which the server 200 receives and collects an electrocardiogram signal from the electrocardiogram measuring apparatus 100.

The ECG signal preprocessing step (S412) is a step of processing noise, signal disconnection, signal interference, etc. mixed in the ECG signal. Specifically, the ECG signal received from the ECG measuring apparatus 100 may be preprocessed using a conventionally known method such as a band pass filter.

The R-peak detection step S413 is a step of detecting the R-peak in the ECG signal. When detecting R-peak, Pan-Tompkins algorithm, Hilbert transform, Kathirvel et al (An efficient R-peak detection based on new nonlinear transformation and first-order Gaussian differentia-tor.Cardiovascular Engineering and Technology and various conventionally known detection algorithms can be used.

The ECG signal splitting step S421 is a step of splitting the ECG signal bit by bit using the detected R-peak.

The step of determining the significance of the bit (S422) is a step of determining whether the split bit corresponds to the actual heartbeat using R-peak. For example, if sudden noise is incorrectly determined as R-peak, the split bits based on the incorrectly determined R-peak do not correspond to the actual heartbeat.

Since human heartbeats have similar lengths, too short or long bits can be considered as false detections during the R-peak detection phase. Specifically, if a bit split using R-peak is statistically too long (e.g., greater than 99 percentile) or too short (e.g., less than 1 percentile), false detection may occur in the R-peak detection phase. In this case, R-peak should be detected again. The algorithm used to detect R-peak again can be used by using algorithms other than the ones used previously, or by modifying the parameters of the previously used algorithms. You can also apply different algorithms for. The reason for detecting the R-peak again is that the R-peak detection algorithm does not guarantee 100% accuracy, especially since the wearable device may generate a lot of noise during data measurement and transmission.

If the bit split in step S422 is not statistically significant (No in S422), another suitable R-peak detection algorithm is selected (S423), and the selected R-peak detection algorithm is used. The R-peak is detected again (S424). The ECG signal is split into bits again using the detected R-peak using the selected R-peak detection algorithm. The R-peak detection algorithm selection and R-peak detection process are repeated until the split bits are determined to be statistically significant.

In the case of determining the significance of the bit (S422), if the split bit is statistically significant (YES in S422), the pseudo bit processing step S430 is performed. The pseudo bit processing step S430 is a step of replacing a current bit with a pre-generated bit if there is a bit similar to the current bit among the pre-generated bits. The pseudo bit processing step S430 is not essential, but the computational amount of the computer may be reduced through the pseudo bit processing step S430.

Feature extraction step (S440) is a step of extracting a feature of each bit. Feature extraction can use a variety of methods such as fast Fourier transform (FFT), Mel Frequency Cepstral Coefficient (MFCC), Filter Bank, Wavelet widely used in signal processing.

Intermediate neural network analysis step (S450) is a step of obtaining the intermediate output data by inputting the feature of each bit to the intermediate artificial neural network. The intermediate output data differs from the output values of ordinary neural networks in that the intermediate output data is not the value to be finally obtained using artificial intelligence. Detailed description of the intermediate artificial neural network and the intermediate artificial neural network analyzing step S450 will be described in detail later with reference to FIGS. 7A to 12.

The final artificial neural network analysis step (S460) is a step of obtaining final output data by receiving intermediate output data from the intermediate artificial neural network. The final artificial neural network is an artificial neural network located at a terminal of an artificial intelligence system composed of a plurality of artificial neural networks according to an embodiment of the present invention, and the final output data output from the final artificial neural network is finally obtained by using artificial intelligence. The value to get. A detailed description of the final artificial neural network and the final artificial neural network analysis step (S460) will be described in detail later with reference to FIGS. 7A to 12.

The determination result reporting step (S470) is a step of generating and reporting a diagnosis result using the final output data obtained in the final artificial neural network analysis step (S460). Specifically, the determination result reporting step S470 may be to analyze the ECG signal to determine whether the subject has arrhythmia and then report the determination result.

As shown in FIG. 5, the ECG signal may be split in units of bits based on the R-peak. In detail, the left side of the R-peak or the right side of the R-peak may be split in the ECG signal. If the ECG signal is split based on the R-peak, the position for splitting the ECG signal may be any position. In general, the waveform of the heartbeat is divided into P-QRS-T, and FIG. 5 reflects this to split the ECG signal so that the R-peak is at the center of each bit.

Long-term data collection is required to determine arrhythmias using ECG signals. Arrhythmia is characterized by an irregular occurrence, it is not possible to determine the time of occurrence. Therefore, the electrocardiogram measuring apparatus 100 in the form of a wearable device is attached to a subject to measure the electrocardiogram for a long time over several days.

When analyzing the ECG signal by transmitting the ECG signal measured by the ECG apparatus 100 to the server 200, if there are similar bits among the bits that have already occurred before inputting data to the artificial neural network, they may be generated in advance. You can speed up analysis by analyzing only the bits and using the results together.

As shown in Fig. 6A, the similarity is calculated by comparing the current bit K with the pre-generated bits A, B, C, D, E, F, G, H, I and J. The similarity between the bit to be determined currently and the N bits previously generated is calculated, and the most similar bit among the pre-generated bits similar to or above a threshold is selected as the replacement bit.

If the most similar bit among the pre-generated bits similar to the current bit K above the threshold is bit G (see FIG. 6B), bit G is inserted instead of bit K. If there is no similar bit, keep bit K.

Since only N pieces of pre-bit data are retained, the similarity of the next bit is determined as shown in FIG. 6C.

By using the similarity-based bit connection information generated in this way, the analysis result of the bit G is shared without performing a separate analysis on the bit K. In addition, the connection information generated in this step is used to restore the result of bit K after the diagnosis is complete. The similarity calculation algorithm may use any of the well-known similarity algorithms such as L1, L2, and DTW. Human ECG tends to repeat similar signals, which can greatly reduce the amount of computation through similar filtering.

According to the ECG analysis method according to an embodiment of the present invention as described above, after splitting the ECG signal in units of bits using R-peak, the features of each bit are extracted, and the features of each bit are the intermediate artificial neural network. Type in

Figure 7a is a diagram illustrating an intermediate artificial neural network in accordance with an embodiment of the present invention. As shown in FIG. 7A, when features t0, t1, ..., tv are extracted for any one bit, e.g., bit A, the v + 1 features of bit A are in the input layer of the intermediate neural network. It is input to v + 1 nodes.

Data input to the intermediate neural network is output to m nodes in the output layer via the hidden layer. The value output to the m nodes in the output layer of the intermediate artificial neural network may indicate the characteristic of bit A. It should be noted, however, that the values output to the m nodes in the output layer of the intermediate neural network are not values that are finally obtained by comprehensively analyzing the ECG signal using artificial intelligence, but rather that one bit, for example, bit A The feature is expressed in m dimensions. The intermediate neural network may be learning bit A through a map model, and in this case, the value output to m nodes in the output layer of the intermediate neural network may be any prediction value or classification value for bit A. The intermediate neural network may be learning bit A through an unsupervised model, and in this case, a value output to m nodes in the output layer of the intermediate neural network may be a value for clustering bit A. In any case, the values output to the m nodes in the output layer of the intermediate artificial neural network are only the characteristics of the bit A in m-dimensions, and are not the values to be finally obtained using artificial intelligence. It is preferable that two or more features are extracted at this time. That is, it is preferable that m is two or more natural numbers.

Meanwhile, although FIG. 7A illustrates that the intermediate artificial neural network has an RNN structure, the intermediate artificial neural network of FIG. 7A is not limited to the RNN structure, and may have a CNN structure or other artificial neural network structures.

7B is a diagram illustrating a final artificial neural network according to an embodiment of the present invention. As shown in FIG. 7B, values output from m nodes of the output layer of the intermediate artificial neural network are input to m nodes of the input layer of the final artificial neural network.

The final neural network may be an RNN, in which case, in order to reflect the time-varying characteristics of consecutive bits, the m-dimensional characteristic value of the previous bit of the bit to be determined in the input layer of the final artificial neural network, The m-dimensional feature value and the m-dimensional feature value for the next bit of the bit to be determined may be sequentially input.

The ECG signal is divided into nine beats as shown in FIGS. 3A and 3B, and each bit is named A, B, C, D, E, F, G, H, and I. When the bits of and subsequent bits are input together and analyzed, the final neural network includes m-dimensional feature values for bit A, m-dimensional feature values for bit B, and m-dimensional feature values for bit C. Can be input sequentially sequentially. When the whole ECG signal is divided into 9 beats, and each bit is A, B, C, D, E, F, G, H, I, the final neural network receives data in the following order. Can be.

(1) m-dimensional feature values for bit A, m-dimensional feature values for bit B, and m-dimensional feature values for bit C.

(2) m-dimensional feature values for bit B, m-dimensional feature values for bit C, and m-dimensional feature values for bit D.

(3) m-dimensional feature values for bit C, m-dimensional feature values for bit D, and m-dimensional feature values for bit E.

(4) m-dimensional feature values for bit D, m-dimensional feature values for bit E, and m-dimensional feature values for bit F.

(5) m-dimensional feature values for bit E, m-dimensional feature values for bit F, and m-dimensional feature values for bit G.

(6) m-dimensional feature values for bit F, m-dimensional feature values for bit G, and m-dimensional feature values for bit H.

(7) m-dimensional feature values for bit G, m-dimensional feature values for bit H, and m-dimensional feature values for bit I.

Data input to the final neural network is output to the c nodes in the output layer through the hidden layer.

The final neural network may be a trained whole ECG signal through a map model. In this case, the value output to the c nodes in the output layer of the final artificial neural network may be a value for classifying the entire ECG signal into c classifications.

As shown in FIG. 7B, the final ANN is preferably an RNN structure that reflects time-varying characteristics of data. The RNN structure has a feature that the connection between nodes has a recursive structure, which can effectively handle time-varying features. The final artificial neural network receives an m-dimensional feature of consecutive bits sequentially and receives a whole set of consecutive bits. Since the ECG signal is classified into c, the structure of the final artificial neural network is preferably an RNN structure. However, the final artificial neural network is not limited to the RNN structure, and may be any other structure as long as the artificial neural network structure can reflect the time-varying characteristics of the time series data. That is, the final artificial neural network is not limited to the RNN structure.

FIG. 8 illustrates an example of analyzing an electrocardiogram signal using a time series data analysis method using artificial intelligence according to an embodiment of the present invention. As described above, this is only for convenience of explanation and an embodiment of the present invention. The time series data analysis method using artificial intelligence according to an example is not limited to ECG signal analysis.

Referring to FIG. 8, a process of analyzing time series data, for example, an ECG signal, using artificial intelligence according to an exemplary embodiment of the present invention will be described.

When the ECG signal 810 is input, the ECG signal is split into a plurality of

bits

821, 822, 823 ... according to a predetermined reference. For example, after detecting the R-peak from the ECG signal 810, the ECG signal may be split into a plurality of

bits

821, 822, 823... Using R-peak.

When generalizing this to time series data, when time series data is input, it may be described as splitting time series data into a plurality of units according to a predetermined criterion.

Next, an operation of extracting the

features

831, 832, 833... Of each bit is performed for each of the plurality of

bits

821, 822, 823. Feature extraction can use a variety of methods such as fast Fourier transform (FFT), Mel Frequency Cepstral Coefficient (MFCC), Filter Bank, Wavelet widely used in signal processing.

When generalizing this to time series data, when time series data is split into a plurality of units, it may be described as extracting a feature of each unit for each of the plurality of units.

Next, a feature of inputting the

features

831, 832, 833... Of each bit into the intermediate artificial

neural network

841, 842, 843. If the ECG signal was split into n bits, all n features would have been extracted. In this case, the intermediate neural network is also preferably n. A feature of each of the n bits is input to each input layer of the n intermediate neural networks to obtain m-dimensional feature values for each of the n bits in the output layer of each of the n intermediate neural networks. In this case, the value output from the intermediate artificial neural network is called 'intermediate output data'.

Extracting a plurality of features (831, 832, 833 ...) from the plurality of bits (821, 822, 823 ...) is performed by fast fourier transform (FFT), Mel Frequency Cepstral Coefficient (MFCC), Filter Bank, Although it uses a well-known algorithm such as Wavelet, the task of obtaining a plurality of m-dimensional feature values from a plurality of bit features (831, 832, 833 ...) is a result of analysis using artificial intelligence. .

When generalizing this to time series data, when time series data is split into n units and the features of each unit are extracted, the features of each unit may be described as being input to the n intermediate artificial neural networks. A feature of each of the n units is input to each input layer of the n intermediate neural networks to obtain m-dimensional feature values for each of the n units in the output layer of each of the n intermediate neural networks.

Next, the m-dimensional feature values for each of the n bits output from the n intermediate artificial neural networks are input to the final artificial neural network 850 in the order of the time axis of each bit. The final neural network may be an RNN. In this case, in order to reflect the time-varying characteristics of successive bits, the final artificial neural network may have an m-dimensional characteristic value for the previous bit of the bit to be determined in the input layer of the final neural network, The m-dimensional feature values for the m-dimensional feature and the m-dimensional feature values for the next bit of the bit to be determined may be sequentially input.

Data input to the final artificial neural network is output to c nodes in the output layer through the hidden layer, and when the final artificial neural network is a classification model, the value output to the c nodes is a value for classifying the entire ECG signal into c classifications. Can be. If the final artificial neural network is a predictive model, the values output to the c nodes may be values for predicting a result within a specific range from the entire ECG signal. The value output from the final neural network is called 'final output data'.

Since the final artificial neural network receives the m-dimensional feature values for each bit sequentially, it is preferable that the nodes of the input layer of the final artificial neural network be m. Of course, it is also possible to have m or more nodes in the input layer of the final artificial neural network so that other values or parameters may be input in addition to the m-dimensional feature values for each bit.

As is commonly used in the art to which the present invention pertains, the term 'classification model' herein refers to an artificial intelligence model for the purpose of figuring out which group the input data belongs to, 'Prediction model' means an artificial intelligence model in which the result can be any value within the range of the training data.

When generalizing this to time series data, when time series data is split into n units, n-dimensional feature values for each of n units are obtained using n intermediate artificial neural networks. The final neural network receives m-dimensional feature values for each of the n units from the n intermediate artificial neural networks in the order of the time axis of each unit, and performs tasks such as classification and prediction of the entire time series data.

In order to reflect the time-varying characteristics of successive units, the m-dimensional characteristic values of the unit to be determined and the m-dimensional characteristic values of the unit immediately adjacent to the unit to be determined are temporally determined in the input layer of the final artificial neural network. It can be entered continuously into the final neural network. In the case of an ECG signal, the pre-bit of the bit to be discriminated, the bit to be discriminated and the 3 bits after the bit to be discriminated must be continuously input, but in the case of general time series data, the pre-unit of the unit to be discriminated and the unit to be discriminated. It is also possible to enter only two together, or to enter a post-unit of the unit to be determined and only two units to be determined. Of course, it is also possible to input all three of the pre-unit of the unit to be determined, the unit to be determined and the post-unit of the unit to be determined.

Since the final artificial neural network receives the m-dimensional feature values for each unit sequentially, the node of the input layer of the final artificial neural network is preferably m. Of course, it is also possible to have m or more nodes of the input layer of the final artificial neural network so that other values or parameters may be input in addition to the m-dimensional feature values for each unit.

9A and 9B are diagrams illustrating node structures of an intermediate neural network and a final artificial neural network used in a time series data analysis method using artificial intelligence according to an embodiment of the present invention.

9A and 9B illustrate that the feature 831 of the first bit 821 obtained by splitting the ECG signal 810 is input to the first intermediate neural network 841, and the first intermediate neural network ( The intermediate output data of 841 is input to the final artificial neural network 850.

Assuming that a feature extracted from each bit using a fast fourier transform (FFT), mel frequency cepstral coefficient (MFCC), filter bank, wavelet, etc. is a d-dimensional value, each intermediate neural network (841, 842, 843) It is preferable that the number of nodes of the input layer is also d. Of course, it is also possible to have more than d nodes of the input layer of the intermediate artificial neural network so that other values or parameters may be input in addition to the d-dimensional feature extracted from each bit.

When the d-dimensional feature 831 is extracted from the first bit 821, the d-dimensional feature 831 is input to d nodes in the input layer 841a of the first intermediate artificial neural network 841. The input data is analyzed via the hidden layer 841b of the first intermediate neural network 841, and the analysis result is output at m nodes in the output layer 841c of the first intermediate neural network 841.

The intermediate output data of the first intermediate neural network 841 output from the m nodes in the output layer 841c of the first intermediate neural network 841 is m of the input layers 850a of the final artificial neural network 850. It is entered into the node. The final neural network 850 not only receives the intermediate output data of the first intermediate neural network 841, but also receives intermediate output data from other intermediate

neural networks

842, 843, and the hidden layer 850b. After analyzing the time-varying characteristics of consecutive bits, the final output data is output. The final output data is output at c nodes in the output layer 850c of the final neural network 850.

FIG. 10 is a diagram for describing a process of inputting intermediate output data output from a plurality of intermediate artificial neural networks according to an embodiment of the present invention into a final artificial neural network.

When the bit to be determined in the ECG signal is the second bit 822, the m-dimensions of the pre-bit 821 of the bit to be determined, the bit 822 to be determined, and the after bit 832 of the bit to be determined. The intermediate output data of must be input continuously.

That is, the m-dimensional intermediate output data 841c obtained by inputting the d-dimensional feature 831 extracted from the dictionary bit 821 of the bit to be determined to the first intermediate neural network 841 and the bit to be determined ( M-dimensional intermediate output data 842c obtained by inputting the d-dimensional feature 832 extracted from 822 to the second intermediate neural network 842,

The m-dimensional intermediate output data 843c obtained by inputting the d-dimensional feature 833 extracted from the post bit 823 of the bit to be discriminated into the third intermediate neural network 843 is sequentially input to the final artificial neural network. .

The above is summarized as follows.

Deep learning may be used to determine an arrhythmia by analyzing an ECG signal. The neural network model includes a CNN model and an RNN model. However, the CNN model is inadequate for time-series data analysis with time-varying features and is less accurate. Since the CNN method is based on the location information of the feature, there is a problem that the discriminating power is greatly reduced even for small deformation or noise. (Sabour, S., Frosst, N. Hinton, G. E. Dynamic routing between capsules.In Advances in Neural Information Processing Systems, pp. 3859-3869, 2017).

To overcome this problem, the RNN model can be used to analyze time series data with time-varying features. The biggest problem with RNN models is that they are slow to learn and analyze. Although accuracy is improved because more information is available and time-varying features can be reflected, it is more complex because more information needs to be stored than CNN. In addition, since the association with successive units must be used as information, the successive units must be processed sequentially. Therefore, long inputs must be processed, which requires more time for learning and analysis.

According to an embodiment of the present invention, in order to solve the problem of low accuracy and variable length input processing of the CNN and the slow learning and analysis speed problem of the RNN, a plurality of intermediate artificial neural networks and one final artificial neural network are provided. We propose a RNN with variable length boundaries for parallelization model.

In the conventional RNN model, n consecutive bits were considered as one input. However, in the artificial intelligence model according to an embodiment of the present invention, the learning and analysis are divided into two stages, a short term RNN processed by a plurality of intermediate artificial neural networks, and a long term RNN processed by one final neural network. First, each of the n consecutive bits is used as an input of the intermediate artificial neural network. As a result, an m-dimensional vector of one bit is obtained and inputted to the final artificial neural network to analyze an ECG signal to determine a disease such as arrhythmia.

The plurality of intermediate artificial neural networks process a plurality of bits, respectively, and the plurality of intermediate artificial neural networks are the same model. Therefore, when processing multiple bits, parallel processing is possible because there is no dependency between each other. Therefore, compared to the conventional RNN method in which all ECG signals must be inputted, the number or parallel processing of computing devices that perform parallel processing is performed. The operation may be performed quickly in proportion to the number of processors to perform. Of course, a structure in which a plurality of intermediate artificial neural networks process a plurality of input bits may be possible depending on resource constraints. This structure processes each bit in parallel in a plurality of intermediate artificial neural networks and processes shortened intermediate output data in the final artificial neural network, thereby greatly reducing the number of nodes in the hidden layer of each artificial neural network, thereby increasing the available memory. do. This brings with it an improvement in diagnostic speeds in GPU environments that are constrained by video memory. In addition, due to the shorter length of data to be processed in the final neural network, the complexity (complexity) is reduced, which greatly improves the diagnostic speed.

When the length of the input data when analyzing the ECG signal of FIG. 3 using the time series data analysis method using artificial intelligence according to an embodiment of the present invention is as follows.

The ECG signal of FIGS. 3A and 3B is divided into nine beats. Supposing that one bit is sampled at 250 Hz, the length of processing when extracting the feature from nine bits is 250Х9 = 2250.

When nine features extracted from nine bits are input to nine intermediate neural networks, the processing length is 1Х9 = 9.

When the m-dimensional intermediate output data output from the nine intermediate artificial neural networks is input to the final artificial neural network, m-dimensional intermediate output data for the pre-bit, the determination target bit, and the post-bit are continuously input. As described above, data is input to the final artificial neural network in the following order.

That is, since the three-length input is performed seven times, the length of processing when the m-dimensional intermediate output data outputted from nine intermediate artificial neural networks is input to the final artificial neural network becomes 3Х7 = 21.

Adding up all the processing lengths is 2250 + 9 + 21 = 2280. It can be seen that the processing length was reduced by one third as compared to the processing length of the conventional method illustrated in FIGS. 3A and 3B being 5250.

As described above, according to the time series data analysis method using artificial intelligence according to an embodiment of the present invention, the analysis accuracy is higher than that of the conventional CNN-based electrocardiogram analysis method, and the conventional RNN-based electrocardiogram analysis method Compared to this, the length of data to be processed is shortened, thereby increasing the speed of analysis, thereby ensuring real-time electrocardiogram analysis. Through this, the ECG signal can be accurately analyzed in real time, and the subject is notified before the heart disease occurs, so that it is possible to take action in advance with medication or hospital treatment. In addition, agile initial response to fatal heart disease is possible and can be used for urgent lifesaving services.

As shown in FIG. 11, in the method of analyzing time series data using artificial intelligence according to an embodiment of the present invention, for each of a plurality of units in which time series data is split on a time axis, an intermediate artificial feature may be used. Inputting to the neural network (S1110), obtaining intermediate output data of dimension m (m is a natural number of 2 or more) from the intermediate artificial neural network (S1120), and inputting intermediate output data of a plurality of units immediately adjacent in time to the final artificial neural network; The method may include obtaining final output data output from the final artificial neural network (S1130) and generating analysis results of time series data using the final output data (S1140).

Since each step has been described in detail above, a description of repeated content is omitted.

As shown in Figure 12, in the time series data analysis method using artificial intelligence according to an embodiment of the present invention, the intermediate artificial neural network can be implemented in a plurality of levels.

The intermediate neural network of each level outputs the feature value of the input data, and the feature value output from the intermediate neural network of the previous level is input to the intermediate neural network of the next level.

When the intermediate neural network is composed of k levels, the features of each of the plurality of units are input into the level 1 intermediate neural network, and the feature values output from the intermediate neural network of each level are input into the intermediate neural network of the next level. The feature value output from the intermediate neural network of level k-1 is input to the intermediate neural network of level k, and the m-dimensional feature value output from the intermediate neural network of level k is input to the final artificial neural network.

In FIG. 12, the intermediate artificial neural network having a plurality of levels is designed in such a manner that a plurality of feature values output from the plurality of intermediate artificial neural networks of the previous level are input to one intermediate neural network of the next level. Processes data in each unit unit (first unit) in a second, and processes data in a second unit grouping a plurality of units temporally adjacent in a level 2 intermediate neural network, and processes a plurality of temporally adjacent in a level 3 intermediate neural network. An example of processing data in a manner of processing data of a third unit grouping data of a second unit is illustrated. In this case, the m-dimensional feature value output from the intermediate artificial neural network at level k will be the m-dimensional feature value for the data of the kth unit. In this case, the number of intermediate neural networks of each level may vary according to the grouping criteria.

Of course, the intermediate neural network having a plurality of levels may be designed in such a manner that one feature value output from one intermediate neural network of the previous level is input 1: 1 to one intermediate neural network of the next level. In this case, the intermediate neural networks of each level can be implemented with the same number. For example, if the number of level 1 intermediate artificial neural network is n, the number of level 2 intermediate artificial neural network can be implemented as n number of level 2 intermediate artificial neural network, n, number n of level k intermediate artificial neural network.

In the time series data analysis method using artificial intelligence according to an embodiment of the present invention, when the intermediate artificial neural network has a plurality of levels, the time series data analysis method includes: for each of a plurality of units in which time series data is split on the time axis, Inputting a feature of each unit into the level 1 intermediate neural network (S1310), and inputting intermediate output data output from the level i (i is one or more natural numbers) artificial neural network into the level i + 1 intermediate neural network; (S1320), obtaining intermediate output data from the level k intermediate neural network (S1330) and sequentially inputting level k intermediate output data into the final artificial neural network to obtain final output data from the final artificial neural network (S1340). Can be.

If the intermediate neural network having a plurality of levels is designed in such a manner that a plurality of feature values output from the plurality of intermediate artificial neural networks of the previous level are input to one intermediate neural network of the next level, The step of sequentially inputting into the final artificial neural network may include sequentially inputting level k intermediate output data for the data of the k-th unit immediately adjacent in time to the final artificial neural network.

On the other hand, when the intermediate artificial neural network having a plurality of levels is designed in such a manner that one feature value output from one intermediate neural network of the previous level is input 1: 1 to one intermediate neural network of the next level, The step of sequentially inputting the level k intermediate output data of immediately adjacent units into the final artificial neural network is the same as the case where the level of the intermediate artificial neural network described with reference to FIGS. 9 and 10 is omitted. .

In the above, the learning method of time series data using artificial intelligence according to an embodiment of the present invention has been described.

Hereinafter, a method of learning time series data using artificial intelligence according to an embodiment of the present invention will be described with reference to FIGS. 14 to 17. In detail, a process of learning an ECG signal using a plurality of artificial neural networks according to an embodiment of the present invention will be described to explain a method of learning time series data using artificial intelligence according to an embodiment of the present invention.

As is well known, the learning process of artificial neural network using learning data is basically similar to the process of analyzing input data using artificial neural network. However, in the process of learning artificial neural network, there is a difference in that the step of calculating the error by comparing the output value and the actual value, and changing the weight of the artificial neural network by back propagating the error is different.

Hereinafter, a method of learning time series data according to an embodiment of the present invention will be described based on differences from the method for analyzing time series data according to an embodiment of the present invention.

As shown in FIG. 14, an ECG learning method using artificial intelligence according to an embodiment of the present invention includes an ECG signal collection step S1410, an ECG signal split step S1420, a bit combining step S1430, and a feature ) May include an extraction step (S1440) and an artificial neural network learning step (S1450).

The electrocardiogram signal collecting step S1410 is a step in which the server 200 collects the electrocardiogram signal received from the electrocardiogram measuring apparatus 100 or the electrocardiogram signal directly input by the user. In general, the whole cardiogram signal measured for a long time is not labeled with the discriminant arrhythmias, but is labeled with the discriminant arrhythmias in bits. Well-known data includes the MIT-BIH Arrhythmia Database.

The ECG signal splitting step S1421 is a step of splitting the ECG signal in bit units.

The bit combining step S1430 is a step of combining a pre-bit and b post-bits with labeled bits.

As shown in FIG. 15, an ECG learning method using artificial intelligence according to an embodiment of the present invention combines a pre-bit and b post-bits with labeled bits and uses them as learning data.

An analysis method of time series data using artificial intelligence according to an embodiment of the present invention does not analyze a single bit, but generates a time-varying feature generated by concatenating a plurality of adjacent bits in time, such as a pre or post bit. Since the gap, pre-bit, or post-bit difference is analyzed together, the pre- and post-bits must be trained by combining the pre- and post-bits with the labeled bits, instead of learning the labeled bits alone. .

The number of pre-bits a and the number of post-bits b to be coupled to the labeled bits can be arbitrarily determined. At this time, the number of pre-bits a and the number of post-bits b should be used as inputs for analysis.

The feature extraction step S1440 is a step of extracting a feature of each bit. Feature extraction can use a variety of methods such as fast Fourier transform (FFT), Mel Frequency Cepstral Coefficient (MFCC), Filter Bank, Wavelet widely used in signal processing. It is preferable to use the same feature extraction algorithm used for learning and the feature extraction algorithm used for analysis.

The artificial neural network learning step (S1450) is a step of learning the artificial neural network by using well-known techniques such as back propagation and gradient reduction. Hereinafter, the neural network learning step S1450 will be described in detail with reference to FIG. 16.

A process of learning time series data, for example, an ECG signal, using artificial intelligence according to an embodiment of the present invention will be described with reference to FIG. 16.

When using a database labeled in units of bits such as the MIT-BIH Arrhythmia Database, the training data 1610 combines one or more pre-bits and one or more post-bits into labeled bits through a bit combining step (S1430). It should be an electrocardiogram signal. If the result of analyzing the entire ECG signal instead of the bit unit is used as a database that is labeled with each ECG signal, the labeled ECG signal may be used as the training data 1610 without the bit combining step (S1430). have.

Generalizing this to time series data, when using a database labeled in units, the training data should be time series data combining one or more units in a labeled unit. If the results of analyzing the entire time series data instead of unit units are used as a database in which each time series data is labeled, the labeled time series data may be used as training data.

When the training data 1610 is input, the training data is split into a plurality of

bits

1621, 1622, 1623 ... according to a predetermined criterion. It is preferable to use the same split method for the ECG signal used for learning and split method for the ECG signal used for analysis.

When generalizing this to time series data, when learning data is input, it can be described as splitting learning data into a plurality of units according to a predetermined criterion. The split method of time series data used for learning and the split method of time series data used for analysis are preferably used.

Next, for each of the plurality of

bits

1621, 1622, 1623..., A feature of extracting the

features

1631, 1632, 1633... Of each bit is performed. The feature extraction algorithm of the ECG signal used in learning and the feature extraction algorithm of the ECG signal used in analysis are preferably the same.

When generalizing this to time series data, when time series data is split into a plurality of units, it may be described as extracting a feature of each unit for each of the plurality of units. It is preferable to use the same feature extraction algorithm of the time series data used for learning and the feature extraction algorithm of the time series data used for analysis.

Next, a function of inputting the

features

1631, 1632, 1633... Of each bit into the intermediate artificial

neural network

1641, 1642, 1643. If the training data 1610 is a combination of n bits, then all n features would have been extracted, in which case the intermediate artificial neural network is preferably n.

When generalizing this to time series data, when time series data is split into n units and the features of each unit are extracted, the features of each unit may be described as being input to the n intermediate artificial neural networks.

Next, the m-dimensional feature values for each of the n bits output from the n intermediate artificial neural networks are input to the final artificial neural network 1650 in order on the time axis of each bit to obtain final output data for the training data. .

When generalizing this to time series data, when time series data is split into n units, m intermediate feature values for each of n units are obtained using n intermediate artificial neural networks, and the final artificial neural network is n. M-dimensional feature values for each of the units are provided from the n intermediate neural networks in the order of the time axis of each unit to obtain classification values, prediction values, and the like for the training data.

When the final output data for the training data is obtained from the final neural network 1650, the final output data for the training data is compared with a label attached to the training data. At this time, the final output data for the training data is an output value, and the label attached to the training data is an actual value. The error between the output value and the actual value is calculated using a loss function or the like and then back propagated.

The backpropagation adjusts the weight of the final neural network 1650 and the weights of the intermediate artificial

neural networks

1641, 1642, 1643... Weight adjustment method of backpropagation and artificial neural network is described in detail in the literature such as https://en.wikipedia.org/wiki/Backpropagation, so the detailed description is omitted.

At this time, care should be taken to set the learning rate of the final artificial neural network (1650) and intermediate artificial neural network (1641, 1642, 1643 ...). The 'learning rate' refers to the size of the step to find the lowest value in the gradient reduction algorithm for weight adjustment of the artificial neural network.

Intermediate neural networks cannot be trained in parallel. This is because there is only one correct answer for a plurality of input bits. Because of this, backpropagation must be performed for a plurality of input bits for one correct answer. If the number of bits is n and the number of intermediate artificial neural networks is n, the n intermediate artificial neural networks must share weights with each other, and the learning rate of the intermediate artificial neural network during backpropagation is 1 / th of the learning rate of the final artificial neural network. It should be set to n times.

On the other hand, even if the intermediate artificial neural network is implemented in a plurality of levels the learning method of the artificial neural network is the same. After analyzing the training data in the same manner as the method of analyzing the artificial neural network when the intermediate artificial neural network is implemented at a plurality of levels (see FIGS. 12 and 13), when the final output data of the training data is obtained from the final artificial neural network, the training is performed. The final output data (output value) for the data is compared with the label (actual value) attached to the training data. The error between the output value and the actual value is calculated using a loss function or the like and then back propagated to adjust the weight of the final neural network and the weights of the intermediate neural networks of the plurality of levels.

As shown in FIG. 17, in the time series data learning method using artificial intelligence according to an embodiment of the present invention, for each of a plurality of units in which time series data is split on a time axis, an artificial feature of each unit is selected. Inputting to the neural network (S1710), obtaining intermediate output data of the dimension m (m is a natural number of 2 or more) from the intermediate artificial neural network (S1720), and inputting intermediate output data of a plurality of units immediately adjacent in time to the final artificial neural network; Obtaining final output data output from the final artificial neural network (S1730), comparing the final output data with a label attached to time series data, calculating an error (S1740), and back propagating the error to the intermediate artificial neural network and the final artificial Adjusting the weight of the neural network (S1740) may be included.

As illustrated in FIG. 18, an exemplary computing device capable of implementing a time series data analysis method or a time series data learning method using artificial intelligence according to embodiments of the present disclosure may include one or more processors 1810, storage 1820, It may include a memory 1830, a network interface 1840, and a bus that load a computer program executed by the processor 1810. 18, only components relevant to embodiments of the present invention are shown. Accordingly, it will be appreciated by those skilled in the art that other general purpose components may be further included in addition to the components illustrated in FIG. 18.

The processor 1810 controls the overall operation of each component of the computing device that can implement the time series data analysis method or the time series data learning method using artificial intelligence according to embodiments of the present invention. The processor 1810 includes a central processing unit (CPU), a micro processor unit (MPU), a micro controller unit (MCU), a graphics processing unit (GPU), or any type of processor well known in the art. Can be. In addition, the processor 1810 may perform an operation on at least one application or program for executing a method according to embodiments of the present invention. A computing device capable of implementing a method for analyzing time series data or a method for learning time series data using artificial intelligence according to embodiments of the present invention may include one or more processors.

The storage 1820 can non-temporarily store one or more computer programs. The storage 1820 may be a non-volatile memory such as a read only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, a hard disk, a removable disk, or a technical field to which the present invention pertains. It may comprise any known type of computer readable recording medium. The storage 1820 stores a computer program for performing a time series data analysis method or a time series data learning method using artificial intelligence according to embodiments of the present invention.

The memory 1830 stores various data, commands, and / or information. The memory 1830 may load one or more computer programs from the storage 1820 to execute the time series data analysis method or the time series data learning method using artificial intelligence according to embodiments of the present invention.

The bus provides a communication function between components of a computing device capable of implementing a time series data analysis method or a time series data learning method using artificial intelligence according to embodiments of the present invention. The bus may be implemented as various types of buses such as an address bus, a data bus, and a control bus.

The network interface 1840 supports wired / wireless communication of a computing device capable of implementing a time series data analysis method or a time series data learning method using artificial intelligence according to embodiments of the present invention. In addition, the network interface 1840 may support various communication methods other than Internet communication. To this end, the network interface 1840 may be configured to include a communication module well known in the art.

A computer program for performing the time series data analysis method or the time series data learning method using artificial intelligence according to the embodiments of the present invention is loaded in the memory 1830, and causes the processor 1810 according to the embodiments of the present invention. Instructions for performing a time series data analysis method or a time series data learning method may be included.

Hereinafter, an apparatus for analyzing time series data using artificial intelligence according to embodiments of the present invention will be described.

An apparatus for analyzing time series data according to an embodiment of the present invention includes a network interface 1840 for receiving time series data, one or more processors 1810, a memory 1830 for loading a computer program executed by the processor 1841, and Storage 1820, which stores a computer program, the computer program for each of a plurality of units in which time series data has been split in the time axis, inputting features of each of the units into the intermediate neural network and Instruction for obtaining intermediate output data of dimension m (m is a natural number of 2 or more), instruction for inputting intermediate output data of a plurality of units immediately adjacent in time to the final neural network, and obtaining final output data output from the final artificial neural network. Using output data to generate analysis results of time series data Scotland may include Sean truck.

Since each instruction of the computer program stored in the storage 1820 of the apparatus for analyzing time series data according to an embodiment of the present invention performs the method for analyzing time series data using artificial intelligence described above, a description of repeated content will be omitted.

Hereinafter, a time series data learning apparatus using artificial intelligence according to embodiments of the present invention will be described.

An apparatus for learning time series data according to an embodiment of the present invention includes a network interface 1840 for receiving time series data, one or more processors 1810, a memory 1830 for loading a computer program executed by the processor 1841, and Storage 1820, which stores a computer program, the computer program for each of a plurality of units in which time series data has been split in the time axis, inputting features of each of the units into the intermediate neural network and instructions for obtaining intermediate output data in the dimension of m (m is a natural number of 2 or more), instructions for inputting intermediate output data of a plurality of units immediately adjacent in time to the final neural network, and obtaining final output data output from the final artificial neural network; Compare output data with labels attached to time series data Scotland may include Sean truck.

Since each instruction of the computer program stored in the storage 1820 of the time series data learning apparatus according to an embodiment of the present invention performs the time series data learning method using artificial intelligence described above, a description of repeated content is omitted.

Meanwhile, the inventive concept described with reference to the drawings may be embodied in computer readable codes on a computer readable medium. The computer-readable recording medium may be a removable recording medium (CD, DVD, Blu-ray Disc, USB storage device, removable hard disk) or a fixed recording medium (ROM, RAM, computer-equipped hard disk). A computer program recorded on a computer readable recording medium may be transmitted to another computing device and installed in another computing device via a network such as the Internet, thereby being used in another computing device.

Although the operations are shown in a specific order in the drawings, it should not be understood that the operations must be performed in the specific order or sequential order shown or that all the illustrated operations must be executed to achieve the desired results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of the various configurations in the embodiments described above should not be understood as such separation being necessary, and the described program components and systems may generally be integrated together into a single software product or packaged into multiple software products. Should be understood.

Although embodiments of the present invention have been described above with reference to the accompanying drawings, those skilled in the art to which the present invention pertains may implement the present invention in other specific forms without changing the technical spirit or essential features thereof. I can understand that. Therefore, it is to be understood that the embodiments described above are exemplary in all respects and not restrictive.

Claims

In the analysis method of time series data performed by a computing device,

Inputting a feature of each of the units into an intermediate neural network for each of a plurality of units in which the time series data is split on a time axis;

Obtaining intermediate output data of the dimension m (m is a natural number of two or more) from the intermediate artificial neural network;

Inputting the intermediate output data of a plurality of units immediately adjacent in time to a final artificial neural network to obtain final output data output from the final artificial neural network; And

Generating an analysis result of the time series data using the final output data;

How to analyze time series data.
The method of claim 1,

The intermediate neural network and the final artificial neural network is RNN (Recurrent Neural Network),

How to analyze time series data.
The method of claim 1,

M nodes of the input layer of the final artificial neural network,

How to analyze time series data.
The method of claim 1,

Obtaining the intermediate output data,

A first computing device, inputting features of a first unit included in the plurality of units into the intermediate artificial neural network implemented in the first computing device and obtaining m-dimensional intermediate output data from the intermediate artificial neural network; And

A second computing device, inputting features of a second unit included in the plurality of units into an intermediate neural network implemented in the second computing device and obtaining m-dimensional intermediate output data from the intermediate neural network; ,

How to analyze time series data.
The method of claim 4, wherein

Obtaining the final output data,

A third computing device receives intermediate output data of the first unit from the first computing device, intermediate output data of the second unit from the second computing device, and intermediate output data of the first unit And sequentially inputting intermediate output data of the second unit into the final artificial neural network, wherein the first unit and the second unit are immediately adjacent units in time.

How to analyze time series data.
The method of claim 1,

The intermediate neural network is composed of k (k is a natural number of 2 or more),

Input to the intermediate artificial neural network,

Inputting a feature of each of the plurality of units into a level 1 intermediate neural network;

Inputting level i output data obtained from the level i (i is one or more natural number) intermediate neural network into the level i + 1 intermediate neural network; And

Obtaining level i + 1 output data from the level i + 1 intermediate neural network;

The intermediate output data is data output from the level k intermediate neural network,

How to analyze time series data.
The method of claim 1,

The time series data is electrocardiogram data,

How to analyze time series data.
The method of claim 7, wherein

The analysis result is an electrocardiogram diagnosis result,

How to analyze time series data.
The method of claim 7, wherein

The n units are beat units that are split using R-peak,

How to analyze time series data.
The method of claim 9,

The split n bit units,

Splitting the time series data and comparing similarities between the split bits to replace bits having a similarity with a predetermined specific bit with a predetermined value or more with the feature unit;

How to analyze time series data.
In the learning method of time series data performed by a computing device,

Inputting a feature of each of the units into an intermediate neural network for each of a plurality of units in which the time series data is split on a time axis;

Obtaining intermediate output data of the dimension m (m is a natural number of two or more) from the intermediate artificial neural network;

Inputting the intermediate output data of a plurality of units immediately adjacent in time to a final artificial neural network to obtain final output data output from the final artificial neural network; And

Comparing the final output data with a label attached to the time series data;

How to learn time series data.
The method of claim 11,

The intermediate neural network and the final artificial neural network is RNN (Recurrent Neural Network),

How to learn time series data.
The method of claim 11,

M nodes of the input layer of the final artificial neural network,

How to learn time series data.
The method of claim 11,

The comparing step includes comparing the final output data with a label attached to the time series data to calculate an error,

After the comparing, further comprising the step of back propagating the error to adjust the weight of the intermediate artificial neural network and the weight of the final artificial neural network,

How to learn time series data.
The method of claim 14,

Adjusting the weights,

And adjusting the weight of the intermediate artificial neural network by making the learning rate of the intermediate artificial neural network 1 / n times the learning rate of the final artificial neural network.

How to learn time series data.
The method of claim 11,

The intermediate neural network is composed of k (k is a natural number of 2 or more),

Input to the intermediate artificial neural network,

Inputting a feature of each of the plurality of units into a level 1 intermediate neural network;

Inputting level i output data obtained from the level i (i is one or more natural number) intermediate neural network into the level i + 1 intermediate neural network; And

Obtaining level i + 1 output data from the level i + 1 intermediate neural network;

The intermediate output data is data output from the level k intermediate neural network,

How to learn time series data.
The method of claim 11,

The time series data is electrocardiogram data,

How to learn time series data.
The method of claim 17,

The n units are beat units that are split using R-peak,

How to learn time series data.
A network interface for receiving time series data;

One or more processors;

A memory for loading a computer program executed by the processor; And

Storage for storing the computer program,

The computer program,

For each of a plurality of units in which the time series data is split on the time axis, input a feature of each of the units into an intermediate neural network and obtain intermediate output data in the dimension m (m is a natural number of two or more) from the intermediate neural network. Obtaining instructions;

Instructions for inputting the intermediate output data of a plurality of units immediately adjacent in time to a final artificial neural network, and obtaining final output data output from the final artificial neural network; And

Instructions for generating an analysis result of the time series data using the final output data;

Time series data analysis device.
A network interface for receiving time series data;

One or more processors;

A memory for loading a computer program executed by the processor; And

Storage for storing the computer program,

The computer program,

For each of a plurality of units in which the time series data is split on the time axis, input a feature of each of the units into an intermediate neural network and obtain intermediate output data in the dimension m (m is a natural number of two or more) from the intermediate neural network. Obtaining instructions;

Instructions for inputting the intermediate output data of a plurality of units immediately adjacent in time to a final artificial neural network, and obtaining final output data output from the final artificial neural network; And

Instructions for comparing the final output data with a label attached to the time series data;

Time series data learning device.