KR102589471B1 - Apparatus and method for augmentating of data - Google Patents

Abstract

A data enhancement device and method are provided, the data enhancement device comprising an input unit for acquiring at least one raw electrocardiogram data including at least one heart rate signal, classifying the at least one heart rate signal into a corresponding heart beat type, and Obtaining augmented ECG data by increasing the heart rate signal for a rare type with few heart rate signals among the rare types, selecting a reference heart rate signal from the heart rate signals of the rare type, and selecting at least one peripheral heart rate signal around the reference heart rate signal; , It may include a processor that generates a random heart rate signal between the reference heart rate signal and the peripheral heart rate signal to enhance the sparse type heart rate signal.

Description

Data augmentation apparatus and method {APPARATUS AND METHOD FOR AUGMENTATING OF DATA}

It relates to a data augmentation device and method.

Muscles contract or relax according to electrical signals. The same goes for the heart muscle. An electrocardiogram (ECG) is time series data obtained by periodically measuring changes in electrical signals (myocardial activity current) according to the activity of the heart muscle, and is commonly used to diagnose heart diseases such as arrhythmia. An electrocardiogram consists of P waves, QRS complex, and T waves. P waves are measured when an electrical signal occurs in the sinoatrial node of the right atrium. The generated electrical signal passes through the atrioventricular node and Bundle of His, is divided into the right and left horns, and is transmitted to the Purkinje fibers. In this process, the ventricle undergoes polarization and depolarization and contracts, and at this time, the ECG QRS complex is measured. T waves are measured during ventricular repolarization. Since these electrocardiograms generally have a regular and stereotypical rhythm, if a disease occurs in the heart, the shape (type) changes, such as the P wave not being measured or the R peak of the QRS complex facing the opposite direction. Therefore, when abnormal heart rhythm occurs, quickly classifying it into a specific type is necessary for immediate replacement for heart disease. However, because it is not easy for medical personnel or patients to continuously monitor the electrocardiogram, a heart rate classification system that can automatically classify heartbeats and detect and warn of the occurrence of abnormal types of heartbeats at an early stage has been researched and developed. However, the databases commonly used to implement such classification systems not only have biased data toward specific types, but also lack an absolute amount, making it difficult to build a system at the level required by users and developers.
The present invention is a related research project of local governments (research project name: Ansan City small business development support project / Ansan City small business development support project / Ansan city small business development support project, research management agency: Korea Institute of Industrial Technology, host organization: Korea University This is the result of Ansan Hospital, research period: 2021.01.01 ~ 2021.12.31).

Chinese Patent Publication No. 112686091 (published on April 20, 2021) Public Patent Publication No. 10-2020-0109754 (published on September 23, 2020) Japanese Patent Publication Patent No. 6831944 (published on February 17, 2021) Public Patent Publication No. 10-2019-0141326 (published on December 24, 2019)

The goal is to provide a data augmentation device and method that can improve the imbalance of heart rate data by amplifying the relatively insufficient amount of rare heart rate data so that rare heart rate types can be accurately classified.

In order to solve the above-mentioned problems, a data enhancement device and method are provided.

The data enhancement device includes an input unit that acquires at least one raw electrocardiogram data including at least one heart rate signal, and classifies the at least one heart rate signal into corresponding heart beat types, and selects a rare type with few heart rate signals among the heart beat types. Augmented electrocardiogram data is obtained by increasing the heart rate signal, selecting a reference heart rate signal from among the sparse heart rate signals, selecting at least one peripheral heart rate signal around the reference heart rate signal, and selecting the reference heart rate signal and the peripheral heart rate. It may include a processor that enhances the sparse heart rate signal by generating a random heart rate signal between the signals.

The processor may remove noise from the raw ECG data to obtain noise-removed ECG data.

The processor performs normalization on the raw ECG data or the noise-removed ECG data to obtain normalized ECG data, and uses the raw ECG data, the noise-removed ECG data or the normalized ECG data to generate the augmented ECG data. Data can also be obtained.

The processor may detect an R peak from the raw ECG data, the noise-removed ECG data, or the normalized ECG data, and extract a signal from the R peak within a certain reference range to obtain the heart rate signal.

The processor may divide the augmented ECG data to generate training data, verification data, and test data. In this case, the test data belongs to both the original ECG data and the augmented ECG data. It may contain only the heart rate signal.

The processor trains a learning model using the training data, and the learning model may include a one-dimensional convolutional neural network.

The data augmentation method includes obtaining at least one raw electrocardiogram data including at least one heart rate signal, classifying the at least one heart rate signal into a corresponding heart beat type, and sparse types with fewer heart rate signals among the heart rate types. It may include the step of acquiring augmented ECG data by increasing the heart rate signal, wherein the step of acquiring augmented ECG data by increasing the heart rate signal for a rare type with a small heart rate signal among the heart rate types may include obtaining augmented ECG data. selecting a reference heart rate signal from among types of heart rate signals, selecting at least one peripheral heart rate signal surrounding the reference heart rate signal, and generating a random heart rate signal between the reference heart rate signal and the peripheral heart rate signal to determine the sparse heart rate signal. It may include the step of enhancing a tangible heart rate signal.

The data enhancement method may further include removing noise from the original ECG data to obtain noise-removed ECG data.

The data augmentation method may further include performing normalization on the raw ECG data or the noise-removed ECG data to obtain normalized ECG data. In this case, the step of acquiring the augmented ECG data includes: It may include acquiring the augmented ECG data using raw ECG data, the noise-removed ECG data, or the normalized ECG data.

The data enhancement method includes detecting an R peak from the raw ECG data, the noise-removed ECG data, or the normalized ECG data, and extracting a signal from the R peak within a certain reference range to obtain the heart rate signal. It is possible to include more.

The data augmentation method may include dividing the augmented ECG data to generate training data, verification data, and test data, where the test data includes the original ECG data and the augmented ECG data. It may be designed to include only heart rate signals belonging to both.

The data augmentation method may include training a learning model using the training data, where the learning model may include a one-dimensional convolutional neural network.

According to the above-described data augmentation device and method, it is possible to obtain the advantage of improving the imbalance of given heart rate data by amplifying the amount of sparse heart rate data that is relatively insufficient among the total heart rate data.

According to the above-described data augmentation device and method, by improving the imbalance of heart rate data and making the distribution of the heart rate data set uniform, the performance of the learning model is prevented from deteriorating due to the imbalance and the classification ability of the learning model for heart rate type is further improved. You can also achieve the desired effect.

According to the above-described data augmentation device and method, it is possible to achieve the effect of strengthening the generality and versatility of the learning model for classifying heart rate data as heart rate data increases.

According to the above-described data augmentation device and method, it is possible to automatically classify heart rate data measured from a subject into a specific type with high accuracy, thereby reducing classification errors due to human subjectivity, thereby reducing the risk of heart disease, etc. There is also the advantage of improving the accuracy of diagnosis.

1 is a block diagram of an embodiment of a data enhancement device.
Figure 2 is a diagram illustrating an example of an electrocardiogram signal.
Figure 3 is a diagram showing an example of the types of heart beats.
Figure 4 is a chart to explain the difference in the number of data between original ECG data and augmented ECG data.
Figure 5 is a diagram for explaining the process of generating augmented ECG data.
Figure 6 is a block diagram of an embodiment of the training unit.
Figure 7 is a diagram to explain the performance of a trained learning model.
Figure 8 is a flow chart of one embodiment of a data enhancement method.

Throughout the specification below, the same reference signs refer to the same components unless there are special circumstances. Terms with the addition of 'unit' used below may be implemented as software and/or hardware, and depending on the embodiment, one 'unit' may be implemented as one physical or logical part, or a plurality of 'units' may be implemented. It is also possible to be implemented with one physical or logical part, or one 'part' to be implemented with a plurality of physical or logical parts. When a part is said to be connected to another part throughout the specification, this may mean that the part and the other part are physically connected to each other and/or electrically connected. In addition, when a part includes another part, this does not mean excluding another part other than the other part unless specifically stated to the contrary, and means that another part may be included depending on the designer's choice. do. Expressions such as the first to Nth (N is a natural number greater than or equal to 1) are intended to distinguish at least one part(s) from other part(s), and do not necessarily mean that they are sequential unless otherwise specified. Additionally, singular expressions may include plural expressions, unless the context clearly makes an exception.

Hereinafter, an embodiment of the data enhancement device will be described with reference to FIGS. 1 to 7.

1 is a block diagram of an embodiment of a data enhancement device.

As shown in FIG. 1, the data enhancement device 200 may include an input unit 101, a storage unit 103, an output unit 105, and a processor 200. At least two of the input unit 101, storage unit 103, output unit 105, and processor 200 unilaterally or bilaterally transmit data, settings, instructions, or programs (which may be referred to as applications, apps, or software, etc.). They may be electrically connected for transmission or may be connected using a wireless communication network. Depending on the embodiment, at least one of the input unit 101, the storage unit 103, and the output unit 105 may be omitted.

The input unit 101 receives data to be augmented, for example, raw ECG data 110, directly from the user, receives it from another external device (for example, a portable memory device, etc.), and/or processes other external information. It may also be received from the device via a wired or wireless communication network. Here, the raw ECG data 110 is data on ECG(s) previously constructed for learning a learning model, etc., and may be constructed in the form of an ECG database. The electrocardiogram database may include, for example, an arrhythmia database such as the MIT-BIH database. In addition, the input unit 101 may receive at least one variable value for a learning model to be trained by the training unit 260 of the processor 200, and may also receive information on the type that will be the standard for augmentation. , at least one program for operating the processor 200 or a user instruction regarding starting operation of the processor 200 may be input. The input unit 101 includes, for example, a keyboard, mouse, tablet, touch screen, touch pad, track ball, track pad, scanner device, image capture module, ultrasonic scanner, motion detection sensor, vibration sensor, light receiving sensor, and pressure sensor. , may include proximity sensors and/or microphones, and can be connected to data input/output terminals capable of receiving data, etc. from other external devices (e.g., portable memory devices, etc.), or to other external devices through a wired or wireless communication network. It may also include a communication module (for example, a LAN card, short-range communication module, or mobile communication module, etc.).

The storage unit 103 acquires data, setting values, or programs (for example, learning models) received by the input unit 101 or generated according to the processing process or results of the processor 200, and stores the acquired data. It can be stored temporarily or non-temporarily. For example, the storage unit 103 may store data to be augmented, for example, original ECG data 110, or augmented data, for example, augmented ECG data 120. In addition, the storage unit 103 stores a previously written program for data augmentation of the processor 200 and transfers all or part of the program according to a call from the processor 200, allowing the processor 200 to perform a data augmentation operation. You can also have it done. A program previously written for data enhancement may be directly written or modified by a designer such as a programmer and then stored in the storage unit 103, or may be transmitted from another physical recording medium (external memory device, compact disk (CD), etc.) It may be received and stored, and/or may be acquired or updated through an electronic software distribution network accessible through a wired or wireless communication network. The storage unit 103 includes, for example, main memory (ROM and/or RAM, etc.) and auxiliary memory (flash memory device (solid state drive (SSD) based on this) etc.), SD (Secure Digital) card, hard disk drive, etc.).

The output unit 105 visually or audibly provides the user with the data 20, 20 stored in the storage unit 103 or the learning algorithm obtained through training of the processor 200, or through another external device. It can be transmitted directly (for example, a portable memory device or other information processing device) or through a wired or wireless network. The output unit 105 may include, for example, a display, a printer device, a speaker device, an image output terminal, a data input/output terminal, and/or a communication module.

The processor 200 performs augmentation processing on raw data (120, for example, raw electrocardiogram data) and, if necessary, creates at least one learning model using the augmented data (120, for example, augmented electrocardiogram data). It is prepared for training. The processor 200 can be implemented using at least one processing device capable of performing calculations or control processing on data, and at least one electronic device is, for example, a central processing unit (CPU). ), graphics processing unit (GPU: Graphic Processing Unit), microcontroller unit (MCU: Micro Controller Unit), application processor (AP: Application Processor), electronic control unit (ECU: Electronic Controlling Unit) and/or microcontroller (Micom: Micro Processor), etc.

According to one embodiment, the processor 200 may include a noise removal unit 210, a normalization unit 220, a beat classification unit 230, a data enhancement unit 240, and a data division unit 250. , Depending on the embodiment, a training unit 260 may be further included. The noise removal unit 210, normalization unit 220, beat classification unit 230, and data division unit 250 can be omitted as needed. Depending on the embodiment, at least two of the noise removal unit 210, normalization unit 220, beat classification unit 230, data enhancement unit 240, data division unit 250, and training unit 260 are physically Or, it may be logically separated. When physically separated, at least two of them may be implemented by processing devices that are physically separate from each other. When logically separated, at least two of them may be implemented by a single physical processing unit.

Figure 2 is a diagram illustrating an example of an electrocardiogram signal.

At least one piece of raw ECG data 110 may include at least one heart rate signal 111 corresponding to a specific type. Here, the heartbeat signal is at least one signal corresponding to the heartbeat. For example, the at least one raw electrocardiogram data 110 may include a heart rate signal 111 corresponding to a normal heart beat that is the same or close to that shown in FIG. 2, and/or corresponding to an abnormal heart beat different therefrom. It may also include heart rate signals. Here, each heart rate signal corresponding to normal or abnormal may be sequentially connected to form raw ECG data 110. In this case, in each of the heart rate signals 111 corresponding to at least one normal heart beat or the heart rate signals corresponding to an abnormal heart rate, there may be additional noise introduced from the outside or generated during the database construction process when measuring the raw ECG data 110. It may be possible. The noise removal unit 210 acquires the raw ECG data 110 from the input unit 101 or the storage unit 103, and removes noise that may exist in the raw ECG data 110, that is, each heart rate signal 111, etc. , can be removed from the original ECG data 110. According to one embodiment, the noise removal unit 210 may be implemented using at least one filter(s) such as a Butterworth filter (for example, a fourth-order Butterworth filter, etc.). In this case, the at least one filter(s) may have a passband of, for example, 0.5 Hz to 15 Hz. ECG data (hereinafter referred to as noise-removed ECG data) obtained by applying filter(s) to the original ECG data 110 may be transmitted to the normalization unit 220.

The normalization unit 220 may perform normalization on the received data (original ECG data 110 or corresponding noise-removed ECG data). Normalization, for example, subtracts the average (or commonly known average ECG signal) of all raw ECG data 110 for each raw ECG data 110, and applies the subtraction result to each raw ECG data 110. ) can also be done by dividing by the standard deviation of Alternatively, normalization may be performed using max-minimum normalization. Accordingly, normalized ECG data corresponding to each of one or more raw ECG data 110 may be obtained.

Figure 3 is a diagram showing an example of the types of heart beats.

The beat classification unit 230 generates a signal 111 (i.e., heart rate signal) corresponding to the heartbeat from the raw ECG data 110, noise-removed ECG data, or normalized ECG data (hereinafter referred to as the raw ECG data 110, etc.). The above is extracted, and each of the extracted one or more heart rate signals 111 may be determined to be at least one type among a plurality of types.

Depending on the embodiment, the beat classification unit 230 may determine the type to be used for classification before classifying the type of each heart rate signal. For example, the beat classification unit 230 selects and determines at least one heartbeat type for classifying a plurality of heartbeat signals from among a plurality of known heartbeat types, as shown in FIG. 3, and selects and determines at least one heartbeat type for classifying a plurality of heartbeat signals. One heart beat type can be used for classification of heart rate signals. In this case, a label (for example, numbers 0 to 10) may be further added to at least one heartbeat type to identify each type. At least one heart beat type, for example, Normal (label 0), Left bundle branch block beat (LBBB, label 1), Right bundle branch block beat (RBBB, label 2), abnormal. Aberrated atrial premature beat (AP, label 3), premature ventricular contraction (PVC, label 4), fusion of ventricular and normal beat (FVN: label 5), Nodal (junctional) premature beat (NP), label 6), Atrial premature contraction (APC), label 7, nodal escape beat (NE: nodal (junctional) escape beat, label 8), pace beat (PACE: Paced beat, label 9) and/or a combination of pace beat and normal beat (FPN: Fusion of paced and normal beat, label 10), etc., but is not limited thereto.

As shown in FIG. 2, at least one heart rate signal 111 transmitted to the beat classification unit 230 may have a QRS complex (Q, R, S) each having an R peak (R), and may have a normal heart rate. In the case of the heart rate signal 111 corresponding to , it may further have a P wave (P) and a T wave (T). According to one embodiment, the beat classification unit 230 detects the R peak (R) from the raw ECG data 110, etc. in order to extract at least one heart rate signal 111, and sets a certain reference range from the R peak. Signals within (z1, z2) can also be detected. Here, a certain reference range (z1, z2) may be defined in consideration of the heartbeat period. Normally, the heart beats once for 0.6 to 1 second. Therefore, in consideration of information loss, the reference range (z1, z2) for extracting the heart rate signal 111 may be determined to be a value corresponding to approximately 1 second or more (for example, 1.6 seconds). For example, the first range (z1) and the second range (z2) may each be defined to include a range from the R peak (R) to a point corresponding to approximately 200 points. That is, the reference ranges (z1, z2) have a length of approximately 400 points (the sum of 200 points corresponding to the first range (z1) and 200 points corresponding to the second range (z2)).

When obtaining at least one heart rate signal 111, the heart rate classification unit 230 may classify each of the at least one heart rate signal 111 into a corresponding heart beat type and store the classification result. For example, if the heart rate classification unit 231 determines that the heart rate signal 111 to be classified corresponds to a normal signal, the heart rate classification unit 231 connects or adds a label (i.e., 0) corresponding to the normal signal to the heart rate signal 111. The classification result for the corresponding heart rate signal 111 can be stored. Accordingly, a group of heart rate signal(s) 21 classified into at least one type (for example, 11 types as shown in FIG. 3) can be obtained. Heart rate signal(s) 21 classified into a predetermined type may be transmitted to the data enhancement unit 240. Depending on the embodiment, the acquired group of heart rate signal(s) 21 may first be temporarily or non-temporarily stored in the storage unit 103.

FIG. 4 is a diagram to explain the difference in the number of data between original ECG data and augmented ECG data, and FIG. 5 is a diagram to explain the process of generating augmented ECG data.

As shown in FIG. 4, data (i.e., heart rate signal)(s) belonging to the original ECG data 110 may be distributed biased toward a specific type. For example, in the MIT-BIH arrhythmia database, there are 72,124 heart rate signals corresponding to normal types, 6,613 heart rate signals for left bundle branch block (LBBB), and heart rate signals for right bundle branch block (RBBB). There are 5,004 of them. Additionally, there are 7,001 heart rate signals for premature ventricular contraction (PVC), 1,960 heart rate signals for atrial premature contraction (APC), and 3,619 heart rate signals corresponding to pace beats (PACE). On the other hand, there are only 150 heart rate signals for abnormal premature beats (AP) in the MIT-BIH arrhythmia database, 802 heart rate signals for combined ventricular and normal beats (VFN), and nodal premature beats ( There are only 83 heart rate signals for NP). In addition, there are 229 heart rate signals for nodular deviant beats (NE), and 260 heart rate signals for combined pace beats and normal beats (FPN). In other words, data on abnormal premature beats (AP), combined ventricular beats and normal beats (VFN), nodal premature beats (NP), nodular stray beats (NE), and combined pace beats and normal beats (FPN). The amount is relatively small and rare. When training a learning model using the entire data including relatively extremely rare types of data, the learning model cannot properly classify the types of these rare types of ECG data.

The data enhancement unit 240 determines a specific type (hereinafter referred to as a sparse type) where the heart rate signal is relatively lacking among the heart beat types, and enhances the sparse type of data (i.e., the heart rate signal) to make the distribution of the overall data relatively even. You can. That is, when a specific type (i.e., sparse type) contains very little data, the data augmentation unit 240 increases the number of sparse type data to a level sufficient for training and learning to create a new set of data (s). ) (for example, augmented ECG data 120) is generated and the learning model is trained based on the augmented ECG data 120, so that the classification performance of the learning model can be improved. For example, the data augmentation unit 240 may detect abnormal premature beats (AP), combined ventricular beats and normal beats (VFN), nodular premature beats (NP), nodular stray beats (NE), and pace beats and normal beats. The number of at least one heart rate signal among the combinations (FPN) of can be increased to a predetermined standard number. Here, the predetermined reference number may be selected by the user or determined by the processor 200 according to a predefined definition. For example, the reference number may be the average or median value of the number of raw ECG data (10), or the data amount of the type that contains the smallest amount of data among the types that are not sparse (e.g., atrial premature contraction (e.g., atrial premature contraction) It may be 1,960), which is the number of data corresponding to APC). In addition, users or designers can define the standard number in various ways according to their arbitrary choice.

According to one embodiment, the data enhancer 240, as shown in FIG. 5, receives heart rate signals 112, 113, and 114 corresponding to at least one rare type among the received heart rate signals, for example, an abnormal heart rate signal. Heart rate signals belonging to at least one of the following types: premature beats (AP), combined ventricular beats and normal beats (VFN), nodal premature beats (NP), nodal stray beats (NE), and combined pace beats and normal beats (FPN) ), and at least one heart rate signal 113 (hereinafter referred to as a reference heart rate signal) can be selected from among the selected heart rate signals 112, 113, and 114. Subsequently, the data enhancer 240 selects other heart rate signals (hereinafter referred to as peripheral heart rate signals) around the selected reference heart rate signal 113. For example, the heart rate signals 112 and 114 that are closest to the selected heart rate signal 113. (possible inclusion) is detected and one or more heart rate signals (121, 122) are newly generated between the selected reference heart rate signal (113) and the detected peripheral heart rate signals (112, 114), thereby generating the overall heart rate signals (112, 113, 114). , 121, 122) can be increased. In this case, one or more generated heart rate signals 121 and 122 may be randomly generated, and the value of the randomly generated heart rate signals 121 and 122 is the value of the reference heart rate signal 113 and the surrounding heart rate. It may be determined within the range of values of signals 112 and 114. Additionally, one or more new heart rate signals 121 and 122 may be generated by applying at least one interpolation method to each value of the reference heart rate signal 113 and the surrounding heart rate signals 112 and 114. The number of heart rate signals 121 and 122 generated may vary depending on the predetermined reference number and the number of heart rate signals 112, 113 and 114 corresponding to rare types. For such data augmentation, for example, SMOTE (synthetic minority oversampling technique) or ADASYN (adaptive synthetic sampling approach for imbalanced learning) may be used, but are not limited thereto.

According to the data augmentation operation of the data augmentation unit 240, as shown in FIG. 4, types with relatively sparse data (abnormal premature beat (AP), combination of ventricular beat and normal beat (VFN)) , the amount of data belonging to at least one of nodal premature beats (NP), nodular stray beats (NE), and combination of pace beats and normal beats (FPN)) increases to approximately 1960. New data obtained by augmenting the data, that is, augmented ECG data 120, may be stored in the storage unit 103 and may be transmitted to the data division unit 250 or the training unit 260, depending on the embodiment.

The data division unit 250 may divide the augmented ECG data 120 according to a predetermined standard to generate one or more data sets. For example, the data division unit 250 classifies each data of the augmented ECG data 120 into at least one of training data, verification data, and test data, into three data sets (training data set, A data set for verification and a data set for testing) may also be obtained. In this case, the data division unit 250 sets at least one of the training data set and the verification data set to include data newly generated by the data augmentation unit 240 (new heart rate signal), and sets the test data set. It is also possible to create a data set to include only data that previously existed in the original ECG data 110 (i.e., heart rate signals belonging to both the original ECG data 110 and the augmented ECG data 120). In other words, the test data set may not include the heart rate signal newly generated by the data augmentation unit 240. One or more data sets obtained by the data division unit 250 may be stored in the storage unit 103 as needed, and may be transmitted to at least one other device (e.g., a portable memory device) through the output unit 105. , may also be transmitted to another information processing device (such as a smartphone, desktop computer, or server hardware device).

The training unit 260 uses the augmented ECG data 120 or uses at least one data set (e.g., a training data set, a verification data) obtained from the augmented ECG data 120 by the data division unit 250. At least one learning model can be trained using the set and test data set. In this case, the training unit 260 may train a learning model using the training data set generated by the data division unit 250 and perform verification of the trained learning model using the verification data set. Additionally, further testing of the learning model can be performed using a test data set. Here, the learning model is, depending on need, a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), or a convolutional recurrent neural network (CRNN). Neural Network, Deep Belief Network (DBN), Deep Q-Networks, Long short term memory (LSTM), Multi-layer Perceptron, Support Vector Machine ( It may include at least one learning model such as a support vector machine (SVM), a generative adversarial network (GAN), and/or a conditional generative adversarial network (cGAN). A trained learning model may be obtained according to training by the training unit 260. The trained learning model may be stored in the storage unit 103 and, if necessary, may be transmitted to at least one other device (for example, a portable memory device or another information processing device) through the output unit 105. .

Figure 6 is a block diagram of an embodiment of the training unit.

For example, as shown in FIG. 6, the training unit 260 is a one-dimensional layer including five feature extraction layers (263-1 to 267-3) and two classification layers (268-1 to 268-3). You can also train a convolutional neural network (261, 1D CNN). Specifically, the one-dimensional convolutional neural network 261 includes an input layer 262 into which data such as augmented electrocardiogram data 120 is input, and a plurality of convolution layers 263-1, 264-1, and 264-1 that perform convolution. 265-1, 266-1, 267-1) and performing batch normalization on the output results of each convolution layer (263-1, 264-1, 265-1, 266-1, 267-1). Batch normalization layers (263-2, 264-2, 265-2, 266-2, 267-2) and batch normalization layers (263-2, 264-2, 265-2, 266-2, 267-2) A pooling layer (263-3, 264-3, 265-3, 266-3, 267-3) that performs pooling (max pooling, average pooling, etc.) on the output results, and a final pooling layer (267-3) It may include fully connected layers 268-1 to 268-3 that receive the output results, sequentially process and transmit the data, and an output layer 169 connected to the last fully connected layer 268-3. Here, the filter sizes and kernel sizes of the convolution layers 263-1, 264-1, 265-1, 266-1, and 267-1 may all be set the same, or some may be set the same. , some others may be determined differently, and all may be determined differently. For example, the size of the filters of each convolutional layer (263-1, 264-1, 265-1, 266-1, 267-1) is given as 8, 16, 32, 32, and 64, sequentially. and kernel sizes can be given as 16, 32, 32, 32, and 24. Similarly, for each pooling layer (263-3, 264-3, 265-3, 266-3, 267-3), the pooling size or stride size may be set the same, and some may be set the same. Some may be determined differently, others may be determined differently, and all may be determined differently. For example, the pooling size of all pooling layers (263-3, 264-3, 265-3, 266-3, and 267-3) may be set to 3 and the stride may be set to 2. Meanwhile, the activation function for at least one of the layers 263-1 to 268-3 may be a Relu function, a sigmoid function, a hyperbolic tangent function, or a soft max function.

Figure 7 is a diagram for explaining the performance of the trained learning model. In Figure 10, the vertical axis indicates the type actually classified, and the horizontal axis indicates the type determined by the trained learning model. The point where the actually classified type and the type determined by the learning model intersect is when data belonging to the types of labels (0 to 10) listed on the left is input, the learning model uses the labels (0 to 10) listed at the bottom. This refers to the probability of outputting a judgment result called type.

As shown in Figure 7, ECG data corresponding to normal (label 0), ECG data corresponding to right bundle branch block (RBBB) (label 2), and ECG data corresponding to abnormal premature beat (AP) (label 3). , ECG data corresponding to nodal premature beats (NP) (label 6), ECG data corresponding to nodular escape beats (NE) (label 8), ECG data corresponding to pace beats (PACE) (label 9), and pace beats. and ECG data (label 10) corresponding to combination of normal beats (FPN) is input, the learning model can determine and classify the corresponding type with complete accuracy. Additionally, ECG data corresponding to ventricular premature contractions (PVC) (label 4), ECG data corresponding to combined ventricular and normal beats (FVN) (label 5), or ECG data corresponding to atrial premature contractions (APC) (label 4). Even when data corresponding to label 7) was entered, the data could be accurately classified with a probability of at least 94%. Therefore, it can be seen that the learning model trained using the augmented ECG data 120, as described above, can determine the heartbeat type corresponding to the ECG data with very high accuracy.

The above-described data enhancement device 20 may be implemented using a specially designed device capable of performing at least one operation of the above-described noise removal, normalization, beat classification, data augmentation, generation of a data set, and training of a learning model. Alternatively, it may be implemented by using one or more information processing devices alone or in combination. Here, one or more information processing devices include, for example, desktop computers, laptop computers, server hardware devices, smart phones, tablet PCs, smart watches, smart bands, head mounted display (HMD) devices, portable devices, etc. or non-portable game consoles, navigation devices, remote controllers (remote controls), digital televisions, set-top boxes, media streaming devices, DVD playback devices, compact disc playback devices, sound playback devices (artificial intelligence speakers, etc.), home appliances (e.g. refrigerators) , fans, air conditioners, ovens or washing machines, etc.), manned or unmanned moving vehicles (e.g. vehicles such as cars, buses or two-wheelers, mobile robots, wireless model vehicles, robot vacuum cleaners, etc.), manned or unmanned flying vehicles (e.g. aircraft, helicopters, etc.) Me, drones, model airplanes, model helicopters, etc.), medical devices (X-ray imaging equipment, CT (Computed Tomography), mammography or MRI (Magnetic Resonance Imaging) equipment, etc.), robots ( It may include, but is not limited to, household, industrial or military use) or mechanical devices (industrial or military use). Depending on the situation or conditions, designers, users, etc. may consider and employ at least one of various devices for computational processing and control of information as the data enhancement device 20 in addition to the information processing device described above.

Hereinafter, an embodiment of a data augmentation method will be described with reference to FIG. 8.

Figure 8 is a flow chart of one embodiment of a data enhancement method.

Referring to FIG. 8, first, raw ECG data is acquired (300). Raw electrocardiogram data may include at least one heart rate signal corresponding to a normal or abnormal heart beat. The original ECG data may be provided in the form of a database, and the database may be provided in the form of an arrhythmia database, such as the MIT-BIH database. Acquisition of raw ECG data may be performed through direct input by the user, delivery from a portable memory device, and/or delivery through a wired or wireless communication network.

Noise removal processing is performed on the raw ECG data (310). According to one embodiment, noise removal processing may be implemented using a filter such as a fourth-order Butterworth filter, and the passband may be given as, for example, 0.5 Hz to 15 Hz.

Normalization may be performed on raw ECG data or noise-removed ECG data (320). Normalization, for example, subtracts the average of all raw ECG data or the average of noise-removed ECG data from each raw ECG data or noise-removed ECG data, and removes the standard deviation or noise of the original ECG data for the subtraction result. This can also be done by dividing the standard deviation of the ECG data.

Once raw ECG data, noise-removed ECG data, or normalized ECG data are obtained, heart rate signals may be obtained and classified (330). Acquisition of the heart rate signal may be performed using the R peak of raw ECG data, noise-removed ECG data, or normalized ECG data. Specifically, for example, acquisition of a heart rate signal may be performed by detecting the R peak from raw ECG data, noise-removed ECG data, or normalized ECG data, and extracting all signals (waves) within a certain reference range from the R peak. . Here, a certain reference range may be defined in consideration of the heartbeat period and, for example, may be determined as a range corresponding to a value of approximately 1 second or more (for example, 1.6 seconds). When at least one heart rate signal is detected from raw ECG data, noise-removed ECG data, or normalized ECG data, classification of the at least one detected heart rate signal is performed. For example, a heartbeat type corresponding to each heartbeat signal may be determined, and a label corresponding to the determined type may be added to the heartbeat signal. Here, the heart rate type is normal, left bundle branch block (LBBB), right bundle branch block (RBBB), abnormal premature cardiac beat (AP), premature ventricular contraction (PVC), combination of ventricular beat and normal beat (FVN), and nodule, as described above. It may also include premature beats (NP), atrial premature beats (APC), nodal escape beats (NE), pace beats (PACE), and/or a combination of pace beats and normal beats (FPN).

Data augmentation processing for sparse types with very little data may then be performed (340). That is, the number of data in a sparse type can be increased. In this case, the number of data after being increased may be equal to or close to the predefined reference number. The reference number may include, for example, an average value of the number of original ECG data, a median value of the number of original ECG data, or an amount of non-sparse type data whose amount is relatively small. The data augmentation operation includes, for example, selecting at least one heart rate signal from among heart rate signals corresponding to a predetermined sparse type, and at least one other heart rate signal located around the selected at least one heart rate signal (for example, in the selected heart rate signal). It may be performed by detecting adjacent heart rate signal(s) and randomly generating a heart rate signal between the selected at least one heart rate signal and the at least one detected adjacent heart rate signal.

Depending on the embodiment, the augmented ECG data obtained through data augmentation processing may be divided into at least two or more data sets (350). For example, the augmented ECG data may be split into a training data set, a validation data set, and a testing data set. In this case, the training data set or verification data set may be generated to include newly generated heart rate signals according to augmentation processing, and the test data set may be generated to include only heart rate signals that previously existed in the original ECG data.

According to one embodiment, training on the learning model may be performed sequentially (360). Training of the learning model is performed using the above-described training data, and may be verified using verification data. Training of the learning model may be performed using augmented ECG data obtained through data augmentation processing. Here, the learning model may include, for example, a convolutional neural network, a multilayer perceptron, a deep neural network, a recurrent neural network, a convolutional recurrent neural network, or a long-term memory, but is not limited thereto.

Acquisition of the above-described raw ECG data (300), noise removal (310), data normalization (320), acquisition and classification of heart rate signals (330), augmentation of rare specific types of data (340), data segmentation (350) and the training process 360 of the learning model may all be performed by the same information processing device, some may be performed by the same information processing device and others may be performed by different information processing devices, and all may be performed by different information processing devices. It may also be performed by In addition, the above-described noise removal process 310, normalization process 320, data division 350, and learning model training process 360 may be omitted, depending on the embodiment.

The data enhancement method according to the above-described embodiment may be implemented in the form of a program that can be driven by a computer device. A program may include instructions, libraries, data files, and/or data structures, etc., singly or in combination, and may be designed and produced using machine code or high-level language code. The program may be specially designed to implement the above-described method, or may be implemented using various functions or definitions that are known and available to those skilled in the art in the computer software field. In addition, here, the computer device may be implemented by including a processor or memory that enables the function of the program, and may further include a communication device if necessary. Additionally, a program for implementing the above-described data enhancement method may be recorded on a recording medium readable by a device such as a computer. Recording media that can be read by a computer include, for example, semiconductor storage media such as ROM, RAM, SD cards, or flash memory (eg, solid state drives (SSD), etc.), or magnetic disk storage such as hard disks or floppy disks. At least one medium capable of temporarily or non-temporarily storing one or more programs that are executed upon call from a device such as a computer, such as an optical recording medium such as a compact disk or DVD, or a magneto-optical recording medium such as a floptical disk. It may include any type of physical storage medium.

Although several embodiments of the data enhancement device and data enhancement method have been described above, the data enhancement device or data enhancement method is not limited to the above-described embodiments. Various other devices or methods that can be implemented by those skilled in the art by modifying and modifying the above-described embodiments based on the above-described embodiments may also be examples of the above-described data enhancement device or data enhancement method. For example, the described method(s) may be performed in an order other than as described, and/or component(s) of the described system, structure, device, circuit, etc. may be combined, connected, or otherwise in a form other than as described. Even if combined or replaced or replaced by other components or equivalents, it can be an embodiment of the data enhancement device and/or data enhancement method described above.

100: Data augmentation device 110: Original electrocardiogram data
120: Augmented ECG data 200: Processor
210: noise removal unit 220: normalization unit
230: Beat classification unit 240: Data augmentation unit
250: data division unit 260: training unit

Claims (12)

An input unit for acquiring at least one raw electrocardiogram data including at least one heart rate signal;
Classify the at least one heart rate signal into a corresponding heart rate type, obtain augmented ECG data by increasing the heart rate signal for a rare type with few heart rate signals among the heart rate types, and select a reference heart rate signal from among the heart rate signals of the rare type. and select at least one peripheral heart rate signal surrounding the reference heart rate signal, and generate one or more new random heart rate signals between the reference heart rate signal and the peripheral heart rate signal to enhance the sparse heart rate signal,
A data enhancement device comprising a processor, wherein the value of the arbitrary heart rate signal is determined within the range of the value of the reference heart rate signal and the value of the peripheral heart rate signal. According to paragraph 1,
The processor is a data enhancement device that removes noise from the original ECG data to obtain noise-removed ECG data. According to paragraph 2,
The processor performs normalization on the raw ECG data or the noise-removed ECG data to obtain normalized ECG data, and uses the raw ECG data, the noise-removed ECG data or the normalized ECG data to generate the augmented ECG data. A data augmentation device that acquires data. According to paragraph 3,
The processor detects an R peak from the raw ECG data, the noise-removed ECG data, or the normalized ECG data, and extracts a signal from the R peak within a certain reference range to obtain the heart rate signal. A data enhancement device. According to paragraph 1,
The processor divides the augmented ECG data to generate training data, verification data, and test data, wherein the test data includes only heart rate signals belonging to both the original ECG data and the augmented ECG data. Data augmentation device. According to clause 5,
The processor trains a learning model using the training data, and the learning model includes a one-dimensional convolutional neural network. A data enhancement device acquiring at least one raw electrocardiogram data including at least one heart rate signal;
classifying, by the data enhancement device, the at least one heart rate signal into a corresponding heart beat type;
Including, wherein the data enhancement device acquires augmented electrocardiogram data by increasing the heart rate signal for a rare type with a small heart rate signal among the heart rate types,
The step of the data enhancement device acquiring augmented ECG data by increasing the heart rate signal for a rare type with a small heart rate signal among the heart rate types,
selecting, by the data enhancement device, a reference heart rate signal from the sparse type of heart rate signal;
selecting, by the data enhancement device, at least one peripheral heart rate signal surrounding the reference heart rate signal; and
The data enhancement device augments the sparse heart rate signal by newly generating one or more random heart rate signals between the reference heart rate signal and the peripheral heart rate signal, and the value of the generated random heart rate signal is the value of the reference heart rate signal. and determining within a range of peripheral heart rate signal values. In clause 7,
The data enhancement method further comprising: removing noise from the original ECG data by the data enhancement device to obtain noise-removed ECG data. According to clause 8,
It further includes the step of the data enhancement device performing normalization on the raw ECG data or the noise-removed ECG data to obtain normalized ECG data,
The step of acquiring the augmented ECG data is
A data enhancement method comprising: acquiring, by the data enhancement device, the augmented ECG data using the raw ECG data, the noise-removed ECG data, or the normalized ECG data. According to clause 9,
detecting, by the data enhancement device, an R peak from the raw ECG data, the noise-removed ECG data, or the normalized ECG data; and
The data enhancement method further comprising the step of the data enhancement device extracting a signal from the R peak within a certain reference range to obtain the heart rate signal. In clause 7,
It includes the step of the data augmentation device dividing the augmented ECG data to generate training data, verification data, and test data,
A data augmentation method wherein the test data includes only heart rate signals belonging to both the original ECG data and the augmented ECG data. According to clause 11,
Comprising: the data augmentation device training a learning model using the training data,
The learning model is a data augmentation method including a one-dimensional convolutional neural network.