KR102635609B1 - Method and apparatus for predicting and classifying irregular clinical time-series data - Google Patents

Abstract

A method and apparatus for predicting and classifying irregular clinical time series data are disclosed. The irregular clinical time series data prediction and classification device includes: a multi-view integrated self-attention module that integrates irregular clinical time series data, observation data and temporal information data to generate a deep representation of the irregular clinical time series data; an auxiliary analysis unit that generates auxiliary missing replacement values using the deep expression; and a prediction and classifier that classifies patient condition prediction or diagnosis using the deep representation.

Description

Method and apparatus for predicting and classifying irregular clinical time-series data}

The present invention relates to a method and device for predicting and classifying irregular clinical time series data.

Electronic health data (EHR) is a collection of patient health data such as diagnoses, vital signs, laboratory test records, procedures, and text descriptions. Over the past decade, significant progress has been made in developing deep learning models using multivariate EHR time series due to the abundance of health datasets.

In particular, clinical data including EHR not only show large irregularities in time and measurement variables, but also show low performance in predicting and classifying patient conditions due to a high missing rate.

Although much research has been conducted to extract and predict the characteristics of measured values using machine learning models, there is a limitation that additional data preprocessing, such as replacing missing values or converting irregular data into regular data, is required.

Recently, deep learning techniques have the advantage of being able to extract features, predict, and classify features at the same time, so they have been in the spotlight in other fields such as natural language processing and have shown great progress at the same time. However, there are limitations in using deep learning techniques for prediction and classification in the field of clinical time series data classification due to factors such as irregularity and high missingness rate.

delete

(01) Patent Document Korean Patent No. 10-2248732 (2021.04.29.)

The present invention is to provide a method and device for predicting and classifying irregular clinical time series data.

In addition, the present invention predicts and classifies irregular clinical time series data that can provide prediction and classification results by modeling irregular clinical data with high missing values and missing pattern information included therein based on a self-attention mechanism. To provide a method and device.

In addition, the present invention integrates information about irregular clinical time series data and missing patterns derived therefrom into a method based on a self-attention mechanism to predict irregular clinical time series data, which can improve prediction and classification performance without additional preprocessing of the data. and to provide a classification method and device.

According to one aspect of the present invention, an apparatus for predicting and classifying irregular clinical time series data is provided.

According to one embodiment of the present invention, there is provided a multi-view integrated self-attention module that integrates regular clinical time series data, observation data, and temporal information data to generate a deep representation of the irregular clinical time series data; an auxiliary analysis unit that generates auxiliary missing replacement values using the deep expression; A prediction and classification device for irregular clinical time series data may be provided, including a prediction and classifier that classifies patient condition prediction or diagnosis using the deep expression.

When learning the prediction and classifier, the auxiliary missing replacement value may be further used.

The multi-view integrated self-attention module obtains each feature value by embedding each of the irregular clinical time series data, the observation status data, and the time information data, and uses a self-attention mechanism to obtain the observation status data and the time information data. Missing pattern information is generated by integrating the feature values, and the missing pattern information and the irregular clinical time series data are integrated using a self-attention block and then applied to a forward neural network (FFN: feed-forward network) module to express the deep expression. can be created.

The auxiliary analysis unit may mask some of the irregular clinical time series data and then restore the masked values using the deep expression to generate auxiliary missing replacement values.

According to another aspect of the present invention, a method for predicting and classifying irregular clinical time series data is provided.

According to an embodiment of the present invention, (a) integrating irregular clinical time series data, observation status data, and time information data to generate a deep representation of the irregular clinical time series data; (b) masking some of the irregular clinical time series data and then restoring the masked values using the deep representation to generate auxiliary missing replacement values; and (c) learning a prediction and classifier model that classifies a patient condition prediction or diagnosis using the deep expression and the auxiliary missing replacement value. A method for predicting and classifying irregular clinical time series data may be provided.

The step (a) includes calculating feature values by embedding each of the irregular clinical time series data, the observation status data, and the time information data; generating missing pattern information by integrating feature values of the observation status data and the time information data based on a first self-attention module; And a step of integrating the feature values of the irregular clinical time series data and the missing pattern information based on a second self-attention module and then applying it to a feed-forward network (FFN) to generate a deep representation of the missing pattern. It can be included.

By providing a method and apparatus for classifying and predicting irregular clinical time series data according to an embodiment of the present invention, clinical data with high missing values and irregular clinical data and missing pattern information included therein are classified based on a self-attention mechanism. There is an advantage in being able to provide accurate prediction and classification results by modeling together.

In addition, the present invention has the advantage of improving prediction and classification performance without additional preprocessing of the data by integrating information about irregular clinical time series data and missing patterns derived therefrom into a method based on a self-attention mechanism.

1 is a diagram schematically showing the configuration of an apparatus for predicting and classifying irregular clinical time series data according to an embodiment of the present invention.
Figure 2 is a diagram schematically showing the internal configuration of a multi-viewpoint integrated self-attention module according to an embodiment of the present invention.
Figure 3 is a block diagram schematically showing the internal configuration of an auxiliary analysis unit according to an embodiment of the present invention.
Figure 4 is a diagram illustrating pseudo code for a method for predicting and classifying irregular clinical time series data according to an embodiment of the present invention.
Figure 5 is a flowchart showing a method for classifying and predicting irregular clinical time series data according to an embodiment of the present invention.

As used herein, singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as “consists of” or “comprises” should not be construed as necessarily including all of the various components or steps described in the specification, and some of the components or steps may be included in the specification. It may not be included, or it should be interpreted as including additional components or steps. In addition, terms such as "... unit" and "module" used in the specification refer to a unit that processes at least one function or operation, which may be implemented as hardware or software, or as a combination of hardware and software. .

Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

Figure 1 is a diagram schematically showing the configuration of an apparatus for predicting and classifying irregular clinical time series data according to an embodiment of the present invention, and Figure 2 is a diagram showing the interior of a multi-view integrated self-attention module according to an embodiment of the present invention. It is a diagram schematically showing the configuration, Figure 3 is a block diagram schematically showing the internal structure of the auxiliary analysis unit according to an embodiment of the present invention, and Figure 4 is a prediction of irregular clinical time series data according to an embodiment of the present invention. and a diagram showing pseudo code for the classification method.

Referring to FIG. 1, the apparatus 100 for predicting and classifying irregular clinical time series data according to an embodiment of the present invention includes a multi-view integrated self-attention module 110, an auxiliary analysis unit 120, a prediction and classifier 130, It is configured to include a memory 140 and a processor 150.

The multi-view integration self-attention module 110 receives irregular clinical time series data, observation data, and time information data as input data, sequentially integrates them, finds features, and applies them to the forward neural network model to the irregular clinical time series data. It is a means to create an in-depth expression of

The multi-view integrated self-attention module 110 includes a data embedding unit 210, an integration module 220, and a forward neural network module 230.

For each subject n, point of length Given a set of D-dimensional multivariate time series of It will be expressed as follows. here, represents the tth observation of all variables, Is Indicates the dth element within.

time series data contains missing values, so to mark observed or missing variables for time series data, Observation information (i.e., missing indicator) with the same size as decided to introduce.

Observation data is When this is observed, , and if not It can be created as follows. Additionally, if observations of irregular clinical time series data are missing, the input The corresponding element within can be set to zero (0). For each variable d, the time interval is It will be expressed as . here, Can be defined as Equation 1.

Irregular clinical time series dataset Given as , predicted label It is possible to build a mapping function for definition.

The data embedding unit 210 derives feature values by embedding each input data (i.e., irregular clinical time series data, observation status data, and time information data). The multi-view integrated self-attention module 110 uses time embedding (TE) as a variation of position encoding that takes continuous time values as input, and can convert the time embedding into an encoding vector representation.

Through this, it is possible to process irregularly sampled clinical time series data by considering the exact time point and time interval.

Time embedding can be expressed as Equation 2.

where t, d, and represents the exact time point, variable index, model dimension, and maximum time length of data, respectively. Temporal embeddings can be added to the learned input embeddings. If this is expressed in a mathematical formula, it is the same as Equation 3 to Equation 5.

here, represents the corresponding embedding weight matrix for the multi-view input, respectively.

Conventionally, only sources of missing values such as missing indicators and time intervals are utilized, but the representation of missing data is not learned, resulting in unreliable evaluation due to inappropriate modeling of missingness and creation of models that are not robust to measurement changes. There was a problem that followed.

Therefore, in one embodiment of the present invention, both missing indicators and time intervals can be effectively utilized to learn a deep representation of irregular clinical time series data.

At any time, given the D measurement value, the multi-view integrated self-attention module 110 embeds the input data for irregular clinical time series data, observational data, and temporal information data, and thereby embeds the input data for each input data (i.e., irregular Feature values for clinical time series data, observation status data, and time information data) can be obtained.

The multi-view integrated self-attention module 110 is a multi-head self-attention block (MHA: It is based on multi-head attention.

Based on the self-attention block, an attention representation of multi-view irregular clinical time series data including observation X can be learned.

For example, each set of inputs Express yourself through self-attention blocks You can learn. Here, each data point can be linearly combined with its own weight matrix to feed into the corresponding Q, K, and V.

The attention expression for each input data can be organized into a mathematical equation, such as Equation 6 to Equation 8.

Here, W represents the set of learnable weight matrices, represents the SoftMax activation function, represents the dimension of the key vector.

The integration module 220 relies on the self-attention block and is a means for integrating feature values of input data.

The integration module 220 can generate complex missing pattern information by integrating the embedded feature values of the observation status data and time information data among the input data. Additionally, the integration module 220 can combine feature values and missing pattern information of irregular clinical time series data in the expression space.

In this way, if the basic expression is learned from the missingness itself, there is no need to impute missing values and there is no need to specify any heuristic function.

Accordingly, the integration module 220 may involve a two-step integration process (a first integration step and a second integration step). Let us explain this in more detail.

That is, in the first integration step, the integration module 220 integrates the expression of the time interval based on the time information data and the expression of the missing indicator based on the observation data using the first self-attention block to express the missing pattern ( ) can be obtained. If this is expressed mathematically, it is as shown in Equation 9.

In the second integration step, the integration module 220 may integrate the representation of irregular clinical time series data and missing pattern information using the second self-attention block. If this is expressed mathematically, it is as shown in Equation 10.

As described above, the integration module 220 may sequentially integrate input data based on a plurality of self-attention modules.

The result of sequentially integrating the input data by the integration module 220 is input to the forward neural network module 230, and joint learning is performed by modeling the relationship between the irregular clinical time series data and the integrated missing pattern through the forward neural network module 230. A deep expression can be output.

The forward neural network module 230 can be applied equally at each time point to model dependencies between variables.

Final deep representation Can be used to generate auxiliary missing replacement values and classify patient condition prediction or diagnosis.

The auxiliary analysis unit 120 is a deep expression output from the multi-view integrated self-attention module 110 ( ) is a means for generating auxiliary missing replacement values.

The auxiliary analysis unit 120 intentionally masks part of the irregular clinical time series data and then restores the masked value using the final deep expression generated by the multi-view integrated self-attention module 110 to provide auxiliary missing replacement values. can be created.

The detailed structure of the auxiliary analysis unit 120 is as shown in FIG. 3.

As shown in FIG. 3, the auxiliary analysis unit 120 includes a multi-head self-attention module 310 and a forward neural network module 320.

The auxiliary analysis unit 120 uses a deep expression output from the multi-view integrated self-attention module 110 as a query ( ) and irregular clinical time series data with keys and values ( ) The multi-head self-attention module 310 can be applied, and the forward neural network module 320 can be placed behind it.

The output layer uses the learned embedding matrix ( ), and the imputed data ( ) can be created. If this is expressed mathematically, it is as shown in Equation 11.

Different masking vectors introduced to represent masked values. Given , impute loss is irregular clinical time series data (X) and Imputed samples for values masked by ( ) can be calculated by the masked mean squared error (MSE) between.

If this is expressed mathematically, it is as shown in Equation 12.

By introducing an imputation loss into the objective function, it is possible to provide auxiliary information to achieve optimal prediction results.

The synthetic loss can be defined as Equation 13 by accumulating the prediction and imputation losses.

here, and represents the hyperparameter for adjusting the ratio between the two losses.

Composite loss ( ), all parameters of the model can be optimized in an end-to-end manner.

The prediction and classifier 130 is a means for predicting or classifying a patient's condition based on the final deep expression output from the multi-view integrated self-attention module 110.

The prediction and classifier 130 may be trained using deep expressions and auxiliary missing replacement values output by the auxiliary analysis unit 120 during learning. The prediction and classifier 130 can be trained using deep representations and auxiliary missing replacement values to minimize loss for auxiliary missing replacement values.

The auxiliary missing replacement value is used only for prediction and training of the classifier 130, and may not be used after training is completed.

The prediction and classifier 130 can predict the patient's condition or classify the diagnosis based on the deep expression.

For example, prediction and classifier 130 may perform in-hospital mortality or LOS prediction.

In-hospital mortality prediction is a binary classification indicating whether the patient died during hospitalization or is scheduled to be discharged, with the target label for Same as

LOS prediction can be a regression task predicting the number of days between a patient's admission and end of hospitalization.

Final deep representation ( ), perform average pooling on the timestamps, and obtain the final pooled representation ( ) followed by a multilayer perceptron (MLP). The final layer depends on the specific task, and sigmoid layers can be used for classification and multi-label classification.

The prediction and classifier 130 uses the target label ( ) and the predicted label ( ) To calculate the classification loss, objective functions such as Equation 14 to Equation 16 can be used.

Binary classification loss can be calculated using Equation 14.

Regression loss can be calculated using Equation 15.

Multi-level classification loss can be calculated using Equation 16.

Here, N represents the total number of patients, K represents the total number of labels in the database, represents the focusing parameter for the minority class, represents the weight for balancing importance between classes.

The memory 140 stores various commands necessary to perform a method for classifying and predicting irregular clinical time series data according to an embodiment of the present invention.

The processor 150 includes internal components (e.g., multi-view integrated self-attention module 110, auxiliary analysis unit 120, prediction and It is a means for controlling the classifier 130, memory 140, etc.). Figure 4 illustrates pseudo code for a method for classifying and predicting irregular clinical time series data according to an embodiment of the present invention.

Figure 5 is a flowchart showing a method for classifying and predicting irregular clinical time series data according to an embodiment of the present invention.

In step 510, the irregular clinical time series data prediction and classification device 100 integrates the irregular clinical time series data, observation data, and time information data to generate an in-depth representation of the irregular clinical time series data.

As already described above, the apparatus 100 for predicting and classifying irregular clinical time series data obtains each feature value by embedding each of the irregular clinical time series data, the observation status data, and the time information data, and uses a self-attention mechanism to obtain the respective feature values. Missing pattern information is generated by integrating the feature values of the observation data and the time information data, and the missing pattern information and the irregular clinical time series data are integrated using a self-attention block, followed by a feed-forward network (FFN). ) module can be applied to generate the deep representation.

In step 515, the irregular clinical time series data prediction and classification device 100 generates auxiliary missing replacement values using the irregular clinical time series data and deep expression.

As already described above, the irregular clinical time series data prediction and classification device 100 masks some of the irregular clinical time series data and then restores the masked values using deep expression to provide auxiliary missing replacement values. can be created.

In step 520, the apparatus 100 for predicting and classifying irregular clinical time series data learns a model (i.e., prediction and classifier 130) using deep expressions and auxiliary missing replacement values. Once this learning process is completed, the patient condition prediction or diagnosis can be classified using the deep expression.

Devices and methods according to embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. Computer-readable media may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on a computer-readable medium may be specially designed and constructed for the present invention or may be known and usable by those skilled in the computer software field. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes magneto-optical media and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

So far, the present invention has been examined focusing on its embodiments. A person skilled in the art to which the present invention pertains will understand that the present invention may be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered from an illustrative rather than a restrictive perspective. The scope of the present invention is indicated in the claims rather than the foregoing description, and all differences within the equivalent scope should be construed as being included in the present invention.

100: Irregular clinical time series data prediction and classification device.
110: Multi-view integrated self-attention module
120: Auxiliary analysis unit
130: Prediction and classifier
140: memory
150: processor

Claims (6)

a multi-view integrated self-attention module that integrates irregular clinical time series data, observational presence data, and temporal information data to generate an in-depth representation of the irregular clinical time series data;
an auxiliary analysis unit that generates auxiliary missing replacement values using the deep expression; and
An irregular clinical time series data prediction and classification device comprising a prediction and classifier that classifies patient condition prediction or diagnosis using the deep representation.
According to claim 1,
Irregular clinical time series data prediction and classification device, characterized in that the auxiliary missing replacement value is further used when learning the prediction and classifier.
According to claim 1,
The multi-view integrated self-attention module is,
Each feature value is obtained by embedding the irregular clinical time series data, the observation data, and the time information data, and the feature values of the observation data and the time information data are integrated using a self-attention mechanism to provide missing pattern information. Generating, integrating the missing pattern information and the irregular clinical time series data using a self-attention block and then applying it to a feed-forward network (FFN) module to generate the deep representation. Time series data prediction and classification device.
According to claim 1,
The auxiliary analysis unit,
An irregular clinical time series data prediction and classification device, characterized in that it masks some of the irregular clinical time series data and then restores the masked values using the deep expression to generate auxiliary missing replacement values.
In the irregular clinical time series data prediction and classification method performed in the irregular clinical time series data prediction and classification device,
(a) integrating irregular clinical time series data, observational availability data, and temporal information data to generate a deep representation of the irregular clinical time series data;
(b) masking some of the irregular clinical time series data and then restoring the masked values using the deep representation to generate auxiliary missing replacement values; and
(c) a method for predicting and classifying irregular clinical time series data, comprising the step of training a prediction and classifier model for classifying patient condition prediction or diagnosis using the deep representation and the auxiliary missing replacement values.
According to clause 5,
In step (a),
Obtaining feature values for each of the irregular clinical time series data, the observation status data, and the time information data;
generating missing pattern information by integrating feature values of the observation status data and the time information data based on a first self-attention module; and
Integrating the feature values of the irregular clinical time series data and the missing pattern information based on a second self-attention module and then applying it to a feed-forward network (FFN) to generate a deep representation of the missing pattern. A method for predicting and classifying irregular clinical time series data.