KR20230013485A - Method and apparatus for predicting and classifying irregular clinical time-series data - Google Patents

Abstract

Disclosed are a method and a device for predicting and classifying irregular clinical time-series data. The device for predicting and classifying irregular clinical time-series data includes: a multi-view integration self-attention module for generating a deep expression of irregular clinical time-series data by integrating the irregular clinical time-series data, observation/absence data, and temporal information data; an auxiliary analysis unit for generating an auxiliary missing replacement value using the deep expression; and a predictor and classifier for classifying a patient condition prediction or diagnosis using the deep expression.

Description

Method and apparatus for predicting and classifying irregular clinical time-series data

The present invention relates to a method and apparatus for predicting and classifying irregular clinical time series data.

Electronic health data (EHR) is a collection of patient health data such as diagnoses, vital signs, laboratory test records, procedures and text descriptions. In the recent decade, significant progress has been made in developing deep learning models using multivariate EHR time series due to rich health data sets.

In particular, clinical data including EHR not only show large irregularities in terms of temporal and measurement variables, but also show low performance in predicting and classifying patient conditions due to high missing rates.

Although many studies have been conducted to extract and predict the characteristics of measured values using machine learning models, there is a limitation that additional data preprocessing, such as replacing missing values or changing irregular data into regular data, exists.

Recent deep learning techniques have the advantage of being able to simultaneously extract, predict, and classify features, so they have been in the limelight in other fields such as various natural language processing and have shown many developments at the same time. However, there are limitations in using deep learning techniques for prediction and classification in the field of clinical time series data classification due to factors such as irregularity or high missing rate.

The present invention is to provide a method and apparatus for predicting and classifying irregular clinical time series data.

In addition, the present invention predicts and classifies irregular clinical time series data that can provide prediction and classification results by modeling irregular clinical data with high missing values and missing pattern information included therein based on a self-attention mechanism. It is to provide a method and apparatus.

In addition, the present invention predicts irregular clinical time series data that can improve the performance of prediction and classification without additional preprocessing of data by integrating irregular clinical time series data and information on missing patterns derived therefrom into an autologous mechanism-based method. and a classification method, and apparatus.

According to one aspect of the present invention, an apparatus for predicting and classifying irregular clinical time series data is provided.

According to an embodiment of the present invention, a multi-view integration self-attention module generating an in-depth representation of the irregular clinical time series data by integrating regular clinical time series data, observation data, and time information data; an auxiliary analyzer generating an auxiliary missing replacement value using the deep expression; and a predictor and classifier for classifying patient condition prediction or diagnosis using the deep expression.

In the prediction and training of the classifier, the auxiliary missing replacement value may be further used.

The multi-view integrated self-attention module obtains a feature value by embedding each of the irregular clinical time-series data, the observation/nonexistence data, and the time information data, and uses the self-attention mechanism to obtain the observation/nonexistence data and the time information data. Missing pattern information is generated by integrating feature values of , and after integrating the missing pattern information and the irregular clinical time series data using an autologous attention block, it is applied to a feed-forward network (FFN) module to express the depth. can create

After masking some of the irregular clinical time series data, the auxiliary analysis unit may generate an auxiliary missing replacement value by restoring the masked value using the deep expression.

According to another aspect of the present invention, a method for predicting and classifying irregular clinical time series data is provided.

According to an embodiment of the present invention, (a) generating an in-depth representation of the irregular clinical time series data by integrating irregular clinical time series data, observation data, and time information data; (b) generating an auxiliary missing replacement value by masking some of the irregular clinical time series data and restoring the masked value using the deep representation; and (c) training a predictor and classifier model for classifying patient condition prediction or diagnosis using the deep representation and the auxiliary missing replacement value.

Step (a) may include: embedding the irregular clinical time-series data, the observation data, and the time information data to obtain respective feature values; generating missing pattern information by integrating feature values of the observation data and the time information data based on a first self-attention module; and integrating the feature values of the irregular clinical time series data and the missing pattern information based on a second self-attention module, and applying the result to a feed-forward network (FFN) to generate a deep representation of the missing pattern. can include

By providing a method and apparatus for classifying and predicting irregular clinical time series data according to an embodiment of the present invention, irregular clinical data with high missing values and missing pattern information included therein based on a self-attention mechanism are provided. It has the advantage of being able to provide accurate prediction and classification results by modeling together.

In addition, the present invention has the advantage of improving the performance of prediction and classification without additional preprocessing of data by integrating irregular clinical time series data and information on missing patterns derived therefrom into an autologous mechanism-based method.

1 is a diagram schematically showing the configuration of an apparatus for predicting and classifying irregular clinical time-series data according to an embodiment of the present invention.
2 is a diagram schematically showing the internal configuration of a multi-view integration self-attention module according to an embodiment of the present invention.
Figure 3 is a block diagram schematically showing the internal configuration of the secondary analysis unit according to an embodiment of the present invention.
4 is a diagram showing pseudo code for a method for predicting and classifying irregular clinical time series data according to an embodiment of the present invention.
5 is a flowchart illustrating a method of classifying and predicting irregular clinical time series data according to an embodiment of the present invention.

Singular expressions used herein include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "consisting of" or "comprising" should not be construed as necessarily including all of the various components or steps described in the specification, and some of the components or some of the steps It should be construed that it may not be included, or may further include additional components or steps. In addition, terms such as "...unit" and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software or a combination of hardware and software. .

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram schematically showing the configuration of an apparatus for predicting and classifying irregular clinical time-series data according to an embodiment of the present invention, and FIG. 2 is an interior of a multi-view integrated self-attention module according to an embodiment of the present invention. Figure 3 is a block diagram schematically showing the internal structure of the auxiliary analysis unit according to an embodiment of the present invention, Figure 4 is a diagram showing irregular clinical time series data prediction according to an embodiment of the present invention and a pseudo code for a classification method.

Referring to FIG. 1 , an apparatus 100 for predicting and classifying irregular clinical time series data according to an embodiment of the present invention includes a multi-view integration self-attention module 110, an auxiliary analysis unit 120, a prediction and classifier 130, It is configured to include a memory 140 and a processor 150 .

The multi-view integration self-attention module 110 receives irregular clinical time series data, observation data, and time information data as input data, sequentially integrates them to find features, and applies them to a forward neural network model to determine irregular clinical time series data. It is a means for creating in-depth representations of

The multi-view integration self-attention module 110 includes a data embedding unit 210, an integration module 220, and a forward neural network module 230.

길이의 시점

의 D-차원 다변량 시계열 집합이 주어지면, 관측 시계열을

와 같이 나타내기로 한다. 여기서,

는 모든 변수의 t번째 관측을 나타내고,

는

내의 d번째 엘리먼트를 나타낸다. For each subject n,

point of length

Given a set of D-dimensional multivariate time series of

to be expressed as here,

denotes the tth observation of all variables,

Is

Indicates the d-th element in

은 결측값(missing values)를 포함하기 때문에, 시계열 데이터에 대해 관측 또는 결측 변수를 마크하기 위해

과 동일한 사이즈를 가지는 관측 여부 정보(즉, 결측 지표(missing indicator))

를 도입하기로 한다. time series data

to mark observed or missing variables for time series data, since they contain missing values.

Observation information having the same size as (i.e., missing indicator)

to introduce

이 관측되면,

이며, 그렇지 않은 경우

과 같이 생성될 수 있다. 또한, 불규칙 임상 시계열 데이터의 관측이 결측되는 경우, 입력은

내에 상응하는 요소에 대해 제로(0)으로 설정될 수 있다. 각 변수 d에 대해 시간 간격을

로 나타내기로 한다. 여기서,

은 수학식 1과 같이 정의될 수 있다. Observation data is

If this is observed,

and, if not

can be created as In addition, if observations of irregular clinical time series data are missing, the input is

It can be set to zero (0) for the corresponding element in For each variable d, the time interval

to be represented by here,

Can be defined as in Equation 1.

와 같이 주어지면, 예측 라벨

정의를 위한 매핑 함수 구축이 가능하다. Irregular clinical time series dataset

Given as, the predicted label

It is possible to build a mapping function for definition.

The data embedding unit 210 derives feature values by embedding each input data (ie, irregular clinical time series data, observation data, and time information data). The multi-view integration self-attention module 110 may use time embedding (TE) as a variant of positional encoding that takes continuous time values as input, and converts the time embedding into an encoding vector representation.

Through this, it is possible to process irregularly sampled clinical time series data in consideration of the exact time point and time interval.

Time embedding can be expressed as in Equation 2.

및

는 각각 정확한 시간 포인트, 변수의 인덱스, 모델 차원, 데이터의 최대 시간 길이를 나타낸다. 시간 임베딩은 학습된 입력 임베딩에 추가될 수 있다. 이를 수학식으로 나타내면 수학식 3 내지 수학식 5와 같다. Here, t, d,

and

represents the exact time point, the index of the variable, the model dimension, and the maximum time length of the data, respectively. Temporal embeddings can be added to learned input embeddings. If this is expressed as an equation, it is equivalent to Equations 3 to 5.

는 각각 다시점 입력에 대한 상응하는 임베딩 가중치 행렬을 나타낸다. here,

denotes the corresponding embedding weight matrix for each multi-view input.

In the past, only missing data sources such as missing indexes and time intervals were utilized, but representations of missing data were not learned, resulting in unreliable evaluation due to inadequate modeling of missing data and creation of models that were not robust to measurement changes. There was a follow up problem.

Therefore, in an embodiment of the present invention, it is possible to learn a deep representation of irregular clinical time series data by effectively utilizing both the missing indicator and the time interval.

At all times, when the measured value D is given, the multi-view integrated self-attention module 110 embeds input data for irregular clinical time series data, observation data, and time information data, and through this, each input data (i.e., irregular feature values for clinical time series data, observation data, and time information data) can be obtained.

The multi-view integrated self-attention module 110 is a multi-headed self-attention block (MHA: It is based on multi-head attention.

An attention representation of multi-view irregular clinical time series data including observation X may be learned based on the self-attention block.

은 자가주의 블록을 통해 자신의 표현

을 학습할 수 있다. 여기서, 각 데이터 포인트는 자체 가중치 행렬과 선형적으로 결합되어 해당하는 Q, K 및 V로 공급될 수 있다. For example, each input set

self-expression through self-attention blocks

can learn Here, each data point can be linearly combined with its own weight matrix and fed into the corresponding Q, K and V.

The expression of attention for each input data can be organized into equations such as Equations 6 to 8.

는 SoftMax 활성화 함수를 나타내고,

는 키 벡터의 차원을 나타낸다. Here, W represents a set of learnable weight matrices,

denotes the SoftMax activation function,

represents the dimension of the key vector.

The integration module 220 relies on the autoattention block and is a means for integrating the feature values of the input data.

The integration module 220 may generate complex missing pattern information by integrating feature values embedded in observation data and time information data among input data. In addition, the integration module 220 may combine feature values of irregular clinical time series data and missing pattern information in the expression space.

In this way, if the basic expression is learned from the missing value itself, there is no need to impute the missing value, and there is no need to specify any heuristic function.

Accordingly, the integration module 220 may involve a two-step integration process (a first integration step and a second integration step). This will be described in more detail.

)을 획득할 수 있다. 이를 수학식으로 나타내면 수학식 9와 같다. That is, in the first integration step, the integration module 220 integrates the expression of the time interval based on the time information data and the expression of the missing indicator based on the observation data using the first self-attention block to express the missing pattern (

) can be obtained. If this is expressed as an equation, it is equal to Equation 9.

In the second integration step, the integration module 220 may integrate irregular clinical time series data and representation of missing pattern information using the second autologous attention block. If this is expressed as an equation, it is equivalent to Equation 10.

As described above, the integration module 220 may sequentially integrate input data based on a plurality of self-attention modules.

The result of sequential integration of the input data by the integration module 220 is input to the forward neural network module 230, and joint learning is performed by modeling the relationship between the irregular clinical time series data and the integrated missing pattern through the forward neural network module 230. In-depth expression can be output.

The forward neural network module 230 can be equally applied at each point in time to model dependencies between variables.

은 보조적인 결측 대체값 생성 및 환자 상태 예측 또는 진단을 분류에 이용될 수 있다. final deep expression

can be used to generate auxiliary missing replacement values and classify patient condition prediction or diagnosis.

)을 이용하여 보조적인 결측 대체값을 생성하기 위한 수단이다. The auxiliary analysis unit 120 is a deep expression output from the multi-view integration self-attention module 110 (

) is used to generate an auxiliary missing replacement value.

The auxiliary analysis unit 120 deliberately masks part of the irregular clinical time series data, and then restores the masked value using the final deep expression generated by the multi-view integration self-attention module 110 to obtain an auxiliary missing replacement value. can create

A detailed structure of the auxiliary analysis unit 120 is as shown in FIG. 3 .

As shown in FIG. 3 , the auxiliary analysis unit 120 includes a multi-head self-attention module 310 and a forward neural network module 320 .

)과 키와 값으로 불규칙 임상 시계열 데이터(

) 사이의 다중헤드 자가주의 모듈(310)을 적용하고, 순방향 신경망 모듈(320)을 뒤에 배치할 수 있다. The auxiliary analysis unit 120 outputs the deep expression from the multi-view integration self-attention module 110 as a query (

) and irregular clinical time series data as keys and values (

), and the forward neural network module 320 can be placed behind.

)을 사용하여 타겟 시간 시퀀스에 매핑되고, 대치된 데이터(

)를 생성할 수 있다. 이를 수학식으로 나타내면 수학식 11과 같다. The output layer is the learned embedding matrix (

) to the target time sequence, and the imputed data (

) can be created. This is expressed as Equation 11.

가 주어지면, 전가 손실

은 불규칙한 임상 시계열 데이터(X)와

에 의해 마스킹된 값을 위한 전가된 샘플(

) 사이의 마스킹된 평균 제곱 에러(MSE: mean squared error)에 의해 계산될 수 있다. Another masking vector introduced to represent the masked value

Given , the imputed loss

is irregular clinical time series data (X) and

Imputed samples for values masked by (

) can be calculated by the masked mean squared error (MSE) between

If this is expressed as an equation, it is equivalent to Equation 12.

By introducing imputation loss into the objective function, it is possible to provide auxiliary information for achieving optimal prediction results.

The synthetic loss can be defined as Equation 13 by accumulating prediction and imputation losses.

와

는 두 손실 사이의 비율 조절을 위한 하이퍼매개변수를 나타낸다. here,

Wow

represents a hyperparameter for adjusting the ratio between the two losses.

)을 통해 단대단(end-to-end) 방식으로 모델의 모든 매개변수가 최적화실 수 있다. composite loss (

) allows all parameters of the model to be optimized in an end-to-end manner.

The predictor and classifier 130 is a means for classifying a patient condition prediction or diagnosis based on the final deep expression output from the multi-view integration autoattention module 110 .

The predictor and classifier 130 may be trained using a deep expression and an auxiliary missing replacement value output by the auxiliary analyzer 120 during learning. The predictor and classifier 130 may be trained to minimize the loss value for the auxiliary missing replacement value using the deep representation and the auxiliary missing replacement value.

The auxiliary missing replacement value is used only for learning of the predictor and classifier 130, and may not be used after learning is completed.

The predictor and classifier 130 may predict a patient condition or classify a diagnosis based on the deep representation.

For example, predictor and classifier 130 may perform in-hospital mortality or LOS prediction.

과 같다. In-hospital mortality prediction is a binary classification that indicates whether a patient has died during hospitalization or is expected to be discharged, with a target label for

Same as

LOS prediction can be a regression task that predicts the number of days between a patient's hospitalization and the end of hospitalization.

)이 주어지면, 타임스탬프에 대한 평균 풀링을 수행하고, 풀링된 최종 표현(

) 다음에 다층 퍼셉트론(MLP)가 뒤 따른다. 최종 계층은 특정 작업에 따라 달라지며 분류 및 다중 레이블 분류에 시그모이드 계층이 사용될 수 있다. final deep expression (

), perform mean pooling over timestamps, and pool the final expression (

) followed by the multilayer perceptron (MLP). The final layer depends on the specific task, and sigmoid layers can be used for classification and multi-label classification.

)과 예측된 레이블(

) 사이의 분류 손실을 계산하기 위해 수학식 14 내지 수학식 16과 같은 목적 함수를 이용할 수 있다. The predictor and classifier 130 is a target label (

) and predicted labels (

), objective functions such as Equations 14 to 16 may be used to calculate the classification loss between

The binary classification loss can be calculated using Equation 14.

The regression loss can be calculated using Equation 15.

The multi-level classification loss can be calculated using Equation 16.

는 마이너리티 클래스를 위한 포커싱 매개변수를 나타내며,

는 클래스간의 중요도 균형을 위한 가중치를 나타낸다. where N represents the total number of patients, K represents the total number of labels in the database,

represents the focusing parameter for the minority class,

represents the weight for the balance of importance between classes.

The memory 140 stores various commands required to perform the method of classifying and predicting irregular clinical time series data according to an embodiment of the present invention.

The processor 150 includes internal components (eg, the multi-view integration self-attention module 110, the auxiliary analysis unit 120, prediction and It is a means for controlling the classifier 130, memory 140, etc.). 4 illustrates pseudo code for a method for classifying and predicting irregular clinical time series data according to an embodiment of the present invention.

5 is a flowchart illustrating a method of classifying and predicting irregular clinical time series data according to an embodiment of the present invention.

In step 510, the apparatus 100 for predicting and classifying irregular clinical time series data generates an in-depth representation of the irregular clinical time series data by integrating the irregular clinical time series data, observation data, and time information data.

As already described above, the apparatus 100 for predicting and classifying irregular clinical time series data obtains each characteristic value by embedding the irregular clinical time series data, the observation data, and the time information data, and uses a self-attention mechanism to Missing pattern information is generated by integrating feature values of the observation data and the time information data, and after integrating the missing pattern information and the irregular clinical time series data using a self-attention block, a feed-forward network (FFN) ) module to generate the deep expression.

In step 515, the apparatus 100 for predicting and classifying irregular clinical time series data generates an auxiliary missing replacement value using the irregular clinical time series data and deep expression.

As already described above, the apparatus 100 for predicting and classifying irregular clinical time series data masks some of the irregular clinical time series data, and then restores the masked values using a deep representation to generate an auxiliary missing replacement value. can create

In step 520, the apparatus 100 for predicting and classifying irregular clinical time series data learns a model (ie, the predictor and classifier 130) using the deep representation and the auxiliary missing replacement value. When such a learning process is completed, patient state prediction or diagnosis may be classified using deep expression.

Devices and methods according to embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in computer readable media. Computer readable media may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on a computer readable medium may be specially designed and configured for the present invention, or may be known and usable to those skilled in the art in the field of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - Includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media and ROM, RAM, flash memory, etc. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler.

The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

So far, the present invention has been looked at mainly by its embodiments. Those skilled in the art to which the present invention pertains will be able to understand that the present invention can be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered from a descriptive point of view rather than a limiting point of view. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the equivalent scope will be construed as being included in the present invention.

100: Irregular clinical time series data prediction and classification device.
110: multi-view integration self-attention module
120: auxiliary analysis unit
130: predictor and classifier
140: memory
150: processor

Claims (6)

a multi-view integration self-attention module for generating an in-depth representation of the irregular clinical time-series data by integrating irregular clinical time-series data, observation data, and time information data;
an auxiliary analyzer generating an auxiliary missing replacement value using the deep expression; and
An apparatus for predicting and classifying irregular clinical time series data including a predictor and classifier for classifying patient condition prediction or diagnosis using the deep expression.
According to claim 1,
The apparatus for predicting and classifying irregular clinical time series data, characterized in that the auxiliary missing replacement value is further used when learning the prediction and classifier.
According to claim 1,
The multi-view integration self-attention module,
Each feature value is obtained by embedding the irregular clinical time series data, the observation data, and the time information data, and missing pattern information is obtained by integrating the observation data and the feature value of the time information data using an autoattention mechanism. and generating the deep expression by integrating the missing pattern information and the irregular clinical time series data using an autologous attention block and then applying it to a feed-forward network (FFN) module. Time series data forecasting and classification device.
According to claim 1,
The auxiliary analysis unit,
An apparatus for predicting and classifying irregular clinical time series data, characterized in that, after masking some of the irregular clinical time series data, restoring the masked value using the deep expression to generate an auxiliary missing replacement value.
(a) generating an in-depth representation of the irregular clinical time series data by integrating irregular clinical time series data, observation data, and time information data;
(b) generating an auxiliary missing replacement value by masking some of the irregular clinical time series data and restoring the masked value using the deep representation; and
(c) training a predictor and classifier model for classifying patient condition prediction or diagnosis using the deep representation and the auxiliary missing replacement value.
According to claim 5,
In step (a),
obtaining feature values by embedding each of the irregular clinical time-series data, the observation data, and the time information data;
generating missing pattern information by integrating feature values of the observation data and the time information data based on a first self-attention module; and
Including integrating the feature values of the irregular clinical time series data and the missing pattern information based on a second self-attention module and then applying them to a feed-forward network (FFN) to generate a deep representation of the missing pattern. A method for predicting and classifying irregular clinical time series data, characterized in that.

Images