METHODS, SYSTEMS AND COMPUTER PROGRAMS USING MACHINE LEARNING TO OPTIMIZE PREDICTION OF AN OCCURRENCE OF A RECURRING MEDICAL SYMPTOM OR BODY BEHAVIOR The present invention relates to a computer-implemented method for optimizing a prediction of an occurrence of a recurring medical symptom or body behavior of a human being using machine learning, to a computer-implemented method for training at least one machine learning model for use in predicting occurrence of a recurring medical symptom or body behavior, and to corresponding systems, computer programs and non-transitory, computer-readable media. Migraine is the most prevalent neurological disorder affecting 12% of the global population. It is influenced by multiple contributing factors, including dietary habits, daily routines, genetics, as well as physiological and biochemical factors. Migraine manifests as recurring, incapacitating headache episodes, occurring either sporadically or persistently, with or without accompanying sensory disturbances. Migraine ranks as the second most significant contributor to the global disability, primarily affecting individuals between the productive ages of 22 and 55, with a higher prevalence among women. This condition represents a substantial source of economic losses on a global scale. However, predicting migraine occurrences and identifying triggers for individual patients remain challenging tasks. Predicting migraine occurrences and identifying triggers remain challenging due to the wide variability in individual triggers, which can include stress, hormonal changes, certain foods, environmental factors, and sleep patterns (see Khan, Johra, et al. “Genetics, pathophysiology, diagnosis, treatment, management, and prevention of migraine”). The complexity is compounded by the cumulative and inconsistent nature of these triggers, individual physiological differences, and the subjective nature of patient reporting. Electronic health records (EHRs) serve as a digital repository for patient health information. While their primarily provide a wealth of data for monitoring and tracking symptoms, thereby aiding in the management and prevention of a wide range of
health conditions and diseases. A significant portion of EHRs is time-stamped, facilitating the longitudinal tracking of a patient’s health status. EHRs have expanded beyond hospital systems to include health diaries collected by smartphone-based applications. These applications allow users to record various lifestyle factors that may influence their health over time. The use of such health apps represents an advancement in healthcare technology, enabling more proactive and personalized approaches to managing health conditions such as migraines. The growing volume of such data presented an opportunity to create comprehensive and practical machine learning models for risk stratification, disease prognosis, and chronic disease management. EHRs have become invaluable tools for tracking various health conditions over time. Researchers have developed effective prediction tools by utilizing deep learning architectures that handle sequential data, such as recurrent neural networks (RNNs), long short-term memory (LSTM) networks, and Gated Recurrent Units (GRUs). These advanced models leverage the temporal nature of EHRs to predict potential future diseases, providing crucial insights for preventative healthcare. Deep learning architectures like recurrent neural networks (RNNs), long short-term memory (LSTM), and Gated Recurrent Units (GRUs) were effective for representing temporal EHRs due to their ability to handle sequential data. These models excel in capturing complex temporal patterns, making them superior choices for various EHR applications. Convolutional neural networks (CNNs), originally designed for image processing, can also be applied to one-dimensional time series data like EHR. They offer parameter efficiency through sparse interactions and parameter sharing as each convolution filter is applied across the entire input. In the literature, a hybrid model combining CNN and RNN (or LSTM) architectures has been proposed for processing temporal EHR data. This hybrid approach leverages the strengths of both architectures: CNNs excel at spatial feature extraction, while RNNs excel at capturing temporal dependencies, making the model well-suited for time- series data.
Doctor AI is a well-known example of a model that uses longitudinal time-stamped EHR data to perform multilabel prediction (Choi, Edward, et al. “Doctor AI: Predicting clinical events via recurrent neural networks“). This model was developed using RNN and applied on a cohort of 260K patients over an 8-year period. The main aim of this model was to utilize longitudinal records of patient past hospital visits to predict the diagnosis and the medication prescribed by physicians during the next visit. This model exemplifies the effective application of EHR data in predictive healthcare analytics. In Choi, Edward, et al. “Using recurrent neural network models for early detection of heart failure onset“, researchers investigated the use of deep learning to enhance the prediction of heart failure onset by analyzing EHR data. They used data from 3,884 heart failure cases and 28,903 control subjects, leveraging RNN with GRU models to identify temporal relationships among events within a 12- to 18-month observation window and predict the likelihood of patient's heart failure. In Stubberud, Anker, et al.: “Forecasting migraine with machine learning based on mobile phone diary and wearable data“, scientists employed mobile phone-based headache diaries and wearable devices to automatically monitor potential migraine indicators such as heart rate, peripheral skin temperature, and muscle tension. They aimed to predict migraine occurrences for the following day using various conventional machine learning methods. The top-performing model was Random Forest. It achieved accuracy, recall (sensitivity), and specificity values of 0.56, 0.0, and 1.0, respectively, on a held-out test set. This means the model correctly identified all negative cases, achieving perfect specificity, but failed to identify any positive cases. Despite its insights, the study has limitations. Firstly, the sample size was small, involving only 18 patients. Secondly, the study duration was relatively short, lasting 295 days. Consequently, the machine learning models used in the study exhibited a limited generalization capability. In Houle, Timothy T., et al.: “Forecasting individual headache attacks using perceived stress: development of a multivariable prediction model for persons with episodic migraine“, researchers developed a multivariable prediction model to forecast headache attacks within 24-hour period based only on stress levels
perceived by migraine patients, while overlooking other important migraine trigger factors. Mobile applications dedicated to headache disorders allow users to log numerous headache characteristics, such as start and end times, pain intensity, and whether the headache was unilateral or throbbing, if the headache worsened with physical activity, was accompanied by nausea, vomiting, light sensitivity (photophobia), sound sensitivity (phonophobia), or aura. Moreover, these apps enable users to monitor lifestyle factors that may influence their headaches, for example, users can record their caffeine intake, fluid consumption, alcohol consumption, missed meals, and other health-related factors such as stress levels, energy levels, sleep duration and quality, and activity levels. Additionally, these apps can gather weather-related data and other sensor information, as discussed in Raffaelli, Bianca, et al.: “Determining the evolution of headache among regular users of a daily electronic diary via a smartphone app: observational study“. This comprehensive tracking can provide valuable insights for both users and healthcare professionals, facilitating more effective management and treatment of headache disorders. It is therefore an objective of the present invention to improve and further develop a method, system and computer program for predicting occurrence of a recurring medical symptom or body behavior, such as a migraine attack. In accordance with the invention, the aforementioned objective is addressed by a computer-implemented method comprising the features of claim 1. According to this claim, such a method for optimizing a prediction of an occurrence of a recurring medical symptom or body behavior of a human being using machine learning comprises obtaining input data comprising features related to the occurrence of the recurring medical symptom or body behavior. For example, the input data may be derived from or include data of an electronic health record. For example, the input data may be generated by a mobile application being executed on a mobile device, such as a smartphone, smartwatch or other type of (medical) wearable device. Alternatively, the input data may be generated by an application running on a server and be derived from data provided by the human being (and its mobile device(s)). The method comprises dividing the features included in the input data into two or
more groups of features. The method comprises encoding, using a trained encoder machine learning model, the features of the two or more groups of features into two or more embeddings. Each embedding represents a group of features. The method comprises inputting the two or more embeddings into a prediction machine learning model being trained to predict the occurrence of a recurring medical symptom or body behavior based on the two or more embeddings. The method comprises providing a prediction of the recurring medical symptom or body behavior based on an output of the prediction machine learning model. The proposed concept is based on the finding, that, by grouping the features into groups of features (also denoted “concepts” in the present disclosure) that are condensed in a lower-dimensional embedding, additional context is encoded into the representations of the respective features, with a focus on features that are relevant for the subsequent prediction of the occurrence of the recurring medical symptom or body behavior. For example, the technique outlined in Koh, Pang Wei, et al. “Concept bottleneck models” may be used for this purpose. This way, less relevant features are deemphasized in the respective context, which facilitates the training and improves the predictive capabilities of the prediction machine learning model being used to make the prediction. The predictive capabilities of the machine learning pipeline (comprising the encoder machine learning model and the prediction machine learning model) can be applied to recurring medical symptoms, such as migraine attacks, as well as to the analysis of features related to sleep, i.e. to determine which features or groups of features primarily impact the sleep of the human being. To further improve the predictive capabilities of the machine learning pipeline, the temporal context of the input data can be taken into account. For example, the input data may be longitudinal input data comprising a plurality of observations of the same subject over a plurality of points in time. In this context, longitudinal data, also known as panel data, is a type of data that involves observations of the same subjects (such as individuals, groups, or organizations) repeatedly over a period of time. This longitudinal data is represented in the embeddings being generated, by generating embeddings that each represent multiple time steps. Thus, the two or more embeddings may be multi time-step embeddings each representing a group
of features over multiple points in time. By providing the additional temporal context, the predictive performance of the machine learning pipeline can be improved further. For example, the two or more multi time-step embeddings may each comprise a concatenation of single time-step embeddings output by the trained encoder machine learning model, with each single time-step embedding representing a point of time. This facilitates generation of the multi time-step embeddings as part of the machine learning pipeline, as the single time-step embeddings can be used across multiple time-steps. In general, each of the multi time-step embeddings may represent a window of time that shifts according to the time step for which the prediction is to be performed. Thus, the two or more multi time-step embeddings may be encoded based on a window of points of time (i.e., a subset of points of time that is selected according to a window of time) within the plurality of points in time. In addition to the time-step concatenation (or as an alternative), the embeddings may also be concatenated across groups of features. In other words, the method may comprise concatenating the two or more embeddings to obtain a concatenated embedding representing the groups of features and providing the concatenated embedding as input for the prediction machine learning model. By using a concatenated embedding representing the groups of features, a single, time-aligned input can be provided to the prediction machine learning model, ensuring that the prediction is based on the same point(s) in time of input data. The proposed concept is based on grouping together features into groups (also denoted “concepts”). These high-level groups or concepts can be inferred using a clustering algorithm (e.g., K-means clustering, hierarchical clustering, Gaussian Mixture Models) or specified by domain experts to allow for more comprehensive representation learning. In particular, features may be grouped together that are topically related, e.g., as shown in Fig.7, where the features are grouped into the concepts “demographics” (including gender and age at signup), “behavioral” (smoking, alcohol consumption, caffeine consumption, fluid intake, missed meals), “health factors” (sleep duration, sleep quality, stress, activity, energy, mood),
“immutable” (air pressure, humidity, temperature, weekday, season) and “medication” (medication, medical class, non-opioid analgesic, triptane, betablocker, magnesium, anti-epileptic, combination analgesic, anti-depressant, calcium antagonist, CGRP (calcitonin gene-related peptide) blocker). Thus, the two or more groups of features may comprise at least one group comprising topically related features. For example, the two or more groups of features may comprise at least one of a group of features related to a behavior of the human being, a group of features related to a demographic of the human being, a group of features related to health features of the human being, a group of features related to medication taken by the human being and a group of features related to environmental factors (“immutables”). Such grouping can improve the predictive capabilities of the machine learning pipeline. In addition to the prediction of the recurring medical symptom or body behavior, another output of the machine learning pipeline may include an explanation of the prediction, i.e., an explanation as to which feature has an increased impact on the prediction. This can be done using explainable artificial intelligence (AI) techniques, such as SHAP (SHapley Additive exPlanations) or LIME (Local Interpretable Model- agnostic Explanations). In particular, the method may comprise providing, based on a set of Shapley values, information on an impact of features of the features related to the occurrence of the recurring medical symptom or body behavior on the prediction. This may help the human being understand what the likely cause of a predicted recurrence of the medical symptom is, e.g., of the migraine attack. This way, the human being can make minor adjustments in their behavior or medication based on the explanation to avoid recurrence of the recurring medical symptom. There are various ways for implementing the respective machine learning models. For example, the encoder machine learning model may be implemented using one of a Long Short-Term Memory (LSTM), a Convolutional Neural Network (CNN), and a Long Short-Term Memory-Convolutional Neural Network hybrid. By using a temporally aware machine learning model, such as an LSTM or an LSTM hybrid, the temporal information contained in the input data can be used as additional context when encoding the groups of features. For example, the prediction machine learning model may be implemented using one of a random forest, a support vector
machine and a neural network. By inputting the features as embedding into the prediction machine learning model, simpler model types, such as random forests or support vector machines, or relatively simple neural networks can be used as predictor, which can decrease the effort for training the prediction machine learning model and for performing inference using the prediction machine learning model. Further aspects of the present disclosure relate to a corresponding computer program, non-transitory, computer-readable medium comprising a program code and system for performing the above method. For example, an aspect relates to a computer program comprising instructions which, when the program may be executed by a computer, cause the computer to carry out the above method. Another aspect relates to a non-transitory, computer-readable medium comprising a program code that, when the program code may be executed on a processor, a computer, or a programmable hardware component, causes the processor, computer, or programmable hardware component to perform the above method. Moreover, another aspect relates to a system comprising interface circuitry, machine-readable instructions, and processor circuitry to execute the machine- readable instructions to carry out the above method. Another set of aspects of the present disclosure relate to a computer-implemented method, computer program, program code and system for training at least one machine learning model for use in predicting occurrence of a recurring medical symptom or body behavior, e.g., of a target human being. The method comprises obtaining a set of training data. The set of training data comprises a plurality of samples of input data. As outlined above, the samples of input data may comprise or be derived from electronic health records and may be provided by a mobile application or server application. Each sample of input data comprises features related to the occurrence of the recurring medical symptom or body behavior. The method comprises dividing the features included in the samples of input data into two or more groups of features. The method comprises training an autoencoder comprising an encoder machine learning model and a decoder machine learning model using the set of training data. The autoencoder is trained to encode, using the encoder machine learning model, the respective groups of features into an embedding representing the respective group of features. The resulting embeddings
have a lower dimensionality than the respective groups of features represented by the embeddings. The autoencoder is trained to decode, using the decoder machine learning model, the embeddings to restore the groups of features. The method comprises providing the encoder machine learning model of the autoencoder for use in encoding the features of the two or more groups of features into embeddings. By grouping the features into groups of features (also denoted “concepts” in the present disclosure) that are condensed in a lower-dimensional embedding, additional context is encoded into the representations of the respective features, with a focus on features that are relevant for the subsequent prediction of the occurrence of the recurring medical symptom or body behavior. This way, less relevant features are deemphasized in the respective context, which facilitates the training and improves the predictive capabilities of a prediction machine learning model being used in combination with the encoder machine learning model to make the prediction. The predictive capabilities of the entire machine learning pipeline (comprising the encoder machine learning model and the prediction machine learning model) can be applied to recurring medical symptoms, such as migraine attacks, as well as to the analysis of features related to sleep, i.e. to determine which features or groups of features primarily impact the sleep of the human being. The proposed concept is based on applying machine learning to generate embeddings and use the embeddings to predict occurrence of a recurring medical symptom or body behavior. Machine learning is a branch of artificial intelligence that involves the development of algorithms and models that allow computers to learn and make predictions or decisions without being explicitly programmed. It focuses on creating systems that can improve their performance over time by learning from data. Training a machine-learning model refers to the process of teaching the model to make accurate predictions or decisions. During training, the model is exposed to a large amount of data, which is used to adjust the model's internal parameters or weights. The model learns patterns, relationships, or rules from the training data, allowing it to generalize and make predictions on new, unseen data.
Training data is the set of examples or instances that is used to teach a machine- learning model. It is often labeled data, meaning that each example is associated with a known outcome or target value. The training data consists of both input features and the corresponding output or target variable. The model learns from this data by analyzing the patterns and relationships between the input features and the target variable. Training algorithms, such as supervised learning, semi-supervised learning, unsupervised learning or reinforcement learning may be used for training the machine-learning model. In the present concept, three machine learning models are used – an encoder machine learning model, a prediction machine learning model, and a decoder machine learning model (that is only used during training of the encoder machine learning model). Machine learning models are trained using machine learning. In particular, the autoencoder, and thus the encoder machine learning model and the decoder machine learning model, may be trained using unsupervised learning. An autoencoder is a type of machine learning model (e.g., neural network) that learns to copy its input to its output using unsupervised learning. It comprises two main parts: the encoder and the decoder. The encoder compresses the input data into a latent-space representation or encoded form (i.e., into the embeddings), and the decoder attempts to reconstruct the input data from this encoded form. The training of an autoencoder is unsupervised because it does not require any labels or target data; it essentially learns from the input data itself. To train the autoencoder, the samples of input data are input into the encoder machine learning model, which compresses it. This compressed data is passed through the decoder machine learning model which then attempts to recreate the original input data from this compressed form. The training process involves minimizing a loss function that measures the difference between the output of the decoder machine learning model (the reconstructed inputs) and the original inputs. By minimizing this loss, the autoencoder learns efficient data codings in an unsupervised manner. The resulting encodings capture the most salient features of the data necessary to reconstruct the input as closely as possible, thus learning a compressed knowledge representation of the data.
The other model being used by the method being used to predict occurrence of the recurring medical symptom or body behavior is the prediction machine learning model. Accordingly, the method may comprise training a prediction machine learning model to predict the occurrence of the recurring medical symptom or body behavior based on the embeddings output by the encoder machine learning model. In particular, the prediction encoder model may be trained using supervised learning. For this purpose, the training data may comprise, for each or a subset of training samples, a desired output for the prediction machine learning model. For example, the label may be a binary label indicating whether the recurring medical symptom or body behavior (e.g., the migraine attack) has occurred at a given point in time relative to the point(s) in time the input data samples represent. Thus, the prediction machine learning model may be trained as a classifier, with the output representing whether the recurring medical symptom or body behavior is predicted to occur at the given point in time relative to the input data being fed into the prediction machine learning model (and the encoder machine learning model, to prepare the embeddings that are used as input for the prediction machine learning model). The method may further comprise providing the prediction machine learning model together with the encoder machine learning model. The encoder machine learning model and the prediction machine learning model are used together as part of a machine learning pipeline. In particular, input data (e.g., the input data samples of the training data during training, or new input data during production use) is input into the encoder machine learning model, which outputs embeddings representing the input data. The resulting embeddings are input into the prediction machine learning model, resulting in a machine learning pipeline comprising both the encoder machine learning model and the prediction machine learning model. Between the two machine learning models, some data processing may be performed to adapt the raw embeddings output by the encoder machine learning model for the prediction machine learning model. In particular, each sample of input data may be longitudinal input data comprising a plurality of observations of the same subject over a plurality of points in time. These points in time may be (partially) represented by the embeddings. For example, the encoder machine learning model may be used to output, for each point in time represented
by the input data and for each group of features, an embedding. These embeddings may be concatenated, using a windowing approach and by group, into multi time- step embeddings. Thus, training the prediction machine learning model may comprise determining two or more multi time-step embeddings each representing a group of features over multiple points in time (by concatenating the single time-step embeddings into the multi time-step embeddings). For example, the two or more multi time-step embeddings may be encoded based on a window of points of time within the points in time represented by the respective input data sample. For example, the method may comprise concatenating the embeddings output by the encoder machine learning model to obtain the two or more multi time-step embeddings. The two or more multi time-step embeddings may then be input into the prediction machine learning model. Moreover, concatenation might not only be performed across time-steps, but also across groups. For example, the method may comprise concatenating the two or more multi time-step embeddings to generate a concatenated embedding, with the concatenated embedding being input into the prediction machine learning model. Machine-learning models, such as the machine-learning model being trained in the present disclosure, are often implemented as Artificial Neural Networks (ANNs), and in particular Deep Neural Networks, Support Vector Machines, Decision Tree models, or Random Forest models. In particular, as outlined above, the encoder machine learning model may be implemented using one of a LSTM, a CNN, and a LSTM-CNN hybrid. The prediction machine learning model may be implemented using one of a random forest, a support vector machine and a neural network. The best prediction performance may be achieved by tailoring the machine learning pipeline to a specific user (a target human being), i.e., to the user that is to use the machine learning pipeline. This can be done by using input data that is provided by the target human being, e.g., through use of a mobile application being executed by a mobile device of the target human being. Thus, the encoder machine learning model and/or the prediction machine learning model may be at least partially trained (e.g., fine-tuned) based on input data samples related to a target human being. To give a suitable starting point for fine-tuning, and to obtain models that are better at
generalization, the models may be pre-trained using the input data of other human beings. In other words, the encoder machine learning model and/or the prediction machine learning model may be at least partially trained based on input data samples related to a plurality of human beings. One core aspect of the proposed concept lies in the division of the features into groups. In particular, the features are divided into the two or more groups of features according to a mapping between features and groups of features. This mapping may be determined, e.g., as part of the training, using a clustering algorithm. For example, a contextual analysis module may assess the relevance and relationships of features based on the data, application context, and expert knowledge, to group features into groups (concepts) and allow the integration of new concepts as new features are introduced. For example, the groups may be dynamically adjusted. In particular, machine learning clustering techniques may be used to dynamically adjust the concept groups. For instance, if certain concepts become more relevant or change their relationships, the grouping algorithm may update the mapping, and thus the groupings, accordingly. In an alternative implementation, the mapping between features and groups of features may be pre-defined according to expert knowledge. In addition to the prediction of the recurring medical symptom or body behavior, another output of the machine learning pipeline may include the explanation of the prediction, i.e., an explanation as to which feature has an increased impact on the prediction. This can be done using explainable artificial intelligence (AI) techniques, such as SHAP (SHapley Additive exPlanations) or LIME (Local Interpretable Model- agnostic Explanations). In the context of the training method, the method may comprise computing, using the encoder machine learning model and the prediction machine learning model, a set of Shapley values. The set of Shapley values represents an impact of features of the features related to the occurrence of the recurring medical symptom or body behavior on a prediction provided by the prediction machine learning model. This set of Shapley values may be used, as part of an inference operation, to provide an explanation of the prediction being provided by the machine learning pipeline. This may help the human being understand what the likely cause of a predicted recurrence of the medical symptom is, e.g., of the
migraine attack. This way, the human being can make minor adjustments in their behavior or medication based on the explanation to avoid recurrence of the recurring medical symptom. Further aspects of the present disclosure relate to a corresponding computer program, non-transitory, computer-readable medium comprising a program code and system for performing the above method. For example, an aspect relates to a computer program comprising instructions which, when the program may be executed by a computer, cause the computer to carry out the above method. Another aspect relates to a non-transitory, computer-readable medium comprising a program code that, when the program code may be executed on a processor, a computer, or a programmable hardware component, causes the processor, computer, or programmable hardware component to perform the above method. Moreover, another aspect relates to a system comprising interface circuitry, machine-readable instructions, and processor circuitry to execute the machine- readable instructions to carry out the above method. There are several ways how to design and further develop the teaching of the present invention in an advantageous way. To this end it is to be referred to the dependent claims on the one hand and to the following explanation of preferred embodiments of the invention by way of example, illustrated by the figure on the other hand. In connection with the explanation of the preferred embodiments of the invention by the aid of the figure, generally preferred embodiments and further developments of the teaching will be explained. In the drawing Fig. 1a shows a flow chart of a computer-implemented method for optimizing a prediction of an occurrence of a recurring medical symptom or body behavior of a human being using machine learning; Fig. 1b shows a block diagram of a system for optimizing a prediction of an occurrence of a recurring medical symptom or body behavior of a human being using machine learning;
Fig. 2a shows a flow chart of a computer-implemented method for training at least one machine learning model for use in predicting occurrence of a recurring medical symptom or body behavior; Fig. 2b shows a block diagram of a system for training at least one machine learning model for use in predicting occurrence of a recurring medical symptom or body behavior; Figs. 3a to 3d give an overview over a dataset; Fig. 4 shows an overview of a training portion of a Temporal Recurrent Autoencoder for Concept Embedding (TRACE) algorithm; Fig. 5 shows an overview of an integration of TRACE with a trained classifier; Fig. 6 provides a detailed outline of an example implementation of the TRACE algorithm; Fig. 7 shows an example feature list; Fig. 8a and 8b compare a per-class recall across five distinct runs for competing methods using 2-day forecast and 3-day forecast datasets; Figs. 9a to 9c present a comparative analysis of three key performance metrics: precision, recall, and macro F1 score; Fig. 10 shows the corresponding weighted key performance metrics: precision, recall, and F1 score values; Fig. 11 summarizes an average training time in minutes for each of the competing model and the standard deviation in minutes across 5 runs for each mode;
Fig. 12 provides a global overview of feature importance derived from a Random Forest model leveraged in the TRACE algorithm; Fig. 13 shows an ablation analysis illustrating the recall per class for individual feature concepts and their combination using the 3-day forecast dataset; Fig. 14 shows precision, recall, macro F1 score, and the corresponding weighted values across five different runs for individual feature concepts and their combination using the 3-day forecast dataset; Fig. 15a to 15j display SHAP summary plots of the top 10 features influencing migraine occurrences for users who consistently used the application over an extended period; and Fig. 16 shows another illustration of an analysis of feature importance for a specific user. Embodiments of the present disclosure relate to a method, system and computer program for optimizing a prediction of an occurrence of a recurring medical symptom or body behavior of a human being using machine learning, and to a method, system and computer program for training the machine learning models being used by such a method, system and computer program. The method, system and computer program for optimizing a prediction of an occurrence of a recurring medical symptom or body behavior of a human being using machine learning is discussed in connection with Figs.1a and 1b, while the method, system and computer program for training the machine learning models is discussed in connection with Figs. 2a and 2b. Figs.3a to 15 show an example implementation, denoted TRACE, of the methods, systems and computer programs discussed in connection with Figs.1a to 2b.
Fig. 1a shows a flow chart of a computer-implemented method for optimizing a prediction of an occurrence of a recurring medical symptom or body behavior of a human being using machine learning. The method comprises obtaining 110 input data comprising features related to the occurrence of the recurring medical symptom or body behavior. The method comprises dividing 120 the features included in the input data into two or more groups of features. The method comprises encoding 130, using a trained encoder machine learning model, the features of the two or more groups of features into two or more embeddings, with each embedding representing a group of features. Optionally, the method comprises concatenating 140 the two or more embeddings to obtain a concatenated embedding representing the groups of features. The method comprises inputting 150 the two or more embeddings (e.g., their concatenation) into a prediction machine learning model being trained to predict the occurrence of a recurring medical symptom or body behavior based on the two or more embeddings. The method comprises providing 160 a prediction of the recurring medical symptom or body behavior based on an output of the prediction machine learning model. Optionally, the method further comprises the method comprises providing 170, based on a set of Shapley values and based on the input data, information on an impact of features of the features related to the occurrence of the recurring medical symptom or body behavior on the prediction. The method of Fig.1a may be performed by the corresponding system 10 shown in Fig.1b. Fig.1b shows a block diagram of a system 10 for optimizing a prediction of an occurrence of a recurring medical symptom or body behavior of a human being using machine learning. The system 10 shown in Fig. 1b is a computer system comprising interface circuitry 12, processor circuitry 14, machine-readable instructions, and, optionally, memory and/or storage circuitry 16. The interface circuitry 12 may be used to facilitate communication with other components, such as sensors, or with systems of computers. This communication may occur via a bus or a network, for example. The processor circuitry 14 may be used to provide the functionality of the system, for example, in conjunction with the interface circuitry 12 (for exchanging information) and/or optional memory or storage circuitry 16 (for storing information, such as machine-readable instructions). The processor circuitry 14 is therefore coupled with the interface circuitry 12 and, optionally, with the
memory or storage circuitry 16. For instance, the system 10 may comprise machine- readable instructions, i.e., a computer program, which may prompt the one or more processors to execute at least one of the methods introduced in connection with Fig. 1a. In accordance with the aforementioned, the method introduced in connection with Fig. 1a may be carried out by the one or more processors executing the machine-readable instructions. For example, the interface circuitry 12 may include or correspond to a network interface circuitry and/or a device interface circuitry configured to be communicatively coupled to one or more other devices, such as the one or more processors. For example, the interface circuitry may include a transmitter, a receiver, or a combination thereof (e.g., a transceiver), and may enable wired communication, wireless communication, or a combination thereof. For example, the processor circuitry 14 may include or correspond to one or more of a digital signal processor circuitry (DSP), a graphical processing unit (GPU), and/or a central processing unit (CPU). For example, the memory and/or storage circuitry 16 may include or correspond to volatile or nonvolatile storage circuitry, such as Random Access Memory (RAM), magnetic disks, optical disks, or flash memory devices. The one or more memory/storage devices may include both removable and non- removable memory devices. Fig.2a shows a flow chart of a computer-implemented method for training at least one machine learning model for use in predicting occurrence of a recurring medical symptom or body behavior, e.g., of a target human being, e.g., for use by the method and system discussed in connection with Figs.1a and 1b. The method comprises obtaining 210 a set of training data. The set of training data comprises a plurality of samples of input data. Each sample of input data comprises features related to the occurrence of the recurring medical symptom or body behavior. The method comprises dividing 230, according to a mapping between features and groups of features, the features included in the samples of input data into two or more groups of features. For example, the mapping between features and groups of features may be pre-defined according to expert knowledge, or may be determined 220 using a clustering algorithm. The method comprises training 240 an autoencoder comprising an encoder machine learning model and a decoder machine learning
model using the set of training data. The autoencoder is trained to encode, using the encoder machine learning model, the respective groups of features into an embedding representing the respective group of features. The resulting embeddings have a lower dimensionality than the respective groups of features represented by the embedding, and to decode, using the decoder machine learning model, the embeddings to restore the groups of features. Optionally, the method comprises training 260 a prediction machine learning model to predict the occurrence of the recurring medical symptom or body behavior based on the embeddings output by the encoder machine learning model. For example, the embeddings output by the encoder machine learning model may be concatenated 250 before they are input into the prediction machine learning model. The method comprises providing 270 a) the encoder machine learning model of the autoencoder for use in encoding the features of the two or more groups of features into embeddings, and b) (optionally) the prediction machine learning model. Further optionally, the method comprises computing and providing 280, using the encoder machine learning model and the prediction machine learning model, a set of Shapley values, with the set of Shapley values representing an impact of features of the features related to the occurrence of the recurring medical symptom or body behavior on a prediction provided by the prediction machine learning model. The method of Fig.2a may be performed by the corresponding system 20 shown in Fig.2b. Fig.2b shows a block diagram of a system for training at least one machine learning model for use in predicting occurrence of a recurring medical symptom or body behavior. The system 20 shown in Fig.2b is a computer system comprising interface circuitry 22, processor circuitry 24, machine-readable instructions, and, optionally, memory and/or storage circuitry 26. The interface circuitry 22 may be used to facilitate communication with other components, such as sensors, or with systems of computers. This communication may occur via a bus or a network, for example. The processor circuitry 24 may be used to provide the functionality of the system, for example, in conjunction with the interface circuitry 22 (for exchanging information) and/or optional memory or storage circuitry 26 (for storing information, such as machine-readable instructions). The processor circuitry 24 is therefore coupled with the interface circuitry 22 and, optionally, with the memory or storage circuitry 26. For instance, the system 20 may comprise machine-readable
instructions, i.e., a computer program, which may prompt the one or more processors to execute at least one of the methods introduced in connection with Fig. 2a. In accordance with the aforementioned, the method introduced in connection with Fig. 2a may be carried out by the one or more processors executing the machine-readable instructions. For example, the interface circuitry 22 may include or correspond to a network interface circuitry and/or a device interface circuitry configured to be communicatively coupled to one or more other devices, such as the one or more processors. For example, the interface circuitry may include a transmitter, a receiver, or a combination thereof (e.g., a transceiver), and may enable wired communication, wireless communication, or a combination thereof. For example, the processor circuitry 24 may include or correspond to one or more of a digital signal processor circuitry (DSP), a graphical processing unit (GPU), and/or a central processing unit (CPU). For example, the memory and/or storage circuitry 26 may include or correspond to volatile or nonvolatile storage circuitry, such as Random Access Memory (RAM), magnetic disks, optical disks, or flash memory devices. The one or more memory/storage devices may include both removable and non- removable memory devices. In the following, an example implementation of the methods, systems and computer programs of Figs. 1a to 2b is provided. In the present disclosure, an AI-powered system for forecasting migraine attacks and identifying their triggers is presented. Forecasting migraine attacks before they occur can significantly enhance the quality of life for sufferers by allowing them to take preemptive measures to mitigate or prevent the onset of symptoms. In the following, a technique is discussed that aims to predict migraine attacks, with an example implementation that leverages a comprehensive migraine dataset collected from a mobile app between 2016 and 2022, encompassing approximately 43,000 users and 7 million daily records. In the present disclosure, TRACE (Temporal Recurrent Autoencoder for Concept Embedding) is introduced, a novel approach developed to anticipate and forecast migraine attacks before they occur using mobile app-based headache diaries and/or electronic health records. The
proposed method leverages diverse headache and life style features relevant to migraine attacks (e.g., the features related to the occurrence of the recurring medical symptom or body behavior). It begins by generating feature concepts, i.e., relevant feature groups (see Koh, Pang Wei, et al. "Concept bottleneck models"), based on timestamps. Then, a (deep learning-based) autoencoder (e.g., the encoder machine learning model) may be used to generate lower-dimensional embeddings (e.g., the two or more embeddings) from these timestamped concepts. This strategy aims at facilitating a deeper understanding of migraine patterns and potential triggers by considering both the feature concepts and their timing. These embeddings may be subsequently input into a simpler model (e.g., the prediction machine learning model) such as Random Forest to predict the occurrence of a migraine attack the following day. Additionally, TRACE may leverage advanced Explainable AI (XAI) techniques to identify both individual-specific and global migraine triggers across diverse user populations. On two held-out sets for 2-day and 3-day forecasts, TRACE successfully predicts approximately 70% of migraine episodes before their onset, outperforming all competing methods while maintaining comparable precision and F1 score. To understand the predictive factors driving model performance, both global and local feature importance analyses are conducted. Global importance is assessed using ablation analysis and Random Forest, while local importance is evaluated on a per- patient basis using SHapley Additive exPlanations (SHAP). The SHAP analysis shows that while common migraine triggers exist, individual triggers can vary significantly, highlighting the need for personalized predictive tools. The present technique relates to health Informatics, deep learning, electronic health records, autoencoder, and explainable AI. TRACE begins by generating feature concepts (e.g., the two or more groups of features), i.e., relevant feature groups, based on timestamps. An (LSTM-based) autoencoder (e.g., the encoder machine learning model) then generates lower- dimensional embeddings from these timestamped concepts. This strategy aims at facilitating a deeper understanding of migraine patterns and potential triggers by considering both the feature concepts and their timing. These embeddings are subsequently input into a (Random Forest) model (e.g., the prediction machine
learning model) to predict the occurrence of a migraine attack the following day. TRACE is evaluated against several established methods, including LSTM (Long Short-Term Memory), CNN (Convolutional Neural Network), a hybrid LSTM-CNN model, and a stand-alone Random Forest. Additionally, an LSTM model with SHapley Additive exPlanations (SHAP) is leveraged to gain further insights into patient-specific migraine triggers. The example implementation discussed herein uses the M-sense dataset. M-sense was a smartphone-based electronic headache diary administered by patients. This mobile app was available in Germany, Austria, and Switzerland for Android from 2016 and iOS from 2017 until 2022. With M-sense, users logged different features of their headaches, such as the start and end times, the maximum intensity of pain (rated on a scale from 0 to 10), and binary features like whether the headache was unilateral, throbbing, worsened with physical activity, accompanied by nausea or vomiting, and whether there was sensitivity to light (photophobia), sound (phonophobia), or if they experienced migraine aura. Using these headache features, M-sense classified each headache occurrence as migraine, tension-type headache (TTH), or non-headache using an algorithm that adheres to the International Classification of Headache Disorder guidelines (ICHD-3). M-sense users were also able to document prescribed or administered medications, including the specific name and the time it was taken. They could log their medications, irrespective of whether they were prescribed for migraines or other chronic or acute conditions. Additionally, M-sense enabled users to track various lifestyle factors that might have influenced their headaches. Users could record their caffeine consumption, fluid intake, alcohol consumption, missed meals, as well as other health-related factors such as stress levels, energy levels, sleep duration, sleep quality, activity levels, and mood. For health-related factors, the app user is requested to assign numerical values ranging from 1 to 10, with 1 being the minimum and 10 being the maximum. Additionally, the app automatically gathered weather data, i.e., temperature, humidity, and air pressure. M-sense stored demographic data, namely, age and gender. Many of these features have been shown in previous studies to potentially trigger or contribute to prolonged headache attacks.
In a prior publication, a subset of the M-sense data, containing 1545 users, was used in an observational study aimed at tracking the progression of headaches among regular M-sense users. A larger subset of the same dataset, which included 2325 users, was leveraged to investigate the prevalence of headaches during the COVID-19 lockdown in Germany. The entire dataset collected by M-sense from 2016 to 2022 was acquired, which includes data from approximately 43,000 users with 7 million daily records. This dataset is composed of structured data and does not include any unstructured entries such as user-input diaries. For the purpose of the example implementation of the proposed technique, daily records labeled as either migraine or no-headache instances were extracted, excluding tension-type headaches (TTH) since TTH accounted for less than 5% of the records. To ensure the integrity and completeness of the dataset, a stringent filter was applied to retain only those records that exhibited no missing values across six key features. These features, which are referred to in the following as ‘daily health factors’, include sleep duration, sleep quality, energy level, mood, stress, and activity. Imputing missing values for these features posed a considerable challenge. All users who experienced at least a single migraine attack while using the app were included, as the goal is to forecast migraines and identify their triggers. Headache- related features were excluded from the predictive modeling task because these features were used to determine the label of each daily record, which is categorized as either ’migraine’ or ’no-headache’. In summary, abstracting from the M-sense dataset, the proposed concept leverages comprehensive headache datasets that encompass longitudinal data such as headache features, lifestyle factors, health variables, sensor data, and medical data and medical imaging collected over different time points for various users who suffered from migraine. Feature pre-processing was conducted on the dataset, including mean normalization per user for features such as sleep duration and quality, alcohol consumption, smoking count and caffeine intake. One-hot encoding was applied to features such as dataset comprises 3,819,618 daily records from 22,619 users. The
number of daily records labeled as ‘migraine’ is 554,625, while the number of daily records labeled as ‘no headache’ is 3,264,993. Therefore, the approximate ratio of records labeled ‘migraine’ to those labeled ‘no headache’ is 1:6. Figs.3a to 3d give an overview over the dataset. Fig. 3a shows the distribution of users by gender. Migraine prevalence is higher among females compared to males in the dataset. Fig.3b shows daily record counts, grouped by label and gender. Fig.3c shows age distribution. The majority of migraine patients fall between the ages of 22 and 55. Fig. 3d shows the average number of migraine days per month experienced by users. The majority of users in the dataset reported having fewer than ten migraine days per month. Assuming a set of N migraine patients; each patient, pi, is observed at Ti different time points, and each observation comprises or consists of F features. This data can be viewed as a set of multivariate time series, where each pi is represented by ^^ ∈ ℝ ^^×^. Here, the following notation is used: Xi,t,f is the value of the f-th feature for the i-th patient at the t-th time point, Xi,t is the feature vector for the i-th patient at the t-th time point, and yi,t is the label for the i-th patient at the t-th time point. For the example implementation and technique, daily records were considered, where each record comprises or consists of various features collected over a single day for an individual patient. To prepare time series datasets for forecasting migraine attacks, samples were created by sliding a window of size w, and extracting sequences of length w days. In more detail, for each patient pi, a sequence was extracted at each time ^ ∶ ^ < ^ ≤ ^^, such that: - a sequence at time t is extracted only if there is no headache occurrence at the previous w days, i.e., ^^,^ = 0 for all ^ ∈ {^ − ^, ... , ^ − 1}. - The sequence is built from the feature vector
from the previous w days. - Extract label ^^,^ for the current day ^. ^"migraine" if ^^,^ is a migraine event
"^^ ℎ^^^^^ℎ^" ^^ℎ^^^^^^ - Form training samples as pairs (^^,(^^^:^^^), ^^,^). Here,
represents the feature matrix of the days preceding the current day t. These features
predict whether or not the user pi will experience a migraine attack on the day t, as indicated by the label yi,t. - Ni represents the number of valid extracted time series sequences, i.e.,
patient pi. Two datasets were retrieved corresponding to w = 2 and w = 3, referred to as the 2- day and 3-day forecasts, respectively. Each dataset can be represented by: ^
is the number of valid sequences for all users. Initially, the Temporal Recurrent Autoencoder for Concept Embedding (TRACE) method may build upon the structure of a traditional autoencoder. An autoencoder is a type of neural network designed to compress input data ^ into a lower- dimensional representation, denoted as ^, and then reconstruct an approximation of the original data, ^^, from ^. During a training phase, the autoencoder aims to approximate the identity function, ℎ^,^(^) ≈ ^, by setting the target values to be equal to the inputs. Through unsupervised learning, autoencoders may thus identify latent variables that influence the underlying data distribution. These latent variables, collectively known as the latent space, represent the most important information in the input data. The autoencoder learns to use these variables to accurately reconstruct the original data. The learning process involves backpropagation, where the target values are set to be equal to the inputs, making the autoencoder learn an approximation to the identity function, ℎ^,^(^) ≈ ^, where W represents the weight matrix and b represents the bias vector of the neural network. By imposing constraints on the network, such as limiting the number of hidden units, interesting structures within the data can be uncovered (see Goodfellow et al.: Deep Learning). TRACE enhances the capabilities of a standard autoencoder by generating embeddings for feature concepts, not raw features, and by taking into consideration the temporal nature of input data. In the proposed method, features are grouped into relevant high-level groups (i.e., concepts), which can be inferred using a clustering algorithm (e.g., K-means clustering, hierarchical clustering, Gaussian Mixture Models) or specified by domain experts to allow for more comprehensive representation learning (see Koh, Prang Wei et al.: “Concept Bottleneck Models”).
For example, features such as air pressure, humidity, temperature, day of the week, and season are grouped under the “weather” concept, while features like smoking, alcohol consumption, caffeine intake, and missed meals fall under the “behavioral” concept. Additionally, TRACE incorporates a time-step aspect where each time-step (i.e., a fixed unit of time, such as an hour, a day or a month) is encoded separately then their representation is concatenated to generate ^. By leveraging this structure, TRACE can effectively compress sequential data into a lower-dimensional space while preserving its temporal characteristics. This makes TRACE particularly well- suited for tasks such as time-series forecasting (see Fig. 4). With respect to the methods shown in Figs.1a and 1b, this means that the input data used therein may be longitudinal input data comprising a plurality of observations of the same subject over a plurality of points in time. The two or more embeddings being generated with help of the encoder machine learning models multi time-step embeddings each representing a group of features over multiple points in time. These can be obtained by concatenating the single time-step embeddings output by the trained encoder machine learning model. Further optionally, the multi time-step embeddings may be concatenated, such that the concatenated embedding represents multiple groups of features, over multiple points in time. Fig. 4 shows an overview of a training portion of TRACE. Fig. 4 may provide an example implementation of the method of Fig.2a. F refers to the total number of features, N refers to the number of samples, and w refers to the observation window (i.e., total number of time points). In Fig. 4, first, the proposed method groups features into relevant high-level concepts (groups of features), i.e., ^^, ^^, … , ^^. Second, it splits each concept by time-step (with each time-step representing a point in time). For simplicity, the figure illustrates an example with three-time steps denoted as
^^, and ^^. Third, the encoder converts time-step concepts into lower- dimensional representations, ^^, ^^, … , ^^ (the embeddings). Then, using the latent space, the decoder follows the same steps as the encoder but in reverse to reconstruct an image of the original data. The learning process involves backpropagation, where the target values are set to be equal to the inputs, making the autoencoder learn an approximation to the identity function, ℎ^,^(^) ≈ ^,
where ^ represents the weight matrix and ^ represents the bias vector of the neural network. By imposing constraints on the network, such as limiting the number of hidden units, interesting structures can be uncovered within the data. As an example both the encoder and decoder can be implemented using LSTM networks or CNN or a hybrid model of LSTM-CNN. In Fig. 5, which provides an example implementation of the method of Fig. 1a, the proposed method integrates the trained encoders (e.g., the encoder machine learning model) with a classifier (e.g., the prediction machine learning model). A random forest model is used as an example; however, it can be replaced with other models depending on the requirements and complexity of the input data, the random forest can be replaced with a simpler model like Support Vector Machine (SVM) or more complex models like neural networks. The trained encoders are leveraged to generate meaningful representations that capture important features and temporal dependencies from the input concepts. These learned representations for each concept are then used as inputs to Random Forest for predicting future migraine attacks. Fig. 6 provides a detailed outline of an example implementation of the TRACE algorithm. In the following, a mathematical representation of the concept is provided. The notation is simplified by dropping the patient’s index. The focus is on the relationship between features and concepts. Let ^^ ∈
represent the feature matrix, where ^ ^ ^,^ indicates the f-th feature on the j-th time point in the sequence ending at time point t. Specifically, ^ ^ ^,^ is an element of the feature matrix
where ^ is the window size, representing the number of preceding time points that the model considers to predict the event at the current time point, ^.
contains features from the previous w days up to, but not including, day t. Let ^ ∈ × ^ represent the concept-feature mapping matrix, where: M^,^ = ^1 if feature f is part of concept c 0 otherwise
The aim is to compute the concept tensor ^^∈ℝ^×^, where K denotes the total number of concepts. The transformation from the feature tensor ^^ to the concept tensor ^^ is given by:
where represents the concept matrix for the time-series sequence ending at time t, and ^ ^ ^,^ represents the concept vector at the j-th time point within the matrix ^^ ^ . M and K are predefined based on expert knowledge, and they can also be inferred through clustering algorithms. The table shown in Fig. 7 outlines the example feature list used in the example implementation and their corresponding concepts. The encoder function, f, in the TRACE model encodes each concept at every time step, and Z is computed by concatenating the representations across all concepts and time steps within window w:
where ^ denotes the concatenation operator for encoded representations of the concepts at different time points. The decoder function, g, decodes the encoded representation Z to reconstruct an approximation of the input data: ^^ = ^(^). In the example implementation, LSTM networks are used for both encoder and decoder functions f^and g. Other models, e.g., CNN, LSTM-CNN, etc. can be used. Additionally, the Adam optimizer was leveraged during training due to its adaptive learning rate properties, while other optimizers can be used as well. After training TRACE to generate representations (embeddings) for each concept, the trained encoders are saved. These encoders are then utilized for representation learning, where the autoencoder is trained to generate meaningful representations
capturing important features and temporal dependencies from the input sequences. These learned representations (i.e., Z) can be used for downstream tasks such as forecasting or classification. In summary, after training TRACE to generate representations for each concept, the method or system saves the trained encoders. These encoders can be then utilized for representation learning, wherein the autoencoder is trained to generate meaningful representations capturing important features and temporal dependencies from the input sequences. These learned representations can be leveraged for downstream tasks such as forecasting or classification using. To forecast the occurrence of future migraine attacks, a simpler model, such as Random Forest, may be used on the temporal encoded feature concepts. In addition to forecasting migraine, the proposed system can be leveraged in predicting other various health conditions (e.g., recurring medical symptoms) such as periodic illnesses, also known as recurrent or episodic illnesses, are conditions that occur in cycles or with regular flare-ups and symptom-free periods. Examples of periodic illnesses include autoimmune diseases like lupus and Crohn’s disease, and other illnesses like Irritable Bowel Syndrome (IBS). These health conditions alternate between phases of active symptoms (flare-ups) and remission, where symptoms decrease or temporarily disappear. The timing and triggers for these cycles vary, and due to their periodic nature, they are often managed with lifestyle modifications, dietary adjustments, and stress management, with additional treatments as needed during flare-ups. By applying the predictive system (TRACE), the timing of these flare-ups can be predicted, their triggers can be identified, and patients can be helped to enhance their quality of life. Further, the predictive system can be applied to a sleep diary project that aims to improve sleep quality (as an example of body behaviors) The proposed TRACE algorithm operates on data being used to train the autoencoder and, eventually, by processing corresponding input data. Therefore, longitudinal data (e.g., the input data / samples of input data) is collected electronically, e.g., from migraine patients, in both a structured (tabular) format and as user-provided text notes or doctor’s notes in electronic headache diaries. In this
context, longitudinal data, also known as panel data, is a type of data that involves observations of the same subjects (such as individuals, groups, or organizations) repeatedly over a period of time. This kind of data collection allows researchers to analyze changes over time and to understand dynamics across various subjects or entities. The idea is to observe how certain variables evolve, how interventions impact outcomes, or to identify trends that only appear over the course of several measurements. Key characteristics and uses of longitudinal data include use of a time dimension, which allows tracking of changes in variables with respect to time, and repeated observation (where each subject in the dataset is observed multiple times, enabling the analysis of intra-individual changes and variations. A time-series dataset may then be prepared using a sliding window technique. An output of the proposed algorithm is a patient-specific prediction, e.g., of future migraine attacks. The output may optionally further include an explanation, in which a set of potential patient-specific migraine trigger factors can be derived using model-agnostic XAI approaches such as SHAP or LIME. The output may optionally further include a trigger identification (e.g., an explanation), which can be obtained by analyzing temporal patterns, such as the specific day or time point when each trigger occurs. For example, fluctuations in caffeine intake between workdays and weekends may increase the likelihood of a migraine attack. For this purpose, the method of Fig.1a may comprises providing 170, based on the set of Shapley values, information on an impact of features of the features related to the occurrence of the recurring medical symptom or body behavior on the prediction, and the method of Fig.2a may comprise computing 280, using the encoder machine learning model and the prediction machine learning model, the set of Shapley values. The proposed concept includes two components – a training component, discussed in connection with Figs. 2a, 2b and 4, in which TRACE is trained, and an application/integration component, discussed in connection with Figs. 1a, 1b and 5, in which the trained encoders are integrated with a classifier, e.g., for forecasting migraine attacks. The training component, which is illustrated in Fig.4, comprises one or more of the following operations and/or concepts. For example, the training component may comprise grouping features into high-level concepts, in which the features are organized into relevant, high-level concepts (i.e., categories), based on
either domain expert or using an existing clustering algorithm. The training component may comprise splitting the concepts by time-step, thereby dividing each concept according to different time steps. For illustration, in Fig.4, three time steps are shown, denoted as
^^, ^^. The training component may further comprise encoding time-step concepts, in which the encoder processes the time-step concepts and converts them into lower-dimensional representations. The representations of individual time-steps (i.e., the single time-step embeddings) may be concatenated to generate ^^^^^^^^ (the multi time-step embeddings), a lower- dimensional representation for each concept. The training component may further comprise decoding the lower-dimensional representation to reconstruct the original data. Using the latent space, the decoder may reconstruct an image of the original data by reversing the steps of the encoder. The integration / application component may comprise generating meaningful representations, in which the trained encoders are used to create meaningful representations (embeddings) that capture important features and temporal dependencies from the input concepts. The integration / application component may further comprise predicting future migraine attacks (or other recurring medical symptoms or behaviors), by using the learned representations ^^^^^^^^ as inputs to a classifier, such as Random Forest, to predict the future recurring medical symptoms or behaviors, such as future migraine attacks. The integration / application component may optionally further comprise using model-agnostic explainable AI (XAI) approaches to identify a set of potential patient-specific migraine triggers, with a focus on time-sensitive patterns. By analyzing time-specific variations, such as changes in routine or environmental factors, this approach uncovers triggers that are not only unique to each patient but may also fluctuate based on the timing of exposure, helping to better understand and anticipate migraine onset. Various examples of the proposed concept may include a temporal alignment component that ensures that the latent representations and their associated concepts remain consistent and relevant as the data evolves during the training phase. Various examples of the proposed concept may include a contextual analysis module that assesses the relevance and relationships of features based on the data, application context, and expert knowledge. This module groups features into concepts and allows the integration of new concepts as new features are
introduced. Various examples of the proposed concept may perform dynamic group adjustment, in which machine learning clustering techniques are used to dynamically adjust the concept groups. For instance, if certain concepts become more relevant or change their relationships, the grouping algorithm updates the groupings accordingly. TRACE enhances encoding and decoding accuracy through temporal alignment and contextual embeddings. The proposed method clusters features into concepts to allow the autoencoder to generate relevant feature embeddings. Additionally, it preserves the temporal nature of the input data, ensuring that latent representations and their associated concepts remain consistent and relevant as the data evolves during training. This innovative approach allows for more accurate and contextually aware modeling of complex, time-dependent data. TRACE can handle a variety of features, e.g., headache-related features, lifestyle factors, medications, medical exam results (such as thyroid function tests, Blood pressure readings, blood sugar levels, etc.), features extracted from medical imaging, and features collected from sensors. TRACE groups features into concepts and then generates embeddings for these concepts, rather than for the raw features. This approach enhances the embeddings, enabling them to learn more useful patterns from relevant features. Previous research, on the other hand, focuses on limited set of features and use conventional machine learning models to forecast migraine attacks. Existing concept bottleneck (CBM) models in the literature first infer an intermediate set of human-defined concepts, which are then used to predict the final output in an interactive manner. These models have been applied to image data, such as knee x-ray images (see Koh, Prang Wei, et al. "Concept bottleneck models.") and medical time-series data where features are grouped into concepts that correspond to meaningful clinical ideas (see Wu, Carissa, et al. "Learning optimal summaries of clinical time-series with concept bottleneck models."), CBMs demonstrated improved performance and explainability. Further, CMBs allow users to directly modify concept predictions and have those changes propagated to the final prediction. This capability enables real-time corrections by experts, e.g., a
radiologist can correct mistakes in concept predictions, thereby enhancing user interaction and accuracy. However, such models lack the concept embedding and the temporal alignment components thus they can be less efficient (in terms of speed and accuracy) than TRACE (see Koh, Prang Wei, et al. "Concept bottleneck models" and Wu, Carissa, et al. "Learning optimal summaries of clinical time-series with concept bottleneck models.") The model generated using TRACE was benchmarked against LSTM, CNN, a hybrid LSTM-CNN model, and a stand-alone Random Forest. The evaluations are conducted on two datasets: the 2-day forecast and the 3-day forecast. The datasets were split based on user ID, ensuring that the training, validation, and test sets consisted of mutually exclusive users. The data was divided into 70%^for training and 30%^for validation and testing. Training and testing predictive models on different user sets is considered a challenging task, yet it is necessary for robust real-world applications. Ideally, it is beneficial to leverage user’s historical data to forecast their future headache episodes. However, when a user is new to the system or does not have enough historical records, then data from existing users can be leveraged to enhance migraine predictions. This strategy not only tests the model’s capacity to generalize across a variety of users but also strengthens its practicality, especially in real-world scenarios where new users are consistently joining the system or existing users lack sufficient historical records. The performance of each method is evaluated over 5^random runs using precision, recall, macro F1 score, weighted precision, weighted recall, weighted F1 score, and per-class recall (also known as true positive rate for each class). The Macro F1 score calculates the F1 score for each class independently and then averages them, treating all classes equally regardless of size. Macro F1 score is sensitive to minority class performance, making it ideal for ensuring balanced performance across all classes, especially for imbalanced datasets. The TRACE model is benchmarked with several models previously leveraged for EHR time series forecasting. These include LSTM, CNN, a hybrid LSTM-CNN
model, and a stand-alone Random Forest (see B. Shickel et al.: “Deep ehr: a survey of recent advances in deep learning techniques for electronic health record (ehr) analysis” and Francois Chollet. Deep learning with Python.). Here, a high-level overview of each of these competing methods is provided. A first benchmark method uses an LSTM (Long Short-Term Memory). LSTM is a type of Recurrent Neural Network (RNN) that is capable of learning long-term dependencies in sequence data. This is useful for time series forecasting, where the temporal dependencies are crucial for making accurate predictions. The LSTM cell consists of three main components: the forget gate (ft), the input gate (it), and the output gate (ot). These gates collectively decide how much information to keep from the past, how much current information to let in, and how much information to output to the next time step (see P. L Benitez et al: “An experimental review on´ deep learning architectures for time series forecasting”). The equations governing the LSTM operations are as follows: ^^ = ^^^^[ℎ^^^ + ^^] + ^^^ ^^ = ^(^^|ℎ^^^ + ^^| + ^^) ^^ = ^(^^[ℎ^^^ + ^^] + ^^) ^^ = tanh(^^[ℎ^^^ + ^^] + ^^) ^^ = ^^ ⊙ ^^^^ + ^^ ⊙ ^^ ℎ^ = ^^ ⊙ tanh(^^) Here, Xt is the input at time step t, ht−1 is the hidden state from the previous time step, and ct−1 is the cell state from the previous time step. The weights (W) and biases (b) are the model parameters learned during training. The σ function represents the sigmoid activation function, and the tanh function represents the hyperbolic tangent activation function. The operations are element-wise. The model trained using the Adam optimizer and is compiled with a categorical cross-entropy loss function:
N is the number of samples in the dataset. • M is the number of classes in the classification task. • yij is the true label of sample i for class j. It is 1 if the sample belongs to class j and 0 otherwise.
• y0ij is the predicted probability that sample i belongs to class j according to the model. In the context of time series forecasting, the LSTM model is trained on a sequence of historical data, and once trained, it can predict future values. The ability of LSTM to remember long-term dependencies makes it a popular choice for such tasks. A second benchmark method uses an CNN (Convolutional Neural Network). Additionally, a CNN architecture is employed as a benchmark model for the time- series forecasting task. The model begins with an input layer that takes a sequence of data with a specific shape (batch size,sequence length,in This is followed by convolutional layers that extract features from the input data using a convolution operation (see I. Koprinska et al.: “Convolutional neural networks for time series forecasting“):
where Xt represents the input at time step t, W is the filter (or kernel), b is the bias, f is the ReLU activation function: f(x) = max(0,x), and k is the size of the filter (or kernel). To reduce computational complexity, pooling layers are used to downsample the feature maps generated by the convolutional layers, often using max pooling. The output of these layers is then flattened into a vector that can be fed into fully connected (FC) layers. These layers learn complex patterns in the features extracted by the convolutional layers, using a ReLU activation function:
where Xflatten represents the flattened feature maps, WFC are the weights, bFC are the biases, and f is the ReLU activation function. The final predictions are produced by the output layer, which uses a softmax activation function:
where Ypred are the predicted probabilities for each class, Wout are the output layer weights, and bout are the output layer biases.
Similar to LSTM, this model is compiled with a categorical cross-entropy loss function and the Adam optimizer. A third benchmark method uses hybrid model including a LSTM and a CNN. The above two architectures are combined to create a hybrid LSTM-CNN model for benchmarking. Combining LSTM networks with CNN has emerged as a powerful approach for time series forecasting. LSTMs are well-suited for capturing long-term dependencies in sequential data, while CNNs excel at capturing local patterns. By combining these two architectures, the model can effectively capture both short- term and long-term dependencies in the time series data. Where, the LSTM layer is first used to learn temporal dependencies from the input data, and the output of the LSTM layer is fed into the CNN layers to capture spatial patterns. This combined architecture allows for more robust feature extraction and has been shown to outperform traditional models in various time series forecasting tasks (see C. Lin et al.: “Early diagnosis and prediction of sepsis shock by combining static and dynamic information using convolutional-lstm”, K. Li et al.: “Convolutional recurrent neural networks for glucose prediction” and G. Harerimana et al.: “Hsga: A hybrid lstm-cnn self-guided attention to predict the future diagnosis from discharge narratives”). A third benchmark method uses a random forest. Random forest is a powerful machine learning algorithm widely used for time series forecasting in various domains, including Electronic Health Records (EHR). By leveraging the ensemble of decision trees, random forest models can capture complex temporal patterns and relationships within the patient data, providing accurate forecasts for healthcare providers to make informed decisions (see S. Wongvibulsin et al.: “Clinical risk prediction with random forests for survival, longitudinal, and multivariate (rf-slam) data analysis”, J. Wang et al.: “. Risk assessment of coronary heart disease based on cloud-random forest” and M. J. Kane et al.: “Comparison of arima and random forest time series models for prediction of avian influenza h5n1 outbreaks”). LSTM, LSTM-CNN, and TRACE models were implemented using Python version 3.11.5, Tensorflow version 2.15.0, and a GeForce RTX 2080^GPU model. Different sets of hyper-parameters (e.g., embedding dimension, convolution kernel sizes,
LSTM hidden units, batch size, etc.) were explored using random search. For each of these models, the maximum epoch number is set to 50. Finally, the following early stopping strategy was implemented during training: if the validation recall did not improve over 10^epochs (i.e.,patience parameter), the training process was halted. All competing models leveraged weighted training to address the issue of class imbalance. Figs. 8a and 8b compare the per-class recall across five distinct runs for competing methods using 2-day forecast and 3-day forecast datasets, each initiated with a unique random seed. In these datasets, the models leverage relevant migraine features from the preceding 2 or 3 days, to predict migraine attacks the following day. The average migraine recall rates for the 2-day forecast dataset are as follows: 0.22 for Random Forest, 0.54 for CNN, 0.58 for LSTM, 0.60 for LSTM-CNN, and 0.67 for TRACE. For the 3-day forecast dataset, the average values are: 0.19 for Random Forest, 0.62 for CNN, 0.58 for LSTM, 0.54 for LSTM-CNN, and 0.68 for TRACE. In both datasets, the TRACE achieves the highest migraine recall of almost 70%, indicating that nearly 70% of all relevant migraine attacks were correctly forecasted. However, it is crucial to note the trade-off between forecasting migraines and no headache days. For example, the Random Forest model demonstrates the highest recall for headache-free days but the lowest recall for migraine episodes. Migraine recall was prioritized due to the potentially higher costs associated with unforeseen migraine attacks. For example, migraine sufferers can mitigate the severity of an attack by adhering to specific routines, such as ensuring adequate sleep, following a particular diet, and avoiding excessive alcohol consumption, which is known to exacerbate headache frequency and intensity. Figs. 9a, 9b and 9c present a comparative analysis of three key performance metrics: precision, recall, and macro F1 score. Figs. 9a and 9b show precision, recall, and macro F1 score values across five different runs for the competing models: Random Forest, CNN, LSTM, LSTM-CNN hybrid, and TRACE. The evaluation is performed using two datasets: 2-day forecast and 3-day forecast. Fig. 9c also shows precision, recall, and macro F1 score values across five different runs for the competing models: Random Forest, CNN, LSTM, LSTM-CNN hybrid, and
TRACE. The evaluation is performed using two datasets: 2-day forecast and 3-day forecast. Fig. 10 shows the corresponding weighted key performance metrics: precision, recall, and F1 score values across five different runs for the competing models: Random Forest, CNN, LSTM, LSTM-CNN hybrid, and TRACE. The evaluation is performed using two datasets: 2-day forecast and 3-day forecast. Similarly to Figs. 9a, 9b and 9c, the performance metrics are computed over 5 distinct runs on 2-day and 3day forecast datasets. All competing methods demonstrate comparable precision and F1 scores, with TRACE achieving the highest recall rates for both datasets. One metric being used is the macro-averaged F1 score. When calculating the macro-averaged F1 score across multiple classes, the F1 score is computed for each class c, then average these scores. It is defined as F1_macro
Where N is the total number of classes and ^1^ is the F1 score for class c. In scikit-learn, ‘f1 score‘ with ‘average=’macro’‘ implements this macro-averaging process automatically. It computes F1 c for each class and then averages these scores to give you ‘f1 test macro‘, the macro-averaged F1 score across all classes. Using this metric ensures that all classes are treated equally, which is especially valuable when assessing minority class performance. Another metric being used is the weighted precision score. The precision score for each class is computed and then averaged by the support, i.e., the number of true instances for each label.
where ^^^^^^^^^^ is the precision of class i and ^^^^^^^^ is the number of true instances of class i.
Another metric being used is the weighted recall score. The recall score for each class is computed and then averaged by the support, i.e., the number of true instances for each label.
where ^^^^^^^ is the recall of class i and ^^^^^^^^ is the number of true instances of class i. Another metric being used is the weighted F1 score. The function f1score from sklearn.metrics is used to compute the F1 score. For weighted F1 score, the parameter average=’weighted’ is added, which calculates the F1 score for each class individually and then computes the weighted average of these scores based on the support (the number of true instances for each class), see scikit-learn contributors. scikit-learn: Machine Learning in Python, 2022 .
where ^^^^^^^^^^ is the precision for class c, ^^^^^^^ is the recall for class c. The weighted average F1 score is as:
where ^^^^^^^^ is the number of true instances for class c. Another metric being used is the per-class recall score. Considering “no headache” as the negative class and “migraine” as the positive class, the per-class recall is computed as follows. For the ”no headache” class (negative class):
For the ”migraine” class:
The table shown in Fig.11 summarizes the average training time in minutes for each of the competing model and the standard deviation in minutes across 5 runs for each
model. In the 2-day forecast dataset, the LSTM-CNN model had the longest training time with significant variance. Individual runs showed that the training time ranged from 67 to 230 minutes. However, this model exhibited less variance in training time for the 3-day forecast. It is assumed that this is because the 3-day forecast dataset includes more historical data, which may provide a more stable training process. The training time for TRACE includes the total duration needed to generate the concept embeddings and to train the random forest model on these embeddings. The model operates in parallel on each concept to create the corresponding embeddings. While training the random forest typically takes less than five minutes on average, generating the concept embeddings is a more time-consuming step. Consequently, TRACE generally requires more training time compared to other models. Lastly, Fig. 12 provides a global overview of feature importance derived from the Random Forest model leveraged in the TRACE algorithm across 5 runs on the 3-day forecast dataset. To understand the importance of the different concepts, an ablation study was conducted to evaluate the performance of the TRACE method when each concept is considered individually. The goal is to determine the impact of each feature concept on the overall performance. Fig. 13 shows an ablation analysis illustrating the recall per class for individual feature concepts and their combination using the 3-day forecast dataset acrossfive different runs. Considering each concept, the health factor concept, denoted as Chealthfactor, achieves the highest recall for the no-headache class, while, the medication concept, denoted as Cmedication, provides the highest recall for the migraine class. While each concept individually demonstrates acceptable performance, their combination results in an enhanced recall for the migraine class. Fig. 14 shows precision, recall, macro F1 score, and the corresponding weighted values across five different runs for individual feature concepts and their combination using the 3- day forecast dataset. Some examples of the proposed concept provide personalized explanations using SHAP. In healthcare, machine learning model interpretability and transparency are
beneficial. Knowing why a model makes a specific prediction can directly impact patient care, regulatory compliance, and trust in AI-driven systems. Using SHAP for explainable AI (XAI) methods allows to pinpoint specific triggers at the patient level, supporting precision medicine initiatives. For instance, SHAP values can reveal individual sensitivities, such as a patient’s increased likelihood of migraines following caffeine consumption, enabling tailored recommendations based on personal triggers. One of the key challenges to tackle is model explainability to ensure safe model deployment and build user trust. Consequently, various eXplanation Artificial Intelligence (XAI) techniques have been proposed to gain insights and provide local and global explanations for models behavior. In Duell et al.: “A Comparison of Explanations Given by Explainable Artificial Intelligence Methods on Analysing Electronic Health Records”, the researchers evaluated three XAI techniques for analyzing EHRs: SHapley Additive exPlanations (SHAP) (see also Lundberg et al.: “A Unified Approach to Interpreting Model Predictions”), Local Interpretable Model- agnostic Explanations (LIME) (Ribeiro et al.: “Why Should I Trust You. Explaining the Predictions of Any Classifier), and Scoped Rules (see Ribeiro et al.: “Anchors: High-Precision Model-Agnostic Explanations). These techniques are feature attribution methods that assign weights to the features used for model predictions. Their benchmark focused on the ability of these methods to provide both local and global explanations, supporting human reasoning and improving trust in AI models. According to their analysis, SHAP was considered the most effective XAI method, offering clarity in communicating both local and global explanations. Further, SHAP visualizations effectively supported clinicians’ needs by providing reasoning and enhancing trust in human-AI collaboration. In Wu et al.: “Learning Optimal Summaries of Clinical Time-series with Concept Bottleneck Models, a different XAI approach was introduced. The researchers aimed to facilitate human inspection of clinical time-series prediction models by learning ‘concepts’ that group features into high-level categories. They proposed an optimization method to select the most important features within each concept, resulting in the learning of sparse prediction models, i.e., models that have a limited number of features. These models strike a balance between complexity and
interpretability, allowing them to be effectively analyzed and understood by human experts, which is crucial in clinical settings where decisions based on machine learning need to be justified and understood by domain experts. In the following, the LSTM model is leveraged to generate SHAP per-user insights of migraine trigger factors. SHAP calculates feature importance scores by considering the contribution of each feature to the model’s output. It assigns each feature an importance score by computing the average marginal contribution of that feature across all possible combination of features. It ensures a consistent and interpretable measure of feature impact based on game theory’s Shapley values. Figs. 15a to 15j display SHAP summary plots of the top 10 features influencing migraine occurrences for 10 users who consistently used the app over an extended period. In Fig. 15a, the SHAP analysis indicates that higher medication usage is associated with a reduced likelihood of migraine occurrence. Furthermore, reduced sleep duration and quality increase the chances of a migraine attack for this user. In Figure 15b, the SHAP analysis reveals that caffeine overconsumption on days -2 and -3, coupled with reduced caffeine intake on day -1, is linked to a higher likelihood of experiencing migraines for this user, similar pattern is observed in other patients, see Figs.15c, 15e, 15f, 15g and 15j. Figs. 15a to 15j show SHAP summary plots illustrating the top 10 features and their impact on predicting migraine occurrences among users who consistently used the app over an extended period. In thisfigure, (day -i) refers to the i’th day preceding a migraine attack. The user shown in Fig.15a actively engaged with the M-sense mobile app from April 13, 2017, to October 08, 2022, contributing a total of 1871 daily records. Analysis of SHAP top features revealed that medication use ranked prominently among the most influential factors in preventing migraine occurrences. Increased stress, alcohol consumption, and reduced sleep duration and quality on the day preceding the attack contribute to a higher likelihood of migraine occurrence for this user. The user shown in Fig.15b actively engaged with the M-sense mobile app from January 27, 2018, to October 02, 2022, contributing a total of 1687 daily records. An analysis
of SHAP top feature lists revealed that variations in normalized caffeine consumption are among the most influential factors in the occurrence of migraine attacks for this user. Additionally, temperature changes in the two days preceding a migraine attack significantly impact its onset. Furthermore, a decline in normalized sleep duration on the day immediately prior to the attack indicates a higher likelihood of experiencing a migraine. The user shown in Fig. 15c actively engaged with the M-sense mobile app from June 25, 2018, to March 02, 2022, contributing a total of 1287 daily records. The top features contributing to the likelihood of migraine occurrence include caffeine, alcohol, stress, and period (i.e., menstrual cycle). The user shown in Fig. 15d actively engaged with the M-sense mobile app from January 27, 2018, to September 13, 2022, contributing a total of 1529 daily records. The top features contributing to the likelihood of migraine occurrence include medication, caffeine, temperature, and stress. The user shown in Fig. 15e actively engaged with the M-sense mobile app from November 08, 2018, to October 05, 2022, contributing a total of 1294 daily records. The top features contributing to the likelihood of migraine occurrence include stress, caffeine, medication, and alcohol consumption. The user shown in Fig.15f actively engaged with the M-sense mobile app from January 11, 2019, to August 11, 2022, contributing a total of 1179 daily records. The top features contributing to the likelihood of migraine occurrence include stress, caffeine, and sleep (both quality and duration). The user shown in Fig. 15g actively engaged with the M-sense mobile app from November 11, 2018, to September 30, 2021, contributing a total of 978 daily records. The top features contributing to the likelihood of migraine occurrence include stress, caffeine, alcohol, and temperature. The user shown in Fig. 15h actively engaged with the M-sensemobile app from April 30, 2018, to October 09, 2022, contributing a total of 1394 daily records. The top features contributing to the likelihood of migraine occurrence include smoking, caffeine, temperature, and medication. The user shown in Fig. 15i actively engaged with the M-sensemobile app from July 18, 2017, to October 09, 2022, contributing a total of 1815 daily records. The top features contributing to the likelihood of migraine occurrence include stress, caffeine, medication, and sleep duration. The user shown in Fig.15j
actively engaged with the M-sensemobile app from January 28, 2018, to September 11, 2021, contributing a total of 1297 daily records. The top features contributing to the likelihood of migraine occurrence include alcohol, caffeine, sleep duration, and stress. The observations indicate that higher temperatures on the day before a migraine increase the likelihood of its occurrence. In contrast, lower temperatures two days prior seem to have a similar effect, see Figs.15a, 15b, 15d and 15f. Similar findings were reported in previous research, which investigated how both temperature levels and fluctuations influence the probability of a migraine attack for some patients. Fig. 16 shows another illustration of an analysis of feature importance for a specific user. The example shown in Fig. 16 shows that reduced caffeine consumption one day before a migraine attack (day -1) is associated with a higher likelihood of migraine, while reducing caffeine two days prior (day -2) appears to be associated with no migraine. This suggests that variations in caffeine intake may contribute to migraine onset. Additionally, increased stress levels are observed to heighten the likelihood of migraine occurrence for this user. Stress was the most prevalent trigger for migraines in the dataset, with elevated stress levels significantly increasing the likelihood of a migraine the following day, consistent with prior research. Sleep duration and/or quality emerged as another common trigger in the dataset, with shorter sleep periods and lower sleep quality significantly increasing the likelihood of a migraine the next day. Furthermore, the analysis reveals that, although common migraine triggers exist, individual triggers can vary significantly, for example, patient-specific triggers were observed, such as period (i.e., menstruation) in Fig. 15c, smoking in Fig. 15h, and excessive alcohol consumption in Fig.15j. The study demonstrates the effectiveness of TRACE in forecasting migraine attacks, achieving approximately 70% accuracy in 2-day and 3-day forecast datasets. This predictive capability significantly enhances the potential for proactive migraine management, which can greatly improve the quality of life for sufferers. By leveraging a comprehensive dataset and employing both global and local feature
importance analyses, critical predictive factors were identified and the variability of migraine triggers for different patients was underscored. TRACE may be refined to further improve its performance and adaptability across diverse patient datasets, such as including more diverse demographics, sensor data, and clinical variables such as brain imaging. A key direction involves conducting an in-depth analysis of complex feature engineering techniques to better understand the correlations and causations between features and migraine onset. Additionally, integrating real-time data collection and feedback mechanisms could enhance the model’s responsiveness and accuracy, ultimately contributing to more effective and tailored treatment strategies. The present invention may be implemented as a computer-implemented method, computer system (comprising one or more processors and one or more storage devices) configured to perform the computer-implemented method and/or as a computer program for performing the computer-implemented method. For example, the computer-implemented method may include one or more steps and/or operations discussed above. Various aspects of the present invention relates to machine learning. In particular, the model mentioned above may be a machine learning model. Machine learning is a branch of artificial intelligence that involves the development of algorithms and models that allow computers to learn and make predictions or decisions without being explicitly programmed. It focuses on creating systems that can improve their performance over time by learning from data. Training a machine-learning model refers to the process of teaching the model to make accurate predictions or decisions. During training, the model is exposed to a large amount of data, which is used to adjust the model's internal parameters or weights. The model learns patterns, relationships, or rules from the training data, allowing it to generalize and make predictions on new, unseen data.
Training data is the set of examples or instances that is used to teach a machine- learning model. It is often labeled data, meaning that each example is associated with a known outcome or target value. The training data consists of both input features and the corresponding output or target variable. The model learns from this data by analyzing the patterns and relationships between the input features and the target variable. Training algorithms, such as supervised learning, semi-supervised learning, unsupervised learning or reinforcement learning may be used for training the machine-learning model. Machine-learning models, such as the machine-learning model being trained in the present disclosure, are often implemented as Artificial Neural Networks (ANNs), and in particular Deep Neural Networks, Support Vector Machines, Decision Tree models, or Random Forest models. Examples may involve or relate to computer programs, including program codes to execute one or more of the mentioned methods when the program is executed on a computer, processor, or other programmable hardware component. As a result, steps, operations, or processes from various methods described above can also be executed by computers, processors, or other programmable hardware components. Examples may additionally cover program storage devices, such as digital data storage media, which are machine-, processor-, or computer-readable and encode and/or contain machine-executable, processor-executable, or computer-executable programs and instructions. These devices may include or be digital storage devices, magnetic storage media like magnetic disks and tapes, hard disk drives, or optically readable digital data storage media, for instance. Other examples encompass computers, processors, control units, field programmable logic arrays (FPLAs), field programmable gate arrays (FPGAs), graphics processing units (GPUs), application- specific integrated circuits (ASICs), integrated circuits (ICs), or system-on-a-chip (SoC) systems that are programmed to carry out the steps of the aforementioned methods. In simpler terms, examples may involve computer programs and storage media comprising computer programs, as well as hardware components like processors and control units, which can be programmed to execute the methods described above.
When certain aspects are mentioned in relation to a device or system, they should also be considered as descriptions of the corresponding methods. For example, a block, component, or functional aspect of the device or system may correspond to a method step or feature of the related method. Therefore, aspects described regarding a method should also be understood as depicting a corresponding element, property, or functional feature of the corresponding device or system. In simpler terms, if something is described in relation to a device or system, it can also be applied to the corresponding method, and vice versa. Many modifications and other embodiments of the invention set forth herein will come to mind to the one skilled in the art to which the invention pertains having the benefit of the teachings presented in the foregoing description and the associated drawings. Therefore, it is to be understood that the invention is not to be limited to the specific embodiments disclosed and that modifications and other embodiments are intended to be included within the scope of the appended claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.
L i s t o f r e f e r e n c e s i g n s 10, 20 System 12, 22 Interface circuitry 14, 24 Processor circuitry 16, 26 Memory/storage circuitry 110 Obtaining input data 120 Dividing features into groups 130 Encoding groups of features into embeddings 140 Concatenating embeddings 150 Inputting embeddings into prediction machine learning model 160 Providing a prediction 170 Providing an explanation 210 Obtaining training data with samples of input data 220 Determining a mapping between features and groups of features 230 Dividing the features of the samples of input data into the groups of features 240 Training an autoencoder (including an encoder ML model) 250 Concatenating embeddings 260 Training a prediction ML model 270 Providing the ML model(s) 280 Computing/providing Shapley values