WO2025212992A1

WO2025212992A1 - System and method of categorizing public biomedical databases using large language models

Info

Publication number: WO2025212992A1
Application number: PCT/US2025/023111
Authority: WO
Inventors: Kaitlyn GAYVERT; Wei Keat Lim; Joseph Cooper DEVLIN
Original assignee: Regeneron Pharmaceuticals Inc
Current assignee: Regeneron Pharmaceuticals Inc
Priority date: 2024-04-05
Filing date: 2025-04-04
Publication date: 2025-10-09
Anticipated expiration: 2026-10-05

Abstract

A computer-implemented method for deriving study types from a dataset is disclosed. The method includes: (i) tokenizing extracted text from a plurality of metadata of a dataset to identify a plurality of tokens; (ii) generating a first numerical representation of each token of the plurality of tokens; (iii) partitioning the plurality of metadata into a plurality of clusters according to a predetermined criterion; (iv) for each cluster of the plurality of clusters, identifying a subset of tokens of the plurality of tokens that are common to all metadata of that cluster; and (vi) for each identified subset of tokens corresponding to each cluster of the plurality of clusters: (a) generating a second numerical representation of each token of that subset of tokens; and (b) based on the second numerical representation of each token, generating a respective study type corresponding to each cluster of the plurality of clusters.

Description

SYSTEM AND METHOD OF CATEGORIZING PUBLIC BIOMEDICAL DATABASES USING LARGE LANGUAGE MODELS

TECHNICAL FIELD

[0001] This disclosure relates generally to systems and methods of categorizing public biomedical databases using large language models (LLMs).

BACKGROUND

[0002] In the recent times, biomedical data has been used in the realm of research, for example, for providing unbiased and profound insights into the intricate mechanisms of biological systems. The norm of sharing biomedical datasets in conjunction with publications has fostered an environment of transparency and cooperative work within the scientific community. With the increasing affordability of these technologies, there has been a corresponding surge in the amount of data generated. This has led to a significant growth of public data repositories that host the shared data, thereby providing a rich resource for further discovery'. One popular forum for sharing processed genomic data is Gene Expression Omnibus (GEO), which includes data collected from hundreds of thousands of studies and millions of samples. Each submission to the GEO generally includes precomputed gene expression levels or signatures, along with a reference to the Sequence Reads Archive (SRA), another data repository' where raw genomic data is often stored and shared. However, these databases often lack detailed annotations. Accordingly, utilizing the available data efficiently is challenging.

[0003] Additionally, biomedical databases have continued to grow exponentially in recent years, especially as journals have begun mandating data sharing with publications. Each year, millions of new publications are added into PubMed* and tens of thousands of new studies are added into GEO and SRA (FIG. 8A). Additionally, millions of samples are added each year associated with those studies. The modalities covered in the genomic databases have also changed over time w ith the development of new' technologies, with high throughput sequencing-based methods overtaking microarray -based methods in GEO over the past few years (FIG. 8B). There are some efforts to develop curated subsets of these databases, for example GEO’s DataSets collections. However, this manual process is time consuming and as a result, these curated collections only cover a fraction of possible entries, with GEO DataSets covering less than 2% of GEO Series.

[0004] While the challenges posed by the metadata in the free-form text format to utilize the available data efficiently are addressed using the natural language processing (NLP) methods, the NLP methods are focused on processing and making sense of text, with applications performing a variety of tasks. The variety of tasks include topic modeling, named entity recognition (NER), and text generation (e.g., summarization, question and answering). Before any of these tasks can be performed, the free-form text format data is converted into numerical representations using a method based on term frequency approaches. In one example, the method generally known as a bag-of-words model is used in which the number of times each word occurs is tallied as term frequencies (TF). and term frequency-inverse document frequency (TF-IDF), in which the frequencies are weighted by an importance metric, is generated. Further, large language models (LLMs) utilizing deep neural networks trained or configured to learn based on large sets of text corpora are used for sophisticated and semantic representations of text as word or sentence embeddings. However, the LLMs get influenced by the text corpora used for training the neural networks or the domain-specific text for applications in, for example, the biomedical domain. BioSentV ec (Zhang 2019), PubMedBERT (Gu 2021), and BioGPT (Luo 2022) are such examples of LLMs that have been trained on biomedical corpora, including PubMed® abstracts. PubMedCentral® articles, and clinical trial notes.

[0005] Accordingly, a system and a method of categorizing public biomedical databases, including annotated datasets, to automatically extract study types, are needed to address the above-mentioned drawbacks associated with currently available public gene expression datasets and methods for automatic derivation of gene expression study types.

BRIEF DESCRIPTION

[0006] In one aspect, a computer-implemented method for deriving study types from a dataset is disclosed. The dataset includes a set of documents, wherein each document includes text data and metadata associated with the text data, such that the dataset includes a plurality of metadata. The computer-implemented method includes: (i) extracting, by a processor of a computing device, text from the plurality of metadata; (ii) tokenizing, by the processor, the extracted text to identify a plurality’ of tokens; (iii) generating, by the processor, a first numerical representation of each token of the plurality of tokens, the first numerical representation including a numerical representation of the importance of that token to the set of documents, to generate a plurality' of first numerical representations; (iv) partitioning, by the processor and based on the first numerical representation of each token of the plurality of tokens, the plurality of metadata into a plurality of clusters according to a predetermined criterion; (v) for each cluster of the plurality of clusters, identifying, by the processor, a subset of tokens of the plurality of tokens that are common to all metadata of that cluster; and (vi) for each identified subset of tokens corresponding to each cluster of the plurality of clusters: (a) generating, by the processor, a second numerical representation of each token of that subset of tokens, the second numerical representation including a numerical vector representation of that token corresponding with a measure of semantic similarity’ of that token to another token in that subset of tokens; and (b) based on the second numerical representation of each token of the subset of tokens, generating, by the processor, a respective study type corresponding to each cluster of the plurality' of clusters.

[0007] In another aspect, a system for deriving study types from a dataset is disclosed. The dataset includes a set of documents, wherein each document includes text data and metadata associated with the text data, such that the dataset includes a plurality of metadata. The system includes at least one memory configured to store instructions, and at least one processor configured to execute the stored instructions, which when executed cause the at least one processor to perform operations including (i) extracting text from the plurality of metadata; (ii) tokenizing the extracted text to identify a plurality of tokens; (iii) generating a first numerical representation of each token of the plurality of tokens; (iv) partitioning, based on the first numerical representation of each token of the plurality of tokens, the plurality of metadata into a plurality' of clusters according to a predetermined criterion; (v) for each cluster of the plurality of clusters, identifying a subset of tokens of the plurality of tokens that are common to all metadata of that cluster; and (vi) for each identified subset of tokens corresponding to each cluster of the plurality of clusters: (a) generating a second numerical representation of each token of that subset of tokens, the second numerical representation including a numerical vector representation of that token corresponding with a measure of semantic similarity of that token to another token in that subset of tokens; and (b) based on the second numerical representation of each token of the subset of tokens, generating a respective study type corresponding to each cluster of the plurality of clusters.

[0008] In yet another aspect, a non-transitory computer-readable medium (CRM) embodying programmed instructions is disclosed. The instructions, when executed by at least one processor of a backend system, cause the at least one processor to derive study types from a dataset including a set of documents, wherein each document includes text data and metadata associated with the text data, such that the dataset includes a plurality of metadata, by performing operations including (i) extracting text from the plurality of metadata; (ii) tokenizing the extracted text to identify a plurality' of tokens; (iii) generating a first numerical representation of each token of the plurality of tokens; (iv) partitioning, based on the first numerical representation of each token of the plurality of tokens, the plurality of metadata into a plurality of clusters according to a predetermined criterion; (v) for each cluster of the plurality of clusters, identifying a subset of tokens of the plurality of tokens that are common to all metadata of that cluster; and (vi) for each identified subset of tokens corresponding to each cluster of the plurality of clusters: (a) generating a second numerical representation of each token of that subset of tokens, the second numerical representation including a numerical vector representation of that token corresponding with a measure of semantic similarity of that token to another token in that subset of tokens; and (b) based on the second numerical representation of each token of the subset of tokens, generating a respective study type corresponding to each cluster of the plurality of clusters.

BRIEF DESCRIPTION OF THE DRAWINGS

[0009] FIG. 1 illustrates a schematic diagram of the NLP data extraction pipeline and the NLP annotation pipeline of HyDRA modeling.

[0010] FIG. 1 con’t illustrates a continuation of the schematic diagram of FIG. 1.

[0011] FIG. 2 A illustrates a chart corresponding to TF-IDF of top keywords found in study descriptions of disease, drug, and gene perturbation studies.

[0012] FIG. 2B illustrates a chart corresponding frequency of disease, drug, and gene terms identified by NER models in disease, drug, and gene perturbation studies. [0013] FIG. 2C illustrates a chart corresponding to top cluster keywords found in disease, drug, and gene studies and their frequencies.

[0014] FIG. 2D illustrates a dimensional graph representation of TF-IDF representations of study descriptions for dataset categories, such as, diseases, drugs, and genes.

[0015] FIG. 2E illustrates a dimensional graph representation of embeddings representation of sample keywords, for study categories, such as, diseases, drugs, and genes.

[0016] FIG. 2F illustrates a dimensional graph representation of TF-IDF representations of sample keywords for all disease studies according to case and control labels.

[0017] FIG. 2G illustrates a dimensional graph representation of embeddings representation of sample keywords for all disease studies according to case and control labels.

[0018] FIG. 3 A illustrates a graph showing a receiver operator characteristics (ROC) curve for the disease BOA model.

[0019] FIG. 3B illustrates a comparison chart showing a comparison of repeated 10-fold cross validation of HyDRA BOA (study type) and CoBRA (case/control) models, assessed using area under the ROC curve (AUC).

[0020] FIG. 3C illustrates a bar plot of feature importance for Naive Bayes model for the disease BOA TF-IDF submodel measured by the inverse log odds for each class.

[0021] FIG. 3D illustrates adimensional graph representation of embedding space of all words found in the CoBRA TF-IDF submodels for disease, drug and gene based upon case and control labels. [0022] FTG. 4A illustrates a bar plot comparing a frequency of high-level disease concepts found in two independent manually curated databases of gene expression studies: CREEDS and Array Studio.

[0023] FTG. 4B illustrates a bar plot showing area under the ROC curves (AUC) for the prediction of disease and drug perturbation study type labels in the independent Array Studio curated dataset.

[0024] FIG. 4C illustrates a graph corresponding to an ROC curve for case/control prediction in curated disease vs healthy studies in the independent ArrayStudio dataset.

[0025] FIG. 4D illustrates a violin plot of accuracy per data when predicting case/control in studies with ‘Previously Observed’ and ‘New’ disease terms.

[0026] FIG. 5 A illustrates a diagram showing a bar plot of a number of GEO human studies and a number of mouse studies.

[0027] FIG. 5B illustrates a chart showing a frequency of high-level disease concepts in human and mouse studies.

[0028] FIG. 5C illustrates a bar plot corresponding to percentage of descendant terms that correspond to high-level disease concepts.

[0029] FIG. 6 illustrates an example flow-chart of method operations according to an example embodiment of the present disclosure.

[0030] FIG. 7 illustrates an example block diagram of a computing device performing method operations described in FIG. 6.

[0031] FIG. 8A illustrates the number of new publications (top), or genomic datasets added to the Sequence Reads Archive (SRA)(middle) and Gene Expression Omnibus (GEO) (bottom) each year. [0032] FTG. 8B illustrates the number of new genomic datasets by platform added to all of GEO (top), human GEO datasets (middle) and mouse GEO datasets (bottom) each year.

[0033] FTG. 8C illustrates an example of the type of input the Annotation of Dataset-Derived Entities and their Resolved concepts (ADDER) model expects and what ty pes of annotations the ADDER model provides.

[0034] FIG. 8D illustrates the number of new human genomic datasets added each year to GEO, broken down by disease area.

[0035] FIG. 8E illustrates examples of the three major types of experimental designs predicted by an embodiment of the HyDRA model of the present disclosure.

[0036] FIG. 9A illustrates a chart showing CRowd Extracted Expression of Differential Signatures (CREEDS) automatic predictions as compared to HyDRA predictions for disease, drug and gene perturbation studies.

[0037] FIG. 9B illustrates a chart of studies overlapped with ArrayStudio and CREEDS automatic predictions.

DETAILED DESCRIPTION OF THE DISCLOSURE

[0038] The following detailed description illustrates embodiments of the disclosure by way of example and not by way of limitation. It is contemplated that the disclosure has general application to annotation and derivation of study types .

[0039] NLP methods are used for querying and utilizing large public biomedical databases. For example, case and control groups are identified according to the text descriptions of the samples in previous studies using ExpressionBLAST (Zinman 2013) and CREEDS (Wang 2016). ExpressionBLAST applied rule-based pattern matching. CREEDS (Wang 2016) applied NLP methods using TF-IDF representations to train machine learning models for the prediction of case or control groups and study types. These pretrained models were subsequently applied to all 31,905 human, mouse, and rat microarrays studies in the GEO at the time of the study. However due to the challenges of predicting study type, using the models thus trained, only high-quality predictions for less than one percent of tested studies.

[0040] The LLMs are used for querying of large text corpora in the biomedical domain for searching through and annotating large collections of publications and clinical notes. NER training datasets and models are released by NCBI and EMBO for biomedical-related tasks such as identifying diseases, tissues, cell types, and gene names from free-form text. Commercial libraries, such as John Snow Labs, have scaled this up with larger datasets and collections of hundreds of pre-trained biomedical-focused models that perform with high accuracy. However, no methods or techniques are available for categorizing the public genomic repositories.

[0041] Accordingly, various embodiments are described in the present disclosure for using a Hybrid NLP approach to Deriving Repository' Annotations (HyDRA). which utilizes both TF-IDF and LLM representations of dataset and sample level metadata to construct models for the Annotation of Dataset-Derived Entities and their Resolved concepts (ADDER), Binning of Analysis ty pe (BOA), and Control or Case prediction By Respective Analysis type (CoBRA). In other words, NLP data extraction includes extracting dataset metadata and deriving sample cluster features and text-based features, and NLP annotation includes tasks such as ADDER, BOA, and CoBRA model construction or generation.

[0042] In some embodiments, for example, to facilitate efficient annotation and querying of public genomic data repositories, such as GEO, pre-trained LLM-based NER models are applied to extract annotations of key entities (disease, tissue, cell type, gene name) from free-form text, such as study descriptions or publication abstracts (See FIG. 8C). Resolver models are then trained to map these terms to controlled vocabulary which are linked to established ontologies, such as the human disease ontology knowledgebase (DO- KB) for disease terms and the uber-anatomy ontology (UBERON) for cell types.

[0043] Further, when the ADDER model, constructed based on NLP annotation, as described herein, was applied to a manually annotated collection of 542 disease, 337 single drug perturbation, and 1,206 gene perturbation experiments, it has been found that these annotations are highly consistent with the manually annotated disease and cell type names as shown in Table 1 below.

[0044] Table 1

[0045] Additionally, when the ADDER model constructed based on NLP annotation, as described herein, is applied to the 56,521 human and 41 ,034 mouse studies available in the GEOmetadb database, it w as found that while a great diversity of terms w ere used to describe key study design components, these terms often captured similar concepts, as shown in Table 2 below.

[0046] Table 2: Unstructured and structured terms in gene expression databases.

[0047] For example, 18,351 terms were used across these studies to describe only 1,528 distinct disease concepts. The mapping of concepts onto ontologies also further enriches these annotations and enables similar terms to be linked to higher-order terms, such as disease families or anatomical organs. For example, under this approach the term “chronic kidney failure’' would be mapped to “chronic kidney disease” and can be further annotated under the broader umbrella of ‘'urinary system diseases” (FIG. 8C). The linking of these annotations to higher-order terms can enable more efficient querying and can also provide insights into trends in genomic research. For example, when all human GEO studies were collapsed to disease family terms, it was observed that more than half of studies focused on the oncology space (FIG. 8D).

[0048] In some embodiments, NLP -based methods as described herein may be used to annotate other information about studies, such as experimental design and the experimental groups within a study. Various objectives sought by experiments on the different types of studies curated in the CREEDS dataset include, for example, assessing the effects of perturbations (e.g., gene knockout, drug treatment), and characterization of different conditions (e.g., disease vs healthy).

[0049] Three types of experimental designs (study types) may include, but are not limited to, disease, drug, and gene perturbations. (FIG. 8E). When assessing a collection of 1,934 manually annotated studies, it was found that study descriptions include many general terms that are found across all study types and related to the study background (e.g., human, Affymetrix) and data type and analytical techniques (e.g., Affymetrix, gene expression, differentially expressed). Generally, the terms that are more distinct to each type of experimental design may not occur at high frequencies, or not generally appearing in more than half of studies. Additionally, the study descriptions include information that may be tangential to the study design. For example, disease and drug perturbation study descriptions often note genes that are not directly relevant to the experiment, such as a proposed disease driver or the target of the drug. Accordingly, when pre-trained NER models, as described herein, are applied to the CREEDS dataset, over half of disease and drug studies reference gene names and almost half of dataset-level descriptions from both drug (45%) and gene (45%) perturbation experiments may be found to reference disease names. Accordingly, study level information may not be sufficient to capture differences between study types.

[0050] As described herein, in some embodiments, key terms from study’s sample descriptions may be used to overcome this limitation. For each of these studies, sample descriptors may be clustered to identify groups of samples described by consistent and distinct terms. Such cluster-defining keywords are enriched for specific experimental design terms. In one example, a cluster-defining keyword “healthy” may be used to specify disease studies, and another cluster-defining keyword “knockout” may be used for gene perturbation studies.

[0051] In some embodiments. LLMs may be used to provide or obtain additional insights, as described herein. For example, based upon qualitative comparison of LLM embeddings with term-frequency representations by applying each technique to both the study-level and sample-level descriptions and visualizing using Uniform Manifold Approximation and Projection (UMAP) plots, it may be found that while the term-frequency representation of study descriptions used by conventional approaches demonstrated discriminative ability (UMAP_1 : H = 0.124, p =0.118, UMAP_2: H = 0.237, p = 7xl0’⁵), a much clearer separation between study types may be obtained while representing using LLM representations of sample keywords (UMAP_1 : H = 0.519, p =2x10'¹⁵, UMAP_2: H = 0.382, p = 2x10’¹⁵). Similarly, both term-frequency and LLM representations may generate visually separated case and control samples.

[0052] Based upon the results describes above, term-frequency and LLM representations both may provide meaningful representations of the text. Accordingly, a hybrid approach that utilizes both conventional term frequencies and LLM-derived embeddings may be used to classify and annotate these studies.

[0053] As described herein, a hybrid approach may utilize term frequencies (TF) and LLMs to classify dataset types with higher predictive power. In some embodiments, term-frequency and LLM-derived embedding representations of the sample keywords may be used to predict study types and experimental groups. A set of models for Binning of Analysis type (BOA) may be trained to predict study type: disease, drug, and/or gene perturbations. Additionally, models may be trained for each study type to predict case and control groups (Control or case prediction By Respective Analysis type (CoBRA)). Accordingly, for each approach and perturbation type, an ensemble model may be trained using two feature sets including TF-IDF representations of term-frequencies and LLM- derived embeddings, and five different training methods including naive bayes (NB), gradient boosting (GBC), neural net (NN), random forest (RF) and logistic regression (LR). The models described herein may then be combined using a soft voting procedure and evaluated using repeated 10-fold cross validation. [0054] In the example embodiments, both BOA and CoBRA models may demonstrate high discriminative power, accuracy and area under the receiver operator cun e (AUC) as described in Table 3 below. Accordingly, strong predictive performances may be observed for the disease (AUC: BOA = 0.92. CoBRA = 0.95), drug perturbation (AUC: BOA = 0.88, CoBRA = 0.92), and gene perturbation (AUC: BOA = 0.91, CoBRA = 0.90) models. Additionally, or alternatively, the predictive performance based on both BOA and CoBRA models may be comparable or outperform models that only utilized term frequencies.

[0055] Table 3

[0056] For each of the sub-models, the most important features affecting the model performance may be identified. For example, for the CoBRA disease sub-models trained on term-frequency representations, the most important features that are strongly associated with the healthy class may include classic control terms, such as “control,” “normal,” “healthy,” and “non.” Similarly, the terms associated with the case (disease) class may include disease-specific terms, such as “autism,” “carcinoma,” and “psoriasis,” which correspond to the most frequent disease types found in the training set and may reflect the biases of the model. This is important because it may limit the model’s ability to predict datasets with out-of-vocabulary (OOV) key terms, since term-frequency based models are limited to words that were observed in the training set. To better understand the LLM feature space, each of the features identified in the term-frequency analysis may be plotted along with the top vectors associated with the case and control group from each embeddings model in a UMAP plot. While the control group vectors are most closely related to each other, each of the case groups may be distinct and clustered closer to the top study -type specific terms that highlights the importance of training distinct CoBRA models based on study type classifications.

[0057] In some embodiments, HyDRA models may be generalized to an independent, manually curated set of 708 disease studies. The set of 708 disease studies may include many diseases that are not previously represented in the training dataset. The HyDRA models may perform well in both classifying study type (BOA AUC: Disease = 0.93, Drug = 0.85) and experimental groups (CoBRA AUC: Disease = 0.99, Drug = 0.95) when tested on new studies. Accordingly, the models are found to perform equally well on both diseases seen and not seen in the training set, when assessed using overall performance across studies and when assessing the percent of correctly annotated case/control samples within studies.

[0058] In one embodiment, while there was minimal overlap with the training set, the independent dataset overlapped with 80 studies with high-quality predictions released with a conventional approach (CREEDS). In these cases, the CREEDS and HyDRA models displayed identical, high accuracy (95%) in predicted disease studies, however HyDRA models more accurately identified drug perturbation experiments (86%) vs CREEDS (59%). (See FIGS. 9A and 9B).

[0059] In some embodiments, the HyDRA models may be generalizable to vocabulary unobserved by the models during training. In some embodiments, the HyDRA models may enable querying of thousands of annotated human and mouse GEO expression studies. In some embodiments, resources, such as dashboards, to facilitate searching through these annotations may be built based on the HYDRA models of the present disclosure. These dashboards may be used to search by disease and view summaries of what data is available. In one example, when HyDRA models are applied to the over 83,000 additional human and mouse GEO expression studies not represented in the manually curated training sets, roughly half of the studies may be predicted to fall in the three major study type categories. Notably, there were major differences between predicted types of human and mouse studies, with human studies largely predicted to be disease (n= 17,082) and drug perturbation (n=26,l 16) comparisons, whereas mouse studies were predominantly drug (n=14,869) and gene (n=15,543) perturbation comparisons with far fewer predicted disease (n=l,552) comparisons. A small subset of these studies, such as. 3% mouse, 4% human, may be predicted to match multiple study types, which is possible in studies with more complex designs.

[0060] In some embodiments, when comparing the diseases captured in GEO studies to the composition of the manually curated studies, using HyDRA models, 1,376 distinct new diseases, including 697 additional disease concepts in predicted disease vs healthy studies, may be identified. By way of a non-limiting example, these additional diseases may be captured across the different categories of diseases. Altogether, these results are consistent with the distinct roles for human and mouse studies in biomedical research and suggest that curated datasets are limited in their ability to fully capture and represent the large gene expression repositories that are available in the public domain.

[0061] Accordingly, the rapidly growing collection of public gene expression studies have become a valuable tool for researchers, while their use is limited due to the lack of structured information describing the studies.

Using the currently known methods to identify studies in the broader GEO database, in particular for highly studied diseases, is cumbersome. However, as described herein, in some embodiments, annotation of the studies with controlled key terms, such as disease and tissue types, may enable more efficient queries to be formulated, which ensures that relevant studies are not missed due to the use of synonyms. In some embodiments. HyDRA utilizes semantic representations of the study-associated text to annotate public gene expression studies with study type, sample labels, and other keywords. In some embodiments, HyDRA may be able to recapitulate manually annotated labels with high accuracy on two independent manually- curated datasets. Additionally, the approach described herein may be generalizable to other ty pes of biomedical databases, as well.

[0062] It has been observed that existing repositories of hand curated signatures tend to be biased towards older studies and well-studied phenotypes. For example, one of the manually curated datasets (CREEDS) consists entirely of microarray studies, and over 90% of ArrayStudio’s dataset is microarray studies. Additionally, over 50% of the ArrayStudio’s studies focuses on the same ten diseases. Accordingly, large automatically derived libraries may be useful to help screen against larger sets of signatures to help identify novel connections between signatures such as pathways, treatments, and diseases. However, approaches for automatically processing gene expression studies may not substitute manual derivation of signatures, HyDRA models may process gene expression studies with high accuracy, and may complement the manual curation efforts by supplementing existing repositories.

[0063] Strengths of the HyDRA models may include multiple layers of information, including annotations and model confidence scores. Additionally, using the LLMs enabled terms to be represented by their meaning, terms that are not in the training set may be included in new test cases. However, LLMs may be susceptible to bias that can be introduced by their training sets. Accordingly, pre-trained and fine-tuned LLMs on biomedical datasets may be used to avoid bias. Additionally, or alternatively, classifiers may be used instead of generative models to perform prediction tasks to not have the issue of hallucinations that models like those reported with LLM models.

[0064] In various aspects, the models disclosed herein offer a powerful tool for the annotation and classification of biomedical datasets. In various aspects, the models disclosed herein may provide the following technical effects: (1) enhance the efficiency of public biomedical database management; (2) provide valuable insights into the evolving landscape of genomic research; and (3) enable formulation of more efficient queries of large public biomedical databases and ensure that relevant studies are not missed due to the use of synonyms in such studies.

[0065] In the following sections, tasks of HyDRA modeling are described in detail. By way of a non-limiting examples, HyDRA modeling may include data processing, feature extraction. BOA and CoBRA model training and validation, ADDER model training and validation, and data access tasks.

[0066] Data Processing

[0067] In some embodiments, all of the study labels utilized for training models may be dow nloaded from the CREEDS database web portal, which included Disease, Perturbation and Drug labels for each study as well as Case and Control labels for each sample. To obtain text information for the Gene Expression Omnibus (GEO) accession identifiers, study descriptions and sample descriptions may be collected and queried from the R Bioconductor package “GEOMetaDB” and associated SQLite database. The ‘’Title,” “Design” and “Summary'” columns from the study (gse) table and the “SampleName,” “SourceName,” “Characteristics” and “Description” columns from the sample (gsm) table may be collected for cleaning and processing. Samples appearing in more than one GEO accession, such as in a SuperSeries, or redundant GEO entries may be filtered out. This data processing task may be performed for all Disease, Perturbation and Drug datasets in the manual and automatic CREEDS partitions. Additionally, Array Studio’s DiseaseLand labels and GEO accession identifiers may be obtained from a 3^rd party vendor and queried from GEOMetaDB and processed in the same manner as CREEDS.

[0068] Feature Extraction

[0069] In some embodiments, after data processing task, processed text data from CREEDS and Omicsoff s Diseaseland Database may be converted into trainable feature matrices through unsupervised clustering, term frequency -inverse document frequency (TF- IDF) generation and embeddings vector generation. By way of a non-limiting example, an unsupervised clustering approach which uses a bag of words representation of sample text and selects the optimal number of sample clusters per study may be used for feature extraction task. Based on silhouette score, poorly clustered studies (silhouette score < 0.75) may be dropped from the training data. In the example embodiment, sample clusters may be filtered to include strings that are unique to each cluster, removing redundant and uninformative phrases. Such filtered sample clusters with specific keywords may be used to generate TF-IDF representations. With labeled training data, the TF-IDF matrices undergo supervised feature extraction based on a minimum fold change difference between class labels to reduce the number of unnecessary’ features. Further, cluster-specific keywords may be used to generate Embeddings from, for example, the BioSentVec model, which converts string information into a 200-dimensional vector. The embedding may be calculated for each item in the list of keywords and averaged together to arrive at a single 200-dimension vector for each cluster. Both the TF-IDF reduced matrix and embeddings matrix may be used to train difference models, as discussed herein.

[0070] BOA and CoBRA Model Training and Validation [0071 ] In the example embodiment, and by way of anon-limiting example, model parameters may be selected first by training with GridSearchCV from skleam with a 10-fold split to select optimal parameters for skleam classification models (Naive Bayes (NB), GradientBoostingClassifier (GBC), MLPClassifier (NN), RandomForestClassifier (RF) and LogisticRegression (LR)). All classification models may be trained and evaluated with, for example, 10-fold cross validation repeated 10 times and evaluated by area under the receiver operator curve (AUC). BOA models, which classify study type may be trained using the manually annotated CREEDS Disease, Drug and Perturbation studies. Each study type model may be trained individually by binarizing the labels as Disease vs. Other, Drug vs. Other and Perturbation vs. Other. Within each study category, CoBRA models may be trained to identify case and control groups. For each classification task, 5 different model ty pes (NB, GBC, NN, RF, LR) may be trained with 2 different inputs, the TD-IDF feature matrix and embeddings feature matrix, for a total of 30 BOA submodels and 30 CoBRA submodels. These submodels may be used to train ensemble models and a “sofifyvote” classifier may be used to determine a final model prediction.

[0072] ADDER Model Training and Validation

[0073] In the example embodiment, entity⁷ recognition models, such as the “Medmentions,” may be used to extract terms related to disease, tissue, and cell types. To convert these terms to controlled vocabulary, ontology resolution models may be trained, for example, using the John Snow Labs “SentenceEntityResolverApproach” function, using the 25 nearest neighbors based on Euclidean distance and Siamese Bidirectional Encoder Representations from Transformers (SBERT) embeddings. Separate resolver models may be developed for disease concepts, which used the Human Disease Ontology as the source of truth, and tissue and cell types, which used a combination of the Uberon multi-species anatomy ontology and the Cell Ontology.

[0074] Data Access

[0075] The NER+Resolver models may be trained using John Snow Labs Healthcare NLP libraries. GEO accession identifiers of the training and testing studies and their labels may be dow nloaded from the CREEDS data portal, however, ArrayStudio labels are proprietary and therefore may not be publicly available. As such, all GEO descriptions and text information is publicly available on the GEO website and from GEOMetaDB.

[0076] Accordingly, as described herein, a method known as ExpressionBlast for identifying case/controls and replicating uses a rule-based pattern matching. In the rule-based pattern matching, pattern matching may be based upon a regular expression (Regex) exact matching model, which may fail to identify a text or a keyword when searching for the text or the keyword in an uncontrolled free text. Additionally, or alternatively, searching based on the Regex exact matching model may map a particular keyword or text to an incorrect concept. The particular keyword or text may be mapped to an incorrect concept when the particular keyword or text is an acronym, and the Regex exact matching model ignores the context in which the particular keyword or text may be used. Similarly, CREEDS used for automatic derivation of gene expression signatures only produced about 6% of datasets associated with 113 diseases, 378 drugs, and/or 2052 gene in response to various searches.

[0077] In some embodiments, for improving search results and to avoid issues described above while using the Regex exact matching model and the CREEDS method, a model referenced herein as HyDRA may be used. The HyDRA model may generate a database, which may be referenced herein as a HyDRA-DB including annotated studies and gene expression study types. The gene expression study types may be automatically extracted gene expression study types. In particular, the HyDRA method may extract study metadata. The dataset may include one or more documents or samples, and each document or sample may have corresponding metadata. The study metadata may include one or more metadata elements. The one or more metadata elements of the dataset may include, but is not limited to, a title of the study, a name of the study, summary or description of the study, a category, or a theme of the study, one or more tags/keywords, and/or design of the study, and so on. Metadata of each document or sample may include a name, a title, a source, characteristics, and/or description or summary, and so on. Text included in the one or more metadata elements of the study, and/or text included in metadata of each document or sample may be extracted.

[0078] A plurality of tokens may be extracted from the extracted text of the study metadata and/or metadata of each document or sample using tokenizing text from the study metadata and/or metadata of each document or sample. A plurality of tokens may be extracted from the study metadata and/or metadata of each document or sample using a whitespace, a comma, a period, a semicolon, a colon, and so on, as a delimiter between words, phrases, or terms of the study metadata and/or metadata of each document or sample in the free-form text.

[0079] From the plurality of tokens, a subset of tokens (or referenced herein as named entities) corresponding to at least one of (i) a disease; (ii) a drug; and/or (iii) a treatment may be identified. The subset of token or the named entities may then be converted into a corresponding numeric vector using a term frequency-inverse document frequency (TF-IDF) algorithm. Alternatively, other algorithms such as a bag of words method, and/or a word2Vec method, and so on, may also be used to generate a numeric vector corresponding to each token or named entity of the subset of tokens. The subset of tokens converted to their respective numeric vectors (e g., w-dimensional vectors, where n is more than 1) using the TF-IDF, the bag of words, and/or the word2Vec algorithms or methods thus represents them based on context, meaning, and similarities between them.

[0080] Additionally, or alternatively, sentence embedding that is numeric vectorization of sentences in the study metadata and/or metadata of each document or sample of the study may also be used. In some embodiments, and by way of a non-limiting example, pretrained word embeddings and pretrained sentence embeddings, for example, BioWordVec and BioSentV ec, respectively, may be used for word embedding and sentence embedding.

[0081] Based on the subset of tokens converted to numeric vectors and each token of the subset of tokens assigned a respective TF-IDF numerical value (or weight), the plurality of documents or samples of the study may be clustered (or grouped). By way of a non-limiting example, clustering (or grouping) of the plurality of documents or samples of the study may be performed using k-means clustering and in accordance with the respective TF-IDF numerical value (or weight) of each document or sample of the plurality of documents or samples. A number of clusters (a value of k) for the k-means clustering may be dynamically selected. By way of a non-limiting example, the value k may be determined using the Silhouette method in which the silhouette value describing similarity of a numerical vector corresponding to a datapoint representing a token to its own cluster (cohesion) compared to other clusters (separation) may be measured. Based on the silhouette value measured for each datapoint representing a token of the subset of tokens corresponding to different number of clusters, an optimum value of k may be determined for k-means clustering.

[0082] Each document or sample of the plurality of documents or samples thus may be in its respective cluster. Since clustering identifies samples that are similar in characteristics, one or more words that are common among each sample in the cluster may be identified. The one or more words that are common among each sample in the cluster may provide keywords to identity' a particular cluster.

[0083] For each cluster, keywords identifying a cluster may be converted into numerical word embedding presentation, as described herein, using aTF-IDF algorithm. Based on the keywords identifying a respective cluster, the keywords may be binned or grouped in a plurality' of bins. The plurality' of bins may be corresponding to a disease, a treatment, and/or a gene perturbation, and so on. Accordingly, the keywords of each cluster may be binned or grouped in a bin of the plurality of bins.

[0084] Based upon the plurality' of samples clustered using K-means clustering, as described herein, and based on keywords that are common among each sample of a cluster binned into the plurality of bins, one or more models of annotated metadata representing archetypes (e.g., patterns or models) of each bin of the plurality of bins may be generated, for example, using binning of analysis (BOA), as described herein. By way of a non-limiting example, BOA may improve accuracy of the one or more models of annotated metadata representing archetypes because the BOA may identify outliers, invalid and missing values of various numeric variables, etc. Further, using a keyword-ontology bridging algorithm, keyword ontology for diseases, tissues, and/or drugs may be generated, for example, using ADDER model training and validation, as described herein. The keyword ontology for diseases, tissues, and/or drugs, and/or the plurality of samples with differential expression analysis may be added to the HyDRA-DB. The differential expression analysis for the plurality' of samples may include quantitative changes in the plurality of samples for case clusters and/or control clusters. [0085] Whether a particular cluster is a case cluster or a control cluster is determined based upon keywords identified as common keywords among all samples of the particular cluster. In some embodiments, the HyDRA-DB includes annotated studies and/or automatically extracted gene study types. By way of a non-limiting example, the automatically extracted gene study types may be added to the HyDRA-DB in accordance with predictive models configured to identify sample type from the text included in the study and the plurality⁷ of documents or samples of the study.

[0086] As described herein, the process of generating HyDRA-DB is based upon NER and disease ontology resolution pipeline. The NER may include word embedding and/or sentence embedding using one or more algorithms based on grammar, statistical natural language processing (NLP) models, and/or predictive models. The one or more algorithms used for NER may be an artificial intelligence (Al) or machine learning (ML) based algorithm. The AI/ML algorithms may be supervised and/or unsupervised AI/ML algorithms, rules-based algorithms, and/or dictionary-based algorithms. Additionally, or alternatively, the one or more algorithms may be in accordance with the neural networkbased systems.

[0087] In some embodiments, and by way of a non-limiting example, an entity resolver model may be used to link or associate an entity (e.g., a token "diabetes") using the NER model to an entity type (e.g., a disease or a syndrome, a mental or a behavioral dysfunction, a sign or a symptom, and/or a neoplastic process, and so on) of a controlled ontology or a concept identified by a concept ID. Additionally, or alternatively, a resolver confidence (or a score) corresponding to each linking or association of an entity with a respective entity type and the respective concept may also be generated and saved in a database storing each linking or association as a separate entry or a record.

[0088] In some examples, a combined use of the NER model and the entity⁷ resolver model, as described herein, may have a higher coverage of annotated studies or gene expression study types in comparison with the state-of-the-art Regex based searching methods or models. Various performance metrices, for example, precision (or positive predictive value defined as (1-FDR) where FDR stands for false discovery⁷ rate) and recall/sensitivity (or a true positive rate) that are focused on the positive class alone have a higher coverage in comparison with the state-of-the-art Regex based searching methods or models.

[0089] Various embodiments presented in the disclosure are described in more detail using FIGs. 1 -7.

[0090] FIGs. 1 and 1 con’t illustrate a schematic diagram 100 of NLP data extraction pipeline and NLP annotation pipeline of HyDRA modeling. The NLP data extraction pipeline includes extracting dataset metadata 102, deriving sample cluster features 104, and deriving text-based features 106. The NLP annotation pipeline includes generating ADDER models 108, performing binning of analysis types 110, and generating CoBRA models 112. As described herein. NLP data extraction pipeline utilizes both TF-IDF and LLM representations of dataset and sample level metadata to construct models such as ADDER models, BOA models, and CoBRA models. Since the tasks of NLP data extraction pipeline and the NLP annotation pipeline are described in detail, those details are not repeated here for brevity.

[0091] FIGs. 2A-2G illustrate word frequencies and embeddings of separate datasets by category' and samples by cases and controls. FIG. 2A illustrates a chart 200a corresponding to TF-IDF of top keywords found in dataset descriptions of disease, drug, and gene perturbation studies. In the chart 200a, rows are grouped by whether terms are found in multiple dataset types (top panel), or specific to a specific study ty pe (in order: disease, drug, and gene-specific terms). FIG. 2B illustrates a chart 200b corresponding frequency of disease, drug, and gene terms identified by NER models in disease, drug, and gene perturbation studies. FIG. 2C illustrates a chart 200c corresponding to top sample keywords by dataset type found in disease, drug, and gene studies and their frequencies. FIG. 2D illustrates a dimensional graph representation 200d of TF-IDF representations of dataset descriptions for dataset categories, such as, diseases, drugs, and genes. FIG. 2E illustrates a dimensional graph representation 200e of embeddings representation of sample keywords, for dataset categories, such as, diseases, drugs, and genes. FIG. 2F illustrates a dimensional graph representation 200f of TF-IDF representations of sample keywords for all disease datasets according to case and control labels. FIG. 2G illustrates a dimensional graph representation 200g of embeddings representation of sample keywords for all disease Datasets according to case and control labels. [0092] FTGs. 3A-3D illustrate representations of dataset and case/control predictions. FIG. 3A illustrates a graph 300a showing a receiver operator characteristics (ROC) curve for the disease BOA model generated, trained, and validated to predict disease, for example, in CREEDS datasets. FIG. 3B illustrates a comparison chart 300b corresponding comparison of repeated 10-fold cross validation area under the ROC curve (AUC) with HyDRA. FIG. 3C illustrates a bar plot 300c of feature importance for Naive Bayes model for disease datasets predicting case or control measured by the inverse log odds for each class. FIG. 3D illustrates a dimensional graph representation 300d of embedding space of all words found in TF-IDF models for diseases, drugs and genes dataset predictions based upon case and control labels.

[0093] FIGs. 4A-4D illustrate dataset and case/control predictions as generalizable to an independent curated dataset. FIG. 4A illustrates a bar plot 400a showing a frequency of high-level disease concepts found in CREEDS compared to the independent Array Studio curated dataset. FIG. 4B illustrates a bar plot 400b showing area under the ROC curves (AUC) for the prediction of disease and drug perturbation dataset labels in the independent ArrayStudio curated dataset. FIG. 4C illustrates a graph 400c corresponding to an ROC curve for case/control prediction in curated disease vs healthy datasets in the independent ArrayStudio dataset, for all studies, studies with diseases included in the training dataset (‘Previously Observed'), and studies with diseases not included in the training dataset (“New Disease’). FIG. 4D illustrates a violin plot 400d of accuracy per data when predicting case/control in studies with ‘Previously Observed’ and ‘New’ disease terms.

[0094] FIGs. 5A-5C illustrate application of HyDRA models to all GEO datasets. FIG. 5A illustrates a diagram 500a showing a bar plot of a number of GEO human studies (left) and a number of mouse studies (right) that were predicted by BOA models to be disease, drug, or gene perturbation studies, or are likely to have some other type of design (“Other”). Percentage relative to the total number of mouse or human studies is shown on the bar plot. FIG. 5B illustrates a chart 500b showing a frequency of high-level disease concepts in human and mouse studies. FIG. 5C illustrates a bar plot 500c corresponding to percentage of descendant terms that correspond to high-level disease concepts that are found in the CREEDS and ArrayStudio curated datasets, and in GEO overall. [0095] FTG. 6 illustrates an example flow-chart 600 of method operations according to some embodiments of the present disclosure. The method operations may be performed on one or more studies. Each of the one or more studies may include one or more documents or samples, and each document or sample may have corresponding metadata. The study metadata may include one or more metadata elements. The one or more metadata elements of the study may include, but is not limited to, a title of the study, a name of the study, summary' or description of the study, a category', or a theme of the study, one or more tags/keywords, and/or design of the study, and so on. Metadata of each document or sample may include a name, a title, a source, characteristics, and/or description or summary, and so on. Text from a plurality of metadata including, but not limited to, text of the metadata of each document or sample, and/or text of metadata of study may be extracted 602.

[0096] A plurality of tokens may be extracted 604 from the extracted text of the study metadata and/or metadata of each document or sample using tokenizing text from the study metadata and/or metadata of each document or sample. The plurality⁷ of tokens may be extracted 604 from the study metadata and/or metadata of each document or sample using a whitespace, a comma, a period, a semicolon, a colon, and so on, as a delimiter between words, phrases, or terms of the study metadata and/or metadata of each document or sample in the free-form text.

[0097] From the generated plurality of tokens, a subset of tokens (or named entities) corresponding to at least one of (i) a disease; (ii) a drug; and/or (iii) a treatment may be identified. A numeric vector, or a first numerical representation, corresponding to each token of the subset of token (or each named entity of the named entities), may be generated 606 using a term frequency-inverse document frequency (TF-IDF) algorithm. Alternatively, other algorithms such as a bag of words method, and/or a word2V ec method, and so on, may also be used to generate the numeric vector (or the first numerical representation) corresponding to each token or named entity' of the subset of tokens. The subset of tokens converted to their respective numeric vectors (e.g., //-dimensional vectors, where n is more than 1 ) using the TF-IDF, the bag of words, and/or the word2Vec algorithms or methods thus represents them based on context, meaning, importance, and/or similarities between them.

[0098] Based on the subset of tokens converted to numeric vectors and each token of the subset of tokens assigned a respective TF-IDF numerical value (or weight), the plurality of metadata corresponding to the plurality of documents or samples of the dataset may be partitioned or grouped 608 into a plurality of clusters according to a predetermined criterion. By way of a non-limiting example, partitioning or grouping of the plurality of metadata may be performed using k-means clustering and in accordance with the respective TF-IDF numerical value (or weight) of each document or sample of the plurality of documents or samples. The predetermined criterion may include a specific number of clusters (a value of k) for the k-means clustering. By way of a non-limiting example, the value k may be dynamically selected, for example, based upon the Silhouette method in which the silhouette value describing similarity of a numerical vector corresponding to a datapoint representing a token to its own cluster (cohesion) compared to other clusters (separation) may be measured. Based on the silhouette value measured for each datapoint representing a token of the subset of tokens corresponding to a different number of clusters, an optimum value of k may be determined for k-means clustering.

[0099] For each cluster of the plurality⁷ of clusters, a subset of tokens of the plurality of tokens that are common to all metadata of that cluster may be identified 610. Each document or sample of the plurality of documents or samples thus may be in its respective cluster based on partitioning 608. Since partitioning 608 groups samples that are similar in characteristics, one or more words that are common among each sample in the cluster may be identified 610. The one or more words that are common among each sample in the cluster may thus provide keywords to identify a particular cluster.

[0100] For each cluster, based on the one or more words (also referenced herein as a subset of tokens) that are common among each sample in the cluster, a second numerical representation of each token of the subset of tokens may be generated 612. As described herein, the second numerical representation may be generated using similar methods as described herein for generating the first numerical representation. The second numerical representation may include a numerical vector representation of each token of the subset of tokens that corresponds with a measure of semantic similarity of each token to another token of the subset of tokens.

[0101] Based on the second numerical representation of each token of the subset of tokens, a respective study type corresponding to each cluster of the plurality of clusters may be generated 614, as described herein. [0102] FTG. 7 illustrates an example block diagram of a computing device 700 performing method operations described in FIG. 6. Computing device 700 may include, but is not limited to, a smart phone, a tablet, a laptop, a server, and/or an electronic device. Computing device 700 may include a processor 704 for executing instructions. In some embodiments, executable instructions may be stored in a memory 706. Processor 704 may include one or more processing units (e.g., in a multi-core configuration). Memory 706 may be any device allowing information such as executable instructions and/or transaction data to be stored and retrieved. Memory’ 706 may include one or more computer readable media.

[0103] Computing device 700 may also include at least one media output component 708 for presenting information to user. Media output component 708 may be any component capable of conveying information to a user. In some embodiments, media output component 708 may include an output adapter (not shown) such as a video adapter and/or an audio adapter. An output adapter may be operatively coupled to processor 704 and operatively couplable to an output device such as a display device (e.g., a cathode ray tube (CRT), liquid crystal display (LCD), light emitting diode (LED) display, or “electronic ink” display) or an audio output device (e.g, a speaker or headphones).

[0104] In some embodiments, media output component 708 may be configured to present a graphical user interface (e.g., a web browser and/or a client application) to a user. A graphical user interface may include, for example, an interface for viewing prompts and data. In some embodiments, user computing device 700 may include an input 710 for receiving input from a user. The user may use input 710 to, without limitation, provide user input.

[0105] Input device 710 may include, for example, a keyboard, a pointing device, a mouse, a sty lus, a touch sensitive panel (e.g., a touch pad or a touch screen), a gyroscope, an accelerometer, a position detector, a biometric input device, at least one vision sensor (e.g., a camera or a video camera), and/or an audio input device. A single component such as a touch screen may function as both an output device of media output component 708 and input device 710.

[0106] User computing device 700 may also include a communication interface 712, communicatively coupled to a backend system or an application server. Communication interface 712 may include, for example, a wired or wireless network adapter and/or a wireless data transceiver for use with a mobile telecommunications network.

[0107] Stored in memory 706 are, for example, computer readable instructions for providing a user interface to user via media output component 708 and, optionally, receiving and processing input from input 710. A user interface may include, among other possibilities, a web browser and/or a client application. Web browsers enable users, such as user, to display and interact with media and other information typically embedded on a web page or a website from the backend system. A client application (e.g. , a frontend application executing on the user device 700) may allow user to interact with, for example, the backend system.

[0108] This written description uses examples to disclose the disclosure, including the best mode, and also to enable any person skilled in the art to practice the disclosure, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the disclosure is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they⁷ include equivalent structural elements with insubstantial differences from the literal languages of the claims.

Claims

CLAIMS What is claimed is:

1. A computer-implemented method for deriving study types from a dataset, the dataset including a set of documents, each document comprising text data and metadata associated with the text data, such that the dataset includes a plurality of metadata, the method comprising: extracting, by a processor of a computing device, text from the plurality of metadata; tokenizing, by the processor, the extracted text to identify a plurality of tokens; generating, by the processor, a first numerical representation of each token of the plurality of tokens, the first numerical representation including a numerical representation of the importance of that token to the set of documents, to generate a plurality' of first numerical representations; partitioning, by the processor and based on the first numerical representation of each token of the plurality of tokens, the plurality of metadata into a plurality of clusters according to a predetermined criterion; for each cluster of the plurality of clusters, identifying, by the processor, a subset of tokens of the plurality of tokens that are common to all metadata of that cluster; and for each identified subset of tokens corresponding to each cluster of the plurality' of clusters: generating, by the processor, a second numerical representation of each token of that subset of tokens, the second numerical representation including a numerical vector representation of that token corresponding with a measure of semantic similarity' of that token to another token in that subset of tokens; and based on the second numerical representation of each token of the subset of tokens, generating, by the processor, a respective study type corresponding to each cluster of the plurality of clusters.

2. The method according to claim 1, wherein the tokenizing comprises identifying the plurality of tokens corresponding to at least one of: disease-specific tokens, treatmentspecific tokens, or gene-specific tokens.

3. The method according to claims 1 or 2, wherein the generating the first numerical representation comprises generating the first numerical representation of the plurality of tokens in accordance with a Term Frequency-Inverse Document Frequency (TF-IDF) representation.

4. The method according to any one of claims 1-3, wherein the generating the second numerical representation comprises generating the second numerical representation in accordance with a Term Frequency-Inverse Document Frequency (TF-IDF) representation.

5. The method according to any one of claims 1-4, further comprising generating a first semantic representation of the plurality' of tokens corresponding to each sample metadata of the plurality of sample metadata, the first semantic representation comprising an embeddings representation.

6. The method according to any one of claims 1-5, further comprising generating a second semantic representation of each token of the subset of tokens, the second semantic representation comprising an embeddings representation.

7. The method according to any one of claims 1-6, wherein the partitioning includes applying a k-means clustering approach on the plurality of tokens.

8. The method according to claim 7, further comprising determining a value of k using a silhouette algorithm.

9. The method according to any one of claims 1 -8, wherein the generating the respective study type comprises generating at least one of: a disease study, a gene study, or a treatment study.

10. The method according to any one of claims 1-9, further comprising categorizing, by the processor, the respective study type corresponding to each cluster of the plurality of clusters into one or more patterns or models.

11. The method according to any one of claims 1-10, further comprising generating, by the processor, a disease identifier based on the first numerical representation of each token of the plurality' of tokens.

12. A system for deriving study types from a dataset, the dataset including a set of documents, each document comprising text data and metadata associated with the text data, such that the dataset includes a plurality of metadata, the system comprising: at least one memory configured to store instructions; and at least one processor configured to execute the stored instructions, which when executed cause the at least one processor to perform operations comprising: extracting text from the plurality of metadata; tokenizing the extracted text to identify a plurality of tokens; generating a first numerical representation of each token of the plurality of tokens; partitioning, based on the first numerical representation of each token of the plurality of tokens, the plurality of metadata into a plurality of clusters according to a predetermined criterion; for each cluster of the plurality of clusters, identifying a subset of tokens of the plurality of tokens that are common to all metadata of that cluster; and for each identified subset of tokens corresponding to each cluster of the plurality of clusters: generating a second numerical representation of each token of that subset of tokens, the second numerical representation including a numerical, vector representation of that token that includes a measure of sematic similarity of that token to another token in that subset of tokens; and based on the second numerical representation of each token of the subset of tokens, generating a respective study type corresponding to each cluster of the plurality of clusters.

13. The system according to claim 12, wherein the plurality of tokens comprises at least one of: disease-specific tokens, treatment-specific tokens, or gene-specific tokens.

14. The system according to claims 12 or 13, wherein at least one of the first numerical representation or the second numerical representation is generated in accordance with a Term Frequency-Inverse Document Frequency (TF-IDF) representation.

15. The system according to any one of claims 12-14, wherein the operations further comprise generating a first semantic representation of the plurality of tokens corresponding to each sample metadata of the plurality of sample metadata, the first semantic representation comprising an embeddings representation.

16. The system according to any one of claims 12-15, wherein the operations further comprise generating a second semantic representation of each token of the subset of tokens, the second semantic representation comprising an embeddings representation.

17. The system according to any one of claims 12-16, wherein the partitioning comprises apply ing a k-means clustering approach on the plurality of tokens, and wherein a value of k is determined using a silhouette algorithm.

18. The system according to any one of claims 12-17, wherein the respective study type comprises at least one of: a disease study, a gene study, or a treatment study.

19. The system according to any one of claims 12-18, wherein the operations further comprising categorizing the respective study type corresponding to each cluster of the plurality of clusters into one or more patterns or models.

20. A non-transitory computer-readable medium (CRM) embodying programmed instructions which, when executed by at least one processor of a backend system, cause the at least one processor to derive study types from a dataset, the dataset including a set of documents, each document comprising text data and metadata associated with the text data, such that the dataset includes a plurality of metadata, by performing operations comprising: extracting text from the plurality⁷ of metadata; tokenizing the extracted text to identify a plurality of tokens; generating a first numerical representation of each token of the plurality of tokens; partitioning, based on the first numerical representation of each token of the plurality of tokens, the plurality of metadata into a plurality of clusters according to a predetermined criterion; for each cluster of the plurality of clusters, identifying a subset of tokens of the plurality of tokens that are common to all metadata of that cluster; and for each identified subset of tokens corresponding to each cluster of the plurality of clusters: generating a second numerical representation of each token of that subset of tokens, the second numerical representation including a numerical, vector representation of that token that includes a measure of sematic similarity of that token to another token in that subset of tokens; and based on the second numerical representation of each token of the subset of tokens, generating a respective study type corresponding to each cluster of the plurality of clusters.

21. The non-transitory CRM of claim 20. wherein the operations further comprise training one or more machine-learning algorithms using the generated respective study type corresponding to each cluster of the plurality of clusters.

22. The non-transitory CRM of claim 20, wherein the operations further comprise: training one or more classification algorithms using the generated respective study type corresponding to each cluster of the plurality of clusters; and classifying a plurality’ of records stored in a database using the one or more classification algorithms trained using the generated respective study ty pe corresponding to each cluster of the plurality of clusters.