EP4669211A1

EP4669211A1 - METHOD FOR THE DIAGNOSTIC DEVELOPMENT OF PANCREASES USING ULTRASOUND IMAGES

Info

Publication number: EP4669211A1
Application number: EP24763315.9A
Authority: EP
Inventors: Adrian SAFTOIU; Lucian-Gheorghe GRUIONU; Anca Loredana UDRISTOIU; Gabriel GRUIONU; Elena Codruta GHEORGHE
Original assignee: Medical Softverse Srl
Current assignee: Medical Softverse Srl
Priority date: 2023-02-28
Filing date: 2024-02-27
Publication date: 2025-12-31
Also published as: WO2024180385A1

Abstract

Artificial Intelligence (AI) and Machine Learning (ML) enabled systems and methods for detecting, marking, and differentiating pancreatic lesions and pancreatic parts from endoscopic ultrasound (EUS) images is disclosed. The system may receive and analyze endoscopic ultrasound images (EUS-images) and patient demographic data to detect, mark and classify anatomic-features. The anatomic-features may be images of pancreatic solid masses (benign or malignant), cystic lesions, and anatomic features of normal pancreas. The EUS-images may include color Doppler or power Doppler EUS-images. The detection-methods may include AI-processing methods which may use machine learning techniques to analyze the EUS-images and to extract relevant information, such as the size, shape, and location of the solid masses (benign or malignant tumor) or cysts. This information may be used to make medical predictions and to determine whether it is necessary or advisable to perform certain medical procedures, such as EUS-guided fine-needle aspiration biopsy (FNAB).

Description

METHODS AND SYSTEMS FOR DETECTING, MARKING, AND DIFFERENTIATING PANCREATIC LESIONS FROM ENDOSCOPIC ULTRASOUND IMAGES

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority from and the benefit of the United States Provisional Patent Application No. 63/449,005 filed on February 28, 2023, and titled "Artificial Intelligence Methods and Systems for Detecting, Marking and Diagnosing Gastro-Intestinal Lesions and Cysts from Endoscopic Ultrasound Images and Patient Demographic Data", which is hereby incorporated by reference for all purposes as if fully set forth herein.

BACKGROUND OF THE INVENTION

I. FIELD OF THE INVENTION

Exemplary embodiments of the present invention relate to methods and systems using Artificial Intelligence to detect, mark and differentiate anatomical features from endoscopic ultrasound images.

II. DISCUSSION OF THE BACKGROUND

Pancreatic lesions (solid masses and cystic neoplasms) are often difficult to detect and diagnose due to the complex anatomy of the pancreas and the limitations of traditional imaging techniques. Endoscopic ultrasound (EUS) is a commonly used diagnostic tool for detecting pancreatic lesions, however, the manual interpretation of EUS images (in regular / traditional EUS procedures) can be time-consuming and prone to human error. As a result, the decision to perform EUS-guided fine needle aspiration biopsy (EUS-FNAB) is often left to the discretion of the endoscopist, leading to a low diagnostic yield or increased percentage of complications.

SUMMARY OF THE INVENTION

Exemplary embodiments of the invention disclose a detection-system configured to receive and/or analyze endoscopic ultrasound images (EUS-images) and patient demographic data to detect, mark and classify anatomic-features. The anatomic-features may be images of pancreatic solid masses (benign or malignant), cystic lesions, and anatomic features of normal pancreas. The EUS-images may include color Doppler or power Doppler EUS-images. The detection-system may be configured to perform various detection-methods. The detectionmethods may include Al-processing methods which may use machine learning techniques to analyze the EUS-images and to extract relevant information, such as the size, shape, and location of the solid masses (benign or malignant tumor) or cysts. This information may be used to make medical predictions and to determine whether it is necessary or advisable to perform certain medical procedures, such as EUS-guided fine-needle aspiration biopsy (FNAB) according to current protocols / guidelines.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention, and together with the description serve to explain the principles of the invention.

FIG 1. shows an exemplary embodiment of an Al enabled detection-system physical components.

FIG 2. shows a flowchart of an exemplary embodiment of an Al model development module.

FIG 3. shows an exemplary embodiment of a system for medical imaging analysis.

FIG 4. shows a flowchart of an exemplary embodiment of a method for performing Al model training.

FIG 5. shows a flowchart of an exemplary embodiment of a data preparation method.

FIG 6. shows a flowchart of an exemplary embodiment of a method for generating an Al model.

FIG 7. shows an exemplary embodiment of an Al enabled imaging apparatus. FIG 8. shows an exemplary embodiment of an Al imaging system.

FIG 9. shows an exemplary image of normal pancreas detection.

FIG 10. shows an exemplary image of the detection and segmentation of the normal pancreas body by using a CNN1 (Convolutional Neural Network 1).

FIG 11. shows an exemplary image of the detection and segmentation of the normal pancreas body by using a CNN2 (Convolutional Neural Network 2).

FIG 12. shows an exemplary image of the detection and segmentation of a cystic lesion inside the normal pancreas head (CNN1).

Figure 13. shows an exemplary image of the detection and segmentation of a cystic lesion inside the normal pancreas head (CNN2).

Figure 14. shows an exemplary image of the detection and segmentation of a solid tumor inside the normal pancreas head (CNN1).

Figure 15. shows an exemplary image of the detection and segmentation of a solid tumor inside the normal pancreas head (CNN2).

Figure 16. shows a user-interface enabling users (e.g. clinicians, medical doctors) to use the Al enabled systems and methods of this application.

DETAILED DESCRIPTION

The invention is described more fully hereinafter with reference to the accompanying drawings, in which embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure is thorough, and will fully convey the scope of the invention to those skilled in the art. Like reference numerals in the drawings denote like elements.

The following detailed description is provided to gain a comprehensive understanding of the methods, apparatuses and/or systems described herein. Various changes, modifications, and equivalents of the systems, apparatuses and/or methods described herein will suggest themselves to those of ordinary skill in the art. Descriptions of well-known functions and structures are omitted to enhance clarity and conciseness.

The detection-methods disclosed herein may provide significant advantages over the traditional ways of reading and interpreting EUS images performed by humans (medical professionals such as radiologists). The detection-methods may lead to increased accuracy and consistency in detecting and marking pancreatic solid masses and cysts, as well as in detecting and marking features of the normal pancreas, thereby reducing the risk of human error. The detection-methods may lead to faster and more efficient analysis of EUS images, thereby reducing the time required for diagnosis and treatment planning, such as performance of EUS- guided FNAB. The detection-methods may allow for the dynamic analysis of pancreatic lesions in time through serial measurements and evaluation of other characteristics pre-therapy and post-therapy (size, vascularity, stiffness, etc.). The detection-methods may allow for the automatic back-up of images (or sequence of images such as movies) received by the detectionsystem, without the need for human intervention. The detection-methods may include capabilities of automatic generation of structured reports for EUS, according to existing protocols and/or guidelines.

The detection-system and detection-methods disclosed herein may be used to detect and mark pancreatic masses and cysts from EUS images / movies. The detection-system and detection-methods disclosed herein may use EUS-images, information extracted from the EUS- images via the detection-methods, and patient demographic data for the diagnosis and therapy of pancreatic disease. The detection-methods have the potential to improve patient outcomes and reduce the time and resources required for diagnosis and treatment planning.

(1). The detection-system

An exemplary embodiment of the detection-system 100 is described with reference to FIG. 1. The detection-system may include a computer-system 101. The computer-system may include one or more of a high-performance processor, graphics card, and memory configured to run Al algorithms and to store large amounts of patient data on HDD/cloud.

The detection-system may further include an EUS-system 102 configured to acquire images (EUS-images) of the pancreas of a patient (the imaged-pancreas). The images may come as timed image-sequences (e.g. movies) of the imaged-pancreas. The images may include anatomical-features of the imaged-pancreas such as solid masses (e.g. tumors), cysts, and parts of normal pancreas images. The anatomical-features may be analyzed and classified by the detection system.

The detection-system may further include or may be connected to a patient-database 103 configured to store patient demographic data such as age, gender, weight, medical history, relevant medical conditions, medical history, etc.

The detection-system may further include an interface-module 104 configured to input and retrieve patient data and images / movies. The images and/or movies may come in DICOM format, with capabilities to export in other well-known formats. The interface-module may further be used by operators (e.g. physicians) to input information about images, to mark the images, and to annotate the images.

The detection-system may further include an image processing and analysis module (hereinafter image-processing-module 105) that uses machine learning techniques to detect and mark, on the EUS-images, pancreatic lesions (masses and/or cysts), as well as normal parts of the pancreas in the images. The image-processing-module may be configured to receive EUS-images, to process and analyze the EUS-images, and to generate a processed-EUS-image. The processing and analysis of the EUS-images may include Artificial Intelligence (Al) and Machine Learning (ML) procedures. The processed-images may include feature-markings on the EUS-images showing lesions, cysts, anatomical-features. The processed-images may also include image-annotations or markings describing risk predictions and other information relevant to diagnosis and possible therapy.

The detection-system may further include a prediction-module 106 that uses the pancreatic lesion (solid masses and/or cysts segmentation) to make predictions about the risk of pancreatic disease and the need for follow-up actions, such as: therapy / treatment procedures and the necessity of performing EUS-guided FNAB. The prediction-module may include or employ Al and ML procedures.

The detection-system may further include a graphical user interface (GUI) 107 configured to display EUS-images (as acquired from the EUS-system), processed-images, analysis results, and auxiliary information such as patient information, information about image / data acquisition and information about processing procedures.

The detection-system may further include a reporting-module 108 configured to generate reports including relevant information about patients and their anatomy (e.g. information about pancreas and/or pancreatic lesions such as masses and/or cysts). The reports may be used by medical professionals for diagnosis and treatment.

The detection-system may further include a secure-data-system 109 configured to protect patient privacy and data. The secure-system may be closed and may be protected via encryption, cybersecurity protocols and data privacy procedures.

The detection-system may further include a continuous learning module 110 configured to improve over time the prediction accuracy, the processing of the EUS-images, and the analysis of the EUS-images. The continuous-learning-module may use data acquired over time, data from many patients, feedback and input from medical professionals, and patient data (e.g., demographic and health history data). The continuous-learning-module may include and/or employ Al and ML processes.

(2). The Model-Development-Module.

The detection-system 100 may include a model-development-module 200. Modeldevelopment-methods for operating the model-development-module are also disclosed. Exemplary embodiments of the model-development-module and model-development-methods are described with reference to FIG. 2. The model-development-methods may include AI/ML processes, such as the continuous training of an Al model which uses Endoscopic Ultrasound (EUS) images and video data. The model-development-methods may be divided into distinct phases involving both medical doctors and Al developers. The model-development-methods may include one or more of the operations, procedures, and phases described hereinafter. The methods may include a data-preparation-phase, an Al training and model validation phase, a testing and diagnostic phase, and a continuous learning phase.

The model-development-module may include an EUS-data-repository 201. The EUS- data-repository may include a set of EUS-datasets. Each of the EUS-datasets may include EUS videos and/or EUS images acquired during an EUS procedure performed on the pancreas of a patient. The EUS-data-repository may include a relatively large number of EUS-datasets, each of the EUS-datasets corresponding to an EUS procedure performed on the pancreas of a patient.

The model-development-module may include data-selection-module 202 configured to select relevant EUS-datasets from the EUS-data-repository. The data-selection-module 202 is configured to receive input from medical professionals and/or Al professionals about which EUS-datasets are suitable for training procedures. The data-selection-module may select the suitable data according to the input from the medical professionals and Al professionals.

The model-development-module may include data-split-module 203 configured to separate the selected data into a plurality of datasets which may include: a training-dataset (comprising EUS-datasets), a validation-dataset (comprising EUS-datasets), and a testingdataset (comprising EUS-datasets). The training-dataset may further include a supervised- training-dataset (comprising EUS-datasets) which is sent to a supervised-training-module. The training-dataset may further include an unsupervised-training-dataset (comprising EUS- datasets) which is sent to the unsupervised-data-module 221.

The model-development-methods may include supervised-training procedures and unsupervised-training procedures.

The Supervised-training-module.

The supervised-training-module may include modules 204 to 210 shown in FIG. 2 and described hereinafter. The supervised-training-module includes a data-module 204 receiving the supervised-training-dataset from data-split-module 203. The supervised-training-module may include data-annotation-module 205 configured to receive input from medical and Al professionals and to generate markings, labels, and annotations for each of the EUS-datasets. The markings, labels, and annotations are appended to the EUS-datasets and are used as labels for Al model training.

The supervised-training-module may further include a second-data-splitting-module 206 configured to split the supervised-training-dataset into supervised-training-data which is passed to training-validation-module 207 and testing-data which is passed to the testing-module 210. The supervised-training-module may further include an Al-model-training-module 208.

The Al-model-training-module may perform Al-training on one or more of the following data: the EUS-datasets (including the markings, labels, and annotations on the EUS-datasets) of the supervised-training-data received from 207; EUS-datasets received from the unsupervised- training-procedures (see e.g. datasets received from the unsupervised-data-selection-module 223); new EUS-datasets 233 received from the new-data-feed-module 233 which has been detected by the detection-system 100 and/or has been analyzed by medical professionals. Al- model-training may be performed according to procedures known in the AI/ML art. The Al- model-training module may generate an Al-Model and its model-parameters (e.g. weights). The Al-model-training-module 208 is configured to detect and extract features (e.g., solid masses, cysts, anatomical parts of the pancreas) in the EUS-images, and to mark and label the features.

The supervised-training-module may further include an Al-model-validation-module 209 configured to receive an Al-Model from the Al-model-training-module 208 and to evaluate and/or validate the performance of the Al-Model against the EUS-datasets in the testing-data received from the testing-module 210. The performance of the Al-Model may be represented by parameters such as: accuracy of feature detection, precision, etc.

If the evaluation/validation results of the Al-Model show that an increase in performance is required, then the Al-model-validation-module 209 sends an increase-precision- message to the unsupervised-learning-modules 220 instructing the increase of a precisionparameter.

If the evaluation/validation results of the Al-Model show that performance is as required, the Al-model-validation-module 209 may send a precision-as-required-message instructing the unsupervised-learning-modules 220 that increased precision is not required. The Al-Model may be updated according to the validation results.

Supervised-continuous-learning may be performed by feeding the new data (e.g., EUS- datasets 233) to the Al-model-training-module 208.

The Unsupervised-Training-Module.

The unsupervised-training-module 220 is configured to perform unsupervised training on the unsupervised-training-dataset. The unsupervised-training-module 220 may include the unsupervised-data-module 221; the unsupervised-detection-module 222; the unsupervised- data-selection-module 223; the precision-detection-module 224; the precision-adjustment- module 225; and the model-update-module 226.

The unsupervised-data-module 221 may receive the unsupervised-training-dataset from data-split-module 203.

The unsupervised-detection-module 222 may employ AI/ML to learn patterns and detect features (e.g. tumors, cysts, parts of the pancreas) in the EUS-datasets of the unsupervised-training-datasets without using labeled data. The EUS-datasets of the unsupervised-training-datasets include EUS-videos with no annotations / labels. The unsupervised-detection-module 222 may use Convolutional Neural Networks (CNNs) to detect, in the images of the EUS-datasets, anatomical-features such as tumors, cysts, and parts of the pancreas. The unsupervised-detection-module 222 may use / incorporate the Al-Model 250 (or may use information from the Al-Model) to perform the detection and the learning.

The unsupervised-training-module may include an unsupervised-data-selection-module 223 configured to perform selection operations as described hereinafter. The Al-Model may be configured to calculate a confidence-parameter ("CONF") for each EUS-dataset of the unsupervised-training-dataset. The unsupervised-data-selection-module 223 may determine, for each processed EUS-dataset (after processing with the Al-Model), if the confidenceparameter for the EUS-dataset is larger than a precision-parameter-threshold (p). EUS-datasets having a confidence-parameter smaller than the precision-parameter p are not selected and are not passed to the Al-model-training-module 208. EUS-datasets having a confidence-parameter CONF larger than the precision-parameter-threshold p are selected and passed to the Al-model- training-module 208.

The Al-model-training module 208 and the Al-Model-Validation-Module 209 may perform on the updated / selected EUS-datasets (received from 223) the same operations as described above with reference to modules 208 and 209 and may send either an increase- precision-message or a precision-as-required-message to the precision-detection-module 224. If an increase-precision-message is sent, then the operations described above with reference to modules 224, 225, 223, 208, and 209 are repeated in a loop until the Al-Model-Validation- Module 209 generates a precision-as-required-message.

When a precision-as-required-message is received by module 224 an instruction is sent to the model-update-module 226 to update the Al-Model at 250 according to the current state of the Al-Model (and its model-parameters, e.g., weights) at the Al-model-training-module 208 and/or the validated model at Al-model-validation-module 209.

The idea is to use Al-segmented images for additional training. However, if there's no improvement in the network performance after training with these new images, the algorithm will increase its confidence threshold. This results in segmenting fewer but more reliable images, which are then used for another round of training.

Diagnostics on New-EUS-Videos.

The model-development-module 200 may further include or may be connected to a new-data-module 230, a detection-module 231; a verification-module 232; a diagnostic-module 235; and a new-data-feed-module 233.

The new-data-module 230 may be configured to store a new EUS-video of a patient's pancreas or to acquire a new EUS-video of a patient's pancreas. The new EUS-video may be passed to the detection-module 231. The detection-module 231 employs the validated Al- Model at module 250 to detect features of the patient's pancreas (e.g. tumors, cysts, parts of the pancreas) on the new EUS-video. The detection-module 231 may generate a description of the detected features and/or may create markings showing the features on the images/frames of the new EUS-video. The detection-module 231 may associate the detected features to a specific type of tissue (e.g. tumors tissue, cysts, pancreas parts). The detection-module 231 may also provide diagnostic-predictions for the EUS-Video. The above information generated or inferred by the detection-module 231 may be included in a descriptive-data of the EUS-video.

The verification-module 232 may receive from the detection-module 231 the EUS-video and the information inferred by the detection module (e.g. feature descriptions, markings, annotations, diagnostic-predictions). The verification-module 232 may enable a medical professional to verify the accuracy of the inferred-information generated by the detectionmodule 231, to make corrections to the descriptions and markings, and to generate corrected- information for the EUS-Video. The corrected-information may be sent to the diagnosticmodule 235 which may generate a diagnostic for the EUS-Video and the patient.

The verification-module 232 may determine if the new EUS-Video and the corrected- information are suitable for Al model training. If the EUS-Video and the corrected information are suitable for Al model training, the verification-module 232 may send the new EUS-Video and the corrected-information to the new-data-feed-module 233.

The new-data-feed-module 233 may provide the new data (EUS-Video and corrected- information) to the Al-model-training-module 208 for retraining the Al-Model and generating improved Al Models.

Continuous Learning Phases.

The model-development-module 200 is configured to periodically or/and continuously receive new data (via modules 201 and 230) and to perform the operations described above with reference to the modules in FIG. 2 so as to improve/update the Al-Model according to: the new data, the input received from the medical professionals (e.g. markings, labels, annotations), and the input received from Al professionals. The new data includes new EUS- Videos acquired from patients (images / videos of patients' pancreas).

The model-development-module 200 is configured to perform supervised continuous learning and unsupervised continuous learning. During supervised continuous learning (e.g. operations performed by modules 202 to 210 and 230 to 235) the model may continue to learn (e.g. improve the Al-Model) from new data under supervision and may involve feedback from medical doctors. Medical doctors participate in the annotation and validation of data, ensuring that the Al-Model's outputs are accurate and clinically relevant. The operations of the supervised continuous learning may be repeated in a loop whenever new data is received.

During unsupervised continuous learning (e.g., operations performed by modules 221 to 226) the model may continue to learn (e.g., improve the Al-Model) from new data in an unsupervised manner. The operations of the unsupervised continuous learning may be repeated in a loop whenever new data is received.

The loop allows the Al-Model to adapt to new data and potentially improve over time, making the Al-Model more robust and precise in its diagnostic capabilities. (3). The System for medical imaging analysis.

A closed-loop imaging-system 300 is described hereinafter with reference to FIG. 3. The imaging-system 300 may be designed to leverage the strengths of both fast Al and precise Al to provide high-quality diagnostic support. The imaging-system 300 may integrate Al in medical diagnostics and continuous learning. The imaging-system 300 may be configured to periodically receive input from medical professionals and to periodically improve the Al model. The imaging-system 300 is connected with or may include EUS-Equipment 301 which is configured to acquire EUS-data (e.g. videos, images of the pancreas of a patient).

The imaging-system 300 may further include an Al-Workstation 302 which may further comprise a Precise-AI-Module 303 and a Fast-AI-Module 304. The EUS-data acquired by the EUS-Equipment is sent to the Al-Workstation. The Fast-AI-Module employs a Fast-Al-Model which quickly processes the EUS-data for object recognition and segmentation. The Precise-AI- Module employs a Precise-Al-Model which provides more accurate and detailed object recognition and segmentation of the EUS-data. The imaging-system 300 may include a switch 315 configured to select between the Precise-AI-Module 303 and the Fast-AI-Module 304.

The imaging-system 300 may further include an EUS-Monitor 305 and an Al-Monitor 306. The EUS-Monitor 305 is configured to display EUS-data (e.g. raw ultrasound images) received from the EUS-Equipment. The Al-Monitor 306 is configured to display Al-processed- data (e.g. EUS images processed by the Fast-AI-Module and/or Precise-AI-Module) received from the Al-Workstation. The EUS-Monitors 305 and 306 may be implemented via computer monitors, displays, or screens.

The imaging-system 300 may further include a decision-module 308. The decisionmodule may analyze the results of the Al processing (e.g., the Al-processed-data including processed, annotated, and/or marked EUS images) and determine whether the results satisfy certain results-requirements.

A medical-doctor 307 may view the EUS-data (e.g., EUS raw images) and/or the Al- processed-data (e.g., Al processed, annotated, and marked EUS images) displayed on the monitors 305 and 306. For an EUS-data (e.g., EUS videos and images) acquired from a patient, the two monitors may display the raw and processed EUS images simultaneously so that the medical doctor can easily compare them. The medical-doctor may review the images and associated information (i.e. results of the Al processing) displayed on the monitors 305 and 306 and decide if the observed results (e.g. images, markings, labels) satisfy certain resultrequirements (e.g. the images are good, the results are clear, the results are accurate). The medical-doctor may use the decision-module 308 to provide input to the imaging-system 300 that the results satisfy the results-requirements or do not satisfy the results-requirements.

In case the results satisfy the results-requirements, the specific EUS-data and/or the corresponding Al-processed-data may be used for diagnosis, may be labeled as "illustrative- data", and/or may be sent by the decision-module 308 to the lllustrative-data-module 309. The illustrative-data may be stored on a Cloud-Storage 310. The illustrative-data may be used for further training of the Al models.

In case the results do not satisfy the result-requirements, the medical doctor may provide review and feedback to the decision-module 308. The decision-module 308 may send instructions to the Al Workstation to re-process the EUS-data via the Precise-AI-Module for increased precision.

The imaging-system 300 may include an Al-Server 311 configured to manage the Al- Models used by the Al-Workstation. The Al-Server 311 may at various times update the Al- Models (e.g. the weights of the Al-Models) as new training is performed. Training may be performed continuously, periodically or at a sequence of times. The Al-Server 311 may be connected to a second-decision-module 312 configured to determine whether sufficient new data has been accumulated in the Cloud-Storage so as to warrant the retraining of the Al- Models. When the second-decision-module finds that sufficient new data has been accumulated in the Cloud-Storage, the Al-Models are retrained by using the new data. The updated Al-Models are used by the Al-Workstation to perform Al processing on future EUS- data. The Al-Models are therefore updated over time as more EUS-data is acquired and the Al- Models are thereby over time improved as more EUS-data is acquired. The imaging-system is designed to learn from new data and from feedback received from the medical doctors to enhance its precision and reliability.

(4). Methods for performing Al Model training. A training-method 400 for performing Al model training is described with reference to FIG. 4. The training-method may be implemented as a collaborative effort between medical personnel and IT specialists in creating and training an Al tool for medical diagnostics.

The training-method 400 may include one or more of the steps/actions described hereinafter. Some of the steps/actions may be performed by medical doctors, some of the steps/actions may be performed by IT professionals, and/or some of the steps/action may be performed automatically by computers and other equipment. The training-method may include processing EUS (Endoscopic Ultrasound) video / images data and developing a trained Al model. The training-method 400 may include one or more of steps 401 to 410 shown in FIG. 4.

The step 401 comprises the receiving of EUS-video-data (e.g. endoscopic ultrasound videos/images of the pancreas of one or more patients) from EUS-Equipment or from a data storage or repository.

The step 402 comprises video trimming or pancreas selection actions in which a medical doctor may select and/or trim the videos to focus on relevant sections (e.g. the sections showing the pancreas).

The step 403 comprises actions reducing the number of frames per second (FPS) of the video which may decrease the volume of data for processing. During step 403 the videos may be processed so as to remove from the video a certain set/number of frames (e.g. the less relevant frames). This way the obtained video includes less frames/information than the original video.

The step 404 comprises actions in which a number of frames are extracted from the video for further processing and the extracted frames are cropped as needed. This step may be performed with input from an IT professional and/or by one or more of the modules and systems described in this application.

The step 405 comprises labeling actions in which images of the EUS-video-data are labeled with one or more labels. The labeling may be performed by one or more of the modules and systems described in this application (e.g. module 205 in FIG. 2) with input from a medical doctor and/or from an IT professional. The step 406 may be performed by module 204 of the model-development-module 200. Examples of labels which may be attached are "pancreas", "tumor", "solid mass", and "cyst".

The step 406 comprises data preparation actions in which the labeled data undergoes further preparation actions, such as: organizing data, splitting data into training and validation sets, and/or data augmentation. The step 406 may be performed by module 204 of the modeldevelopment-module 200 described in FIG. 2.

The step 407 comprises model training actions in which the prepared data and/or the labeled data are used to train the Al-Models of the detection-system 100, the Al-Models of the model-development-module 200, the Al-Models of the imaging-system 300, and any other system in this application. New Al-Models are obtained as a result of the training (e.g. new weights are obtained for the Al-Models).

The step 408 comprises model evaluation actions in which, after training, the performance of the newly trained Al-Models is evaluated. The evaluation may be performed against a set of validation data to ensure the accuracy of the newly trained Al-Models.

The step 409 comprises model validation actions in which a medical doctor reviews the results of using the newly trained Al-Models to confirm their medical accuracy and reliability.

If the steps 408 and 409 confirm that the newly trained Al-Models satisfy the required accuracy and reliability, the Al-Models are updated according to the newly trained Al-Models (e.g. the weights of the Al-Models are updated with the new weights). The updated trained Al- Models 410 may have an improved performance over the Al-Models before training and may be deployed for use on patients.

(5). Data Preparation Methods.

A Data-Preparation-Method 500 including the preparing of datasets from the EUS-videos are described with reference to FIG. 5. Medical doctors may be involved in the initial video trimming and frame extraction processes, ensuring that only medically relevant data is processed. Al developers may be involved at the later stages, including two rounds of frame annotation and the final dataset export, which indicates their role in preparing the data for Al development. The Data-Preparation-Method 500 may include one or more of steps 501 to 507 described hereinafter. Step 501 may include the trimming of EUS videos which may comprise cutting out unnecessary parts of the video to focus on the relevant sections for analysis.

Step 502 may include a process where the frame rate of the videos is reduced. This means that fewer frames per second are kept, which can help in reducing the amount of data to be processed.

Step 503 may include a process of extracting frames from the videos in which individual frames are extracted from the videos to be used for further analysis.

Step 504 may include a process in which the size of the extracted frames is reduced. This may involve resizing the frames to a smaller dimension, which can be important for standardizing input data for Al models and reducing computational load.

Step 505 may include a process in which a first round of annotations is performed which involves labeling or marking frames with relevant information such as the location of anatomical features or abnormalities. The first-round of annotations may be performed by a medical doctor.

Step 506 may include a process in which a second round of annotations is performed which may comprise a review of the initial annotations and/or adding of an additional layer of information. Some of the processes at step 506 may be performed by an Al developer.

Step 507 may include a process in which the processed and annotated frames are exported as a dataset. The datasets may be further used in training or evaluating Al models.

Steps 501 and 505 may include receiving input from medical doctors. Steps 502, 503, 504, 506 and 507 may include receiving input from Al developers.

(6). Methods for generating an Al model.

An Al-Model-Generation method is described with reference to FIG. 6. The Al-Model- Generation 600 may include one or more of steps 601 to 611 described hereinafter (see corresponding blocks in FIG. 6). In an exemplary embodiment, steps 601 to 611 may be performed in a sequential manner. FIG. 6 shows the process flow and input/output dependencies from one step to the next.

The Al-Model-Generation 600 may be divided into three main phases outlining the learning model development process: a data preparation phase comprising steps 601 to 603; a training phase comprising steps 604 to 609; and a testing phase comprising steps 610 to 611. The Al-Model-Generation 600 may ensure the creation of an Al model that balances speed and precision.

Step 601 may include a data acquisition process in which EUS data to be used for training Al models is acquired/collected.

Step 602 may include a data annotation process in which the acquired data is labeled to provide ground truth for the learning process.

Step 603 may include a dataset split process in which the annotated data is divided into several datasets, such as: a training-dataset, a validation-dataset, and a testing-dataset.

Step 604 may include performing Fast-AI-Processing and/or Precise-AI-Processing. Step 604 may act as a decision node selecting the type of Al processing to be performed (Fast-AI- Processing and/or Precise-AI-Processing) function of model requirements.

Step 605 may include selecting the main architecture and/or the backbone of the Al model to be used for feature extraction or pattern learning.

Step 606 may include evaluating different data augmentation techniques and testing various data augmentation methods to improve model robustness and performance by artificially expanding the dataset.

Step 607 may include the selection of data augmentation techniques which provide the most improvements in model performance.

Step 608 may include training the final Al model. The training may be performed by using the selected backbone (at step 605) and data augmentation techniques (at step 607) to train the Al model with the training-dataset (at step 603). A Trained-Al-Model (including trained weights and parameters) is obtained as a result of the training.

Step 609 may include saving the trained weights and parameters of the Trained-Al- Model.

Step 610 may include applying the Al-Trained-Model to the testing-dataset (see step 603) to evaluate the performance of the Al-Trained-Model. The testing-dataset was not used during the training.

Step 611 may include evaluating the performance of the Trained-Al-Model which may include evaluating the outcomes of the model, the accuracy of the model outcomes, the precision of the outcomes, and other relevant metrics. For example, the outcome may be that the patient is positive for a disease / medical condition (e.g., a tumor or cyst is detected by the Al-Trained-Model) or the outcome may be that the patient is negative for a disease of medical condition (e.g., there is no tumor or cysts detected). Examples of metrics for the performance of the Trained-Al-Model are: True positive (TP) which is the number of cases correctly identified as solid tumor/cyst; False positive (FP) which is the number of cases incorrectly identified as solid tumor/cyst; True negative (TN) which is the number of cases correctly identified as normal pancreas; False negative (FN) which is the number of cases incorrectly identified as normal pancreas; Accuracy which is calculated as (TP+TN) / (TP+TN+FP+FN); Recall which is calculated as TP / (TP+FN); and Precision which is calculated as TP / (TP+FP).

The Al-Model-Generation methods disclosed herein may be implemented by the system described with reference to FIGS. 1, 2, 3, 7 and 8. The Al-Model-Generation methods disclosed herein may implement the methods and processes described with reference to FIGS. 1-8. The Al-Model-Generation methods disclosed herein may be included in the methods and processes described with reference to FIGS. 1-8.

(7). AI-Enabled-lmaging-Apparatus.

An AI-Enabled-lmaging-Apparatus 700 is described with reference to FIG. 7 and may include one or more of the following: EUS-Equipment 701 configured to acquired EUS-videos of the pancreas of a patient; an Al-Workstation 702 configured to perform Al processing of the EUS- Videos and to generate Al-processed-data for each of the EUS-videos; an EUS-display 703 configured to display the EUS-videos, received from the EUS-Equipment, so that a Physician can watch the EUS-videos; and an Al-display 704 configured to display the Al-processed-data so that the Physician can visualize the Al-processed-data. The Al-processed-data may include EUS images and videos including annotations and markings made on the videos and images by the Al- Workstation.

The displays 703 and 704 may be disposed so that the Physician can view the EUS-videos and the Al-processed-data simultaneously and/or so that the Physician can easily compare the EUS-data and the Al-processed-data. The Physician may be enabled to watch the EUS-videos and Al-processed-data in essentially "real-time". A method of operating the AI-Enabled-lmaging-Apparatus 700 is described hereinafter. The method incorporates the use of Al processes and includes the use of Al as a complementary tool in medical diagnostics. The method may include one or more of the following steps.

The patient is connected to the EUS-Equipment and an Endoscopic Ultrasound (EUS) procedure is performed on the pancreas of the patient. The EUS-Equipment acquires EUS-videos of the pancreas. The EUS-Equipment may be connected to two outputs. At the first output the EUS-videos are sent to the EUS-Display 703 where they are displayed for viewing (the viewing may be live viewing). At the second output the EUS-videos are sent to the Al-Workstation 702 which performs Al processing and generates Al-processed-data for each of the EUS-videos. The Al-processed-data is displayed on the Al-Display 704. The Al-processed-data may include EUS images and the results of the Al processing and analysis (e.g. labels, annotations, markings on images of the video frames).

The AI-Enabled-lmaging-Apparatus 700 enables the physician to use and view both the regular EUS display showing the EUS-videos (comprising ultrasound images as they are acquired by the EUS-Equipment) and a separate display that shows the results of the Al processing and analysis.

The Physician may view the EUS-videos and the Al-processed-data simultaneously and/or may compare the EUS-data and the Al-processed-data. The Physician may be enabled to watch the EUS-videos and Al-processed-data in essentially "real-time". This way the Physician will be able to provide diagnostic support, to provide second opinions, and/or to assist in identifying areas/regions of interest within the ultrasound images.

The AI-Enabled-lmaging-Apparatus 700 disclosed herein may be implemented by the systems described with reference to FIGS. 1, 2, 3, 7 and 8. The AI-Enabled-lmaging-Apparatus 700 may implement the methods and processes described with reference to FIGS. 1-8.

(8). The Al-lmaging-System.

An Al-lmaging-System 800 is described hereinafter with reference to FIG. 8. The Al- lmaging-System may be used to train and refine Al models, which may be then (after training and refining) deployed back into the clinical setting for practical use. The Al-lmaging-System may be configured to satisfy requirements related to data security and compliance with health data protection standards. The diagram in FIG. 8 outlines the workflow of a continuous development process involving a clinical-site and a development-site for an Al application in a medical setting via the Al-lmaging-System 800.

The Al-lmaging-System 800 may include one or more EUS-Equipment-Modules 801 configured to acquire EUS-images from patients (e.g. pancreas images). The EUS-Equipment- Modules may be manufactured by companies such as Pentax, Olympus, Fuji or any other company. The Al-lmaging-System 800 may include one or more Al-Desktops 802 connected to and receiving EUS-images from the EUS-Equipment-Modules 801. The Al-Desktops 802 may be configured to perform Al processing of the EUS-images such as extracting anatomical features (e.g. cysts, pancreas parts, tumor areas) from the EUS-images.

The Al-Desktops 802 are connected to a Data-Storage-Module 803 which may be in a cloud-storage-system 804. The data on the Data-Storage-Module 803 and the cloud-storage- system 803 may be protected according to the requirements of the Health Insurance Portability and Accountability Act (HIPAA). The Data-Storage-Module 803 may include at least Al annotate and anonymized frames diagnostic-data. The Al-lmaging-System 800 may further include an Al- Application-Update-Module 806.

The Al-lmaging-System 800 may further include an Al-Data-Server 810 which may act as a bridge between the clinical-site and the development-site. The Al-Data-Server 810 may receive Medical-Data (e.g. EUS images, Al processed EUS images, annotated EUS images, data associated with the EUS images) from the Data-Storage-Module 803. The Al-Data-Server 810 may send the Medical-Data to the development-site. The Al-Data-Server 810 may receive New- Al-Weights (e.g. ML updated weights) from the development-site. The diagram in FIG. 8 shows the flow of information between the clinical-site and the development-site via the Al-Data- Server 810.

The Al-lmaging-System 800 may further include a Data-Labeling-Verification-Module 811, a Model-Training-Module 812, a Model-Evaluation-Module 813, and a Model-Validation- Module 814. The modules 811, 812, 813, and 814 may be connected in an iterative process. The modules 811, 812, 813, and 814 may be part of the development-site. A Clinicians- Development-Team may provide feedback and input to the modules 811, 812, 813 and 814. The Al-lmaging-System 800 may further include an Al-Application-Update-Module 815 configured to receive updated / new versions of Al-Models (e.g. new Al weights) from the Model-Validation-Module 814. The Al-Application-Update-Module 815 is further configured to provide the new-versions / updated Al-Models to the clinical site modules. The updated / new versions of Al-Models (e.g. new Al weights) may be used by the Al-Desktops to improve the Al processing of the EUS-images received from the EUS-Equipment 801. The Al-Application- Update-Modules 815 and 806 may be implemented via the same module or via separate modules.

The Al-lmaging-System 800 and/or the modules it includes may be implemented by the systems described with reference to FIGS. 1, 2, 3, and 7. The Al-lmaging-System 800 may implement the methods and processes described with reference to FIGS. 1-8.

(9). Examples of EUS-images and marked-up/annotated EUS-images are described with respect to FIGS. 9-15. FIG 9. shows an exemplary image of normal pancreas detection. FIG 10. shows an exemplary image of the detection and segmentation of the normal pancreas body by using a CNN1 (Convolutional Neural Network 1). FIG 11. shows an exemplary image of the detection and segmentation of the normal pancreas body by using a CNN2 (Convolutional Neural Network 2). FIG 12. shows an exemplary image of the detection and segmentation of a cystic lesion inside the normal pancreas head (CNN1). Figure 13. shows an exemplary image of the detection and segmentation of a cystic lesion inside the normal pancreas head (CNN2). Figure 14. shows an exemplary image of the detection and segmentation of a solid tumor inside the normal pancreas head (CNN1). Figure 15. shows an exemplary image of the detection and segmentation of a solid tumor inside the normal pancreas head (CNN2).

(10). The User-Interface.

A user-interface is described with reference to Figure 16. The user-interface allows users (e.g., clinicians, medical doctors) to use the Al enabled systems of this application (e.g., the systems described with reference to FIGS. 1, 2, 3 and 7) and the Al enabled methods of this application (e.g., the methods described with reference FIGS. 1-14). FIG. 16 shows the userinterface displayed on a screen (e.g., a computer display). The user-interface may be used for diagnosing pancreatic conditions. The displayed user-interface may include a left-panel showing a grayscale EUS-image of a pancreas without Al-processing. The EUS-image may be a frame of a EUS-video or a function of frames (e.g., an average of multiple frames) of a EUS-video. The displayed user-interface may include a right-panel showing the same EUS-image after Al processing. The Al processed EUS-image includes markings and/or annotations, such as color- coded areas, lines, and annotations text. For example, the green shading shows the pancreas area, the red shading shows a region which the Al-enabled methods disclosed herein have identified as a tumor, and the label "tumor 0.89" shows the confidence score (0.89 on a scale from zero to 1) assigned by the Al-enabled-methods for the presence of a tumor. The green area surrounding the red area represents the boundary of the pancreas as identified by the Al.

The user-interface may further include one or more controls (e.g., "Pancreas", "Tumor", and "Cyst") which control whether to display a specific segmentation (e.g., pancreas, tumor, cysts). The user-interface may further include one or more sliders for "Pancreas", "Tumor" and "Cyst" degree of confidence (e.g., on a scale from 0 to 1). The sliders may determine the minimum confidence level for which an area is displayed as a specific anatomical feature / segmentation (e.g., as pancreas, tumor, or cyst).

The user-interface may further include controls (e.g., buttons) enabling the user to select a "Persistence" attribute. The "Persistence" attribute determines how long a region must be continuously present in the EUS-videos in order for it to be shown in the EUS-image displayed on the screen. Depending on the chosen level of "Persistence," the system either displays regions that appear briefly, stay visible for a moderate duration, or require a more extended period of continuous presence before showing them on the screen.

The "Persistence" attribute may include a level "None" for which a region is considered present (and displayed on the screen) even if it appears in just a single frame.

The "Persistence" attribute may include a level "Medium" for which a region is considered present (and displayed on the screen) only if the region is visible in at least a first- number (relatively small) of consecutive frames (e.g. three consecutive frames).

The "Persistence" attribute may include a level "High" for which a region is considered present (and displayed on the screen) only if the region is visible in at least a second-number (relatively large) of consecutive frames (e.g. five consecutive frames). (11). Methods of using and/or operating the systems in this application.

Exemplary embodiments of methods of using and operating the systems and modules described in this application are disclosed hereinafter. The methods may be implemented via the systems and modules described with reference to FIGS. 1, 2, 3, 7 , 8 and 15. The Al- processing methods may be implemented via the methods and processes described with reference to FIGS. 1-15.

In an exemplary embodiment it is disclosed a method for detecting and marking pancreatic lesions (e.g., masses and/or cysts) in EUS-images. The method may comprise one or more of the steps described hereinafter. The method may include receiving one or more EUS- movies and/or EUS images of the pancreas of a patient (e.g. acquired via EUS imaging equipment). The method may include receiving one or more multiparametric images of the pancreas of the patient. The multiparametric images may include one or more of: EUS imaging data, color / power Doppler imaging data, contrast harmonic imaging data, real-time elastography data, CT/MR fusion data, and 3D data.

The method may further include using Al-processing methods to analyze the EUS images and the multiparametric images and to detect pancreatic lesions (e.g. masses, tumors, and/or cysts) and anatomical features / parts of the pancreas. The Al-processing methods may be implemented via the methods and processes described with reference to FIGS. 1-8.

The method may further include marking and/or segmenting the detected pancreatic lesions (e.g. masses, tumors, and/or cysts) and the anatomical features of the pancreas in the EUS images and/or the multi-parametric images. The marking may include displaying certain structures as a color overlay on the gray-scale image (see e.g., FIG. 15 right panel showing the red color dispose over the tumor region and the green color dispose over the pancreas tissue area). One or more of the steps may be performed in real-time.

In an exemplary embodiment it is disclosed a method for determining the type of pancreatic lesions (e.g., masses, tumors, and/or cysts) and anatomical features / parts of the pancreas in EUS-movies and/or EUS images and in multiparametric images of the pancreas of a patient. The method may include receiving at a computer one or more EUS-movies and/or EUS images of the pancreas of a patient (e.g., acquired via EUS imaging equipment). The method may include receiving at the computer one or more multiparametric images of the pancreas of the patient.

The method may further include using Al-processing methods to analyze the EUS images and the multiparametric images and to detect pancreatic lesions (e.g., masses, tumors, and/or cysts) and anatomical features/parts of the pancreas. The Al-processing methods may be implemented via the methods and processes described with reference to FIGS. 1-8.

The method may further include determining the type of the detected focal pancreatic lesions based on the analysis of the EUS and multiparametric images.

In an exemplary embodiment it is disclosed a method for predicting the risk of malignancy of pancreatic lesions (e.g., masses and cysts), as well as normal pancreas, from EUS- movies and/or EUS images and in multiparametric images of the pancreas of a patient.

The method may include receiving at a computer one or more EUS-movies and/or EUS images of the pancreas of a patient (e.g., acquired via EUS imaging equipment). The method may include receiving at the computer one or more multiparametric images of the pancreas of the patient. The method may further include analyzing in real-time each of the images.

The method may further include using Al-processing methods to detect focal pancreatic lesions (e.g., masses and/or cysts) and anatomical features/parts of the pancreas. The Al- processing methods may be implemented via the methods and processes described with reference to FIGS. 1-8.

The method may further include predicting the risk of malignancy for the detected focal pancreatic lesions (solid masses and cystic neoplasms) based on the analysis of gray-scale images (e.g. EUS images not processed by Al) and/or multiparametric EUS images (color / power Doppler imaging, contrast harmonic imaging, real-time elastography, fusion CT/MR imaging, 3D).

In an exemplary embodiment it is disclosed a method including one or more of: suggesting a choice of EUS-guided fine-needle aspiration biopsy (FNAB) and suggesting the performance of further EUS imaging for determining focal pancreatic lesions (solid masses and cystic neoplasms) in EUS images.

The method may further include using Al-processing methods to analyze the images and to detect focal pancreatic lesions (e.g., solid masses and/or neoplastic cysts). The Al-processing methods may be implemented via the methods and processes described with reference to FIGS. 1-8.

The method may further include suggesting a choice of EUS-guided fine-needle aspiration biopsy (FNAB) and/or suggesting the performance of further EUS imaging for determining focal pancreatic lesions (solid masses and cystic neoplasms) in EUS images.

In an exemplary embodiment it is disclosed a method-5 including one or more of the following: using patient demographic data to inform the characterization of focal pancreatic lesions in EUS images.

The method may include receiving at a computer one or more EUS-movies and/or EUS images of the pancreas of a patient (e.g. acquired via EUS imaging equipment). The method may include receiving at the computer one or more multiparametric images of the pancreas of the patient. The method may further include analyzing in real-time each of the images.

The method may include obtaining patient-demographic-data, such as age, weight, race, gender, and preexisting conditions.

The method may further include using Al-processing methods to analyze the images and the patient-demographic-data and based on the analysis to characterize focal pancreatic lesions (e.g. solid masses and/or neoplastic cysts). The Al-processing methods may be implemented via the methods and processes described with reference to FIGS. 1-8.

In an exemplary embodiment it is disclosed an Al enabled method for interpreting images of rapid on-site evaluation (ROSE) cytopathology results. The method may include receiving images from equipment for acquiring cytopathology images from patients during ROSE.

The method may further include performing cytopathology analysis of the images (e.g. by using Al-processing methods) to diagnose focal pancreatic lesions based on cytopathology analysis.

The method may further include suggesting a choice of further management of the patient based on one or more of the following: the analysis of the images, pathology diagnosis and patient demographic data. The method may be compliant with international guidelines.

In an exemplary embodiment it is disclosed an Al enabled method for interpreting the images of macroscopic on-site evaluation (MOSE) histopathology results.

The method may include receiving images from equipment for obtaining histopathology images from patients during MOSE histopathology procedures.

The method may further include performing histopathology analysis of the images (e.g. by using Al-processing methods) to diagnose focal pancreatic lesions based on the histopathology analysis. The histopathology analysis may include performing immunohistochemistry.

In an exemplary embodiment it is disclosed a computer-implemented method. The method may include detecting and marking pancreatic lesions (focal masses and neoplastic cysts) and normal pancreas regions. The method may further include generating a personalized structured report, based on the detection and the marking of the pancreatic lesions and the normal pancreas regions. The personalized structured report may include one or more of the following: text, images, movies. The method may include receiving at a computer one or more EUS-movies and/or EUS images of the pancreas of a patient (e.g., acquired via EUS imaging equipment). The computer may be configured to analyze in real-time each individual EUS image.

The method may further include using Al-processing methods to analyze the images and to detect focal pancreatic lesions (e.g., focal masses and/or neoplastic cysts) and/or normal pancreas regions. The Al-processing methods may be implemented via the methods and processes described with reference to FIGS. 1-8.

The method may further include marking the detected pancreatic lesions (focal masses and neoplastic cysts) in the images and/or marking the normal pancreas regions in the images.

The method may further include determining the characteristics, the size, and the type of the detected pancreatic lesions (focal masses and neoplastic cysts) in the images and/or of the normal pancreas regions in the images. The method may further include predicting the malignancy of the detected pancreatic lesions based on cytopathology results (obtained by ROSE) and/or on histopathology results (obtained by MOSE).

The method may further include designing of a structured report using a personalized large language model (LLM) using text and images / movies acquired via EUS examination (e.g. using EUS equipment). Designing the structured report may include using Al processing.

In an exemplary embodiment it is disclosed a system for detecting and marking pancreatic lesions and cysts. The system may include an endoscopic ultrasound imaging device, a computer configured to perform Al processing / analysis of the images and to detect pancreatic lesions and cysts in the images. The system may be further configured to perform marking of the detected lesions and cysts in the images (e.g., add markings). The system may include a user-input-module enabling a medical doctor to add and edit the markings.

In an exemplary embodiment it is disclosed a method for determining the feasibility of drainage in complex EUS cases. The method may include obtaining EUS images / movies from patients, during conventional EUS examinations, as well as multiparametric EUS imaging (color / power Doppler imaging, contrast harmonic imaging, CT/MR fusion, 3D, etc.).

The method may further include using Al-processing methods to analyze the images and to detect the common bile duct / pancreatic duct in order to establish the feasibility of drainage. The detection and analysis may be performed in real-time. The Al-processing methods may be implemented via the methods and processes described with reference to FIGS. 1-8.

The method may further include tracing the common bile duct / pancreatic duct towards the major / minor papilla, in the EUS movies in real-time, whilst displaying these structures as a color overlay over-imposed on the gray-scale image.

In an exemplary embodiment it is disclosed a method for determining the feasibility of radiofrequency / microwave ablation (RFA/MWA) in solid pancreatic masses. The method may include obtaining EUS movies from patients, during conventional EUS examinations, as well as multiparametric EUS imaging (color / power Doppler imaging, contrast harmonic imaging, CT/MR fusion, 3D, etc.).

The method may further include using Al-processing methods to analyze the images, to detect, and to characterize (differentiate) the solid pancreatic mass in order to establish the feasibility of RFA/MWA. The detection and analysis may be performed in real-time. The Al- processing methods may be implemented via the methods and processes described with reference to FIGS. 1-8.

The method may further include performing an ablation procedure, whilst displaying the target lesion during ablation, in a color overlay over-imposed on the gray-scale image.

The above embodiments presented in this disclosure merely serve as exemplary embodiments and it will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present disclosure. The inventions herein may be embodied in many different forms and should not be construed as limited to the exemplary embodiments set forth herein. Rather, these exemplary embodiments are provided so that this disclosure is thorough, and will fully convey the scope of the invention to those skilled in the art.

The aspects of the invention in this application are not limited to the disclosed operations and sequence of operations. For instance, operations may be performed by various elements and modules, may be consolidated, may be omitted, and may be altered without

800 departing from the spirit and scope of the present invention.

Claims

LIST OF CLAIMS

1. A method for performing medical diagnostics on a patient's pancreas, the method comprising: receiving at a computer one or more endoscopic ultrasound images (EUS-images) of the pancreas; wherein the EUS-images comprise one or more of: images acquired via EUS imaging equipment; multiparametric images; using Al-processing methods to analyze the EUS-images and to detect pancreatic lesions and anatomical features and/or parts of the pancreas, wherein the detected pancreatic lesions comprise one or more of: solid masses, malignant tumors, benign tumors, and cysts; and marking and/or segmenting the detected pancreatic lesions and the anatomical features and/or parts of the pancreas in the EUS-images.

2. The method of claim 1 for performing medical diagnostics on a patient's pancreas, further comprising: determining, based on the Al processing and analysis of the EUS-images, a type of pancreatic focal lesions corresponding to the detected pancreatic lesions.

3. The method of claim 1 for performing medical diagnostics on a patient's pancreas, further comprising: predicting a risk of malignancy for the detected pancreatic lesions; and predicting a risk of malignancy for the detected anatomical features and/or parts of the pancreas based on based on the Al processing and analysis of the EUS-images.

4. The method of claim 1 for performing medical diagnostics on a patient's pancreas, further comprising one or more of: suggesting a choice of EUS-guided fine-needle aspiration biopsy (FNAB); and suggesting the performance of further EUS imaging on the patient for detecting focal pancreatic lesions.

5. The method of claim 1 for performing medical diagnostics on a patient's pancreas, further comprising: receiving at the computer patient-demographic-data comprising one or more of the following: age, weight, race, gender, preexisting conditions of the patient, and medical history of the patient; using Al-processing methods to analyze the EUS-images together with the patientdemographic-data; and characterizing the pancreatic lesions based on the analysis of the EUS-images and the patient-demographic-data.

6. The method of claim 1 for performing medical diagnostics on a patient's pancreas, further comprising: receiving cytopathology-images for the patient, wherein the cytopathology-images are acquired during a rapid on-site evaluation (ROSE) cytopathology procedure; using Al-processing methods to perform cytopathology analysis of the cytopathology- images and to perform diagnosis of the pancreatic lesions based on the cytopathology analysis; receiving at the computer patient-demographic-data comprising one or more of the following: age, weight, race, gender, preexisting conditions of the patient, and medical history of the patient; and suggesting a choice of further management of the patient based on one or more of the following: the analysis of the EUS-images, the analysis of the cytopathology-images and the patient-demographic-data.

7. The method of claim 1 for performing medical diagnostics on a patient's pancreas, further comprising: receiving histopathology-images from equipment for obtaining histopathology-images from patients during macroscopic on-site evaluation (MOSE) histopathology procedures; using Al-processing methods to perform histopathology analysis of the histopathologyimages, to interpret the histopathology-images, and to diagnose focal pancreatic lesions based on the histopathology analysis; receiving at the computer patient-demographic-data; and suggesting a choice of further management of the patient based on one or more of the following: the analysis of the EUS-images, the analysis of the histopathology-images and the patient-demographic-data; wherein the histopathology analysis comprises immunohistochemistry procedures.

8. The method of claim 1 for performing medical diagnostics on a patient's pancreas, further comprising: generating a personalized structured report, based on the detection and the marking of the pancreatic lesions and the anatomical features and/or parts of the pancreas; determining characteristics of the detected pancreatic lesions in the EUS-images and/or characteristics of the normal pancreas regions in the EUS-images predicting the malignancy of the detected pancreatic lesions based on results of MOSE histopathology procedures performed on the patient and/or on ROSE cytopathology procedures performed on the patient; and designing the structured-report using Al-processing methods and/or a personalized large language model (LLM) using text and the EUS-images; wherein the personalized structured report comprises one or more of the following: text, images, and movies.

9. The method of claim 1 for performing medical diagnostics on a patient's pancreas, wherein the marking comprises displaying a first color overlay over a pancreatic lesions and displaying a second color over an anatomical feature of the pancreas;

10. The method of claim 1 for performing medical diagnostics on a patient's pancreas, wherein the detection, the Al-processing, and the marking are performed in real-time.

11. The method of claim 1 for performing medical diagnostics on a patient's pancreas, wherein the multiparametric images comprise one of more of: EUS imaging data, color/ power Doppler imaging data, contrast harmonic imaging data, real-time elastography data, CT/MR fusion data, and 3D data.

12. The method of claim 1 for performing medical diagnostics on a patient's pancreas, further comprising: using Al-processing methods to analyze the EUS-images and to detect a common bile duct and/or a pancreatic duct in order to establish the feasibility of drainage; tracing the common bile duct and/or the pancreatic duct towards the major / minor papilla in the EUS-images whilst displaying the common bile duct and/or the pancreatic duct as color overlays over-imposed on a gray-scale image.

13. The method of claim 1 for performing medical diagnostics on a patient's pancreas, further comprising: using Al-processing methods to characterize the pancreatic lesions and anatomical features and/or parts of the pancreas with the purpose of determining whether radiofrequency and/or microwave ablation (RFA/MWA) procedures are suitable for the patient; evaluating the feasibility of performing radiofrequency / microwave ablation (RFA/MWA) in solid pancreatic masses; and performing an ablation procedure, whilst displaying the target lesion during ablation, in a color overlay over-imposed on a gray-scale image.

14. A system for detecting and marking pancreatic lesions and cysts, the system comprising: an endoscopic ultrasound imaging device configured to acquire EUS-images; a computer configured to perform Al processing and/or analysis of the EUS-images and to detect pancreatic lesions and cysts in the images; and a user-input-module enabling a medical doctor to add and/or edit markings and on the EUS-images and to add annotations to the EUS-images; wherein the system is configured to perform marking of the detected lesions and cysts in the images (e.g., add markings).