KR20230050319A - Systems and methods for artificial intelligence based image analysis for detection and characterization of lesions - Google Patents

Abstract

Presented herein are systems and methods that provide improved detection and characterization of lesions in a subject through automated analysis of nuclear medicine images, such as positron emission tomography (PET) and single photon emission computed tomography (SPECT) images. do. In particular, in certain embodiments, the approaches described herein utilize artificial intelligence (AI) to detect regions of 3D nuclear medicine images that correspond to hotspots indicative of potentially cancerous lesions within a subject. Machine learning modules can be used to detect the presence and locations of these regions in an image, as well as segment the region corresponding to a lesion and/or classify these hotspots based on their likelihood of representing a true underlying cancerous lesion. . Such AI-based lesion detection, segmentation and classification can provide a basis for further characterization of lesions, overall tumor burden, and disease severity and risk estimation.

Description

Systems and methods for artificial intelligence based image analysis for detection and characterization of lesions

[0001] This application is filed with U.S. Provisional Patent Application No. 63/048,436, filed Jul. 6, 2020, and U.S. Provisional Patent Application No. 17/008,411, filed Aug. 31, 2020, Dec. 2020 U.S. Provisional Patent Application No. 63/127,666, filed on June 18, and U.S. Provisional Patent Application No. 63/209,317, filed on June 10, 2021, hereby claim priority to, and the benefit of, each of those applications The contents of are incorporated by reference into this application in their entirety.

[0002] The present invention relates generally to systems and methods for generating, analyzing and/or presenting medical imaging data. More specifically, in certain embodiments, the present invention relates to systems and methods for automated analysis of medical images to identify and/or characterize cancerous lesions.

[0003] Nuclear medicine imaging involves the use of radiolabeled compounds called radiopharmaceuticals. Radiopharmaceuticals are administered to patients and accumulate in various areas of the body in a manner dependent on the biophysical and/or biochemical properties of tissues affected by the presence and/or condition of a disease such as cancer. For example, certain radiopharmaceuticals, after being administered to a patient, accumulate in areas of abnormal bone formation associated with malignant bone lesions that exhibit metastases. Other radiopharmaceuticals may bind to specific receptors, enzymes and proteins in the body that are altered during the evolution of disease. After these molecules are administered to a patient, they circulate in the blood until they find their intended target. The bound radiopharmaceutical remains at the site of disease, while the remainder of the agent is eliminated from the body.

[0004] Nuclear medicine imaging techniques capture images by detecting radiation emitted from a radioactive portion of a radiopharmaceutical. Accumulated radiopharmaceuticals act as beacons so that images depicting disease location and concentration can be obtained using commonly available nuclear medicine techniques. Examples of nuclear medicine imaging techniques include bone scan imaging (also called scintigraphy), single photon emission computed tomography (SPECT) and positron emission tomography (PET). Bone scan, SPECT and PET imaging systems can be found in most hospitals worldwide. The selection of a particular imaging technique depends on and/or is dictated by the particular radiopharmaceutical being used. For example, while technetium 99 m ( ^{99 m} Tc ) labeled compounds are compatible with bone scan imaging and SPECT imaging, PET imaging often uses 18F labeled fluorinated compounds. The compound ^{99 m} Tc methylenediphosphonate ( ^{99 m} Tc MDP) is a popular radiopharmaceutical used in bone scan imaging to detect metastases. Radioactively labeled prostate specific antigen (PSMA) targeting compounds such as ^{99 m} Tc-labeled 1404 and PyL ^TM (also known as [ 18 F]DCFPyL) can be used in conjunction with SPECT and PET imaging, respectively, for highly specific prostate cancer detection. provides the possibility for

[0005] Thus, nuclear medicine imaging is a valuable technique for providing physicians with information that can be used to determine the presence and extent of a patient's disease. Doctors can use this information to provide patients with recommended courses of treatment and to track disease progression.

[0006] For example, an oncologist can determine whether a patient has a particular disease, such as prostate cancer, what stage of the disease is evident, what (if any) recommended course of treatment, whether surgical intervention is involved. In addition, nuclear medicine images can be input and used in patient research in evaluating the possibility of prognosis. The oncologist may use the radiologist's report in this evaluation. A radiologist report is a descriptive evaluation of nuclear medicine images prepared by a radiologist for a physician requesting an imaging study, including, for example, the type of study performed, clinical history, comparisons between images, and information used to conduct the study. It includes techniques, radiologist observations and opinions, as well as overall impressions and recommendations that radiologists may have based on the results of imaging studies. The signed radiologist report is sent to the physician directing the study for physician review, followed by discussion between physician and patient about the results and treatment recommendations.

[0007] Therefore, in this process, the radiologist conducts an imaging study on the patient, analyzes the acquired images, writes a radiologist report, delivers the report to the requesting doctor, and the doctor makes evaluation and treatment recommendations. , and involves the physician communicating the results, recommendations, and risks to the patient. This process may include repeating the imaging study due to inconclusive results or ordering additional tests based on initial results. If an imaging study indicates that a patient has a particular disease or condition (eg, cancer), doctors may choose to do nothing, wait carefully, or adopt an active surveillance approach rather than surgery, as well as a variety of treatment options, including surgery. Discuss the risks of

[0008] Accordingly, the process of reviewing and analyzing various patient images over time plays an important role in cancer diagnosis and treatment. There is a critical need for improved tools that facilitate and enhance the accuracy of image review and analysis for cancer diagnosis and treatment. Improving the toolkit utilized by physicians, radiologists and other medical professionals in this way provides significant improvements in standard care and patient experience.

[0009] Systems and methods that provide improved detection and characterization of lesions in a subject through automated analysis of nuclear medicine images, such as positron emission tomography (PET) and single photon emission computed tomography (SPECT) images, are disclosed. presented in the application. In particular, in certain embodiments, the approaches described herein utilize artificial intelligence (AI) to detect regions of 3D nuclear medicine images that represent potentially cancerous lesions within a subject. In certain embodiments, these areas correspond to localized areas of high intensity relative to the surrounding environment (hotspots) due to increased uptake of the radiopharmaceutical within the lesions. The systems and methods described in this application can use machine learning modules to detect the presence and locations of these hotspots in an image, as well as segment regions corresponding to hotspots and/or analyze the likelihood that they actually correspond to true underlying cancerous lesions. It can be used to classify hotspots based on These AI-based lesion detection, segmentation and classification approaches can provide a basis for further characterization of lesions, overall tumor burden, and disease severity and risk estimation.

[0010] For example, once an imaging hotspot representing lesions is detected, segmented and classified, lesion index values are used to provide a dimension and/or magnitude (e.g., volume) of radiopharmaceutical uptake within the underlying lesion. can be calculated for The calculated lesion index values can be aggregated in turn to provide an overall estimate of tumor burden, disease severity, risk of metastasis, etc. for a subject. In a particular embodiment, lesion index values are calculated by comparing the dimensions of intensities within segmented hotspot volumes to those of specific reference organs, such as liver and aortic segments. By using the reference organs in this way, lesion index values can be measured on a normalized scale that can be compared between images of different subjects. In certain embodiments, the approaches described in this application provide multiple imaging regions corresponding to organs and tissue regions where radiopharmaceuticals accumulate at high levels under normal circumstances, such as kidney, liver, bladder (eg, urinary bladder). It includes a technique for suppressing intensity bleed of Intensities in areas of nuclear medicine images corresponding to these organs are typically high even for normal, healthy subjects, but do not necessarily indicate cancer. Moreover, the high accumulation of radiopharmaceuticals in these organs results in high levels of radioactivity release. The increased emitted radiation may be scattered, resulting in high intensities within areas of nuclear medicine images corresponding to the organs themselves, as well as high intensities outside nearby voxels. This intensity bleed into imaging areas outside and around areas corresponding to organs involved in high uptake can interfere with the detection of nearby lesions and cause inaccuracies in uptake measurements therein. Correction of these intensity bleed effects thus improves the accuracy of lesion detection and quantification.

[0011] In a specific embodiment, the AI-based lesion detection technology described in this application is functional information obtained from nuclear medicine images with anatomical information obtained from anatomical images such as x-ray computed tomography (CT) images. increases For example, the machine learning modules used in the approaches described in this application include a first channel corresponding to a portion of a functional nuclear medicine image (eg, PET image; eg, SPECT image) as follows. Multiple input channels may be received as well as additional channels corresponding to co-aligned anatomical (eg, CT) images and/or portions of anatomical information derived therefrom. Adding the anatomical environment in this way can improve the accuracy of lesion detection approaches. Anatomical information can also be incorporated into lesion classification approaches applied after detection. For example, in addition to calculating lesion index values based on the intensities of detected hotspots, hotspots may be assigned an anatomical label based on their location. For example, the detected hotspots may be locations such as prostate, pelvic lymph nodes, non-pelvic lymph nodes, bone. Alternatively, a label (eg, an alphanumeric label) may be automatically assigned based on whether or not it corresponds to locations in a soft tissue area outside the prostate and lymph nodes.

[0012] In a particular embodiment, detected hotspots and related information, such as computed lesion index values and anatomical labeling, are communicated to allow for review by a medical professional such as a physician, radiologist, technologist, and the like. It is presented as an interactive graphical user interface (GUI). Thus, medical professionals can use the GUI to review and verify the accuracy of the detected hotspots, as well as the corresponding index values and/or anatomical labeling. In certain embodiments, the GUI also allows the user to identify and segment (eg, manually) additional hotspots within the medical image so that the medical professional can identify additional potential lesions that the medical professional believes the automated detection process may have missed. make it possible Once identified, lesion index values and/or anatomical labeling may be determined for manually identified and segmented lesions. Once the user is satisfied with the set of hotspots detected and the information calculated therefrom, approval can be confirmed and a final signed report that can be reviewed and used, for example, to discuss outcomes and diagnosis with the patient and evaluate prognosis and treatment options. can create

[0013] In this way, the approaches described herein provide AI-based tools for lesion detection and analysis that can improve and rationalize the accuracy of assessment of a subject's disease (eg, cancer) status and progression. to provide. This facilitates diagnosis, prognosis and assessment of response to treatment to improve patient outcomes.

[0014] In one aspect, the present invention relates to a method for automatically processing 3D images of a subject to identify and/or characterize (eg, rank) cancerous lesions in the subject, the method comprising: (a) by a processor of a computing device, a functional imaging technique (eg, positron emission tomography (PET); Receiving (eg, and/or accessing) a 3D functional image of the object obtained using, for example, single photon emission computed tomography (SPECT) [eg, where the 3D functional image is obtained by plural includes voxels of , each representing a specific physical volume within the object and having an intensity value (eg, standard absorption value (SUV)) representing detected radiation emitted from the specific physical volume, wherein a plurality of 3D functional images at least some of the voxels in represent a physical volume within the target tissue area]; (b) by the processor, using a machine learning module (eg, a pre-trained machine learning module (eg, having predetermined (eg, fixed) parameters determined through a training procedure)) to perform 3D rendering; Automatically detects one or more hotspots within a functional image by (i) for each hotspot, a hotspot list identifying the hotspot's location (e.g., a list of coordinates (e.g., image coordinates; e.g., physical spatial coordinates); e.g., a mask identifying voxels of the 3D functional image corresponding to the location of the detected hotspot (e.g., center of mass)], and (ii) for each hotspot, a corresponding 3D hotspot within the 3D functional image volume {eg, where the 3D hotspot map is a segmentation map (eg, comprising one or more segmentation masks) identifying for each hotspot voxels within the 3D functional image corresponding to the 3D hotspot volume of each hotspot [e.g., where the 3D hotspot map is an artificial intelligence-based segmentation of a functional image (e.g., a machine learning module that receives at least a 3D functional image as input and generates a 3D hotspot map as output to segment hotspots) used)]; For example, here, the 3D hotspot map includes, for each hotspot, a 3D boundary (eg, an irregular boundary) of the hotspot (eg, a 3D boundary surrounding the 3D hotspot volume, and other voxels of the 3D functional image). generating one or both of the 3D hotspot maps identifying the characteristic voxels of the 3D functional image that make up the 3D hotspot volume from each hotspot, each hotspot corresponding to a local region of high intensity with respect to its surroundings; indicates (eg, points to) a potential cancerous lesion in the subject -}; and (c) storing and/or providing the hotspot list and/or 3D hotspot map for display and/or further processing.

[0015] In a particular embodiment, the machine learning module receives as input at least a portion of a 3D functional image and automatically detects one or more hotspots based at least in part on intensities of voxels of the received portion of the 3D functional image. In a specific embodiment, the machine learning module receives as input a 3D segmentation map identifying one or more volumes of interest (VOIs) within the 3D functional image, each VOI having a specific target tissue region and/or a specific anatomical region within the object [ eg soft tissue areas (eg prostate, lymph nodes, lungs, breast); For example, one or more specific bones; For example, the entire skeleton region].

[0016] In a specific embodiment, the method may, by a processor, an anatomical imaging technique [eg, x-ray computed tomography (CT); For example, magnetic resonance imaging (MRI); receiving a 3D anatomical image of the object obtained using eg, ultrasound], wherein the 3D anatomical image is a graphic representation of a tissue (eg, soft tissue and/or bone) in the object A first input channel corresponding to at least a portion of a 3D anatomical image and a second input channel corresponding to at least a portion of a 3D functional image [eg For example, here, the machine learning module receives the PET image and the CT image as separate channels (e.g., separate channels representing the same volume) (e.g., two colors of color photographic images). Similar to reception by machine learning module of channels (RGB)].

[0017] In a specific embodiment, a machine learning module inputs and receives a 3D segmentation map identifying one or more volumes of interest (VOIs) within the 3D functional image and/or the 3D anatomical image, each VOI having a specific Corresponds to a target tissue region and/or a specific anatomical region. In a particular embodiment, the method includes generating, by a processor, a 3D segmentation map by automatically segmenting a 3D anatomical image.

[0018] In a specific embodiment, the machine learning module is a region specific machine learning module that receives a specific part of a 3D functional image corresponding to one or more specific tissue regions and/or anatomical regions of an object as an input.

[0019] In a particular embodiment, the machine learning module generates as output a list of hotspots [eg, where the machine learning module determines one or more hotspots each based on intensities of voxels of at least some of the 3D functional image. implements a machine learning algorithm (eg, an artificial neural network (ANN)) trained to determine one or more positions (eg, 3D coordinates) corresponding to a position of one of:

[0020] In a particular embodiment, the machine learning module generates a 3D hotspot map as output [eg, where the machine learning module identifies 3D hotspot volumes of the 3D hotspot map (eg, the 3D hotspot map identifies 3D hotspot volumes (eg, surrounded by 3D hotspot boundaries) by depicting a 3D boundary (eg, irregular boundary) for each hotspot) 3D functional image (eg, 3D boundary) implement a machine learning algorithm (eg, an artificial neural network (ANN)) trained to segment (based at least in part on intensities of voxels of the functional image); For example, here, the machine learning module implements, for each voxel of at least a portion of the 3D functional image, a machine learning algorithm trained to determine a hotspot likelihood value representing a likelihood that the voxel will correspond to a hotspot (e.g. For example, step (b) includes performing one or more subsequent post-processing steps, such as setting a threshold to identify 3D hotspot volumes of the 3D hotspot map using the hotspot likelihood values (e.g., 3D hotspot The map identifies, for each hotspot, 3D hotspot volumes (eg, surrounded by the 3D hotspot boundary) by depicting the hotspot's 3D boundary (eg, an irregular boundary).

[0021] In a particular embodiment, the method comprises (d) classifying, by the processor, for each hotspot of at least some of the hotspots, a lesion likelihood corresponding to a likelihood of the hotspot representing a lesion in the subject [e.g., if the hotspot is binary classification indicating whether it is a true lesion; determining a likelihood value on a scale (eg, a floating point value from 0 to 1) representing the likelihood of a hotspot representing, for example, a true lesion].

[0022] In a particular embodiment, step (d) includes using a second machine learning module to determine a lesion likelihood classification for each hotspot of the portion [eg, wherein the machine learning module implement a trained machine learning algorithm to detect hotspots (e.g., to generate a hotspot list and/or a 3D hotspot map as output) and for each hotspot to determine a lesion likelihood classification for the hotspot] . In certain embodiments, step (d) includes determining a lesion likelihood classification for each hotspot using a second machine learning module (eg, a hotspot classification module) [eg, a 3D functional image based at least in part on one or more members selected from the group consisting of intensities of , a hotspot list, a 3D hotspot map, intensities of a 3D anatomical image, and a 3D segmentation map; For example, the second machine learning module may receive one or more inputs corresponding to one or more members selected from the group consisting of intensities of a 3D functional image, a hotspot list, a 3D hotspot map, intensities of a 3D anatomical image, and a 3D segmentation map. receiving channels].

[0023] In a particular embodiment, the method includes determining, by a processor, a set of one or more hotspot features for each hotspot and using the set of one or more hotspot features as an input to a second machine learning module. .

[0024] In a particular embodiment, the method further comprises (e) a subset of, by the processor, one or more hotspots corresponding to hotspots that have a high likelihood of corresponding to cancerous lesions based at least in part on a lesion likelihood classification for the hotspots. selecting a set (eg, for inclusion in a report; eg, for use in calculating one or more risk index values for a subject).

[0025] In a particular embodiment, the method further comprises (f) [prior to step (b)] adjusting, by the processor, intensities of voxels of the 3D functional image to bleed intensity from one or more high-intensity volumes of the 3D functional image. bleed (e.g., cross-talk), wherein each of the one or more high-intensity volumes corresponds to a region of high absorption tissue in the subject associated with high radiopharmaceutical absorption under normal circumstances (must be cancer does not represent). In certain embodiments, step (f) includes correcting the intensity bleed from the plurality of high intensity volumes one at a time in a sequential manner [e.g., from a first high intensity volume to generate a first calibrated image. first adjusting the intensities of voxels of the 3D functional image to correct the intensity bleed, and then adjusting the intensities of the voxels of the first corrected image to correct the intensity bleed, etc. from the second high intensity volume]. In a particular embodiment, the one or more high intensity volumes correspond to one or more highly absorbent tissue regions selected from the group consisting of kidney, liver and bladder (eg, urinary bladder).

[0026] In a particular embodiment, the method further comprises (g) determining, by the processor, for each of at least a portion of one or more hotspots, the size and/or internal (e.g., volume) of the underlying lesion to which the hotspot corresponds. and determining a corresponding lesion index indicative of the radiopharmaceutical absorption level. In a particular embodiment, step (g) includes the intensity (intensities) of one or more voxels associated with a hotspot (eg, at and/or relative to the location of the hotspot; eg, within the volume of the hotspot) ( comparing standard absorption values (eg, corresponding to SUVs) to one or more reference values, each reference value being a specific reference tissue region within the subject (eg, liver; eg, aorta). intensities (e.g., SUV values) of a reference volume (e.g., as an average (e.g., a robust mean, such as the average of interquartile range values)) associated with a portion) and corresponding to a reference tissue region. ) is determined based on In certain embodiments, the one or more reference values include one or more members selected from the group consisting of an aortic reference value relating to an aortic portion of a subject and a liver reference value relating to a liver of a subject.

[0027] In a particular embodiment, for at least one particular reference value associated with a particular reference tissue region, determining the particular reference value is a multi-component mixed model (e.g., a particular reference value for a two-component Gaussian model). fitting the intensities of voxels within a specific reference volume corresponding to a reference tissue region [eg fitting a distribution of intensities of voxels (eg fitting a histogram of voxel intensities)] [ For example, identifying one or more minor peaks in the distribution of voxel intensities, the minor peaks corresponding to voxels associated with normal absorption, and excluding those voxels from determining a reference value (e.g., thereby describes the effects of abnormally low radiopharmaceutical uptake in certain parts of reference tissue regions, such as parts of the liver)].

[0028] In certain embodiments, the method comprises calculating (eg, automatically by a processor) an overall risk index for a subject indicative of a cancer status and/or risk for the subject using the determined lesion index values. includes

[0029] In a particular embodiment, the method is such that for each hotspot, by a processor (eg, automatically), a potential cancerous lesion indicative of the hotspot is located [eg, prostate, pelvic lymph nodes, nasal pelvis within lymph nodes, bone (e.g., bone metastases) and within soft tissue regions not located in the prostate or lymph nodes] [e.g., by a processor (e.g., based on received and/or determined 3D segmentation maps) )] determining an anatomical classification corresponding to a specific anatomical region and/or group of anatomical regions within the object.

[0030] In a particular embodiment, the method includes (h) causing, by the processor, a graphical representation of at least some of the one or more hotspots for review by a user, for display in a graphical user interface (GUI). do. In certain embodiments, the method includes (i) receiving, by a processor, a user selection via a GUI of a subset of the one or more hotspots identified through user review as likely representing underlying cancerous lesions in the subject. include

[0031] In certain embodiments, the 3D functional images include PET or SPECT images obtained after administration of an agent (eg, a radiopharmaceutical; eg, an imaging agent) to a subject. In certain embodiments, the formulation includes a PSMA binder. In certain embodiments, the formulation includes ¹⁸ F. In certain embodiments, the formulation comprises [ 18 F]DCFPyL. In certain embodiments, the formulation includes PSMA-11 (eg, ⁶⁸ Ga-PSMA-11). In certain embodiments, the agent comprises one or more members selected from the group consisting of ^99m Tc, ⁶⁸ Ga, ¹⁷⁷ Lu, ²²⁵ Ac, ¹¹¹ In, ¹²³ I, ¹²⁴ I and ¹³¹ I.

[0032] In a particular embodiment, the machine learning module is a neural network [eg, an artificial neural network (ANN); For example, it implements a convolutional neural network (CNN)].

[0033] In a particular embodiment, the processor is a processor of a cloud based system.

[0034] In another aspect, the present invention relates to a method for automatically processing 3D images of a subject to identify and/or characterize (eg, rank) cancerous lesions in the subject, the method comprising ( a) by a processor of a computing device, a functional imaging technique [eg, positron emission tomography (PET); Receiving (eg, and/or accessing) a 3D functional image of the object obtained using, for example, single photon emission computed tomography (SPECT)] [eg, where the 3D functional image is obtained in a plurality of It includes voxels of, each representing a specific physical volume within the object and having an intensity value representing detected radiation emitted from the specific physical volume, wherein at least some of the plurality of voxels of the 3D functional image are physical in the target tissue region. representing volumes]; (b) by the processor, an anatomical imaging technique [eg, x-ray computed tomography (CT); For example, magnetic resonance imaging (MRI); eg, receiving a 3D anatomical image of the object acquired using ultrasound], wherein the 3D anatomical image includes a graphic representation of a tissue (eg, soft tissue and/or bone) in the object; (c) by the processor, automatically detecting one or more hotspots in the 3D functional image using a machine learning module to: (i) for each hotspot, a hotspot list identifying the location of the hotspot; (ii) each hotspot , a corresponding 3D hotspot volume in the 3D functional image {eg, where the 3D hotspot map is a segmentation map identifying, for each hotspot, voxels in the 3D functional image corresponding to the 3D hotspot volume of each hotspot (eg eg, including one or more segmentation masks) [eg, where the 3D hotspot map is an artificial intelligence-based segmentation of a functional image (eg, receiving at least a 3D functional image as input and a 3D hotspot map as output). using a machine learning module to generate and segment hotspots]; For example, here, the 3D hotspot map includes, for each hotspot, a 3D boundary (eg, an irregular boundary) of the hotspot (eg, a 3D boundary surrounding the 3D hotspot volume, and other voxels of the 3D functional image). generating one or both of the 3D hotspot maps identifying the characteristic voxels of the 3D functional image that make up the 3D hotspot volume from and indicates (eg, points to) a potential cancerous lesion in the subject, the machine learning module receives at least two input channels, the input channels comprising: a first input channel corresponding to at least a portion of the 3D anatomical image and a 3D A second input channel corresponding to at least a portion of the functional image [eg, where the machine learning module receives the PET image and the CT image as separate channels (eg, separate channels representing the same volume) ( eg, similar to reception by a machine learning module of the two color channels (RGB) of a color photographic image)] and/or anatomical information derived from them [eg, one or more a 3D segmentation map identifying volumes of interest (VOIs), each VOI corresponding to a specific target tissue region and/or a specific anatomical region; and (d) storing and/or providing the hotspot list and/or 3D hotspot map for display and/or further processing.

[0035] In another aspect, the present invention relates to a method for automatically processing 3D images of a subject to identify and/or characterize (eg, rank) cancerous lesions in the subject, the method comprising ( a) by a processor of a computing device, a functional imaging technique [eg, positron emission tomography (PET); Receiving (eg, and/or accessing) a 3D functional image of the object obtained using, for example, single photon emission computed tomography (SPECT)] [eg, where the 3D functional image is obtained in a plurality of It includes voxels of, each representing a specific physical volume within the object and having an intensity value representing detected radiation emitted from the specific physical volume, wherein at least some of the plurality of voxels of the 3D functional image are physical in the target tissue region. representing volumes]; (b) a list of hotspots, by the processor, automatically detecting one or more hotspots within the 3D functional image using a first machine learning module to identify, for each hotspot, a location of the hotspot [e.g., where machine learning The module uses a machine learning algorithm (eg, one or more positions (eg, 3D coordinates) each corresponding to a position of one of the one or more hotspots based on intensities of voxels of at least a portion of the 3D functional image). generating a machine learning algorithm (e.g., implementing an artificial neural network (ANN)) trained to determine , wherein each hotspot corresponds to a local area of high intensity to its surroundings and is a potential cancerous lesion in the subject. represents (eg, points to) - (c) by the processor, for each of the one or more hotspots, automatically determining a corresponding 3D hotspot volume in the 3D functional image using a second machine learning module and the hotspot list; , generating a 3D hotspot map [eg, wherein the second machine learning module is based at least in part on a hotspot list along with intensities of voxels of the 3D functional image to identify 3D hotspot volumes of the 3D hotspot map. Implement a machine learning algorithm (e.g., an artificial neural network (ANN)) trained to segment a 3D functional image, wherein the machine learning module, for each voxel of at least a portion of the 3D functional image, the voxel corresponds to a hotspot. implementing a machine learning module algorithm trained to determine a hotspot likelihood value representing a likelihood of performing one or more subsequent post-processing steps such as)], [eg, where the 3D hotspot map corresponds to and/or is based on an output from (eg, a second machine learning module) is a segmentation map (e.g., comprising one or more segmentation masks) that identifies, for each hotspot, the voxels in the 3D functional image corresponding to the 3D hotspot volume of each hotspot, generated using ; For example, here, the 3D hotspot map includes, for each hotspot, a 3D boundary (eg, an irregular boundary) of the hotspot (eg, a 3D boundary surrounding the 3D hotspot volume, and other voxels of the 3D functional image). characteristic voxels of a 3D functional image that create a 3D hotspot volume from )]; and (d) storing and/or providing the hotspot list and/or 3D hotspot map for display and/or further processing.

[0036] In a particular embodiment, the method includes (e) determining, by the processor, for each hotspot of the at least some of the hotspots, a lesion likelihood classification corresponding to a likelihood of the hotspot indicative of a lesion in the subject. In certain embodiments, step (e) includes determining a lesion likelihood classification for each hotspot using a third machine learning module (eg, a hotspot classification module) [eg, a 3D functional image based at least in part on one or more members selected from the group consisting of intensities of , a hotspot list, a 3D hotspot map, intensities of a 3D anatomical image, and a 3D segmentation map; For example, the third machine learning module may receive one or more inputs corresponding to one or more members selected from the group consisting of intensities of a 3D functional image, a hotspot list, a 3D hotspot map, intensities of a 3D anatomical image, and a 3D segmentation map. receiving channels].

[0037] In a particular embodiment, the method further comprises (f) a subset of one or more hotspots that correspond, by the processor, to hotspots that have a high likelihood of corresponding to cancerous lesions based at least in part on a lesion likelihood classification for the hotspots. (eg, for inclusion in a report; eg, for use in calculating one or more risk index values for a subject).

[0038] In another aspect, the present invention refers to preventing effects from tissue regions associated with low (eg, abnormally low) radiopharmaceutical uptake (eg, due to tumors with no tracer uptake). A method of measuring intensity values in a reference volume corresponding to a tissue region (eg, a liver volume associated with a liver of an object), the method comprising: (a) by a processor of a computing device, a functional imaging technique [eg, For example, positron emission tomography (PET); Receiving (eg, and/or accessing) a 3D functional image of the object obtained using, for example, single photon emission computed tomography (SPECT)] [eg, wherein the 3D functional image is obtained by plural It includes voxels of, each representing a specific physical volume within the object and having an intensity value representing detected radiation emitted from the specific physical volume, wherein at least some of the plurality of voxels of the 3D functional image are physical in the target tissue region. represents the volume]; (b) identifying, by the processor, a reference volume within the 3D functional image; (c) fitting, by the processor, the intensities of voxels in the reference volume to a multi-component mixture model (eg, a two-component Gaussian mixture model) [multiple to the distribution (eg, histogram) of intensities of voxels in the reference volume; fitting component mixture models]; (d) identifying, by the processor, dominant modes of the multi-component model; (e) determining, by the processor, a dimension (eg, mean, maximum, mode, median, etc.) of intensities corresponding to a principal mode that (i) are within the reference tissue volume and (ii) are associated with the principal mode. determining a reference intensity value corresponding to a dimension of the intensity of (and eg, excluding voxels having intensities associated with minor modes in the reference value calculation) (eg, thereby tissue associated with low radiopharmaceutical uptake); prevent influence from areas); (f) detecting, by the processor, one or more hotspots corresponding to potentially cancerous lesions within the functional image; and (g) by the process, for each hotspot of at least some of the detected hotspots, a lesion index value using at least the reference intensity value [e.g., a dimension of intensities of voxels corresponding to the detected hotspot and (ii) ) lesion index value based on the reference intensity value].

[0039] In another aspect, the invention relates to intensity bleed (e.g., crosstalk (e.g., not necessarily indicative of cancer) due to areas of high absorption tissue in a subject associated with high absorption of radiopharmaceuticals under normal circumstances). cross-talk), the method comprising: (a) by a processor of a computing device, a functional imaging technique [eg, positron emission tomography (PET); Receiving (eg, and/or accessing) a 3D functional image of the object obtained using, eg, single photon emission computed tomography (SPECT)] [eg, where the 3D functional image is obtained by plural It includes voxels of, each representing a specific physical volume within the object and having an intensity value representing detected radiation emitted from the specific physical volume, wherein at least some of the plurality of voxels of the 3D functional image are physical in the target tissue region. represents the volume]; (b) identifying, by the processor, a high-intensity volume within the 3D functional image, wherein the high-intensity volume is a specific high-absorption tissue region where high radiopharmaceutical uptake occurs under normal circumstances (eg, kidney; eg, liver; eg, bladder) -; (c) identifying, by the processor, a suppression volume in the 3D functional image based on the identified high-intensity volume, the suppression volume being within and outside a predetermined attenuation distance from the boundary of the identified high-intensity volume; Corresponds to ; (d) determining, by a processor, a background image corresponding to the 3D functional image with intensities of voxels in the high-intensity volume replaced with interpolated values determined based on intensities of voxels of the 3D functional image in the suppression volume; (e) determining, by a processor, an estimated image by subtracting intensities of voxels in the background image from intensities of voxels in the 3D functional image (eg, by performing voxel-by-voxel subtraction); (f) extrapolating, by a processor, intensities of voxels of an estimated image corresponding to the high-intensity volume to locations of voxels within the suppression volume to determine intensities of voxels of a suppression map corresponding to the suppression volume; determining a suppression map by setting intensities of voxels of the suppression map corresponding to the extrinsic locations to zero; and (g) the high intensity volume by the processor to adjust intensities of voxels of the 3D functional image based on the suppression map (eg, by subtracting intensities of voxels of the suppression map from intensities of voxels of the 3D functional image). and correcting the intensity bleed from

[0040] In a particular embodiment, the method includes performing, for each of a plurality of high intensity volumes, steps (b) to (g) in a sequential manner to correct intensity bleed from each of a plurality of high intensity volumes. includes

[0041] In a particular embodiment, the plurality of high intensity volumes include one or more members selected from the group consisting of kidney, liver and bladder (eg, urinary bladder).

[0042] In another aspect, the present invention relates to a method for automatically processing 3D images of a subject to identify and/or characterize (eg, rank) cancerous lesions in the subject, the method comprising: a) by a processor of a computing device, a functional imaging technique [eg, positron emission tomography (PET); Receiving (eg, and/or approaching) a 3D functional image of the object obtained using, for example, single photon emission computed tomography (SPECT)] [eg, where the 3D functional image is obtained by plural It includes voxels of, each representing a specific physical volume within the object and having an intensity value representing detected radiation emitted from the specific physical volume, wherein at least some of the plurality of voxels of the 3D functional image are physical in the target tissue region. represents the volume]; (b) automatically detecting, by the processor, one or more hotspots in the 3D functional image, each hotspot corresponding to a local area of high intensity to its surroundings and representing a potential cancerous lesion in the subject (e.g., pointing) ― ; (c) causing, by the processor, rendering of a graphical representation of the one or more hotspots for display in an interactive graphical user interface (GUI) (eg, a quality management and reporting GUI); (d) receiving, by the processor, a user selection (e.g., report, report) of a final set of hotspots that includes at least a portion (e.g., up to all) of the one or more automatically detected hotspots. for inclusion in); and (e) storing and/or providing the final set of hotspots for display and/or further processing.

[0043] In a particular embodiment, the method includes (f) receiving, by a processor, via a GUI a user selection of one or more additional user-identified hotspots for inclusion in a final set of hotspots; and (g) updating, by the processor, the last hotspot configured to include one or more additional user identification hotspots.

[0044] In a particular embodiment, step (b) includes using one or more machine learning modules.

[0045] In another aspect, the invention relates to a method for automatically processing 3D images of a subject to identify and characterize (eg, rank) cancerous lesions in the subject, the method comprising (a) Functional imaging techniques [eg, positron emission tomography (PET); Receiving (eg, and/or accessing) a 3D functional image of the object obtained using, for example, single photon emission computed tomography (SPECT)] [eg, where the 3D functional image is obtained in a plurality of It includes voxels of, each representing a specific physical volume within the object and having an intensity value representing detected radiation emitted from the specific physical volume, wherein at least some of the plurality of voxels of the 3D functional image are physical in the target tissue region. represents the volume]; (b) automatically detecting, by the processor, one or more hotspots in the 3D functional image, each hotspot corresponding to a local area of high intensity to its surroundings and representing a potential cancerous lesion in the subject (e.g., pointing) ― ; (c) for each hotspot, by the processor, a potentially cancerous lesion indicative of the hotspot is located [eg, in the prostate, pelvic lymph nodes, non-pelvic lymph nodes, in bone (eg, areas of bone metastasis) and in the prostate or determining an anatomical classification corresponding to a specific anatomical region and/or group of anatomical regions within the determined object [eg, by a processor (eg, by a processor (eg, receive and/or or based on the determined 3D segmentation map)]; and (d) storing and/or providing for each hotspot an identification of one or more hotspots along with an anatomical classification corresponding to the hotspot for display and/or further processing.

[0046] In a particular embodiment, step (b) includes using one or more machine learning modules.

[0047] In another aspect, the present invention relates to a system for automatically processing 3D images of a subject to identify and/or characterize (eg, grade) cancerous lesions in the subject, the system comprising a computing device of the processor; and a memory having stored therein instructions, which when executed by the processor cause the processor to: (a) by the processor of the computing device, a functional imaging technique [e.g., positron emission tomography (PET); ); receive (eg, and/or access) a 3D functional image of an object obtained using, for example, single photon emission computed tomography (SPECT)] and [eg, wherein the 3D functional image is obtained by multiple includes voxels of , each representing a specific physical volume within the object and having an intensity value (eg, standard absorption value (SUV)) representing detected radiation emitted from the specific physical volume, wherein a plurality of 3D functional images at least some of the voxels in represent a physical volume within the target tissue area]; (b) by the processor, using a machine learning module [eg, a pre-trained machine learning module [eg, having predetermined (eg, fixed) parameters determined through a training procedure]] to automatically detect one or more hotspots within a functional image, such that (i) for each hotspot, a hotspot list identifying the hotspot's location [e.g., a list of coordinates (e.g., image coordinates; e.g., physical spatial coordinates); For example, a mask identifying respective voxels of the 3D functional image corresponding to the location of the detected hotspot (eg, center of mass)], and (ii) for each hotspot, a corresponding mask in the 3D functional image 3D hotspot volume {e.g., where the 3D hotspot map is a segmentation map (e.g., comprising one or more segmentation masks) identifying, for each hotspot, voxels within the 3D functional image corresponding to the 3D hotspot volume of each hotspot. [e.g., where the 3D hotspot map is an artificial intelligence-based segmentation of a functional image (e.g., machine learning that receives at least a 3D functional image as input and generates a 3D hotspot map as output to segment hotspots). Obtained via); For example, here, the 3D hotspot map includes, for each hotspot, a 3D boundary (eg, an irregular boundary) of the hotspot (eg, a 3D boundary surrounding the 3D hotspot volume, and other voxels of the 3D functional image). generate one or both of the 3D hotspot maps that identify the characteristic voxels of the 3D functional image that make up the 3D hotspot volume from indicate (eg, point to) potentially cancerous lesions in the subject}; and (c) store and/or provide hotspot lists and/or 3D hotspot maps for display and/or further processing.

[0048] In a particular embodiment, the machine learning module receives as input at least a portion of a 3D functional image and automatically detects one or more hotspots based at least in part on intensities of voxels of the received portion of the 3D functional image.

[0049] In a particular embodiment, the machine learning module receives as input a 3D segmentation map identifying one or more volumes of interest (VOIs) within a 3D functional image, each VOI having a specific target tissue region and/or specific anatomy within an object. enemy areas [eg, soft tissue areas (eg, prostate, lymph nodes, lungs, breast); For example, one or more specific bones; For example, the entire skeleton region].

[0050] In a particular embodiment, the instructions may cause the processor to perform an anatomical imaging technique [eg, x-ray computed tomography (CT); For example, magnetic resonance imaging (MRI); For example, to receive (and/or access) a 3D anatomical image of an object acquired using ultrasound]. Here, the 3D anatomical image includes a graphical representation of a tissue (eg, soft tissue and/or bone) in the object, and the machine learning module receives at least two input channels, the input channels representing the 3D anatomical image. A first input channel corresponding to at least a portion and a second input channel corresponding to at least a portion of the 3D functional image [eg, here, the machine learning module converts the PET image and the CT image into separate channels (eg, the same receiving with separate channels representing the volume (e.g., similar to receiving by a machine learning module of two color channels (RGB) of a color photographic image)].

[0051] In a particular embodiment, a machine learning module receives as input a 3D segmentation map identifying one or more volumes of interest (VOIs) within the 3D functional image and/or the 3D anatomical image, each VOI having a specific Corresponds to a target tissue region and/or a specific anatomical region.

[0052] In a particular embodiment, the instructions cause a processor to automatically segment a 3D anatomical image to create a 3D segmentation map.

[0053] In a specific embodiment, the machine learning module is a region specific machine learning module that receives a specific part of a 3D functional image corresponding to one or more specific tissue regions and/or anatomical regions of an object as an input.

[0054] In a particular embodiment, the machine learning module generates as output a list of hotspots [eg, where the machine learning module identifies one or more hotspots each based on intensities of voxels of at least some of the 3D functional images. implements a machine learning algorithm (eg, an artificial neural network (ANN)) trained to determine one or more positions (eg, 3D coordinates) corresponding to a position of one of

[0055] In a particular embodiment, the machine learning module generates a 3D hotspot map as output [eg, where the machine learning module identifies 3D hotspot volumes of the 3D hotspot map (eg, the 3D hotspot map 3D functional image (eg, to identify 3D hotspot volumes (eg, surrounded by 3D hotspot boundaries)) by delineating a 3D boundary (eg, an irregular boundary) for each hotspot. implement a machine learning algorithm (eg, an artificial neural network (ANN)) trained to segment (based at least in part on intensities of voxels of the 3D functional image); For example, here, the machine learning module implements, for each voxel of at least a portion of the 3D functional image, a machine learning algorithm trained to determine a hotspot likelihood value representing a likelihood that the voxel will correspond to a hotspot (eg, Step (b) uses the hotspot likelihood values to identify 3D hotspot volumes of a 3D hotspot map (e.g., the 3D hotspot map depicts, for each hotspot, a 3D boundary (e.g., an irregular boundary) of the hotspot). and performing one or more subsequent post-processing steps, such as setting a threshold (eg, to identify 3D hotspot volumes surrounded by a 3D hotspot boundary)].

[0056] In a particular embodiment, the instructions cause the processor to: (d) for each hotspot of at least some of the hotspots, classify a lesion likelihood corresponding to a likelihood of the hotspot representing a lesion in the subject [e.g., whether the hotspot is a true lesion; binary classification indicating whether or not; For example, a likelihood value on a scale (eg, a floating point value from 0 to 1) representing the likelihood of a hotspot representing a true lesion] is determined.

[0057] In a particular embodiment, in step (d), the instructions cause the processor to use a machine learning module to determine a lesion likelihood classification for each hotspot of the portion [eg, wherein the machine learning module implement a trained machine learning algorithm to detect hotspots (e.g., to generate a hotspot list and/or a 3D hotspot map as output) and for each hotspot to determine a lesion likelihood classification for the hotspot] .

[0058] In a particular embodiment, the instructions in step (d) cause the processor to determine a lesion likelihood classification for each hotspot using a second machine learning module (eg, a hotspot classification module) [eg, a hotspot classification module]. , intensities of a 3D functional image, a hotspot list, a 3D hotspot map, intensities of a 3D anatomical image, and a 3D segmentation map; For example, the second machine learning module may receive one or more inputs corresponding to one or more members selected from the group consisting of intensities of a 3D functional image, a hotspot list, a 3D hotspot map, intensities of a 3D anatomical image, and a 3D segmentation map. receiving channels].

[0059] In a particular embodiment, the instructions cause a processor to determine a set of one or more hotspot characteristics for each hotspot and use the set of one or more hotspot characteristics as input to the second machine learning module.

[0060] In a particular embodiment, the instructions cause the processor to (e) select a subset of one or more hotspots that correspond to hotspots with a high likelihood to correspond to cancerous lesions based at least in part on a lesion likelihood classification for the hotspots. (eg, for inclusion in a report; eg, for use in calculating one or more risk index values for a subject).

[0061] In a particular embodiment, the instructions cause the processor to (f) (prior to step (b)) adjust the intensities of voxels of the 3D functional image by the processor to bleed the intensity from one or more high-intensity volumes of the 3D functional image ( eg, crosstalk), each of the one or more high-intensity volumes corresponds to an area of high absorption tissue in the subject associated with high absorption of radiopharmaceuticals under normal circumstances (not necessarily indicative of cancer).

[0062] In a particular embodiment, in step (f), the instructions cause the processor to correct intensity bleed from the plurality of high-intensity volumes one at a time in a sequential fashion (e.g., a first corrected image first adjusting the intensities of voxels of the 3D functional image to correct intensity bleed from the first high intensity volume to generate, then adjusting the intensities of voxels of the first corrected image to correct intensity bleed etc. from the second high intensity volume box].

[0063] In a particular embodiment, the one or more high intensity volumes correspond to one or more highly absorbent tissue regions selected from the group consisting of kidney, liver and bladder (eg, urinary bladder).

[0064] In a particular embodiment, the instructions may cause the processor to (g) for each of at least a portion of the one or more hotspots, the size and/or interior (e.g., volume) of the radiopharmaceutical uptake of the underlying lesion to which the hotspot corresponds. Determine the corresponding lesion index representing the level.

[0065] In a particular embodiment, the instructions in step (g) direct the processor to one or more voxels associated with a hotspot (eg, at and/or about the location of the hotspot; eg, within a volume of the hotspot). to compare the intensity (intensities) of (e.g., corresponding to standard absorption values (SUVs)) with one or more reference values, each reference value being a specific reference tissue region within the subject (e.g., liver; e.g. Intensities (e.g., SUVs) of a reference volume [e.g., as an average (e.g., a robust average, such as the average of interquartile range values)] associated with a portion of the aorta and corresponding to a reference tissue region. values).

[0066] In a particular embodiment, the one or more reference values include one or more members selected from the group consisting of an aortic reference value relating to an aortic portion of a subject and a liver reference value relating to a liver of a subject.

[0067] In a particular embodiment, for at least one particular reference value associated with a particular reference tissue area, the instructions cause the processor to: By fitting the intensities of voxels within a particular reference volume [eg, by fitting a distribution of intensities of voxels (eg, fitting a histogram of voxel intensities)] to determine a particular reference value [eg, voxels identifying one or more sub-peaks in the distribution of intensities, which sub-peaks correspond to voxels associated with normal uptake, and excluding those voxels from the reference value determination (eg, thereby a reference tissue region such as liver sections) describes the effects of abnormally low absorption of radiopharmaceuticals in certain parts of the spectrum)].

[0068] In a particular embodiment, the instructions cause the processor to use the determined lesion index values to calculate (eg, automatically) an overall risk index for the subject indicative of cancer status and/or risk for the subject.

[0069] In a particular embodiment, the instructions cause the processor to, for each hotspot, locate a potential cancerous lesion indicative of the hotspot (e.g., prostate, pelvic lymph node, non-pelvic lymph node, bone (e.g., bone metastasis area) ) and within soft tissue regions not located in the prostate or lymph nodes] determined [eg, by a processor (eg, based on received and/or determined 3D segmentation maps)] specific anatomical regions within the subject and/or Determine (eg automatically) an anatomical classification corresponding to a group of anatomical regions.

[0070] In a particular embodiment, the instructions cause the processor to cause (h) rendering of a graphical representation of at least a portion of the one or more hotspots for review by a user, for display in a graphical user interface (GUI).

[0071] In a particular embodiment, the instructions cause the processor to (i) receive, via a GUI, a user selection of a subset of the one or more hotspots identified through user review as likely representing underlying cancerous lesions in the subject.

[0072] In certain embodiments, the 3D functional images include PET or SPECT images obtained after administration of an agent (eg, a radiopharmaceutical; eg, an imaging agent) to a subject. In certain embodiments, the formulation includes a PSMA binder. In certain embodiments, the formulation includes ¹⁸ F. In certain embodiments, the formulation comprises [ 18 F]DCFPyL. In certain embodiments, the formulation includes PSMA-11 (eg, ⁶⁸ Ga-PSMA-11). In certain embodiments, the agent comprises one or more members selected from the group consisting of ^99m Tc, ⁶⁸ Ga, ¹⁷⁷ Lu, ²²⁵ Ac, ¹¹¹ In, ¹²³ I, ¹²⁴ I and ¹³¹ I.

[0073] In a particular embodiment, the machine learning module is a neural network [eg, an artificial neural network (ANN); For example, it implements a convolutional neural network (CNN)].

[0074] In a particular embodiment, the processor is a processor of a cloud-based system.

[0075] In another aspect, the present invention relates to a system for automatically processing 3D images of a subject to identify and/or characterize (eg, rank) cancerous lesions in the subject, the system comprising computing the device's processor; and a memory having stored thereon instructions, which when executed by the processor cause the processor to: (a) a functional imaging technique [eg, positron emission tomography (PET); receive (eg, and/or access) a 3D functional image of an object obtained using, for example, single photon emission computed tomography (SPECT)] and [eg, wherein the 3D functional image is obtained by multiple It includes voxels of, each representing a specific physical volume within the object and having an intensity value representing detected radiation emitted from the specific physical volume, wherein at least some of the plurality of voxels of the 3D functional image are physical in the target tissue region. representing volumes]; (b) anatomical imaging techniques [eg, x-ray computed tomography (CT); For example, magnetic resonance imaging (MRI); eg, ultrasound] to receive (eg, and/or access) a 3D anatomical image of an object acquired using a 3D anatomical image of a tissue (eg, soft tissue and/or or bones); (c) using a machine learning module to automatically detect one or more hotspots within the 3D functional image such that (i) for each hotspot, a hotspot list identifying the hotspot's location; (ii) for each hotspot, a 3D The corresponding 3D hotspot volume in the functional image {eg, where the 3D hotspot map is a segmentation map (eg, one including the above segmentation masks) [e.g., where the 3D hotspot map is an artificial intelligence-based segmentation of a functional image (e.g., receiving at least a 3D functional image as input and generating a 3D hotspot map as output to generate a hotspot Obtained through using a machine learning module that divides ]; For example, here, the 3D hotspot map includes, for each hotspot, a 3D boundary (eg, an irregular boundary) of the hotspot (eg, a 3D boundary surrounding the 3D hotspot volume, and other voxels of the 3D functional image). create one or both of the 3D hotspot maps that identify the characteristic voxels of the 3D functional image that make up the 3D hotspot volume from and indicates (eg, points to) a potential cancerous lesion in the subject, the machine learning module receives at least two input channels, the input channels comprising: a first input channel corresponding to at least a portion of the 3D anatomical image and a 3D A second input channel corresponding to at least a portion of the functional image [eg, where the machine learning module receives the PET image and the CT image as separate channels (eg, separate channels representing the same volume) ( eg, similar to reception by a machine learning module of the two color channels (RGB) of a color photographic image)] and/or anatomical information derived from them [eg, one or more a 3D segmentation map identifying volumes of interest (VOIs), where each VOI corresponds to a specific target tissue region and/or a specific anatomical region; (d) store and/or provide hotspot lists and/or 3D hotspot maps for display and/or further processing;

[0076] In another aspect, the present invention is directed to a system for automatically processing 3D images of a subject to identify and/or characterize (eg, rank) cancerous lesions in the subject, the system comprising computing the device's processor; and a memory having stored thereon instructions, which when executed by the processor cause the processor to: (a) a functional imaging technique [eg, positron emission tomography (PET); receive (eg, and/or access) a 3D functional image of an object obtained using, for example, single photon emission computed tomography (SPECT)] and [eg, wherein the 3D functional image is obtained by multiple It includes voxels of, each representing a specific physical volume within the object and having an intensity value representing detected radiation emitted from the specific physical volume, wherein at least some of the plurality of voxels of the 3D functional image are physical in the target tissue region. representing volumes]; (b) use a first machine learning module to automatically detect one or more hotspots within the 3D functional image, so that for each hotspot, a hotspot list identifying the location of the hotspot [e.g., wherein the machine learning module is a machine A learning algorithm (eg, to determine one or more locations (eg, 3D coordinates) each corresponding to a location of one of the one or more hotspots based on intensities of voxels of at least some of the 3D functional image) implement a trained machine learning algorithm (e.g., an artificial neural network (ANN))], each hotspot corresponds to a local area of high intensity to its surroundings and represents a potential cancerous lesion in the subject ( (e.g., pointing to); (c) for each of the one or more hotspots, use a second machine learning module and the hotspot list to automatically determine a corresponding 3D hotspot volume in the 3D functional image to generate a 3D hotspot map [ For example, where the second machine learning module performs a machine learning algorithm (eg, based at least in part on a hotspot list along with intensities of voxels of the 3D functional image to identify 3D hotspot volumes of the 3D hotspot map). An artificial neural network (ANN) trained to segment a functional image; e.g., wherein the machine learning module determines, for each voxel of at least a portion of the 3D functional image, a hotspot likelihood value indicating the likelihood that the voxel corresponds to a hotspot. (e.g., step (b) performs one or more subsequent post-processing steps, such as setting a threshold to identify 3D hotspot volumes of the 3D hotspot map using the hotspot likelihood values) [e.g., wherein a 3D hotspot map is generated using (eg, corresponding to and/or based on an output from) a second machine learning module to map each hotspot. a segmentation map (eg, comprising one or more segmentation masks) identifying voxels in the 3D functional image corresponding to the 3D hotspot volume of each hotspot for ; For example, here, the 3D hotspot map includes, for each hotspot, a 3D boundary (eg, an irregular boundary) of the hotspot (eg, a 3D boundary surrounding the 3D hotspot volume, and other voxels of the 3D functional image). depicting characteristic voxels of a 3D functional image) creating a 3D hotspot volume from]; and (d) store and/or provide hotspot lists and/or 3D hotspot maps for display and/or further processing.

[0077] In a particular embodiment, the instructions cause the processor to (e) determine, for each hotspot of the at least some of the hotspots, a lesion likelihood classification corresponding to a likelihood of the hotspot representing a lesion in the subject.

[0078] In a particular embodiment, in step (e), the instructions cause the processor to determine a lesion likelihood classification for each hotspot using a third machine learning module (eg, a hotspot classification module) [eg, based at least in part on one or more members selected from the group consisting of, eg, intensities of a 3D functional image, a hotspot list, a 3D hotspot map, intensities of a 3D anatomical image, and a 3D segmentation map; For example, the third machine learning module may receive one or more inputs corresponding to one or more members selected from the group consisting of intensities of a 3D functional image, a hotspot list, a 3D hotspot map, intensities of a 3D anatomical image, and a 3D segmentation map. receiving channels].

[0079] In a particular embodiment, the instructions cause the processor to (f) select a subset of one or more hotspots that correspond to hotspots with a high likelihood to correspond to cancerous lesions based at least in part on a lesion likelihood classification for the hotspots. (eg, for inclusion in a report; eg, for use in calculating one or more risk index values for a subject).

[0080] In another aspect, the present invention refers to preventing effects from tissue areas associated with low (eg, abnormally low) radiopharmaceutical uptake (eg, due to tumors with no tracer uptake). A system for measuring intensity values in a reference volume corresponding to a tissue region (eg, a liver volume associated with a liver of a subject), the system comprising: a processor of a computing device; and a memory in which instructions are stored, wherein when the instructions are executed by the processor, the processor causes (a) a functional imaging technique [eg, positron emission tomography (PET); receive (eg, and/or access) a 3D functional image of an object obtained using, for example, single photon emission computed tomography (SPECT)] and [eg, wherein the 3D functional image is obtained by multiple It includes voxels of, each representing a specific physical volume within the object and having an intensity value representing detected radiation emitted from the specific physical volume, wherein at least some of the plurality of voxels of the 3D functional image are physical in the target tissue region. represents the volume]; (b) to identify a reference volume within the 3D functional image; (c) to fit a multi-component mixing model (e.g., a two-component Gaussian mixing model) to the intensities of voxels in the reference volume [multi-component mixing to the distribution (e.g., histogram) of intensities of voxels in the reference volume fitting the model]; (d) identify the dominant modes of the multi-component model; (e) to determine the dimension (e.g., mean, maximum, mode, median, etc.) of the intensities corresponding to the principal mode, so that voxels (i) within the reference tissue volume and (ii) associated with the principal mode (and example Determining a reference intensity value corresponding to a dimension of the intensity (eg, excluding voxels having intensities associated with minor modes in the reference value calculation) (eg, thereby from tissue regions associated with low radiopharmaceutical uptake). prevent the effect of); (f) detect one or more hotspots corresponding to potentially cancerous lesions within the 3D functional image; and (g) for each hotspot of at least some of the detected hotspots, a lesion index value using at least a reference intensity value [e.g., (i) a dimension of intensities of voxels corresponding to the detected hotspot and (ii) lesion index value based on the reference intensity value] is determined. In another aspect, the present invention provides a method for correcting intensity bleed (e.g., crosstalk) due to areas of high absorption tissue in a subject associated with high absorption of radiopharmaceuticals under normal circumstances (e.g., not necessarily indicative of cancer). It relates to a system for, the method comprising (a) functional imaging techniques [eg, positron emission tomography (PET); Receiving (eg, and/or accessing) a 3D functional image of the object obtained using, for example, single photon emission computed tomography (SPECT)] [eg, where the 3D functional image is obtained in a plurality of It includes voxels of, each representing a specific physical volume within the object and having an intensity value representing detected radiation emitted from the specific physical volume, wherein at least some of the plurality of voxels of the 3D functional image are physical in the target tissue region. represents the volume]; (b) identifying a high-intensity volume in the 3D functional image - the high-intensity volume is a specific high-absorption tissue region where high radiopharmaceutical uptake occurs under normal circumstances (eg, kidney; eg, liver; eg, Bladder) corresponding to ; (c) identifying a suppression volume in the 3D functional image based on the identified high-intensity volume, the suppression volume corresponding to a volume within and outside a predetermined attenuation distance from the boundary of the identified high-intensity volume; (d) determining a background image corresponding to the 3D functional image having intensities of voxels in the high-intensity volume replaced with interpolated values determined based on intensities of voxels of the 3D functional image in the suppression volume; (e) determining an estimated image by subtracting intensities of voxels in the background image from intensities of voxels in the 3D functional image (eg, by performing voxel-by-voxel subtraction); (f) extrapolate intensities of voxels of an estimated image corresponding to the high-intensity volume to locations of voxels within the suppression volume to determine intensities of voxels of the suppression map corresponding to the suppression volume, and to locations outside the suppression volume. determining a suppression map by setting intensities of voxels of the corresponding suppression map to zero; and (g) adjusting intensities of voxels of the 3D functional image based on the suppression map (eg, by subtracting intensities of voxels of the suppression map from those of the 3D functional image) to obtain an intensity bleed from the high-intensity volume. It includes a correction step.

[0081] In a particular embodiment, the instructions cause a processor to perform, for each of a plurality of high intensity volumes, steps (b) through (g) in a sequential manner to correct intensity bleed from each of a plurality of high intensity volumes. do.

[0082] In a particular embodiment, the plurality of high intensity volumes include one or more members selected from the group consisting of kidney, liver and bladder (eg, urinary bladder).

[0083] In another aspect, the present invention relates to a system for automatically processing 3D images of a subject to identify and/or characterize (eg, grade) cancerous lesions in the subject, the system comprising a computing device of the processor; and a memory having stored thereon instructions, which when executed by the processor cause the processor to: (a) functional imaging techniques [eg, positron emission tomography (PET); receive (eg, and/or access) a 3D functional image of an object obtained using, for example, single photon emission computed tomography (SPECT)] and [eg, wherein the 3D functional image is obtained by multiple It includes voxels of, each representing a specific physical volume within the object and having an intensity value representing detected radiation emitted from the specific physical volume, wherein at least some of the plurality of voxels of the 3D functional image are physical in the target tissue region. represents the volume]; (b) automatically detect one or more hotspots within the 3D functional image, each hotspot corresponding to its surroundings a local area of high intensity and representing (eg, pointing to) a potential cancerous lesion in the subject; (c) cause rendering of a graphical representation of one or more hotspots for display within an interactive graphical user interface (GUI) (eg, a quality management and reporting GUI); (d) receive via the interactive GUI (e.g., to include in a report) a user selection of a final set of hotspots that includes at least some (e.g., up to all) of the one or more automatically detected hotspots; ); and (e) store and/or present the final set of hotspots for display and/or further processing.

[0084] In a particular embodiment, the instructions cause the processor to (f) receive, via the GUI, a user selection of one or more additional user-identified hotspots for inclusion in a final set of hotspots; (g) Update the last hotspot configured to include one or more additional user-identified hotspots.

[0085] In a particular embodiment, in step (b), the instructions cause the processor to use one or more machine learning modules.

[0086] In another aspect, the present invention relates to a method for automatically processing 3D images of a subject to identify and/or characterize (eg, rank) cancerous lesions in the subject, the method comprising ( a) by a processor of a computing device, a functional imaging technique [eg, positron emission tomography (PET); receiving (eg, and/or accessing) a 3D functional image of the object obtained using, eg, single photon emission tomography (SPECT)]; (b) a 3D anatomical image of the object obtained by the processor using an anatomical imaging technique [eg, a computed tomography (CT) image; receiving (eg, and/or accessing) a magnetic resonance (MR) image]; (c) by the processor, one or more specific tissue region(s) or group(s) of tissue regions within the 3D functional image and/or within the 3D anatomical image (e.g., a tissue region corresponding to a specific anatomical region); receiving a 3D segmentation map identifying a set of tissue regions, eg, groups of tissue regions comprising organs in which high or low radiopharmaceutical uptake occurs; (d) by the processor, automatically detecting and/or segmenting a set of one or more hotspots within the 3D functional image using one or more machine learning module(s) to (i) identify for each hotspot a location of the hotspot; a list of hotspots (e.g., detected by one or more machine learning module(s)), and (ii) for each hotspot, a 3D hotspot map identifying a corresponding 3D hotspot volume within the 3D functional image [e.g. . (e.g., a 3D boundary enclosing a 3D hotspot volume)] - at least one (e.g., maximum total) of the one or more machine learning module(s) as input (i) receiving a 3D functional image, (ii) a 3D anatomical image and (iii) a 3D segmentation map, each hotspot corresponding to a local area of high intensity to its surroundings and representing a potential cancerous lesion within the subject; and (e) storing and/or providing the hotspot list and/or 3D hotspot map for display and/or further processing.

[0087] In a particular embodiment, the method may include, by a processor, one or more (eg, multiple) specific tissue regions (eg, organs and/or specific bones) within a 3D anatomical image and a 3D functional image. Receiving an initial 3D segmentation map identifying ; and identifying, by the processor, at least a portion of the one or more specific tissue regions as belonging to the specific one of the one or more tissue grouping(s) (eg, predefined groupings); and updating, by the processor, the 3D segmentation map to indicate the identified specific regions as belonging to a specific tissue grouping; and using, by the processor, the updated 3D segmentation map as an input to at least one of the one or more machine learning modules.

[0088] In a particular embodiment, one or more tissue groupings include a soft tissue grouping such that specific tissue regions representative of soft tissue are identified as belonging to the soft tissue grouping. In certain embodiments, one or more tissue groupings include a bone tissue grouping such that specific tissue regions representing bone are identified as belonging to the bone tissue grouping. In certain embodiments, the one or more tissue groupings include a high uptake organ grouping where one or more organs associated with high radiopharmaceutical uptake (eg, not necessarily due to the presence of lesions, under normal circumstances) are in a high uptake grouping. to be identified as belonging to.

[0089] In a particular embodiment, the method includes, for each detected and/or segmented hotspot, determining, by a processor, a classification for the hotspot [eg, according to an anatomical location, eg eg, classifying the hotspot as bony, lymphatic, or prostate, assigning an alphanumeric code based on the determined (eg, by a processor) location of the hotspot in the subject, eg, as in the labeling scheme of Table 1].

[0090] In a particular embodiment, the method includes using at least one of the one or more machine learning modules to determine a classification for a hotspot for each detected and/or segmented lesion (e.g. e.g., where a single machine learning module performs detection, segmentation, and classification).

[0091] In a particular embodiment, one or more machine learning modules include (A) a whole body lesion detection module that detects and/or segments hotspots across the whole body; and (B) a prostate lesion module that detects and/or segments hot spots within the prostate. In a particular embodiment, the method includes generating hotspot lists and/or maps using (A) and (B), respectively, and merging the results.

[0092] In a particular embodiment, step (d) divides and classifies a set of one or more hotspots to identify for each hotspot a corresponding 3D hotspot volume within the 3D functional image and each hotspot volume is classified into a plurality of hotspot classes. A particular hotspot class of classes [e.g., each hotspot class identifying a specific anatomical and/or tissue region (eg, lymph, bone, prostate) in which the lesion represented by the hotspot is determined to be located] Generating a labeled 3D hotspot map, labeled as belonging to , by segmenting a first initial set of one or more hotspots in the 3D functional image using a first machine learning module to determine the number of hotspot volumes. generating a first initial 3D hotspot map identifying a first set - the first machine learning module segments the hotspots of the 3D functional image according to a single hotspot class [e.g., all hotspots belong to a single hotspot class; to differentiate between background regions and hotspot volumes (eg, not between different types of hotspots) (eg, each hotspot volume identified by the first 3D hotspot map is are labeled as belonging to a single hotspot class, conversely)] ― ; generating a second initial 3D hotspot map identifying a second set of initial hotspot volumes by segmenting the second initial set of one or more hotspots within the 3D functional image using a second machine learning module - the second machine learning module. segment the 3D functional image according to a plurality of different hotspot classes such that the second initial 3D hotspot map is a multi-class 3D hotspot map in which each hotspot volume is labeled as belonging to a particular one of a plurality of different hotspot classes; ham (to differentiate not only between hotspot volumes corresponding to different hotspot classes, but also between hotspot volumes and background regions) - ; and merging, by a processor, a first initial 3D hotspot map and a second initial 3D hotspot map for at least a portion of a hotspot volume identified by the first initial 3D hotspot map by: Identifying a matching hotspot volume of a second initial 3D hotspot map (eg, by identifying substantially overlapping hotspot volumes of the first and second initial 3D hotspot maps) - matching the second 3D hotspot map a hotspot volume labeled as belonging to a particular hotspot class of a plurality of different hotspot classes; and labeling specific hotspot volumes in the first initial 3D hotspot map as belonging to a specific hotspot class (eg, matching hotspot volumes are labeled as belonging) to identify matching hotspot volumes in the second 3D hotspot map as belonging. generating a merged 3D hotspot map comprising segmented hotspot volumes of the first 3D hotspot map labeled according to classes, wherein step (e) generates the merged 3D hotspot map for display and/or further processing; storing and/or providing the map.

[0093] In a particular embodiment, the plurality of different hotspot classes include one or more members selected from the group consisting of: (i) to represent a lesion located in bone (eg, by a second machine learning module) (ii) lymphatic hotspots determined to represent lesions located in the lymph nodes (eg, by a second machine learning module), and (iii) lesions located in the prostate (eg, Prostate hotspots determined (by a second machine learning module).

[0094] In a particular embodiment, the method includes (f) receiving and/or accessing a list of hotspots; and (g) for each hotspot in the hotspot list, segmenting the hotspot using the analytical model [e.g., thereby generating a 3D map of the hotspots segmented analytically (e.g., the 3D map is identifying, for each hotspot, a hotspot volume including voxels of the hotspot of the 3D anatomical image and/or functional image surrounded by the segmented hotspot region)].

[0095] In a particular embodiment, the method includes (h) receiving and/or accessing a hotspot map; and (i) for each hotspot in the hotspot map, segmenting the hotspot using the analytical model [e.g., thereby generating a 3D map of the hotspots segmented analytically (e.g., the 3D map is identifying, for each hotspot, a hotspot volume including voxels of the hotspot of the 3D anatomical image and/or functional image surrounded by the segmented hotspot region)].

[0096] In a specific embodiment, the analytic model is an adaptive threshold setting method, wherein step (i) is each based on a dimension of intensities of voxels of a 3D functional image located within a specific reference volume corresponding to a specific reference tissue region. based on one or more reference values (e.g., a blood pool reference value determined based on intensity in an aortic volume corresponding to a portion of the subject's aorta; e.g., based on intensity in a liver volume corresponding to the subject's liver) determining the determined liver reference value); and for each particular hotspot volume in the 3D hotspot map, determining, by the processor, a corresponding hotspot intensity based on intensities of voxels within the particular hotspot volume [eg, where the hotspot intensity is the particular hotspot volume are the maximal intensities of voxels within (e.g. representing SUVs)]; and determining, by the processor, a hotspot specific threshold for the particular hotspot based on at least one of (i) the corresponding hotspot intensity and (ii) the one or more reference value(s).

[0097] In a particular embodiment, a hotspot specific threshold is determined using a particular threshold function selected from a plurality of threshold functions, the particular threshold function being based on a comparison of the corresponding hotspot intensity to at least one reference value. [e.g., each of the plurality of threshold functions is associated with a specific range of intensity (eg, SUV) values, and the specific threshold function is the hotspot intensity and/or its (eg, predetermined) selected according to the particular range to which the percentage falls (eg, wherein each particular range of intensity values is at least partially limited by a multiple of at least one reference value)].

[0098] In certain embodiments, a hotspot specific threshold is determined (eg, by a specific threshold function) as a variable percentage of the corresponding hotspot intensity, where the variable percentage decreases with increasing hotspot intensity. [eg, where the variable percentage itself is a function (eg, a decreasing function) of the corresponding hotspot intensity].

[0099] In another aspect, the present invention provides a method for identifying and/or characterizing (eg, grading, eg, classifying, eg, indicating a particular lesion type) cancerous lesions in a subject. It relates to a method for automatically processing 3D images of, the method comprising: (a) by a processor of a computing device, a functional imaging technique [eg, positron emission tomography (PET); receiving (eg, and/or accessing) a 3D functional image of the object obtained using, eg, single photon emission tomography (SPECT)]; (b) automatically segmenting, by a processor, a first initial set of one or more hotspots in the 3D functional image using a first machine learning module to generate a first initial 3D hotspot map identifying a first set of hotspot volumes; step - the first machine learning module segments the hotspots of the 3D functional image according to a single hotspot class (eg, differentiating between background regions and hotspot volumes by identifying all hotspots as belonging to a single hotspot class; (eg, not between different types of hotspots) (eg, each hotspot volume identified by the first 3D hotspot map is labeled as belonging to a single hotspot class, as opposed to the background)] ; (c) generating, by the processor, a second initial 3D hotspot map identifying a second set of initial hotspot volumes by segmenting a second initial set of one or more hotspots within the 3D functional image using a second machine learning module - The second machine learning module segments the 3D functional image according to a plurality of different hotspot classes so that the second initial 3D hotspot map generates multiple hotspot volumes in which each hotspot volume is labeled as belonging to a particular one of a plurality of different hotspot classes. to be a class 3D hotspot map (to differentiate not only between hotspot volumes corresponding to different hotspot classes, but also between hotspot volumes and background regions) - ; (d) generating, by the processor, a first initial 3D hotspot map and a second initial 3D hotspot map for each particular hotspot volume of at least a portion of the first set of initial hotspot volumes identified by the first initial 3D hotspot map as follows: merging by: identifying a matching hotspot volume of the second initial 3D hotspot map (e.g., identifying substantially overlapping hotspot volumes of the first and second initial 3D hotspot maps); by) - a matching hotspot volume in the second 3D hotspot map is labeled as belonging to a particular hotspot class of a plurality of different hotspot classes; and labeling a particular hotspot volume in the first initial 3D hotspot map as belonging to a particular hotspot class (matching hotspot volumes are labeled as belonging) to the classes that matching hotspot volumes in the second 3D hotspot map are identified as belonging to. generating a merged 3D hotspot map comprising segmented hotspot volumes of the labeled first 3D hotspot map, and (e) storing and/or providing the merged 3D hotspot map for display and/or further processing. Include steps.

[0100] In a particular embodiment, the plurality of different hotspot classes include one or more members selected from the group consisting of: (i) to indicate a lesion located in bone (e.g., by a second machine learning module); ) bone hotspots determined, (ii) lymphatic hotspots determined to represent lesions located in the lymph nodes (eg, by a second machine learning module), and (iii) lesions located in the prostate (eg, , prostate hotspots determined by the second machine learning module).

[0101] In another aspect, the present invention provides a method for identifying and/or characterizing (eg, grading; eg, classifying; eg, indicative of a particular lesion type) cancerous lesions in a subject. It relates to a method for automatically processing 3D images of through an adaptive threshold setting approach, the method comprising (a) by a processor of a computing device, a functional imaging technique [eg, positron emission tomography (PET) ); receiving (eg, and/or accessing) a 3D functional image of the object obtained using, eg, single photon emission tomography (SPECT)]; (b) receiving (eg, and/or accessing), by a processor, a preliminary 3D hotspot map that identifies one or more preliminary hotspot volumes within the 3D functional image; (c) by the processor, one or more reference values (e.g., corresponding to a part of the aorta of the object) each based on a dimension of intensities of voxels of the 3D functional image located in a specific reference volume corresponding to a specific reference tissue region determining a blood pool reference value determined based on intensity within an aortic volume of the subject (eg, a liver reference value determined based on intensity within a liver volume corresponding to a liver of a subject); (d) by the processor, for each particular preliminary hotspot volume of at least a portion of the one or more preliminary hotspot volumes identified by the preliminary 3D hotspot map, using an adaptive threshold-based segmentation based on the preliminary hotspot volumes to: generating a refined 3D hotspot map by: determining a corresponding hotspot intensity based on intensities of voxels within a specific preliminary hotspot volume [eg, where the hotspot intensity is a voxel within a specific preliminary hotspot volume; are the maximal intensities of (e.g., representing SUVs)]; determining a hotspot specific threshold for a particular preliminary hotspot volume based on at least one of (i) the corresponding hotspot intensity and (ii) one or more reference value(s); Using a threshold-based segmentation algorithm that performs image segmentation using a hotspot-specific threshold determined for a specific preliminary hotspot volume, segmenting at least a portion of a 3D feature (e.g., a subvolume of a specific preliminary hotspot volume) [eg For example, clusters of voxels with intensities above the hotspot specific threshold and including the maximum intensity voxel of the preliminary hotspot volume (eg, n-connected component scheme (eg, where n = 6, n = 18) identifying 3D clusters of voxels)) connected to each other, etc.)] determining a refined and analytically segmented hotspot volume corresponding to a particular preliminary hotspot volume; and including the refined hotspot volume in the refined 3D hotspot map; and (e) storing and/or providing the refined 3D hotspot map for display and/or further processing.

[0102] In a particular embodiment, the hotspot specific threshold is determined using a particular threshold function selected from a plurality of threshold functions, the particular threshold function being based on a comparison of the corresponding hotspot intensity to at least one reference value. [e.g., each of a plurality of threshold functions is associated with a specific range of intensity (eg, SUV) values, and the specific threshold function is a hotspot intensity and/or its (eg, predetermined ) according to the particular range to which the percentage falls (eg, wherein each particular range of intensity values is at least partially limited by a multiple of at least one reference value)].

[0103] In certain embodiments, a hotspot specific threshold is determined (eg, by a specific threshold function) as a variable percentage of the corresponding hotspot intensity, where the variable percentage decreases with increasing hotspot intensity. [eg, where the variable percentage itself is a function (eg, a decreasing function) of the corresponding hotspot intensity].

[0104] In another aspect, the present invention provides a method for identifying and/or characterizing (eg, grading; eg, classifying; eg, representing a particular lesion type) cancerous lesions in a subject. It relates to a method for automatically processing 3D images of, the method comprising (a) by a processor of a computing device, an anatomical imaging technique [eg, x-ray computed tomography (CT); For example, magnetic resonance imaging (MRI); eg, receiving a 3D anatomical image of an object obtained using ultrasound], wherein the 3D anatomical image includes a graphic representation of a tissue (eg, soft tissue and/or bone) in the object; (b) by the processor, the 3D anatomical image is automatically segmented to include a liver volume corresponding to the liver of the object and an aorta volume corresponding to the aorta portion (eg, thoracic and/or abdominal portion) in the 3D anatomical image; generating a 3D segmentation map identifying a plurality of volumes of interest (VOIs); (c) by a processor, a functional imaging technique [eg, positron emission tomography (PET); Receiving (eg, and/or accessing) a 3D functional image of the object obtained using, for example, single photon emission computed tomography (SPECT)] [eg, where the 3D functional image is obtained in a plurality of includes voxels of , each representing a specific physical volume within the object and having an intensity value representing detected radiation emitted from the specific physical volume, wherein at least some of the plurality of voxels of the 3D functional image represent a physical volume within the target tissue region representing volumes]; (d) automatically segmenting, by the processor, one or more hotspots in the 3D functional image, each segmented hotspot corresponding to a local area of high intensity to its surroundings and representing a potential cancerous lesion in the subject (e.g. , indicating) - identifying one or more automatically segmented hotspot volumes; (e) causing, by the processor, rendering of a graphical representation of the one or more automatically segmented hotspot volumes for display in an interactive graphical user interface (GUI) (eg, a quality management and reporting GUI); (f) receiving, by the processor, a user selection of a final hotspot set comprising at least a portion (eg, up to all) of the one or more automatically segmented hotspot volumes via an interactive GUI; (g) for each hotspot volume of the final set, by the processor, (i) the intensities of the voxels of the functional image corresponding to the hotspot volume (e.g., located within) and (ii) the liver volume and aorta volume determining a lesion index value based on one or more reference values determined using intensities of voxels of the functional image corresponding to ; and (e) storing and/or providing the final hotspot set and/or lesion index values for display and/or further processing.

[0105] In a particular embodiment, step (b) comprises segmenting the anatomical image such that the 3D segmentation map identifies one or more bone volumes corresponding to one or more bones of the object, and step (d) comprises functional image identifying a bone using one or more bone volumes within the bone volume and segmenting one or more bone hotspot volumes located within the bone volume (eg, by applying a difference of one or more Gaussian filters and thresholding the bone volume). include

[0106] In a particular embodiment, step (b) includes the 3D segmentation map of the subject's soft tissue organs [eg, left/right lungs, left/right gluteus maximus, urinary bladder, liver, left/right kidney, gallbladder, spleen Segmenting the anatomical image to identify one or more organ volumes corresponding to , thoracic and abdominal aorta and optionally (eg, prostate for patients who have not undergone radical prostatectomy); d) uses the one or more segmented organ volumes within the functional image to identify one or more soft tissue (eg, lymphatic and optionally prostate) volumes and segment one or more lymphatic and/or prostate hotspot volumes located within the soft tissue volume. (eg, by applying a Laplacian of one or more Gaussian filters and thresholding the soft tissue volume).

[0107] In a particular embodiment, step (d) includes adjusting intensities of the functional image to suppress intensity from one or more hyperabsorbent tissue regions prior to segmenting one or more lymphatic and/or prostate hotspot volumes (e.g., , using one or more inhibition methods described herein).

[0108] In a particular embodiment, step (g) includes determining a liver reference value using intensities of voxels of the functional image corresponding to the liver volume.

[0109] In a particular embodiment, a two-component Gaussian mixture model fit is used to fit a two-component Gaussian mixture model to a histogram of the intensities of functional imaging voxels corresponding to the liver volume using a two-component Gaussian mixture model fit to determine areas of abnormally low absorption and identifying and excluding voxels with relevant intensities, and determining an inter-reference value using intensities of remaining (eg, non-excluded) voxels.

[0110] In another aspect, the invention provides a method for identifying and/or characterizing (eg, grading; eg, classifying; eg, representing a particular lesion type) cancerous lesions in a subject. It relates to a system for automatically processing 3D images of, the system comprising: a processor of a computing device; and a memory having stored therein instructions, which when executed by the processor cause the processor to: (a) use a functional imaging technique [eg, positron emission tomography (PET); receive (eg, and/or access) a 3D functional image of an object acquired using, eg, single photon emission computed tomography (SPECT)]; (b) anatomical imaging techniques [eg, computed tomography (CT) images; receive (eg, and/or access) a 3D anatomical image of an object obtained using, for example, magnetic resonance (MR) imaging]; (c) one or more specific tissue region(s) or group(s) of tissue regions within a 3D functional image and/or within a 3D anatomical image (eg, a set of tissue regions corresponding to a specific anatomical region; e.g. receive a 3D segmentation map identifying groups of tissue regions, eg, including organs where high or low radiopharmaceutical uptake occurs; (d) use one or more machine learning module(s) to automatically detect and/or segment a set of one or more hotspots within the 3D functional image, such that (i) for each hotspot, a hotspot list identifying the location of the hotspot [ eg, detected by one or more machine learning module(s)], and (ii) for each hotspot, a 3D hotspot map identifying a corresponding 3D hotspot volume within the 3D functional image [eg, one or more determined through segmentation performed by the machine learning module(s)] [e.g., where the 3D hotspot map depicts, for each hotspot, the 3D hotspot boundaries (e.g., irregular boundaries) of the hotspot (e.g., eg, a 3D boundary enclosing a 3D hotspot volume), and at least one of the one or more machine learning module(s) (e.g., a maximum total) receives as input: (i) a 3D functional image , receiving (ii) a 3D anatomical image and (iii) a 3D segmentation map, each hotspot corresponding to a local area of high intensity to its surroundings and representing a potential cancerous lesion within the subject; and (e) store and/or provide hotspot lists and/or 3D hotspot maps for display and/or further processing.

[0111] In a particular embodiment, the instructions cause the processor to identify one or more (eg, multiple) specific tissue regions (eg, organs and/or specific bones) within the 3D anatomical image and the 3D functional image. receive an initial 3D segmentation map of identify at least a portion of the one or more specific tissue regions as belonging to a specific one of the one or more tissue grouping(s) (eg, predefined groupings); update the 3D segmentation map to indicate the identified specific regions as belonging to a specific tissue grouping; Use the updated 3D segmentation map as an input to at least one of the one or more machine learning modules.

[0112] In a particular embodiment, one or more tissue groupings include a soft tissue grouping such that specific tissue regions representative of soft tissue are identified as belonging to the soft tissue grouping. In certain embodiments, one or more tissue groupings include a bone tissue grouping such that specific tissue regions representing bone are identified as belonging to the bone tissue grouping. In certain embodiments, the one or more tissue groupings include a high uptake organ grouping where one or more organs associated with high radiopharmaceutical uptake (eg, not necessarily due to the presence of lesions, under normal circumstances) are in a high uptake grouping. to be identified as belonging to.

[0113] In a particular embodiment, the instructions cause the processor to determine, for each detected and/or segmented hotspot, a classification for the hotspot [e.g., according to anatomical location, e.g., bone lesion , classification as lymphatic or prostate, assigning an alphanumeric code based on the determined (eg, by a processor) location of the hotspot in relation to the subject, eg, as in the labeling scheme of Table 1].

[0114] In a particular embodiment, the instructions cause the processor to use, for each detected and/or segmented hotspot, at least one of the one or more machine learning modules to determine a classification for the hotspot (eg, where , a single machine learning module performs detection, segmentation, and classification).

[0115] In a particular embodiment, one or more machine learning modules include (A) a whole body lesion detection module that detects and/or segments hotspots across the whole body; and (B) a prostate lesion module that detects and/or segments hot spots within the prostate. In a particular embodiment, instructions cause a processor to generate the hotspot list and/or maps using (A) and (B), respectively, and merge the results.

[0116] In a particular embodiment, the instructions in step (d) cause a processor to segment and classify a set of one or more hotspots to identify for each hotspot a corresponding 3D hotspot volume within the 3D functional image, and each hotspot to be classified into a plurality of hotspots. Of the hotspot classes, a specific hotspot class [eg, each identifying a specific anatomical and/or tissue region (eg, lymph, bone, prostate) in which the lesion represented by the hotspot is determined to be located. hotspot class] by: segmenting a first initial set of one or more hotspots in the 3D functional image using a first machine learning module to generate hotspot volumes of generating a first initial 3D hotspot map identifying a first set - the first machine learning module segments the hotspots of the 3D functional image according to a single hotspot class [e.g., all hotspots belong to a single hotspot class; to differentiate between background regions and hotspot volumes (eg, not between different types of hotspots) (eg, each hotspot volume identified by the first 3D hotspot map is are labeled as belonging to a single hotspot class, conversely)] ― ; segmenting a second initial set of one or more hotspots within the 3D functional image using a second machine learning module to generate a second initial 3D hotspot map identifying a second set of initial hotspot volumes - the second machine learning module segment the 3D functional image according to a plurality of different hotspot classes such that the second initial 3D hotspot map is a multi-class 3D hotspot map in which each hotspot volume is labeled as belonging to a particular one of a plurality of different hotspot classes; ham (to differentiate not only between hotspot volumes corresponding to different hotspot classes, but also between hotspot volumes and background regions) - ; and merging the first initial 3D hotspot map and the second initial 3D hotspot map of at least a portion of the hotspot volume identified by the first initial 3D hotspot map by: the second initial 3D hotspot map; Identifying a matching hotspot volume of a map (eg, by identifying substantially overlapping hotspot volumes of first and second initial 3D hotspot maps) - wherein the matching hotspot volume of the second 3D hotspot map is a plurality of labeled as belonging to a particular hotspot class of different hotspot classes; and labeling specific hotspot volumes in the first initial 3D hotspot map as belonging to a specific hotspot class (matching hotspot volumes are labeled as belonging), thereby labeling matching hotspots in the second 3D hotspot map according to the classes identified to belong to. generating a merged 3D hotspot map comprising the segmented hotspot volumes of the first 3D hotspot map, wherein the instructions in step (e) cause the processor to generate the merged 3D hotspot map for display and/or further processing; to store and/or provide.

[0117] In a particular embodiment, the plurality of different hotspot classes include one or more members selected from the group consisting of: (i) to represent a lesion located in bone (e.g., by a second machine learning module) ) bone hotspots determined, (ii) lymphatic hotspots determined to represent lesions located in the lymph nodes (eg, by a second machine learning module), and (iii) lesions located in the prostate (eg, , prostate hotspots determined by the second machine learning module).

[0118] In a particular embodiment, the instructions further cause the processor to: (f) receive and/or access a list of hotspots; (g) for each hotspot in the hotspot list, segment the hotspot using an analytical model [e.g., thereby generating a 3D map of the hotspots segmented analytically (e.g., the 3D map is each Identifying a hotspot volume containing voxels of the hotspot of the 3D anatomical image and/or functional image surrounded by the segmented hotspot region, for the hotspot of )].

[0119] In a particular embodiment, the instructions further cause the processor to: (h) receive and/or access a hotspot map; (i) for each hotspot in the hotspot map, segment the hotspot using an analytical model [e.g., thereby generating a 3D map of the hotspots segmented analytically (e.g., the 3D map is each identifying a hotspot volume containing voxels of the hotspot of the 3D anatomical image and/or functional image surrounded by the segmented hotspot region, for the hotspot of )].

[0120] In a particular embodiment, the analytic model is an adaptive thresholding method, wherein in step (i) the instructions cause the processor to measure the intensities of voxels of a 3D functional image located within a particular reference volume corresponding to a particular reference tissue region. One or more reference values (e.g., a blood pool reference value determined based on intensity within an aorta volume corresponding to a portion of the subject's aorta; e.g., based on intensity within a liver volume corresponding to a subject's liver) each based on to determine the determined liver reference value); For each specific hotspot volume in the 3D hotspot map, determine a corresponding hotspot intensity based on the intensities of voxels within the specific hotspot volume [eg, where the hotspot intensity is the maximum intensities of voxels within the specific hotspot volume (eg, representing SUVs)]; determine a hotspot specific threshold for a particular hotspot based on at least one of (i) the corresponding hotspot intensity and (ii) one or more reference value(s).

[0121] In a particular embodiment, the hotspot specific threshold is determined using a particular threshold function selected from a plurality of threshold functions, the particular threshold function being based on a comparison of the corresponding hotspot intensity to at least one reference value. [e.g., each of a plurality of threshold functions is associated with a specific range of intensity (eg, SUV) values, and the specific threshold function is a hotspot intensity and/or its (eg, predetermined ) according to the particular range to which the percentage falls (eg, wherein each particular range of intensity values is at least partially limited by a multiple of at least one reference value)].

[0122] In certain embodiments, a hotspot specific threshold is determined (eg, by a specific threshold function) as a variable percentage of the corresponding hotspot intensity, where the variable percentage decreases with increasing hotspot intensity. [eg, where the variable percentage itself is a function (eg, a decreasing function) of the corresponding hotspot intensity].

[0123] In another aspect, the present invention provides a method for identifying and/or characterizing (eg, grading, eg, classifying, eg, indicating a particular lesion type) cancerous lesions in the subject. It relates to a system for automatically processing 3D images of, the system comprising: a processor of a computing device; and a memory having stored therein instructions, which when executed by the processor cause the processor to: (a) use a functional imaging technique [eg, positron emission tomography (PET); receive (eg, and/or approach) a 3D functional image of the object acquired using, eg, single photon emission tomography (SPECT)]; (b) a first method for automatically segmenting a first initial set of one or more hotspots in the 3D functional image using a first machine learning module to identify a corresponding 3D hotspot volume, a first set of hotspot volumes, in the 3D functional image; generate an initial 3D hotspot map, wherein the first machine learning module segments the hotspots of the 3D functional image according to a single hotspot class [e.g., identifying all hotspots as belonging to a single hotspot class to identify background regions and hotspots; Differentiate between volumes (eg, not between different types of hotspots) (eg, consider each hotspot volume identified by the first 3D hotspot map to belong to a single hotspot class as opposed to the background) labeled)]; (c) segment a second initial set of one or more hotspots within the 3D functional image using a second machine learning module to generate a second initial 3D hotspot map identifying a second set of initial hotspot volumes; A machine learning module segments the 3D functional image according to a plurality of different hotspot classes such that the second initial 3D hotspot map is a multi-class 3D hotspot in which each hotspot volume is labeled as belonging to a particular one of a plurality of different hotspot classes. to be a map (to differentiate between hotspot volumes and background areas as well as between hotspot volumes corresponding to different hotspot classes); (d) for each particular hotspot volume of at least a portion of the first set of initial hotspot volumes identified by the first initial 3D hotspot map, a first initial 3D hotspot map and a second initial 3D hotspot map are prepared by: merge: identifying a matching hotspot volume of the second initial 3D hotspot map (eg, by identifying substantially overlapping hotspot volumes of the first and second initial 3D hotspot maps) - the second 3D hotspot A matching hotspot volume in the map is labeled as belonging to a particular hotspot class of multiple different hotspot classes; and labeling a particular hotspot volume in the first initial 3D hotspot map as belonging to a particular hotspot class (matching hotspot volumes are labeled as belonging) to the classes that matching hotspot volumes in the second 3D hotspot map are identified as belonging to. generating a merged 3D hotspot map comprising segmented hotspot volumes of the labeled first 3D hotspot map; and (e) store and/or present the merged 3D hotspot map for display and/or further processing.

[0124] In a particular embodiment, the plurality of different hotspot classes include one or more members selected from the group consisting of: (i) to indicate a lesion located in bone (e.g., by a second machine learning module) ) bone hotspots determined, (ii) lymphatic hotspots determined to represent lesions located in the lymph nodes (eg, by a second machine learning module), and (iii) lesions located in the prostate (eg, , prostate hotspots determined by the second machine learning module).

[0125] In another aspect, the present invention provides a method for identifying and/or characterizing (eg, grading; eg, classifying; eg, representing a particular lesion type) cancerous lesions in a subject. A system for automatically processing 3D images of a through an adaptive threshold setting approach, the system comprising: a processor of a computing device; and a memory having stored thereon instructions, which when executed by the processor cause the processor to: (a) functional imaging techniques [eg, positron emission tomography (PET); receive (eg, and/or access) a 3D functional image of an object acquired using, eg, single photon emission computed tomography (SPECT)]; (b) receive (eg, and/or access) a preliminary 3D hotspot map that identifies one or more preliminary hotspot volumes within the 3D functional image; (c) one or more reference values (eg, in an aortic volume corresponding to a part of the aorta of the object) each based on a dimension of intensities of voxels of the 3D functional image located in a specific reference volume corresponding to a specific reference tissue region; determine a blood pool reference value determined based on the intensity; eg, a liver reference value determined based on intensity in a liver volume corresponding to the liver of the subject); (d) for each particular preliminary hotspot volume of at least a portion of the one or more preliminary hotspot volumes identified by the preliminary 3D hotspot map, using adaptive threshold-based segmentation based on the preliminary hotspot volumes, by To generate a refined 3D hotspot map: determining a corresponding hotspot intensity based on intensities of voxels within a specific preliminary hotspot volume [eg, where the hotspot intensity is the maximum intensities of voxels within a specific preliminary hotspot volume (eg, representing SUVs)]; determining a hotspot specific threshold for a specific preliminary hotspot based on at least one of (i) the corresponding hotspot intensity and (ii) one or more reference value(s); By segmenting at least a part of the 3D feature (eg, a subvolume of a specific preliminary hotspot volume) using a threshold-based segmentation algorithm that performs image segmentation using a hotspot-specific threshold determined for a specific preliminary hotspot [eg, For example, clusters of voxels with intensities above a hotspot specific threshold and containing the maximum intensity voxel of the preliminary hotspot volume (eg, in an n-connected component way (eg, where n = 6, n = 18, etc.) )) determining a refined and analytically segmented hotspot volume corresponding to a particular preliminary hotspot volume; and including the refined hotspot volume in the refined 3D hotspot map; and (e) store and/or present the refined 3D hotspot map for display and/or further processing.

[0126] In a particular embodiment, the hotspot specific threshold is determined using a particular threshold function selected from a plurality of threshold functions, the particular threshold function being based on a comparison of the corresponding hotspot intensity to at least one reference value. [e.g., each of a plurality of threshold functions is associated with a specific range of intensity (eg, SUV) values, and the specific threshold function is a hotspot intensity and/or its (eg, predetermined ) according to the particular range to which the percentage falls (eg, wherein each particular range of intensity values is at least partially limited by a multiple of at least one reference value)].

[0127] In certain embodiments, a hotspot specific threshold is determined (eg, by a specific threshold function) as a variable percentage of the corresponding hotspot intensity, where the variable percentage decreases with increasing hotspot intensity. [eg, where the variable percentage itself is a function (eg, a decreasing function) of the corresponding hotspot intensity].

[0128] In another aspect, the invention provides a method for identifying and/or characterizing (eg, grading; eg, classifying; eg, representing a particular lesion type) cancerous lesions in a subject. It relates to a system for automatically processing 3D images of, the system comprising: a processor of a computing device; and a memory having stored therein instructions, which when executed by the processor cause the processor to: (a) an anatomical imaging technique [eg, x-ray computed tomography (CT); For example, magnetic resonance imaging (MRI); eg, ultrasound] to receive (eg, and/or approach) a 3D anatomical image of an object acquired using a tissue (eg, soft tissue and/or or bones); (b) automatically segmenting the 3D anatomical image to obtain a plurality of interests including a liver volume corresponding to the liver of the object and an aortic volume corresponding to the aorta portion (eg, thoracic and/or abdominal portion) in the 3D anatomical image create a 3D segmentation map identifying volumes (VOIs); (c) functional imaging techniques [eg, positron emission tomography (PET); receive (eg, and/or access) a 3D functional image of an object obtained using, for example, single photon emission computed tomography (SPECT)] and [eg, wherein the 3D functional image is obtained by multiple includes voxels of , each representing a specific physical volume within the object and having an intensity value representing detected radiation emitted from the specific physical volume, wherein at least some of the plurality of voxels of the 3D functional image represent a physical volume within the target tissue region representing volumes]; (d) automatically segment one or more hotspots in the 3D functional image to identify one or more automatically segmented hotspot volumes, each segmented hotspot corresponding to a local region of high intensity with respect to its surroundings and providing potential in the object; indicate (eg, point to) a cancerous lesion; identify one or more automatically partitioned hotspot volumes; (e) cause rendering of a graphical representation of one or more automatically segmented hotspot volumes for display within an interactive graphical user interface (GUI) (eg, a quality management and reporting GUI); (f) receive, via the interactive GUI, a user selection of a final hotspot set comprising at least a portion (eg, up to all) of the one or more automatically segmented hotspot volumes; (g) for each hotspot volume in the final set (i) the intensities of the voxels of the functional image corresponding to the hotspot volume (e.g., located internally) and (ii) the functional image corresponding to the liver volume and the aortic volume. determine a lesion index value based on one or more reference values determined using intensities of voxels of the image; and (e) store and/or present the final set of hotspots and/or lesion index values for display and/or further processing.

[0129] In a particular embodiment, the instructions in step (b) cause the processor to segment the anatomical image such that the 3D segmentation map identifies one or more bone volumes corresponding to one or more bones of the object, and the instructions in step (d) causes the processor to identify bone using one or more bone volumes within the functional image and segment one or more bone hotspot volumes located within the bone volume (e.g., apply a difference of one or more Gaussian filters and threshold the bone volume). by setting a value).

[0130] In a particular embodiment, the instructions in step (b) cause the processor to generate a 3D segmentation map of the subject's soft tissue organs (e.g., left/right lungs, left/right gluteus maximus, urinary bladder, liver, left/right kidney). , to segment the anatomical image to identify one or more organ volumes corresponding to the gallbladder, spleen, thoracic and abdominal aorta, and optionally (e.g., prostate for patients who have not undergone radical prostatectomy); The instructions in d) cause the processor to use the segmented organ volumes within the functional image to identify a soft tissue (eg, lymphatic and optionally prostate) volume and segment one or more lymphatic and/or prostate hotspot volumes located within the soft tissue volume. (eg, by applying a Laplacian of one or more Gaussian filters and thresholding the soft tissue volume).

[0131] In a particular embodiment, the instructions in step (d) cause the processor to adjust the intensities of the functional image to suppress intensity from one or more highly absorptive tissue regions prior to segmenting the one or more lymphatic and/or prostate hotspot volumes. (eg, using one or more inhibition methods described herein).

[0132] In a particular embodiment, the instructions in step (g) cause the processor to determine a liver reference value using intensities of voxels of the functional image corresponding to the liver volume.

[0133] In a particular embodiment, the instructions cause the processor to fit a two-component Gaussian mixture model to a histogram of intensities of functional image voxels corresponding to the liver volume using a two-component Gaussian mixture model fit to determine areas of abnormally low absorption from the liver volume. voxels having intensities related to v are identified and excluded, and an inter-reference value is determined using intensities of the remaining (eg, non-excluded) voxels.

[0134] Features of the embodiments described for one aspect of the invention may be applied to another aspect of the invention.

[0135] The foregoing and other objects, aspects, features and advantages of the present disclosure will become more apparent and better understood by referring to the following description taken in conjunction with the accompanying drawings:
[0136] FIG. 1A is a block flow diagram of an example process for artificial intelligence (AI) based lesion detection, in accordance with the illustrative embodiment.
[0137] FIG. 1B is a block flow diagram of an exemplary process for AI-based lesion detection, in accordance with an illustrative embodiment.
[0138] FIG. 1C is a block flow diagram of an exemplary process for AI-based lesion detection, in accordance with an illustrative embodiment.
[0139] FIG. 2A is a graph showing a histogram of liver SUV values overlaid with a two-component Gaussian mixture model, in accordance with an illustrative embodiment.
[0140] FIG. 2B is a PET image superimposed on CT images showing a portion of a liver volume used for calculation of a liver reference value, according to an illustrative embodiment.
[0141] FIG. 2C is a block flow diagram of an exemplary process for calculating reference intensity values to prevent/reduce effects from tissue regions associated with low radiopharmaceutical uptake, in accordance with the illustrative embodiment.
[0142] FIG. 3 is a block flow diagram of an exemplary process for correcting intensity bleed from one or more tissue regions associated with high radiopharmaceutical uptake, in accordance with the illustrative embodiment.
4 is a block flow diagram of an exemplary process for anatomically labeling hotspots corresponding to detected lesions, in accordance with the illustrative embodiment.
[0144] FIG. 5A is a block flow diagram of an exemplary process for interactive lesion detection allowing user feedback and review via a graphical user interface (GUI), in accordance with the illustrative embodiment.
[0145] FIG. 5B is an example process for user review, quality control and reporting of automatically detected lesions, in accordance with the illustrative embodiment.
[0146] FIG. 6A is a screenshot of a GUI used to verify correct segmentation of a liver reference volume, in accordance with the illustrative embodiment.
[0147] FIG. 6B is a screen shot of a GUI used to confirm accurate segmentation of an aortic segment (blood pool) reference volume, in accordance with the illustrative embodiment.
[0148] FIG. 6C is a screenshot of a GUI used for user selection and/or validation of automatically segmented hotspots corresponding to detected lesions in a subject, in accordance with the illustrative embodiment.
[0149] FIG. 6D is a screenshot of a portion of a GUI that allows a user to manually identify lesions in an image, in accordance with the illustrative embodiment.
[0150] FIG. 6E is a screenshot of another portion of a GUI that allows a user to manually identify lesions in an image, in accordance with the illustrative embodiment.
[0151] FIG. 7 is a screenshot of a portion of a GUI showing a quality control checklist, in accordance with the illustrative embodiment.
8 is a screenshot of a report generated by a user using an embodiment of the automated lesion detection tool described herein, in accordance with an illustrative embodiment.
[0153] FIG. 9 is a block flow diagram illustrating an exemplary architecture for hotspot (lesion) segmentation via a machine learning module receiving as inputs 3D anatomical images, 3D functional images and 3D segmentation maps in accordance with the illustrative embodiment. .
[0154] FIG. 10A is a block flow diagram illustrating an exemplary process in which lesion type mapping is performed subsequent to hotspot segmentation, in accordance with the illustrative embodiment.
[0155] FIG. 10B is a block flow diagram illustrating an exemplary process in which lesion type mapping is performed subsequent to hotspot segmentation, illustrating the use of a 3D segmentation map, in accordance with the illustrated embodiment.
[0156] FIG. 11A is a block flow diagram illustrating a process for detecting and/or segmenting hotspots representing lesions using a systemic network and a prostate specific network, in accordance with the illustrated embodiment.
[0157] FIG. 11B is a block flow diagram illustrating a process for detecting and/or segmenting hotspots representing lesions using a systemic network and a prostate specific network, in accordance with the illustrated embodiment.
12 is a block flow diagram illustrating use of an analytic segmentation step following AI-based hotspot segmentation, in accordance with the illustrative embodiment.
[0159] Fig. 13A is a block diagram illustrating an example U-net architecture used for hotspot partitioning, in accordance with the illustrative embodiment.
13B and 13C are block diagrams illustrating example FPN architectures for hotspot segmentation, in accordance with an illustrative embodiment.
[0161] FIGS. 14A, 14B, and 14C show example images illustrating segmentation of hotspots using the U-net architecture according to the illustrative embodiment.
[0162] Figures 15A and 15B show example images illustrating segmentation of hotspots using the FPN architecture in accordance with the illustrative embodiment.
[0163] Figures 16A, 16B, 16C, 16D and 16E are screen shots of an exemplary GUI for uploading, analyzing and generating reports from medical imaging data, in accordance with the illustrative embodiment.
[0164] Figures 17A and 17B are block flow diagrams of example processes for segmenting and classifying hotspots using two parallel machine learning modules, in accordance with the illustrative embodiment.
[0165] FIG. 17C shows between various software modules (eg, APIs) of an example implementation of a process for segmenting and classifying hotspots using two parallel machine learning modules, in accordance with the illustrative embodiment. It is a block flow diagram illustrating the interaction and data flow of
[0166] Figure 18A is a block flow diagram of an example process for segmenting hotspots by an analytic model using an adaptive threshold setting method, in accordance with the illustrative embodiment.
[0167] Figures 18B and 18C are graphs illustrating the change of a hotspot specific threshold used in the adaptive threshold method as a function of hotspot strength (SUVmax), according to the illustrative embodiment.
[0168] Figures 18D, 18E and 18F are diagrams illustrating certain threshold setting techniques, in accordance with the illustrative embodiment.
[0169] Fig. 18G is a diagram showing intensities of prostate voxels along axial, sagittal and coronal planes with an illustrative setting of a histogram of prostate voxel intensity values and a threshold scaling factor, according to an illustrative embodiment.
[0170] Figure 19A is a block flow diagram illustrating hotspot segmentation using conventional manual ROI definition and conventional fixed and/or relative threshold setting, in accordance with the illustrative embodiment.
[0171] Fig. 19B is a block flow diagram illustrating hotspot segmentation using an AI-based approach combined with an adaptive threshold setting method, in accordance with the illustrative embodiment.
[0172] FIG. 20 is a set of images comparing exemplary segmentation results for threshold setting alone with segmentation results obtained through an AI-based approach combined with an adaptive threshold setting method, according to an illustrative embodiment. am.
[0173] Figures 21A, 21B, 21C, 21D, 21E, 21F, 21G, 21H and 21I show 2D slices of a series of 3D PET images moving along the vertical direction in the abdominal area . The images are compared with the results of the machine learning approach (right images) and the hotspot segmentation results (left images) in the abdominal region performed by the threshold setting method alone according to the specific embodiment described in this application, and each method Show the hotspot regions identified by p overlaid on the PET image slices.
22 is a block flow diagram of a process for uploading and analyzing PET/CT image data using a CAD device that provides automated image analysis, in accordance with certain embodiments described herein.
[0175] FIG. 23 is a screenshot of an example GUI that allows a user to upload image data for review and analysis via a CAD device that provides automated image analysis in accordance with certain embodiments described herein.
[0176] FIG. 24 is an exemplary GUI viewer allowing a user to review and analyze medical image data (eg, 3D PET/CT images) and results of automated image analysis, in accordance with the illustrative embodiment. This is a screenshot of the viewer.
[0177] Fig. 25 is a screenshot of an automatically generated report, in accordance with an illustrative embodiment.
[0178] Figure 26 is a block flow diagram of an example workflow for analysis of medical image data providing automated analysis with user input and review, in accordance with the illustrative embodiment.
[0179] Figure 27 shows three views of a CT image with segmented bone and soft tissue volumes superimposed according to the illustrative embodiment.
[0180] Figure 28 is a block flow diagram of an analytical model for segmentation hotspots, in accordance with an illustrative embodiment.
[0181] Figure 29A is a block diagram of a cloud computing architecture used in a particular embodiment.
[0182] Figure 29B is a block diagram of an example microservice communication flow used in a particular embodiment.
[0183] Figure 30 is a block diagram of a cloud computing environment used in a particular embodiment.
[0184] Figure 31 is a block diagram of an example computing device and an example mobile computing device used in a particular embodiment.
[0185] Features and advantages of the present disclosure will become more apparent from the detailed description set forth below, in conjunction with drawings in which like reference characters identify corresponding elements throughout. In the drawings, like reference numbers generally indicate identical or functionally similar and/or structurally similar elements.

[0186] The systems, devices, methods and processes of the claimed invention are intended to include modifications and transformations developed using information from the embodiments described herein. Conversions and/or modifications of the systems, devices, methods and procedures described in this application may be performed by those skilled in the art.

[0187] Throughout this specification, articles, devices, and systems are described as having, including, or involving particular components, or processes and methods having particular steps or Where described as including, consisting of, or consisting of, there are articles, devices, and systems of the invention that consist essentially of or consist of the recited elements, and of the invention that consist essentially of or consist of the recited process steps. It is contemplated that there are processes and methods according to.

[0188] It should be understood that the order of steps or order of performing particular actions is immaterial so long as the invention remains operable. Moreover, two or more steps or actions may be performed concurrently.

[0189] A reference in this application to any publication, eg, in the Background section, is an admission that the publication serves as prior art to any of the claims presented in this application. no. The Background section is provided for purposes of clarity and is not meant to be a description of the prior art to any claim.

[0190] Headers are provided for the convenience of the reader (the existence and/or placement of headers do not limit the scope of the subject matter described in this application).

[0191] In this application, unless the context clearly dictates otherwise: (i) the term "a" may be understood to mean "at least one"; (ii) the term “or” may be understood to mean “and/or”; (iii) the terms "comprising" and "including" may be understood to include itemized components or steps, whether presented alone or in conjunction with one or more additional components or steps; (iv) the terms “about” and “approximately” may be understood to allow for standard variations as understood by those skilled in the art; (v) Where ranges are provided, endpoints are included.

[0192] In certain embodiments, the term "about" when used in this application in reference to a value refers to a value similar to the referenced value. In general, those skilled in the art familiar with the context will understand the variance in the degree of relevance encompassed by "about" in that context. For example, in some embodiments, the term “about” means 25%, 20%, 19%, 18%, 17%, 16%, 15%, 14%, 13%, 12%, 11% of a stated value. %, 10%, 9%, 8%, 7%, 6%, 5%, 4%, 3%, 2%, 1% or less.

A. Nuclear medicine images

[0193] Nuclear medicine images are acquired using nuclear imaging techniques such as bone scan imaging, positron emission tomography (PET) imaging, and single photon emission computed tomography (SPECT) imaging.

[0194] As used in this application, “image”—eg, a 3D image of a mammal—is any visual representation, such as a photograph, video frame, or streaming video, as well as any electronic, digital, or mathematical representation of a photograph, video frame, or streaming video. Including analog. Any device described in this application, in certain embodiments, includes a display for displaying an image or any other result produced by the processor. Any method described herein, in certain embodiments, includes displaying an image or any other result generated through the method.

[0195] As used in this application, “3-D” or “three-dimensional” in relation to “image” means conveying information about three dimensions. A 3-D image may be rendered as a three-dimensional data set and/or displayed as a set of two-dimensional representations or a three-dimensional representation.

[0196] In a particular embodiment, nuclear medicine imaging uses imaging agents that include radiopharmaceuticals. Nuclear medicine images can be acquired after administration of a radiopharmaceutical to a patient (eg, a human subject) and provide information about the distribution of the radiopharmaceutical within the patient. Radiopharmaceuticals are compounds containing radionuclides.

[0197] As used herein, “administering” an agent means introducing a substance (eg, an imaging agent) to a subject. In general, for example, parenteral (eg intravenous), oral, topical, subcutaneous, peritoneal, intraarterial, inhalation, vaginal, rectal, nasal, introduction into cerebrospinal fluid or infusion into body compartments. Any route of administration may be used, including

[0198] "Radionuclide" as used in this application means a portion containing radioactive isotopes of at least one element. Exemplary and suitable radionuclides include, but are not limited to, those described in this application. In some embodiments, the radionuclides are those used in Positron Emission Tomography (PET). In some embodiments, the radionuclides are those used in single photon emission computed tomography (SPECT). In some embodiments, the non-limiting list of radionuclides is ^99m Tc, ¹¹¹ In, ⁶⁴ Cu, ⁶⁷ Ga, ⁶⁸ Ga, ¹⁸⁶ Re, ¹⁸⁸ Re, ¹⁵³ Sm, ¹⁷⁷ Lu, ⁶⁷ Cu, ¹²³ I, ¹²⁴ I, ¹²⁵ I , ¹²⁶ I, ¹³¹ I, ¹¹ C, ¹³ N, ¹⁵ O, ¹⁸ F, ¹⁵³ SM, ¹⁶⁶ Ho, ¹⁷⁷ Lu, ¹⁴⁹ Pm, ⁹⁰ Y, ²¹³ Bi, ¹⁰³ Pd, ¹⁰⁹ Pd, ¹⁵⁹ Gd, ¹⁴⁰ La, ¹⁹⁸ Au, ¹⁹⁹ Au, ¹⁶⁹ Yb, ¹⁷⁵ Yb, ¹⁶⁵ Dy, ¹⁶⁶ Dy, ¹⁰⁵ Rh, ¹¹¹ Ag, ⁸⁹ Zr, ²²⁵ Ac, ⁸² Rb, ⁷⁵ Br, ⁷⁶ Br, ⁷⁷ Br, ⁸⁰ Br, ^80m Br, ⁸² Br, ⁸³ Br, ²¹¹ At and ¹⁹² Ir.

[0199] The term "radiopharmaceutical" as used in this application refers to a compound containing a radionuclide. In certain embodiments, radiopharmaceuticals are used for diagnostic and/or therapeutic purposes. In certain embodiments, the radiopharmaceutical is small molecules labeled with one or more radionuclide(s), antibodies labeled with one or more radionuclide(s), and antigen-binding portions of antibodies labeled with one or more radionuclide(s). include them

[0200] Nuclear medicine images (eg, PET scans; eg, SPECT scans; eg, whole body bone scans; eg, composite PET-CT images; eg, composite SPECT-CT images s) form images by detecting radiation emitted from radionuclides in radiopharmaceuticals. The distribution of a particular radiopharmaceutical within a patient may be determined by biological mechanisms such as blood flow or perfusion as well as specific enzymatic or receptor binding interactions. Different radiopharmaceuticals can be designed to take advantage of different biological mechanisms and/or unique specific enzyme or receptor binding interactions, and thus selectively focus within specific types of tissue and/or regions within a patient when administered to a patient. do. A greater amount of radiation is emitted from areas within the patient where the concentrations of the radiopharmaceutical are higher than other areas so that these areas appear brighter in nuclear medicine images. Thus, intensity changes in nuclear medicine images can be used to map the distribution of radiopharmaceuticals within a patient. This mapped distribution of the radiopharmaceutical within the patient can be used, for example, to infer the presence of cancerous tissue within various regions of the patient's body.

[0201] For example, when administered to a patient, technetium 99m methylenediphosphonate ( ^99m Tc MDP) selectively accumulates in the patient's skeletal region, particularly in areas of abnormal bone formation associated with malignant bone lesions. Selective focus of radiopharmaceuticals at these sites creates discernible hotspots - localized areas of high intensity in nuclear medicine images. Thus, the presence of malignant bone lesions associated with metastatic prostate cancer can be inferred by identifying these hotspots within a patient's whole body scan. As described below, risk indices that correlate with other prognostic metrics representing patient overall survival and disease status, progression, treatment efficacy, etc., intensity of whole-body scans obtained after administration of ^99m Tc MDP to patients It can be calculated based on an automated analysis of changes. In certain embodiments, other radiopharmaceuticals may also be used in a manner similar to ^99m Tc MDP.

[0202] In certain embodiments, the particular radiopharmaceutical used depends on the particular nuclear medicine imaging technique used. For example, 18F sodium fluoride (NaF) also accumulates in bone lesions similar to ^99m Tc MDP but can be used with PET imaging. In certain embodiments, PET imaging may also utilize a radioactive form of vitamin choline that is readily taken up by prostate cancer cells.

[0203] In certain embodiments, a radiopharmaceutical that selectively binds to specific proteins or receptors of interest, particularly proteins or receptors whose expression is increased in cancerous tissue, may be used. These proteins or receptors of interest include CEA expressed in colorectal carcinoma, Her2/neu expressed in a number of cancers, BRCA 1 and BRCA 2 expressed in breast and ovarian cancer, TRP-1 and TRP expressed in melanoma; 2, but are not limited thereto.

[0204] For example, human prostate specific antigen (PSMA) is upregulated in prostate cancer, including metastatic disease. PSMA is expressed by virtually all prostate cancers and its expression is further increased in poorly differentiated, metastatic and hormone refractory carcinomas. Thus, radiopharmaceuticals corresponding to PSMA binding agents (eg, compounds with high affinity to PSMA) labeled with one or more radionuclide(s) may be used in various regions of the patient (eg, including skeletal regions). (but is not limited thereto)) to obtain nuclear medicine images from a patient in which the presence and/or status of prostate cancer can be assessed. In a specific embodiment, nuclear medicine images obtained using PSMA binders are used to identify the presence of cancerous tissue within the prostate when disease is localized. In a specific embodiment, nuclear medicine images acquired using radiopharmaceuticals containing PSMA binders include not only the prostate but also other organs and tissue regions such as lungs, lymph nodes, and bones that are involved when disease metastasizes. It is used to identify the presence of cancerous tissue in various areas.

[0205] In particular, upon administration to a patient, radionuclide-labeled PSMA binders selectively accumulate within cancerous tissue based on their affinity for PSMA. In a manner similar to that described above for ^{the 99m} Tc MDP, selective concentration of radionuclide-labeled PSMA binders at specific sites within the patient creates detectable hotspots in nuclear medicine images. Because PSMA binding agents are concentrated in various cancerous tissues and regions of the body that express PSMA, localized cancer within the patient's prostate and/or metastatic cancer of various regions of the patient's body can be detected and evaluated. Automated analysis of intensity changes in nuclear medicine images obtained after administration of a PSMA-binding radiopharmaceutical to a patient for risk indices that correlate with other prognostic metrics representing patient overall survival and disease status, progression, treatment efficacy, etc. can be calculated based on

[0206] A variety of radionuclide labeled PSMA binders can be used as radiopharmaceutical imaging agents for nuclear medicine imaging to detect and evaluate prostate cancer. In certain embodiments, the specific radionuclide labeled PSMA binder used is specific to the specific imaging technique (eg PET; eg SPECT) and specific areas of the patient to be imaged (eg organs), such as depend on factors For example, certain radionuclide labeled PSMA binders are suitable for PET imaging, while others are suitable for SPECT imaging. For example, certain radionuclide labeled PSMA binders facilitate imaging of a patient's prostate and are primarily used when disease is localized, while others facilitate imaging of organs and regions throughout the patient's body. and useful for evaluating metastatic prostate cancer.

[0207] Various PSMA binders and radionuclide labeled versions thereof are described in U.S. Patent Nos. 8,778,305, 8,211,401, and 8,962,799, each of which is incorporated herein by reference in its entirety. Several PSMA binders and radionuclide-labeled versions thereof are also described in PCT Application PCT/US2017/058418, filed on October 26, 2017 (PCT Publication WO 2018/081354), the contents of which are hereby incorporated in their entirety. incorporated by reference into the application. Section J below also describes some exemplary PSMA binders and radionuclide labeled versions thereof.

B. Automated lesion detection and analysis

i. Automated lesion detection

[0208] In a particular embodiment, the systems and methods described herein utilize machine learning techniques for automated image segmentation and detection of hotspots that correspond to and represent possible cancerous lesions in a subject.

[0209] In a particular embodiment, the systems and methods described in this application are based on a cloud-based platform as described, for example, in PCT/US2017/058418, filed on Oct. 26, 2017 (PCT Publication WO 2018/081354). It can be implemented in, the contents of which are incorporated by reference in this application in its entirety.

[0210] In a particular embodiment, machine learning modules, as described herein, include one or more machine learning modules, such as random forest classifiers, artificial neural networks (ANNs), convolutional neural networks (CNNs), and the like. implement the techniques. In a particular embodiment, machine learning modules implementing machine learning techniques are trained using manually segmented and/or labeled images, for example, to identify and/or classify portions of images. Such training may be used to determine various parameters of machine learning algorithms implemented by the machine learning module, such as weights associated with layers of neural networks. In certain embodiments, once the machine learning module is trained, the values of the determined parameters are fixed and the machine learning module (eg, to achieve a specific task, such as identifying specific target regions within images) , immutable, static) to process new data (e.g. different from training data) and without further updates on those parameters (e.g. the machine learning module provides feedback and/or updates) not received) is used to perform the trained task. In certain embodiments, machine learning modules may receive feedback based, for example, on a user's accuracy review, and such feedback may be used as additional training data to dynamically update the machine learning module. In some embodiments, the trained machine learning module is a classification algorithm, eg, a random forest classifier, with tunable and/or fixed (eg, locked) parameters.

[0211] In a particular embodiment, machine learning techniques may use a volume of interest corresponding to particular organs (eg, prostate, lymph node regions, kidney, liver, bladder, aortic region) as well as specific target tissue regions such as bone. It is used to automatically segment anatomical structures in anatomical images, such as CT, MRI, ultrasound, etc., to identify them. In this way, machine learning modules segmentation that can be mapped (eg, projected) onto functional images, such as PET or SPECT images, to provide an anatomical environment for evaluating intensity fluctuations therein. may be used to generate masks and/or segmentation maps (eg, comprising a plurality of segmentation masks, each corresponding to and identifying a particular target tissue region). Approaches that segment images and use the acquired anatomical environment for analysis of nuclear medicine images are described, for example, in PCT/US2019/012486, filed Jan. 7, 2019 (PCT Publication No. WO 2019/136349) and 2020 PCT/EP2020/050132 (PCT Publication WO 2020/144134), filed on January 6, 2020, the contents of each of which are incorporated herein by reference in their entirety.

[0212] In a particular embodiment, occult lesions are detected as areas of high intensity locally on functional images, such as PET images. These localized regions of high intensity, also called hotspots, can be detected using image processing techniques that do not necessarily involve machine learning, such as filtering and thresholding, and can be segmented using approaches such as fast-forward methods. Anatomical information established from segmentation of anatomical images allows anatomical labeling of detected hotspots indicative of potential lesions. The anatomical environment can be useful to allow different detection and segmentation techniques to be used for hotspot detection in various anatomical regions, which can increase sensitivity and performance.

[0213] In a particular embodiment, the automatically detected hotspots may be presented to the user through an interactive graphical user interface (GUI). In certain embodiments, a manual segmentation tool is included in the GUI to allow the user to select lesions of all shapes and sizes to account for target lesions detected by the user (eg, physician) but missed or poorly segmented by the system. Allows you to manually "paint" regions of images that you recognize as corresponding. These manually segmented lesions can then be included in later generated reports along with selected automatically detected target lesions.

ii. AI-based lesion detection

[0214] In a particular embodiment, the systems and methods described herein utilize one or more machine learning modules to analyze intensities of 3D functional images and detect hotspots representing potential lesions. For example, training data for AI-based lesion detection algorithms may be obtained by collecting a data set of PET/CT images obtained by manually detecting and segmenting hotspots representing lesions. These manually labeled images can be used to train one or more machine learning algorithms to automatically analyze functional images (eg, PET images) to accurately detect and segment hotspots corresponding to cancerous lesions.

[0215] FIG. 1A shows an example process 100a for automated lesion detection and/or segmentation using machine learning modules implementing machine learning algorithms such as ANNs, CNNs, and the like. As can be seen in FIG. 1A , a 3D functional image 102 , such as a PET or SPECT image, is received 106 and used as input to a machine learning module 110 . 1A shows an exemplary PET image obtained using PyL ^™ as the radiopharmaceutical 102a. The PET image 102a is shown superimposed on the CT image (eg, as a PET/CT image), but the machine learning module 110 is the PET (eg, or other functional image) itself (eg, as a PET/CT image). , CT or other anatomical images) can be received as input. In certain embodiments, anatomical images may also be received as input, as described below. The machine learning module automatically detects and/or segments hotspots 120 determined (by the machine learning module) to represent potentially cancerous lesions. An example image showing hotspots appearing in PET image 120b is also shown in FIG. 1A. The machine learning module thus produces as output one or both of (i) a hotspot list 130 and (ii) a hotspot map 132 . In a particular embodiment, the hotspot list identifies locations (eg, centers of mass) of detected hotspots. In a particular embodiment, the hotspot map identifies 3D volumes and/or depicts 3D boundaries of detected hotspots as determined through image segmentation performed by machine learning module 110 . The hotspot list and/or hotspot map may be stored and/or provided (eg, to another software module) for display and/or further processing 140 .

[0216] In a particular embodiment, machine learning based lesion detection algorithms may be trained on and use anatomical information as well as functional image information (eg, from PET images). For example, in a particular embodiment, one or more machine learning modules used for lesion detection and segmentation may use two channels - a first channel corresponding to a portion of a PET image and a second channel corresponding to a portion of a CT image. It can be received as input and trained. In certain embodiments, information derived from anatomical (eg, CT) images may also be used as input to machine learning modules for lesion detection and/or segmentation. For example, in certain embodiments, 3D segmentation maps identifying various tissue regions within an anatomical and/or functional image may also be used to provide an anatomical environment (e.g., as an input, e.g., received as a separate input channel by one or more machine learning modules).

[0217] FIG. 1B shows that both a 3D anatomical image 104 and a 3D functional image 102, such as a CT or MR image, are received 108 and the 3D anatomical image 104 and 3D functional image 104 and 3D functional image as described herein are received. (102) Exemplary process 100b used as input to machine learning module 112 that performs hotspot detection and/or segmentation 122 based on information (eg, voxel intensities) from all show Hotspot list 130 and/or hotspot map 132 may be generated as output from a machine learning module and subjected to further processing (eg, rendering graphics for display, follow-up by other software modules, etc.) 140 ) can be stored / provided for.

[0218] In a particular embodiment, automated lesion detection and analysis (eg, for inclusion in a report) includes three tasks: (i) detection of hotspots corresponding to the lesions, (ii) ) segmentation of the detected hotspots (eg, identifying a 3D volume corresponding to each lesion within the functional image), (iii) having a high or low probability corresponding to a true lesion in the subject (eg, Classification of detected hotspots as whether they are therefore appropriate for inclusion in the radiologist's report. In a particular embodiment, one or more machine learning modules may be used to accomplish these three tasks, for example one by one (eg, in sequence) or in combination. For example, in a particular embodiment, a first machine learning module is trained to detect hotspots and identify hotspot locations, a second machine learning module is trained to segment hotspots, and a third machine learning module is trained to segment hotspots, and a third machine learning module is trained to, for example, other It is trained to classify the detected hotspots using the information obtained from the two machine learning modules.

[0219] For example, as seen in the exemplary process 100c of FIG. 1C, the 3D functional image 102 is received 106 and sent to a first machine learning module 114 that performs automated hotspot detection. can be used as an input for The first machine learning module 114 automatically detects 124 one or more hotspots in the 3D functional image and generates a hotspot list 130 as output. The second machine learning module 116 may receive the hotspot list 130 together with the 3D functional image as input and perform automated hotspot segmentation 126 to generate the hotspot map 132 . As noted previously, the hotspot list 130 as well as the hotspot map 132 may be stored and/or presented for further processing 140 .

[0220] In a particular embodiment, a single machine learning module uses hotspots within images (eg, 3D functional images; eg, to generate a 3D hotspot map identifying volumes corresponding to detected hotspots). It combines the first two steps of segmentation and detection of hotspots. A second machine learning module may then be used to classify the detected hotspots based on, for example, previously determined segmented hotspots. In a particular embodiment, a single machine learning module can be trained to accomplish all three tasks (detection, segmentation and classification) in a single step.

iii. Lesion index values

[0221] In certain embodiments, a lesion index value is calculated for a detected hotspot to provide, for example, a measure of relative uptake and/or size within the corresponding physical lesion. In certain embodiments, the lesion index value is determined for a particular hotspot based on reference values corresponding to (i) a measure of intensity for the hotspot and (ii) measures of intensity in one or more reference volumes, each corresponding to a particular reference tissue. is calculated for For example, in certain embodiments, the reference values are an aortic reference value (also referred to as a blood pool reference) that measures intensity within an aortic volume corresponding to a portion of the aorta and a liver reference value that measures intensity within a liver volume corresponding to a liver of a subject. contains the value In certain embodiments, intensities of voxels of a nuclear medicine image, eg, a PET image, represent standard absorption values (SUVs) (eg, corrected for radiopharmaceutical injection dose and/or patient weight), and hotspots Measures of intensity and/or measures of reference values are SUV values. The use of these reference values for calculating lesion index values is described in more detail, for example, in PCT/EP2020/050132, filed 6 Jan. 2020, the contents of which are incorporated herein by reference in their entirety. included

[0222] In a particular embodiment, a segmentation mask is used, for example, to identify a specific reference volume in a PET image. In the case of a specific reference volume, a segmentation mask identifying the reference volume can be obtained through segmentation of an anatomical, eg, CT image. For example, in certain embodiments (eg, as described in PCT/EP2020/050132), segmentation of a 3D anatomical image is a segmentation map comprising a plurality of segmentation masks each identifying a specific tissue region of interest. can be performed to generate Accordingly, one or more segmentation masks of a segmentation map created in this way may be used to identify one or more reference volumes.

[0223] In a particular embodiment, the mask is eroded a fixed distance (eg, at least one voxel) to identify the voxels of the reference volume to be used for calculation of the corresponding reference value, a physical area within the reference tissue region. It is possible to create a reference organ mask that identifies a reference volume that fully corresponds to . For example, erosion distances of 3 mm and 9 mm may be used for the aorta and liver reference volumes, respectively. Other erosion distances may also be used. Additional mask refinement may be performed, eg, as described below for the liver reference volume (eg, to select a desired specific set of voxels for use in calculating a reference value).

[0224] Various intensity dimensions within the reference volumes may be used. For example, in a particular embodiment, a robust average of voxel intensities inside a reference volume (eg, defined by a reference volume segmentation mask after erosion) is the average of values in the interquartile range of voxel intensities (IQR _mean ) can be determined as Other dimensions such as peak, maximum, median, etc. may also be determined. In a specific embodiment, the aortic reference value is determined as a robust average of SUVs from voxels inside the aortic mask. The robust mean is calculated as the average IQR _mean of the values in the interquartile range.

[0225] In a particular embodiment, a subset of voxels within the reference volume is selected to avoid influence from reference tissue regions that may have abnormally low radiopharmaceutical uptake. Although the automated segmentation techniques described and referenced in this application can provide accurate contouring (eg, identification) of regions of images corresponding to specific tissue regions, abnormally low levels of the liver should be excluded from reference value calculations. Absorptive regions are often present. For example, a liver reference value (eg, liver SUV value) prevents effects from areas of the liver with very low tracer (radiopharmaceutical) activity, which may result, for example, from tumors lacking tracer uptake. can be calculated to In a particular embodiment, calculating a reference value for the liver to account for the effect of abnormally low absorption in the reference tissue regions involves a histogram of the intensities of voxels corresponding to the liver (eg, voxels within an identified liver reference volume). Analyze and remove (e.g., exclude) intensities if those intensities form a second histogram peak of lower intensities, so that only intensities related to the higher intensity value peak are included.

[0226] For example, in the case of the liver, the reference SUV is 2 fitted to a histogram of the SUVs of the voxels within the liver reference volume (e.g. identified by a liver segmentation mask, e.g. according to the erosion procedure described above). It can be calculated as the average SUV of principal components (eg, also referred to as "modes", eg, "major modes") in a component Gaussian mixture model. In a particular embodiment, if a subcomponent has a larger average SUV than the main component and the subcomponent has a weight of at least 0.33, an error is generated and no reference value for the liver is determined. In certain embodiments, if the subcomponent has a larger average than the main peak, the inter-reference mask is left untouched. Otherwise, a separate SUV threshold is calculated. In a particular embodiment, the segregation threshold is defined such that the probability of belonging to a major component for a larger SUV at the threshold is equal to the probability of belonging to a minor component for a smaller SUV at the segregation threshold. Next, the inter-reference mask is refined by removing voxels whose SUVs are smaller than the separation threshold. A liver reference value may then be determined as a dimension of the intensity (eg SUV) values of the voxels identified by the liver reference mask, eg as described herein for the aorta reference. 2A illustrates an example liver reference calculation, a histogram of liver SUV values with Gaussian mixture components (major component 244 and minor component 246) shown in red and a separation threshold 242 shown in green. shows

[0227] Figure 2B shows the resulting portion of the liver volume used to calculate the liver reference value, with voxels corresponding to the lower value peak excluded from the reference value calculation. Contours 252a and 252b of the refined liver volume mask, with voxels corresponding to lower value peaks (e.g., having intensities below the separation threshold 242) excluded, are shown in the respective images of FIG. 2B. is shown in As confirmed in the figure, lower intensity regions towards the lower part of the liver as well as regions closer to the liver edge were excluded.

[0228] FIG. 2C shows an exemplary process 200 in which a multi-component mixed model is used to avoid effects from regions with low tracer uptake as described herein for liver reference volume calculation. The procedure described herein for the liver and shown in FIG. 2C also measures the strength of other organs and tissue regions of interest, such as the aorta (eg, an aortic portion such as a thoracic aortic portion or an abdominal aortic portion), parotid gland, gluteus muscle. can be similarly applied to the calculation of As described in this application and identified in FIG. 2C , in a first step a 3D functional image 202 is received and a reference volume corresponding to a particular reference tissue region (eg liver, aorta, parotid gland) is placed inside it. is identified in (208). Next, the multi-component mixing model 210 is fitted to the distribution intensities (eg, a histogram of intensities) of the reference volume (eg, interior) and the dominant modes of the mixing model are identified (212). A dimension of the intensities associated with the major mode (eg, excluding contributions from intensities associated with other minor modes) is determined 214 and used as a reference intensity value for the identified reference volume. In certain embodiments, as described herein, the dimensions of the intensities associated with the dominant mode are determined by identifying a segregation threshold, such that intensities above the segregation threshold are determined to be associated with the dominant mode, and intensities below the segregation threshold are determined to be associated with the dominant mode. It is determined to be related to the mode. Voxels with intensities above the separation threshold are used to determine the reference intensity value, while voxels with intensities below the separation threshold are excluded from the reference intensity value calculation.

[0229] In a particular embodiment, hotspots are detected 216 and the reference intensity value determined in this way is determined by, for example, PCT/US2019/012486 filed on Jan. 7, 2019 and filed on Jan. 6, 2020. can be used to determine lesion index values for hotspots 218 detected through approaches such as those described in published PCT/EP2020/050132, the contents of each of which are incorporated herein by reference in their entirety.

iv. Suppression of intensity bleed associated with normal absorption in highly absorbing organs

[0230] In a specific embodiment, the intensities of the voxels of the functional image are adjusted to suppress/correct intensity bleed associated with a specific organ where hyperabsorption occurs under normal circumstances. This approach can be used for organs such as kidney, liver and urinary bladder, for example. In certain embodiments, correction for intensity bleed associated with multiple organs is performed one organ at a time in a step-by-step fashion. For example, in certain embodiments, first renal absorption is inhibited, then hepatic absorption, and then urinary bladder absorption is inhibited. Thus, the input for liver suppression is an image corrected for renal uptake (eg, the input for bladder suppression is an image corrected for renal and hepatic uptake).

[0231] Figure 3 shows an exemplary process 300 for correcting intensity bleed from a region of high absorption tissue. As can be seen in FIG. 3, a 3D functional image is received (304) and a high-intensity volume corresponding to the highly absorbent tissue region is identified (306). In another step, a containment volume outside the high-intensity volume is identified (308). In certain embodiments, as described herein, the containment volume may be determined as a volume that encloses an area outside of, but within a predetermined distance from, the high-intensity volume. In another step, the high-intensity volume intensity determined, eg, through interpolation (eg, using convolution), eg, based on intensities outside the high-intensity volume (eg, within the containment volume). A background image is determined by allocating voxels within the (310). In another step, an estimated image is determined by subtracting the background image from the 3D functional image (eg, via voxel-by-voxel intensity subtraction) (312). In another step, a suppression map is determined (314). As described herein, in certain embodiments, a suppression map is determined using an estimated image by extrapolating the intensity values of voxels within the high intensity volume to locations outside the high intensity volume. In a particular embodiment, the intensities are extrapolated only to locations within the suppression volume, and the intensities of voxels outside the suppression volume are set to zero. The suppression map is then used to adjust the intensities of the 3D functional image 316, for example by subtracting the suppression map from the 3D functional image (eg, performing voxel-by-voxel intensity subtraction).

[0232] An exemplary approach for suppression/correction of intensity bleed from a specific organ (in certain embodiments, kidneys are treated together) on a PET/CT composite image is as follows:

One. The projected CT organ mask segmentation is adjusted to the high-intensity regions of the PET image to account for PET/CT misalignment. If the PET calibrated organ mask is less than 10 pixels, no suppression is made for this organ.

2. A “background image” is calculated to replace any high absorption with an interpolated background absorption within the attenuation distance from the PET-adjusted organ mask. This is done using convolution with Gaussian kernels.

3. Intensities to be considered when estimating suppression are calculated as the difference between the input PET and the background image. This "estimation image" has high intensity inside a given organ and zero intensity at locations farther than the attenuation distance from the given organ.

4. A suppression map is estimated from the estimated image using an exponential model. The suppression map is non-zero only in the region within the attenuation distance of the PET-adjusted organ segmentation.

5. A suppression map is subtracted from the original PET image.

[0233] As described above, these five steps may be repeated in a sequential manner for each of several sets of organs.

v. Anatomical labeling of detected lesions

[0234] In a particular embodiment, the detected hotspots are assigned (eg, automatically) to anatomical labels identifying specific anatomical regions and/or groups of regions in which the lesions they represent are determined to be located. . For example, as shown in exemplary process 400 of FIG. 4 . The 3D functional image may be received 404 and used to automatically detect hotspots 406 via, for example, any of the approaches described herein. Once hotspots are detected, anatomical classifications for each hotspot can be automatically determined 408 and each hotspot is labeled with the determined anatomical classification. Automated anatomical labeling uses automatically determined locations of detected hotspots together with anatomical information provided, for example, by a 3D segmentation map that identifies specific tissue regions and/or image regions corresponding to an anatomical image. can be performed Respective hotspots and anatomical labeling may be stored and/or presented for further processing 410 .

[0235] For example, detected hotspots may be automatically classified into one of five classes as follows.

T (전립선 종양)

T (prostate tumor)

N (골반 림프절)

N (pelvic lymph nodes)

Ma (비골반 림프)

Ma (non-pelvic lymph)

Mb (골 전이)

Mb (bone metastasis)

Mc (전립선 또는 림프절 내에 위치하지 않는 연조직 전이)

Mc (soft tissue metastases not located within the prostate or lymph nodes)

[0236] Table 1 below lists tissue regions associated with each of the five classes. Accordingly, hotspots corresponding to locations within any of the tissue areas associated with a particular class may be automatically assigned to that class.

[ Table 1 ]

List of tissue regions corresponding to the 5 classes of the lesion anatomical labeling approach

vi. Graphical user interface and quality control and reporting

[0237] In a particular embodiment, the detected hotspots and related information, such as calculated lesion index values and anatomical labeling, are interactive graphics user to allow review by a medical professional such as a physician, radiologist, technologist, etc. It is displayed as an interface (GUI). Therefore, medical professionals can use the GUI to review and verify the accuracy of the detected hotspots, as well as the corresponding index values and/or anatomical labeling. In certain embodiments, the GUI may also allow a user to identify and segment (eg, manually) additional hotspots within the medical image to identify additional potential lesions that the medical professional believes the automated detection process may have missed. let it be Once identified, lesion index values and/or anatomical labeling may be determined for manually identified and segmented lesions. For example, as shown in FIG. 5B , the user can review the determined locations for each hotspot as well as the anatomical labeling such as (eg, automatically determined) miTNM classification. A miTNM classification scheme is described, eg, in Eiber et al., "Prostate Cancer Molecular Imaging Standardized Evaluation (PROMISE): Proposed miTNM Classification for the Interpretation of PSMA—Ligand PET/CT," J. Nucl. Med., vol. 59, pg. 469-78 (2018)] for further details, the contents of which are incorporated herein by reference in their entirety. Once the user is satisfied with the set of detected hotspots and the information computed therefrom, they can confirm approval and generate a final signed report that can be reviewed and used to discuss the results and diagnosis with the patient and evaluate prognosis and treatment options. can

[0238] In an exemplary process 500 for interactive hotspot review and detection, eg, as can be seen in FIG. 5A, a 3D functional image is received 504 and hotspots are automated, eg, as described herein. automatically detected using any of the detected detection approaches (506). A set of automatically detected hotspots are graphically displayed and rendered within the interactive GUI 508 for user review. The user may select at least some (eg, at most all) of the automatically determined hotspots for inclusion in the final set of hotspots 510, which may require additional calculations to determine risk index values for the patient, for example. (512).

[0239] FIG. 5B shows an exemplary workflow 520 for user review of detected lesions and lesion index values for quality control and reporting. An exemplary workflow allows user review of segmented lesions as well as liver and aortic segmentation used for calculation of lesion index values as described herein. For example, in a first step, the user determines the quality 522 and the accuracy of the automated segmentation used to obtain the liver and blood pool (eg, aorta) reference values 524 (eg, the image (eg, aorta)). , CT images). As can be seen in FIGS. 6A and 6B , the GUI allows the user to ensure that the automated segmentation of the liver (purple 602 in FIG. 6A ) is within healthy liver tissue and the blood pool (aortic portion 604 shown in salmon color in FIG. 6B ). ) evaluates the images to ensure that the automated segmentation of ) is within the aorta and left ventricle and enables overlapping segmentation.

[0240] In another step 526, the user confirms the automatically detected hotspots and/or identifies additional hotspots to generate, eg, a final set of hotspots corresponding to the lesions, for inclusion in the generated report. do. As can be seen in FIG. 6C, the user can select an automatically identified hotspot by moving the mouse pointer over a graphical representation of the hotspot displayed within the GUI (eg, as an overlapping and/or labeled area on the PET and/or CT image). can To facilitate hotspot selection, the particular selected hotspot may be displayed to the user through a color change (eg, turning green). The user may then click on the hotspot to select it, which may be visually confirmed to the user through another color change. For example, as can be seen in FIG. 6C , the hotspot turns pink when selected. Upon user selection, quantitatively determined values such as lesion index and/or anatomical labeling may be displayed to the user to allow them to automatically verify the determined values 528 .

[0241] In a particular embodiment, the GUI allows the user to (automatically) select hotspots from a set of pre-identified hotspots to verify that the hotspots in fact represent lesions 526a and also automatically correspond to lesions not detected. to identify additional hotspots 562b.

[0242] As can be seen in FIGS. 6D and 6E, the user uses a GUI tool to slice slices of images (eg, PET images and/or CT images; for example, PET images superimposed on CT images). ) to mark the regions corresponding to the manually identified new lesions. Quantitative information such as lesion index and/or anatomical labeling can be automatically determined for manually identified lesions or manually entered by the user.

[0243] At another step, for example, once the user has selected and/or manually identified all lesions, the GUI displays a quality control checklist 530 for the user to review, as seen in FIG. 7 . Once the user has reviewed and completed the checklist, they can click "Generate Report" to sign and generate a final report 532 . An example of the generated report is shown in FIG. 8 .

C. Exemplary Machine Learning Network Architectures for Lesion Segmentation

i. Machine learning module input and architecture

[0244] Referring to Figure 9, which illustrates an exemplary hotspot detection and segmentation process 900 in some embodiments, the hotspot detection and/or segmentation receives functional 902 and anatomical 904 images as input. by the machine learning module 908 receiving as input a map 906 that provides segmentation of various organs as described in this application as well as segmentation of various tissue regions such as, for example, soft tissue and bone. is carried out

[0245] The functional image 902 may be a PET image. The intensities of the voxels of functional image 902 can be scaled to represent SUV values as described herein. Other functional images as described herein may also be used in certain embodiments. The anatomical image 904 may be a CT image. In a particular embodiment, the voxel intensities of the CT image 904 are scaled to represent Hounsfield units. In certain embodiments, other anatomical images as described herein may be used.

[0246] In some embodiments, machine learning module 908 implements machine learning algorithms using the U-net architecture. In some embodiments, machine learning module 908 implements a machine learning algorithm that uses a feature pyramid network (FPN) architecture. In some embodiments, various other machine learning architectures may be used to detect and/or segment lesions. In certain embodiments, machine learning modules as described herein perform semantic segmentation. In certain embodiments, machine learning modules as described herein perform instance segmentation to distinguish one lesion from another, for example.

[0247] In some embodiments, the three-dimensional segmentation map 906 received as input by the machine learning module is specific to certain organs (e.g., prostate, liver, aorta, bladder, various other organs described herein, etc.) ) and/or identify various volumes (eg, via multiple 3D segmentation masks) in the received 3D anatomical and/or functional images as corresponding to specific tissue regions of interest, such as bone. Additionally or alternatively, the machine learning module may receive a 3D segmentation map 906 identifying groupings of tissue regions. For example, in some embodiments, a 3D segmentation map may be used that identifies soft tissue regions, bone, then background regions. In some embodiments, the 3D segmentation map can identify groups of highly absorbing organs where high levels of radiopharmaceutical absorption occur. A group of highly absorbing organs may include, for example, the liver, spleen, kidney and urinary bladder. In some embodiments, the 3D segmentation map identifies a group of high absorption organs along with one or more other organs, such as the aorta (eg, a low absorption soft tissue organ). Other groupings of tissue areas may also be used.

[0248] The functional image, anatomical image, and segmentation map input to the machine learning module 908 may have various sizes and dimensions. For example, in a specific embodiment, each of the functional image, anatomical image, and segmentation map are patches of three-dimensional images (eg, represented by a three-dimensional matrix). In some embodiments, each of the patches has the same size - eg, each input is a [32 x 32 x 32] or [64 x 64 x 64] patch of voxels.

[0249] The machine learning module 908 segments the hotspots and creates a 3D hotspot map 910 identifying one or more hotspot volumes. For example, the 3D hotspot map 910 may include one or more masks that identify one or more hotspot volumes and have the same dimensions as one or more functional image, anatomical image, or segmentation map inputs. In this way, the 3D hotspot map 910 can be used to identify functional images, anatomical images, or volumes in a segmentation map corresponding to hotspots and thus physical lesions.

[0250] In some embodiments, the machine learning module 908 divides the hotspot volumes by differentiating between background (ie, non-hotspot) areas and the hotspot volume. For example, the machine learning module 908 can be a binary classifier that classifies voxels as background or belongs to a single hotspot class. Thus, the machine learning module 908 identifies the hotspot volume but does not differentiate between different anatomical locations and/or types of lesions that particular hotspot volumes may represent - eg, bone metastases, lymph nodes, local prostate. A class independent (eg, 'single class') 3D hotspot map may be generated as output. In some embodiments, machine learning module 908 segments the hotspot volumes and classifies the hotspots according to a plurality of hotspot classes, each representing a specific anatomical location and/or type of lesion represented by the hotspot. In this way, the machine learning module 908 can directly generate a multi-class 3D hotspot map that identifies one or more hotspot volumes and labels each hotspot volume as belonging to a particular one of a plurality of hotspot classes. For example, detected hotspots may be classified as bone metastases, lymph nodes or prostate lesions. In some embodiments, other soft tissue classes may be included.

[0251] This classification may be performed in addition or alternatively to the classification of hotspots according to the likelihood of representing a true lesion, eg as described in section B.ii herein.

ii. Lesion classification post-processing and/or output

[0252] In some embodiments, referring to FIGS. 10A and 10B , after hotspot detection and/or segmentation (the terms “lesions” and “hotspots” refer to physical lesions, e.g., in functional images, to hotspots) In the images by one or more machine learning modules, post-processing 1000 is performed to label hotspots as belonging to a particular hotspot class. For example, detected hotspots may be classified as bone metastases, lymph nodes or prostate lesions. In some embodiments, the labeling scheme of Table 1 may be used. In some embodiments, such labeling may be performed by a machine learning module, which may be the same machine learning module used to perform segmentation and/or detection of hotspots, or a separate module receiving a list of detected hotspots. (e.g., identifying its locations) and/or a 3D hotspot map (e.g., depicting hotspot boundaries determined through segmentation) individually or as a 3D functional image, 3D anatomical image as described herein. and/or use as an input along with other inputs such as segmentation maps. In some embodiments, as can be seen in FIG. 10B , the segmentation map 906 used as an input to the machine learning module 908 to perform lesion detection and/or segmentation is also dependent on, for example, anatomical location. Can be used to classify lesions. In some embodiments, other (eg, different) segmentation maps may be used (eg, not necessarily the same segmentation map supplied to the machine learning module as input).

iii. Parallel Organ Specific Lesion Detection Modules

[0253] In some embodiments, referring to FIGS. 11A and 11B, the one or more machine learning modules include one or more organ-specific modules that perform detection and/or segmentation of hotspots located within a corresponding organ. For example, as seen in exemplary processes 1100 and 1150 of FIGS. 11A and 11B , respectively, prostate module 1108a may be used to perform detection and/or segmentation in a prostate region. In some embodiments, one or more organ specific modules are used in conjunction with the whole body module 1108b to detect and/or segment hotspots throughout a subject's body. In some embodiments, the results 1100a for one or more organ specific modules are merged with the results 1110b from the full body module to form the final hotspot list and/or hotspot map 1112 . In some embodiments, merging is performed by merging the results (eg, hotspot lists and/or 3D hotspot maps) 1110a and 1110b into another, such as 3D hotspot map 1114 generated by segmenting the hotspots using other methods. This may include combining the output, which may include the use of other machine learning modules and/or techniques as well as other segmentation approaches. In some embodiments, an additional segmentation approach may be performed after detection and/or segmentation of hotspots by one or more machine learning modules. This additional segmentation step may use, for example, hotspot segmentation and/or detection results obtained from one or more machine learning modules as input. In certain embodiments, as can be seen in FIG. 11B , as described herein, for example, in section C. iv. The analytic segmentation approach 1122 in can be used with organ specific lesion detection modules. Analytical segmentation 1122 utilizes an analytic segmentation technique (e.g., machine learning) using the PET image 1102 along with the results 1110b and 1110a from upstream machine learning modules 1108b and 1108a. not) to segment the hotspots and create an analytically segmented 3D hotspot map 1124.

iv. analytic segmentation

[0254] In some embodiments, referring to FIG. 12 , machine learning techniques may be used to perform hotspot detection and/or initial segmentation, e.g., performing final segmentation for each hotspot as a subsequent step. An analytic model is used to

[0255] As used in this application, the terms "analytic model" and "analytic segmentation" refer to a segmentation method that is based on (eg, uses) predetermined rules and/or functions (eg, mathematical functions). refers to them For example, in certain embodiments, an analytic segmentation method identifies hotspots using one or more predetermined rules, such as an ordered sequence of image processing steps, application of one or more mathematical functions to an image, conditional logic branches, and the like. can be divided Analytical segmentation methods include threshold-based methods (e.g., including image thresholding), level-setting methods (e.g., fast-forward method), graph-cut methods (e.g., Watershed segmentation) or active contour models may be included, but are not limited thereto. In a particular embodiment, analytic segmentation approaches do not depend on training phase. In contrast, in a particular embodiment, a machine learning model uses a training data set (including examples of hotspots and segmented images, for example, manually by a radiologist or other practitioner) to spot hotspots. We will segment the hotspot using a model that has been automatically trained to pre-segment the segments and aims to mimic the segmentation behavior in the training set.

[0256] For example, the use of an analytic segmentation model to determine the final segmentation may be advantageous because, in some cases, analytic models may be more easily understood and debugged than machine learning approaches. In some embodiments, these analytic segmentation approaches can operate on 3D functional images with lesion segmentation generated by machine learning techniques.

[0257] In an example process 1200 for hotspot segmentation using an analytical model, as can be seen, for example in FIG. Receives the segmentation map 1206 as input. The machine learning module 1208 performs segmentation to generate a 3D hotspot map 1210 identifying one or more hotspot volumes. The analytic segmentation model 1212 uses the PET image 1202 and the 3D hotspot map 1210 generated by the machine learning module to perform segmentation and analytically generate a 3D hotspot map 1214 that identifies the segmented hotspot volumes. do.

v. Exemplary Hotspot Segmentation

[0258] Figures 13A and 13B show examples of machine learning module architectures for hotspot detection and/or segmentation. Fig. 13a shows an exemplary U-net architecture ("N = " in parentheses in Fig. 13a identifies multiple filters in each layer), and Fig. 13b illustrates an exemplary FPN architecture. 13C shows another exemplary FPN architecture.

[0259] Figures 14A-14C show example results for hotspot segmentation obtained using a machine learning module implementing the U-net architecture. The crosshairs and bright spots in the images indicate a segmented hotspot 1402 (indicating a potential lesion). 15A and 15B show example hotspot segmentation results obtained using a machine learning module implementing FPN. In particular, FIG. 15A shows an input PET image superimposed on a CT image. 15B shows an example hotspot map determined using a machine learning module implementing FPN superimposed on a CT image. The overlaid hotspot map shows hotspot volumes 1502 in dark red near the subject's spine.

D. Exemplary Graphical User Interface

[0260] In certain embodiments, the lesion detection, segmentation, classification, and related techniques described herein require user interaction (eg, with a software program implementing the various approaches described herein) and/or or a GUI that facilitates review of the results. For example, in certain embodiments, GUI portions and windows allow, among other things, a user to upload and manage data to be analyzed, visualize images and results generated through the approaches described herein, Allows you to generate a report summarizing the results. Screenshots of certain example GUI views are shown in FIGS. 16A-16E .

[0261] For example, FIG. 16A shows the uploading and viewing of studies such as PET images and PET images and CT images collected via PET/CT scans [same scans and/or scans (e.g. , image data collected during Digital Imaging and Communications in Medicine (DICOM) standard). In certain embodiments, uploaded studies are automatically added to a patient list that lists the identifiers of subjects/patients from which one or more PET/CT images have been uploaded. For each entry in the patient list shown in Figure 16, the patient ID is displayed along with the available PET/CT studies for that patient as well as the corresponding reports. In certain embodiments, the team concept enables the creation of groupings of multiple users (eg, teams) who work on and are provided access to a particular subset of uploaded data. In certain embodiments, a patient list may be automatically shared and associated with a specific team to provide each member of the team access to the patient list.

[0262] FIG. 16B shows an example GUI viewer 1610 that allows a user to view medical image data. In a particular embodiment, the viewer is a multi-modal viewer, allowing the user to view multiple imaging techniques as well as various formats and/or combinations thereof. For example, the viewer shown in FIG. 16B allows a user to view PET and/or CT images as well as fusions (eg, overlaps) thereof. In a particular embodiment, a viewer allows a user to view 3D medical image data in a variety of formats. For example, the viewer may allow a user to select and view various 2D slices along specific (eg, selected) cross-sectional planes of 3D images. In certain embodiments, a viewer allows a user to view a maximum intensity projection (MIP) of 3D image data. Other ways of visualizing 3D image data may also be provided. In this example, as can be seen in FIG. 16B , a control panel graphical widget 1612 is provided on the left side of the viewer and allows the user to view available study information such as date, various patient data and imaging parameters, etc. to be able to see

[0263] Referring to Fig. 16C, in a particular embodiment the GUI viewer allows the user to select lesion volumes that are, for example, Volumes of Interest (VOIs) of an image to identify and select as likely representing true underlying physical lesions. Includes a lesion selection tool that In a particular embodiment, lesion volumes are selected from a series of hotspot volumes that are automatically identified and segmented via, for example, any of the approaches described herein. Selected lesion volumes may be saved for inclusion in a final set of identified lesion volumes that may be used for reporting and/or further quantitative analysis. In certain embodiments, upon user selection of a particular lesion volume, for example, as seen in FIG. 16C , various characteristics/quantitative metrics of a particular lesion [eg, maximum intensity, peak intensity, average intensity, volume, lesion index (LI), anatomical classification (eg, miTNM class, location, etc.), etc.] (1614).

[0264] Referring to Fig. 16D, a GUI viewer additionally or alternatively allows a user to view the results of automated segmentation performed according to various embodiments described herein. Segmentation may be performed through automated analysis of CT images as described herein and may include identification and segmentation of 3D volumes representing the liver and/or aorta. Segmentation results may be superimposed on representations of medical image data, such as CT and/or PET image representations.

[0265] FIG. 16E shows an example report 1620 generated through analysis of medical imaging data as described herein. In this example, report 1620 summarizes the results for the reviewed study and provides characteristics and quantitative metrics characterizing lesion volumes 1622 selected (eg, by the user). For example, as can be seen in FIG. 16E , the report reports, for each selected lesion volume, Lesion ID, Lesion Type (eg, miTNM Classification), Lesion Location, SUV-Max, SUV-Peak, SUV- Mean values, volume and lesion index values are included.

E. Hotspot Segmentation and Classification Using Multiple Machine Learning Modules

[0266] In a particular embodiment, multiple machine learning modules are used in parallel to segment and classify hotspots. 17A is a block flow diagram of an exemplary process 1700 for segmenting and classifying hotspots. The exemplary process 1700 performs image segmentation on the 3D PET/CT image to segment hotspot volumes and divides each segmented hotspot volume according to an (automatically) determined anatomical location - in particular, a lymphatic, bone or prostate hotspot. Classify.

[0267] The exemplary process 1700 receives and operates on a 3D PET image 1702 and a 3D CT image 1704 as inputs. The CT image 1704 is segmented to identify 3D volumes in the CT image that represent specific tissue regions and/or organs of interest or anatomical groupings of multiple (eg, related) tissue regions and/or organs. It is input to the first organ segmentation machine learning module 1706 to perform. Thus, the organ segmentation machine learning module 1706 is used to generate a 3D segmentation map 1708 that identifies specific tissue regions and/or organs of interest or anatomical groupings thereof within the CT image. For example, in a particular embodiment segmentation map 1708 identifies two volumes of interest corresponding to two anatomical groupings of organs - one highly absorbent soft tissue organ including liver, spleen, kidney and urinary bladder. corresponds to the anatomical grouping of , and the second corresponds to the aorta (eg, thoracic and abdominal segments), a low-absorption soft tissue organ. In certain embodiments, organ segmentation machine learning module 1706 generates an initial segmentation map with output identifying various individual organs, including those that make up the anatomical groupings of segmentation map 1708, as well as others in particular embodiments. and segmentation map 1708 is created from the initial segmentation map (eg, by assigning volumes corresponding to individual organs of an anatomical group grouping with the same label). Thus, in a particular embodiment, the 3D segmentation map 1708 is composed of (i) voxels belonging to high absorption soft tissue organs, (ii) low absorption soft tissue organs - i.e., the aorta - and (iii) between different regions as background. It uses three labels to identify and differentiate

[0268] In the exemplary process 1700 shown in FIG. 17A. The long-term segmentation machine learning module 1706 implements the U-net architecture. Other architectures (eg FPN's) may be used. The PET image 1702, CT image 1704 and 3D segmentation map 1708 are used as inputs to two parallel hotspot segmentation modules.

[0269] In a particular embodiment, the example process 1700 uses two machine learning modules in parallel to segment and classify hotspots in different ways and then merge the results. For example, the machine learning module performs more accurate segmentation when identifying only a single class of hotspots - eg, whether image regions are hotspots - than several desired hotspot classes - lymph, bone, prostate. appeared to do Thus, process 1700 uses a first single-class hotspot segmentation module 1712 to perform precise segmentation and a second multi-class hotspot segmentation module 1714 to classify hotspots into the desired three categories.

[0270] In particular, the first single-class hotspot segmentation module 1712 performs segmentation to create a first single-class 3D hotspot map 1716 that identifies 3D volumes representing hotspots along with other image regions identified as a background. carry out Accordingly, single class hotspot segmentation module 1712 performs binary classification to label image voxels as belonging to one of two classes (background or single hotspot class). A second multi-class hotspot segmentation module 1714 segments the hotspots and assigns the segmented hotspot volumes to one of a plurality of hotspot classification labels as opposed to using a single hotspot class. In particular, the multi-class hotspot segmentation module 1714 classifies the segmented hotspot volumes into lymph, bone or prostate hotspots. The multi-class hotspot segmentation module thus creates a second multi-class 3D hotspot map 1718 that identifies 3D volumes representing the hotspots and labels them as lymph, bone or prostate along with other image regions identified as background. In step 1700, the single-class hotspot segmentation module and the multi-class hotspot segmentation module respectively implement the FPN architecture. Other machine learning architectures (eg U-net) may be used.

[0271] In a particular embodiment, the single-class hotspot map 1716 and multi-class hotspot map 1718 are merged (1722) to create a final 3D hotspot map of segmented and classified hotspots 1724. In particular, each hotspot volume of the single-class hotspot map 1716 is compared to the hotspot volumes of the multi-class hotspot map 1718 to thus identify matching hotspot volumes representing the same physical location and the same (potential) physical lesion. . Consistent hotspot volumes may be identified based on various dimensions, such as spatial overlap (eg, percentage volume overlap), proximity (eg, centers of gravity within a threshold distance), and the like. there is. The hotspot volumes of the single-class hotspot map 1716 in which the matching hotspot volume from the multi-class hotspot map 1718 is identified are assigned the label of the matching hotspot volume - lymph, bone, or prostate. In this way, hotspots are accurately segmented via single-class hotspot segmentation module 1712 and then labeled using the results of multi-class hotspot segmentation module 1714.

[0272] Referring to FIG. 17B, in certain cases, for a particular hotspot volume in the single-class hotspot map 1716, no matching hotspot volume is found from the multi-class hotspot map 1718. These hotspot volumes are labeled based on comparison with segmentation map 1708 and possibly a 3D segmentation map 1738 that identifies 3D volumes corresponding to lymphatic and bony regions.

[0273] In a particular embodiment, the single class hotspot segmentation module 1712 may not segment the hotspots of the prostate region such that the single class hotspot map does not include any hotspot in the prostate regions. Hotspot volumes labeled as prostate hotspots from the multi-class hotspot map 1718 can be used for inclusion in the merged hotspot map 1724. In certain embodiments, single-class hotspot segmentation module 1712 may segment some hotspots in the prostate region, but additional hotspots (eg, not identified in single-class hotspot map 1716) are multi-class hotspot segmentation. Prostate hotspots can be segmented and identified by module 1714. These additional hotspot volumes present in the multi-class hotspot map 1718 may be included in the merged hotspot map 1724.

[0274] Accordingly, in a particular embodiment, information from CT image 1704, PET image 1702, 3D organ segmentation map 1738, single class hotspot map 1716 and multi-class hotspot map 1718 is a hotspot It is used in a merge step 1722 to create a merged 3D hotspot map of the segmented and classified hotspot volumes 1724.

[0275] In one exemplary merging approach, overlap (eg, between two hotspot volumes) is any two from the hotspot volume of the multi-class hotspot map and the single-class hotspot map that correspond to/represent the same physical location. It is determined when the number of voxels is determined. If a particular hotspot volume in a single-class hotspot map overlaps with only one hotspot volume in a multi-class hotspot map (i.e., only one matching hotspot volume from an identified multi-class hotspot map), then a particular hotspot volume in a single-class hotspot map Overlapped hotspot volumes in the multi-class hotspot map are labeled according to the class they are identified as belonging to. If a particular hotspot volume overlaps with two or more hotspot volumes from a multi-class hotspot map (each identified as belonging to a different hotspot class), then each voxel of the single-class hotspot volume is the same as the number of overlapping hotspot volumes from the multi-class hotspot map. It is assigned the same class as the nearest voxel. If a particular hotspot volume of a single-class hotspot map does not overlap with a hotspot volume of a multi-class hotspot map, the particular hotspot volume is compared to a 3D segmentation map identifying soft tissue regions (e.g., organs) and/or bones. assigned to a hotspot class based on For example, in some embodiments, a particular hotspot volume may be labeled as belonging to a bone class if any of the following statements are true:

(i) more than 20% of the hotspot volume overlaps with the rib segment;

(ii) if the hotspot volume does not overlap with any label of the organ segment and the CT average value of the hotspot mask is greater than 100 Hounsfield units;

(iii) when the SUV _max location of the hotspot volume overlaps with the bone label in organ segmentation; or

(iv) If more than 50% of the hotspot volume overlaps with bone labels in organ segmentation.

[0276] In some embodiments, a particular hotspot volume may be identified as lymph if 50% or more of the particular hotspot volume does not overlap with bone labels in organ segmentation.

[0277] In some embodiments, when all hotspot volumes of a single class hotspot map are classified as lymph, bone or prostate, any remaining prostate hotspots from the multi-class model are overlaid on the single class hotspot map and the merged hotspot map included in

[0278] FIG. 17C shows an example computer process 1750 for implementing the hotspot segmentation and classification approach according to the embodiment described with respect to FIGS. 17A and 17B.

F. Analytical Segmentation via Adaptive Thresholding Approach

[0279] In certain embodiments, image analysis techniques as described in this application, for example as described in section C.iv of this application, are used to refine hotspot segmentations determined via a machine learning module as described in this application. To do this, an analytic segmentation step is used. For example, in certain embodiments, a 3D hotspot map generated by machine learning approaches as described herein is used as an initial input to an analytic segmentation model that refines and/or performs an entirely new segmentation.

[0280] In a particular embodiment, the analytic segmentation model uses a thresholding algorithm to identify hotspots on anatomical images (eg, CT images, MR images) and/or functional images (eg, SPECT images, PET images). ) (eg, an anatomical and functional composite image such as a PET/CT or SPECT/CT image) by comparing intensities of voxels with one or more threshold values.

[0281] In a particular embodiment, with reference to FIG. 18A, an adaptive threshold approach whereby, for a particular hotspot, the intensities within the initial hotspot volume determined for the particular hotspot are machine learning, e.g., as described herein. The approaches are compared to one or more reference values to determine a threshold for a particular hotspot. The threshold for a particular hotspot is then used by the analytic segmentation model to segment the particular hotspot and determine the final hotspot volume.

[0282] FIG. 18A shows an example process 1800 for segmenting hotspots via an adaptive threshold setting approach. Process 1800 uses an initial 3D hotspot map 1802 to identify one or more 3D hotspot volumes, a PET image 1804 and a 3D organ segmentation map 1806 . The initial 3D hotspot map 1802 may be determined automatically through various machine learning approaches described herein and/or based on user interaction with a GUI. For example, a user can refine the automatically determined set of hotspot volumes by selecting a subset for inclusion in the 3D hotspot map 1802 . Additionally or alternatively, the user may determine the 3D hotspot volumes manually, for example by creating a boundary on the image with a GUI.

[0283] In a particular embodiment, the 3D organ segmentation map identifies one or more reference volumes corresponding to particular reference tissue regions, such as an aorta portion and/or liver. As described herein, for example in Section B.iii, the intensities of voxels within particular reference volumes may be used to compute associated reference values 1808, for which the intensities of identified and segmented hotspots are can be compared (eg acting as a 'measuring stick'). For example, the liver volume may be used to calculate liver reference values and the aortic portion to be used to calculate aorta or blood pool reference values. In process 1800, the intensities of the aortic portion are used to calculate 1808 a blood pool reference value 1810. The blood pool reference value 1810 is used in combination with the initial 3D hotspot map 1802 and the PET image 1804 to determine thresholds for performing a threshold based analytic segmentation of the hotspots in the initial 3D hotspot map 1802 .

[0284] In particular, for a particular hotspot volume identified in the initial 3D hotspot map 1802 (which identifies a particular hotspot representing a physical lesion), the intensities of the PET image 1804 voxels located within the particular hotspot volume correspond to the particular hotspot. used to determine the hotspot intensity for In a particular embodiment, the hotspot intensities are maximum intensities of voxels located within a particular hotspot volume. For example, for PET image intensities representing SUVs, the maximum SUV (SUV _max ) within a particular hotspot volume is determined. Other measures may be used, such as peak value (eg SUV _peak ), mean, median, quartile mean (IQR _mean ).

[0285] In a particular embodiment, a hotspot specific threshold for a particular hotspot is determined based on a comparison of a blood pool reference value and hotspot intensity. In certain embodiments, the comparison between the hotspot intensity and the blood pool reference value is used to select one of a plurality of (eg, predefined) threshold functions, the selected threshold function being hotspot specific for a particular hotspot. used to calculate the threshold. In certain embodiments, the threshold function calculates a hotspot specific threshold as a function of a hotspot intensity (eg, maximum intensity) of the specific hotspot and/or a blood pool reference value. For example, a threshold function may calculate a hotspot specific threshold as the product of (i) a scaling coefficient and (ii) a hotspot intensity (or other intensity measure) of a specific hotspot and/or blood pool reference. In certain embodiments, the scaling factor is a constant. In certain embodiments, the scaling factor is an interpolated value determined as a function of the intensity dimension of a particular hotspot. In certain embodiments and/or for certain threshold functions, the scaling factor is used to determine the plateau level corresponding to the maximum threshold value, as described in more detail, eg, in Section G of this application. is a constant that becomes

[0286] Pseudocode for an example approach to, for example, computing various threshold functions and choosing between them (eg, via conditional logic) is shown below.

If 90% of SUVmax ≤ [see blood pool], the threshold = 90% of SUVmax.

Alternatively, if 50% of SUVmax ≥ 2 x [see blood pool], the threshold = 2 x [see blood pool].

Alternatively, linear interpolation is used to determine the percentage of SUVmax, starting with 90% at [[blood pool reference]/0.9] and ending at [2 x [blood pool reference]/0.5] with 50%.

If [interpolated percentage] x SUVmax is less than 2 x [refer to blood pool], threshold = [interpolated percentage] x SUVmax.

Alternatively, threshold = 2 x [see blood pool].

[0287] Figures 18B and 18C show a specific example adaptive threshold setting approach implemented by the pseudo code above. 18B displays the change in threshold 1832 as a function of hotspot intensity - SUVmax in the example - for a particular hotspot. 18C plots the change in the hotspot specific threshold as a percentage of SUVmax for the specific hotspot as a function of SUVmax for the specific hotspot. The dotted lines in each graph represent specific values for the blood pool reference (with an SUV of 1.5 in the example plots of FIGS. 18B and 18C), and also represent 90% and 50% of the SUV _max in FIG. 18C.

[0288] Referring to FIGS. 18D-18F, the adaptive threshold setting approach as described herein addresses the problems and drawbacks associated with previous threshold setting techniques using fixed or relative thresholds. In particular, threshold-based lesion segmentation based on maximum standard absorption value (SUVmax) provides a transparent and reproducible method of segmenting hotspot volumes for estimation of parameters such as absorption volume and SUV _mean in certain embodiments, but conventional fixation and relative thresholds do not work well under the full dynamic range of the lesion SUV _max . The fixed threshold approach uses a single value, eg a user-defined SUV value, as the threshold to use for segmenting hotspots in the image. For example, a user may set a fixed threshold level to a value of 4.5. The relative threshold approach segments hotspots using a specific constant, fraction or percentage and using a local threshold for each hotspot set to a specific percentage or percentage of the hotspot maximum SUV. For example, the user may set the relative threshold to 40% to segment each hotspot using a threshold calculated as 40% of the maximum hotspot SUV value. Both of these approaches - conventional fixed thresholds and relative thresholds - have drawbacks. For example, it is difficult to define appropriate fixed thresholds that work well across patients. Defining the threshold as a fixed percentage of the hotspot maximum or peak intensity is also problematic because hotspots with lower overall intensity are segmented using lower thresholds. As a result, segmenting low-intensity hotspots that may represent smaller lesions with relatively low absorption using a low threshold may in fact result in a larger identified hotspot volume than for high-intensity hotspots that represent physically larger lesions. .

[0289] For example, FIGS. 18D and 18E illustrate segmentation of two hotspots using a threshold determined to be 50% of the maximum hotspot intensity (eg, 50% SUV _max ). Each figure shows a line cut through the hotspot and plots the intensity on a vertical line as a function of position. 18D shows a graph 1840 illustrating the change in intensity for high-intensity hotspots representing large physical lesions 1848. The hotspot intensity 1842 peaks near the center of the hotspot and the hotspot threshold 1844 is set to 50% of the maximum value of the hotspot intensity 1842 . Segmenting the hotspot using the hotspot threshold 1844 creates a segmented volume that approximately matches the size of the physical lesion as shown, for example by comparing the linear dimension 1846 to the illustrated lesion 1848. . 18E shows a graph 1850 illustrating the change in intensity for low intensity hotspots representing small physical lesions 1858. The hotspot intensity 1852 also peaks near the center of the hotspot and the hotspot threshold 1854 is also set to 50% of the maximum value of the hotspot intensity 1852 . However, since hotspot intensity 1852 peaks less sharply and has a lower intensity peak than hotspot intensity 1842 for high-intensity hotspots, setting a threshold relative to the maximum of hotspot intensity results in a much higher absolute threshold. It gets lower. As a result, threshold-based segmentation results in a greater hotspot volume compared to hotspots with smaller but higher intensity indicated physical lesions, as shown, for example, by comparing the linear size 1856 to the illustrated lesion 1858. generate Relative thresholds can thus produce larger apparent hotspot volumes for smaller physical lesions. This is particularly problematic for the assessment of treatment response, as lesions of low intensity will have lower thresholds and thus lesions responding to treatment may appear to increase in volume.

[0290] In certain embodiments, adaptive threshold setting as described herein addresses these drawbacks by using an adaptive threshold calculated as a percentage of the hotspot intensity, which percentage is (i) the hotspot intensity (e.g. For example, SUV _max ) decreases with increasing (ii) both hotspot intensity (eg, SUV _max ) and overall physiological uptake (eg, as measured by a reference value such as a blood pool reference value) depends on Thus, unlike conventional relative thresholding approaches, the specific fraction/percentage of hotspot intensity used in the adaptive thresholding approach described in this application varies, which is itself a function of hotspot intensity and, in certain embodiments, physiologically Absorption is also explained. For example, as can be seen in the example plot 1860 of FIG. 18F , threshold 1864 can be set to a higher peak hotspot intensity 1852 using a variable, adaptive threshold setting approach as described herein. Set to a percentage - for example, 90% as shown in Fig. 18F. As illustrated in FIG. 18F , doing so enables a threshold based segmentation to identify a hotspot volume that more accurately reflects the true size of the lesion 1866 represented by the hotspot.

[0291] In certain embodiments, thresholding is facilitated by first dividing non-uniform lesions into homogeneous sub-components and finally excluding absorption from nearby intensity peaks using a watershed algorithm. As described in this application, adaptive threshold setting can be applied to manually pre-segmented lesions as well as automated detections by, for example, deep neural networks implemented via machine learning modules as described in this application. This can improve reproducibility and robustness and add explainability.

G. Example Study Comparing Exemplary Threshold Setting Functions and Scaling Factors for PYL-PET/CT Imaging

[0292] This example evaluates various parameters for use in the adaptive threshold setting approach as described in this application, e.g., Section F, and uses manually annotated lesions as a reference to set fixed and relative thresholds. Describe studies performed for comparison.

[0293] This exemplary study used ¹⁸ F-DCFPyL PET/CT scans on 242 patients with hotspots corresponding to bone, lymphatic and prostate lesions manually segmented by an experienced nuclear medicine reader. A total of 792 hotspot volumes across 167 patients were annotated. Two studies were conducted to evaluate threshold setting algorithms. In the first study, manually annotated hotspots were refined with different thresholding algorithms and how well the order of magnitude was preserved, i.e., smaller hotspot volumes remained smaller than larger hotspot volumes initially after refinement. The extent to which it is has been estimated. In a second study, according to various embodiments described in this application, refinement by thresholding of suspicious hotspots automatically detected by a machine learning approach was performed and compared with manual annotations.

[0294] The PET image intensities in this example were calibrated to represent standard absorption values (SUV) and referred to as absorption or absorption intensities in this section. Other threshold setting algorithms compared were: a fixed threshold of SUV = 2.5, a relative threshold of 50% of SUV _max and variants of adaptive thresholds. Adaptive thresholds were defined as the decreasing percentage of SUV _max with or without the maximum threshold level. The stability level was set to be absorption intensities above normal in areas corresponding to healthy tissue. To select an appropriate resting level, two supporting investigations were performed, one studying the normal absorption intensity of the aorta and the other studying the normal absorption intensity of the prostate. Among other things, thresholding approaches were evaluated based on conservation of order of magnitude compared to annotations performed by a nuclear medicine reader. For example, if a nuclear medicine reader has manually segmented hotspots and sorted the manually segmented hotspot volumes according to size, preservation of size order is the hotspot volume created by segmenting the same hotspots using an automated thresholding approach. hotspot volumes are ordered according to their size in the same way (eg, without user interaction). According to the weighted rank correlation measure, both embodiments of the adaptive thresholding approach achieved the best performance in terms of magnitude order preservation. These two adaptive thresholding methods set thresholds starting at 90% of the SUV _max for low-intensity lesions and stabilizing at twice the blood pool reference value (e.g., 2 x [aortic reference uptake]). used The first method (called “P9050―sat”) reached stability when the stability level was 50% of SUV _max , and the other (called “P9040—sat”) reached stability when SUV _max was 40%. did

[0295] It has also been shown that refining automatically detected and segmented hotspots with threshold settings changes the precision-recall tradeoff. The original automatically detected and segmented hotspots had high recall and low precision, but refining the segments with the P9050-sat thresholding method produced a more balanced performance in terms of precision and recall.

[0296] Improved relative size conservation indicates that treatment response assessment will be improved/more accurate because the algorithm better captures the order of magnitude of the nuclear medicine reader's annotations. Dealing with the balance between over-segmentation and under-segmentation can be decoupled from the detection step by introducing a separation threshold setting method - i.e. automated hotspot detection performed using machine learning approaches as described in this application and In addition to the segmentation approach, we use the analytic, adaptive segmentation approach as described in this application.

[0297] Examples of supporting studies described in this application were used to determine the scaling factors used to calculate the stability values corresponding to maximum thresholds. For investigation, as described in this application, these scaling factors in this example were determined based on the intensities of normal healthy tissue at various reference regions. For example, multiplying the blood pool reference by a factor of 1.6 based on the intensities of the aortic region produced levels generally greater than 95% of aortic intensity values but less than normal normal uptake in the prostate. Therefore, higher values were used in the specific example threshold functions. In particular, factor 2 was determined to achieve a level higher than most intensities generally in normal prostate tissue. This value was determined manually based on examinations of histograms and image projections of sagittal, coronal and transverse sections of PET image voxels within prostate volumes, but any portions corresponding to tumor uptake were excluded. Exemplary image slices showing scaling factors and corresponding histograms are shown in FIG. 18G.

i. introduction

[0298] Defining the volumes of a lesion in PET/CT can be a subjective process, as lesions appear as hotspots in PET and generally do not have clear boundaries. The volumes of the lesion can often be segmented based on the anatomical extent of the CT image, but this approach will ignore specific information about tracer uptake since full uptake will not be included. Also, certain lesions can be seen on functional PET images, but not on CT images. This section describes an exemplary study designed threshold setting methods that aim to accurately identify hotspot volumes that reflect the physiological uptake volume, i.e. the volume where uptake is higher than the background. To perform segmentation and identify hotspot volumes in this way, thresholds are chosen to balance the risk of including the background against the risk of not segmenting a hotspot volume that is large enough to reflect the entire absorbent volume.

[0299] This risk balance is typically addressed by selecting 50% or 40% of the SUV _max value determined for the potential hotspot volume as a threshold. The rationale for this approach is to set the threshold higher for high-absorption lesions (e.g., corresponding to high-intensity hotspots on the PET image) than for low-absorptive lesions (e.g., corresponding to lower intensity hotspots). While it can be, it maintains the same level of risk as not dividing the volume representing the total absorption volume as the hotspot volume. However, for hotspots with a low signal-to-noise ratio, using a threshold of 50% of SUV _max will include the background in the segmentation. To avoid this, one can use a decreasing percentage of SUVmax starting at 90% or 75% for low intensity hotspots, for example. Moreover, the risk of background inclusion is lowered as soon as the threshold is sufficiently higher than the background level, which occurs for thresholds well below 50% of SUVmax for highly absorptive lesions. Thus, the threshold can be limited at a stable level higher than typical background intensities.

[0300] One reference for a level of absorption intensity much higher than the normal background absorption intensity is average liver absorption. Based on actual background absorption intensities, other reference levels may be desirable. Background uptake intensities are different in bone, lymph and prostate, with bone having the lowest background uptake and prostate having the highest background uptake. It is advantageous/preferable to use the same threshold setting method regardless of tissue, as the same segmentation method can be used regardless of the location and/or classification of a particular lesion. Thus, this example study evaluates thresholds using the same threshold parameter for lesions in all three tissue types. The adaptive thresholding variants evaluated in this example include variants that are stable at hepatic uptake intensity, stable at the estimated level of aortic uptake, and several variants that are stable at the estimated level above the prostate uptake.

[0301] Certain previous approaches have determined mediastinal blood pool uptake intensity calculated by adding 2x standard deviation of the blood pool uptake intensity to the mean of the blood pool uptake intensity (e.g., mean of the blood pool uptake intensity + 2 x SD). The level was determined as a function of However, this approach, which relies on an estimate of the standard deviation, can introduce undesirable errors and noise sensitivities. In particular, estimating the standard deviation is much less robust than estimating the mean and may be affected by noise, minor segmentation errors, or PET/CT misalignment. A more robust method of estimating levels above blood absorption intensity uses a fixed factor multiplied by the mean or aortic reference value. In order to find a suitable factor, the distributions of absorption intensity in the aorta are studied and described in this example. Normal prostate absorption intensities were studied to determine appropriate factors that could be applied to the reference aortic absorption to calculate levels above normal prostate intensities in general.

ii. methods

Threshold setting manual annotations

[0302] This study used a subset of the data that included only lesions with at least one other lesion of the same type in the same patient. These results resulted in a data set with 684 manually segmented lesion resorption volumes (278 from bone, 357 from lymph nodes, and 49 from prostate) across 92 patients. Automatic refinement by threshold setting was performed and the output was compared to the original volumes. Performance was measured by a weighted average of rank correlations between refined and original volumes within the patient and tissue type, with a weight given by the number of volumes of segmented hotspots in the patient. This performance measure indicates whether the relative sizes between the segmented hotspot volumes are preserved, but ignores the optionally defined absolute sizes since the absorption volumes do not have clear boundaries. However, for a particular patient and tissue type, the same nuclear medicine reader created all annotations and therefore they were written in a systematic manner with smaller lesion annotations reflecting smaller uptake volumes compared to larger lesion annotations in practice. it can be assumed that

Threshold setting of automatically detected lesions

[0303] This study used a subset of the data not used to train the machine learning modules used for hotspot detection and segmentation and 285 manually segmented lesion absorption volumes (104 Dog bones, 129 lymph, 52 prostate) were created. Refined (and unrefined) automatically detected lesions consistent with manually segmented lesions (sensitivity 90% to 91% for bone, 92% to 93% for lymph, and 94% to 98% for prostate). Precision and recall were measured between volumes. These performance measures quantify the similarity between automatically detected and possibly refined hotspots and manually annotated hotspots.

blood absorption

[0304] For 242 patients, the thoracic portion of the aorta was segmented in the CT component using a deep learning pipeline. The segmented aortic volume was projected onto the PET space and eroded by 3 mm to minimize the risk of the aortic volume including areas outside the aorta or in the vessel wall while maximally maintaining uptake inside the aorta. For the remaining absorption intensities, the coefficient q = (aortaMEAN + 2 x aortaSD) / aortaMEAN was calculated for each patient.

prostate absorption

[0305] In 29 patients, normal resorption of the prostate was studied. This study was performed using segmented prostate volumes determined through a machine learning module. Absorption intensities of manually annotated prostate lesions were excluded. Referring to the example in FIG. 18G , the remaining absorption intensities normalized to the aortic reference absorption intensities were visualized by histograms as well as maximum intensity projections in the axial, sagittal and coronal planes. The purpose of maximal projection was to find explanations for the extrinsic intensities of the histograms, especially intensities associated with bladder uptake that are higher than the maximal uptake in healthy tissue.

Threshold setting methods

[0306] The two basic methods (a fixed threshold at SUV = 2.5 and a relative threshold at 50% of SUVmax) were compared with 6 variants of adaptive thresholds. Adaptive thresholds were defined using three threshold functions, each associated with a specific range of SUVmax values. especially:

(1) Low Range Threshold Function: The thresholds for SUV _max values in the low range were calculated using the first threshold function. The first threshold function computed threshold values as a fixed (high) percentage of SUV _max .

(2) Mid-Range Threshold Function: The second threshold function was used to calculate the thresholds for mid-range SUV _max values. The second threshold function calculated the threshold as a percentage of the linear decrease in SUV _max and bounded to the maximum threshold equal to the threshold at the upper end of the range.

(3) High Range Threshold Functions: Thresholds for SUV _max values in the high range were calculated using the high range threshold function. The high range threshold function set thresholds at either a maximum fixed threshold (saturation thresholds) or a fixed (lower) percentage of the SUV _max (non-saturation thresholds).

[0307] The exact parameters of the three threshold functions and ranges described above vary among the various adaptive threshold algorithms and are listed in Table 2 below.

[ Table 2 ]

Ranges and Parameters for Threshold Functions of Adaptive Threshold Algorithms

[0308] The interpolated percentage used in the middle SUV _max range was calculated for P9050-sat in the following way:

Here, 50% of SUV _high is 2 x [aortic absorption intensity], and 90% of SUV _low is equal to aortic absorption intensity.

[0309] Interpolated percentages used in other adaptive threshold setting algorithms are calculated similarly. Then, the mid-range threshold is:

And similar to other adaptive threshold setting algorithms.

iii. results

Threshold setting of manually annotated lesions

[0310] The highest weighted rank correlations (0.81) were obtained with the P9050-sat and P9040-sat methods, with P7540-sat, A9050-sat and L9050-sat also providing high values. The 50% of relative SUV _max (0.37) and P9050-non-sat (0.61) thresholding approaches exhibited the lowest weighted rank correlations. A fixed threshold at SUV = 2.5 showed a rank correlation lower than most of the adaptive threshold setting approaches (0.74). The results of the weighted rank correlation for each of the threshold approaches are summarized in Table 3 below.

[ Table 3 ]

Weighted average of rank correlations for thresholding approaches evaluated

Threshold of automatically detected lesions

Unrefined automatic hotspot detections had low precision (0.31 to 0.47) but high recall (0.83 to 0.92) with excessive segmentation. Refinement using a 50% threshold setting algorithm of relative SUVmax improved precision (0.70 to 0.77), but reduced recall to about 50% (0.44 to 0.58). Refinement using P9050-sat improved precision (0.51 to 0.84) and reduced the decrease in recall (0.61 to 0.89), showing a balance with fewer over-divisions but more under-divisions (0.61-0.89). P9040-sat performed similarly to P9050-sat in these respects, while L9050-sat had the highest precision (0.85 to 0.95) but the lowest recall (0.31 to 0.56). Tables 4a-e show the overall results for precision and recall.

[ Table 4a ]

Precision and recall values without analytical segmentation refinement

[ Table 4b ]

Precision and recall values refined through a 50% threshold approach of the relative SUV _max

[ Table 4c ]

Precision and recall values with adaptive segmentation using the P9050—sat implementation

[ Table 4d ]

Precision and recall values with adaptive split t using P9040—sat implementation

[ Table 4e ]

Precision and recall values with adaptive segmentation using the L9050-sat implementation

Support for Threshold Setting Methods: Blood Absorption

[0312] For the resulting coefficients, qMEAN + 2 x qSD was 1.54, so using a factor of 1.6 was determined to be a good candidate for achieving a threshold level that exceeds most blood absorption intensity values. In the exemplary study, only 3 patients had aortaMEAN + 2 x aortaSD greater than 1.6 x aortaMEAN. Three external patients had q = 1.64, 1.92 and 1.61, the patient with a factor of 1.92 had an erroneous aortic segment spill into the spleen and the other patients had coefficients close to 1.6.

Support for Threshold Setting Methods: Prostate Absorption

[0313] Based on manual review of histograms of normal prostate intensities with projections of axial, sagittal and coronal planes in mind, a value of 2.0 applied to the aortic reference value is an appropriate scale to obtain a level higher than normal prostate absorption intensity. It is a factor.

H. Example: Use of AI-based hotspot segmentation compared to thresholding alone

[0314] In this example, hotspot detection and segmentation performed using an AI-based approach using machine learning modules to segment and classify hotspots as described in this application differs from the conventional approach using threshold-based segmentation alone. have been compared

[0315] FIG. 19A shows a conventional hotspot segmentation approach 1900 without using machine learning techniques. Instead, hotspot segmentation is performed based on manual description of hotspots by the user, followed by intensity (eg SUV) based threshold setting 1904 . The user masks manually place 1922 a circular marker representing a region of interest (ROI) 1924 within the image 1920 . Once the ROI is placed, either a fixed or relative threshold approach can be used to segment hotspots within the manually placed ROI 1926 . The relative thresholding approach individually sets the threshold for a particular ROI to a fixed percentage of the maximal SUV within the ROI, and segments each user-identified hotspot using an SUV-based thresholding approach to segment the boundaries created by the initial user. split the This conventional approach can be time consuming, as it relies on the user manually identifying and creating the boundaries of the hotspots, furthermore the segmentation results as well as the downstream quantification 1906 (e.g., of the hotspot metrics). calculation) may vary from user to user. Furthermore, as conceptually illustrated in images 1928 and 1930, depending on a particular threshold, different thresholds may produce different hotspot segmentations 1929, 1931. SUV threshold levels can also be adjusted to detect early-stage disease, but doing so often results in a large number of false positives, distracting attention from true positives. Finally, conventional fixed or relative SUV-based threshold approaches, eg, as described herein, suffer from over- and/or under-estimation of lesion size.

[0316] Referring to Fig. 19B, instead of using manual user-based selection of ROIs containing hotspots in conjunction with SUV-based threshold setting, an AI-based approach 1950 in accordance with certain embodiments described herein may be used to detect hotspots ( 1956) to automatically analyze CT 1954 and PET images 1952 (eg, synthetic PET/CT) to detect, segment, and classify. As described in further detail in this application, machine learning based hotspot segmentation and classification can be used to generate an initial 3D hotspot map, which is then described in this application using the adaptive thresholding technique, e.g., Sections F and It can be used as an input to an analytic segmentation method (1958) as described in G. Among other things, the use of machine learning approaches reduces user subjectivity and time required to review images (eg, by medical practitioners such as radiologists). Additionally, AI models can perform complex tasks and identify high-stakes metastatic disease as well as early-stage lesions while maintaining low false-positive rates. Improved hotspot segmentation in this way improves the accuracy of related downstream quantifications 1960 to measure metrics that may be used to assess disease severity, prognosis, treatment response, and the like, among others.

[0317] Figure 20 shows the improved performance of the machine learning based segmentation approach compared to conventional thresholding methods. In a machine learning based approach, hotspot segmentation was performed by first detecting and segmenting hotspots using machine learning modules, as described in this application (eg, in Section E) along with the adaptation described in Sections F and G. was refined using an analytical model implementing a version of the thresholding technique. A conventional threshold setting method performs segmentation of clusters of voxels having intensities exceeding a fixed threshold using a fixed threshold setting. As can be seen in FIG. 20, the conventional threshold setting method generates false positives (2002a and 2002b) due to absorption of radiopharmaceuticals in the urethra, whereas the machine learning segmentation technique accurately ignores urethral absorption and produces prostate lesions (2004). and 2006) only.

[0318] FIGS. 21A to 21I show hotspot segmentation results in the abdominal region performed by a conventional threshold setting method (left images) and results (right images) of a machine learning approach according to an embodiment described in the present application. Compare. 21A to 21I show a series of 2D slices of a 3D image moving along the vertical direction of an abdominal area with hotspot areas identified by each method overlaid. The results shown in the figures show that abdominal uptake is a problem for the conventional thresholding approach, with large false positive areas appearing in the left lateral images. This can result from high absorption in the kidneys and bladder. Conventional segmentation approaches require complex methods to suppress this uptake and limit these false positives. In contrast, the machine learning model used to segment the images shown in FIGS. 21A-21I did not rely on any such suppression and instead learned to ignore this kind of absorption.

I. CAD Device Implementation Example

[0319] This section describes example CAD device implementations in accordance with the specific embodiments described herein. The CAD device described in this example is called “aPROMISE” and uses several machine learning modules to perform automated organ segmentation. An example CAD device implementation uses analytical models to perform hotspot detection and segmentation.

[0320] The aPROMISE (Automated PRO state specific Membrane Antigen Imaging SE gmentation) implementation example described in this example allows users to convert PSMA PET/CT image data into DICOM files. It uses a cloud-based software platform with a web interface to upload scans, review patient studies within teams, and share study evaluations. This software complies with the DICOM (Digital Imaging and Communications in Medicine) 3 standard. Multiple scans can be uploaded for each patient and the system provides a separate review for each study. The software provides a review page that allows users to display and view studies in a 4-panel view showing PET, CT, PET/CT fusion and maximum intensity projection (MIP) simultaneously. It includes a GUI and includes options to display each view separately. This device is used to review full patient studies using image visualization and analysis tools to allow users to identify and mark regions of interest (ROIs). While reviewing the image data, users can either select from predefined hotspots that are highlighted when hovering the mouse pointer over the segmented area or manually create (e.g., select individual image slices to include as hotspots). Selecting voxels) can display the ROI. Quantitative analysis is performed automatically on selected or (manually) created hotspots. The user can review the results of this quantitative analysis and decide which hotspots should be reported as suspicious lesions. In aPROMISE, a region of interest (ROI) represents a contiguous sub-portion of an image, a hotspot refers to a ROI with high local intensity (eg, exhibits high absorption) (eg, relative to surrounding regions), and a lesion represents a user-defined or user-selected ROI in which the disease is considered suspect.

[0321] To generate the report, the software of the example implementation must allow the user to sign to verify quality control and electronically sign the report preview. Signed reports are stored on the device and can be exported as JPG or DICOM files.

[0322] The aPROMISE device is implemented with a microservices architecture, as described in more detail in this application and shown in FIGS. 29A and 29B.

i. work flow

[0323] Figure 22 depicts the workflow of an aPROMISE device from uploading DICOM files to exporting electronically signed reports. Upon logging in, the user can import DICOM files into aPROMISE. Imported DICOM files are uploaded to a patient list where the user can click on a patient to display the corresponding studies available for review. The layout principles for the patient list are shown in FIG. 23 .

[0324] This view 2300 lists all patients with studies uploaded within the team and displays patient information (name, ID and gender), date of recent study upload, and study status. Study status is: studies ready for review (blue symbols, 2302), studies with errors (red symbols, 2304), studies under calculation (orange symbols, 2304), and studies with available reports per patient (black). symbol, 2308) indicates cognition. The number in the upper right corner of the condition symbol indicates the number of studies with a particular condition per patient. Review of a study begins by clicking on a patient, selecting a study, and identifying whether the patient has had a prostatectomy. The study data is opened and displayed in the review window.

[0325] FIG. 24 shows a review window 2400 through which a user can review PET/CT image data. The lesions are manually marked and reported by the user selecting from either pre-defined hotspots segmented by the software or user-defined hotspots created using a creation tool to select voxels to include as hotspots in the program. . Predefined hotspots, regions of interest with high local intensity absorption, are automatically segmented using specific methods for soft tissue (prostate and lymph nodes) and bone and highlighted when hovering the mouse pointer over the segmented region. The user may choose to turn on the Split Display option to simultaneously visually display splits of predefined hotspots. The selected or created hotspots are subjects for automated quantitative analysis and are detailed in panels 2402 , 2422 and 2442 .

[0326] The collapsible panel 2402 on the left summarizes the patient and study information extracted from the DICOM data. Panel 2402 also displays and lists quantitative information about user-selected hotspots. Hotspot location and type are manually validated—T: limited to primary tumor, N: locally metastatic disease, Ma/b/c: distant metastatic disease (lymph node, bone and soft tissue). The device displays an automated quantitative analysis of user-selected hotspots—SUV-Max, SUV-Peak, SUV-Mean, Lesion Volume, Lesion Index (LI)—to indicate which hotspot the user will report as lesions in a standardized report. review them and make decisions.

[0327] Middle panel 2422 includes a four panel view representation of DICOM image data. A CT image is displayed in the upper left corner, a PET/CT fusion view is displayed in the upper right corner, a PET image is displayed in the lower left corner, and a MIP is shown in the lower right corner.

[0328] MIP is a visualization method of volume data that displays a 2D projection of a 3D image volume from various viewing angles. MIP imaging was performed by Wallis JW, Miller TR, Lerner CA, Kleerup EC. 3D representation in nuclear medicine. IEEE Transmedical Imaging. 1989;8(4):297-30. doi: 10.1109/42.41482. PMID: 18230529.

[0329] The collapsible right panel 2442 includes the following visualization controls for optimizing image review and shortcut keys for manipulating the image for review purposes.

[0330] Viewport:

십자선의 유무

presence or absence of crosshairs

PET/CT 융합 영상에 대한 페이딩(fading) 옵션

Fading options for PET/CT fusion images

PET 추적자 흡수 강도들을 시각화하기 위한 표준 핵의학 컬러맵 선택.

Selection of a standard nuclear medicine colormap to visualize PET tracer uptake intensities.

[0331] SUV and CT windows:

명암 스트레칭, 히스토그램 변경 또는 명암 향상으로도 알려진 영상들의 윈도우잉(windowing)은 영상들이 강도를 통해 조작되어 특정 구조들을 강조하기 위해 그림의 모양을 변경한다.

Windowing of images, also known as intensity stretching, histogram alteration or intensity enhancement, is where the images are manipulated through their intensity to change the shape of the picture to emphasize certain structures.

SUV 창에서, SUV 강도들에 대한 창 프리셋(preset)들은 슬라이더 또는 단축키들로 조정할 수 있다.

In the SUV window, window presets for SUV intensities can be adjusted with sliders or hotkeys.

CT 창에서, 하운스필드 강도들에 대한 창 프리셋들은 단축키들을 사용하거나 클릭 앤 드래그(click―and―drag) 입력을 사용하여 드롭―다운(drop―down) 목록에서 선택할 수 있다. 여기에서 영상의 밝기는 창 수준을 통해 조정되고 대비는 창 넓이를 통해 조정된다.

In the CT window, window presets for Hounsfield intensities can be selected from a drop-down list using shortcuts or click-and-drag input. Here, the brightness of the image is adjusted through the window level and the contrast is adjusted through the window width.

[0332] division

장기 분할 표시 옵션은 참조 장기들 분할 또는 전신 분할의 시각화를 켜거나 끈다.

The Show Organ Segmentation option turns on or off the visualization of reference organs segmentation or whole body segmentation.

사용자는 장기 분할을 표시하기 위한 패널 보기들을 선택할 수 있다.

The user can select panel views for displaying the organ segmentation.

핫스팟 분할 표시 옵션은 선택된 영역들에서 미리 정의된 핫스팟들의 표시를 켜거나 끄도록 선택한다 (골반 영역, 뼈 또는 모든 핫스팟들에서).

The Show Hotspot Splits option chooses to turn on or off the display of predefined hotspots in selected areas (in the pelvic area, bone or all hotspots).

[0333] Viewer gestures

줌(Zoom), 팬(Pan), CT 창(window), 슬라이스 변환 및 검토 창의 핫스팟 숨김에 대한 단축키 및 조합들.

Shortcuts and combinations for zoom, pan, CT window, transform slice and hide hotspot in review window.

[0334] To proceed with generating the report, the signing user clicks the Generate Report button 2462. Users should check the following quality control items before generating a report:

영상 품질은 허용 가능하다

Video quality is acceptable

PET 및 CT 영상들이 올바르게 정렬된다

PET and CT images are correctly aligned

환자 연구 데이터는 정확하다

Patient study data are accurate

참조 값들 (혈액 풀, 간)은 허용 가능하다

Reference values (blood pool, liver) are acceptable

연구는 수퍼스캔이 아니다

Research is not superscan

[0335] After checking the quality control items, a report preview is displayed for the user's electronic signature. The report includes a patient summary to be identified as a lesion by the user, a quantitative assessment of the total quantitative lesion burden and individual lesions of the user selected hotspots.

[0336] FIG. 25 shows an exemplary generated report 2500. Report 2500 includes three sections 2502, 2522 and 2542.

[0337] Section 2502 of report 2500 provides a summary of patient data obtained from DICOM tags. This includes a summary of the patient - patient name, patient ID, age and weight - and summary of study data - study date, dose at injection, radiopharmaceutical imaging tracer used and its half-life, and time between tracer injection and image data acquisition. time - included.

[0338] Section 2522 of report 2500 provides quantitative information summarized from hotspots selected by the user to be included as lesions. Summarized quantitative information indicates total lesion burden per lesion type (primary prostate tumor (T), regional/regional pelvic lymph nodes (N) and distant metastases - lymph nodes, bones or soft tissue organs (Ma/b/c)). Summary section 2522 also displays the observed quantitative uptake (SUV-average) in the reference organs.

[0339] Section 2542 of report 2500 is a detailed quantitative assessment and location of each lesion from selected hotspots identified by the user. When reviewing the report, the user must electronically sign their patient study review results, including hotspots and quantifications selected as lesions. The report is then saved to the device and can be exported as a JPG or DICOM file.

ii. image processing

Preprocessing of DICOM input data

[0340] Image input data is provided in DICOM format, which is a rich data expression. DICOM data includes intensity data as well as meta data and communication structures. To optimize data for use with aPROMISE, data is passed through microservices that re-encode, compress, and remove unnecessary or sensitive information. It also collects intensity data from separate DICOM series and encodes the data into a single lossless PNG file with an associated JSON meta-information file.

[0341] Data processing of PET image data includes estimation of the SUV (standard absorption value) factor included in the JSON meta information file. The SUV factor is a scalar used to convert image intensities to SUV values. SUV factors are calculated according to QIBA (Quantitative Imaging Biomarkers Alliance) guidelines.

Algorithmic image processing

[0342] FIG. 26 shows an example image processing workflow (process) 2600.

[0343] aPROMISE segments 2602 the patient skeleton and selected organs using a convolutional neural network (CNN) model. Organ segmentation 2602 allows for automated calculation of standard absorption value (SUV) references in the patient's 2604 aorta and liver. The SUV references for the aorta and liver are then used as reference values when determining specific SUV-value based quantitative indices such as the lesion index (LI) and intensity-weighted tissue lesion volume (ITLV). A detailed description of the quantitative indices is provided in Table 6 below.

[0344] User selecting from either predefined hotspots 2608a segmented by software and user defined hotspots created using authoring tool 2608b to select voxels to include as hotspots in the program. (2608) manually marks and reports lesions. Predefined hotspots, regions of interest with high local intensity, are automatically segmented using certain specific methods for soft tissue (prostate and lymph nodes) and bone (e.g. one for bone, as seen in FIG. 28). A specific method of segmentation of , and other specific segmentation methods for soft tissue regions may be used). Based on the organ segmentation, the software determines the type and location of selected hotspots in prostate, lymph or bony regions. The determined types and locations are displayed in a list of selected hotspots shown in panel 2502 of viewer 2400. The type and location of selected hotspots in other regions (eg not located in prostate, lymph or bony regions) are manually added by the user. The user can add and edit all applicable hotspot types and locations at any time during hotspot selection. Hotspot type is determined using the miTNM system, which is a clinical standard and notation system for reporting cancer metastasis. In this approach, individual hotspots are assigned a type according to a character-based code representing specific physical characteristics, such as:

T는 원발성 종양을 나타낸다

T indicates primary tumor

N은 원발성 종양의 영향을 받는 근처의 림프절을 나타낸다

N indicates nearby lymph nodes affected by the primary tumor

M은 원격 전이를 나타낸다

M indicates distant metastasis

[0345] For distant metastatic lesions, the localization is grouped into a/b/c - system corresponding to additional pelvic lymph nodes (a), bones (b) and soft tissue organs (c).

[0346] For all hotspots selected for inclusion as lesions, SUV-values and indices are calculated (2610) and displayed in a report.

Organ segmentation on CT

[0347] Organ segmentation 2602 is performed using CT images as input. Starting with two coarse segmentations from the full image, smaller image sections are extracted and selected to contain a given set of organs. Fine segmentation of organs is performed in each imaging section. Finally, all segmented organs from all image sections are assembled into a full image segmentation displayed in aPROMISE. Successfully completed segmentation identified 52 different bones and 13 soft tissue organs as visualized in FIG. 27 and is presented in Table 5. Both coarse and fine segmentation processes include the following three steps:

1. Preprocessing of CT images,

2. CNN segmentation, and

3. Post-process the segmentation.

[0348] CT image preprocessing prior to coarse segmentation involves three steps: (1) removing image slices representing only air (e.g., having less than 0 Hounsfield units), (2) fixing the image. re-sampling to a size equal to the size specified, and (3) normalizing the image as described below based on the mean and standard deviation for the training data;

[0349] In CNN models, a label corresponding to a background or a segmented organ is assigned to each pixel of an input image to perform semantic segmentation where a label map of the same size is input data.

[0350] Post-processing is performed after segmentation and includes the following steps:

- Absorbs adjacent pixel clusters once.

- Absorb adjacent pixel clusters until no such cluster exists.

- Remove all non-largest clusters of each label.

— Discard skeleton parts from segmentation; Some segmentation models segment skeletal parts as reference points when segmenting soft tissue. It is believed that the skeletal parts of these models will be removed after segmentation is complete.

[0351] Two different coarse segmentation neural networks and 10 different fine segmentation neural networks are used, including segmentation of the prostate. If the patient had a prostatectomy prior to examination—information provided by the user when validating the patient study background prior to opening the study for review—the prostate is not segmented. Combinations of fine and coarse segmentation and the body parts each combination provides are presented in Table 5.

[ Table 5 ]

Summary of how to segment different body parts by combining coarse and fine segmentation networks

[0352] CNN model training involves an iterative minimization problem in which the training algorithm updates model parameters to lower the segmentation error. Segmentation error is defined as the deviation from perfect overlap between manual segmentation and CNN model segmentation. Each neural network used for organ segmentation was trained to construct optimal parameters and weights. Training data for developing neural networks for aPROMISE consisted of low-dose CT images with manually segmented and labeled body parts as described above. CT images for training segmentation networks were obtained as part of the NIMSA project (https://nimsa.se/) and clinicaltrials.gov (https://www.clinicaltrials.gov/ct2/show/NCT01667536?term=99mTc―MIP -1404&draw=2&rank=5) was collected during the phase 2 clinical trial of registered drug candidate 99mTc-MIP-1404. The NIMSA project consisted of 184 patients and the 99mTc-MIP-1404-data consisted of 62 patients.

Calculation of reference data (SUV reference) in PSMA PET

[0353] Reference values are used when evaluating the physiological uptake of the PSMA tracer. Current clinical practice is to use SUV intensities of identified volumes corresponding to blood pool or liver or both tissues as reference values. For PSMA tracer intensity, the blood pool is measured as aortic volume.

[0354] The intensities of the volumes corresponding to the thoracic portion of the aorta and liver in aPROMISE SUV are used as reference values. The uptake registered on the PET image along with organ segmentation of the aortic and liver volumes is the criterion for calculating the SUV reference in each organ.

[0355] Aorta. The segmented aortic volume is reduced to ensure that portions of the image corresponding to the vessel wall are not included in the volume used to compute the SUV reference for the aortic region. Segmentation reduction (3 mm) was chosen empirically to balance the tradeoff of keeping as much aortic volume as possible while not including vessel wall regions. The reference SUV for the blood pool is the robust average of the SUVs at the pixels inside the reduced segmentation mask identifying the aortic volume. The robust mean is calculated as the mean of the values in the interquartile range.

[0356] Liver. When measuring the reference value in the liver volume, the segmentation is reduced along the edges to create a buffer that adjusts for possible misalignment between the PET and CT images. The amount of reduction (9 mm) was determined empirically using manual observations of images with PET/CT misalignment.

[0357] A cyst or malignancy of the liver may result in areas of low tracer uptake in the liver. To reduce the influence from these local differences in tracer uptake on the calculation of the SUV reference, a two-component Gaussian mixture model approach was used according to the embodiment described in Section B.iii above with respect to FIG. 2A. Specifically, a two-component Gaussian mixture model was fitted to the SUVs from the voxels inside the reference organ mask and the major and minor components of the identified distribution. The SUV reference for liver volume was initially calculated as the mean SUV of principal components from a Gaussian mixture model. If a subcomponent is determined to have a larger average SUV than the main component, then the liver reference organ mask remains unchanged unless the weight of the subcomponent is greater than or equal to 0.33 - in which case, if the weight of the subcomponent was greater than or equal to 0.33. An error will occur and the inter-reference value will not be calculated.

[0358] If the minor component has a smaller average SUV than the major component, a separation threshold is calculated, for example, as can be seen in FIG. 2A. The separation threshold is defined as:

임계값 이상에서 SUV에 대한 주요 성분에 속할 확률, 및;

the probability of belonging to the principal component for the SUV above the threshold, and;

임계값 이하에서 SUV에 대한 부 성분에 속할 확률은

Below the threshold, the probability of belonging to the minor component for SUV is

same.

[0359] The reference mask is then refined by removing pixels below the separation threshold.

Pre-definition of hotspots in PSMA PET

[0360] Referring to Fig. 28, in the aPROMISE implementation of this example, segmentation of regions with high local intensity of PSMA PET by aPROMISE, so-called predefined hotspots, are determined in PET images 2802 and CT images, performed by the analytic model 2800 based on input from the organ segmentation map 2804 projected into PET space. For the software to segment bony hotspots, the original PET images 2802 are used and the PET images are processed to suppress normal PET tracer uptake 2806 to segment lymphatic and prostate hotspots. A graphical overview of the analytic model used in this exemplary implementation is presented in FIG. 28 . The analytical method, as further described below, is designed to find regions of high local absorption intensity that can represent ROIs without an excessive number of extraneous regions or PET tracer background noise. An analytical method was developed from a labeled data set consisting of PSMA PET/CT images.

[0361] Suppression of normal PSMA tracer uptake intensity (2806) was performed in one high uptake organ at the time. First, the intensity of absorption in the kidney is suppressed, then the liver, and finally the urinary bladder. Suppression is performed by applying the estimated suppression map to the high-intensity regions of the PET. The suppression map is created using the organ map previously segmented from the CT and projected and adjusted onto the PET image to create a PET adjusted organ mask.

[0362] This adjustment corrects for small misalignments between the PET and CT images. Using the adjusted map, a background image is computed. This background image is subtracted from the original PET image to create an absorption estimation image. Then, we estimate the suppression map from the absorption estimate image using an exponential function that depends on the Euclidean distance from the voxel outside the segmentation to the PET-adjusted organ mask. An exponential function is used because the intensity of absorption decreases exponentially with distance from the organ. Finally, a suppression map is subtracted from the original PET image to suppress the intensities associated with normal high uptake in the organ.

[0363] After suppression of normal PSMA tracer uptake intensity, hotspots are segmented in prostate and lymph 2812 using organ segmentation mask 2804 and suppressed PET image 2808 produced by suppression step 2806. Prostate hotspots are not segmented for patients undergoing prostatectomy. Bone and lymphatic hotspot segmentations are applicable for all patients. Each hotspot is segmented using a fast-marching method in which the base PET image is used as a velocity map and the volume of the input region determines the moving time. The input region is also used as an initial segmentation mask to identify the volume of interest for the fast-paced method and is created differently depending on whether the hotspot segmentation is performed on bone or soft tissue. Bone hotspots are segmented using the rapid progression method and Gaussian's difference (DoG) filtering approach 2810 and limp, where applicable, prostate hotspots are segmented using the rapid progression method and Gaussian's Laplacian (LoG) filtering approach 2812. Divided.

[0364] For detection and segmentation of bone hotspots, a bone region mask is created to identify a bone volume in which bone hotspots can be detected. The bone region mask consists of the following bone regions: thoracic vertebrae (1-12), lumbar vertebrae (1-5), clavicle (L+R), scapula (L+R), sternal ribs (L+R, 1- 12), hip bone (L+R), femur (L+R), sacrum and coccyx. The masked image is normalized based on the mean intensity of healthy bone tissue in the PET image by iteratively normalizing the image using DoG filtering. The filter sizes used in DoG are 3 mm/spacing and 5 mm/spacing. DoG filtering acts as a band pass filter that impairs signals farther from the band center in the image, which emphasizes clusters of voxels with higher intensities relative to their periphery. Thresholding the normalized image acquired in this way creates clusters of voxels that can be differentiated from the background and segmented accordingly to create a 3D segmentation map 2814 that identifies hotspot volumes located in bone regions.

[0365] For detection and segmentation of lymphatic hotspots, a lymphatic area mask capable of detecting hotspots corresponding to potential lymph nodes is created. The lymphatic region mask includes voxels within the bounding box surrounding all segmented bone and organ regions, but excludes voxels within the segmented organs themselves, and voxels excluding lung volumes are retained. Another prostate area mask is created so that hotspots corresponding to potential prostate tumors can be detected. This prostate region mask is a 1 voxel extension of the prostate volume determined from the organ segmentation step described in this application. Applying a lymphatic region mask to a PET image creates a masked image that includes voxels within the lymphatic region (eg, excluding other voxels), and similarly, applying a prostate region mask to a PET image includes voxels within the prostate volume. A masked image is created.

[0366] Soft tissue hotspots - i.e., lymph and prostate hotspots - were placed on three different sizes of LoG filters on the lymph and/or prostate masked images - one 4 mm/spacing XYZ, one 8 mm/spacing XYZ, and one 12 mm/spacing XYZ - separately applied to generate three LoG filtered images for each of the two soft tissue types (prostate and lymph). For each soft tissue type, the three corresponding LoG filtered images are thresholded using minus 70% of the aortic SUV reference, then a local minimum is found using a 3 x 3 x 3 minimum filter. This approach produces three filtered images each containing clusters of voxels corresponding to hotspots. The three filtered images are combined by taking the sum of local minima from the three images to create a hotspot area mask. Each component of the hotspot area mask is segmented using a leveling method to determine one or more hotspot volumes. This segmentation approach is performed for both prostate and lymphatic hotspots to automatically segment hotspots in the prostate and lymphatic regions.

iii. quantification

[0367] Table 6 identifies the values calculated by the software that are displayed for each hotspot after the user selects it. ITLV is an aggregated value and only appears in reports. All calculations are SUV variants of PSMA PET/CT.

[Table 6]

Values calculated by aPROMISE

iv. Web-based platform architecture

[0368] aPROMISE uses a microservices architecture. Deployment to AWS is handled in cloud formation scripts located in the AWS code repository. The aPROMISE cloud architecture is provided in Fig. 29a, and the microservice communication design chart is provided in Fig. 29b.

J. Contrast

i. PET Imaging Radionuclide Labeled PSMA Binder

[0369] In certain embodiments, the radionuclide-labeled PSMA binder is a radionuclide-labeled PSMA binder suitable for PET imaging.

[0370] In certain embodiments, the radionuclide labeled PSMA binder comprises [ 18 F]DCFPyL (also referred to as PyL ^TM ; also referred to as DCFPyL-18F):

[18F] DCFPyL,

or a pharmaceutically acceptable salt thereof.

[0371] In certain embodiments, the radionuclide labeled PSMA binder comprises [ 18 F]DCFBC:

[18F] DCFBC,

or a pharmaceutically acceptable salt thereof.

[0372] In certain embodiments, the radionuclide labeled PSMA binder comprises ⁶⁸ Ga-PSMA-HBED-CC (also referred to as ⁶⁸ Ga-PSMA-11):

^68Ga -PSMA-HBED-CC,

or a pharmaceutically acceptable salt thereof.

[0373] In certain embodiments, the radionuclide labeled PSMA binder comprises PSMA-617:

PSMA-617;

or a pharmaceutically acceptable salt thereof. ^In certain embodiments, the radionuclide labeled PSMA binder comprises ⁶⁸ Ga-PSMA-617, which is PSMA-617 labeled as 68 Ga, or a pharmaceutically acceptable salt thereof. In ^certain embodiments, the radionuclide labeled PSMA binder comprises ¹⁷⁷ Lu-PSMA-617, which is PSMA-617 labeled with 177 Lu, or a pharmaceutically acceptable salt thereof.

[0374] In certain embodiments, the radionuclide labeled PSMA binder comprises PSMA-I&T:

PSMA-I&T,

or a pharmaceutically acceptable salt thereof. In certain embodiments, the radionuclide labeled PSMA binder comprises ⁶⁸ Ga-PSMA-I&T, which is PSMA-I&T labeled with ⁶⁸ Ga, or a pharmaceutically acceptable salt thereof.

[0375] In certain embodiments, the radionuclide labeled PSMA binder comprises PSMA-1007:

PSMA-1007;

or a pharmaceutically acceptable salt thereof. In certain embodiments, the radionuclide labeled PSMA binder comprises ¹⁸ F-PSMA-1007, which is PSMA-1007 labeled with ¹⁸ F, or a pharmaceutically acceptable salt thereof.

ii. SPECT Imaging Radionuclide Labeled PSMA Binder

[0376] In certain embodiments, the radionuclide labeled PSMA binder is a radionuclide labeled PSMA binder suitable for SPECT imaging.

[0377] In certain embodiments, the radionuclide labeled PSMA binder comprises 1404 (also referred to as MIP-1404):

1404;

or a pharmaceutically acceptable salt thereof.

[0378] In certain embodiments, the radionuclide labeled PSMA binder comprises 1405 (also referred to as MIP-1405):

1405;

or a pharmaceutically acceptable salt thereof.

[0379] In certain embodiments, the radionuclide labeled PSMA binder comprises 1427 (also referred to as MIP-1427):

1427;

or a pharmaceutically acceptable salt thereof.

[0380] In certain embodiments, the radionuclide labeled PSMA binder comprises 1428 (also referred to as MIP-1428):

1428;

or a pharmaceutically acceptable salt thereof.

[0381] In certain embodiments, the PSMA binder can be combined with a radioactive isotope of a metal [eg, a radioactive isotope of technetium (Tc) (eg, technetium-99m ( ⁹⁹ mTc)); For example, radioactive isotopes of rhenium (Re) (eg, rhenium-188( ¹⁸⁸ Re); eg, rhenium-186( ¹⁸⁶ Re)); For example, a radioactive isotope of yttrium (Y) (eg, ⁹⁰ Y); For example, radioactive isotopes of lutetium (Lu) (eg, ¹⁷⁷ Lu); For example, a radioactive isotope of gallium (Ga) (eg, ⁶⁸ Ga; eg, ⁶⁷ Ga); For example, radioactive isotopes of indium (eg, ¹¹¹ In); For example, it is labeled as a radionuclide by chelating to a radioactive isotope of copper (Cu) (eg, ⁶⁷ Cu)].

[0382] In certain embodiments, 1404 is labeled as a radionuclide (eg, chelated to a radioactive isotope of a metal). In certain embodiments, the radionuclide-labeled PSMA binder comprises ^99m Tc-MIP-1404, which is 1404 labeled with ^99m Tc (e.g., chelated to ^99m Tc):

^99m Tc-MIP-1404,

or a pharmaceutically acceptable salt thereof. In certain embodiments, 1404 is another metal radioisotope [eg, a radioactive isotope of rhenium (Re) (eg, rhenium-188( ¹⁸⁸ Re); eg, rhenium-186( ¹⁸⁶ Re)) ; For example, a radioactive isotope of yttrium (Y) (eg, ⁹⁰ Y); For example, a radioactive isotope of lutetium (Lu) (eg, ¹⁷⁷ Lu); For example, a radioactive isotope of gallium (Ga) (eg, ⁶⁸ Ga; eg, ⁶⁷ Ga); For example, radioactive isotopes of indium (eg, ¹¹¹ In); for example, a radioactive isotope of copper (Cu) (e.g., ⁶⁷ Cu)] to form a compound having a structure similar to that shown above for ^99m Tc-MIP-1404, and other metal radioactive isotopes. replaces ^99m Tc.

[0383] In certain embodiments, 1405 is labeled as a radionuclide (eg, chelated to a radioactive isotope of a metal). In certain embodiments, the radionuclide labeled PSMA binder comprises ^99m Tc-MIP-1405, which is 1405 labeled with ^99m Tc (e.g., chelated to ^99m Tc):

^99m Tc-MIP-1405,

or a pharmaceutically acceptable salt thereof. In certain embodiments, 1405 is another metal radioactive isotope (eg, a radioactive isotope of rhenium (Re) (eg, rhenium-188( ¹⁸⁸ Re); eg, rhenium-186( ¹⁸⁶ Re))) ; For example, a radioactive isotope of yttrium (Y) (eg, ⁹⁰ Y); For example, radioactive isotopes of lutetium (Lu) (eg, ¹⁷⁷ Lu); For example, a radioactive isotope of gallium (Ga) (eg, ⁶⁸ Ga; eg, ⁶⁷ Ga); For example, radioactive isotopes of indium (eg, ¹¹¹ In); for example, a radioactive isotope of copper (Cu) (e.g., ⁶⁷ Cu)] to form a compound having a structure similar to that shown above for ^99m Tc-MIP-1405, and other metal radioactive isotopes. replaces ^99m Tc.

[0384] In certain embodiments, 1427 is labeled (e.g., chelated) with a radioactive isotope of a metal to form a compound according to the formula:

1427 chelated to metal,

or a pharmaceutically acceptable salt thereof, wherein M is a metal radioisotope labeling 1427 [eg, a radioactive isotope of technetium (Tc) (eg, technetium-99m( ^99m Tc)); For example, radioactive isotopes of rhenium (Re) (eg, rhenium-188( ¹⁸⁸ Re); eg, rhenium-186( ¹⁸⁶ Re)); For example, a radioactive isotope of yttrium (Y) (eg, ⁹⁰ Y); For example, radioactive isotopes of lutetium (Lu) (eg, ¹⁷⁷ Lu); For example, a radioactive isotope of gallium (Ga) (eg, ⁶⁸ Ga; eg, ⁶⁷ Ga); For example, radioactive isotopes of indium (eg, ¹¹¹ In); For example, a radioactive isotope of copper (Cu) (eg, ⁶⁷ Cu)].

[0385] In certain embodiments, 1428 is labeled (e.g., chelated) with a radioactive isotope of a metal to form a compound according to the formula:

1428 chelated to metal,

or a pharmaceutically acceptable salt thereof, wherein M is a metal radioisotope labeling 1428 [eg, a radioactive isotope of technetium (Tc) (eg, technetium-99m( ^99m Tc)); For example, radioactive isotopes of rhenium (Re) (eg, rhenium-188( ¹⁸⁸ Re); eg, rhenium-186( ¹⁸⁶ Re)); For example, a radioactive isotope of yttrium (Y) (eg, ⁹⁰ Y); For example, radioactive isotopes of lutetium (Lu) (eg, ¹⁷⁷ Lu); For example, a radioactive isotope of gallium (Ga) (eg, ⁶⁸ Ga; eg, ⁶⁷ Ga); For example, radioactive isotopes of indium (eg, ¹¹¹ In); For example, a radioactive isotope of copper (Cu) (eg, ⁶⁷ Cu)].

[0386] In certain embodiments, the radionuclide labeled PSMA binder comprises PSMA I&S:

PSMA I&S,

or a pharmaceutically acceptable salt thereof. In certain embodiments, the radionuclide labeled PSMA binder comprises ^99m Tc-PSMA I&S, which is PSMA I&S labeled with ^99m Tc-PSMA I&S, or a pharmaceutically acceptable salt thereof.

K. Computer Systems and Network Architecture

[0387] As can be seen in FIG. 30, an implementation of a network environment 3000 for use in providing the systems, methods and architectures described herein is shown and described. For a brief overview, referring now to FIG. 30 , a block diagram of an exemplary cloud computing environment 3000 is shown and described. The cloud computing environment 3000 can include one or more resource providers 3002a, 3002b, 3002c (collectively 3002). Each resource provider 3002 may include computing resources. In some embodiments, computing resources may include hardware and/or software used to process data. For example, computing resources may include hardware and/or software capable of executing algorithms, computer programs and/or computer applications. In some embodiments, exemplary computing resources may include application servers and/or databases with storage and retrieval capabilities. Each resource provider 3002 can be coupled to any other resource provider 3002 in the cloud computing environment 3000 . In some embodiments, resource providers 3002 may be connected through a computer network 3008. Each resource provider 3002 can be coupled to one or more computing devices 3004a, 3004b, 3004c (collectively, 3004) via a computer network 3008.

[0388] The cloud computing environment 3000 may include a resource manager 3006. Resource manager 3006 can be coupled to resource providers 3002 and computing devices 3004 via computer network 3008 . In some embodiments, resource manager 3006 may facilitate the provision of computing resources to one or more computing devices 3004 by one or more resource providers 3002 . Resource manager 3006 can receive a request for a computing resource from a particular computing device 3004 . Resource manager 3006 can identify one or more resource providers 3002 that can provide the computing resource requested by computing device 3004 . Resource manager 3006 can select resource provider 3002 to provide computing resources. A resource manager 3006 can facilitate a connection between a resource provider 3002 and a particular computing device 3004 . In some embodiments, resource manager 3006 may establish a connection between a particular resource provider 3002 and a particular computing device 3004 . In some embodiments, resource manager 3006 may redirect a particular computing device 3004 with a requested computing resource to a particular resource provider 3002 .

[0389] FIG. 31 shows examples of a computing device 3100 and a mobile computing device 3150 that can be used to implement the techniques described in this disclosure. Computing device 3100 includes laptops, desktops, workstations, personal digital assistants (PDAs), servers, blade servers, mainframes and It is intended to represent various types of digital computers, such as other suitable computers. Mobile computing device 3150 is intended to represent various forms of mobile devices such as personal digital assistants, cellular phones, smart-phones, and other similar computing devices. Components shown herein, their connections and relationships, and functions are examples only and are not meant to be limiting.

[0390] The computing device 3100 includes a processor 3102, a memory 3104, a storage device 3106, a high-speed interface 3108 coupled to the memory 3104 and a number of high-speed expansion ports. 3110 and a low-speed expansion port 3114 and a low-speed interface 3112 coupled to the storage device 3106. Processor 3102, memory 3104, storage device 3106, high-speed interface 3108, high-speed expansion port 3110, and low-speed interface 3112 are each interconnected using various buses, and a typical motherboard It can be mounted on a motherboard or in other ways suitable. Processor 3102 processes commands for execution within computing device 3100, including those stored in memory 3104 or storage device 3106, to provide graphical information for a GUI to a display coupled to high-speed interface 3108. It can be displayed on an external input/output device such as (display) 3116. In another embodiment, multiple processors and/or multiple buses may be used with multiple memories and types of memory as appropriate. Also, several computing devices can be connected, each providing some of the necessary tasks (eg, as a server bank, group of blade servers, or multiprocessor system). Thus, as a term is used in this application in which a plurality of functions are described as being performed by a “processor,” this means that the plurality of functions are performed by any number (one or more) of the processors of any number (one or more) of computing devices. It includes implementations performed by Also, where functionality is described as being performed by a “processor,” this includes implementations in which the functionality is performed by any number (one or more) processors of any number (one or more) computing devices ( For example, distributed computing systems).

[0391] The memory 3104 stores information within the computing device 3100. In some embodiments, memory 3104 is a volatile memory unit or units. In some embodiments, memory 3104 is a non-volatile memory unit or units. Memory 3104 can also be other forms of computer readable media, such as magnetic or optical disks.

[0392] The storage device 3106 can provide mass storage for the computing device 3100. In some embodiments, the storage device 3106 is a floppy disk device, hard disk device, optical disk device or tape device, flash memory, or other similar solid state device. It may be or include a computer readable medium, such as a solid state memory device, or may include an array of devices including a storage area network or other constructs. Instructions may be stored in an information carrier. Instructions when executed by one or more processing devices (eg, processor 3102) perform one or more methods as described above. Instructions may also be stored by one or more storage devices, such as computer readable or machine readable media (eg, memory 3104 , storage device 3106 , or memory on processor 3102 ).

[0393] High speed interface 3108 manages bandwidth intensive operations for computing device 3100, while low speed interface 3112 manages low bandwidth intensive operations. The assignment of these functions is exemplary only. In some embodiments, high-speed interface 3108 includes memory 3104, display 3116 (eg, via a graphics processor or accelerator), and high-speed expansion ports that can accommodate various expansion cards (not shown). (3110). In an implementation, low-speed interface 3112 is coupled to storage device 3106 and low-speed expansion port 3114 . Low-speed expansion port 3114, which may include various communication ports (e.g., USB, Bluetooth®, Ethernet, wireless Ethernet) is used for keyboard, pointing device, scanner It may be connected to one or more input/output devices such as a scanner, or to a networking device such as a switch or router (eg, via a network adapter).

[0394] The computing device 3100 may be implemented in various forms as shown in the figure. For example, it may be implemented as a standard server 3120 or multiple times in a group of such servers. It may also be implemented in a personal computer such as laptop computer 3122. It may also be implemented as part of a rack server system 3124. Alternatively, components from computing device 3100 may be combined with other components of a mobile device (not shown), such as mobile computing device 3150 . Each of these devices may include one or more of computing device 3100 and mobile computing device 3150, and the overall system may be comprised of multiple computing devices that communicate with each other.

[0395] The mobile computing device 3150 includes, among other components, a processor 3152, a memory 3164, an input/output device such as a display 3154, a communication interface 3166, and a transceiver 3168. includes The mobile computing device 3150 may be provided with a storage device such as a micro-drive or other device to provide additional storage. Each of the processor 3152, memory 3164, display 3154, communication interface 3166, and transceiver 3168 are interconnected using various buses, and the various components may be mounted on a common motherboard or in other ways as appropriate. can

[0396] Processor 3152 may execute instructions within mobile computing device 3150, including instructions stored in memory 3164. The processor 3152 may be implemented as a chipset of chips including individual and multiple analog and digital processors. Processor 3152 is responsible for managing other components of mobile computing device 3150, such as, for example, control of user interfaces, applications executed by mobile computing device 3150, and wireless communications by mobile computing device 3150. adjustments can be provided.

[0397] The processor 3152 may communicate with a user via a control interface 3158 and a display interface 3156 coupled to the display 3154. Display 3154 may be, for example, a TFT (Thin Film Transistor Liquid Crystal Display) display or an OLED (Organic Light Emitting Diode) display or other suitable display technology. Display interface 3156 may include suitable circuitry for driving display 3154 to provide graphics and other information to a user. Control interface 3158 can receive commands from a user and translate them for submission to processor 3152 . Additionally, external interface 3162 can provide communication with processor 3152 to enable short-range communication between mobile computing device 3150 and other devices. External interface 3162 may provide, for example, for wired communication in some implementations, or for wireless communication in other implementations, and multiple interfaces may also be used.

[0398] Memory 3164 stores information within mobile computing device 3150. The memory 3164 may be implemented as one or more of a computer-readable medium or media, a volatile memory unit or units, or a non-volatile memory unit or units. Expansion memory 3174 may also be provided and coupled to mobile computing device 3150 via expansion interface 3172 , which may include, for example, a single in line memory module (SIMM) card interface. Expansion memory 3174 may provide additional storage space for mobile computing device 3150 or store applications or other information for mobile computing device 3150 . Specifically, the expansion memory 3174 may include instructions for performing or supplementing the above process, and may also include security information. Thus, for example, expansion memory 3174 can be provided as a security module for mobile computing device 3150 and can be programmed with instructions allowing secure use of mobile computing device 3150 . Security applications may also be provided with additional information, such as placing identification information on the SIMM card in an unhackable manner via the SIMM card.

[0399] The memory may include, for example, flash memory and/or NVRAM memory (non-volatile random access memory) as described below. In some embodiments, instructions are stored in an information carrier. The instructions when executed by one or more processing devices (eg, processor 3152) perform one or more methods as described above. Instructions may also be stored by one or more storage devices, such as one or more computer-readable or machine-readable media (e.g., memory 3164, expansion memory 3174, or memory on processor 3152). . In some embodiments, the instructions may be received in a signal propagated through transceiver 3168 or external interface 3162, for example.

[0400] Mobile computing device 3150 may communicate wirelessly via communication interface 3166, which may include digital signal processing circuitry if desired. The communication interface 3166 is a GSM voice call (global system for mobile communication), SMS (Short Message Service), EMS (Enhanced Messaging Service) or MMS messaging (Multimedia Messaging Service), CDMA (code division multiple access), TDMA (time division multiple access), PDC (personal digital cellular), WCDMA (wideband code division multiple access) , CDMA2000 or General Packet Radio Service (GPRS), and the like, may provide communication under various modes or protocols. Such communication may occur via transceiver 3168 using radio frequency, for example. Short-range communication may also occur, such as using Bluetooth®, Wi-Fi™ or other transceivers (not shown). Additionally, a Global Positioning System (GPS) receiver module 3170 may be used by applications running on the mobile computing device 3150 to provide additional navigation and location-related wireless data to the mobile computing device 3150 as appropriate. can be provided to

[0401] The mobile computing device 3150 may also communicate audibly using an audio codec 3160, which may receive voice information from a user and convert it into usable digital information. Audio codec 3160 may likewise produce audible sound for a user through a speaker, such as, for example, in a handset of mobile computing device 3150 . Such sounds may include sounds from voice phone calls, may include recorded sounds (eg, voice messages, music files, etc.), and may also include applications running on mobile computing device 3150. may include sounds generated by

[0402] The mobile computing device 3150 may be implemented in various forms as shown in the figure. For example, it may be implemented in cellular telephone 3180. It may also be implemented as part of a smartphone 3182, PDA, or other similar mobile device.

[0403] The various implementations of the systems and techniques described in this application may be realized in digital electronic circuitry, integrated circuitry, specially designed application specific integrated circuits (ASICs), computer hardware, firmware, software, and/or combinations thereof. It can be. These various implementations include programming that includes instructions for at least one programmable processor and storage system, at least one input device and at least one output device, which may be special or general purpose coupled to receive data and instructions and transmit data. may include implementation in one or more computer programs executable and/or interpretable on a capable system.

[0404] Such computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and may be implemented in a high-level procedural and/or object-oriented programming language and/or assembly/machine language. there is. As used herein, the terms machine readable medium and computer readable medium are used to provide machine instructions and/or data for a programmable processor including a machine readable medium that receives machine instructions as a machine readable signal. means any computer program product, apparatus and/or device (eg, magnetic disk, optical disk, memory, programmable logic devices (PLDs)) that is used. The term machine-readable signal refers to any signal used to provide machine instructions and/or data to a programmable processor.

[0405] To provide interaction with a user, the systems and techniques described in this application include a display device (eg, a cathode ray tube (CRT) or liquid crystal display (LCD) monitor) for displaying information to a user and It can be implemented on a computer having a keyboard and pointing device (eg, mouse or trackball) through which a user can provide input to the computer. Other types of devices may also be used to provide interaction with the user, for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback). can; Input from the user may be received in any form including acoustic, voice, or tactile input.

[0406] The systems and techniques described in this application may include a back end component (eg, as a data server) or include a middleware component (eg, an application server), or may include a front end component (eg, as a data server). A front end component (eg, a client computer with a Web browser or graphical user interface through which a user can interact with an implementation of the systems and techniques described herein), or It may be implemented in a computing system that includes a combination of these back-end, middleware, or front-end components. Components of the system may be interconnected by any form or medium of digital data communication (eg, a communication network). Examples of communication networks include LAN (Local Area Network), WAN (Wide Area Network) and Internet.

[0407] A computing system may include a client and a server. Clients and servers are usually remote from each other and usually interact through a communication network. The relationship of client and server arises because of computer programs running on the respective computers and having a client-server relationship to each other.

[0408] In some embodiments, the various modules described in this application may be separated, combined or integrated into single or combined modules. The modules depicted in the figures are not intended to limit the systems described herein to the software architectures shown herein.

[0409] Elements of different embodiments described in this application may be combined to form other embodiments not specifically described above. Elements may be excluded from the processes, computer programs, databases, etc. described in this application without adversely affecting operation. Further, the logic flows depicted in the figures do not require any specific order presented or sequential order to obtain desired results. Various individual elements may be combined into one or more individual elements to perform the functions described herein.

[0410] Throughout the specification, devices and systems are described as having, including, or comprising particular components, processes and methods having, including, or comprising particular steps. , where described as comprising, additionally there are devices and systems of the present invention consisting essentially of or consisting of the recited elements and processes according to the invention consisting essentially of or consisting of the recited processing steps. and methods exist.

[0411] The various described embodiments of the present invention may be used with one or more other embodiments unless technically incompatible. It should be understood that the order of steps or order of performing particular actions is immaterial so long as the invention remains operable. Moreover, two or more steps or actions may be performed concurrently.

[0412] Although this invention has been specifically shown and described with reference to specific preferred embodiments, various changes may be made in form and detail without departing from the spirit and scope of the invention as defined in the appended claims. It should be understood by those skilled in the art that there is.

Claims (148)

A method for automatically processing 3D images of an object to identify and/or characterize cancerous lesions in the object, comprising:
(a) receiving, by a processor of a computing device, a 3D functional image of an object obtained using a functional imaging technique;
(b) automatically detecting, by the processor, one or more hotspots in the 3D functional image using a machine learning module, each hotspot corresponding to a local region of high intensity relative to the surroundings and generating a potential within the object; Indicates a cancerous lesion that is, (i) for each hotspot, a hotspot list identifying the location of the hotspot and (ii) for each hotspot, identifying a corresponding 3D hotspot volume in the 3D functional image. generating one or both of the 3D hotspot maps; and
(c) storing and/or providing the hotspot list and/or the 3D hotspot map for display and/or further processing. According to claim 1,
The machine learning module receives at least a portion of the 3D functional image as an input and automatically detects one or more hotspots based at least in part on intensities of voxels of the received portion of the 3D functional image. A method for automatically processing 3D images of According to claim 1 or 2,
The machine learning module receives as input a 3D segmentation map identifying one or more volumes of interest (VOIs) in the 3D functional image, each VOI corresponding to a specific target tissue region and/or a specific anatomical region within the object. , A method for automatically processing 3D images of an object. According to any one of claims 1 to 3,
receiving, by the processor, a 3D anatomical image of the object obtained using an anatomical imaging technique, wherein the 3D anatomical image includes a graphical representation of a tissue in the object;
The machine learning module receives at least two input channels, the input channels comprising a first input channel corresponding to at least a portion of the 3D anatomical image and a second input channel corresponding to at least a portion of the 3D functional image A method for automatically processing 3D images of an object. According to claim 4,
The machine learning module receives as input a 3D segmentation map identifying one or more volumes of interest (VOIs) within the 3D functional image and/or the 3D anatomical image, each VOI having a specific target tissue region and/or specific anatomy. A method for automatically processing 3D images of an object corresponding to an enemy region. According to claim 5,
and generating the 3D segmentation map by automatically segmenting the 3D anatomical image by the processor. According to any one of claims 1 to 6,
The machine learning module is a specific region machine learning module that receives a specific part of the 3D functional image corresponding to one or more specific tissue regions and/or anatomical regions of the object as an input, and automatically converts 3D images of the object. way to deal with it. According to any one of claims 1 to 7,
The method of claim 1 , wherein the machine learning module generates the hotspot list as an output. According to any one of claims 1 to 8,
The method of claim 1 , wherein the machine learning module generates the 3D hotspot map as an output. According to any one of claims 1 to 9,
(d) determining, by the processor, for each hotspot of at least some of the hotspots, a lesion likelihood classification corresponding to a likelihood of the hotspot representing a lesion in the object; How to automatically process them. According to claim 10,
Step (d) comprises determining the lesion likelihood classification for each hotspot of the portion using the machine learning module. According to claim 10,
Step (d) comprises determining the lesion likelihood classification for each hotspot using a second machine learning module. According to claim 12,
Automatically generating, by the processor, 3D images of an object, comprising determining a set of one or more hotspot features for each hotspot and using the set of one or more hotspot features as an input to the second machine learning module. way to deal with it. According to any one of claims 10 to 13,
(e) selecting, by the processor, a subset of one or more hotspots corresponding to hotspots with a high likelihood of corresponding to cancerous lesions based at least in part on the lesion likelihood classification for the hotspots. A method for automatically processing 3D images of an object, comprising: According to any one of claims 1 to 14,
(f) correcting, by the processor, intensity bleed from one or more high intensity volumes of the 3D functional image by adjusting intensities of voxels of the 3D functional image, wherein the one or more high intensity volumes A method for automatically processing 3D images of a subject, each corresponding to a tissue region of high absorption in the subject associated with high absorption of radiopharmaceuticals under normal circumstances. According to claim 15,
The method for automatically processing 3D images of an object, wherein step (f) includes correcting intensity bleed from a plurality of high-intensity volumes one at a time in a sequential manner. According to claim 15 or 16,
The method of claim 1 , wherein the one or more high-intensity volumes correspond to one or more highly absorptive tissue regions selected from the group consisting of kidney, liver, and bladder. According to any one of claims 1 to 17,
(g) determining, by the processor, for each of the at least some of the one or more hotspots, a corresponding lesion index indicative of the size and/or level of radiopharmaceutical absorption within the underlying lesion to which the hotspot corresponds. A method for automatically processing 3D images of an object. According to claim 18,
Step (g) comprises comparing the intensity (intensities) of one or more voxels associated with the hotspot to one or more reference values, each reference value being the reference tissue region of a reference volume corresponding to a specific reference tissue region. A method for automatically processing 3D images of an object, related to. According to claim 19,
The one or more reference values include one or more members selected from the group consisting of an aortic reference value related to an aortic portion of the object and a liver reference value related to the liver of the object. Method for automatically processing 3D images of an object. According to claim 19 or 20,
For at least one specific reference value associated with a specific reference tissue region, determining the specific reference value comprises fitting intensities of voxels in a specific reference volume corresponding to the specific reference tissue region to a multi-component mixture model. , A method for automatically processing 3D images of an object. According to any one of claims 18 to 21,
A method for automatically processing 3D images of an object, comprising calculating an overall risk index for the object representing a cancer state and/or risk for the object using the determined lesion index values. 23. The method of any one of claims 1 to 22,
determining, by the processor, for each hotspot, an anatomical classification corresponding to a specific anatomical region and/or group of anatomical regions within the object for which the potential cancerous lesion represented by the hotspot is determined to be located. A method for automatically processing 3D images of an object, comprising: 24. The method of any one of claims 1 to 23,
(h) causing, by the processor, rendering of a graphical representation of at least a portion of the one or more hotspots for review by a user, for display in a graphical user interface (GUI). A method for automatically processing 3D images. According to claim 24,
(i) receiving, via the GUI, a user selection, by the processor, of a subset of the one or more hotspots identified through user review as likely indicative of underlying cancerous lesions in the subject; A method for automatically processing 3D images. 26. The method of any one of claims 1 to 25,
The method for automatically processing 3D images of an object, wherein the 3D functional image includes a PET or SPECT image obtained after administration of an agent to the object. 27. The method of claim 26,
A method for automatically processing 3D images of an object, wherein the agent comprises a PSMA binder. According to claim 26 or 27,
A method for automatically processing 3D images of an object, wherein the agent comprises ¹⁸ F. According to claim 27 or 28,
The method of claim 1 , wherein the agent comprises [ 18 F]DCFPyL. According to claim 27 or 28,
A method for automatically processing 3D images of an object, wherein the agent comprises PSMA-11. According to claim 26 or 27,
The method of automatically processing 3D images of an object, wherein the agent comprises one or more members selected from the group consisting of ^99m Tc, ⁶⁸ Ga, ¹⁷⁷ Lu, ²²⁵ Ac, ¹¹¹ In, ¹²³ I, ¹²⁴ I and ¹³¹ I. 32. The method of any one of claims 1 to 31,
The method for automatically processing 3D images of an object, wherein the machine learning module implements a neural network. 33. The method of any one of claims 1 to 32,
The method for automatically processing 3D images of an object, wherein the processor is a processor of a cloud-based system. A method for automatically processing 3D images of an object to identify and/or characterize cancerous lesions in the object, comprising:
(a) receiving, by a processor of a computing device, a 3D functional image of the object obtained using a functional imaging technique;
(b) receiving, by the processor, a 3D anatomical image of the object obtained using an anatomical imaging technique, wherein the 3D anatomical image includes a graphic representation of a tissue in the object;
(c) by the processor, automatically detecting one or more hotspots in the 3D functional image using a machine learning module, each hotspot corresponding to a local area of high intensity relative to the surroundings and potentially a cancerous lesion in the subject represents one or both of (i) for each hotspot a hotspot list identifying the hotspot's location and (ii) for each hotspot a 3D hotspot map identifying a corresponding 3D hotspot volume within the 3D functional image steps to create all
- the machine learning module receives at least two input channels, a first input channel corresponding to at least a part of the 3D anatomical image and a second input channel corresponding to at least a part of the 3D functional image and and/or contains anatomical information derived therefrom; and
(d) storing and/or providing the hotspot list and/or the 3D hotspot map for display and/or further processing. A method for automatically processing 3D images of an object to identify and/or characterize cancerous lesions in the object, comprising:
(a) receiving, by a processor of a computing device, a 3D functional image of the object obtained using a functional imaging technique;
(b) by the processor, automatically detecting one or more hotspots in the 3D functional image using a machine learning module, each hotspot corresponding to a local area of high intensity relative to the surroundings and potentially a cancerous lesion in the subject Indicates -, for each hotspot, generating a hotspot list identifying the location of the hotspot;
(c) automatically determining, by the processor, a corresponding 3D hotspot volume in the 3D functional image for each of the one or more hotspots using a second machine learning module and the hotspot list to generate a 3D hotspot map; ; and
(d) storing and/or providing the hotspot list and/or the 3D hotspot map for display and/or further processing. 36. The method of claim 35,
(e) determining, by the processor, for each hotspot of at least some of the hotspots, a lesion likelihood classification corresponding to a likelihood of the hotspot representing a lesion in the object; way to deal with it. 37. The method of claim 36,
The method for automatically processing 3D images of an object, wherein step (e) includes determining the lesion likelihood classification for each hotspot using a third machine learning module. The method of any one of claims 35 to 37,
(f) selecting, by the processor, a subset of the one or more hotspots corresponding to hotspots with a high likelihood of corresponding to cancerous lesions based at least in part on the lesion likelihood classification for the hotspots. , A method for automatically processing 3D images of an object. A method of measuring intensity values in a reference volume corresponding to a reference tissue region to prevent effects from tissue regions associated with a low absorption radiopharmaceutical, comprising:
(a) receiving, by a processor of a computing device, a 3D functional image of an object, wherein the 3D functional image is obtained using a functional imaging technique;
(b) identifying, by the processor, a reference volume within the 3D functional image;
(c) fitting, by the processor, a multi-component mixture model to intensities of voxels in a reference volume;
(d) identifying, by the processor, a dominant mode of the multi-component model;
(e) determining, by the processor, dimensions of intensities corresponding to the principal mode to determine reference intensity values corresponding to dimensions of intensities of voxels (i) within the reference tissue volume and (ii) associated with the principal mode; doing;
(f) detecting, by the processor, one or more hotspots corresponding to potentially cancerous lesions within the functional image; and
(g) determining, by the processor, for each hotspot of the at least some of the detected hotspots, a lesion index value using at least the reference intensity value in a reference volume corresponding to a reference tissue region. How to measure intensity values. A method of correcting intensity bleed due to areas of high absorption tissue in a subject associated with high absorption of radiopharmaceuticals under normal circumstances, comprising:
(a) receiving, by a processor of a computing device, a 3D functional image of the object, wherein the 3D functional image is obtained using a functional imaging technique;
(b) identifying, by the processor, a high-intensity volume in the 3D functional image, the high-intensity volume corresponding to a specific high-absorption tissue region where high radiopharmaceutical uptake occurs under normal circumstances;
(c) identifying, by the processor, a suppression volume in the 3D functional image based on the identified high-intensity volume, wherein the suppression volume is a volume within and outside a predetermined attenuation distance from a boundary of the identified high-intensity volume. Corresponds to ― ;
(d) determining, by the processor, a background image corresponding to the 3D functional image with intensities of voxels in the high-intensity volume replaced with interpolated values determined based on intensities of voxels of the 3D functional image in the containment volume; doing;
(e) determining, by the processor, an estimated image obtained by subtracting intensities of voxels of the background image from intensities of voxels of the 3D functional image;
(f) by the processor;
extrapolate intensities of voxels of the estimated image corresponding to the high intensity volume to locations of voxels within the suppression volume to determine intensities of voxels of a suppression map corresponding to the suppression volume;
determining a suppression map by setting intensities of voxels of the suppression map corresponding to locations outside the suppression volume to zero; and
(g) correcting, by the processor, intensity bleed from the high intensity volume by adjusting intensities of voxels of the 3D functional image based on the suppression map. 41. The method of claim 40,
A method of correcting intensity bleed comprising performing steps (b) to (g) on each of a plurality of high intensity volumes in a sequential manner to correct intensity bleed from each of the plurality of high intensity volumes. 41. The method of claim 41,
wherein the plurality of high intensity volumes comprises one or more members selected from the group consisting of kidney, liver and bladder. A method for automatically processing 3D images of an object to identify and/or characterize cancerous lesions in the object, comprising:
(a) receiving, by a processor of a computing device, a 3D functional image of the object obtained using a functional imaging technique;
(b) automatically detecting, by the processor, one or more hotspots in the 3D functional image, each hotspot corresponding to a local area of high intensity compared to the surroundings and representing a potential cancerous lesion in the object;
(c) causing, by the processor, rendering of a graphical representation of the one or more hotspots for display within an interactive graphical user interface (GUI);
(d) receiving, by the processor, a user selection of a final hotspot set comprising at least a portion of the one or more automatically detected hotspots via the interactive GUI; and
(e) storing and/or providing the final set of hotspots for display and/or further processing. 44. The method of claim 43,
(f) receiving, by the processor, a user selection through the GUI of one or more additional user-identified hotspots for inclusion in the final set of hotspots; and
(g) updating, by the processor, the final hotspot set to include the one or more additional user-identified hotspots. The method of claim 43 or 44,
A method for automatically processing 3D images of an object, wherein step (b) comprises using one or more machine learning modules. A method for automatically processing 3D images of an object to identify and characterize cancerous lesions in the object, comprising:
(a) receiving, by a processor of a computing device, a 3D functional image of the object obtained using a functional imaging technique;
(b) automatically detecting, by the processor, one or more hotspots in the 3D functional image, each hotspot corresponding to a local area of high intensity compared to the surroundings and representing a potential cancerous lesion in the subject;
(c) corresponding to, by the processor, for each of at least some of the one or more hotspots, a specific anatomical region and/or group of anatomical regions within the object that is determined to be located in which a potential cancerous lesion indicative of the hotspot is to be located; automatically determining an anatomical classification for and
(d) storing and/or providing, for each hotspot, an identification of the one or more hotspots along with an anatomical classification corresponding to the hotspot, for display and/or further processing. How to do it automatically. 47. The method of claim 46,
A method for automatically processing 3D images of an object, wherein step (b) comprises using one or more machine learning modules. A system for automatically processing 3D images of an object to identify and/or characterize cancerous lesions in the object, comprising:
a processor of the computing device; and
a memory having instructions stored therein, which instructions, when executed by the processor, cause the processor to:
(a) receiving a 3D functional image of the object obtained using a functional imaging technique;
(b) automatically detecting one or more hotspots within the 3D functional image using a machine learning module to obtain (i) for each hotspot a hotspot list identifying the hotspot's location and (ii) for each hotspot the hotspot list generate one or both of 3D hotspot maps that identify corresponding 3D hotspot volumes in the 3D functional image, each hotspot corresponding to a local area of high intensity relative to the surroundings and representing a potential cancerous lesion in the subject; ;
(c) A system for automatically processing 3D images of an object, allowing storage and/or presentation of the hotspot list and/or the 3D hotspot map for display and/or further processing. 49. The method of claim 48,
wherein the machine learning module receives at least a portion of the 3D functional image as an input and automatically detects the one or more hotspots based at least in part on intensities of voxels of the received portion of the 3D functional image. A system for automatically processing images. The method of claim 48 or 49,
The machine learning module receives as input a 3D segmentation map identifying one or more volumes of interest (VOIs) in the 3D functional image, each VOI corresponding to a specific target tissue region and/or a specific anatomical region within the object. A system for automatically processing 3D images of an object. The method of any one of claims 48 to 50,
The instructions cause the processor to:
receive a 3D anatomical image of the object obtained using an anatomical imaging technique, the 3D anatomical image including a graphic representation of a tissue in the object;
The machine learning module receives at least two input channels, the input channels comprising a first input channel corresponding to at least a portion of the 3D anatomical image and a second input channel corresponding to at least a portion of the 3D functional image A system for automatically processing 3D images of an object. 51. The method of claim 51,
The machine learning module receives as input a 3D segmentation map identifying one or more volumes of interest (VOIs) within the 3D functional image and/or the 3D anatomical image, each VOI having a specific target tissue region and/or specific A system for automatically processing 3D images of an object corresponding to an anatomical region. 52. The method of claim 52,
wherein the instructions cause the processor to automatically segment the 3D anatomical image to generate the 3D segmentation map. The method of any one of claims 48 to 53,
The machine learning module is a specific region machine learning module that receives a specific part of the 3D functional image corresponding to one or more specific tissue regions and/or anatomical regions of the object as an input, and automatically converts 3D images of the object. system for processing. The method of any one of claims 48 to 54,
The system for automatically processing 3D images of an object, wherein the machine learning module generates the hotspot list as an output. 56. The method of any one of claims 48 to 55,
The system for automatically processing 3D images of an object, wherein the machine learning module generates the 3D hotspot map as an output. The method of any one of claims 48 to 56,
The instructions cause the processor to:
(d) for each hotspot of at least some of the hotspots, determine a lesion likelihood classification corresponding to a likelihood of the hotspot representing a lesion in the subject. 58. The method of claim 57,
In step (d), the instructions cause the processor to determine a lesion likelihood classification for each hotspot in the portion using the machine learning module. 58. The method of claim 57,
In step (d), the instructions cause the processor to determine a lesion likelihood classification for each hotspot using a second machine learning module. The method of claim 59,
The instructions cause the processor to automatically generate 3D images of an object, which causes the processor to determine a set of one or more hotspot features for each hotspot and use the set of one or more hotspot features as input to the second machine learning module. system for processing. The method of any one of claims 57 to 60,
The instructions cause the processor to:
(e) automatically processing 3D images of a subject to select a subset of the one or more hotspots corresponding to hotspots with a high likelihood of corresponding to cancerous lesions based at least in part on a lesion likelihood classification for the hotspots. system to do. The method of any one of claims 48 to 61,
The instructions cause the processor to:
(f) adjusting, by the processor, intensities of voxels of the 3D functional image to correct intensity bleed from one or more high-intensity volumes of the 3D functional image, each of which is highly absorptive under normal circumstances; A system for automatically processing 3D images of an object corresponding to areas of high absorption tissue in the object related to radiopharmaceuticals. 63. The method of claim 62,
In step (f), the instructions cause the processor to correct intensity bleed from a plurality of high intensity volumes one at a time in a sequential manner. The method of claim 62 or 63,
The system for automatically processing 3D images of an object, wherein the one or more high-intensity volumes correspond to one or more highly absorptive tissue regions selected from the group consisting of kidney, liver, and bladder. The method of any one of claims 48 to 64,
The instructions cause the processor to:
(g) for each of the at least some of the one or more hotspots, automatically generating 3D images of the subject to determine a corresponding lesion index indicative of the size and/or level of radiopharmaceutical uptake therein of the underlying lesion to which the hotspot corresponds. system for processing. 66. The method of claim 65,
In step (g), the instructions cause the processor to compare intensities (intensities) of one or more voxels associated with the hotspot with one or more reference values, each reference value being associated with a specific reference tissue region within the object. A system for automatically processing 3D images of an object, determined based on intensities of a reference volume associated with and corresponding to the reference tissue region. 67. The method of claim 66,
The system for automatically processing 3D images of an object, wherein the one or more reference values include one or more members selected from the group consisting of an aorta reference value related to an aortic portion of the object and a liver reference value related to a liver of the object. The method of claim 66 or 67,
For at least one specific reference value associated with a specific reference tissue region, the instructions cause the processor to fit the intensities of voxels within a specific reference volume corresponding to the specific reference tissue region to a multi-component mixture model to determine the specific reference value. A system for automatically processing 3D images of an object, allowing a decision to be made. 69. The method of any one of claims 65 to 68,
The instructions are configured to automatically process 3D images of an object, causing the processor to calculate an overall risk index for the object representing a cancer state and/or risk for the object using the determined lesion index values. system. The method of any one of claims 48 to 69,
The instructions cause the processor to determine, for each hotspot, an anatomical classification that corresponds to a particular anatomical region and/or group of anatomical regions within the subject for which the potential cancerous lesion represented by the hotspot is determined to be located. A system for automatically processing 3D images of an object. The method of any one of claims 48 to 70,
The instructions cause the processor to:
(h) A system for automatically processing 3D images of an object, causing rendering of a graphical representation of at least a portion of said one or more hotspots for review by a user, for display in a graphical user interface (GUI). 71. The method of claim 71,
The instructions cause the processor to:
(i) for automatically processing 3D images of a subject to receive through the GUI a user selection of a subset of the one or more hotspots identified through user review as likely representing underlying cancerous lesions in the subject; system. The method of any one of claims 48 to 72,
The system for automatically processing 3D images of a subject, wherein the 3D functional image includes a PET or SPECT image obtained after administration of an agent to the subject. 74. The method of claim 73,
The system for automatically processing 3D images of an object, wherein the agent comprises a PSMA binder. The method of claim 73 or 74,
The system for automatically processing 3D images of an object, wherein the agent comprises ¹⁸ F. 75. The method of claim 74,
The system for automatically processing 3D images of an object, wherein the agent comprises [ 18 F]DCFPyL. The method of claim 74 or 75,
The system for automatically processing 3D images of an object, wherein the agent comprises PSMA-11. The method of claim 73 or 74,
The system for automatically processing 3D images of an object, wherein the agent comprises one or more members selected from the group consisting of ^99m Tc, ⁶⁸ Ga, ¹⁷⁷ Lu, ²²⁵ Ac, ¹¹¹ In, ¹²³ I, ¹²⁴ I and ¹³¹ I. 79. The method of any one of claims 48 to 78,
The system for automatically processing 3D images of an object, wherein the machine learning module implements a neural network. The method of any one of claims 48 to 79,
The system for automatically processing 3D images of an object, wherein the processor is a processor of a cloud-based system. A system for automatically processing 3D images of an object to identify and/or characterize cancerous lesions in the object, comprising:
a processor of the computing device; and
a memory having instructions stored therein, which instructions, when executed by the processor, cause the processor to:
(a) receiving a 3D functional image of the object obtained using a functional imaging technique;
(b) receive a 3D anatomical image of the object obtained using an anatomical imaging technique, wherein the 3D anatomical image includes a graphic representation of a tissue in the object;
(c) automatically detecting one or more hotspots within the 3D functional image using a machine learning module to obtain (i) for each hotspot a hotspot list identifying the location of the hotspot and (ii) for each hotspot the generate one or both of 3D hotspot maps that identify corresponding 3D hotspot volumes within the 3D functional image, each hotspot corresponding to a local area of high intensity relative to the surroundings and representing a potential cancerous lesion within the subject;
The machine learning module receives at least two input channels, the input channels comprising a first input channel corresponding to at least a portion of the 3D anatomical image and a second input channel corresponding to at least a portion of the 3D functional image and/or or contains anatomical information derived therefrom;
(d) A system for automatically processing 3D images of an object, allowing storage and/or presentation of the hotspot list and/or the 3D hotspot map for display and/or further processing. A system for automatically processing 3D images of an object to identify and/or characterize cancerous lesions in the object, comprising:
a processor of the computing device; and
a memory having instructions stored therein, which instructions, when executed by the processor, cause the processor to:
(a) receiving a 3D functional image of the object obtained using a functional imaging technique;
(b) automatically detect one or more hotspots within the 3D functional image using a first machine learning module to generate for each hotspot a hotspot list identifying the hotspot's location - each hotspot relative to its surroundings; Corresponding to focal areas of high intensity and indicating potentially cancerous lesions in the subject;
(c) for each of the one or more hotspots, automatically determine a corresponding 3D hotspot volume in the 3D functional image using a second machine learning module and the hotspot list to generate a 3D hotspot map;
(d) A system for automatically processing 3D images of an object, allowing storage and/or presentation of the hotspot list and/or the 3D hotspot map for display and/or further processing. 82. The method of claim 82,
The instructions cause the processor to:
(e) determine for each hotspot of at least some of the hotspots a lesion likelihood classification corresponding to a likelihood of the hotspot representing a lesion in the subject. 83. The method of claim 83,
In step (e), the instructions cause the processor to determine a lesion likelihood classification for each hotspot using a third machine learning module. The method of any one of claims 82 to 84,
The instructions cause the processor to:
(f) automatically processing 3D images of a subject to select a subset of the one or more hotspots corresponding to hotspots with a high likelihood of corresponding to cancerous lesions based at least in part on a lesion likelihood classification for the hotspots. system to do. A system for measuring intensity values in a reference volume corresponding to a reference tissue region to prevent effects from tissue regions associated with low absorption radiopharmaceuticals, comprising:
a processor of the computing device; and
a memory having instructions stored therein, which instructions, when executed by the processor, cause the processor to:
(a) receiving a 3D functional image of an object, wherein the 3D functional image is obtained using a functional imaging technique;
(b) identify a reference volume within the 3D functional image;
(c) fit a multi-component mixture model to the intensities of voxels in the reference volume;
(d) identify a dominant mode of the multi-component model;
(e) determine dimensions of intensities corresponding to the principal mode to determine reference intensity values that (i) are within a reference tissue volume and (ii) correspond to measures of intensities of voxels associated with the principal mode;
(f) detect one or more hotspots corresponding to potentially cancerous lesions within the 3D functional image;
(g) for each hotspot of the at least some of the detected hotspots, determine a lesion index value using at least the reference intensity value. A system for correcting intensity bleed due to highly absorptive tissue regions in a subject associated with high radiopharmaceutical absorption under normal circumstances, comprising:
(a) receiving a 3D functional image of an object, wherein the 3D functional image is obtained using a functional imaging technique;
(b) identifying a high-intensity volume in the 3D functional image, the high-intensity volume corresponding to a specific highly absorptive tissue region where high radiopharmaceutical uptake occurs under normal circumstances;
(c) identifying a suppression volume in the 3D functional image based on the identified high-intensity volume, the suppression volume corresponding to a volume within and outside a predetermined attenuation distance from a boundary of the identified high-intensity volume;
(d) determine a background image corresponding to the 3D functional image with intensities of voxels in the high-intensity volume replaced with interpolated values determined based on intensities of voxels of the 3D functional image in the suppression volume;
(e) determining an estimated image by subtracting intensities of voxels of the background image from intensities of voxels from the 3D functional image;
(f) extrapolating intensities of voxels of the estimated image corresponding to the high intensity volume to locations of voxels within the suppression volume to determine intensities of voxels of a suppression map corresponding to the suppression volume;
determine a suppression map by setting intensities of voxels of the suppression map corresponding to locations outside the suppression volume to zero;
(g) correcting intensity bleed from the high intensity volume by adjusting intensities of voxels of the 3D functional image based on the suppression map. 87. The method of claim 87,
wherein the instructions cause the processor to perform steps (b) to (g) on each of a plurality of high intensity volumes in a sequential manner to correct intensity bleed from each of the plurality of high intensity volumes; system. 88. The method of claim 88,
wherein the plurality of high intensity volumes comprises one or more members selected from the group consisting of kidney, liver and bladder. A system for automatically processing 3D images of an object to identify and/or characterize cancerous lesions in the object, comprising:
a processor of the computing device; and
a memory having instructions stored therein, which instructions, when executed by the processor, cause the processor to:
(a) receiving a 3D functional image of the object obtained using a functional imaging technique;
(b) automatically detect one or more hotspots in the 3D functional image, each hotspot corresponding to a local area of high intensity relative to the surroundings and representing a potential cancerous lesion in the subject;
(c) cause rendering of a graphical representation of the one or more hotspots for display within an interactive graphical user interface (GUI);
(d) receive through the interactive GUI a user selection of a final set of hotspots that includes at least a portion of the one or more automatically detected hotspots;
(e) A system for automatically processing 3D images of an object, causing storage and/or presentation of the final set of hotspots for display and/or further processing. 91. The method of claim 90,
The instructions cause the processor to:
(f) receive via the GUI a user selection of one or more additional user-identified hotspots for inclusion in the final set of hotspots;
(g) update the final hotspot set to include the one or more additional user-identified hotspots. The method of claim 90 or 91,
In step (b), the instructions cause the processor to use one or more machine learning modules. A system for automatically processing 3D images of an object to identify and characterize cancerous lesions in the object, comprising:
a processor of the computing device; and
a memory having instructions stored therein, which instructions, when executed by the processor, cause the processor to:
(a) receiving a 3D functional image of the object obtained using a functional imaging technique;
(b) automatically detect one or more hotspots in the 3D functional image, each hotspot corresponding to a local area of high intensity relative to the surroundings and representing a potential cancerous lesion in the subject;
(c) for each of at least a portion of the one or more hotspots, an anatomical classification corresponding to a particular anatomical region and/or group of anatomical regions within the subject for which the potential cancerous lesion represented by the hotspot is determined to be located. let it decide automatically;
(d) automatically processing 3D images of an object to store and/or provide for each hotspot an identification of the one or more hotspots along with an anatomical classification corresponding to the hotspot, for display and/or further processing; system to do. 94. The method of claim 93,
wherein the instructions cause the processor to perform step (b) using one or more machine learning modules. A method for automatically processing 3D images of an object to identify and/or characterize cancerous lesions in the object, comprising:
(a) receiving, by a processor of a computing device, a 3D functional image of the object obtained using a functional imaging technique;
(b) receiving, by the processor, a 3D anatomical image of the object obtained using an anatomical imaging technique;
(c) receiving, by the processor, a 3D segmentation map identifying one or more specific tissue region(s) or tissue region group(s) within the 3D functional image and/or within the 3D anatomical image;
(d) by the processor, automatically detecting and/or segmenting a set of one or more hotspots in the 3D functional image using one or more machine learning module(s) to: (i) for each hotspot, the hotspot generating one or both of a hotspot list identifying the location of and (ii) a 3D hotspot map identifying, for each hotspot, a corresponding 3D hotspot volume within the 3D functional image - each hotspot has a high relative to its surroundings. Corresponding to local areas of intensity and indicating potential cancerous lesions in the subject,
at least one of the one or more machine learning module(s) receives as input: (i) the 3D functional image, (ii) the 3D anatomical image, and (iii) the 3D segmentation map; and
(e) storing and/or providing the hotspot list and/or the 3D hotspot map for display and/or further processing. 95. The method of claim 95,
receiving, by the processor, an initial 3D segmentation map identifying one or more specific tissue regions within the 3D anatomical image and/or the 3D functional image; and
identifying, by the processor, at least a portion of the one or more particular tissue regions as belonging to a particular one of one or more tissue grouping(s); updating the 3D segmentation map to represent regions; and
and using, by the processor, the updated 3D segmentation map as an input to at least one of the one or more machine learning modules. 96. The method of claim 96,
The method for automatically processing 3D images of an object, wherein the one or more tissue groupings include soft tissue groupings so that specific tissue regions representing soft tissues are identified as belonging to the soft tissue groupings. The method of claim 96 or 97,
The method of claim 1 , wherein the one or more tissue groupings include a bone tissue grouping so that specific tissue regions representing bone tissue are identified as belonging to the bone tissue grouping. The method of any one of claims 96 to 98,
The one or more tissue groupings include a high absorption organ grouping so that one or more organs related to the high absorption radiopharmaceutical are identified as belonging to the high absorption grouping. The method of any one of claims 95 to 99,
For each detected and/or segmented hotspot, determining a classification for the hotspot by the processor. 100. The method of claim 100,
A method for automatically processing 3D images of an object comprising using at least one of one or more machine learning modules to determine a classification for the hotspot for each detected and/or segmented lesion. The method of any one of claims 95 to 101,
The one or more machine learning modules,
(A) a systemic lesion detection module that detects and/or segments hotspots throughout the body; and
(B) A method for automatically processing 3D images of an object comprising a prostate lesion module that detects and/or segments hotspots within the prostate. 102. The method of claim 102,
A method for automatically processing 3D images of an object, comprising generating hotspot lists and/or maps using (A) and (B), respectively, and merging the results. The method of any one of claims 95 to 103,
Step (d) is,
segmenting and classifying a set of one or more hotspots to identify for each hotspot a corresponding 3D hotspot volume in the 3D functional image and labeling each hotspot volume as belonging to a particular one of a plurality of hotspot classes ( label), a labeled 3D hotspot map,
generating a first initial 3D hotspot map identifying a first set of initial hotspot volumes by segmenting a first initial set of one or more hotspots within the 3D functional image using a first machine learning module - the first machine learning module; divides the hotspots of the 3D functional image according to a single hotspot class;
generating a second initial 3D hotspot map identifying a second set of initial hotspot volumes by segmenting a second initial set of one or more hotspots within the 3D functional image using a second machine learning module - the second machine learning module; segment the 3D functional image according to a plurality of different hotspot classes such that the second initial 3D hotspot map is a multi-class 3D hotspot map in which each hotspot volume is labeled as belonging to a particular one of a plurality of different hotspot classes; Ham -, generating; and
generating, by the processor, the first initial 3D hotspot map and the second initial 3D hotspot map for at least a portion of the hotspot volume identified by the first initial 3D hotspot map;
identify a matching hotspot volume in the second initial 3D hotspot map, the matching hotspot volume in the second 3D hotspot map being labeled as belonging to a specific one of the plurality of different hotspot classes;
Segmented hotspot volumes of the first 3D hotspot map labeled according to the classes identified by labeling a particular hotspot volume of the first initial 3D hotspot map as belonging to a particular hotspot class to which matching hotspot volumes of the second 3D hotspot map belong. merging by creating a merged 3D hotspot map comprising the hotspot volumes;
The method for automatically processing 3D images of an object, wherein step (e) includes storing and/or providing the merged 3D hotspot map for display and/or further processing. 104. The method of claim 104,
The plurality of different hotspot classes,
(i) bone hotspots determined to represent lesions located in bone;
(ii) lymphatic hotspots determined to represent lesions located in the lymph nodes, and
(iii) prostate hotspots determined to represent lesions located in the prostate
A method for automatically processing 3D images of an object, including one or more members selected from the group consisting of. 106. The method of any one of claims 95 to 105,
(f) receiving and/or accepting the hotspot list; and
(g) segmenting the hotspot using an analysis model for each hotspot in the hotspot list. 106. The method of any one of claims 95 to 105,
(h) receiving and/or accepting the hotspot map; and
(g) segmenting the hotspot using an analysis model for each hotspot of the hotspot map. 107. The method of claim 107,
The analysis model is an adaptive threshold setting method,
Step (i) is
determining one or more reference values based on dimensions of intensities of voxels of the 3D functional image located in a specific reference volume corresponding to a specific reference tissue region; and
For each specific hotspot volume in the 3D hotspot map,
determining, by the processor, a corresponding hotspot intensity based on intensities of voxels within the particular hotspot volume; and
determining, by the processor, a hotspot specific threshold for a specific hotspot based on at least one of (i) the corresponding hotspot intensity and (ii) the one or more reference value(s). A method for automatically processing images. 108. The method of claim 108,
The hotspot specific threshold is determined using a specific threshold function selected from a plurality of threshold functions, and the specific threshold function is selected based on a comparison of a corresponding hotspot intensity with at least one reference value. A method for automatically processing 3D images. 109. The method of claim 108 or 109,
The method of claim 1 , wherein the hotspot specific threshold is determined as a variable percentage of a corresponding hotspot intensity, and wherein the variable percentage decreases with increasing hotspot intensity. A method for automatically processing 3D images of an object to identify and/or characterize cancerous lesions in the object, comprising:
(a) receiving, by a processor of a computing device, a 3D functional image of the object obtained using a functional imaging technique;
(b) a first initial 3D hotspot map identifying, by the processor, a first set of initial hotspot volumes by automatically segmenting the first initial set of one or more hotspots in the 3D functional image using a first machine learning module; Generating, wherein the first machine learning module segments hotspots of the 3D functional image according to a single hotspot class;
(c) a second initial 3D hotspot map identifying, by the processor, a second set of initial hotspot volumes by automatically segmenting the second initial set of one or more hotspots within the 3D functional image using a second machine learning module; generating: the second machine learning module segments the 3D functional image according to a plurality of different hotspot classes such that the second initial 3D hotspot map is such that each hotspot volume is a specific one of the plurality of different hotspot classes. to be a multi-class 3D hotspot map labeled as belonging to ;
(d) generating, by the processor, the first initial 3D hotspot map and the second initial 3D hotspot map for each particular hotspot volume of at least a portion of the first set of initial hotspot volumes identified by the first initial 3D hotspot map; hotspot map,
identify a matching hotspot volume in the second initial 3D hotspot map, the matching hotspot volume in the second 3D hotspot map being labeled as belonging to a specific one of the plurality of different hotspot classes;
Segmented hotspots of the first 3D hotspot map labeled according to classes identified by labeling a particular hotspot volume of the first initial 3D hotspot map as belonging to a particular hotspot class to which matching hotspots of the second 3D hotspot map belong. merging by creating a merged 3D hotspot map comprising the volumes; and
(e) storing and/or providing the merged 3D hotspot map for display and/or further processing. 111. The method of claim 111,
The plurality of different hotspot classes,
(i) bone hotspots determined to represent lesions located in bone;
(ii) lymphatic hotspots determined to represent lesions located in the lymph nodes, and
(iii) prostate hotspots determined to represent lesions located in the prostate
A method for automatically processing 3D images of an object, including one or more members selected from the group consisting of. A method for automatically processing 3D images of an object through an adaptive threshold setting approach to identify and/or characterize cancerous lesions in the object, comprising:
(a) receiving, by a processor of a computing device, a 3D functional image of the object obtained using a functional imaging technique;
(b) receiving, by the processor, a preliminary 3D hotspot map identifying one or more preliminary hotspot volumes within the 3D functional image;
(c) determining, by the processor, one or more reference values based on dimensions of intensities of voxels of the 3D functional image located in a specific reference volume corresponding to a specific reference tissue region;
(d) by the processor, for each particular preliminary hotspot volume of at least a portion of the one or more preliminary hotspot volumes identified by the preliminary 3D hotspot map and based on the preliminary hotspot volumes;
determine a corresponding hotspot intensity based on intensities of voxels within the specific preliminary hotspot volume;
determine a hotspot specific threshold for the specific preliminary hotspot volume based on at least one of (i) the corresponding hotspot intensity and (ii) the one or more reference value(s);
Refined and analytically segmented corresponding to the specific preliminary hotspot volume by segmenting at least a portion of the 3D functional image using a threshold-based segmentation algorithm that performs image segmentation using the determined hotspot specific threshold for the specific preliminary hotspot volume. determine the hotspot volume;
generating a refined 3D hotspot map using adaptive threshold based segmentation by including the refined hotspot volume within the refined 3D hotspot map; and
(e) storing and/or providing the refined 3D hotspot map for display and/or further processing. 113. The method of claim 113,
The hotspot specific threshold is determined using a specific threshold function selected from a plurality of threshold functions, and the specific threshold function is selected based on a comparison of a corresponding hotspot intensity with at least one reference value. A method for automatically processing 3D images. The method of claim 113 or 114,
The method of claim 1 , wherein the hotspot specific threshold is determined as a variable percentage of a corresponding hotspot intensity, and wherein the variable percentage decreases with increasing hotspot intensity. A method for automatically processing 3D images of an object to identify and/or characterize cancerous lesions in the object, comprising:
(a) receiving, by a processor of a computing device, a 3D anatomical image of the object obtained using an anatomical imaging technique, wherein the 3D anatomical image includes a graphic representation of a tissue in the object;
(b) a plurality of volumes of interest (VOIs) including a liver volume corresponding to the liver of the object and an aorta volume corresponding to the aorta portion in the 3D anatomical image by automatically segmenting the 3D anatomical image by the processor; generating a 3D segmentation map identifying the segments;
(c) receiving, by the processor, a 3D functional image of the object obtained using a functional imaging technique;
(d) automatically segmenting, by the processor, one or more hotspots in the 3D functional image to identify one or more automatically segmented hotspot volumes, each segmented hotspot corresponding to a local region of high intensity relative to its surroundings. and indicates potential cancerous lesions in the subject;
(e) causing, by the processor, rendering of a graphical representation of the one or more automatically segmented hotspot volumes for display within an interactive graphical user interface (GUI);
(f) receiving, by the processor, a user selection of a final hotspot set comprising at least a portion of the one or more automatically segmented hotspot volumes through the interactive GUI;
(g) by the processor, for each hotspot volume of the final set, (i) intensities of voxels of the functional image corresponding to the hotspot volume and (ii) the functional image corresponding to liver volume and aortic volume determining a lesion index value based on one or more reference values determined using intensities of voxels of ; and
(h) storing and/or providing the final hotspot set and/or lesion index values for display and/or further processing. 116. The method of claim 116,
Step (b) comprises segmenting the anatomical image so that the 3D segmentation map identifies one or more bone volumes corresponding to one or more bones of the object;
Step (d) automatically processes 3D images of the object, including identifying a bone volume using the one or more bone volumes in the functional image and segmenting one or more bone hotspot volumes located within the bone volume. way to do it. The method of claim 116 or 117,
Step (b) comprises segmenting the anatomical image so that the 3D segmentation map identifies one or more organ volumes corresponding to soft tissue organs of the object;
Step (d) comprises identifying one or more soft tissue volumes using the one or more segmented organ volumes within the functional image and segmenting one or more lymphatic and/or prostate hotspot volumes located within the soft tissue volume. A method for automatically processing 3D images of an object. 118. The method of claim 118,
Step (d) produces 3D images of the object, further comprising adjusting intensities of the functional image to suppress intensity from one or more highly absorptive tissue regions before segmenting the one or more lymphatic and/or prostate hotspot volumes. How to do it automatically. The method of any one of claims 116 to 119,
The method for automatically processing 3D images of an object, wherein step (g) includes determining a liver reference value using intensities of voxels of the functional image corresponding to the liver volume. 120. The method of claim 120,
Intensity associated with abnormally low absorption regions from liver volume by fitting a two-component Gaussian mixture model to the histogram of intensities of functional imaging voxels corresponding to liver volume using a two-component Gaussian mixture model fit A method for automatically processing 3D images of an object, comprising identifying and excluding voxels having , and determining an inter-reference value using intensities of remaining voxels. A system for automatically processing 3D images of an object to identify and/or characterize cancerous lesions in the object, comprising:
a processor of the computing device; and
a memory in which instructions are stored, wherein the instructions, when executed by the processor, cause the processor to:
(a) receiving a 3D functional image of the object obtained using a functional imaging technique;
(b) receiving a 3D anatomical image of the object obtained using an anatomical imaging technique;
(c) receive a 3D segmentation map identifying one or more specific tissue region(s) or tissue region group(s) within the 3D functional image and/or within the 3D anatomical image;
(d) automatically detecting and/or segmenting a set of one or more hotspots within the 3D functional image using one or more machine learning module(s) to (i) identify for each hotspot a location of the hotspot; generate one or both of a hotspot list and (ii) a 3D hotspot map identifying, for each hotspot, a corresponding 3D hotspot volume within the 3D functional image - each hotspot is in a local area of high intensity relative to its surroundings; and represents a potential cancerous lesion in the subject, wherein at least one of the one or more machine learning module(s) produces (i) the 3D functional image, (ii) the 3D anatomical image, and (iii) the 3D segmentation map. received as input ― ;
(e) A system for automatically processing 3D images of an object, allowing storage and/or presentation of the hotspot list and/or the 3D hotspot map for display and/or further processing. 122. The method of claim 122,
The instructions cause the processor to:
receive an initial 3D segmentation map identifying one or more specific tissue regions within the 3D anatomical image and/or the 3D functional image;
identify at least some of the one or more specific tissue regions as belonging to a specific one of one or more tissue groupings, and update the 3D segmentation map to indicate the identified specific regions as belonging to the specific tissue grouping; do;
A system for automatically processing 3D images of an object to use the updated 3D segmentation map as an input to at least one of the one or more machine learning modules. 123. The method of claim 123,
The system for automatically processing 3D images of an object, wherein the one or more tissue groupings include a soft tissue grouping so that specific tissue regions representing soft tissue are identified as belonging to the soft tissue grouping. The method of claim 123 or 124,
The system for automatically processing 3D images of an object, wherein the one or more tissue groupings include a bone tissue grouping so that specific tissue regions representing a bone are identified as belonging to the bone tissue grouping. The method of any one of claims 123 to 125,
The one or more tissue groupings include a high absorption organ grouping so that one or more organs related to the high absorption radiopharmaceutical are identified as belonging to the high absorption grouping. The method of any one of claims 122 to 126,
wherein the instructions cause the processor to determine, for each detected and/or segmented hotspot, a classification for the hotspot. 127. The method of claim 127,
The instructions may cause the processor to, for each detected and/or segmented hotspot, use at least one of the one or more machine learning modules to determine a classification for the hotspot. system for processing. The method of any one of claims 122 to 128,
The one or more machine learning modules,
(A) a systemic lesion detection module that detects and/or segments hotspots throughout the body; and
(B) A system for automatically processing 3D images of an object, comprising a prostate lesion module that detects and/or segments hotspots within the prostate. 129. The method of claim 129,
wherein the instructions cause the processor to generate the hotspot list and/or maps using (A) and (B), respectively, and merge the results. The method of any one of claims 122 to 130,
In step (d), the instructions cause the processor to segment and classify a set of one or more hotspots to identify for each hotspot a corresponding 3D hotspot volume within the 3D functional image, each hotspot being a plurality of hotspots. A labeled 3D hotspot map labeled as belonging to a particular hotspot class among the classes;
generating a first initial 3D hotspot map identifying a first set of initial hotspot volumes by segmenting a first initial set of one or more hotspots within the 3D functional image using a first machine learning module - the first machine learning module; divides the hotspots of the 3D functional image according to a single hotspot class;
generating a second initial 3D hotspot map identifying a second set of initial hotspot volumes by segmenting the second initial set of one or more hotspots within the 3D functional image using a second machine learning module - the second machine learning module divides the 3D functional image according to a plurality of different hotspot classes so that the second initial 3D hotspot map is a multi-class 3D hotspot map in which each hotspot volume is labeled as belonging to a particular one of the plurality of different hotspot classes. make it happen ;
the first initial 3D hotspot map and the second initial 3D hotspot map of at least a portion of the hotspot volume identified by the first initial 3D hotspot map;
identify a matching hotspot volume in the second initial 3D hotspot map, the matching hotspot volume in the second 3D hotspot map being labeled as belonging to a specific one of the plurality of different hotspot classes;
Segmented hotspots of the first 3D hotspot map labeled according to classes identified by labeling a particular hotspot volume of the first initial 3D hotspot map as belonging to a particular hotspot class to which matching hotspots of the second 3D hotspot map belong. generate and merge, thereby creating a merged 3D hotspot map comprising the volumes;
In step (e), the instructions cause the processor to store and/or provide the merged 3D hotspot map for display and/or further processing. 131. The method of claim 131,
The plurality of different hotspot classes are
(i) bone hotspots determined to represent lesions located in bone;
(ii) lymphatic hotspots determined to represent lesions located in the lymph nodes, and
(iii) prostate hotspots determined to represent lesions located in the prostate
A system for automatically processing 3D images of an object, including one or more members selected from the group consisting of. The method of any one of claims 122 to 132,
The instructions further cause the processor to:
(f) receive and/or approve the hotspot list;
(g) a system for automatically processing 3D images of an object, to segment the hotspot using an analysis model for each hotspot in the hotspot list. The method of any one of claims 122 to 133,
The instructions further cause the processor to:
(h) receive and/or accept the hotspot map;
(i) a system for automatically processing 3D images of an object, to segment the hotspot using an analysis model for each hotspot in the hotspot map. 134. The method of claim 134,
The analysis model is an adaptive threshold setting method,
In step (i), the instructions cause the processor to:
determine one or more reference values based on dimensions of intensities of voxels of the 3D functional image located in a specific reference volume corresponding to a specific reference tissue region, respectively;
For each specific hotspot map of the 3D hotspot map,
determine a corresponding hotspot intensity based on intensities of voxels within the particular hotspot volume;
for automatically processing 3D images of an object, to determine a hotspot specific threshold for the specific hotspot based on at least one of (i) the corresponding hotspot intensity and (ii) the one or more reference value(s). system. 135. The method of claim 135,
The hotspot specific threshold is determined using a specific threshold function selected from a plurality of threshold functions, and the specific threshold function is selected based on a comparison of a corresponding hotspot intensity with at least one reference value. A system for automatically processing 3D images. 136. The method of claim 135 or 136,
The system for automatically processing 3D images of an object, wherein the hotspot specific threshold is determined as a variable percentage of a corresponding hotspot intensity, and wherein the variable percentage decreases with increasing hotspot intensity. A system for automatically processing 3D images of an object to identify and/or characterize cancerous lesions in the object, comprising:
a processor of the computing device; and
a memory having stored therein instructions, which when executed by the processor cause the processor to:
(a) receiving a 3D functional image of the object obtained using a functional imaging technique;
(b) automatically segmenting a first initial set of one or more hotspots in the 3D functional image using a first machine learning module, thereby identifying a first set of initial hotspot volumes that is a corresponding 3D hotspot volume in the 3D functional image. generate a first initial 3D hotspot map, wherein the first machine learning module segments hotspots of the 3D functional image according to a single hotspot class;
(c) generate a second initial 3D hotspot map identifying a second set of initial hotspot volumes by automatically segmenting a second initial set of one or more hotspots within the 3D functional image using a second machine learning module; wherein the second machine learning module segments the 3D functional image according to a plurality of different hotspot classes so that the second initial 3D hotspot map labels each hotspot volume as belonging to a particular one of the plurality of different hotspot classes. Make it a multi-class 3D hotspot map - ;
(d) the first initial 3D hotspot map and the second initial 3D hotspot map for each particular hotspot volume of at least a portion of the first set of initial hotspot volumes identified by the first initial 3D hotspot map;
identify a matching hotspot volume in the second initial 3D hotspot map, the matching hotspot volume in the second 3D hotspot map being labeled as belonging to a specific one of the plurality of different hotspot classes;
Segmented hotspots of the first 3D hotspot map labeled according to classes identified by labeling a particular hotspot volume of the first initial 3D hotspot map as belonging to a particular hotspot class to which matching hotspots of the second 3D hotspot map belong. merge by creating a merged 3D hotspot map containing the volumes;
(e) A system for automatically processing 3D images of an object, enabling storage and/or presentation of the merged 3D hotspot map for display and/or further processing. 138. The method of claim 138,
The plurality of different hotspot classes are
(i) bone hotspots determined to represent lesions located in bone;
(ii) lymphatic hotspots determined to represent lesions located in the lymph nodes, and
(i) Prostate hotspots determined to represent lesions located in the prostate
A system for automatically processing 3D images of an object, including one or more members selected from the group consisting of. A system for automatically processing 3D images of an object through an adaptive thresholding approach to identify and/or characterize cancerous lesions in the object, comprising:
a processor of the computing device; and
a memory having instructions stored therein, which instructions, when executed by the processor, cause the processor to:
(a) receiving a 3D functional image of the object obtained using a functional imaging technique;
(b) receive a preliminary 3D hotspot map identifying one or more preliminary hotspot volumes within the 3D functional image;
(c) determine one or more reference values based on dimensions of intensities of voxels of the 3D functional image, respectively, located within a specific reference volume corresponding to a specific reference tissue region;
(d) for each particular preliminary hotspot volume of at least a portion of the one or more preliminary hotspot volumes identified by the preliminary 3D hotspot map and based on the preliminary hotspot volumes;
determine a corresponding hotspot intensity based on intensities of voxels within the specific preliminary hotspot volume;
determine a hotspot specific threshold for the specific preliminary hotspot based on at least one of (i) the corresponding hotspot intensity and (ii) the one or more reference value(s);
A refined and analytically segmented hotspot corresponding to the specific preliminary hotspot volume by segmenting at least a portion of the functional 3D using a threshold-based segmentation algorithm that performs image segmentation using a hotspot specific threshold determined for the specific preliminary hotspot. determine the volume;
include the refined hotspot volume within the refined 3D hotspot map to generate a refined 3D hotspot map using adaptive threshold based segmentation;
(e) A system for automatically processing 3D images of an object, allowing storage and/or providing the refined 3D hotspot map for display and/or further processing. 140. The method of claim 140,
The hotspot specific threshold is determined using a specific threshold function selected from a plurality of threshold functions, and the specific threshold function is selected based on a comparison of a corresponding hotspot intensity with at least one reference value. A system for automatically processing 3D images. 141. The method of claim 140 or 141,
The system for automatically processing 3D images of an object, wherein the hotspot specific threshold is determined as a variable percentage of a corresponding hotspot intensity, and wherein the variable percentage decreases with increasing hotspot intensity. A system for automatically processing 3D images of an object to identify and/or characterize cancerous lesions in the object, comprising:
a processor of the computing device;
a memory having instructions stored therein, which instructions, when executed by the processor, cause the processor to:
(a) receiving a 3D anatomical image of the object obtained using an anatomical imaging technique, wherein the 3D anatomical image includes a graphic representation of a tissue in the object;
(b) 3D segmentation for identifying a plurality of volumes of interest (VOIs) including a liver volume corresponding to the liver of the object and an aorta volume corresponding to the aorta portion in the 3D anatomical image by automatically segmenting the 3D anatomical image create a map;
(c) receiving a 3D functional image of the object obtained using a functional imaging technique;
(d) automatically segment one or more hotspots in the 3D functional image to identify one or more automatically segmented hotspot volumes, each segmented hotspot corresponding to a local region of high intensity relative to its surroundings and providing potential in the object; - indicates cancerous lesions;
(e) cause rendering of a graphical representation of the one or more automatically segmented hotspot volumes for display within an interactive graphical user interface (GUI);
(f) receive, via the interactive GUI, a user selection of a final hotspot set comprising at least a portion of the one or more automatically segmented hotspot volumes;
(g) for each hotspot volume in the final set, using (i) the intensities of voxels of the functional image corresponding to the hotspot volume and (ii) the intensities of voxels of the functional image corresponding to liver volume and aortic volume determine a lesion index value based on the determined one or more reference values;
(h) a system for automatically processing 3D images of an object, allowing storage and/or presentation of the final set of hotspots and/or lesion index values for display and/or further processing. 143. The method of claim 143,
In step (b), the instructions cause the processor to segment the anatomical image so that the 3D segmentation map identifies one or more bone volumes corresponding to one or more bones of the object;
In step (d), the instructions cause the processor to identify a bone volume using one or more bone volumes within the functional image and segment one or more bone hotspot volumes located within the bone volume. A system for automatically processing images. The method of claim 143 or 144,
In step (b), the instructions cause the processor to segment the anatomical image so that the 3D segmentation map identifies one or more organ volumes corresponding to soft tissue organs of the object;
In step (d), the instructions cause the processor to identify a soft tissue volume within the functional image using the one or more segmented organ volumes and locate one or more lymphatic and/or prostate hotspot volumes located within the soft tissue volume. A system for automatically processing 3D images of an object, leading to segmentation. 145. The method of claim 145,
In step (d), the instructions cause the processor to adjust intensities of the functional image to suppress intensity from one or more highly absorptive tissue regions prior to segmenting the one or more lymphatic and/or prostate hotspot volumes. , A system for automatically processing 3D images of an object. The method of any one of claims 143 to 146,
In step (g), the instructions cause the processor to determine a liver reference value using intensities of voxels of the functional image corresponding to the liver volume. 147. The method of claim 147,
The instructions cause the processor to:
Fit a two-component Gaussian mixture model to the histogram of intensities of functional image voxels corresponding to liver volumes;
identify and exclude voxels with intensities associated with abnormally low absorption regions from the liver volume using a two-component Gaussian mixture model fit;
A system for automatically processing 3D images of an object, allowing intensities of remaining voxels to be used to determine a liver reference value.

Images