JP3243177U - Thermal ablation monitoring using B-mode ultrasound image registration - Google Patents

Abstract

Systems, displays and methods are provided for performing ultrasound image registration and for using ultrasound images to guide thermal ablation. Registration involves correlating successive ultrasound images, identifying keyframes from the correlation values, identifying periodic changes corresponding to respiration and heartbeat, and determining the same phase for the identified periodic changes. This is done by correlating pixels in successive keyframes with Based on this alignment, the onset of ablation is detected, bubbles formed in this ablation procedure are identified, and their movement is tracked. They all use only B-mode ultrasound images. Using the identified gas bubbles, thermally damaged tissue regions are defined and provided in real-time with accuracy similar to prior art post-ablation results. The disclosed systems and methods therefore provide physicians with an ultrasound-based method of monitoring thermal ablation in real time. [Selection drawing] Fig. 1A

Description

The present invention relates to the field of thermal ablation and, more particularly, to ultrasonic monitoring of thermal ablation procedures.

Current practice uses MRI (Magnetic Resonance Imaging) or CT (Computer Tomography) to measure thermal ablation damage to tissue and ablation procedure efficiency 24 hours after the procedure is performed and after the tissue has settled. evaluated.

U.S. Pat. Nos. 7,367,944 and 8,603,015 teach methods and systems for monitoring tissue ablation and are hereby incorporated by reference in their entirety. .

U.S. Pat. No. 7,367,944 U.S. Pat. No. 8,603,015

SUMMARY The following is a simplified summary to provide an initial understanding of the invention. This summary does not necessarily identify key elements or limit the scope of the invention, but merely serves as an introduction to the description that follows.

One aspect of the present invention is an ultrasound display associated with a thermal ablation unit comprising a repeatedly updated B-mode ultrasound image and thermally damaged tissue aligned with the repeatedly updated B-mode ultrasound image. and at least one indication of.

One aspect of the present invention is a method of ultrasound image registration performed on a plurality of consecutive ultrasound images, comprising the steps of calculating and correlating the consecutive ultrasound images to obtain a correlation value; identifying a plurality of keyframes from the values; identifying at least one periodic change in the sequential ultrasound images from the correlation values; and performing ultrasound image registration by correlating pixels in successive keyframes with phases.

One aspect of the invention is a method of monitoring a thermal ablation procedure, identifying the onset of ablation by a change in correlation value that occurs within a specified duration and above a specified threshold. and identifying bubbles formed by the ablation in the registered ultrasound images; detecting movement of the bubbles in the registered ultrasound images; delimiting (demarcating) a thermally damaged tissue region with respect to and excluding detected bubble motion, in a targeted and real-time manner.

One aspect of the present invention is to calculate and correlate received consecutive ultrasound images to obtain a correlation value, identify a plurality of keyframes from the correlation value, Performing ultrasound image registration by identifying at least one periodic change in the image and correlating pixels in successive keyframes having the same phase with respect to the identified at least one periodic change. An ultrasonic alignment module is provided that is configured to:

One aspect of the invention is an ultrasound image-guided system for thermal ablation, comprising an ultrasound registration module configured to receive a plurality of sequential ultrasound images and a specified duration of time. an ablation monitoring module configured to identify the onset of ablation by a change in correlation value occurring within and exceeding a specified threshold.

These, additional and/or other aspects and/or advantages of the invention are set forth in, or likely can be inferred from, the detailed description and/or learned by practice of the invention, which follows.

For a better understanding of embodiments of the invention and to show how embodiments of the invention may be practiced, reference will now be made, purely by way of example, to the accompanying drawings. In the drawings, like numerals represent corresponding elements or parts throughout.

1A-1E are high-level schematic block diagrams of ultrasound image-guided systems for thermal ablation, according to some embodiments of the present invention. 1A-1E are high-level schematic block diagrams of ultrasound image-guided systems for thermal ablation, according to some embodiments of the present invention. 1A-1E are high-level schematic block diagrams of ultrasound image-guided systems for thermal ablation, according to some embodiments of the present invention. 1A-1E are high-level schematic block diagrams of ultrasound image-guided systems for thermal ablation, according to some embodiments of the present invention. 1A-1E are high-level schematic block diagrams of ultrasound image-guided systems for thermal ablation, according to some embodiments of the present invention. 2A and 2B provide schematic non-limiting examples for the derivation and use of keyframes, according to some embodiments of the present invention. 2A and 2B provide schematic non-limiting examples for the derivation and use of keyframes, according to some embodiments of the present invention. FIG. 3 provides a schematic non-limiting example of periodic change detection using frequency analysis, according to some embodiments of the present invention. FIG. 4 provides a schematic non-limiting example of detecting the onset of ablation, according to some embodiments of the present invention. FIG. 5 provides a schematic, non-limiting example of monitoring bubble motion dynamics and its relationship to a blood vessel, according to some embodiments of the present invention. FIG. 6 provides a non-limiting example of handling shadowed regions according to some embodiments of the present invention. FIG. 7A is a high-level schematic illustration of the relationship between bubble release and tissue coagulation, according to some embodiments of the present invention. FIG. 7B is a high-level schematic diagram of a color representation of the bubble development phase, according to some embodiments of the present invention. FIG. 7C is a high-level schematic illustration of sequential ultrasound images with respective defined regions of damaged tissue, according to some embodiments of the present invention. 7D and 7E are high level schematic examples of gray level histograms, according to some embodiments of the present invention. 7D and 7E are high level schematic examples of gray level histograms, according to some embodiments of the present invention. 8A and 8B provide non-limiting examples of tracking treatment regions within enhanced ultrasound images, demonstrating maintenance of proper registration, according to some embodiments of the present invention. , demonstrating tracking through movement in the image. 8A and 8B provide non-limiting examples of tracking treatment regions within enhanced ultrasound images, demonstrating maintenance of proper registration, according to some embodiments of the present invention. , demonstrating tracking through movement in the image. FIG. 9A is a high-level schematic diagram of an ultrasound display, according to some embodiments of the present invention. FIG. 9B is a high-level schematic illustration of the use of CT-based images and ultrasound images for planning an ablation procedure, according to some embodiments of the present invention. FIG. 10 is a high-level flowchart illustrating a method according to some embodiments of the invention. This is a continuation of FIG. 10(1). This is a continuation of FIG. 10(2). FIG. 11A provides an illustration of the compared damaged tissue areas. FIG. 11B provides a Bland-Altman plot presenting the level of similarity between the defined damaged tissue and the comparative experimental images, according to some embodiments of the present invention. FIG. 11C provides injured tissue measured by contrast-enhanced CT 24 hours after ablation, as measured in the prior art.

The following description describes various aspects of the invention. For purposes of explanation, specific configurations and details are set forth in order to provide a thorough understanding of the invention. However, it will also be apparent to those skilled in the art that the present invention may be practiced without the specific details presented herein. Additionally, well-known features may be omitted or simplified in order not to obscure the invention. With particular reference to the drawings, the details shown are for purposes of illustration and are intended for exemplary discussion of the invention only, and are the most useful and readily understood of the principles and conceptual aspects of the invention. It is emphasized that it is presented to provide what is believed to be an explanation that In this regard, while no attempt is made to show structural details of the invention in more detail than is necessary for a basic understanding of the invention, the description accompanying the drawings provides an understanding that some aspects of the invention are: It will be clear to those skilled in the art how it may be embodied in practice.

Before describing at least one embodiment of the invention in detail, it is important to note that this invention is not limited in its application to the details of construction and arrangement of components set forth in the following description or shown in the drawings. Please understand. The invention is applicable to other embodiments and combinations of the disclosed embodiments that may be practiced or carried out in various ways. Also, it is to be understood that the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting.

Throughout this specification, unless otherwise indicated, as will be apparent from the discussion below, ”, “emphasis”, “derivation”, etc. are expressed as physical quantities (e.g., electronic quantities) in the registers and/or memory of a computing system. Computing, manipulating and/or transforming data into other data similarly represented as physical quantities within a computing system's memory, registers, or other apparatus for the storage, transmission, or display of such information It should be understood to refer to the operations and/or processes of a system or computer, or similar electronic computing device.

Embodiments of the present invention provide efficient and economical methods and mechanisms for image-guided thermal ablation (thermal ablation) of, for example, malignant tumors, particularly using ultrasound imaging, thereby providing medical It provides improvements to the art of ablation and ultrasound imaging. Image-guided thermal ablation (IGA) of malignant tumors is gradually becoming the first line of treatment and is being explored in a wide variety of clinical settings as a viable alternative to tumor resection in open surgery (laparotomy, thoracotomy). applied in the field. The disclosed IGA methods are configured, in at least some embodiments, to provide a cytolethal ablation zone with clinically effective margins and minimal collateral damage at least equivalent to the results of open surgery. be done.

Disclosed embodiments improve and/or any of the four typical phases of effective IGA treatment, including pre-treatment imaging, pre-treatment planning, and, among other things, online-monitored tumor ablation and post-treatment assessment. provide a solution. Moreover, for viable percutaneous tumor ablation procedures (procedures) where the tumor is percutaneously accessible, the disclosed embodiments are useful for pretreatment imaging and planning and, in particular, damage to adjacent tissue. and provide controllable thermal ablation of tumors that allows the practitioner to visualize, confirm and verify treatment response non-invasively, possibly in real-time. In particular, the disclosed real-time or near-real-time feedback enables accurate, complete, and identifiable thermal ablation of tumors, in contrast to prior art methods that typically require waiting one day for imaging feedback on tissue damage. enable Advantageously, as opposed to prior art attempts to monitor ablation, the disclosed embodiments use a biological signature of tissue, i.e., how tissue responds to thermal ablation, to quantify ablation damage. According to a typical embodiment.

Systems, displays and methods are provided for performing ultrasound image registration and for using ultrasound images to guide thermal ablation. Registration computes and correlates consecutive ultrasound images, identifies keyframes from the correlation values, identifies periodic change(s) corresponding to respiration and heartbeat, and for the identified periodic changes: It is performed by correlating pixels in consecutive keyframes that have the same phase. Based on this alignment, the onset of ablation is detected, bubbles formed in this ablation procedure are identified, and their movement is tracked. They all use only B-mode ultrasound images. Using the identified gas bubbles, thermally damaged tissue regions are defined and provided in real-time with accuracy similar to prior art post-ablation results. The disclosed systems and methods therefore provide physicians with an ultrasound-based method of monitoring thermal ablation in real time. Additional features include assessment of ultrasound image quality, safety alerts, ablation analysis, and simulation and planning of ablation procedures.

1A-1E are high-level schematic block diagrams of an ultrasound image-guided system 100 for thermal ablation, according to some embodiments of the present invention. FIG. 1A schematically illustrates a thermal ablation procedure performed using the thermal ablation tool 80 and modules of system 100, and FIG. 1B shows ultrasonic alignment module 101 and safety module 103 and image quality assessment within system 100. Details of module 105 are schematically shown, and FIG. 1C schematically shows details of ablation monitoring module 102 of system 100 . The separation of system operations into modules is schematic, non-limiting, and is intended for illustrative purposes only. FIG. 1D schematically illustrates an algorithm flow for estimating liver tissue damage, according to some embodiments of the present invention. FIG. 1E schematically illustrates a computing device 109, described below.

FIG. 1A schematically illustrates a thermal ablation procedure performed using a thermal ablation tool 80, for example using RF or possibly laser energy, to ablate tissue 70, for example a tumor in the liver. shown in The thermal ablation procedure is monitored in real time by an ultrasound module 85 operating in B-mode, which identifies, for example, ablated tissue and target tissue 70 to monitor the thermal ablation procedure. Distinguish from surrounding tissue, provide feedback on the progress of the thermal ablation procedure, indicate the extent of target tissue 70 removed, warn about surrounding tissue ablated and/or approaching safety boundaries, and target tissue. A series of ultrasound images 90 used by the ultrasound image-guided system 100 are provided to provide a real-time assessment of the success of the thermal ablation procedure in removing 70 . Current technology requires a period of about one day to elapse before reliable imaging can be used to determine the extent of the ablated target tissue 70 and damage to surrounding tissue; Note that another procedure should then be performed to correct any deficiencies. In the disclosed embodiments, the feedback provided is immediate and allows optimization of the thermal ablation procedure during its execution.

In various embodiments, an ultrasound image-guided system 100 for thermal ablation includes an ultrasound registration module 101 configured to provide continuity and spatial orientation from incoming successive ultrasound images 90. , and an ablation monitoring module 102 configured to detect and monitor the thermal ablation procedure. Additional modules are used to provide safety warnings (e.g. safety module 103), assess image quality (e.g. image quality assessment module 105) and possibly improve image quality, and after the thermal ablation procedure is completed. Reports on the thermal ablation procedure may be provided, additional data and analysis may be derived, etc. (eg, ablation analysis module 104), and the thermal ablation procedure may be planned (eg, simulation and planning module 106).

The ultrasound registration module 101 (see, for example, FIG. 1B) calculates and monitors at least one correlation value 110 between received consecutive ultrasound images 90, the calculated correlation value(s) 110 identifying a plurality of keyframes 120 from, identifying at least one periodic change 130 in successive ultrasound images 90 from the values of the calculated correlation values 110; and identifying the identified periodic change 130, It may be configured to perform ultrasound image registration 140 by correlating pixels in consecutive keyframes 120 that have the same phase.

2A and 2B provide schematic non-limiting examples of the derivation and use of keyframes 120 according to some embodiments of the present invention. The keyframes 120 may be grouped in a keyframe bank 121 containing, for example, several, tens or hundreds of keyframes 120, and the incoming image 90 may be compared to these keyframes 120. . For example, incoming images 90 may be compared to the last keyframe and previous keyframes based on their periodicity with respect to image features and periodicity 130 . For example, motion parameter autocorrelation may be used to select keyframes 120 from keyframe bank 121 for incoming image 90 . Switching the keyframes 120 used may be configured to overcome the drift problem. In certain embodiments, the selection of the keyframe 120 as a starting point for a model predicting motion parameters (e.g., velocity model), possibly applying enhanced correlation coefficient (ECC) maximization. may be applied for

In particular embodiments, circular frame buffer 122 may contain hundreds or thousands of consecutive frames representing one or more cycles of periodic variation 130 . Incoming images 90 may be compared to the last and any frame in circular frame buffer 122 based on their periodicity with respect to image characteristics and periodicity 130 .

FIG. 2B provides a schematic non-limiting example for implementing periodic change 130 detection and keyframe selection, according to some embodiments of the present invention. In particular embodiments, the keyframes 120 may be selected from the frame buffer 121 using, for example, autocorrelation of motion parameters computed using a fixed-size sliding window 123 to correlate with the past. FIG. 2B provides an example of a sliding window 123 (sliding back with respect to time) applied to Pearson's θ and the resulting autocorrelation values over time. Time point 124 provides an autocorrelation peak, indicating the periodicity of periodic variation 130 . Clearly, alternative methods of detecting periodicity may be applied to the image data or portions thereof (e.g., periodicity may be associated with, for example, blood vessels, treatment regions, or other portions of the image). may be detected in certain parts).

In various embodiments, the ultrasound image registration 140 uses parametric image alignment using extended correlation coefficient maximization (ECC) and/or Dense Optical Flow using two-frame motion estimation based on polynomial expansion. ) to one or more of the keyframes 120 . One or more of known alignment techniques may be applied to achieve alignment between keyframes 120 and/or to predict the required transformation of a particular keyframe 120 . Ultrasound image registration 140 presents challenges inherent in registering 2D images during ablation, i.e. non-rigid transformations between frames due to plane changes in different 2D images, captured sections in different images. correspond to each other, as well as tissue flexibility and apparent ablation effects that periodically obscure elements in ultrasound images.

Before and during an ablation procedure, the physician changes the orientation of the ultrasound image plane, resulting in an ultrasound image that cannot be registered with other ultrasound images in the sequence because it does not display the corresponding tissue cross-section. Note that you may The ultrasound registration module 101 may be configured to detect out-of-plane images and/or align only ultrasound images that are in the same plane.

In various embodiments, an ultrasound image-guided system 100 for thermal ablation receives a plurality of consecutive ultrasound images 90, identifies at least one keyframe 120 from the received ultrasound images 90, Applying at least two registration algorithms to the ultrasound image 90 received for that keyframe 120 to obtain at least two corresponding confidence scores, where the at least two confidence scores exceed a specified threshold ( includes an ultrasound registration module 101 configured to align the received ultrasound images (e.g., 80%, 85, 90%, 95%, or above intermediate). For example, using two different types of registration algorithms, such as ECC and optical flow, allows out-of-plane images to be detected as images with at least one of the confidence scores being low (e.g., less than 80%). Sometimes.

In particular, registration is difficult during an ablation procedure due to the sometimes drastic changes in the ultrasound image and the continuous changes in the extent of the ablated tissue. Using two or more registration algorithms with their respective confidence scores may allow out-of-plane images to be detected, thus maintaining surveillance of the ablated region. For example, since ECC and optical flow are inherently different in the way they compute registration, ultrasound images 90 with high confidence scores by both algorithms should be the respective reference against which they are registered. It is most likely in the same plane as the key image 120 .

Note that one keyframe 120 at a time may be sufficient in some embodiments and may be updated as the ablation progresses. Alignment algorithms may be applied to compare predicted images with respect to keyframes 120 and/or to use one or more keyframes 120 as anchors to avoid alignment drift. .

In certain embodiments, ultrasound registration module 101 may be configured to derive an aggregate confidence score by performing a voting procedure on the results of at least two registration algorithms. Specifically, the confidence scores may be used to discard outliers (if transforms from all alignments yield the same output image, confidence is high, whereas they are On the one hand, the voting process may be used to ensure that the alignment is not between unrelated images (e.g., images from different planes).

As disclosed herein, the ablation monitoring module 102 identifies the onset of ablation in the registered images, identifies bubbles formed by the ablation in the registered ultrasound images, and locates them. Detecting bubble movement in the registered ultrasound images, indicating ablation areas on the registered ultrasound images, e.g., indicating borders of heat-damaged tissue, indicating colored areas indicating heat-damaged tissue, and/or Alternatively, it may be configured to indicate bubble formation (phase 1), bubble development (phase 2) and bubble saturation (phase 3).

If the ablated region appears to shrink or change unexpectedly, the confidence score derived by the registration algorithm either skips each image until the imaging plane returns to resemble keyframe 120, or Or it may be used to avoid false indications of ablated regions (eg, to display respective out-of-plane indications).

In embodiments that include multiple monitoring planes, the ultrasound image may be registered with each plane as disclosed herein.

Ultrasound image registration 140 may be performed by applying known registration techniques such as ECC and Dense Optical Flow directly to specific keyframes 120 and/or to match other specified keyframes 120. It may be performed to derive a transform for a particular keyframe 120 . Multiple registration techniques may be applied to the specified keyframe 120 and/or related keyframes to yield a prediction of the specified keyframe 120, and the ultrasound image registration 140 may include multiple Select aligned keyframes by processing the candidates and deriving their respective confidence scores that can be used to arrive at a consensus vote and/or confidence score for a given keyframe 120 It may be configured as

In certain embodiments, the ultrasound registration module 101 may be configured to perform ultrasound image registration 140 on received consecutive ultrasound images 90 in real time. Real-time registration 140 combines the technical implementation of the ultrasound registration module 101, the ultrasound image quality, the ultrasound operator's proficiency, possibly with the slight delay required to perform the image processing procedure. etc., and may be performed during a thermal ablation procedure. The term “real-time” is used herein for no delay, a few milliseconds (eg, 1-5 milliseconds, 5-10 milliseconds, or any delays of tens of milliseconds (eg, 10-50 ms, 50-100 ms, or any intermediate range), delays of hundreds of milliseconds (eg, 100-500 ms, 500-500 ms, 1000 milliseconds, or any intermediate range), or delays up to several seconds (eg, 1-5 seconds, or any intermediate range).

The ultrasound registration module 101 may be configured to process the ultrasound image 90 with respect to the ultrasound imaging module 85 type. In various embodiments, the ultrasound registration module 101 may further include an ultrasound imaging module 85 and may be tailored to the type of ultrasound imaging module 85 . In various embodiments, the ultrasound registration module 101 and the ultrasound imaging module 85 may be partially or fully integrated.

In certain embodiments, the ultrasound registration module 101 is configured to derive medical parameters 135, such as respiratory rate and/or heart rate, and/or to indicate patient motion from the identified periodic changes 130. may be For example, medical parameters 135 may be derived from ultrasound image 90 and/or frequency analysis 131 of designated regions thereof. The identified frequencies (of periodic changes 130) are used to monitor medical parameters 135, such as heart rate and respiration rate, and to enhance vessel detection 172, 174 and/or bubble movement 170 through vessels. may also be used.

FIG. 3 provides a schematic non-limiting example for detection of periodic changes 130 using frequency analysis 131, according to some embodiments of the present invention. Frequency analysis 131 may involve applying a Fourier transform 132 to the sequence of images 90 (and/or image portions using, for example, masks) and detecting salient frequencies. In the illustrated non-limiting example, a significant frequency of 0.25-0.35 Hz may be associated with the patient's respiratory rate, while a significant frequency of 1.86 Hz was identified in the analysis. may, but need not be immediately associated with a particular medical parameter. Identification of the medical significance of what is detected may be achieved indirectly by relationship to the monitored medical parameter (eg, heart rate) and/or directly by further analysis 131 . In the illustrated non-limiting example, the image analysis and/or enhancement 133 includes, for example, an ultrasound (US) image 90 arriving through a temporal bandpass filter at the detected frequencies and/or enhanced (e.g., motion may be applied at the detected frequencies by applying 134 to the ultrasound image 185 (compensated). In the example provided, a prominent (temporal) frequency of 1.86 Hz, which could be suspected to be related to heart rate, indeed appears to be spatially correlated with blood vessels in ultrasound images, making them more distinct. It may be used to further enhance the ultrasound image 185 for viewing purposes. As discussed herein (see, eg, tissue cooling model 116 ), vascular cooling and bubble transport effects may be used in estimating damaged tissue area 180 .

In certain embodiments, the identified bubbles 160 (see below) move the vessels according to their moving pulse frequency, possibly in correlation with previously identified vessels (eg, using CT data). It may be further identified as moving through. The detected medical parameters 135 may also be used to reacquire the treatment area in case of patient motion. First, the patient's movement itself may be monitored and used to indicate the return of the patient to the original posture, and second, the regularity of the medical parameters 135, e.g. may be used to identify the focal plane prior to and re-establish keyframes corresponding to the treatment region. Reacquisition or loss-of-tracking recovery of the treatment region, in some cases, utilizes data collected until loss of tracking, e.g., regarding image characteristics of the tracking region, and/or orthogonal planes as disclosed herein. may be achieved by any of the registration methods disclosed herein that utilize the images in the .

In certain embodiments, the ultrasound registration module 101 may be configured to detect out-of-plane frames 111 by a decrease in monitored correlation value 110 greater than a specified threshold. For example, frequency analysis may be used to detect the regular return of the treatment area to the plane of ultrasound measurement, thereby determining keyframes 120 (see, eg, FIG. 3). Additionally, frequency analysis may be used to remove noise (eg, speckle) from an image, thereby improving image quality.

Note that using the correlation value(s) 110 provides a non-limiting example of processing consecutive ultrasound images 90 . Correlation between ultrasound images 90 may be performed by a variety of methods, including any of the following: For example, Proceedings of the MICCAI 2014 workshop - Challenge on Liver Ultrasound Tracking, CLUST 2014 - (which is incorporated herein by reference in its entirety). Sum of Absolute Differences (SAD), Sum of Squared Differences (SSD), Normalized Cross Correlation (NCC), kernel-based tracking algorithms, normalized gradient fields, as taught in , 2D motion tracking using dense optical flow, Bayesian algorithms, motion estimation frameworks, etc.

In certain embodiments, the ultrasound registration module 101 may be configured to assess the image quality 105 of successive ultrasound images 90 and remove low quality images. In certain embodiments, the ultrasound registration module 101 performs machine learning 114 on the automatically tagged databanks 112 formed by the ultrasound registration module 101 on previously received ultrasound images 90 . may be configured to perform image quality assessment 105 by Tagging may be performed in various ways, e.g., in relation to various measures of image quality, in relation to measures of its usefulness in the registration process (e.g., similarity to a particular keyframe 120), etc. may Image quality 105 may be assessed by a variety of methods, examples of which include non-reference and full-reference image quality assessments, which may relate to quantitative and qualitative aspects of images and image portions associated with ablation procedures. artificial intelligence methods such as deep convolutional neural networks for

In some embodiments, successive ultrasound images 90 may be captured in two mutually orthogonal planes or in 3D, and the ultrasound registration module 101 aligns the successive ultrasound images 90. It may be configured to perform respective alignments. Data from one orthogonal plane may be used to enhance registration in another plane. In certain embodiments, two mutually orthogonal ultrasound planes (referred to as the x-plane and the y-plane) may have an angle of 90° between them, or other angles such as x, y-planes may not be perpendicular to each other) is used in determining keyframes 120 and, if necessary, in reacquiring the treatment region following patient movement. may be Because out-of-plane movement in one plane (e.g., the x-plane) may be tracked and monitored in the other plane (e.g., the y-plane), identifying a new out-of-plane image after the movement has occurred. (because movement is manifested in both planes). The ultrasound registration module 101 may be configured to monitor and control the line of intersection between two planes and use it for registration of ultrasound images in either of the planes. In certain embodiments, an at least partially 3D ultrasound image in which the two planes have details regarding the volumetric extent of the treated tissue, the ablation region, and other nearby features (e.g., blood vessels), or It may be used to derive extended 2D ultrasound images. In certain embodiments, data from both planes may be used to verify that the ablation region covers the entire required volume and that the ablation procedure avoids damage around the lesion ( See safety module 103 below).

The ultrasound image-guided system 100 initiates ablation by detecting a change in the correlation value 110 monitored by the ultrasound registration module 101 that occurs within a specified duration and above a specified threshold. It may further include an ablation monitoring module 102 configured to identify 150 . The ablation initiation 150 may be used to identify a new ablation initiation, for example, when multiple power sources (e.g., multiple RF electrodes or laser focus) are used, or when power sources are used intermittently. may be monitored during For separate ablation sites, ablation monitoring module 102 may be configured to integrate damaged tissue regions from separate ablation sites.

FIG. 4 provides a schematic non-limiting example of detecting the onset of ablation 150, according to some embodiments of the present invention. Detection of changes in incoming image 90 defined by certain parameters is illustrated as probabilities in image 151 derived from changes in image 90 using, for example, frequency analysis 131 . Ablation monitoring module 102 is configured to detect the onset of ablation 150, possibly relative to the detected tip of tool 80, without requiring data from ablation tool 80 itself. As illustrated in FIG. 4, multiple (simultaneous and/or sequential) events of ablation onset 150 are independently detected and overcome prior art difficulties such as masking by needle tip reflections and disturbing factors such as noise and motion. can be overcome. Furthermore, detection of the onset of ablation 150 provides precise timing of lesion formation and improves segmentation accuracy.

The ablation monitoring module 102 is further adapted to identify bubbles 160 formed by ablation in the registered ultrasound images and/or to detect movement 170 of the bubbles in the registered ultrasound images. may be configured. The disclosed system synthesizes grayscale B-mode ultrasound images (e.g., Doppler processing to reduce spatial and temporal resolution) according to at least keyframes 120 and/or periodic changes 130 identified by ultrasound registration module 101. not required) to provide detection of bubble movement 170 above. The dynamics of the bubble movement 170 can vary from tissue regions that still have their own blood supply (due to the vessels that drain the bubble) to regions that do not have an active blood supply to themselves (the bubble is static and not drained). ) tissue regions, the latter corresponding to tissue that has consequently atrophied and may be included in the defined tissue 180, but which is still not directly and sufficiently damaged by the ablation procedure. do. In this way, bubble monitoring can be used to predict the resulting indirect tissue damage, which in the prior art would require waiting 24 hours and performing another imaging procedure. (see validation of this predictive power in the experimental results below).

In certain embodiments, ablation monitoring module 102 may be configured to identify cessation of bubble movement 170, which may be used to indicate a collapsed vessel. Collapsed blood vessels (and tissue supplied thereby) may then be applied to the defined damaged tissue 180 .

FIG. 5 provides a schematic, non-limiting example for monitoring the dynamics of bubble movement 170 and its relationship to blood vessels, according to some embodiments of the present invention. An enhanced ultrasound image 185A illustrates damaged tissue 180 in contact with an intact blood vessel 172 that supports bubble motion 171 and provides a cooling effect 116 that may be modeled. In forming enhanced ultrasound image 185B, ablation monitoring module 102 may be configured to color code damaged tissue 180, intact blood vessel 172, and damaged blood vessel 174, which may indicate air bubbles. It may be identified by cessation of motion 170 . According to the analysis, the damaged tissue area 180 may be applied as, for example, the enhanced ultrasound image 185C, and the ablation increases the extent of the damaged area and over time as the blood vessel damages, coagulates and collapses. It may be shown to develop into a serially enhanced ultrasound image (eg, 185D) with .

The ablation monitoring module 102 is further configured to dynamically and in real-time define the thermally damaged tissue region 180 with respect to incoming sequential ultrasound images and with respect to and to the exclusion of detected bubble motion 170. may be Specifically, the inventors have found that bubble motion 170 may be used to indicate tissue that has not yet been damaged and is therefore expelled from the damaged tissue area 180 . Bubble movement 170 may be monitored for specified frequencies identified as being associated with blood pulsations through blood vessels (see Heart Rate Identification above). Thus, air bubbles may be used as a contrast agent under ultrasonography to mark sustained and stopped blood flow within the ablation region. The cessation of blood flow may then be used to indicate tissue damage and define additions to the damaged area, as disclosed herein.

The ablation monitoring module 102 may be further configured to dynamically and in real-time define an intact blood vessel 172 with respect to the detected bubble motion 170 with respect to incoming sequential ultrasound images (see, eg, FIG. 1C). . In certain embodiments, ablation monitoring module 102 may be configured to correlate defined vessels 172 with vessels identified in enhanced preliminary CT images 175 . This correlation may be used as a validation tool, as a method of enhancing CT images, and/or for ablation analysis 104 and/or sequential ablation simulation and planning 106 . The ablation monitoring module 102 may be further configured to dynamically and in real-time identify the damaged vessel 174 on successive incoming ultrasound images, eg, according to the detected cessation of bubble motion 170 . Ablation monitoring module 102 may be further configured to add the identified damaged vessel 174 to the defined thermally damaged tissue region 180 . For example, the inventors identified a vessel collapse 176 event and accordingly updated the defined damage area 180 by adding the area of the collapsed vessel 176 to the defined damage area 180. We have identified a typical signature of vascular rupture 176 that can be used for

In some embodiments, the ablation monitoring module 102 further determines the typical manner in which tissue responds to thermal ablation by its biological characteristics, e.g., the type of bubbles formed and their through-tissue characteristics. It may be configured to characterize thermally ablated tissue by propagation, rate of damage development within the tissue, distribution of blood vessels across the tissue, and the like. Biological features may also be used to enhance ablation, e.g. tumors supplied by numerous blood vessels may be less than healthy tissue due to the extensive cooling of the ablation area provided by the blood vessels. Note that it may be ablated more slowly than Thus, the rate at which ablation has progressed may be used to characterize the tissue being ablated and possibly to fine-tune the ablation procedure itself.

In certain embodiments, for example, additional data from other ultrasound modes may be used to further enhance the disclosed B-mode-based ultrasound imaging. For example, M-mode or Doppler data may be derived and used to enhance flow in a presented ultrasound image.

FIG. 1D schematically illustrates an algorithmic flow for estimating liver tissue damage, according to some embodiments of the present invention. The algorithm flow includes a preparation stage 202, a perception stage 204 (optionally at least partially performed by the ablation monitoring module 102), and a segmentation stage 206 (optionally at least partially performed by the ablation analysis module 104). ), which may also be part of the method 200 described in FIG. 10 below. The disclosed algorithms may be configured to receive ultrasound images in real time and deliver an enhanced ultrasound image 185 that includes an indication of ongoing thermal ablation effects such as defined damaged tissue 180. good. Examples of such displays include delineating the damaged area, filing the damaged area with one or more colors, altering ultrasound image pixels (e.g., making pixels corresponding to damaged tissue darker in the display). or brighter), adding color to a grayscale image, and the like. For example, any of these modifications may include incorporating data from the parametric coagulation map 181 (see FIG. 7A) described below into the enhanced ultrasound image 185. FIG.

For example, preparation 202 may include ultrasound registration and image quality assessment 105 by ultrasound registration module 101 . Perception stage 204 includes detection of ablation initiation 150 and analysis of hydrodynamics 171 of bubble movement (e.g., bubble development and surrounding vascular , and showing their states as disclosed in FIG. 1C) and shadow detection 191 . Segmentation step 206 includes, for example, ablation segmentation 182, which includes identification and classification of tissue damage in segments (one or more pixels) of the image, and ablation segmentation 184, which further includes, for example, enhancement 192, in shaded regions. , organ segmentation 186, for example, relating tissue damage to anatomical details of the treatment area of each organ, such as the liver, and optionally projected onto the 2D ultrasound image in a variety of ways; 3D damage rendering 188, which may be rendered in 3D fashion directly (e.g., using a 3D representation) or indirectly (e.g., allowing the user to change the plane of the 2D image); may contain.

In certain embodiments, the ablation monitoring module 102 further defines a shaded region 190 within the sequential ultrasound images 90 for the identified bubble 160 and uses the shaded region 190 to determine the distal boundary of the thermally damaged tissue region 180 . may be configured to highlight 192 the Defining the shaded area 190 may be performed with respect to the detected bubble 160 and/or with respect to the detected bubble movement 170 . In the defined damaged tissue 180, a shaded area 190 may be highlighted to better distinguish between tissue damage and intact and damaged vessels and vessel collapse 172, 174, 176, respectively, which may be referred to as the defined injury. may be included in organization 180;

In addition to the shadowed area 190 due to air bubbles, which are dynamic and may change during the ablation procedure, additional shadows due to bones (e.g., ribs) and other tissues and elements that are opaque to ultrasound may occur. Note that shadows may also be present in image 90 . Such shadows may vary between images 90 due to motion and periodicity 130, for example, but are typically more static in the sense that they do not evolve over time. The former (dynamic shadows related to the procedure) are closer to the treated area and are transient and variable, whereas the latter (usually static shadows not related to the procedure) are more stable and constant, the former It poses a larger problem than is discussed in the specification. Obviously, corresponding procedures may also be applied to the latter.

Ultrasound registration module 101 and/or ablation monitoring module 102 apply temporal and directional analysis to ultrasound image 90 to identify shadowed regions 190 and regions within image 90 that they affect and are affected by. may be configured to determine the Directional analysis may be applied radially and/or tangentially.

FIG. 6 provides a non-limiting example of handling shadowed regions 190, according to some embodiments of the present invention. Static shadows 191 are typically unrelated to the procedure, but are associated with ultrasound opaque material within image 90 and typically extend over a larger section of image 185, typically on the sides of the image. It is on top of the body and remote from the treatment area and therefore requires relatively little image processing. However, because shadow 191 is typically dark (e.g., bone shadow) and can limit the system's ability to monitor the ablation procedure, if the extent of ablated tissue approaches the shadowed area, A warning may be provided. For example, direct image enhancement can be achieved by increasing contrast (emphasizing differences between pixels) and/or rescaling the grayscale to cover a narrower gray range within shaded area 191. It may be applied to shaded area 191 . Also, the shaded image area 191 may be positioned to improve the efficiency of the registration process, for example, where the shaded area 191 may interfere with the detection of the periodic change 130 (eg, in terms of processing speed, memory requirements, etc.). It may be at least partially excluded from the alignment process.

Bubbles 160 created during ablation are highly reflective and therefore typically dynamic with bubble movement, localized within the treatment area, and darker than static shadows 191. produces a shadow 192 that is diffuse without Procedure-related dynamic shadows 192 typically require more detailed analysis as they can strongly influence the disclosed tissue damage estimates. As the ablated region expands, the size of the bubble shadow 192 and its impact on the derivation of the damaged region 180 becomes more pronounced, and dedicated processing of the shadowed region 192 may be applied as disclosed above ( For example, contrast and/or brightness enhancement, separate analysis by ablation monitoring module 102, etc.). As schematically illustrated in FIG. 6, the spatial extent of the shaded region 192 is with respect to the direction of the ultrasound source (eg, in a corresponding radial direction, or in an approximated direction depending on the type of radiation source). , may be calculated for all identified bubbles and bubble-containing regions 161 detected by ablation monitoring module 102 . The results of the calculation may be represented as a mask corresponding to the shaded regions 192 and may be spatially integrated for the plurality of identified bubbles and bubble-containing regions 161 . Image processing within the calculated shaded area 192 may be performed as disclosed above.

Additionally or complementary, tissue damage may be estimated in terms of grayscale derivatives in image 90 to overcome, at least to some extent, the disturbances caused by shadows. Enhancement of the image 185 may be applied according to the grayscale derivative to enhance the damaged tissue area 180 . Additionally or complementary, the detection threshold may be lowered in the shadowed regions 190 and/or the grayscale may be normalized in the shaded regions 190 . Machine learning, such as deep learning procedures, may be applied to detect shadows through dynamic changes caused by ablation to improve analysis of damaged tissue.

Tissue characterization and image segmentation may be further enhanced with respect to the effects thermal ablation has on respective tissues. The ablation analysis module 104 distinguishes live from dead tissue (including predicting tissue portions that will atrophy after destruction of the corresponding vessel) and distinguishes between tissue types (e.g., among them (regarding bubble volume, size, and formation conditions), possibly incorporating models for heat dissipation and sound velocity through different tissue types. A label may be added to the enhanced ultrasound image 185 to indicate the tissue type that was segmented and/or identified.

FIG. 7A is a high-level schematic illustration of the relationship between bubble release and tissue coagulation, according to some embodiments of the present invention. Bubble identification 160 and damaged tissue area 180 may be made with respect to the following analysis of the spatiotemporal behavior of local thermal bubble activity induced by heat treatment. As shown schematically in Figure 7A, typical bubble behavior during thermal ablation can be divided into three main phases. During the initial phase (denoted as Phase 1), gas bubbles (e.g., containing nitrogen gas and/or oxygen gas) are released from cells and connective tissue by the heat treatment, and at the same time are partially removed from the area by local blood circulation. effectively removed. Accordingly, the bubble concentration in each region increases at a rate that represents a balance between the processes of bubble ejection and ejection. During the following phase (designated Phase 2), the blood vessels coagulate and stop expelling the bubbles that are forming. Therefore, bubbles continue to form but are not expelled, increasing the bubble concentration rate. Finally, during the final phase (designated Phase 3), the bubble concentration reaches saturation with complete tissue coagulation. It should be noted that tissue that has not been coagulated but whose blood supply has been cut off typically undergoes apoptosis due to ischemia within a day or two and can therefore be considered damaged tissue.

Quantitative estimation of damaged tissue 180 may be performed by measuring bubble concentration throughout the tissue according to the indicated phases. For example, in some embodiments, bubble concentration is estimated using tissue echogenicity, eg, as proportional to tissue echogenicity (integrated back scatter—also referred to as IBS). good too. Bubble concentration may be estimated using regular two-dimensional B-mode ultrasound images at the maximum temporal and spatial resolution provided by the respective ultrasound device, as described herein. It is emphasized that ultrasound-based guidance of corresponding high-resolution ablation may be possible. For example, FIG. 7A further provides an example parametric coagulation map 181 derived in relation to the temporal gradient of the B-mode ultrasound image at each point, which is formally represented by Equation 1. 1, σ(t) represents the parametric coagulation map and B(t) represents the B-mode image value proportional to IBS(t), which represents the local tissue echogenicity as a function of time.
σ(t)=(∂B(t))/∂t(B(t)∝logIBS(t)) Equation 1

Note that in this definition, the parametric coagulation map is not sensitive to the incident ultrasound intensity, and is therefore due to bubble absorption artifacts (e.g., bubbles formed between the point of interest and the ultrasound transducer. Shading above). discussion).

FIG. 7B is a high-level schematic illustration of a color representation of the bubble development phase, according to some embodiments of the present invention. The derived parametric coagulation map 181 can be in various manners, e.g., each pixel with a grayscale value B, or (e.g., using the three colors for phases 1, 2, and 3 described above). and) may be displayed as a color overlay map on the ultrasound image, represented by colors that indicate the local state of ablation. A non-limiting example of such a display is 40 seconds pretreatment (a), 20 seconds treatment (b), 25 seconds treatment (c), 30 seconds using a continuous color scale. (d), 35 seconds (e), and 40 seconds (f) tissue damage resulting from a radiofrequency ablation procedure is provided in FIG. 7B.

Note that parametric coagulation map 181 does not rely on knowledge of tissue- or patient-specific parameters to determine ablation zone morphology. Since σ(t) directly estimates both tissue necrosis and expected (e.g., after 24 h) tissue apoptosis in real-time and is used to image them, the physician can With feedback, the ablation process can be adjusted to ensure complete destruction of the tumor 70 and desired margins with minimal damage to surrounding tissue. It is emphasized that all information necessary to provide the physician with the necessary information (parametric coagulation map 181) is derived from commonly available B-mode ultrasound images.

FIG. 7C is a high-level schematic illustration of sequential ultrasound images 90 enhanced 185 at respective defined damaged tissue regions 180, according to some embodiments of the present invention. 7C, starting with an initial ablation 81, the defined damaged tissue region 180 gradually increases as the thermal ablation progresses until it reaches its final size and morphology shown as a coagulation map 181. grow to Note that the enhanced ultrasound image 185 may be one or more indications of the effects of ongoing thermal ablation, such as delineating the damaged area (outlining), filing the damaged area in one or more colors, Altering the ultrasound image pixels (eg, in terms of their brightness), such as adding color to the grayscale image corresponding to the parametric coagulation map 181, may also be included. The accumulation of defined areas for each image reflects the detected bubbles that occur during the thermal ablation procedure, stationary bubble motion and vessel collapse.

7D and 7E are high-level schematic examples of a gray level histogram 152, according to some embodiments of the present invention. In certain embodiments, the ablation monitoring module 102 further derives a gray-level histogram 152 for the specified region and analyzes the histogram 152 to detect ablation initiation 150, bubble 160 and bubble movement 170, and image artifacts. may be configured to identify For example, typical histogram signatures corresponding to ablation onset 150, any of bubble 160 and bubble motion 170, and tissue type may be identified, and the ablation monitoring module 102 may identify these histogram signatures. may be configured to detect and identify the In certain embodiments, machine learning procedures (eg, deep learning) may be employed to characterize histogram features for different case and/or tissue characterizations. In certain embodiments, the histogram is shown schematically for a particular time in, for example, FIG. It may be further processed by altering the histogram to the line shown. These lines may be used for further analysis or display, and the histogram features represent typical lines that can be associated with each event (e.g., ablation initiation 150, bubble 160, and bubble movement 170). is represented by

The ultrasound image-guided system 100 detects the safety boundary 160 for the ablation procedure and sequential ultrasound images 90 (identified by the ultrasound registration module 101) in real-time with respect to the incoming sequential ultrasound images. Safety module 103 (see Figure 1B). For example, safe boundary 160 may be defined as the boundary of an organ to be treated (e.g., liver or liver lobe), adjacent critical structures (e.g., aorta), or other types of tissue where damage should be avoided. . Safety module 103 may be configured to leave critical structures proximal to the treatment site. The disclosed safety module 103 enforces the safety boundary 160 through continuous motion within the ultrasound image 90 due to natural motion (e.g., breathing and heartbeat as examples of periodic changes 130) or motion of the ultrasound transducer. Note that it is configured to track 142 .

8A and 8B for tracking a treatment region within an enhanced ultrasound image 185 demonstrating maintenance of registration 140 and tracking 142 through movement within the image 185, according to some embodiments of the present invention. provides a non-limiting example of In the figure, the treatment area is circled and shown schematically as it moves back and forth without losing alignment in a swing corresponding to the patient's respiratory rate (Fig. 8A). FIG. 8B schematically demonstrates the temporary loss of tracking and alignment due to strong patient motion and the rapid recovery of alignment and tracking. The figures demonstrate the stability and reliability of the disclosed registration and tracking procedures.

Simulation and planning module 106 converts pre-imaging data, such as 3D data, for example from CT or MRI 86, into ultrasound-related data, for example, to provide demarcation of tissue to be ablated. (see comparison of injured tissue and tumors in the examples below).

The simulation and planning module 106 proposes operating parameters (eg, intensity, duration, number, timing, etc. of ablation locations) and/or spatial configurations (eg, location and orientation) to the ablation tool 80 for subsequent ablation procedures. may be configured to plan and/or simulate the The simulation and planning module 106 utilizes pre-imaging, such as 3D CT or MRI images (eg, from CT system 86; see also FIGS. 1C, 8A, and 8B, and experimental data below). may The simulation and planning module 106 may utilize ultrasound data collected from previous or experimental procedures on tissue architecture that correlates the type of thermal ablation with the resulting tissue damage.

Data collected during a thermal ablation procedure may be used to provide that analysis, eg, via ablation analysis module 104 . This data may be further accumulated and processed to provide simulation and planning of ablation procedures, eg, via simulation and planning module 106 .

In certain embodiments, the simulation and planning module 106 uses tissue cooling models 116 derived from various sources such as the ablation analysis module 104 and any of the previous procedures, preliminary CT or MRI images 86, thermal and flow models, and the like. may contain. Tissue cooling model 116 relates tissue structure, such as the number and spatial configuration of blood vessels in the vicinity of the tissue to be treated, to the distribution of heat applied by ablation tool 80 in specified spatial configurations at specified operating parameters. good too.

In certain embodiments, the analysis of preliminary CT images includes diagonal vessels that are not imaged in a single ultrasound (or CT) plane but contribute to tissue cooling and therefore the extent and shape of the damaged tissue region. A spatial configuration of vessels surrounding the tissue to be ablated may be provided, including: Diagonal vessels may be highlighted and used to plan the ablation procedure.

In certain embodiments, an automated learning procedure may be implemented to enhance tissue cooling model 116 and/or enhance pre-imaging analysis. For example, the simulation and planning module 106 is configured to predict the spatial configuration of the vessel, assess the cooling effect of the vessel, and provide an estimate of the predicted bubble movement once the ablation procedure is initiated. good too.

Accordingly, given the following procedure, the simulation and planning module 106 may be configured to utilize the tissue cooling model 116 to predict tissue damage under certain conditions and optimize the ablation procedure. good. The ablation monitoring module 102 is further configured to both adjust ablation parameters during the ablation procedure and update the tissue cooling model 116 as needed (e.g., cooling effects and predicted bubble movements). ) may be configured to monitor the ablation procedure with respect to predictions from the tissue cooling model 116 .

FIG. 1E is a high-level block diagram of an exemplary computing device 109 that may be used with embodiments of the invention. Computing device 109 may be, for example, one or more central processing unit processors (CPUs), one or more graphics processing units (GPUs or general purpose GPUs - GPGPUs), chips or any suitable computing or computing device. It may include a controller or processor 63 , an operating system 61 , memory 62 , storage 65 , input devices 66 and output devices 67 which may be present or include them. Ultrasound image-guided system 100 and/or any of its modules such as ultrasound registration module 101, ablation monitoring module 102, safety module 103, ablation analysis module 104, image quality assessment module 105 and/or planning module 106 , for example, may include at least a portion of the computer system shown in FIG. 1E.

Operating system 61 is any code designed and/or configured to perform tasks including coordinating, scheduling, arbitrating, overseeing, controlling, or otherwise managing the operation of computing device 109, such as scheduling the execution of programs. It may be or contain a segment. Memory 62 includes, for example, random access memory (RAM), read only memory (ROM), dynamic RAM (DRAM), synchronous DRAM (SD-RAM), double data rate (DDR) memory chips, flash memory, volatile memory, It may be or include non-volatile memory, cache memory, buffers, short-term memory units, long-term memory units, or other suitable memory or storage units. The memory 62 may be or include multiple, possibly different memory units. Memory 62 may store, for example, instructions for performing methods (eg, code 64) and/or data such as user responses, interruptions, and the like.

Executable code 64 may be any executable code, such as an application, program, process, task, or script. Executable code 64 may be executed by controller 63 , possibly under control of operating system 61 . For example, executable code 64, when executed, may cause computer code generation or compilation, or application execution, such as VR execution or inference, in accordance with embodiments of the present invention. Executable code 64 may be code generated by the methods described herein. One or more computing devices 109 or components of computing devices 109 may be used for the various modules and functions described herein. Devices that include components similar or different to those included in computing device 109 may be used and may be connected to a network and used as a system. One or more processors 63 may be configured to implement embodiments of the present invention, for example, by executing software or code.

Storage 65 may be, for example, a hard disk drive, a floppy disk drive, a compact disk (CD) drive, a CD recordable (CD-R) drive, a universal serial bus (USB) device, or other suitable removable storage unit. and/or may be or include a permanent storage unit. Data such as instructions, code, VR model data, parameters, etc. may be stored in storage 65 and loaded from storage 65 into memory 62 where they may be processed by controller 63 . In some embodiments, some of the components shown in FIG. 1E may be omitted.

Input device 66 may be or include, for example, a mouse, keyboard, touch screen or touch pad, or any suitable input device. It will be appreciated that any suitable number of input devices may be operatively connected to computing device 109 as indicated by block 66 . Output devices 67 may include one or more displays, speakers, and/or any other suitable output device. It will be appreciated that any suitable number of output devices may be operatively connected to computing device 109 as indicated by block 67 . Any applicable input/output (I/O) device may be connected to computing device 109, such as a wired or wireless network interface card (NIC), modem, printer or facsimile machine, universal serial bus ( USB) devices or external hard drives may be included in input devices 66 and/or output devices 67 .

Embodiments of the present invention comprise one or more articles (e.g., memory 62 or storage 65), such as non-transitory computer or processor readable media, or non-transitory computer or processor storage media such as memory, disk drives, or USB flash memory.

FIG. 9A is a high level schematic diagram of an ultrasound display 195 according to some embodiments of the present invention. The ultrasound display 195 may be associated with the thermal ablation unit 80 and the ultrasound system 85 within the ultrasound image-guided system 100 to provide a repeatedly updated B-mode ultrasound image 185 and a repeatedly updated B-mode ultrasound image 185 . and at least one representation 197 of thermally damaged tissue registered on the ultrasound image 185 . Non-limiting examples of indicia 197 that may be combined and/or substituted include heat-damaged tissue boundaries 197A and/or colored areas 197B indicating heat-damaged tissue (see, e.g., FIG. 9A), which are: It may be updated repeatedly as the corresponding thermal ablation procedure progresses. Alternatively or complementary, display 197 may indicate bubble formation (Phase 1), bubble development (Phase 2) where both bubble formation and bubble evacuation by the vessels occur, and bubble saturation (Phase 3) (e.g., in the diagram above). 7A and 7B), which may be repeatedly updated as the corresponding thermal ablation procedure progresses. In certain embodiments, ultrasound images 185 may be displayed in grayscale, with displays 197, such as stage 1, 2, and 3 displays 197C, for each of the monitored bubbles in successive ultrasound images. Stages 1, 2, and 3 may be displayed in different colors defined for each pixel according to the accumulation rate (accumulation rate) of . Clearly, color may reflect a continuous parameter corresponding to, for example, bubble accumulation rate, bubble mobility, or any other parameter used to quantify extent of tissue damage at each pixel. and/or may be modified and/or discretized to reflect any defined damage threshold.

In certain embodiments, the ultrasound display 195 may be further configured to indicate safety boundaries for corresponding thermal ablation procedures, eg, according to the safety module 103 . In certain embodiments, the ultrasound display 195 further includes a medical image derived from periodic changes 130 in successive ultrasound images 90, such as the patient's respiratory rate and/or heart rate, for example, according to the ultrasound registration module 101. It may be configured to indicate parameters 135 . In certain embodiments, ultrasound display 195 may be configured to display repeatedly updated B-mode ultrasound image 185 and heat-damaged tissue representation 197 in two mutually orthogonal planes.

The ultrasound image-guided system 100 may also be associated with a CT system 86 used to derive preliminary images of the tissue to be treated. The ultrasound display 195 may further be configured to associate the displayed representation(s) 197 with the CT-based image(s) of the tissue. FIG. 9B is a high-level schematic of a CT-based image used to plan an ablation procedure, according to some embodiments of the present invention; A 3D anatomical model for a liver tumor ablation procedure is shown, showing the positioning of an ablation tool 80 and a simulated ablation volume 82. FIG. In certain embodiments, at least a portion of the planning may be performed by the simulation and planning module 106 on ultrasound images, eg, with respect to the tissue cooling model 116 (see FIGS. 1A and 1C). At least a portion of the plan may be performed and/or displayed using ultrasound display 195 which may be associated with simulation and planning module 106 .

FIG. 10 is a high-level flowchart illustrating a method 200 according to some embodiments of the invention. The steps of this method may be performed with respect to the ultrasound image-guided system 100 and/or any of its modules described above, which may optionally be configured to implement the method 200 . Method 200 may be implemented, at least in part, by at least one computer processor, eg, in an ultrasound system or related system. Certain embodiments include a computer program product including a computer-readable storage medium (eg, see FIG. 1E) embodied with a computer-readable program and configured to perform the relevant steps of method 200. Method 200 may include the following steps, in any order.

The method 200 includes calculating and monitoring at least one correlation value between successive ultrasound images (step 220) and identifying a plurality of keyframes from the calculated at least one correlation value (step 222). ), identifying at least one periodic change in successive ultrasound images from the values of the calculated at least one correlation value (step 224); performing ultrasound image registration (step 230) by correlating pixels in successive keyframes with ) to perform ultrasound image registration (step 201). Method 200 may include using ultrasound images in two mutually orthogonal planes or 3D (step 205).

Correlation value(s) or equivalent measures can be calculated in a variety of ways, e.g., sum of absolute differences (SAD), sum of squared differences (SSD), normalized cross-correlation (NCC), kernel-based tracking algorithms, normalized gradient field , 2D motion tracking using dense optical flow, Bayesian algorithms, motion estimation frameworks, etc. may be computed.

In certain embodiments, method 200 may include deriving a medical parameter, such as respiratory rate and/or heart rate, from the identified periodic change(s) (step 210). The method 200 may further include detecting out-of-plane frames by detecting a decrease in the monitored correlation value(s) greater than a specified threshold (step 212). The method 200 may include evaluating image quality of successive ultrasound images and removing low quality images (step 214). In certain embodiments, evaluation 214 automatically tags images with respect to previously received images, possibly forming an automatically tagged data bank, and uses machine learning, for example. may be performed by evaluating the quality of the incoming image with respect to the tagged image.

Alternatively or complementary, method 200 includes performing registration 203 by identifying at least one keyframe from a received ultrasound image (step 232); applying at least two registration algorithms to the obtained ultrasound images to obtain at least two corresponding confidence scores (step 234); registering the ultrasound images (step 236). Alternatively or complementary, method 200 may include deriving an aggregated confidence score (stage 238) by performing a voting procedure on the results of at least two registration algorithms.

The method 200 identifies the onset of ablation by changes in the monitored correlation value(s) that occur within a specified duration and above a specified threshold (step 250); identifying bubbles formed by the ablation in the image (step 260) and/or in the registered ultrasound images according to at least the identified keyframes and/or the identified periodic change(s) and monitoring thermal ablation using the B-mode ultrasound image (step 240), for example, by detecting bubble movement in the B-mode ultrasound image (step 270).

The method 200 may further include defining a thermally damaged tissue region in relation to and to the exclusion of said detected bubble motion dynamically and in real-time with respect to incoming sequential ultrasound images (step 280). The method 200 may include defining an intact vessel with respect to the detected bubble motion dynamically and in real-time with respect to incoming sequential ultrasound images (step 282). In certain embodiments, method 200 may include correlating the defined vessel with the vessel identified in the enhanced preliminary CT image (step 285). The method 200 further includes, dynamically and in real-time with respect to incoming ultrasound images, identifying damaged vessels according to the detected cessation of bubble motion (step 290); and adding blood vessels to the defined heat-damaged tissue region (step 292).

In certain embodiments, the method 200 further includes defining a shaded region in successive ultrasound images for the identified bubble (step 300) and determining and/or defining a distal boundary of the thermally damaged tissue region. and enhancing the shadowed areas (step 305) so as to do so.

In certain embodiments, method 200 further includes, in real-time with respect to incoming ultrasound images, detecting a safe boundary for an ablation procedure (step 310); tracking the safe boundary with respect to change(s) (step 312) and issuing a warning if the ablation approaches the tracked safe boundary (step 315).

The disclosed systems and methods were validated in an experimental setting comparing the defined damaged tissue 180 to enhanced CT (CECT) images taken 24 hours after thermal RF ablation of 20 liver tumors in 19 patients. . A parametric coagulation map 181 was derived according to Equation 1 from ultrasound images 90 acquired during an 80 second treatment with an interframe period of 33 milliseconds. A parametric coagulation map 181 was then used to derive the defined damaged tissue 180 .

11A-11C are Bland-Altman plots presenting the level of similarity between the compared damaged tissue areas, demarcated damaged tissue 180, and comparative experimental images according to some embodiments of the present invention; and provide respective illustrations of damaged tissue measured by CECT 24 hours after ablation measured in the prior art. Comparisons are made as true positives (TP) for the overlap of the defined damaged tissue 180 and the damaged tissue measured by CECT 75 as false positives (FP ) and as a false negative (FN) for damaged tissue measured by CECT 75 extending beyond the defined damaged tissue 180 using the three metrics defined in Equation 2 for the area shown in FIG. executed. True negatives (TN) represent the tissue area outside both the defined damaged tissue 180 and the damaged tissue measured by CECT75.
Sorensen-Dice (Sφrensen-Dice) coefficient = 2TP/(FN + 2TP + FP)
Sensitivity metric = TP/(TP + FN)
Accuracy metric = TP/(TP+FP)
formula 2

FIG. 11B shows total lesion area and 24% of ablation assessed by damaged tissue 180 defined using a Bland-Altman plot (comparing the difference between the areas provided by the two measurement methods to their mean). The mean difference between damaged tissue measured by CECT after hours was μ with standard deviation σ = 0.2 cm2 compared to the mean lesion ^size measured by both methods of 5.03 ± 0.2 ^cm2 . = 0.02 cm ² (95% of measurements are within ±2σ indicated). Accordingly, the disclosed methods and systems provide highly accurate results that correspond very well to prior art post-ablation measurements.

FIG. 11C presents a comparative experiment image comparing the defined damaged tissue 180 (as filled red area) with the actual damaged tissue imaged by CECT 24 hours after treatment (blue area overlapping red area). Note that the lesion areas align not only when the lesions deviate significantly from the theoretical elliptical shape (see schematic in FIG. 1A for continuously heated region 82). Such shapes occur, for example, near relatively large blood vessels that are not damaged during thermal ablation and therefore continue to cool the surrounding tissue, as can be seen from the dynamic representation of bubble motion 170 .

The disclosed procedures, including CT data transfer to system 100, correlation and registration of 3D CT data with 2D ultrasound images 90, ultrasound registration 140, and calculation, analysis, and demarcation of damaged tissue 180 are: Validated in tests performed at the Fraunhofer Institute for Digital Medicine MEVIS.

Aspects of the present invention are described above with reference to flowchart illustrations and/or partial illustrations of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each portion of the flowchart illustrations and/or sub-charts, and combinations of parts within the flowchart illustrations and/or sub-charts, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the instructions executed via the processor of the computer or other programmable data processing apparatus may be used as flowcharts and/or instructions. A machine may be constructed to generate a partial view or means for performing the functions/acts specified in that portion.

These computer program instructions may be stored on a computer-readable medium capable of directing a computer, other programmable data processing apparatus, or other device to function in a specified manner, thereby causing the computer-readable medium to The instructions stored in the flowcharts and/or partial diagrams, or articles of manufacture that include instructions for performing the functions/acts specified in that portion.

The computer program instructions are loaded onto a computer, other programmable data processing apparatus, or other device such that a series of operational steps are performed on the computer, other programmable apparatus, or other device, A computer-implemented process may be generated whereby instructions executing on a computer or other programmable data processing apparatus perform the functions/acts specified in the flowchart illustrations and/or sub-diagrams or portions thereof. Provide a process.

The foregoing flowcharts and diagrams illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each portion in a flowchart or sub-diagram may represent a module, segment, or portion of code, which consists of one or more executable instructions for implementing the specified logical function(s). including. It should also be noted that, in some alternative implementations, the functions noted in the section may occur out of the order noted in the figures. For example, two portions shown in succession may, in fact, be executed substantially concurrently or the portions may sometimes be executed in the reverse order, depending on the functionality involved. good. Each portion of the partial diagrams and/or flowchart illustrations, and combinations of portions of the partial diagrams and/or flowchart illustrations, may represent a dedicated hardware-based system, or a combination of dedicated hardware and computer instructions, that performs the specified functions or acts. Note also that it can be implemented by combination.

In the above description, an embodiment is an example or implementation of the present invention. The various appearances of "one embodiment," "one embodiment," "a particular embodiment," or "some embodiments" do not necessarily all refer to the same embodiment. Although various features of the invention may be described in the context of a single embodiment, those features may be provided separately or in any suitable combination. Conversely, although the invention may be described herein in the context of separate embodiments for clarity, the invention may be implemented in a single embodiment. Certain embodiments of the invention may include features from different embodiments disclosed above, and certain embodiments may incorporate elements from other embodiments disclosed above. Disclosure of elements of the invention in the context of a particular embodiment should not be construed as limiting their use only in that particular embodiment. Furthermore, it is to be understood that the invention may be practiced or carried out in various ways and that the invention may be practiced in specific embodiments other than those outlined in the foregoing description.

The invention is not limited to these figures or the corresponding description. For example, flow need not move through each illustrated box or state, or in exactly the same order as illustrated and described. The meanings of technical and scientific terms used herein are as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise defined. While the invention has been described with respect to a limited number of embodiments, these should not be construed as limitations on the scope of the invention, but rather as exemplifications of some of the preferred embodiments. is. Other possible variations, modifications, and applications are also within the scope of the invention. Accordingly, the scope of the present invention should be limited not by what has been described, but by the attached utility model claims and their legal equivalents.

Claims (25)

An ultrasound display associated with a thermal ablation unit, comprising:
a repeatedly updated B-mode ultrasound image;
and at least one representation of thermally damaged tissue registered with said repeatedly updated B-mode ultrasound image. The ultrasound display of Claim 1, wherein said at least one display includes a boundary of said thermally damaged tissue, said boundary being repeatedly updated as the corresponding thermal ablation procedure progresses. 3. The ultrasound display of Claim 1 or Claim 2, wherein the at least one display includes a colored area indicative of the thermally damaged tissue, the colored area being repeatedly updated as the corresponding thermal ablation procedure progresses. wherein said at least one indication comprises an indication of bubble formation (Phase 1), bubble development with both bubble formation and vascular bubble evacuation (Phase 2), and bubble saturation (Phase 3), said indications corresponding 4. The ultrasound display of any one of claims 1-3, which is repeatedly updated as the thermal ablation procedure progresses. 5. The ultrasound display of claim 4, wherein the ultrasound image is displayed in grayscale and the phase 1, 2 and 3 representations are displayed in different colors. 6. The ultrasound display of claim 4 or claim 5, wherein said phases 1, 2 and 3 are defined pixel by pixel according to respective accumulation rates of monitored bubbles in successive ultrasound images. 7. The ultrasound display of any one of claims 1-6, further configured to indicate a safety boundary for a corresponding thermal ablation procedure. 8. The apparatus of any one of claims 1 to 7, further configured to indicate a medical parameter derived from periodic variations in successive ultrasound images, said medical parameter comprising respiratory rate and/or heart rate. Ultrasound display as described. 9. The apparatus of any one of claims 1-8, further configured to display the repeatedly updated B-mode ultrasound image and at least one representation of the thermally damaged tissue in two mutually orthogonal planes. Ultrasound display as described. 10. The ultrasound display of any one of claims 1-9, further configured to associate the displayed at least one representation with a CT-based display of tissue to be treated. 1. A computer program product comprising a computer readable storage medium embodying a computer readable program and configured to perform ultrasound image registration for a plurality of consecutive ultrasound images, the computer readable program comprising:
a computer readable program configured to correlate the successive ultrasound images to obtain a correlation value;
a computer readable program configured to identify a plurality of keyframes from the correlation values;
a computer readable program configured to identify at least one periodic change in the successive ultrasound images from the correlation value;
and a computer readable program configured to perform said ultrasound image registration by correlating pixels in successive keyframes having the same phase for said at least one identified periodic change. product. An ultrasonic alignment module comprising:
correlating successive ultrasound images received to obtain a correlation value;
identifying a plurality of keyframes from the values of the correlation value;
identifying at least one periodic change in the consecutive ultrasound images from the correlation values; and correlating pixels in consecutive keyframes having the same phase for the identified at least one periodic change. An ultrasound registration module configured to perform ultrasound image registration by: 13. The ultrasonic registration module of claim 12, further comprising an ultrasonic imaging module, wherein the ultrasonic registration module is tailored to the type of the ultrasonic imaging module. 14. The ultrasound registration module of claim 12 or claim 13, further configured to derive a medical parameter including respiratory rate and/or heart rate from the identified at least one periodic change. 15. The ultrasonic registration module of any one of claims 12-14, further configured to detect an out-of-plane frame by a decrease in the correlation value above a specified threshold. An ultrasound image-guided system for thermal ablation comprising:
16. The ultrasound registration module of any one of claims 12-15 configured to receive a plurality of consecutive ultrasound images;
an ablation monitoring module configured to identify the onset of ablation by a change in the correlation value calculated by the ultrasound registration module occurring within a specified duration and above a specified threshold;
an ultrasound image-guided system comprising: 10. The ablation monitoring module is further configured to identify bubbles formed by ablation in the registered ultrasound images and to detect bubble movement in the registered ultrasound images. 17. The ultrasound image-guided system of 16. The ablation monitoring module detects movement of the bubble on a B-mode ultrasound image according to at least the keyframes and/or the at least one periodic change identified by the ultrasound registration module and generates an incoming sequence. 18. The ultrasound image-guided system of claim 17, further configured to dynamically and in real-time with respect to the ultrasound image to define a thermally damaged tissue region with respect to and to exclude the detected bubble motion. . The ablation monitoring module defines a shaded area in the successive ultrasound images for the identified bubble, enhances the shaded area to determine a distal boundary of a thermally damaged tissue area, and 19. Further configured to define a shaded region in the successive ultrasound images with respect to and enhance the shaded region to define a distal boundary of the thermally damaged tissue region. The ultrasound image-guided system of paragraph 1. detecting the safe boundary of the ablation procedure in real-time with respect to the incoming sequential ultrasound images;
tracking the safety boundary with respect to the at least one periodic change in the successive ultrasound images identified by the ultrasound registration module;
20. The superstructure of any one of claims 16-19, further comprising a safety module configured to issue a warning if the ablation monitored by the ablation monitoring module approaches the tracked safety boundary. Acoustic image-guided system. An ultrasound image-guided system for thermal ablation comprising:
receiving a plurality of consecutive ultrasound images;
identifying at least one keyframe from the received ultrasound image;
applying at least two registration algorithms to the ultrasound images received for the at least one keyframe to obtain at least two corresponding confidence scores;
an ultrasound registration module configured to align the received ultrasound images for which the at least two confidence scores exceed a specified threshold;
identifying the onset of ablation in the registered images;
identifying bubbles formed by ablation in the registered ultrasound images and detecting bubble movement in the registered ultrasound images;
an ablation monitoring module configured to display an ablation region on the registered ultrasound image. 22. The ultrasound image of Claim 21, wherein said ultrasound registration module is further configured to derive an aggregated confidence score by performing a voting procedure on the results of said at least two registration algorithms. inductive system. 23. The ultrasonic image-guided system of Claim 21 or Claim 22, wherein said display comprises boundaries of thermally damaged tissue, said boundaries being repeatedly updated as the corresponding thermal ablation procedure progresses. 24. The ultrasonic image-guided ultrasound of any one of claims 21-23, wherein the display comprises a colored area indicative of thermally damaged tissue, the colored area being repeatedly updated as the corresponding thermal ablation procedure progresses. type system. wherein said indication includes indication of bubble formation (Phase 1), bubble development (Phase 2) with both bubble formation and vascular bubble evacuation, and bubble saturation (Phase 3), said indication includes corresponding thermal ablation procedures; 25. The ultrasound image-guided system of any one of claims 21-24, wherein is repeatedly updated as .

Images