NL2029877B1 - Image based analysis of a vessel structure - Google Patents

Abstract

A method, system, and computer-readable medium for image based analysis of a vessel structure (V). A sequence of images (I[t]) is received with pixels values (D) representing the vessel structure (V) at a respective instance of time (t) following injection by a contrast fluid (Fc) flowing through the vessel structure (V). A respective vessel mask (V m) outlines the vessel structure (V) based on respective pixel values (D). The respective vessel mask (V m) is divided in a set of mask segments (V s[n]) coinciding with different sections of the vessel structure (V). A respective distribution of pixel values (D) of pixels (P) in the respective image (I) intersecting the respective mask segment (Vs) is used to determine a respective segment value (S) characterizing the distribution at the respective instance of time (t) of the respective image (1). Based on the respective set of segment values (S[t]) as function of time (t), a respective flow metric (Q) is determined representing the contrast fluid (Fc) flowing through a respective section of the vessel structure (V) corresponding to the respective mask segment (Vs). One or more flow metrics can be used to generate a parametric image (Iq). The one or more flow metrics and/or parametric image can be used as input to improve accuracy and/or computational efficiency of a model for classifying and/or localizing a segment of interest, in particular stenosis.

Description

Title: IMAGE BASED ANALYSIS OF A VESSEL STRUCTURE

TECHNICAL FIELD AND BACKGROUND

The present disclosure relates to computer implemented methods, systems, and computer readable media for image based analysis of a vessel structure, such as the classification and/or localization of stenosis.

As background, US 9,891,044 B2 describes a method and device for determining deviation in pressure in a blood vessel. Deviations of blood pressure due to stenosis caused by plaque pose a health risk. The deviation, often expressed in a fractional flow reserve, may be calculated on a per- location basis using deviations of the local cross-sectional area or local diameter from a reference value representing a healthy vessel.

As further background, US 9,974,508 B2 describes non-invasive systems and methods for determining fractional flow reserve within a luminal organ by positioning a monitoring device external to a luminal organ and near a stenosis, the monitoring device capable of determining at least one characteristic of the stenosis, operating the monitoring device to determine the at least one characteristic of the stenosis, and determining fractional flow reserve at or near the stenosis based upon the at least one characteristic determined by the monitoring device.

As further background, US 9,247,918 B2 describes computing hemodynamic quantities by acquiring angiography data from a patient; calculating a flow and/or calculating a change in pressure in a blood vessel of the patient based on the angiography data; and computing the hemodynamic quantity based on the flow and/or the change in pressure.

As further background, EP 2 946 321 B1 describes a method and system of vascular assessment during a procedure of vascular catheterization of a subject. A computer is communicatively connected and configured to receive, from an angiographic imaging device, a plurality of 2D images captured during the procedure while the subject is catheterized, the plurality of 2D images showing a portion of a vasculature of the subject comprising a stenosis. The computer is further configured to calculate a vascular tree model from the plurality of 2D images while the subject remains in the procedure and catheterized, the vascular tree model based on estimated dimensions of the portion of the vasculature determined from the 2D images. The computer is also configured to determine an index of vascular function which indicates a capacity for restoration of flow by opening of the stenosis based on said vascular tree model, and also while the subject remains in the procedure and catheterized.

There remains a need for further improvements in the analysis of a vessel structure, such as improvements in accuracy and efficiency.

SUMMARY

Aspects of the present disclosure relate to the analysis of a vessel structure, in particular based on a sequence of images showing a progressing flow of contrast fluid through the vessel structure. As will be appreciated, basing the analysis on the manner in which the contrast fluid flows through the vessel structure can reveal dynamic information and characteristics of the vessel structure that are difficult or impossible to determine from any single image. However, the analysis of such sequence is not straightforward, and this has motivated the inventors to invent a set of advantageous analysis techniques that are proven to be particularly suited for the present task. By isolating the vessel structure from the background using a vessel mask, the analysis of the flow can be focused on the most relevant data. By dividing the vessel mask in a set of mask segments, the analysis can differentiate information about different sections of the vessel structure, which can lead to relevant classification and localization of relevant features. By using a distribution of multiple pixel values for each segment and each image, statistically relevant information can be derived, which is e.g. less affected by noise and overall movement of the subject. The characteristics of each distribution can be condensed into a respective segment value. The segment values as function of time can be further condensed to produce a respective flow metric. So a singular value can be derived to characterize an aspect of the flow through a respective section of the vessel structure corresponding to the respective mask segment. For example, the flow metrics of different segments can be used to generate a parametric image.

The flow metrics and/or parametric image can be used in various ways to improve the analysis of the contrast fluid flowing through the vessel structure as function of time. In some embodiments, a system having a processor configured to perform operational acts in accordance with the present methods comprises a display device, configured to display a representation of the parametric image. For example, the inventors find that the (single) displayed parametric image can reveal information about the flow of contrast fluid, and consequently the vessel structure, which is practically impossible to observe in the original sequence of images.

Furthermore, when the parametric image is overlaid with one of the original images, this may further improve analysis of the vessel in its surrounding context, including the background.

The flow metrics and/or parametric image can also be used input into a computer model. For example, the inventors find that use of the flow metrics and/or parametric image as input to the computer model can improve accuracy, reduce computational complexity, and/or save overall computing resources such as processor time and/or memory. For example, the flow metrics and/or parametric image can be used as input instead of the original sequence of images, which sequence can consist of many images, each with many pixel values. It is recognized that most of these original pixels values may have minimal relevance to analysis of the vessel structure, e.g. background, parts of the vessel which have not been reached by the contrast fluid, and subtle variations which may or may not be relevant to determine the model output. By using the flow metrics or parametric image, optionally in combination with one or more original images, the relevant aspects can be more efficiently determined in a respective model, with less consideration of superfluous or irrelevant data.

Also the model, in principle, does not need to be aware of the specific sequence of events, which can greatly simplify the model, e.g. analysis algorithm.

BRIEF DESCRIPTION OF DRAWINGS

These and other features, aspects, and advantages of the apparatus, systems and methods of the present disclosure will become better understood from the following description, appended claims, and accompanying drawing wherein:

FIG 1 illustrates a set of images showing a vessel structure at different times following injection by a contrast fluid;

FIG 2 illustrates corresponding images of vessel mask wherein the vessel structure has been isolated from the background;

FIGs 3A and 3B illustrates an image wherein the vessel mask 1s filled by a gradient as function of distance to an injection location to determine a set of mask segments;

FIGs 4A-4C illustrate distributions of pixel values in respective images intersecting a respective mask segment;

FIGs 5A-5C illustrate determining a set of segment values as function of time for the respective mask segment based on distributions of pixel values;

FIGs 6A-6C illustrate determining a respective flow metric for a respective mask segment based on the respective set of segment values;

FIG 7A illustrates a plot of a first flow metric as function of segment number;

FIG 7B illustrates the first flow metric applied as pixel value of respective vessel segments;

FIG 7C illustrates an overlay of FIG 7B onto one of the images;

FIGs 8A-8C are similar to FIGs 7A-7C but using a different, second flow metric;

FIGs 9A-9C are again similar but using a different, third flow metric; 5 FIG 10 illustrates combining different flow metrics to improve analysis in identifying a segment of interest having a combination of characteristics;

FIG 11A illustrate breaking down a segment of interest according to different branch segments and determining segment values for each branch segment;

FIG 11B illustrates a fourth flow metric applied as pixel value of respective branch segments to determine a branch segment of interest;

FIG 11C illustrates an overlay of FIG 11B onto one of the images;

FIG 12A illustrates validation results for classification of stenosis;

FIG 12B illustrates validation results for localization of a respective vessel segment containing stenosis.

DESCRIPTION OF EMBODIMENTS

Terminology used for describing particular embodiments is not intended to be limiting of the invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. The term "and/or" includes any and all combinations of one or more of the associated listed items. It will be understood that the terms "comprises" and/or "comprising" specify the presence of stated features but do not preclude the presence or addition of one or more other features. It will be further understood that when a particular step of a method is referred to as subsequent to another step, it can directly follow said other step or one or more intermediate steps may be carried out before carrying out the particular step, unless specified otherwise. Likewise it will be understood that when a connection between structures or components is described, this connection may be established directly or through intermediate structures or components unless specified otherwise.

The invention is described more fully hereinafter with reference to the accompanying drawings, in which embodiments of the invention are shown. In the drawings, the absolute and relative sizes of systems, components, layers, and regions may be exaggerated for clarity.

Embodiments may be described with reference to schematic and/or cross- section illustrations of possibly idealized embodiments and intermediate structures of the invention. In the description and drawings, like numbers refer to like elements throughout. Relative terms as well as derivatives thereof should be construed to refer to the orientation as then described or as shown in the drawing under discussion. These relative terms are for convenience of description and do not require that the system be constructed or operated in a particular orientation unless stated otherwise.

FIG 1 illustrates a sequence of images I[t] that can be used in the methods and system for image based analysis of a vessel structure “V”, as described herein. Optionally, the images are preprocessed by contrast enhancement and/or noise suppression.

In some embodiments, each image represents the vessel structure “V” at a respective instance of time “t” following injection by a contrast fluid

Fe flowing through the vessel structure “V”. Typically, a respective image “I” in the sequence of images I[t] comprises a set of pixels “P” having pixels values “D”, e.g. representative of brightness/darkness. For example, the pixel values may indicate the extent of the vessel structure and/or contrast fluid as it flows through the vessel structure. Typically, the vessel structure “V” can be distinguished from the background B by comparing the pixel values “D”.

In some embodiments, different images in the sequence show a different progression of the contrast fluid, e.g. with the contrast fluid having just been injected or passed an injection location Ci in the first image I[1] and the contrast fluid having progressed though the vessel structure (or at least the relevant part thereof) in the last image I[21]. Some embodiments comprise determining an injection location Ci of the contrast fluid Fc entering the vessel structure “V”. For example, the injection location Ci can be identified as an origin of the contrast fluid. In one embodiment, the sequence comprises at least one image per taken per second, preferably, at least two images per second, more preferably at least five images per second, most preferably at least ten images per second, or more. In another or further embodiment, the present methods may use a subselection of the taken or received images. Preferably, the selected images at least have a fixed time interval there between to derive consistent results, e.g. regarding fluctuation, although it can also be envisaged to use a variable time interval, e.g. with more images per unit of time to focus on a period of interest.

In some embodiments, the sequence of images I[t] is received by a processor, e.g. as part of a computer configured to implement the methods described herein. In other or further embodiments, the methods described herein are stored as (non-transitory) computer-readable instructions to be executed by the processor. In one embodiment, the sequence of images I[t] is received from a (medical) imaging device communicatively coupled to the processor. For example, the imaging device comprises an X-ray imaging device or any other imaging device capable of taking the sequence of images.

Alternatively to receiving the images directly from an imaging device, the sequence of images I[t] can also be received from a memory, e.g. previously stored images taken by an imaging device.

FIG 2 illustrates images of a vessel mask Vm corresponding to the images of the vessel structure in FIG 1.

Some embodiments comprise determining a respective vessel mask Vm outlining the vessel structure “V” based on the pixel values “D” in a respective one or more images. For example, the vessel mask can be individually generated for each respective image “I” in the sequence of images I[t], or a common vessel mask can be generated for the whole sequence. The function of the vessel mask may include selection of the vessel structure and/or removal of the background. For example, the vessel mask is formed by a pixel area coinciding with pixels of the vessel structure “V”.

In some embodiments, the set of pixels “P” in an image includes pixels of the vessel structure “V”, whose pixels values “D” are affected by the contrast fluid Fc following the injection, and pixels of a surrounding background B, substantially unaffected by the contrast fluid Fc. In one embodiment, the vessel mask Vm isolates the vessel structure “V” from the surrounding background B in the respective image “I”. In one embodiment, the vessel mask is determined by isolating a mask of pixels representing the vessel structure with possible margin. In another or further embodiment, the vessel mask Vm is configured to outline one or more areas of (contiguous) pixels coinciding with the vessel structure “V”. Typically, the vessel mask 1s determined based on the pixel values and/or contrast between pixel values in the respective image. For example, the contrast can be based on relative pixel value and/or gradient of pixels values.

FIGs 3A and 3B illustrate dividing the respective vessel mask Vm in a set of mask segments Vs[n] coinciding with different sections of the vessel structure. Preferably, each mask segment covers an exclusive part of the vessel mask. In this way, different parts can be efficiently represented by respective mask segments. Alternatively, some mask segments may be partially overlapping. Preferably, the mask segments are contiguous, e.g. covering the whole vessel mask. Alternatively, there may be gaps between adjacent mask segments.

In some embodiments, e.g. as shown, the vessel mask Vm is divided into respective mask segments Vs[n] in accordance with a respective (pixel) distance “d” to a specific location, e.g. (origin) point, line, or area.

Preferably a distance is determined to the location Ci where the contrast fluid first enters the vessel structure “V”. As will be appreciated it can be expected that the fluid will spread through the vessel structure from this origin. So by selecting the division between mask segments relative to this location, the flow can be most efficiently tracked, even through branching vessels (at least in good approximation).

In some embodiments, the vessel mask Vm is divided into respective mask segments Vs[n] according to a predetermined step size (ds) in distance “d” from the specific location Ci. Preferably, a fixed step in distance 1s used. For example, using a step size ds=10 unit (e.g. pixels), the first mask segment Vs[1] covers a part of the vessel structure between the distance of 0 and 10 units from the origin, the second mask segment Vs[2] between 10 and 20 units, the third mask segment Vs[3] between 20 and 30 units, and so on. Of course also other step sizes can be used. It can also be envisaged to use a variable step size, e.g. increasing or decreasing the step size dependent on (average or local) vessel diameter to take into account expected flow rate. Lowering the step size increases the number of mask segments which may improve localization of a segment of interest, but also decreases the number of pixels per segment which may increase contributions of noise and decrease computational efficiency.

In one embodiment, e.g. as shown in FIG 3A, the distance “d” 1s a measured as a radial distance from a point, e.g. calculated as d2=Ax2+Ay? where Ax and Ay are the respective pixel distances to the location Ci. In another or further embodiment, e.g. as shown in FIG 3B, the distance “d” is a rectilinear distance e.g. calculated as d = Ax+Ay. Also other distance measures can be used, such as a preferred embodiment (mot shown) wherein the distance along a respective vessel path is used in dividing the mask segments. For example, this distance can be measured for each vessel branch separately. The images show how the vessel mask can be filled by a gradient pattern and subdivided accordingly. While the gradient may help, in particular for determining a distance along the vessel, the division in mask segments can also be implemented without this gradient.

To improve localization while maintaining efficiency and lowering noise, it is preferred in some embodiments to divide the vessel mask into a number of mask segments, wherein the number is selected from a range between three and ninety, preferably between five and fifty, most preferably between ten and thirty. When mask segments are further subdivided for each vessel branch separately, this number may increase by the number of further subdivisions; or the number may be initially the same, if the subdivision is performed sequentially (described later below with reference to FIGs 11A-11C). To get a statistically relevant distribution of pixel values, it is preferred in other or further embodiments that each mask segments contains at least a minimum number of pixels, e.g. at least ten pixels per mask segment, preferably at least one hundred pixels, more preferably at least one thousand pixels, e.g. up to ten thousand pixels, or more. Using high numbers of pixels may also allow sufficient remaining pixels when using a threshold (described later below with reference to FIGs 5A-5C).

FIGs 4A-4C illustrate distributions of pixel values “D” in respective images I[t=10,12,14] intersecting a respective mask segment

Vs[n=6]. Some embodiments comprise determining a respective distribution

ID} of pixel values “D” of pixels “P” in the respective image “I” intersecting the respective mask segment Vs. In other or further embodiments, comprise determining a respective segment value “S” characterizing the distribution

{D} at the respective instance of time “t” of the respective image “I”. For example, this can be done for respective combinations of a respective mask segment Vs in the set of mask segments Vs[n] and a respective image “I” in the sequence of images I[t], preferably at least for combinations of a respective mask segment intersecting with different images, more preferably repeated for different mask segment each combined with respective images, or even repeated for all (relevant) combinations of mask segments and images.

FIGs 5A-5C illustrate an embodiment which comprises determining a respective set of segment values S[t] based on the respective segment value “S” as function of the respective instance of time “t”. For example, this can be done for a respective mask segment Vs[n=6] in the set of mask segments Vs[n], and preferably repeated for all (relevant) mask segments, e.g. set of mask segments including at least a segment of interest such as suspected stenosis.

All the pixels in the respective image “I” intersecting the respective mask segment Vs form what is referred herein as the full set of intersecting pixels. In principle, the segment value “S” for the respective image “I” intersecting the respective mask segment Vs can be based on the pixel values “D” of the full set of intersecting pixels. Preferably though, the segment value “S” for the respective image “I” intersecting the respective mask segment Vs is based on the pixel values “D” of a subset of the full set of intersecting pixels. The inventors find that using a subset of the full set of intersecting pixels can improve sensitivity to specific aspects of the fluid flow which can be dependent on the specific subselection.

In one embodiment, the subset is determined by exclusively selecting pixels from the full set of intersecting pixels having a respective pixel value D meeting a threshold value T. In another or further embodiment, the threshold T comprises at least one of a minimum pixel value and a maximum pixel value. Preferably, the threshold value T is determined relative to a distribution of all pixel values in the full set of intersecting pixels. The inventors find that using a relative threshold value can improve adaptability to various circumstances, e.g. shifting mean value.

Alternatively, or in addition, also absolute threshold values can be used, e.g. to filter predetermined pixel values irrespective of the measured distribution.

In one embodiment, the threshold value T is determined to select a predetermined fraction of highest pixel values (top percentile) from the full set of intersecting pixels, e.g. 5% darkest pixels (highest concentration of contrast fluid). Of course, also other percentages can be used or a fixed minimum threshold. The inventors find that using a subset of darkest pixels representing the highest concentration of contrast fluid can improve sensitivity in particular to fluid concentrated at a center of the vessel.

In another or further embodiment, the threshold value T is determined to select a predetermined fraction of lowest pixel values (bottom percentile) from the full set of intersecting pixels, e.g. 5% lightest pixels (lowest concentration of contrast fluid). Of course, also other percentages can be used or a fixed maximum threshold. The inventors find that using a subset of lightest pixels representing the lowest concentration of contrast fluid can improve sensitivity in particular to fluctuations of fluid, e.g. arriving in a vessel.

In another or further embodiment, the threshold value T is determined to select an intermediate fraction of pixel values from the full set of intersecting pixels, e.g. using 50% of pixels with values around the mean value and/or excluding a top percentage darkest pixels and/or excluding a bottom percentage of lightest pixels. Of course, also other percentages can be used or a range of fixed minimum and maximum thresholds. The inventors find that using a midrange of pixel values can improve sensitivity in particular to the midrange behavior of the fluid.

FIGs 6A-6C illustrate an embodiment which comprises determining a combined segment value as a respective flow metric “Q” based on a respective set of segment values S[t]. For example, this can be repeated for some mask segments Vs[n=6,7,8] to yield respective flow metrics Q[n=6,7,8], and preferably repeated for all relevant mask segments to improve overall classification and/or localize a segment of interest. Some embodiments comprise determining at least a first metric for each mask segment Vs in the set of mask segments Vs[n]. As will be appreciated, the flow metric of a respective mask segment Vs can be used to characterize, e.g. classify, the contrast fluid Fc flowing through a respective section of the vessel structure “V” corresponding to the respective mask segment. Taken together, the flow metric of all segments can be used to characterize, e.g. classify, the whole vessel structure.

In some embodiments, for each mask segment Vs and each image “I”, at least two different segment values are generated based on the pixel values “D” of respective different subsets of the full set of intersecting pixels.

For example, for each mask segment Vs and each image “I”, a first segment value is generated based on a first subset of the full set of intersecting pixels and a second segment value is generated based on a second subset full set of intersecting pixels, wherein the second subset is different than the first subset, preferably non-overlapping. The inventors find that using a combination of different subsets, e.g. as described above, may yield different flow metrics that can be sensitive to different aspects of the fluid flow. For example, a combination of two, three, or more difference subsets can be used. Each subset may be used to generate a different parametric image.

The combination of different parametric images may be used in a combined display and/or more accurately determine relevant features such as locating a stenosis in the vessel structure by means of a machine learning algorithm or other model.

Alternatively, or in addition, to taking different subsets, different segment values can also be obtained by applying a different function to derive the segment value from the complete or partial distribution of pixel values. For example, a first segment value is determined by a first function taking a distribution of pixel values as input; and a different, second segment value is determined by a different second function taking the same distribution of pixel values as input. For example, the first function is configured to return an mean value of the distribution and the second function is configured to return a standard deviation of the distribution. Of course many different operations characterizing different aspects of the distribution, and many different combinations of those operations, can be envisaged. Advantageous examples, as used herein, include dividing the mean value of the distribution by the standard deviation of the distribution, or the reciprocal value thereof.

In some embodiments, for each mask segment Vs and each image “I”, at least two different sets of segment values are generated. In one embodiment, respective different flow metrics are determined based on the different sets of segment values. For example, for each mask segment Vs and each image “I”, a first flow metric 1s determined based on the first segment value, and a second flow metric is generated based on the second segment value.

Alternatively, or in addition, to deriving different flow metrics from different sets of segment values, different flow metrics can also be obtained from the same set of segment values. In one embodiment, a first flow is determined by a first function taking a set of segment values as input, and a different, second flow metric is determined by a different second function taking the same set of segment values as input. For example, the first function is configured to return an mean value of the set of segment values and the second function is configured to return a standard deviation of the set of segment values. Of course many different operations characterizing different aspects of the set of segment values, and many different combinations of those operations, can be envisaged.

Advantageous examples, as used herein, include dividing the mean value of the set of segment values by the standard deviation of the set of segment values, or the reciprocal value thereof.

FIG 7A illustrates a plot of a first flow metric Q1 as function of segment number “n”. In this case a subselection of pixel values was obtained by using a threshold to retain only the pixels having the lowest five percent of pixel values (5 percentile), e.g. corresponding to the lowest concentration of contrast liquid. This is illustrated on the left hand side of FIGs 5A-5C.

Respective segment values “S” were calculated for each mask segment Vs and each instance of time using the mean over standard deviation

S = Dm/Ds of the thresholded distributions of pixel values. For each set of segment values “S” as function of time, a respective flow metric was calculated using again a calculation of the mean over standard deviation, but now of the set of segment values. The present FIG 7A illustrates the resulting first flow metric Q1 as function of the mask segment number “n”, after normalizing these values in a range between 0 and 1.

FIG 7B illustrates the first flow metric Q1 applied as pixel value of respective vessel segments. Some embodiments comprise generating a parametric image Iq by setting a pixel value of pixels in the parametric image Iq intersecting the respective mask segment Vs based on (the value of) the respective flow metric of the mask segment. For example, the value of the respective flow metric can be used to determine a brightness of the pixels. To improve contrast, the value of the flow metric can, e.g., be normalized between 0 and 1 (here already done in FIG 7A).

FIG 7C illustrates an overlay of FIG 7B onto one of the images e.g. the last image I[21]. Some embodiments comprise combining the parametric image Iq with any image of the sequence of images I[t]. In one embodiment, the parametric is image is combined by overlaying the parametric image Iq onto an original (optionally enhanced) image of the images in the sequence, preferably a later image which most clearly shows the contrast fluid in the whole vessel structure “V”. For example, the parametric image is based on applying the flow metric as pixel values of respective segment, which are then averaged with pixel values of the original image. Also another function of the respective pixel values can be used. For example, a weighted average can be used to control the relative contribution of the original pixel values and parametric pixel values. Of course the pixel values can also be directly applied to, or combined with, the original image as a weighted average or otherwise. So it 1s not necessary to generate an intermediate image per se.

In FIGs 7A-7C, a segment of interest (*) is indicated which corresponds to the lowest value for the first metric Q1. In this case the segment of interest (*) overlaps a stenosis in the vessel structure, which is not a coincidence. Indeed, the inventors find that the first metric Q1 correlates well with this condition. Without being bound by theory, it will be understood that the value of the mean over standard deviation of pixel values in a mask segment will be low when there no contrast fluid and/or there is a large spread of pixel values in the corresponding vessel segment, and high when there is contrast fluid and/or a small spread of pixel values in the corresponding vessel segment. By using the lowest five percentile of pixel values, the present metric is particularly sensitive to the pixels having the lowest intensity of contrast fluid which the inventors find to be mostly at the periphery of the vessel (intensity is higher at the vessel center).

Furthermore, the present metric formed by the mean over standard deviation of the resulting segment values will be relatively high where there is relatively high change in spread of pixel values as function of time (e.g. corresponding to a rapid change in the contrast fluid along the vessel periphery), and relatively low when there is relatively low change in the spread of pixel values as function of time (e.g. more gradual or no change in the contrast fluid passing along the vessel periphery). So the inventors find that a stenosis can be localized using the first metric. It will be noted here that also other combinations of the mean and standard deviation could be used. For example, using the reciprocal value, the results may be inverted but still used in an equivalent way. And of course also equivalent different measures of the average pixel value and/or spread in pixel values could be used with similar result. And also additional or alternative statistical measures could be used to characterize similar or different characteristics of the respective pixel distributions and/or segment values. So it will be understood that the present methods and systems are not limited to the specific arithmetic operations used here.

FIGs 8A-8C are similar to the previous FIGs 7A-7C but showing a second metric Q2, which has been obtained using a different subselection of pixel values, namely those obtained by using a threshold to retain only the pixels having the highest five percent of pixel values (95 percentile). For comparison, the arithmetic operations to obtain the segment values and flow metric Q2 from the different subselection have not been changed compared to the previous figure. As illustrated by the present figures, the segment of interest (*) correlates with the highest value of the second metric. Without being bound by theory, the inventors find that the highest percentile, i.e. relatively darkest pixels in a segment, correlate well with the pixels at the center of the vessel (pixels are relatively lighter at the periphery).

Accordingly, the present metric may be sensitive to a different part of the flow, e.g. at the center where the presence of a stenosis may different effects such as turbulence and/or intermittent flow.

FIGs 9A-9C are similar the previous FIGs 7A-7C and FIGs 8A-8C, and have been obtained in a similar way, but with yet another subselection of pixel values, in this case using a midrange of pixel values (here the inter- quartile range). Again, it may be observed there is a correlation with the segment of interest (*), e.g. steepest drop in the flow metric value Q3 and/or relatively high value just before the stenosis. Without being bound by theory, the inventors find that a midrange of pixel values, e.g. excluding a fraction of the lightest and darkest pixels, may yield particular sensitivity to the average flow behavior, e.g. excluding the extremes.

In some embodiments, for each mask segment Vs and sequence of images I[t], at least two different flow metrics Q1,Q2 are determined, e.g. by taking different subsets and/or using different arithmetic functions to derive the segment values and/or flow metrics. In one embodiment, based on the different flow metrics Q1,Q2, a corresponding set of different parametric images is generated representing different aspects of the contrast fluid Fc flowing through the vessel structure “V” as function of time “t”. In another or further embodiment, a stenosis in the vessel structure “V” is determined based on the set of different parametric images and/or combined images generated by overlaying the different parametric 1mage Iq over a respective copy of a select image the sequence of images I[t].

The flow metrics Q and/or parametric images Iq, as described herein, can be used in various ways to improve the analysis of the contrast fluid Fc flowing through the vessel structure “V” as function of time “t”.

Some embodiments comprise displaying an image, or outputting an image for display, based on the parametric image Iq. The inventors find that the (single) displayed parametric image Iq can reveal information about the flow of contrast fluid, and consequently the vessel structure, which is practically impossible to observe in the original sequence of images. Furthermore, when the parametric image is overlaid with one of the original images, this may further improve analysis of the vessel in its surrounding context, including the background.

In some embodiments, a set of flow metrics Q and/or parametric image Iqis input into a computer model, or other algorithm, to determine a classification of the vessel structure “V” and/or determine a segment of interest (*). The inventors find that use of the flow metrics and/or parametric image Iq as input to the computer model can improve accuracy, reduce computational complexity, and/or save overall computing resources such as processor time and/or memory. For example, the flow metrics and/or parametric image Iq can be used as input instead of the original sequence of images, which sequence can consist of many images, each with many pixel values. It is recognized that most of these original pixels values may have minimal relevance to analysis of the vessel structure, e.g. background, parts of the vessel which have not been reached by the contrast fluid, and subtle variations which may or may not be relevant to determine the model output.

By using the flow metrics or parametric image, optionally in combination with one or more original images, the relevant aspects can be more efficiently determined in a respective model, with less consideration of superfluous or irrelevant data. Also the model, in principle, does not need to be aware of the specific sequence of events, which can greatly simplify the model, e.g. analysis algorithm.

In one embodiment, the computer model comprises a classification model which uses the set of flow metrics Q and/or parametric image Iq as input to determine, e.g. calculate, one or more classifications, e.g. parameters, that characterize a respective aspect of the vessel structure.

For example, the classification comprises determining that the vessel structure has an obstruction and/or narrowing (stenosis) which affects how the contrast fluid flows through the vessel; or conversely that there is no stenosis (healthy vessel). The output parameter classifying the vessel structure may be a binary parameter signifying the presences or absence of a respective indication (e.g. indication of stenosis or not); or comprise a value which signifies a severity and/or confidence in the respective indication.

In another or further embodiment, the computer model comprises a localization model which uses the set of flow metrics Q and/or parametric image Iq as input to determine, e.g. calculate, one or more locations in the vessel structure, in particular a vessel segment meeting a characterization, e.g. according to one or more threshold value(s). For example, the localization model is configured to determine a location of a stenosis or other feature of interest in the vessel structure.

In some embodiments, the computer model 1s based on a machine learning algorithm. In one embodiment, a stenosis is determined (classified and/or localized) by inputting one or more parametric images Ip and/or flow metrics into the machine learning algorithm. In another or further embodiment, the algorithm includes a logistic regression model which may, e.g., use K-fold cross validation and/or Area Under the ROC curve (AUC). In one embodiment, transfer learning from the localization model is used in the classification model, or vice versa. Typically, the machine learning algorithm is trained by comparing a classification and/or localization determined by the algorithm with a reference classification and/or localization. For example, a set of training data comprises a set of image sequences and corresponding (ground truth) classification and/or localization, e.g. determined by a physician or otherwise. The machine learning algorithm may include a set of predetermined (training) weights to classify and/or localize a feature of interest based on the input of a set of flow metrics and/or one or more parametric images. These weights can be stored as part of, or accessible to, a computer program performing the present methods.

Some aspects of the present disclosure can be embodied as one or more (non-transitory) computer-readable media storing instructions that, when executed by one or more processors, cause one or more devices to perform one or more methods as described herein. For example, these methods may include image based analysis of a vessel structure “V7,

determining of flow metrics Q, determining and/or displaying of a parametric image Iq, generating and/or displaying enhanced parametric images of a vessel structure, determining a classification and/or localization of a feature of interest such as stenosis, accessing and/or determining a set of weights used in a machine learning algorithm to classify and/or localize the feature of interest, et cetera.

In some embodiments, a classification of the vessel structure “V” and/or localization of a segment of interest (*) is displayed together with, or as part of, the image for display. Also combinations of the classification and localization model are possible, e.g. determining a presence/severity as a well as a location of a respective feature of interest. By using the parametric image instead of, or addition to, the flow metrics, the further information can be used to improve classification and/or localization. For example, a diameter and/or location of a respective vessel segment can be used to improve classification and/or localization of a stenosis. Further improvements in accuracy can be obtained by also including at least one of the original images, e.g. separately or as part of a combined parametric image.

Alternatively, or in addition to, determining a stenosis, also other features of interest can be determined. For example, the image based analysis of the vessel structure may include determining a (relative) pressure and/or flow rate in respective parts of the vessel structure based on the flow pattern. For example, a parametric image may represent a pressure distribution and/or flow rate map in the vessel structure, or the parametric image can be used as input to a computer model which determines an image of the vessel structure including respective (blood) pressure and/or flow rate values. In this way, the present methods can provide an alternative to conventional (invasive) methods of measuring pressure in a vessel structure, typically using a wire, and/or less conventional ways of measuring flow rate.

In some embodiments, a respective segment value “S” 15 determined using a first algorithm configured to calculate a statistical property of the distribution of pixel values “D”. In other or further embodiments, the respective flow metric “Q” is determined using a second algorithm configured to calculate a statistical property of the set of segment values. In principle, the second algorithm can be the same as the first algorithm, although in general these can be different algorithms. Preferably, the statistical property (used in the first and/or second algorithm) includes at least one of an average value (Sm) and/or spread in value (So), most preferably a combination thereof.

As described herein, the average value represents the average behavior, and can e.g. be calculated using the mean, central, expected value of a set or distribution of values. In one embodiment, the average value is used as part of determining a segment value to represent the average values of a (sub)set of pixel values in a segment. In another or further embodiment, the average is used as part of determining a flow metric to represent the average of a (sub)set of segment values over time. For example, the arithmetic mean can be calculated as the sum of the values divided by the number of values. Alternative or equivalent measures which can provide similar information about the average may include the geometric mean, harmonic mean, population mean (expected value), inner mean, geometric mean, root mean squared, median, et cetera.

As described herein, the spread in value is a measure of the amount of variation or dispersion of a set of values. Examples may include the standard deviation, average deviation, mean absolute deviation, median absolute deviation, variance, et cetera. For example, a low standard deviation indicates that the values tend to be close to the mean (also called the expected value) of the set, while a high standard deviation indicates that the values are spread out over a wider range.

The inventors find that the development of the average value may be useful in characterizing the general amount or density of contrast fluid in a vessel segment, e.g. representing how much and/or how fast the fluid flows through the vessel segment. Furthermore, the inventors find that the spread in value, e.g. standard deviation, may represent the variation or turbulence of the flow, which may be indicative, e.g., of the presence of a stenosis.

While each of the average value and spread of values can be used individually in calculating a segment value “S” and/or flow metric “Q”, preferably these values are combined, e.g. using an arithmetic operation such as division and/or multiplication. Besides the average value and spread of values, also other or further functions can be used to determine a segment value “S” representing the pixel values of a segment and/or a flow metric “Q” representing the segment values over time. For example, the maximum and/or minimum pixel value in a segment can provide an alternative, or additional, useful measure of a segment value at a respective instance in time. For example, the maximum and/or minimum segment value in a segment can provide an alternative, or additional, useful measure for a respective flow metric of a segment. These and other functions can be applied alone or in combination. For example, the minimum and/or maximum can also be used as normalization.

FIG 10 illustrates combining different flow metrics Q1,Q2,Q3 to improve analysis in identifying a segment of interest (*) having a combination of characteristics. As was demonstrated above, different subselections of pixels can be used to accentuate respective features. By combining the different metrics in a model, a combined result can be achieved which can be tuned to specific characteristics of interest and improve overall robustness. For example, the different flow metrics can be combined in a computer model, as described herein, to yield a classification and/or localization. In this case, the computer model has identified a stenosis in the indicated segment of interest (*) with relatively high confidence.

FIG 11A illustrate breaking down the above identified segment of interest according to different branch segments Vb and determining segment values “S” for each branch segment b=1,2,3. FIG 11B illustrates a fourth flow metric Q4 applied as pixel value of respective branch segments to determine a branch segment of interest (*). In this case the fourth metric was derived for the respective branches as standard deviation over mean of the segment values. As illustrated, the branch with the highest standard deviation over mean (most volatile behavior) is found to correspond with the stenosis. FIG 11C illustrates an overlay of FIG 11B onto one of the images e.g. the last image I[21]. For example, the image of FIG 11C is particularly suitable for display to locate the stenosis within the vessel structure.

In some embodiments, the vessel mask Vm is (further) divided into mask segments Vs in accordance with a respective branch Vb of the vessel structure “V” coinciding with the respective mask segment Vs. In principle, the initial mask segments Vs can be determined such that it divides not only different sections of the same branch (e.g. as function of distance to the injection point) but also in accordance with the respective branch Vb, e.g. such that each mask segment is exclusive to a single vessel branch. Alternatively, the subdivision in respective branch segments is performed after an initial one or more flow metrics have been determined for segments spanning multiple branches. For example, the initial one or more flow metrics may indicate a segment of interest which can then be further analyzed. For example, this sequential procedure may improve increase robustness since more pixels are available per segment and/or computational efficiency since less segments in total need to be considered.

The procedure of analyzing the mask segments assigned to individual vessel branches, may in principle proceed in the same way as already described for the mask segments. For example, set of segment values S can be calculated for each branch segment separately, and a respective flow metric “Q” can be calculated based on the set of segment values S. Optionally, the thresholds, subsets, and algorithms for calculating the set of segment values S and/or flow metric “Q” for the individual branches can be adapted. For example, in

FIG 11B, the standard deviation over mean for each curve was used to emphasize that the branch b=3 with the relative largest spread in values is most likely to correspond to a stenosis (indicated by *).

FIG 12A illustrates validation results for classification of stenosis.

The graph was produced based on classification of stenoses of over 75+ patients as calculated by a logistic regression model. FIG 12B illustrates validation results for localization of a respective vessel segment containing stenosis. The graph was produced by localization of vessel segment for 20 stenoses using a combination of seven best correlating metrics. As indicated in the legend, the metrics were optimized by FNR score, F1 score, and combined. These outperformed the physician scores (Circle) for the dataset.

In interpreting the appended claims, it should be understood that the word "comprising" does not exclude the presence of other elements or acts than those listed in a given claim; the word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements; any reference signs in the claims do not limit their scope; several "means" may be represented by the same or different item(s) or implemented structure or function; any of the disclosed devices or portions thereof may be combined together or separated into further portions unless specifically stated otherwise.

Claims (15)

CONCLUSIONS A computer-implemented method for image-based analysis of a vessel structure (V), the method comprising receiving a sequence of images (I[t]), where a respective image (I) in the sequence of images (I[t]) comprises a set of pixels (P) with pixel values (D) representing the vasculature (V) at a respective time (t) after injection with a contrast agent (Fc) flowing through the vasculature (V); for a respective image (I) in the sequence of images (I[t]), determining a respective vessel mask (Vm) outlining the vessel structure (V) based on respective pixel values (D); dividing the respective vessel mask (Vm) into a set of mask segments (Vs[n]) that coincide with different sections of the vessel structure (V); for respective combinations of a respective mask segment (Vs) in the set of mask segments (Vs[n]) and a respective image (I) in the sequence of images (I[t]), determining a respective distribution of pixel values (D) of pixels (P) in the respective image (I) overlapping the respective mask segment (Vs), and determining a respective segment value (S) characterizing the distribution at the respective time (t) of the respective image (I); for a respective mask segment (Vs) in the set of mask segments (Vs[n]), determining a respective set of segment values (S[t]) based on the respective segment value (S) as a function of the respective time point (t); and determining, based on the respective set of segment values (S[t]), a combined segment value as a respective flow metric (Q) of the contrast medium (Fc) flowing through a respective portion of the vascular structure (V) corresponding to the respective mask segment (Vs). The method of the preceding claim, comprising determining at least one flow metric (Q1) for each mask segment (Vs) in the set of mask segments (Vs[n]). The method according to the preceding claim, comprising generating a parametric image (Iq) by setting a pixel value of pixels in the parametric image (Ip) overlapping the respective mask segment (Vs) based on the respective flow metric (Q ) of the mask segment. The method according to the preceding claim, comprising displaying an image based on the parametric image (Iq), or a combination of the parametric image (Iq) with an image (1[21]) of the sequence of images ( I[t]). The method according to any one of the preceding claims, wherein a set of flow metrics (Q) and/or at least one parametric image (Iq) is fed into a computer model to determine a vessel structure classification (Vs) and/or a segment of interest (*). The method according to any one of the preceding claims, comprising determining an injection site (Ci) of the contrast agent (Fc) entering the vessel structure (V), dividing the vessel mask (Vm) into respective mask segments (Vs[n]) in accordance with a respective distance (d) from the injection site (Ci). The method according to any one of the preceding claims, wherein the vessel mask (Vm) is divided into mask segments (Vs) according to a respective branch (Vb) of the vessel structure (V) coincident with the respective mask segment (Vs). The method according to any one of the preceding claims, wherein the respective segment value (S) is determined using a first algorithm configured to calculate a statistical property of the distribution of pixel values (D), and the respective flow metric (Q) is determined using a second algorithm configured to calculate a statistical property of the set of segment values, where the statistical property includes at least one of a mean value (Dm,Sm), range in value (Do,S0), or combination thereof. The method of any one of the preceding claims, wherein all the pixels in the respective image (T) overlapping the respective mask segment (Vs) form a complete set of overlapping pixels, the segment value (S) for the respective image (T) being overlap the respective mask segment (Vs) is based on the pixel values (D) of a subset the full set of overlapping pixels, where the subset is determined by selecting only pixels from the full set of overlapping pixels with a respective pixel value (D) that satisfies a threshold value (T), the threshold (T) comprising at least one of a minimum pixel value and a maximum pixel value, the threshold value (T) being determined relative to a distribution of all pixel values in the full set of overlapping pixels. The method according to any one of the preceding claims, wherein for each mask segment (Vs) and each image (I), at least two different segment values are generated based on the pixel values (D) of respective different subsets of the full set of overlapping pixels. The method according to any one of the preceding claims, wherein for each mask segment (Vs) and each image (I), at least two different sets of segment values are generated, with respective different flow metrics (Q1, Q2) determined based on the different sets of segment values. The method according to any one of the preceding claims, wherein, for each mask segment (Vs) and sequence of images (I[t]), at least two different flow metrics (Q1,Q2) are determined, the presence and/or location of a stenosis in the vascular structure is determined based on the at least two different flow metrics (Q1,Q2). A non-temporary computer-readable medium storing instructions which, when executed by one or more processors, cause an apparatus to perform the method of any preceding claim. The medium of the preceding claim storing a machine learning algorithm and a set of weights configured to determine the presence and/or location of a stenosis based on receipt as input to the algorithm from at least one of a set flow metrics (Q1,Q2,Q3), determined according to the method of any one of claims 1-12; and/or a parametric map (Ip) based on the set of flow metrics (Q1,Q2,Q3). A system for image-based analysis of a vessel structure (V), the system comprising a processor communicatively coupled to a medical imaging device for receiving a sequence of images (I[t]); and a memory that stores non-temporary computer-readable instructions which, when executed, cause the processor to perform the method of any one of claims 1-12.