A method, an apparatus and a computer program for segmenting a structure in a multidimensional dataset
The invention relates to a method of segmenting a structure in a multidimensional dataset, whereby a morphological filter with a structuring element is applied to said dataset.
The invention further relates to an apparatus for enabling a segmentation of a structure in a multi-dimensional dataset, said apparatus comprising: an input for accessing the multi-dimensional dataset; a processor arranged to segment the multi-dimensional dataset using a morphological filter with a structuring element for obtaining the structure in accordance with a comparison between a dimension of the structuring element and a pixel intensity distribution of the multi-dimensional dataset.
The invention still further relates to a computer program.
An embodiment of a method as is set forth in the opening paragraph is known from Yung-Nien Sun et al 'The Computer Image Analysis for the 2 -D Coronary Arteriograms', IEEE TENCON '93/Beijing, p. 978-982. In the known method a segmentation approach is disclosed, which is arranged to use inherent distribution along a direction perpendicular to a vessel direction. Hereby for detecting arterial segments from angiogram, a morpho logical operator is used with a structuring element which shape is similar to the grey- level distribution of blood vessels. As a result, the skeleton of coronal artery from cineangiograms is derived. In order to segment a structure in the multidimensional dataset, the known method uses several complicated computational steps. Firstly, in order to reduce influence of variations in the background in an image obtained from the multi-dimensional dataset, a morphological operator with a structuring element is applied, which size is determined by the largest diameter among all blood vessels in the image. The resulting image, being the background image, is subtracted from the original image. Secondly, the background-subtracted image is processed with a binary filter, subsequently being subjected to binary closing and opening operators. Next, a conditional dilation is carried out to compensate for erroneous removal of vessels with smaller widths
than the applied structuring elements. Finally, to obtain a complete tree structure of coronal arteriograms and successfully track all arterial segments, it is essential in the known method to determine the direction vector at each bifurcation point or terminal point.
It is a disadvantage of the known method that for obtaining a segmented structure, notably to segment a vessel from a suitable dataset, a complicated image segmentation sequence has to be applied, which requires major computational resources and is time consuming.
It is an object of the invention to provide a method of segmenting a structure in a multi-dimensional dataset using accurate yet fast and simple means.
To this end the method according to the invention comprises the following steps: using for the morpho logical filter a multi-scale morpho logical filter comprising a set of structuring elements with different respective dimensions; applying the multi-scale morphological filter to the dataset by consecutively selecting a structuring element with a different dimension from said set; sub- segmenting the structure based on a comparison between the selected structuring element and a pixel intensity distribution in the multi-dimensional dataset; - segmenting the structure based on a combination of said sub-segmentations.
The technical measure of the invention is based on the insight that by using the multi-scale morphological filter, in particular by consecutively applying said filter with increasing element size, a hierarchy of sub-segmentations of the structure is obtained, which are then easily combined to yield the segmented structure. The method according to the invention detects where the morphologic element fits in the intensity landscape of the original dataset. In the first place the method according to the invention delivers a segmentation of the structure, notably, a vessel in a vessel-tree. Secondly, information on local dimensions along the segmented structure is obtained, as this information is directly linked to the applied structuring element of the morpho logic filter. This last feature yields an additional advantage, which is particularly supportive for segmentation of vessels in a vessel- tree, where the branches split up and become thinner towards the periphery. With the multi- scale morphologic filter the segmentation contains inherently the size information along the vessels. Figure 1 presents a schematic illustration of the action of a structuring element of a particular size on a dataset. In accordance with the method of the invention, to obtain an
accurate segmentation of the structure in the multi-dimensional dataset, it is sufficient to consecutively apply the morphological filter, each time selecting a structuring element with a different dimension. It is noted that it is possible to apply the structuring elements with consecutively increasing size, or with consecutively decreasing size. In the former case one yields a multi-scale segmentation, whereby the smallest structuring element segments all vessels up to a fine level and the largest structuring element leaves only vessels corresponding to its size. When a volume rendering operation is applied to such segmentation, in some cases, it may occur that the large vessels are hidden behind the small vessels classification. To avoid this effect a region growing from the large vessels to the small element segmentation may be applied. In any case, the multi-scale morphological filter according to the invention is proven to be fast in its operation and is easy to implement.
In an embodiment of the method according to the invention the respective dimensions of the structuring elements are selected to enable detection of edge pixels.
This technical measure is based on the insight that partial volume effects are present in the image, yielding so-called edge pixels. These pixels will be filtered out if a structuring element is applied, as is, for example, shown by 3 in Figure 1. Therefore, it is preferable to select the dimension of the structuring elements to enable edge pixel detection, for example by defining flanks, or the like, in the leading front and in the trailing front of the structuring element. In a further embodiment of the method according to the invention, the structure comprises a vessel, whereby the respective dimensions of the structuring elements are selected in accordance with variation in a cross-section of the vessel.
This technical measure is based on the insight that by defining the shape and the size of the structuring element in accordance with the scale of cross-sectional dimensions in the vessel under consideration, the morphological filter can be advantageously tailored- made for vessels that become smaller and smaller towards the periphery.
In a still further embodiment of the method according to the invention, the method further comprises the steps of: accessing results of the segmentation of the multi-dimensional dataset providing the structure; determining values of a pre-defined feature of the structure; coding the values of the pre-defined feature using graphic means in accordance with a pre-selected criterion; mapping the coded values of the pre-defined feature to the structure.
This technical measure is based on the insight that the results of the automatic three-dimensional vessel-tree segmentation step can be automatically used for determination of a suitable feature of the vessel, notably a radius, a diameter or another dimension of the vessel, or a circularity of the vessel. Preferably, the pre-defined feature is a dimension of the tubular structure in a direction substantially perpendicular to a longitudinal direction of the tubular structure. As is explained in the foregoing, an application of the multi-scale morphological filter inherently results in a scale of local dimensions of the structure being segmented. Therefore, the true value of the local dimension is obtained and is subsequently used for visualization purposes, irrespective of a future visualization view, thereby avoiding misinterpretation due to a partial projection. Thus, even if a truly large vessel appears tiny on a selected cross-sectional image, for example, because this slice only touches the border of the vessel, its true dimension at this location will be coded and will be shown in a resulting image because its true dimension was initially coded. It must be understood that the term 'mapping' is defined as a procedure of correlating the respective coordinates of the segmented object and the map of coded values. Due to the mapping, any arbitrary projection will be accompanied with the correct map of coded values, which will always be presented in true size. The method according to the invention preserves the original pixel values, whereby the color-coding is a preferred embodiment of the coding operation. However, other graphic coding may be used, like shadowing, hatching, underlying, etc. The coding may be in absolute values, or may be expressed as a percentage of the largest detected value. The size- encoded color overlay reflects the three-dimensional vessel size, rather than the arbitrary size corresponding to a two-dimensional cut-plane.
In a still further embodiment of the method according to the invention, an image of the segmented structure is reproduced on a display, said image being overlaid with the results of said coding.
It is found that the method according to the invention is surprisingly suitable for the application for detection of emboli, whereby the specificity of the detection is substantially improved with respect to a human inspection based on a plurality of two-dimensional slices. The anatomical coarse-to-fine vessel structures in a distal direction are preferably reflected in a color distribution, which human observers can quickly perceive. Also variations between different human observers are minimized.
An apparatus according to the invention comprises: an input for accessing the multi-dimensional dataset;
a processor arranged to segment the multi-dimensional dataset using a morphological filter with a structuring element for obtaining the structure in accordance with a comparison between a dimension of the structuring element and a pixel intensity distribution of the multi-dimensional dataset; - a sequencer arranged to select a structuring element from a pre-defined set of structuring elements with respective dimensions and to consecutively initiate the processor to apply the morphological filter with the selected structuring element to the multi-dimensional dataset to yield respective sub-segmentations; a logic unit arranged to obtain the segmented structure by combining the results of said sub-segmentations.
The apparatus according to the invention is suitable for practicing the method as is set forth in the foregoing. The apparatus according to the invention operates reliably, whereby the computation time is linear with the number of structuring elements in the predefined set. In an embodiment of the apparatus according to the invention, the respective dimensions of the structuring elements are definable within a pre-defined scale of dimensions.
It is found to be particularly advantageous to pre-define a scale of dimensions for the structuring elements to be used in the multi-scale morphological filter. Preferably, the pre-defined scale is selected to correspond to a variation of sizes of the structure being analyzed, notably a variation in diameters of the vessels constituting a vessel tree.
In a further embodiment of the apparatus according to the invention, whereby the segmented structure comprises a tubular structure, the apparatus further comprises an encoder arranged to code a local dimension of the segmented structure in a direction substantially perpendicular to a longitudinal direction of the structure using graphic means in accordance with a pre-determined criterion, said apparatus still further comprising a mapping means arranged to map the coded values of said local dimension to the tubular structure.
In its preferred embodiment the encoder is implemented as a computer program, arranged to calculate a range in the determined values of the pre-defined feature of the structure and to assign a certain graphic feature to each running value of the pre-defined parameter, for example, in accordance with its percentage of the maximum value. This way of coding is particularly suitable for variations within a small range. Alternatively, a higher than a linear power function or a lower than a liner power function may be used to assign a color code or any other suitable graphic coding to a particular running value.
For the mapping means a look-up table with stored coordinates of the structure and the coded values may be used. Alternatively, the mapping means may be adapted to use a suitable function, like a spline to define the object, whereby the coordinates of the coded values are defined within the terms of the function. A suitable combination of the above methods is also possible.
The output of the apparatus according to the invention comprises the structure, segmented within the multi-dimensional dataset, said structure being mapped with coded information about the true value of the pre-defined feature of the object. This output may be remotely accessed, for example using internet or wireless access by a remote user, for visualization and inspection purposed. This embodiment of the apparatus according to the invention is advantageous, as the remote user does not have to acquire all necessary and frequently expensive software means to process the multi-dimensional dataset on his computer.
In an embodiment of the apparatus according to the invention, the apparatus further comprises a graphic unit arranged to overlay an image of the segmented structure with the coded values of its local dimensions.
For inspections and visualizations it is advantageous that the apparatus according to the invention comprises the graphic unit suitable to overlay the image of the segmented structure with the coded values of the pre-determined feature. The output of such a device is an image of the segmented structure appended with a suitable graphic representation of the coded values of the feature of interest. Such images may be advantageously used for archiving purposes, for video-conferencing, reporting, etc. Preferably, the apparatus still further comprises a display for displaying the image of the object overlaid with the coded values of the pre-determined feature for inspection of a suitable medical specialist.
In a further embodiment of the apparatus according to the invention, the apparatus further comprises a data acquisition unit arranged to obtain the multi-dimensional dataset.
For usage in a professional diagnostic environment, like diagnostic procedures in a hospital, it is advantageous that the apparatus according to the invention is further provided with a suitable data acquisition unit. Various embodiments of data acquisition units are contemplated, including, but not limited to, a computer tomography unit, a magnetic resonance imaging unit, an ultra-sound imaging unit, etc.
The computer program according to the invention is arranged to comprise instructions for causing a processor to carry out the steps of the method as is set forth in the foregoing. An operation of the computer program according to the invention will be explained with reference to Figure 5.
These and other aspects of the invention will be discussed in further details with reference to figures.
Fig. 1 presents a schematic illustration of a structuring element in one dimension.
Fig. 2 presents a schematic illustration of a result of an application of the multi-scale morphological filter.
Fig. 3 presents a schematic view of an embodiment of the apparatus according to the invention. Fig. 4 presents in a schematic way an encoded overlay of vessel sizes on a diagnostic image.
Fig. 5 presents a schematic block-scheme of the computer program according to the invention.
Figure 1 presents a schematic illustration of a structuring element in one dimension. When a morphological filter 1 with a structuring element 3 is applied to a function 5 describing a pixel intensity, the resulting convolute 6 is determined by the size and shape of the structuring element 3. All pixels positions where the structuring element fits under the intensity landscape of the original volume keep their original values, all other positions are set to a fixed value, for example to zero. The result is thus a segmentation of structures that are bigger than the structuring element 3, as well as of structures that have a higher intensity value than a predefined intensity value. It must be noted that is some circumstances, in particular when partial volume effects are frequent in the multi-dimensional dataset, it is preferable to define the structuring element so that edge pixels are detected. In one dimension an exemplary shape of the corresponding structuring element is given by the structuring element 4 comprising flanks in a leading front and a trailing front. It is noted that the shape of the flanks may be defined in any appropriate way, not necessarily using a linear function. One skilled in the art will immediately understand how the leading front and the
trailing front may be defined in higher dimensions, without departing from the teaching of the present invention.
Figure 2 presents a schematic illustration of a result of an application of the multi-scale morphological filter 10 in accordance with the invention. Preferably, the multi- scale morphological filter comprising structuring elements with increasing sizes is consecutively applied to the dataset comprising information on a vessel. In this example the respective sizes of the structuring elements are the following: A < B < C < D. The application of such multi-scale morphological filter results in a multi-scale segmentations of the sub- vessels 12, 14, 16, 18, whereby a small structuring element segments all vessels up to a fine level and a larger structuring element discards the tiny branches first, up to the largest structuring element which discards all branches except the biggest one fitting its size. Preferably, the structuring elements are selected to enable a detection of edge pixels corresponding to partial volume penumbra's 12', 14', 16', for example in a similar way with respect to a concept presented with reference to Figure 2. Thus, due to the way the multi- scale structuring element proceeds, one inherently obtains information on the local size along a vessel trunk. By defining the shape and the size of the morphological structuring element per scale of the vessel, the multi-scale morphological filter can be tailored for vessels that become distally thinner.
It must be noted that a vessel structure from a small structuring element contains both small and larger vessels. At the larger vessels it will be even slightly bigger than as segmented with a large structuring element. In a volume rendering this effect may hide the large vessel classifications behind the small vessel classification. This hiding can be avoided by applying a per se known region growing algorithm in a direction from the large vessels to the small vessels. Preferably this is carried out iteratively, whereby for the most cases two iterations will suffice.
Figure 3 presents a schematic view of an embodiment of the apparatus according to the invention. The apparatus 20 comprises an input 22 for accessing and receiving the multi-dimensional dataset in any suitable form. For example, the apparatus 20 may be involved in the acquisition of the image data, whereby the apparatus 20 is further equipped with a suitable data acquisition unit 30. Various embodiments of the data acquisition unit are contemplated, including, but not limited to an X-ray unit, a computer tomography (CT) unit, a magnetic resonance (MR) unit, any combination of the former, etc. In this case the multi-dimensional dataset may be acquired in an analogue form and converted using a suitable A/D converter to a digital form for further processing. The multi-
dimensional dataset may also be received in a digital form, e.g. through direct acquisition in a digital form or via a computer network after having been acquired by another computer/medical instrument. The core of the image processing apparatus is formed by a processor 24 which is arranged to load the multi-dimensional dataset from the input 22 and to segment the structure using the segmentation method with a multi-scale morpho logical filter in accordance with the invention. An example of a suitable processor 24 is a conventional microprocessor or signal processor, a background storage 28 (typically based on a hard disk) and working memory 26 (typically based on RAM). The background storage 28 can be used for storing the multi-dimensional dataset (or parts of it) when not being processed, and for storing results of the image segmentation algorithm. The main memory 26 typically holds the (parts of) multi-dimensional dataset being processed and the results of the coding and referencing operations. The apparatus 20 according to the invention preferably further comprises an encoder 25 arranged to code the determined values of the pre-determined feature using graphic means in accordance with a pre-selected criterion. The criterion may be selectable from a list of valid criteria, stored in a file 25a. The encoder is arranged to assign a suitable coding value to a running value of the determined dimension of the structure, notably a vessel, for example, based on a percentage of the running value in the total range of determined values of the dimensions of the vessel. The apparatus further comprises a sequencer 27 arranged to select a structuring element from a pre-defined set (not shown) of structuring elements and to consecutively initiate the processor 24 to apply the multi-scale morphological filter to the dataset. The apparatus further comprises mapping means 71 arranged to map the coded values of the selected feature to the object. Preferably, the mapping means is arranged to create a file with correlated coordinates of the object and the coordinates of the coded values representing the selected feature. Preferably, the encoder 25, the sequencer 27, the mapping means 71 and the processor 24 are operable by a computer program 23, preferably stored in memory 28. The apparatus further comprises a logic unit 21 arranged to yield the final segmentation of the structure based on a suitable combination of the sub-segmentations. An output 29 is used for outputting the results of the processing, like the segmented structure and, optionally, the results of the mapping step, preferably stored in one file.
Preferably, the apparatus 20 is arranged to further process the multidimensional dataset. The output 29 of the apparatus 20 comprises the segmented structure, notably a vessel, together with graphically coded values of the vessels' dimensions obtained during the process of image segmentation using the multi-scale morpho logical filter in
accordance with the invention. Preferably, the coding comprises a color coding. Preferably, the output 29 comprises an image of the segmented structure, notably a vessel, overlaid with the coded values of the diameter, said image being stored in a suitable file. For this purpose the apparatus 20 comprises a graphic unit 70 arranged to overlay an image of the segmented structure with the coding, notably the color-coding. The output 29 of the apparatus 20 is made available to the further input 35 of a suitable image viewer 31. Preferably, the further input 35 comprises a suitable further processor arranged to operate a suitable interface using a program 36 adapted to control a user interface 34 so that an image 33 is visualized, comprising a graphic representation of overlaid coded values 33a on the segmented structure (not shown). Preferably, for user's convenience, the viewer 31 is provided with a high- resolution display means 32, the user interface being operable by means of a suitable interactive means 37, for example a mouse, a keyboard or any other suitable user's input device.
Figure 4 presents in a schematic way an encoded overlay of vessel sizes on a diagnostic image. However in this example a two-dimensional image is shown, the overlay may also be implemented for a three-dimensional image. Preferably, the overlay of coded data is presented using a suitable user interface 42 of an image viewer. The user interface 42 comprises a graphics window 44 wherein a visualized image of the segmented structure 47 overlaid with a color coded data of local diameter values is presented. The image may also comprise other supplementary information 46, like a surrounding anatomy for localization purposes. The user interface 42 further comprises suitable controls 49, arranged to control the visualization mode. Using the controls 49, the user may adjust an orientation of a projection plane, scroll over different slices of the same projection orientation, etc. The informative window 48 preferably presents absolute values of the dimensions (in this example a running value of the diameter along the vessel) corresponding to respective color codes.
Alternatively, presentation of a percentage difference with respect to a certain value, for example a maximum value within a sub-branch or a full tree is possible.
Figure 5 presents a schematic block-scheme of the computer program according to the invention. The operation of the computer program according to the invention is schematically illustrated by the flow-chart 50. It is noted that each of the structuring blocks of the flow-chart 50 may be implemented by a subroutine or by a plurality of subroutines. At step 51 the computer program according to the invention inputs the multi-dimensional dataset into a suitable storage means, for example into operational memory. At step 52 the computer program applies the multi-scale morpho logical filter to said dataset consecutively. For this
purpose a sequence 52a is defined, which is arranged to access a structuring element from a pre-defined set of structuring elements (not shown). Preferably, the morphological filter operates by selecting the structuring elements in the order of increasing their dimensions. At step 54 an operation of sub-segmenting the multi-dimensional dataset is carried out by application of the morphological filter with the selected structuring element. The result of each sub-segmenting is stored at step 54a. When the loop of sub-segmenting the structure in the multi-dimensional dataset is completed, the computer program according to the invention proceeds to the step 56, whereby the final segmentation of the structure is carried out based on a combination of the sub-segmentations. The named combination may comprise operations, like superimpose, subtract, extract, etc.
When the structure is segmented, the information on the local dimensions of the structure is automatically available due to intrinsic properties of the multi- scale morphological filter, as was explained above. It is found to be particularly advantageous to use this fact for coding the obtained values of the local dimensions of the structure for visualization purposes. Therefore, in its embodiment the computer program according to the invention comprises the step 58, during which the values of the local dimensions of the structure are coded using graphic means. Examples of suitable coding using graphic means comprise color coding, grey-scale coding, hatching, etc. Still preferably, at step 60 a suitable image of the segmented structure is created, notably is computed or is accessed. The image may be a two-dimensional image, or a three dimensional image. At step 62 the computer program carries out the step of overlaying the image of the segmented structure with the results of the coding and at step 64 the overlaid image is displayed on a suitable display means. This way of visualizing the local dimensions of the segmented structure is advantageous, as in all cases the true values of the local dimensions are being coded and thus are being presented to an inspector. This is of particular importance for two-dimensional images, as in oblique projections a dimension of the structure, shown in the image, may substantially deviate from its true dimension. This feature of the computer program substantially improves the reliability of the image analysis, in particular for detection of vessel abnormalities, like embolisms, stenosis, or other pathologies leading to a sudden change in the lumen cross-section.