Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/10—Segmentation; Edge detection
  • G06T7/12—Edge-based segmentation
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/10—Segmentation; Edge detection
  • G06T7/143—Segmentation; Edge detection involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/10—Segmentation; Edge detection
  • G06T7/149—Segmentation; Edge detection involving deformable models, e.g. active contour models
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
  • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
  • G06T7/344—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving models
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V10/00—Arrangements for image or video recognition or understanding
  • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
  • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
  • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
  • G06V10/755—Deformable models or variational models, e.g. snakes or active contours
  • G06V10/7553—Deformable models or variational models, e.g. snakes or active contours based on shape, e.g. active shape models [ASM]
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10072—Tomographic images
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/20—Special algorithmic details
  • G06T2207/20081—Training; Learning
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/20—Special algorithmic details
  • G06T2207/20112—Image segmentation details
  • G06T2207/20124—Active shape model [ASM]
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/30—Subject of image; Context of image processing
  • G06T2207/30004—Biomedical image processing
  • G06T2207/30081—Prostate
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
  • G06V2201/03—Recognition of patterns in medical or anatomical images
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
  • G06V2201/03—Recognition of patterns in medical or anatomical images
  • G06V2201/031—Recognition of patterns in medical or anatomical images of internal organs

Definitions

• the present application relates to a method and apparatus for generating and using a subject-specific statistical motion model, such as may be used in the processing of medical images.
• Statistical shape modelling provides a powerful method to describe and compensate for physical organ shape changes, for example, due to inter-subject anatomical variation, and has been investigated extensively for a range of medical image analysis tasks, including image segmentation [1 ] and image registration [2].
• image segmentation [1 ] and image registration [2] image segmentation [1 ] and image registration [2].
• PCA principal component analysis
• Such models help to ensure that physically implausible organ shapes are not considered, or are at least penalised, when searching for a solution within a segmentation and registration algorithm.
• Variations in the shape of an organ can generally be considered as arising from two sources.
• the first termed inter-subject variation
• the second termed intra-subject variation
• Such intra-subject variations can be caused by many different factors, such as a change in posture of the person, natural organ motion (e.g. due to breathing or the beating heart), tissue deformation by the insertion of a medical instrument, the progression or regression of a disease, etc.
• SSMs that represent intra-subject organ motion may be termed statistical motion models (SMMs) to distinguish them from the more general SSM [6, 7].
• SMMs statistical motion models
• inter- and intra-subject shape variation is illustrated by considering the following: if we have first and second organ states, which each corresponds to a different organ shape that arises from intra-subject organ shape variation, then there must be some physical transition of the organ between the first and second states to produce the change in organ shape.
• a model of such intra-subject variation can be referred to as a SMM (statistical motion model), since the physical transition implies a motion in a general sense (including a shape change due to organ deformation) between the first and second shapes.
• SMM statistical motion model
• the differences between the first and second organ shapes arise from inter-subject variation, then they correspond to different individuals, and there is no reason for such a physical transition to exist between the first and second shapes corresponding to some change in state. Accordingly, this type of variation is represented by a more generic SSM (statistic shape model), since the first and second shapes are not generally related to one another by motion.
• an SSM based at least in part on inter-subject training data, it may permit a variation that is not, in fact, physically plausible for a given organ.
• the separation between the eyes in effect, the skull size
• an SSM which is based on at least in part on inter-subject training data might potentially allow variation in this eye separation, despite the fact that such variation is not physically plausible in the context of an individual subject.
• multilinear analysis [9] has been proposed as a method for dynamic modelling of the heart [10] and cardiac valve [1 1 ] motion.
• This approach enables inter-subject shape variations (e.g., due to subject specific differences in the size and shape of the heart of different individuals) and intra-subject shape variations (e.g. due to physiological heart motion) to be represented by the same statistical model.
• this method requires that inter-subject temporal correspondence between motion subspaces is known - in other words, the states of the organs for different subjects must be correlated, for example, via an independent signal, such as an electrocardiographic (ECG) signal, which measures the electrical activity of the heart and is therefore is inherently correlated to its motion.
• ECG electrocardiographic
• subject-specific dynamic data on organ motion for example, from imaging
• imaging is often extremely limited or not available at all, but many examples exist in the context of image-guided surgical procedures where it is desirable to be able provide models which use learnt information on inter- and intra-subject organ shape variation to predict subject-specific organ shape and motion given only sparse and potentially noisy (intraoperative) spatial data that describe the current organ state.
• MRI magnetic resonance imaging
• intraoperative imaging such as ultrasound
• One well-established way of overcoming this difficulty is to perform a (non-rigid) registration (i.e. spatial alignment) between the pre-operative image and the intraoperative image, thereby allowing structures determined in the pre-operative images to be mapped across to (and displayed in conjunction with) the intra-operative images.
• this is achieved using special-purpose image fusion software, and an important requirement for surgical applications is that the image registration needs to be performed within a timescale that is acceptable in the context of the surgical procedure.
• the image registration should also be able to compensate for movement and deformation in the organ or structures of surgical interest, for example, due to changes in the subject position or posture, or as a direct result of instruments used in the surgical procedure itself (for example, organ deformation arising from ultrasound imaging probe pressure).
• organ deformation arising from ultrasound imaging probe pressure.
• the physical nature of this organ motion in an individual subject suggests that an SMM approach is appropriate, but for cases where dynamic imaging is unavailable, generating a subject- specific SMM by performing biomechanical simulations of organ motion to provide training data is both a complex and time-consuming process, both computationally and from the point of view of implementing within a clinical workflow.
• the approach described herein enables a subject-specific SMM to be built without knowing the correspondence between subject subspaces - in other words, a relationship between the shapes and states of the organs of different subjects does not need to be established.
• this approach involves only limited subject-specific geometric data, for example, a reference shape based on the segmentation of one static image, to predict the organ motion for a new (unseen) subject.
• the application of this approach for deformable registration of MRI and transrectal ultrasound (TRUS) images of the prostate is discussed below.
• a new framework for statistical shape modelling of subject-specific organ motion including soft-tissue organ deformation, etc. is described herein.
• the approach adopted combines modelling of subject-specific probability of shape model parameters with kernel regression analysis to predict organ motion for an unseen subject.
• the approach is computationally efficient, uses only a limited number of training shape samples for each subject, and is highly suited to image analysis problems where prior subject-specific information on organ shape variation is useful, but where obtaining sufficient training data to construct a statistical shape/motion model for each new subject is impractical or technically or logistically difficult.
• subject-specific, MRI-derived statistical prostate motion models are generated and compared with models built using subject-specific training data provided by finite element simulations and mixed subject data that exhibits variation in shape due inter-subject variations and intra-subject prostate motion.
• the results from this investigation show that the two subject-specific models are generally equivalent in terms of output, and provide much higher specificity than can be obtained with a mixed model built using training datasets from a group of subjects.
• Some embodiments of the invention provide a method and an apparatus for building a subject-specific statistical model of shape variation for an anatomical structure using a set of geometric representations of shape for the anatomical structure.
• Each representation is associated with one of multiple subjects, each subject having a respective subset of associated representations, and each subset comprising multiple representations including a reference representation.
• the multiple representations correspond to different shapes of the anatomical structure arising from physical motion and/or deformation thereof.
• the method comprises specifying a set of shape parameters whose values, for any given representation, characterise the shape of the representation, and for each subject, representing a probability distribution of the values of the shape parameters across the subset of representations associated with that subject by a set of subject-specific distribution parameters.
• the method further comprises determining a regression between the subject-specific distribution parameters and the respective reference representation for each subject, and applying said determined regression to a reference representation for a new subject to determine subject-specific distribution parameters for the new subject.
• the subject-specific distribution parameters are then used to build a subject-specific statistical motion model for the new subject.
• the geometric representations may, for example, comprise images.
• the images may be three-dimensional, especially for representing the three-dimensional shape of the anatomical structure.
• the images may also be two dimensional; this may be appropriate, for example, in a facial recognition systems.
• the images may be obtained from one or more suitable imaging modalities, such as (without limitation) magnetic resonance imaging (MRI), X-ray computed tomography (CT), positron emission tomography (PET), ultrasound, photo-acoustic imaging (PAT), optical, X-ray or gamma ray imaging, optical microscopy, electron microscopy, etc.
• suitable imaging modalities such as (without limitation) magnetic resonance imaging (MRI), X-ray computed tomography (CT), positron emission tomography (PET), ultrasound, photo-acoustic imaging (PAT), optical, X-ray or gamma ray imaging, optical microscopy, electron microscopy, etc.
• the geometric representations may also be derived, for example, from biomechanical modelling.
• shape of an anatomical structure for a subject might initially be determined by measurement (whether from an image, or by any other appropriate physical measurement technique), and a model fitted to this shape based on known properties of the materials of the anatomical structure, such as elasticity, etc.
• the model allows the anatomical structure to move or deform in accordance with these known properties, thereby creating other physically realistic shapes for the anatomical structure. Representations of these shapes, in effect, instances of the model
• geometric representations can be provided in any appropriate format, for example, as an image generated from the model, or as a set of positions of particular features or landmarks on the anatomical structure according to the configuration of that particular model instance.
• the geometrical representations may be derived from a combination of sources, for example, from multiple different imaging modalities, and/or from biomechanical modelling.
• the set of shape parameters are determined using principal component analysis (PCA), which provides a compact representation of the domain or space of the geometric representations.
• PCA principal component analysis
• other embodiments may utilise different shape parameters. For example, if the representations are derived from biomechanical modelling, then the shape parameters might potentially be derived directly from (or correspond to) the values of certain model parameters.
• the anatomical structure may represent an organ, such as the prostate, liver, kidney or heart, comprising soft tissue. Modelling of such organs is typically performed for medical reasons.
• the modelling may support a non-rigid image registration, which allows a first image to be mapped (registered) to a second image.
• the first image might be a detailed pre-operative image, such as acquired via MRI
• the second image might be an intra-operative image, such as acquired via ultrasound.
• the modelling described herein can help the non-rigid image registration to accommodate, in a computationally efficient manner, deformation or motion of the organ, such as may arise from a change in posture of the subject, the presence of a surgical instrument, some change in physiological state, and/or any other source of deformation.
• the modelling described herein can also be utilised for non-medical purposes, such as facial recognition.
• Figure 1 is a schematic overview of the method used to build a subject-specific SMM in accordance with some embodiments of the invention.
• Figure 2 shows a schematic representation of the shapes of the prostates of different subjects. Each shape was generated using biomechanical modelling software to predict the physical deformation under different conditions.
• Figure 3 provides a graphical representation of the factorised probability density P(3 ⁇ 4) f° r three different subjects, as determined by a method in accordance with some embodiments of the invention.
• Figure 4 provides a comparison of the factorised probability density for three subjects, compared with for the population as a whole.
• Figures 5 and 6 provide a schematic overview of the main leave-one-out method for a chosen test subject, as part of a validation of a method in accordance with some embodiments of the invention.
• Figure 7 compares, for three test subjects, plots obtained using a regression model to provide a predicted subject-specific probability density function (solid line), compared with a histogram constructed using original data samples (dotted line).
• Figure 8 shows examples of random shape instances generated using (i) the ground-truth biomechanically-based SMM, (ii) the model-predicted subject-specific SMM of a prostate for the same subject in accordance with some embodiments of the invention, and (iii) the mixed-subject SSM, which captures the shape variation over the training population of 36 patient prostates.
• Figure 9, Figure 10 and Figure 1 1 are plots showing the median RMS distance values for the generalisation ability of (i) the model-predicted subject-specific SMM in accordance with some embodiments of the invention, (ii) the biomechanical-based subject-specific SMM generated using the (ground truth) biomechanical simulations, and (iii) the mixed-subject SSM, respectively, for each test subject.
• Figure 12, Figure 13 and Figure 14 are plots showing the median RMS distance values of the specificities of the three linear models, namely (i) the model-predicted subject-specific SMM in accordance with some embodiments of the invention, (ii) the biomechanical-based subject-specific SMM generated using the (ground truth) biomechanical simulations, and (iii) the mixed-subject SSM, respectively, for each test subject.
• Figure 15 shows a pair-wise example of registration of prostate surfaces and anatomical landmarks.
• SSPDF subject-specific probability density function
• FIG. 1 A schematic overview of the method used to build a subject-specific SMM is shown in Figure 1 .
• the stages involved are as follows:
• the SSPDF may itself be expressed in parametric form and represented by a number of distribution parameters - e.g. the mean and variance for a Gaussian distribution;
• the reference shape may describe an organ in its 'resting', un-deformed state, or at a time corresponding to a particular physiological event.
• the reference shape is then represented by its shape parameters;
• the main steps described above are also illustrated in Figure 1 .
• the resulting subject- specific SMM is an alternative to a subject-specific SMM built directly from training data available for this subject (including image-based and simulated training data). Therefore, the subject-specific SMM estimated using this method can be compared directly with one generated using the conventional method.
• an illustration of implementing these steps is provided using the example of building a subject-specific SMM of the prostate that captures deformation caused by the placement of an ultrasound probe in the rectum.
• Figure 2 shows a schematic representation of the shapes of the prostates of / subjects, as represented by triangulated meshes.
• the shape of each mesh has been simulated using finite element (FE) modelling to predict various physical deformations of a reference shape in response to the presence of a transrectal ultrasound (TRUS) probe.
• FE finite element
• TRUS transrectal ultrasound
• the 3D position and orientation of the TRUS probe is represented in Figure 2 by a shaded hollow cylinder for each deformed shape instance.
• the reference shape represents the prostate in the "resting state", obtained by segmenting a T2-weighted MR image that was acquired without an endorectal coil (or other rectal insertion) in place [12].
• the reference shapes are usually normalised to a consistent scale and orientation , so as to provide the most reliable comparability between the different images.
• group-wise surface registration of the meshes can be performed to establish point correspondence between : (i) each deformed shape and the reference shape for each subject, and ii) the reference shapes of different subjects.
• FE simulations are performed to determine the training dataset, the point correspondence between each deformed shape (and the reference shape) is known implicitly.
• the algorithm used to determine the cross-subject (inter-subject) point correspondence is described below.
• the mixed-subject SSM is constructed by applying principal component analysis (PCA) to
• the shape vectors may define either a 3D surface or a volume (for example, represented by the nodes of an FE mesh). Taking advantage of dimensionality reduction by excluding components that explain less variance in the training data, the resulting shape model is approximated by the linear equation using L ⁇ G principal components [3]:
• s is the mean shape vector; e ; is the eigenvector of the covariance matrix of the (mean- subtracted) training shape vectors corresponding to the I th largest eigenvalue, af ; and
• b gl is a scalar shape parameter; the vector b g contains the shape parameters that collectively describe the g th organ shape.
• Eq. (1 ) models mixed-subject individual and motion variations learned from all the training data.
• s and E estimates, from the overall data set, of distribution parameters that apply to the whole data set, and which are used to transform the shape properties for a given sample between two different coordinate systems - s g , which represents the coordinate system of the raw image data, and b g , which represents a reduced dimension coordinate system from the PCA.
• An SSM generated in this way is referred to as the mixed-subject SSM.
• Subject-specific PDF calculation is referred to as the mixed-subject SSM.
• the subject-specific probability density for the ( n subject is denoted by B £ e il , where ® ( is a multivariate random variable and i e .
• L denotes the i th subject subspace (hence the vector shape parameter b g , in respect of subject / ' , comprises samples from S £ ).
• b y contains the shape parameters of the training data by projecting the coordinates s y for the j th shape belonging to the i th subject.
• s y and s g are training shape vectors, where the different subscripts just denote differently grouped data.
• the curves shown on each axis represent the factorised probability densities for the respective principal component, whereas the ellipses represent bounding parameter values of the SSPDFs, P(S j ) based on both principal components.
• the probability density of all the training data that builds the mixed-subject SSM is denoted by P(3 ⁇ 4: 3 ⁇ 4 e D. g ), where the reference space D. g is the union of all the subject subspaces.
• Figure 4 shows some examples of these factorised probability densities using the histograms of the samples ⁇ b y ⁇ from the prostate shape data.
• P(3 ⁇ 4) is different between subjects (i.e. for different values of (/)), and P(3 ⁇ 4) is also different from P(3 ⁇ 4;), corresponding to the mixed- subject (population) SSM.
• the sample distributions have a generally bell-like shape with different widths and centre positions.
• This PDF is considered as a parametric example of the SSPDF for i th subject, and is entirely characterised by the distribution parameters ⁇ , and of .
• the probability distribution associated with a given principal component approximates to a Gaussian distribution which can be parameterised by two standard scalar quantities, namely mean ( ⁇ ) and variance ( ⁇ 2 ).
• K(x, x') exp(-
• the optimal regression parameter for a given data set (e.g. for the set of data illustrated in Figure 2) may be obtained by using a linear least squares technique to minimise the regularised residual sum-of-squares as follows [14]:
• the kernel parameter h is determined by a cross validation method which is described below.
• the shape parameters b new 0 for the new subject can be estimated by first non-rigidly registering to the mean shape of the group-wise registration [1 ], and then projecting onto the principal components of the mixed-subject SSM after removing the rigid component.
• the linear model may be obtained directly by "centering" the predicted diagonal covariance matrix, so that the predicted subject-specific SMM takes the form:
• an optimal value is computed by minimising the cross validation error, defined as the root-mean-square of the regression residuals, as in Eq.(5).
• the regression error is computed for each data in a leave-one-out scheme by comparing the difference between the ground-truth distribution parameters, computed from the training data via Eqs.(3) and (4), and the predicted distribution parameters, computed from the test data via Eqs. (6) and (8).
• a golden search strategy was used to then find the optimal value of x within the predefined interval 1 ⁇ x ⁇ 8, with the cross validation error serving as the objective function to minimise.
• a Golden Search also known as a Golden Section Search, represents a standard numerical optimisation algorithm; the theory is described in Kiefer, J. (1953), "Sequential minimax search for a maximum", Proceedings of the American Mathematical Society 4 (3): 502-506, doi:10.2307/2032161 , while an example implementation is described in Press, WH; Teukolsky, SA; Vetterling, WT; Flannery, BP (2007), “Section 10.2. Golden Section Search in One Dimension", Numerical Recipes: The Art of Scientific Computing (3rd ed.), New York: Cambridge University Press, ISBN 978-0-521 -88068-8). Point Correspondence
• One of the advantages of the modelling technique described herein is that it does not require a timing or phase correspondence (such as the position in a breathing cycle) to be established between the motion subspaces. Since only the probability densities are modelled to describe the subject motions, and the motion data can be grouped in an arbitrary order in the training dataset, which overcomes a number of practical difficulties. However, point correspondence still needs to be established between subject subspaces. In other words, a set of predefined features (e.g.
• anatomical landmarks are specified for an organ (corresponding to the points), and for each image or data set, the positions of these features (points) in the image are determined. This then allows the point correspondence to be determined, i.e. for each point or feature, the (corresponding) position of that feature in each image or data set is determined.
• This point correspondence may be estimated, for example, by group-wise surface registration [1 ].
• inter-subject registration of the training shapes, required to build the mixed-subject SSM was performed using an iterative group-wise registration scheme based on the landmark-guided coherent point drift (LGCPD) method [15], where anatomical apex and base points of the prostate serve as two known corresponding points to assist the registration in finding realistic point correspondence between gland surfaces.
• LGCPD landmark-guided coherent point drift
• Figures 5 and 6 illustrate the leave-one-out method for a chosen test subject, where the three linear models are constructed independently.
• the root-mean-square (RMS)-distance-based generalisation ability and specificity then can be computed for each test subject, as per Figures 5 and 6 respectively.
• the cross-validation method described below provides an overall assessment of the modelling ability. Low RMS distance indicates strong generalisation ability and specificity for the linear model.
• the generalisation ability of a linear model quantifies the ability of the model to describe unseen data, and therefore relates closely to a common application of such models, namely capturing organ motion to provide prior information for registering to unseen data.
• the generalisation ability was measured by a separate, embedded leave-one-out scheme [17].
• the generalisation ability was defined as the RMS Euclidean distance between the mesh nodes of an unseen test dataset and the corresponding nodes of the instantiated model fitted to the test data (i.e. the fitted model).
• the RMS-distance-based generalisation ability is given by:
• N is the number of the mesh nodes of each model
• s test and s fitted are the shape vectors (as defined above) of unseen test data and the instantiated model, respectively.
• the unseen test data was the remaining (left-out) data of the 100 biomechanical simulations of the test subject, in the embedded leave-one-out scheme.
• the biomechanically-based SMM was then built independently using the remaining 99 simulations, as illustrated in Fig. 5.
• the generalisation abilities were computed for the three linear models as mentioned above (subject-specific SMM,
• biomechanically based SMM and mixed-subject SSM) , all based on the subject-level leave-one-out scheme.
• the accuracy of registering a deformable, model-predicted subject- specific SMM, which is based on MR-derived prostate geometry data, to 3D TRUS images was investigated by quantifying the target registration error (TRE) in the alignment of manually-identified, independent anatomical landmarks for 8 patient datasets following registration using the method described in [12].
• the data for these 8 patients was independent of the training data used to build the predictive model.
• This TRE provides an independent measure of the registration performance that can be compared directly with registrations that make use of SMMs that were built using
• Fig. 7 shows example plots of the factorised P(3 ⁇ 4) for the first four predicted SSPDFs for three new test subjects (solid line - bottom plot), compared with the histogram constructed using the original ⁇ by ⁇ (dotted line - top plot). It can be seen that the corresponding curves show excellent agreement between the dotted lines representing P(3 ⁇ 4) of the data used in this study and the solid lines representing the regression-estimated subject-specific probability density curves for the first four principal components for the three patients. The goodness-of-fit between the corresponding curves was evaluated using the X 2 test. The result - an average p>0.78 - indicates excellent agreement, which provides justification for the effectiveness of the kernel regression analysis and the choice of the Gaussian form to model the PDFs in this study.
• Figure 8 shows examples of random shape instances generated using (i) the
• the top row of Figure 8 comprises randomly sampled prostate glands from the ground-truth biomechanically-based SMM of a test subject (as in the leave-one-out validation); the middle row comprises randomly selected samples from the model-predicted subject-specific SMM, which are constructed from data excluding the test subject; and the bottom row comprises randomly selected samples from the mixed subject SSM, which includes both intra- and inter-subject shape variations in the training data.
• the first column of Figure 8 shows the reference shape from each model.
• Figure 9 Figure 10 and Figure 1 1 are plots showing the median RMS distance value for the generalisation ability of (i) the model-predicted, subject-specific SMM, (ii) the biomechanical-based, subject-specific SMM generated using the (ground truth) biomechanical simulations, and (iii) the mixed-subject SSM, respectively, for each test subject.
• the error bars indicate the 5th/95th percentiles of these RMS distances. From inspection of these plots, it can be seen that the two subject-specific SMMs provide lower RMS errors (distances) compared with the mixed-subject SSM.
• paired Kolgomorov-Smirnov tests confirm that 1) the mixed-subject SSM has significantly worse generalisation ability than the model-predicted SMM and the
• the proposed model-predicted SMM has a generalisation ability for unseen data which is comparable with that of the original biomechanically-based SMM, while both of these subject-specific models outperform the mixed-subject SSM in this regard (i.e. for generalisation ability).
• Figure 12 and Figure 14 are plots showing the median RMS distance values of the specificities of the same three linear models, namely (i) the model-predicted subject-specific SMM, (ii) the biomechanical-based subject-specific SMM generated using the (ground truth) biomechanical simulations, and (iii) the mixed-subject SSM, respectively, for each test subject. Again, the error bars indicate the 5th/95th percentiles of these RMS distances. Comparing Figure 12, Figure 13 and Figure 14 reveals that the subject-specific SMMs provide significantly smaller (better) RMS-distance-based specificities.
• Table 1 Summary of Registration TREs using predicted SSMMs and SMMs
• This approach described herein provides a new framework for modelling subject-specific organ motion, in which learnt statistics from a training population are used to predict subject-specific training data for an unseen patient, rather than requiring those data to be provided either by dynamic imaging or computer simulation.
• the disclosed method allows subject-specific organ motion to be modelled implicitly, without knowing the explicit motion correspondence between data sets from different individuals.
• the motion modelling method described herein has been compared with biomechanical modelling as a means of generating subject-specific synthetic training data (rather than using dynamic imaging).
• biomechanical simulations One advantage of using biomechanical simulations is that the point correspondences between successive shapes of the organ of a particular subject are known implicitly, since these are computed relative to a common reference shape.
• a significant advantage of the approach described herein is that, since the method is based on modelling SSPDF and kernel regression analysis, only limited subject-specific data on organ shape change during motion is required. Consequently, the method is computationally efficient and highly suited to applications where comprehensive data on organ motion are difficult or impossible to acquire, for example during a surgical intervention.
• the approach disclosed herein is also helpful in situations where imaging is feasible but has significant practical constraints (for example, limited frame rate), meaning that that only a small number of training shape instances may be obtained.
• the use of just a single reference shape for unseen subjects overcomes practical constraints that are commonly encountered in the clinical setting, since the reference image can be obtained from diagnostic or planning images, which are widely available in the clinical setting (as in the example provided herein).
• the approach described herein has primarily used subject-specific SMMs to describe the motion of the prostate gland alone, the approach is also applicable to other organs as appropriate, including multi-organ motion. Furthermore, the approach may be easily adapted to use an arbitrary probability function, such as a mixture model if multi-modal distribution is observed, or a different regression technique. Although such adaptations would not result in a straightforward linear model directly, random samples of the subject-specific organ shape could then be drawn from the learnt SSPDF, e.g. via a Monte Carlo approach, and then used to build a linear SMM using standard SSM techniques.
• the approach described herein supports the use of group- and pair-wise landmark-guided coherent point drift (LGCPD) algorithms to non-rigidly register training shapes.
• Figure 15 shows a pair-wise example of such registration, in particular, an example of pair-wise registration of prostate surfaces and anatomical landmarks (apex and base) using the original coherent point drift (CPD) and LGCPD algorithms. It can be seen that the landmarks are better aligned (right) after using the LGCPD algorithm, compared with using the CPD algorithm (middle). Accordingly, the approach described herein provides a faster and more robust extension to general- purpose CPD.
• CPD coherent point drift
• the value of L in Eq. (1) may be chosen so that the reference SSM covers of a certain percentage of the cumulative variance (e.g. at least 99%) in the training data.
• the approach described herein may provide an alternative method to determine an optimal value of L, as components with decreasing variance may contain too much noise to be reasonably modelled by Gaussian or captured by kernel regression.
• multiple subject-specific images are generated using a biomechanical model to deform a single reference image for a given subject.
• the multiple subject-specific images may be acquired via other techniques, including: (i) acquiring a "video" of the subject, i.e. a relatively rapid succession of images frames (2-D or 3-D); or (ii) acquiring multiple (still) images of the subject over a period of time.
• a "video" of the subject i.e. a relatively rapid succession of images frames (2-D or 3-D
• acquiring multiple (still) images of the subject over a period of time e.g. changes in organ shape over time may reflect processes other than motion/deformation (e.g. growth or decay) - it may or may not be beneficial for such processes to be incorporated into the model, depending on the details of the desired application.
• the multiple subject-specific images for any given subject may be acquired from two or more different techniques (such as using both biomechanical modelling and video data), and different techniques (or combinations of techniques) may be used for different subjects.
• different techniques or combinations of techniques
• the use of a biomechanical model to deform a single reference image for a given subject has the advantage that once the point correspondence has been determined for the reference image, this can then be automatically preserved through the
• biomechanically modelling for the other images obtained thereby For video data, once a point correspondence has been determined for one image (image frame) in the video, then it is generally feasible to preserve the point correspondence through difference frames on an automated or semi-automated basis, such as by determining motion vectors from one image frame to the next.
• the approach described herein significantly reduces the time required to build a subject- specific SMM, which is important in many applications, such as modelling prostate motion to enable deformable registration of preoperative MR images to intraoperative TRUS images.
• approach described herein reduces the computational time to both constrain the registration and predict tissue displacements inside the prostate, given surface displacements, from a few hours using a GPU-based FE simulator to a just few seconds without compromising registration accuracy.
• Another potential application of the method disclosed herein is to decompose a conventional SSM using any subject-specific measurements, if available, to provide better modelling ability.
• the model might be used, inter alia, to segment the organ from a second image or to perform an image registration between the second image and the reference image (so that the corresponding positions in the reference and second images for a specific location on or in the organ are determined).
• subject identification such as in the context of facial recognition, where there are multiple subjects, each having a respective reference image, and it is to be determined whether or not a newly acquired image matches one of these reference images (and if so, which one).
• the subject-specific SMM may be used to try to map (register) each reference image to the newly acquired image, and a match is determined for a given reference image if such registration is successful (statistically plausible) within the parameters of the SMM.
• the above embodiments involving various data (signal) processing may be performed by specialised hardware, by general purpose hardware running appropriate computer code, or by some combination of the two.
• the general purpose hardware may comprise a personal computer, a computer workstation, etc.
• the computer code may comprise computer program instructions that are executed by one or more processors to perform the desired operations.
• the one or more processors may be located in or integrated into special purpose apparatus, such as a medical imaging system (MRI, ultrasound, etc).
• the one or more processors may comprise digital signal processors, graphics processing units, central processing units, or any other suitable device.
• the computer program code is generally stored in a non-transitory medium such as an optical disk, flash memory (ROM), or hard drive, and then loaded into random access memory (RAM) prior to access by the one or more processors for execution.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Theoretical Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Software Systems (AREA)
• Probability & Statistics with Applications (AREA)
• Databases & Information Systems (AREA)
• Artificial Intelligence (AREA)
• Computing Systems (AREA)
• Health & Medical Sciences (AREA)
• Evolutionary Computation (AREA)
• General Health & Medical Sciences (AREA)
• Medical Informatics (AREA)
• Multimedia (AREA)
• Measuring And Recording Apparatus For Diagnosis (AREA)
• Image Analysis (AREA)
• Magnetic Resonance Imaging Apparatus (AREA)
• Ultra Sonic Daignosis Equipment (AREA)
• Image Processing (AREA)

Priority Applications (5)

Applications Claiming Priority (2)

Publications (1)

Family

ID=50490747

Family Applications (1)

Country Status (7)

Cited By (2)

Families Citing this family (10)

Family Cites Families (10)

• 2014
  • 2014-03-04 GB GB201403772A patent/GB201403772D0/en not_active Ceased
• 2015
  • 2015-03-03 CA CA2940992A patent/CA2940992C/en active Active
  • 2015-03-03 AU AU2015225921A patent/AU2015225921A1/en not_active Abandoned
  • 2015-03-03 JP JP2016555581A patent/JP2017512522A/ja not_active Ceased
  • 2015-03-03 US US15/120,991 patent/US10115201B2/en active Active
  • 2015-03-03 WO PCT/GB2015/050608 patent/WO2015132575A1/en not_active Ceased
  • 2015-03-03 EP EP15709309.7A patent/EP3114645B1/en active Active

Non-Patent Citations (4)

Also Published As

Similar Documents

Legal Events