EP1869620A1

EP1869620A1 - Method for processing image and volume data based on statistical models

Info

Publication number: EP1869620A1
Application number: EP06724278A
Authority: EP
Inventors: Andreas Schilling
Original assignee: Eberhard Karls Universitaet Tuebingen
Current assignee: Eberhard Karls Universitaet Tuebingen
Priority date: 2005-04-12
Filing date: 2006-04-12
Publication date: 2007-12-26
Anticipated expiration: 2026-04-12
Also published as: WO2006108633A1; US20090010506A1; DE502006007692D1; ATE478395T1; US8520926B2; EP1869620B1; DE102005016955A1

Abstract

A method for visually displaying and/or evaluating measurement data from imaging methods involve the following acts: a) calculating a parameterized statistical model from example voxel data sets that map different objects of an identical object class; b) carrying out at least one imaging method on an object to be examined of the object class in order to extract real measurement data; c) setting a set of model parameters of the parameterized statistical model; d) determining a difference between the real measurement data and the parameterized statistical model; e) repeating steps c) and d) while changing the model parameters until the difference between the real measurement data and the parameterized statistical model is minimal; and f) visually displaying and/or evaluating the statistical model parameterized in aforementioned manner.

Description

Method for processing image and volume data based on statistical models

In medicine, various imaging techniques are available that visualize the internal structures of a body to be examined. For a long time there have been X-rays, which provide a two-dimensional image of the transilluminated structures, other methods, such as (computed tomography) CT or (magnetic resonance) MR tomography can also provide three-dimensional, so-called voxel data sets. The term voxel is made up of the terms "volumetry" and "pixels". For a voxel spatial data set that is in discretized form in Cartesian coordinates, a voxel corresponds to an associated discrete value at an XYZ coordinate of the data set. A voxel thus forms a three-dimensional equivalent of a pixel. Usually, the data contained in a voxel data set are scalar quantities, for example intensity values or color values, which are intended for visualization with the means of the volume graphic.

Unfortunately, these latter methods are still expensive and time-consuming today or (in CT) associated with higher radiation exposure. Therefore, often even though - e.g. in surgical planning - three-dimensional images would be desirable to use normal two-dimensional radiographs. Although it does not contain depth information, physicians can visualize the depicted three-dimensional structures based on their experience.

The object of the invention is to make available a method for visualizing and / or evaluating measurement data from imaging methods, which enables a powerful evaluation of the measured data and, in particular, the calculation of volume models from the measurement data of two-dimensional recordings. The invention solves this problem by a method according to claim 1.

According to the invention, the method comprises the steps of: a) calculating a parameterized statistical model from exemplary voxel data sets which map different objects of an identical object class, b) performing at least one imaging method on an object of the object class to be examined for obtaining real measurement data, c Setting a set of model parameters of the parameterized statistical model, d) determining a deviation between the real measurement data and the parameterized statistical model, e) repeating steps c) and d) changing the model parameters until the deviation between the real measurement data and the parameterized statistical model is minimal; and f) visualizing and / or evaluating the statistical model parameterised in this way. The steps a) to f) are preferably carried out in the order mentioned.

In other words, in the method for processing measurement data from imaging methods from example voxel data sets, a parametric statistical model is calculated, model parameters for this model are determined so that data calculated from the model optimally match the measured data, and the model thus obtained is output for visualization or further processing.

Among other things, the method according to the invention makes it possible to calculate volume models from two-dimensional images (for example X-ray images or individual recorded slice images) by calculating the most probable configuration which could have led to the recording with the aid of a statistical model generated from sample data sets. The experience of the doctor, who can imagine a three-dimensional configuration that led to a certain image, corresponds here to the data obtained from statistical data. Corresponding techniques have hitherto been used only in the field of modeling two-dimensional images [1] or three-dimensional surface models (Morphable Models [2]), but not for solid models.

A refinement of the method comprises the steps of: d) calculating virtual measurement data from the parameterized statistical model, and d1) determining the deviation between the real measurement data and the parameterized statistical model by determining a deviation between the real measurement data and the virtual measurement data. Step c1) is preferably carried out after step c) and before step d), and step d1) is preferably carried out after step d) and before step e).

In a development of the method, the example voxel data sets are obtained from CT and / or MR voxel data.

In a development of the method, the real measurement data are obtained on the basis of one or more X-ray images.

In one development of the method, the real measured data are obtained from data which has not yet been backprojected, of one or more CT and / or MR recordings.

In one development of the method, the real measured data is obtained from voxel data, for example from backprojected data of one or more CT and / or MR recordings.

In one development of the method, a reference data record is calculated from the parameterized statistical model, the measured data are registered with the reference data record and those model parameters are calculated which represent the model which best matches the measured data. In a further development of the method, the parameterized statistical model is obtained from a linear combination of example vectors, wherein a respective example vector is assigned to a respectively associated exemplary voxel data set and components of the respective example vector describe position and intensity of volume elements of the associated example voxel data set. The example vectors are determined based on the example voxel data sets. In order to determine or calculate the example vectors, a parameter reduction can be carried out in addition to a reparatization.

In a development of the method, a vector space spanned by the example vectors is reparametrised.

In a development of the method, the evaluation of the parameterized statistical model in step f) comprises a detection of anomalies in the real measured data.

The method according to the invention or specific substeps of the method can preferably be carried out on special hardware, for example on the basis of programmable logic modules.

The invention will now be described by way of typical application scenarios and with reference to the accompanying drawings. Here are shown schematically:

1 shows a block diagram of a device for generating parameterized statistical volume models,

2 shows a block diagram of a device for calculating a volume model from one or more simple X-ray images or other measurement data sets and 3 shows a block diagram of a device for analyzing the model parameters of a parameterized statistical volume model with orthogonal basis vectors from a measured complete data set.

Calculation of a statistical volume model from sample data sets

To compute the statistical volume model, sample volume datasets Vj (j = 1 -.n) are needed that represent all objects of the same object class. Here, first CT-voxel data sets are to be considered, whereby corresponding models can also be generated for MR tomography recordings. The fact that the objects all belong to a common object class means that for each object point in a data record in every other data record a point can be identified that represents a corresponding object feature. In a first step, these correspondences are identified; Generally, this step is called registration of the models. This can be done manually by marking corresponding points in the example data sets and interpolating the intermediate point positions, but also by automatic methods, see e.g. [3] and [4].

Subsequently, a vector B _{j is} calculated from each example data record, which fully describes the data record and is referred to below as an example vector. For example, the first of these example vectors B ₁ may be generated by cascading the first of the example data sets for each voxel with x, y, and z positions and the voxel intensity as vector components, whereby the vector has a number of components corresponding to the vector Four times the voxel number. For all other example data records, the example vectors are formed by the fact that, for each voxel of the first example data record, the cor- responding voxels of the further example data set is determined, and whose x-, y-, and z-position and its voxel intensity as vector components are written one after the other. Such generated vectors can now be combined linearly. The linear combinations represent new volume data sets consisting of positions and intensities. By means of interpolation and resampling, regularly sampled voxel data sets can also be generated from such linear combinations.

The arbitrariness in the assignment of the example vector components to the voxels of the arbitrarily selected first example data set can be removed by repeating the calculation method, wherein the second pass instead of the arbitrarily selected first example data set is a voxel data set calculated from the mean vector of the example vectors for correspondence calculation and example vector generation is used.

In a second step, the vector space spanned by the example vectors B _{1 is} reparametrised. For example, as described in [2], a principal component analysis (PCA) can be used. In the PCA, eigenvectors and eigenvalues of the covariance matrix of the example vectors are calculated after the coordinates of the vectors

1 "~ were transformed such that the mean vector ß ^ -Yß of the

Example vectors in the origin lies. It is known that these can be computed efficiently for small n without explicit computation of the high-dimensional covariance matrix. Linear combinations of the vectors can now be represented as the sum of the mean vector B ₀ and the parameter α, weighted eigenvectors B _{1 of} the covariance matrix, which thus represent a new basis for the vector space of the linear combinations of the example vectors. Assuming that the sample datasets are at most (ni) -dimensional Normal distribution, this distribution can be estimated from the example data sets. In this case, the eigenvalues of the covariance matrix belonging to the eigenvectors represent the variance of the projection of the transformed example vectors on the associated eigenvectors and are used to calculate the probability of their occurrence for specific parameter combinations.

If the sample objects were not already positioned and aligned during the recording, it makes sense before the main component analysis to rotate and move the location coordinates contained in the example vectors so that the position and orientation of the example objects match as well as possible. For the invention described, it does not matter if this happens manually or with automatic methods such as fitting moments. If a viewed object consists of multiple subobjects that can be moved relative to each other (such as multiple bones connected by joints), then that alignment should be performed separately for the subobjects. The parameters of this orientation transformation (eg rotation angles and displacement vectors) can also be understood as vectors that describe the position and orientation of an object or position and orientation of the subobjects, and whose linear combinations also form a vector space, which can also be used with linear or nonlinear techniques such as eg PCA or kernel PCA can be reparametrized. The goal of the reparamethsation, which can also be done separately for subobjects, is the determination of parameters that can describe combinations of the example vectors in such a way that these parameters can be ordered according to their importance. In PCA, the importance of a parameter can be defined by the mean square error that occurs when the corresponding parameter is omitted in the description of the example vectors in the coordinate system given by the PCA. Parameters in which the omission does not cause an error in the Representation of the example vectors generated can be omitted. The remaining parameters provide a redundancy-free representation of the example vectors and - if they represent well the entirety of the volume data records of an object class - also a low-redundancy display possibility for volume data sets of any new object from the considered object class.

Once a statistical model has been obtained in this way, any combination of the original sample data sets can be generated by setting the corresponding parameters. The resulting data sets can be visualized using known volume visualization techniques. For example, it is very easy to calculate an image that corresponds to a normal X-ray image by integration along the radiation lines emanating from a virtual X-ray source from a CT data set.

Calculation of a volume model from an X-ray

An important task which can be solved with the described invention is the calculation of a volume model from a simple X-ray image. This happens in the following way:

a) The statistical parametric volume model is initialized so that the average object is displayed. Normally the parameterization will be chosen so that the average object is represented by the fact that all parameters α have the value 0. Alternatively, an initial parameter set can be set manually.

b) Of this model with the set parameters α. a virtual X-ray is generated. The display parameters (position of the radiation source and position and orientation of the image plane) should be selected similar to the normal X-ray image to be analyzed. c) The virtually generated X-ray image is compared with the real normal X-ray image. For example, the sum of the squared intensity differences can serve as a comparison measure.

d) Now, in an iterative manner, by repeatedly going through steps 2 and 3 with changed model parameters α, and visualization parameters (position of image plane and radiation source), the match between the virtually generated and the real X-ray image is improved. For the adaptation of the parameters, various numerical standard optimization techniques can be used, such as e.g. various hierarchical gradient-based methods, simulated annealing or simplex methods [5].

This procedure corresponds to an analysis-by-synthesis procedure, as it is e.g. is known for the analysis of two-dimensional images [6].

The result of the method is a set of model parameters that characterize the specific volume model represented by the statistical parameterized volume model which best fits the analyzed real X-ray image. This model can now be displayed with standard visualization techniques, eg it is possible to view it from all sides. The procedure can be improved if two or more images are used instead of a single real X-ray image. The model must then be adjusted in the analysis phase in such a way that both recordings agree with their corresponding virtual recordings. For this purpose, in step 3, the sum of the squared intensity differences between each of the recorded images taken and the calculated image corresponding thereto can be used as a comparison measure. The remaining steps remain unchanged. Basically, with the aid of the statistical model, the most plausible model based on the example data sets can be calculated, which led to specific recordings. If a match can not be achieved in certain image areas, the reason is that no model that can be represented as a combination of the example data can produce an image corresponding to the captured image. Areas in which high differences occur can, for example, be color-coded, since they represent conditions deviating from the normal. The same applies if individual model parameters assume sizes that are far outside the variance given by the example data.

If multimodal sample data sets are used for the calculation of the statistical model, further possibilities for the analysis of recorded data arise. If the sample data set contains CT data, e.g. As well as various MR data, an X-ray image can not only be used to determine a CT data record that matches this X-ray image, but also the most plausibly suitable MR data is automatically available for this purpose.

The analysis method described for X-ray images can be applied not only for normal X-ray images, but for all representations that can be calculated from the parameterized model. For example, instead of an X-ray, a single CT scan can also be used. The matching model is determined as described for the X-ray images and represents the model that fits best to this section according to the statistics given by the example data. Another very important application is the analysis of raw data as obtained in the CT scan. Since such representations can also be computed from volume data, it is possible to compute high quality CT images even from a smaller amount of acquired data without performing inverse radon transformation (filtered backprojection). It only needs to be considered here that the sample data sets have enough variation must be able to properly represent the circumstances. If this is not the case, at least a deviation of the measured data from the data derived from the model can be detected and treated separately.

Analysis of complete volume datasets

Also, the analysis of complete volume data sets makes sense. It is done by registration, as used in the creation of the statistical model. For linear vector spaces and a parameterization by PCA, the parameter determination is particularly simple and takes place by simple projection of the resulting vectors onto the main axes determined by the PCA and subtraction of the projections of the average vector of the example vectors on these main axes.

If the parameterized statistical model is not a linear model with orthogonal basis functions, but has e.g. With the aid of non-linear parameter reduction techniques, a complete volume data set can also be analyzed with the method described above for calculating a volume model from an X-ray image, if the complete data record to be analyzed is used instead of the X-ray image. The step of calculating a virtual recording can be omitted here, since the measured volume data set can be compared directly with the volume data set generated from the model parameters.

If the statistical model is multimodal, the other modality can be generated with the parameters thus determined; MR data can be used to calculate a CT representation and vice versa. However, it should be noted that this representation is only a plausible model and not a measurement. Of particular interest is the application in the diagnosis of anomalies, as these are in strong Express deviations of the measured data from the model data. With example data labeled with labels, it would even be possible to automatically assign certain groups (gender, age, specific illnesses) by determining the parameters for measured data. For such diagnostic purposes, however, would offer other parametrizations; In order to distinguish between two groups (eg certain disease present / not present), a discriminant analysis should be carried out instead of the principal component analysis.

Device for generating a parameterized statistical volume model

FIG. 1 shows, by way of example, the block diagram of a device for generating parameterized statistical volume models.

Via an input port 11, sample voxel data sets Vj (j = 1..n) of an object class are loaded into a memory unit 12 for example voxel data sets. From there, the sample voxel data sets are passed to a registration and sample vector calculation processing unit 13, where dot correspondences between the data sets are determined, and then from these correspondences an example vector is computed for each example voxel data set.

Then the average vector B _{0 of} all example vectors is calculated. true, and then subtracted from each example vector. The resulting difference vectors B ₁ = B ₁ -B ₀ are forwarded to a processing unit 14 for reparametrization by PCA, where they are subjected to principal component analysis. The result is a parametric statistical volume model that consists of the vector P ₀ = B ₀ and the eigenvectors P _{1 of} the covariance matrix of the difference vectors, as well as from the associated eigenvalues v. This model can be output via an output port 15, or forwarded to a processing unit 16 for generating a voxel model for specific parameters α, where parameters α, α and β are input via an input port 17

the vector vectors P _1, the sum vector V ₀ = B ₀ + ^] U ₁ P _{1 is} calculated, i represents a defined by the parameters α, concrete volume model. From this sum vector, a voxel model on a regular grid is then calculated by resampling, which can be output via an output port 18.

Device for determining model parameters from one or more X-ray images or other measurement data sets

2 shows, by way of example, the block diagram of a device for calculating a volume model from one or more simple X-ray images or other measurement data sets.

Via an input port 21 is a parameterized statistical volume model in the form of the vectors P ₁ (i = 0..n) and the associated variances V ₁ (i = 1..n) and the measured data M, for example in the form of a simple

X-ray image loaded into a storage unit 22 for volume model and measurement data. For the following description, it is assumed that the measurement data M represents a simple X-ray image. The vectors P ₁ are forwarded to a processing unit 23 for generating a voxel model for certain parameters α. This processing unit is supplied with parameters by a processing unit 24 for optimizing the parameters a _t and the presentation parameters.

From these and the vectors P ₁ , it calculates a voxel model that is sent to a processing unit 25 for generating virtual measurement data M '. is passed from a voxel model. In this processing unit, which can consist of a conventional voxel visualization device, for example, and which also receives the display parameters from the processing unit 24 for optimizing the parameters α and the display parameters, a virtual measurement data record M ', for example a virtual X-ray image, is generated and sent to a processing unit 26 for the calculation of a similarity measure between measured data M and virtual measurement data M 'passed. In this processing unit, the virtual X-ray image is compared with the recorded X-ray image to be analyzed from the storage unit 22. This comparison measure is passed on to the processing unit 24 for optimizing the parameters and the presentation parameters, where according to an optimization algorithm iteratively, starting with an initial parameter set, new parameters a _t and new presentation parameters are generated until the comparison measure reaches an extreme value indicating in that the maximum possible match between real recorded and virtually generated measured data set was achieved. This algorithm may also rely on the variances stored in the volume model and measurement data storage unit 22. The initial parameters α, are either fixed default values (eg 0) or alternatively can be entered via an input port 27. The inital representation parameters should correspond to the acquisition parameters of the measurement data and are likewise input via the input port 27. With multiple sets of presentation parameters, the device can also be used to simultaneously analyze multiple X-ray images of an object.

Once the iteration has come to an end, the voxel model calculated from the optimized parameters is available at an output port 28, which can be visualized using standard visualization methods for voxel models. It makes sense to use a device for this visualization, which corresponds to the processing unit 25 for generating virtual measurement data M 'from a voxel model. At an output port 29 are the optimized model parameters that represent the calculated optimized model very compact, and can also be used to determine the plausibility of the calculated model using the probability distribution given by the statistical model.

Device for analyzing the model parameters of a parameterized statistical volume model with orthogonal basis functions from a measured complete data set

3 shows, by way of example, the block diagram of a device for analyzing the model parameters of a parameterized statistical volume model with orthogonal basis vectors from a measured complete data set.

Via an input port 31, a parametric statistical volume model in the form of the orthogonal normalized vectors P ₁ (i = 0..n) and the measured complete volume data set K are loaded into a volume model and measurement data storage unit 32. The measured complete volume data set K is then registered in a processing unit 33 for registering K with the volume data set represented by the mean vector P _{0 of} the parameterized statistical model. After this registration, as with the generation of the parameterized statistical model, a vector is calculated from the correspondences and the intensities of K that represents the measured complete volume data set to be analyzed. This vector is then transformed by subtracting P ₀ into the coordinate system of the parameterized statistical model. The transformed measured data vector VA "thus calculated is forwarded to a processing unit 34 for the calculation of the model parameters α, where the parameters α, by projecting onto the or- normal basis vectors P _{1 of} the parameterized statistical volume model. The thus calculated parameters a _t are then output via an output port 35.

To summarize, the described method is for processing data of an object that has been processed by imaging techniques such as imaging. MR tomography, CT, or simple X-ray images were obtained by creating a parameterized statistical volume model from sample volume data sets or sample voxel data sets and fitting the model parameters to the data obtained. The creation of the parameterized statistical volume model from example voxel data sets is carried out by manual or automatic registration of the volume or voxel data records, storage of the correspondences as high-dimensional vectors and subsequent parameter reduction. The example voxel data sets may contain additional semantic information such as e.g. generated by manual segmentation. The adaptation of the model parameters to the data to be processed obtained by means of imaging methods is carried out by using analysis-by-synthesis method. The result is a volume model of the recorded object which can be visualized using standard methods on the basis of the example data records. Applications are e.g. the generation of a three-dimensional volume model from a single or a few individual X-ray images, or the automatic detection of unusual structures by comparing the data calculated from the statistical model with the recorded volume data.

developments

A further development of the described methods and devices in various respects is possible. For example, for the determination of initial parameter sets in the analysis of data similar to [6], machine-learning methods can be used. references

[1] T.F. Cootes, GJ. Edwards, and CJ. Taylor. Active appearance mode. In Burkhardt and Neumann, editors, Computer Vision - ECCV'98Vol. II, Freiburg, Germany, 1998. Springer, Lecture Notes in Computer Science 1407.

[2] V. Blanz, T. Vetter. Method and device for the processing of images based on morphable modeis, EP1039417.

[3] D.L. Collins, P. Neelin, T.M. Peters, and A.C. Evans, "Automatic 3D intersubject registration of MR Volumetry data in standardized Talairach space," Journal of Computer Assisted Tomography, Vol. 18, pages 192-205.

[4] J. Modersitzki. Numerical Methods for Image Registration. Numerical Mathematics and Scientific Computation. Oxford University Press (2004).

[5] James C. Spall. Introduction to Stochastic Search and Optimization, ISBN: 0471330523, Wiley lntercience, 2003.

[6] A. Schilling. Tübingen Perception Conference, Feb. 2005, Symposium 1 Symposium 1: Analysis by Synthesis Methods in Computer Vision and Perception, 2005.

[7] Cris Curio. Analysis by Synthesis for Human Pose Tracking: Merging View-Based and Model-Based Approaches, in: H.Bülthoff, H. Mallot, R. Ulrich, F. Wichmann (ed.), Proceedings of TWK05: 8th Tuebingen Perception Conference , Feb. 2005, Symposium 1 Symposium 1: Analysis by Synthesis Methods in Computer Vision and Perception, 2005.

Claims

claims

1. A method for visualizing and / or evaluating measurement data from imaging methods, characterized by the steps of: a) calculating a parameterized statistical model from example voxel data sets that represent different objects of an identical object class, b) performing at least one imaging method on a c) setting a set of model parameters of the parameterized statistical model, d) determining a deviation between the real measurement data and the parameterized statistical model, e) repeating steps c) and d) while changing the object class Model parameters until the deviation between the real measured data and the parameterized statistical model is minimal, and f) visualizing and / or evaluating the thus parameterized statistical model.

2. The method according to claim 1, characterized by the steps of: d) calculating virtual measurement data from the parameterized statistical model, and d1) determining the deviation between the real measurement data and the parameterized statistical model by determining a deviation between the real measurement data and the virtual one measurement data.

3. The method according to claim 1 or 2, characterized in that the sample voxel data sets are obtained from CT and / or MR voxel data.

4. The method according to any one of claims 1 to 3, characterized in that the real measurement data are obtained on the basis of one or more X-ray images.

5. The method according to any one of claims 1 to 3, characterized in that the real measurement data from not yet backprojizier- th data of one or more CT and / or MR images are obtained.

6. The method according to any one of claims 1 to 3, characterized in that the real measurement data is obtained from voxel data.

7. Method according to claim 6, characterized in that a reference data set is calculated from the parameterized statistical model, the measured data are registered with the reference data record, and those model parameters are calculated which represent the model which best matches the measured data.

8. The method according to any one of the preceding claims, characterized in that the parameterized statistical model is obtained from a linear combination of example vectors, wherein a respective example vector is assigned to a respectively associated example voxel data set and components of the respective example vector position and intensity of volume elements of the describe the corresponding sample voxel data set.

9. The method according to claim 8, characterized in that a vector space spanned by the example vectors is repaired.

10. The method according to any one of the preceding claims, characterized in that the evaluation of the parameterized statistical model in step f) comprises a detection of anomalies in the real measurement data.