WO2016072926A1 - Procédé d'enregistrement d'image du corps entier et procédé permettant d'analyser des images de celui-ci - Google Patents

Procédé d'enregistrement d'image du corps entier et procédé permettant d'analyser des images de celui-ci Download PDF

Info

Publication number
WO2016072926A1
WO2016072926A1 PCT/SE2015/051177 SE2015051177W WO2016072926A1 WO 2016072926 A1 WO2016072926 A1 WO 2016072926A1 SE 2015051177 W SE2015051177 W SE 2015051177W WO 2016072926 A1 WO2016072926 A1 WO 2016072926A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
water
images
fat
whole body
Prior art date
Application number
PCT/SE2015/051177
Other languages
English (en)
Inventor
Joel KULLBERG
Håkan AHLSTRÖM
Robin STRAND
Original Assignee
Kullberg Joel
Ahlström Håkan
Strand Robin
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Kullberg Joel, Ahlström Håkan, Strand Robin filed Critical Kullberg Joel
Priority to EP15856647.1A priority Critical patent/EP3286727B1/fr
Priority to US15/524,368 priority patent/US10083506B2/en
Priority to CA2967003A priority patent/CA2967003C/fr
Priority to DK15856647.1T priority patent/DK3286727T3/da
Publication of WO2016072926A1 publication Critical patent/WO2016072926A1/fr

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/05—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves 
    • A61B5/055—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves  involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0033—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room
    • A61B5/0035—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room adapted for acquisition of images from more than one imaging mode, e.g. combining MRI and optical tomography
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/107—Measuring physical dimensions, e.g. size of the entire body or parts thereof
    • A61B5/1073—Measuring volume, e.g. of limbs
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/48—Other medical applications
    • A61B5/4869—Determining body composition
    • A61B5/4872—Body fat
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10088—Magnetic resonance imaging [MRI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30008—Bone
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person

Definitions

  • This disclosure relates to image registration.
  • it provides a method for whole body image registration and a method for analyzing images.
  • measured parameters may for example include subcutaneous fat volume, visceral fat volume, muscle volume, tumor diameter, and liver fat content.
  • Information from imaging comprising millions of measurement samples is often reduced to a few output parameter values.
  • Imaging applications are typically designed to provide answers to questions concerning one or more a priori determined target parameters, for example fat volume or tumor diameter. This is typically achieved by gross filtering of image information resulting in a large data reduction.
  • SPM statistical parameter mapping
  • VBM voxel based morphometry
  • Radiomics a feature analysis of pre-segmented regions can be made (Ref. 2). This is an initiative to use radiology medical imaging to monitor the development and progression of cancer or its response to therapy providing a comprehensive quantification of a tumor phenotype.
  • Radiomics enables high- throughput extraction of a large amount of quantitative features from radiology medical images of a given modality, such as computed tomography (CT), positron emission tomography (PET), and MR, and can provide complementary and interchangeable information compared to sources such as demographics, pathology, blood biomarkers, or genomics, improving individualized treatment selection and monitoring.
  • CT computed tomography
  • PET positron emission tomography
  • MR positron emission tomography
  • the statistical analysis is radiomics is restricted to pre-defined, pre-segmented regions of the images, only.
  • Gupta et al. (Ref. 3) presented a method for building a statistical whole-body atlas using pre-segmented objects, landmarks, by generative or discriminative learning methods.
  • the statistical analysis on non-imaging (meta) data, voxel intensities, statistics on global and local shape deformations or statistics on joint articulations is limited to analyzing one volume at the time of the atlas.
  • such atlases are not based on fat and/ or water MR images.
  • the methods as described above are either focused on too small a region of interest, or use a limited number of bands or images, which restricts the resolution and the accuracy of any possible analyzes of registered images.
  • a method for registration of whole body volume images comprises obtaining a first image and a second image.
  • the first image is a whole body volume image comprising water and fat magnetic resonance image data
  • the second image is a whole body volume image comprising water and fat magnetic resonance image data.
  • Bone tissue is identified from the first image creating a first bone image
  • bone tissue is identified from the second image creating a second bone image.
  • Water components are separated from the first image, generating a first water image based on absolute water content, and water components are separated from the second image, generating a second water image based on absolute water content.
  • Fat components are separated from the first image, generating a first fat image based on absolute fat content, and fat components are separated from the second image, generating a second fat image based on absolute fat content.
  • the first image is registered to the second image by deforming the first bone image according to a bone tissue deformation rule, deforming the first water image according to a water tissue deformation rule under constraints of the deformation of the first bone image and deforming the first fat image according to a fat tissue deformation rule under constraints of the deformation of the first bone image and the deformation of the first water image.
  • a method for analysis of whole body volume images comprises analysing image features of whole body volume images obtainable by a registering method according to the first aspect in a holistic manner.
  • the image features comprises one or more of signal intensities, features derived from a deformation field, and filtered measures derived from imaging parameters and optionally also from non-imaging parameters in all voxels.
  • Embodiments of the present invention carry a number of advantages.
  • One advantage with the proposed technology is that it provides an enhanced quality of image registration achieved by use of tissue-dependent registration rules. Thereby, a holistic analysis approach anywhere in a body allowing statistical analysis and integration of imaging and non-imaging data is enabled. Correlations performed between image data from any region and nonimaging data of the body studied are enabled.
  • Figure 2 illustrates another embodiment of registration and analysis
  • Figure 3 is a flow diagram of steps of an embodiment of a method for registration of whole body volume images
  • Figure 4 illustrates differences between a standard analysis methods and the one enabled by the present disclosure
  • Figure 5 presents examples of input image data and body mask
  • Figure 6 illustrates bone registration, according to embodiments of the present disclosure
  • Figure 7 illustrates an example of a statistical atlas-
  • Figure 8 is a flow diagram of steps of an embodiment of a method for analysis of whole body volume images;
  • Figure 9 presents an illustration of one embodiment using results from image registration in a method for image analysis;
  • Figure 10 presents results from atlas creation and application for anomaly detection
  • Figure 1 1 presents results from whole body comparisons between different groups of subjects
  • Figure 12 presents results from cross sectional correlations studies between different imaging data features and non-imaging data
  • Figure 13 presents an illustration of another embodiment using results from image registration in a method for image analysis
  • Figure 14 presents an example result from a longitudinal study
  • Figure 15 schematically illustrates a method for image analysis
  • Figure 16 presents a flow chart illustrating a method for registration of volume images.
  • imaging of whole bodies and whole-body imaging is herein to be understood as bodies or imaging of bodies, respectively, comprising at least the thoracic, abdominal and pelvic regions.
  • the intensity average and shape information average as mentioned herein are to be understood as general mean values of two or more intensity and shape information values, respectively, of a distribution of values.
  • This disclosure aims at using image data to a higher degree and /or in a more efficient manner than in prior art, involving statistical analysis of image features using all image data that allows integration of non-imaging data into an image analysis.
  • This approach is related to other well-established omics- concepts such as Genomics, Proteomics, Metabolomics and, Lipidomics.
  • image registration i.e. the process in which a first image is deformed to match a second image that is fixed, is crucial for a statistical comparison of whole-body images when bodies or objects that are studied are not pre-segmented.
  • Water-containing tissues have very similar deformation properties, regardless of which organs they belong to. Similarly, fat-containing tissues also have very similar deformation properties, but different from the water-containing tissues. By utilizing such similarities in deformation properties, registration of basically all parts of a body can be performed with a high degree of robustness. In other words, different tissue are registered based on their morphological variation, starting with the tissue with lowest shape variation, bone. Magnetic resonance imaging (MRI) allows imaging and quantification of many properties of human anatomy and physiology, such as morphology, diffusion, perfusion, regional brain tissue activation, etc. Separation of the fat- and water- components from the MRI signal can also be used to generate images that hold absolute fat- and water-content. By quantify the water and fat content in absolute values, tissues being pre-dominantly water-containing or fat-containing can be distinguished.
  • MRI Magnetic resonance imaging
  • tissue -specific features are not considered and the methods to analyze are therefore primarily not based on image registration, for which reason statistical analysis is limited. Furthermore, in ref. 5, tissue-specific features are also not utilized herein and the statistical analysis is limited to pre-defined regions of the abdomen.
  • Figure 1 illustrates an embodiment of registration and analysis. The description is held on an overview level, in order to provide an understanding of the underlying technology.
  • Water and fat whole body magnetic resonance (MR) images are provided in box 100. Absolute water and fat content are used. As will be discussed further below, additional images may also be provided, as well as non-imaging quantities.
  • MR magnetic resonance
  • bone deformation is illustrated.
  • the bone deformation is based on bone images.
  • a division into body parts is made.
  • a number of identified landmarks can be utilized.
  • water deformation is illustrated.
  • the water deformation is based on water images under constraints of the deformation of the bone image.
  • body parts, landmarks and/or identified organs are utilized.
  • fat deformation is illustrated.
  • the fat deformation is based on water images under constraints of the deformation of the bone image and the water image.
  • body parts, landmarks and/or identified organs are utilized.
  • the bone, water and/or fat deformation can be performed at least partly simultaneously and / or intermittently.
  • box 140 the registered images are used for analysis.
  • FIG. 2 illustrates another embodiment of registration and analysis.
  • Water and fat whole body magnetic resonance (MR) images, and optionally additional images, are provided in the box 100.
  • MR magnetic resonance
  • the provided images may in this embodiment be registered by firstly conducting bone registration 110.
  • bone registration a locally affine deformation may be conducted based on information of bone images and optionally of body parts.
  • landmarks may be used in said bone registration.
  • the addition of landmarks, for example, foot, knee, hip and shoulder and/ or shoulder landmarks typically adds robustness to the registration of body parts in whole body images.
  • Water registration 120 is secondly conducted.
  • a first elasticity deformation with a, compared to fat, low elasticity, is conducted based on information from water images, with the fixed bones as constraints.
  • organs may be used in said water registration. The addition of organs adds robustness to registration in abdominal region in whole body images.
  • the first elasticity is typically an elasticity being lower than an elasticity threshold.
  • FIG. 1 illustrates a flow diagram of steps of an embodiment of a method for registration of whole body volume images. The method starts in step 200.
  • a first image and a second image are obtained.
  • the first image is a whole body volume image comprising water and fat magnetic resonance image data.
  • the second image is a whole body volume image comprising water and fat magnetic resonance image data.
  • bone tissue is identified from the first image creating a first bone image.
  • bone tissue is identified from the second image creating a second bone image.
  • water components are separated from the first image, generating a first water image based on absolute water content.
  • water components are separated from the second image, generating a second water image based on absolute water content.
  • fat components are separated from the first image, generating a first fat image based on absolute fat content.
  • fat components are separated from the second image, generating a second fat image based on absolute fat content.
  • the first image is registered to the second image by deforming the first bone image according to a bone tissue deformation rule, deforming the first water image according to a water tissue deformation rule under constraints of the deformation of the first bone image and deforming the first fat image according to a fat tissue deformation rule under constraints of the deformation of the first bone image and the deformation of the first water image.
  • the procedure ends in step 299.
  • the bone tissue deformation rule comprises a bone tissue elasticity
  • the water tissue deformation rule comprises a water- containing tissue elasticity
  • the fat tissue deformation rule comprises a fat-containing tissue elasticity.
  • the bone tissue elasticity is lower than the water-containing tissue elasticity and the water-containing tissue elasticity is lower than the fat-containing tissue elasticity.
  • the deforming of the first bone image is performed prior to the deforming of the first water image, and the deforming of the first water image is performed prior to the deforming of the first fat image.
  • the deforming of the first bone image, the first water image and the first fat image may be performed in a common process.
  • the deformations may thus be performed at least partially simultaneous or interleaved.
  • registering said first volume image, or in a more extended case one or more of a first set of volume images, onto the second volume image, or in the more extended case the second set of volume images comprises determining point-to-point correspondences between the first image and the second image.
  • the first and second volume images within the method comprise images of at least one whole body.
  • Figure 4 illustrates in a schematic manner differences between a standard prior-art analysis method and an embodiment of a method for analyzing images, according to the present disclosure.
  • a cohort of people 410 e.g. groups of healthy and/ or non-healthy persons is investigated by different acquisition techniques.
  • Imaging techniques 400 give raw image data from all subjects in the cohort 410.
  • An explicit measurement of the liver 420 from the images is illustrated and results in the measurements illustrated as 425.
  • Non-imaging techniques give typically one or several values of specified quantities, as raw data 430, also from all subjects in the cohort.
  • explicit measurements 425 such as e.g. a measurement from the liver 420, can be analyzed, possibly together with non-imaging data. This analysis typically gives rise to one or a few values of specified quantities. Irrespective of the quality of such analysis, the total available information in the image data is considerably reduced.
  • a correlation between measurements from imaging data and non-imaging data is illustrated 441. Correlations may then be performed between the limited data deduced from the image data from liver and nonimaging data measured on the same body that was carrying the liver. Conclusions can thereby be drawn about correlations between the different discrete data sources.
  • image data from all subjects of a whole body is integrated in the image analysis, as illustrated in 450.
  • image analysis may therefore be performed in the whole body region 460- 463. Correlations may thus be performed between image data from any region and also including non-imaging data of the body studied.
  • MRI magnetic resonance imaging
  • Fat and water components can be separated from the MRI signals to generate images that hold absolute fat and water content.
  • the absolute fat content is defined as f/ (f+w), where f is the fat signal amplitude and w is the water signal amplitude (real or complex).
  • the absolute water content is defined as w/ (f+w).
  • additional images may be provided such as other types of MR- images and/ or positron emission tomography (PET) images of different PET- tracers.
  • the other types of MR images may comprise diffusion weighted or apparent diffusion coefficient (ADC) MR-images.
  • the PET images may comprise fluorodeoxyglucose (FDG) images.
  • FDG fluorodeoxyglucose
  • PET Positron emission tomography
  • each set of volume images within the method may further comprise one or more types of MR images comprising: diffusion weighted MR images, ADC MR images, ultra-short echo time MR images, and zero echo time MR images.
  • each set of volume images within the method may further comprise one or more types of PET images, comprising static and dynamic FDG images.
  • embodiments of the present disclosure allow not only signal intensity from MRI or PET images to be compared. Also, fusion of image data will also cause different regions of a subject (whole body) to expand/ contract and this can be used to estimate deviations in local volumes, of for example abdominal fat, muscles or liver, and also local deformations. Multiple local feature /texture measures, such as intensity heterogeneity may also be measured, which can be used to determine deviations of tissue heterogeneity, for example due to pathologies such as tumors or fibrosis.
  • the step of registering comprises determining of a volume deformation field and applying of the volume deformation field on the first volume image. This can be applied in general case, but will be discussed more in detail in connection with detailed embodiments presented here below. Thus, the following paragraphs present embodiments of deformation of volume images in a somewhat more detailed manner.
  • volume images are aligned in a reference coordinate system.
  • Image registration relates to the problem of finding a displacement that spatially aligns a moving or movable image to a fixed image.
  • deforming bone tissue and water tissue and fat tissue may comprise determining of a volume deformation field and applying of that volume deformation field on the first image.
  • An image is a mapping 7: ⁇ , where il c l 3 .
  • the transformation is typically in many particular embodiments computed by minimizing a cost function.
  • the cost function can for example be sum of squared intensity differences, mutual information or correlation.
  • Image registration can be either parametric using for example B-splines, thin- plate splines or wavelets, or non-parametric, for example Demons or Morphons.
  • the transform T can also be restricted to rigid or affine transformations.
  • the deformation field In general, restrictions on the deformation field are needed in order to see to that the deformation field is, for example, smooth, invertible, or diffeomorphic and to avoid unexpected deformations.
  • the parametric approach typically involves an implicit regularization since the parametric representation allows only smooth deformations.
  • the regularization In an iterative non-parametric registration process Ti (x - x + Ui (x , the regularization can be obtained by, in each iteration, smoothing the deformation field T t ⁇ x) or the update deformation field
  • an image registration is usually carried out in different scales, starting from a coarse scale where the overall shape is first aligned.
  • fine-scale structures are aligned.
  • This multiscale approach can be realized by using for example, Gaussian scale space and/ or pyramid representations of the image data.
  • GPU Graphical processing unit
  • a variational approach to image registration may be used.
  • the deformation field may be achieved by energy minimization of a metric, i.e. , weighted terms of energy functions and regularization terms.
  • Essential steps in a variational approach to image registration are choice of metric, parameterization of the deformation field, optimization method to find the optimal parameters and interpolation methods.
  • deformable image registration is needed due to the high degree of morphological variability in whole-body images.
  • the deformation fields may be parameterized by, for example, a B-spline representation.
  • optimization method may be used due to its computational efficiency.
  • a multi-resolution approach based on spatial down-sampling and Gaussian scale-space may also be used.
  • Anatomical landmarks can be used to guide the registration process.
  • the mean Euclidean distance error between landmarks in the fixed and moving images may be used.
  • the method for registration of whole body volume images thus comprises the further step of identifying of a plurality of landmarks in the first image and in the second image.
  • the step of registering thereby comprises matching of the identified plurality of landmarks between the first image and the second image. This can be applied in general case, but will be discussed more in detail in connection with detailed embodiments presented further below.
  • anatomical landmarks may be used to guide the registration process.
  • the anatomical landmarks may be obtained manually by a user or in an automated way by processing the image data that is used to achieve an initial, affine, deformation.
  • the anatomical landmarks may also be used to guide the registration process by including information on point-to-point correspondences.
  • identifying bone tissue and water-containing tissue and fat-containing tissue of the first and second images may comprise identifying of landmarks of the first and the second images, and/ or segmenting the water-containing tissue and/ or the fat- containing tissue of the first and the second images.
  • Registering the first image may comprise matching of the landmarks of the first image onto the landmarks of the second image.
  • the method for registration of whole body volume images comprises the further step of segmenting the first image and the second image into segments, wherein the step of registering is performed for each segment.
  • the registering thus comprises matching of said segments between said first image and said second image.
  • the step of segmenting the first image and saheid second image is performed in the first water image and the second water image, identifying organs. This can be applied in general case, but will be discussed more in detail in connection with detailed embodiments presented here below.
  • the method for registration of whole body volume images comprises extracting of a body mask from the water and fat magnetic resonance image data and extracting of body parts from the body mask. The step of registering is then performed for each of these body parts. This can be applied in general case, but will be discussed more in detail in connection with detailed embodiments presented further below.
  • MRI images with absolute fat and water content denoted I FAT and 1 W ATER > and body mask, denoted I B ODY > may be used.
  • Figure 5 illustrates absolute fat 500, absolute water 510 and body mask 520 images which may be used as input image data. These images are coronal example slices.
  • a body mask 520 can typically be extracted from the image data. In an example implementation, this may be obtained by using intensity thresholds followed by morphological operations and region growing. From the body mask, separate sub regions may be extracted, for example arms, head, abdomen, and legs. In one implementation, this may be obtained by morphological analysis of the image data.
  • registering the first image may comprise registering the first image based on segmentation of the water-containing tissue of the first and the second images.
  • the whole-body sub regions may be used in the registration process to divide bone into corresponding sub regions, hereafter denoted bone sections.
  • the bone sections may be deformed by separate affine transformations.
  • a rough segmentation of the skeleton can be computed by extracting regions with low signal in water and fat separated MRI, since air and bone do not produce any signal in water/ fat separated MRI. From this, a segmented skeleton may be obtained by removing segmented lung and abdomen and by utilizing filtering techniques.
  • the first image may be a member of a first set of volume images to be registered, which set comprises one or more volume images.
  • the first set of volume images may be volume images of one and the same whole body.
  • the second image may be a volume image composed from one or more whole bodies.
  • the second image may comprise intensity average and shape information average volume images based on volume images of multiple bodies.
  • the inter-individual morphological variation in bone, lean tissue (water signal) , and adipose tissue (fat signal) may be assumed to be relatively low, medium and high, respectively.
  • the example solution may divide the image registration process into three different steps, where different tissue are registered based on their morphological variation, starting with the tissue with lowest shape variation, bone.
  • the cost functions used in the individual steps of the registration procedure may be weighted sums of cost functions denoted S; .
  • As input to the cost functions may either be a moving image (I M ) and a fixed image (7 F ) or sets of landmarks. Cost functions used in the different steps are described below.
  • Input data to the bone registration may comprise:
  • body parts, bone sections and landmarks could be according to the following.
  • an initial deformation may be obtained by a similarity transform.
  • the initial transform may be obtained by Procrustes analysis of the eight landmarks.
  • the initial transform can be followed by a piecewise affine transform for each of the segmented and labeled bone sections together with the one, two or four closest landmarks, see Table 1.
  • the four closest landmarks are used and a slightly elastic deformation can be allowed to compensate for different articulations of the spine.
  • a distance transform may assign the distance to the closest object voxel to each background voxel.
  • the pairs of landmarks used correspond to the landmarks, as given in Table 1 , in the fixed and moving images.
  • a cost function may be minimized for an affine transform for all objects except the spine k ⁇ 5 for which a deformable transform (a B-spline grid with spacing 64 mm) may be used.
  • the contribution of the DT is high for the low resolutions (for stability) and low for the highest resolution.
  • Two sets of landmark pairs may be computed for each k bone section:
  • LMbone For each bone section deformed by an affine (or elastic for the spine) deformation, a large number of corresponding points pairs in the fixed and moving images may be generated.
  • Each body part may be deformed by the deformation given by the registration of the corresponding bone section. This deformation can be represented by a large number of landmarks, i.e. , corresponding point pairs, in the fixed and moving images.
  • the landmarks may be be used in future processing steps to constrain the deformation in the bone sections and the landmarks LMWB will be used for whole-body approximate registration described in the next section.
  • Figure 6 illustrates bone registration between two whole bodies collected from the same subject at two different time points.
  • This figure illustrates a moving (or movable) image 610 and a deformed moving image 61 1.
  • a fixed image being a reference onto which the moving image 610 is being mapped to create the deformed moving image 61 1 is shown.
  • the symmetric difference between the bone images is also shown before 630 and after 631 the deformation, respectively. Deviations between the moving image and the fixed image are thus illustrated in 630 and 631 in grey.
  • 631 a significantly enhanced overlap and only minor deviations are shown, as compared to what is shown in 630.
  • the local deformations of the bones may be propagated to the whole body and represented by a smooth deformation field by computing a global deformation that minimizes the point-pair distances of LM WB .
  • the cost function may be a linear combination of the mean square distance between the fixed and the moving body mask images and the mean Euclidean distance between the landmarks LM WB in the fixed and moving image.
  • Water images may be registered by minimizing the sum of square differences of the water images and using the global deformation as presented above as initial transform and LM B0NE as landmarks.
  • the cost function may be a linear combination of the mean square distance between the fixed and the moving water images and the mean Euclidean distance between the landmarks LM WB in the fixed and moving image.
  • the mean square error of body masks and fat images may be minimized by LM WATER and LM B0NE as landmarks.
  • the cost function is a linear combination of the mean square distance between the fixed and the moving fat images, the mean square distance between the fixed and the moving body mask images and the mean Euclidean distance between the landmarks LM WB in the fixed and moving image.
  • the first image and the second image are images of one and the same whole body acquired at different times.
  • Each volume image may be associated with volume image features, enabling statistical analysis of said volume image features of various volume images.
  • the method may further comprise storing the registered first images, repeating the identifying, the registering and the storing of the registered first images. This repetition generates a group of multiple first images, which together form an atlas of stored volume images.
  • the storing of the registered first image may also comprise storing of image features of the first images.
  • the image features may comprise one or more of following: volume image intensity values, calculated values based on volume image intensity values, and properties of the volume deformation field of each of the first images.
  • Figure 7 illustrates an example of a statistical atlas, related to embodiments of the present disclosure.
  • This example statistical atlas is generated via fusion of PET and different MRI source images and calculated images from healthy subjects (bodies) into a common reference coordinate system, i.e. registering said images.
  • the thereby created atlas then holds statistics of all image features included, i.e. morphology, function and metabolism.
  • the different kinds of whole body images e.g. PET and different MRI source images from one subject
  • whole body images of many subjects are illustrated.
  • the ensemble of whole body images could e.g. represent a group of healthy subjects, or a statistically selected group of subjects.
  • the ensemble of whole body images are brought into a common reference coordinate system, i.e. registering 799 of the whole body images.
  • the total image becomes an atlas 720 based on volume images of multiple bodies.
  • the atlas comprises at least an intensity averaged and shape information averaged volume image based on said volume images of said multiple bodies.
  • image data from a large number of healthy subjects can be deformed to a common coordinate system to create a mean shape or reference person in which each voxel may be considered to contain a distribution of all imaging parameters 720-722 measured in an entire cohort of individuals.
  • the analysis of images is typically based on multispectral image data collected from different subjects (whole bodies) or at different points in time. This approach may be used to combine whole-body image data from, for instance, healthy subjects to create a statistical atlas that contains detailed statistics on tissue morphology, function, and metabolism in all voxels in the entire body.
  • image data from each individual dataset may then be compared to this statistical atlas, in order to calculate deviations in each pixel from all image parameters collected, such as fat and water content, FDG- uptake, etc. in the entire body, as illustrated by the diagrams in part C of Figure 7.
  • the image registration as described herein allows in general words point-to- point, tissue-to-tissue and organ-to-organ analysis.
  • FIG 8 is a flow diagram of steps of an embodiment of a method for analysis of whole body volume images.
  • the method starts in step 201.
  • image features of whole body volume images obtainable by a registering method according to the description here above are analysed in a holistic manner.
  • the image features comprise one or more of signal intensities, features derived from a deformation field, and filtered measures derived from imaging parameters and optionally also from non-imaging parameters in all voxels.
  • the procedure ends in step 298.
  • the method comprises the further step 280, in which whole body volume images are registered according to the principles described elsewhere in this disclosure.
  • the step of analysing image features is then based on registered whole body volume images obtained by this step 280.
  • FIG. 9 presents an illustration of some embodiments using results from image registration in a method for image analysis.
  • the image analysis comprises cross-sectional 930 studies and prediction 940 cohort studies.
  • the figure illustrates how an atlas 720 can be used to assess deviations 915- 17 in image parameters in whole body images 715-717 and study their associations to non-imaging data 930-933 and 940-943.
  • cross-sectional studies 930-933 associations to scalar valued parameters may be studied.
  • imaging data is typically collected from one point in time, and non-imaging parameters are collected from the same point in time.
  • Non-image parameters may comprise blood samples.
  • Cross-sectional studies may be studied using whole-body "correlation-maps", and groups of subjects, for instance normal versus pre-diabetic, can be compared using whole-body "difference-maps".
  • the step of analyzing image features thus comprises a cross-sectional analysis, analyzing a multitude of whole body volume images from more than one whole body.
  • the cross- sectional analysis comprises collected non-imaging data associated with whole bodies of the whole body volume images.
  • the step of analyzing image features comprises calculating of a correlation between image features of a group of whole body volume images with said collected non-imaging data.
  • Prospective studies for risk prediction 940-943 may further be used to create "prediction-maps" 941 of what whole-body tissue features predict “time to event” or the future risk of, for instance, type-2 diabetes, myocardial infarction, stroke or dementia, respectively.
  • imaging data is collected at one point in time, whereas non-imaging data is typically collected at a later point in time.
  • Figure 10 illustrates the results from atlas creation and anomaly detection.
  • the anomaly detection was performed by comparing intensities of one male subject with high liver fat content 1030 to a preliminary whole body imaging atlas 1010 and 1020, i.e. pointwise distributions 1015 of fat content of 50 male subjects.
  • the results indicate low p-values where expected, i.e. in the abdominal adipose tissue and in the liver, respectively 1040.
  • a coronal example slice is shown from each whole body image volume.
  • Mean 101 1 absolute fat content in the normal material is show in 1010. Values from 0% (black) to 100% (white).
  • Absolute fat content 1031 in a male subject with high liver fat is shown in 1030. Values from 0% (black) to 100% (white).
  • Pointwise p-values of the hypothesis that the fat content in the male subject with high liver fat does not differ from the normal material are shown in 1040. Values from 0 (white) to 1 (black).
  • Pointwise p-values of the hypothesis that the fat content in the male subject with high liver fat does not differ from the normal material thresholded at 0.05 are shown in 1041. Points with p-values below 0.05 are black and the rest white.
  • the step of analyzing image features thus comprises comparing a whole body volume image from one whole body with an atlas.
  • Figure 1 1 shows the point-by-point p-values (p-maps) of local tissue volume were obtained by two-tailed t- tests between groups 1030 of low weight subjects and high weight subjects (20 men and 23 women). As expected, significant differences of local tissue volume (size) were observed between the two groups.
  • a coronal example slice is shown from each whole body image volume. p-maps are shown for male 1110 and female 1120. Values from 0 (white) to 1 (black). p-maps thresholded at 0.05 for male 1 1 11 and female 1 121 are also shown. Significant p-values are shown in black.
  • the step of analyzing image features thus comprises comparing a first group of body volume images to a second group of body volume images.
  • the figure shows pointwise r-values between local tissue volume and body weight in males 1210 and females 1215.
  • the figure shows pointwise r-values between local tissue volume and total body fat mass measured by bioimpedance analysis (BIA) in males 1220 and females 1225.
  • BIOA bioimpedance analysis
  • the figure shows pointwise r-values between fat content and mean liver fat content in males 1230 and females 1235.
  • Figure 13 presents an illustration of some embodiments using results from image registration in a method for a longitudinal image analysis.
  • Longitudinal studies may be used for detailed studies of changes in tissue feature during, for example, oncological or diabetes treatment.
  • image data as well as non- imaging data can be collected at multiple points in time 718a and 718b. The registration of these images can result in a difference image 1320 which contains changes in the different image features 1330 in the whole body region between the time points.
  • the analysis of the image data may be generalized into categories.
  • One category being, a priori driven 1340, i.e. based on a hypothesis 1341 and 1350, and another being open ended 1360, i.e. general search for feature deviations 1361 , 1362, 1370, 1380.
  • the arrow 1341 illustrates the hypothesized a priori direction of change.
  • the r-value illustrated 1361 can be converted into a p-value of statistical significance of the feature change.
  • Figure 14 shows an example from a longitudinal study where the registration method is applied.
  • One coronal slice of the moving 14010, deformed moving 1411 and fixed 1420 images of absolute fat content is shown together with a difference image 1431 between the fixed and deformed moving images.
  • the moving image is deformed to match the fixed image using the described methodology.
  • the subject underwent low calorie diet (LCD) and gastric bypass (GBP) between the moving and fixed MRI-scans.
  • the total weight loss was 14 kg. Note the significant reduction in liver fat.
  • This methodology could for example be used for "weight loss imaging", to visualize where and when volumetric and tissue property (e.g. intensity) changes occur or for studying changes in image features during for example cancer treatment.
  • a coronal example slice is shown from each whole body image volume.
  • the figure shows absolute fat content in the baseline volume (moving) 1410. Values from 0% (black) to 100% (white).
  • the figure shows absolute fat content in the deformed baseline volume (moving) 141 1. Values from 0% (black) to 100% (white).
  • the figure shows absolute fat content in the follow-up volume (fixed) 1420. Values from 0% (black) to 100% (white).
  • the figure shows a difference-image, i.e. the difference in absolute fat content between 1420 and 141 1. Values from -20% (black) to 20% (white).
  • the step of analyzing image features thus comprises a longitudinal analysis, analyzing differences between a multitude of whole body volume images from one and the same whole body acquired at different times.
  • the step of analyzing image features is based on a hypothesis.
  • the step of analyzing image features is based on a general search for feature deviations.
  • the statistical interaction between image features and non-imaging parameters may be studied in the whole-body region using different approaches.
  • correlations such as correlation maps between image features and insulin resistance.
  • Another approach being group-wise comparisons, where associations in image features between two groups of whole body images are studied.
  • the groups are typically divided by non-imaging parameters. For example, to create a voxel-level statistical test of feature difference between two groups of individuals, e.g. normal versus type-2-diabetics, etc.
  • Another approach is multivariate modelling, where different statistical models, such as linear, logistic, Cox hazard, factor analysis, lasso, boosted regression trees, can include multiple variables, e.g. age/blood pressure/ smoking/ physical activity in the analysis.
  • Still another approach being anomaly detection, where differences in image features between an imaging atlas, or group of normality, and a subject with suspected non-normality referring to image features or non-imaging parameters, are studied.
  • Still another may be considered to be cross-sectional analysis where associations between image features and non-imaging parameters in a statistical whole body atlas, or group of whole body images, are studied.
  • Still yet another approach is the longitudinal analysis, where changes /differences in image features in whole body images of the same individual acquired at different time-points, are studied.
  • Attenuation correction may be considered to be one approach. Segmentation, or detection of, of tissue, such as bone, water, fat and pre- segmented organs, by utilizing an imaging atlas holding pre-segmented tissue, provides information to be used in attenuation correction needed in PET-MR imaging.
  • image features wherever mentioned may comprise one or more of signal intensities, features derived from a deformation field, typically regional volumes, and filtered measures, such as texture, segmented tissue.
  • this disclosure aims at using the full potential of image data by avoiding reducing image information to a few measures or parameters, and by carrying out a holistic analysis on each image, and on large numbers of collected images by a statistical representation and also by allowing integration of non-imaging biomarkers in an analysis of images.
  • Embodiments herein described are based on an image registration method which uses tissue-specific handling of fat/water MR images and pre- segmented bone.
  • Figure 15 schematically illustrates a method for image analysis, according to some embodiments.
  • the method comprises performing a comparison of where and how image features, or image features derived from a deformation field, of the method for registration of volume images of whole bodies differ within a first group of first sets of volume images and within a second group of first sets of volume images, wherein said first and said second group of first sets of volume images are comprised within the atlas of volume images.
  • the method alternatively comprises calculating a correlation between image features, or image features derived from a deformation field, of the method for registration of volume images of a group of first sets of volume images of whole bodies of the volume image atlas and collected non-imaging data associated with said whole bodies comprising one or more of: blood pressure or measurement results from blood analysis, measurement results from non-imaging analysis, biomarkers or data from longitudinal studies of disease development.
  • the method alternatively comprises performing a comparison of image features, or image features derived from a deformation field, of the method for registration of volume images from a set of registered volume images of a whole body and a volume image atlas of registered volume images of multiple whole bodies enabling detecting where and how image features from a whole body deviate from image features from multiple whole bodies of the atlas of volume images.
  • the method alternatively comprises using the method for registration of volume images to identify bone tissue and at least one other tissue of the volume images, enabling tissue quantification in MR-type imaging or improving attenuation correction in PET-MR imaging.
  • the method alternatively comprises performing comparisons of where and how image features differ between sets of registered volume images of one and the same whole body acquired at different time points.
  • FIG. 16 presents a flow chart illustrating a method for registration of volume images of one or more whole bodies. The method comprises identifying 42 bone tissue and at least one other tissue of a first set and a second set of volume images, each set of volume images comprising at least water and fat
  • the method also comprises registering 44 one or more of the first set of volume images, onto the second set of volume images, using tissue- dependent registration rules comprising a registration rule for bone tissue and at least one other registration rule for said at least one other tissue, based on identified bone tissue and identified at least one other tissue.
  • the tissue-dependent registration rules in registering one or more of the first set of volume images may comprise tissue-dependent deformation rules, and registering one or more of the first set of volume images, may comprise deforming bone tissue of said one or more of the first set of volume images using a bone tissue deformation rule and deforming said at least one other tissue of said one or more of the first set of volume images using at least one other tissue deformation rule.
  • the tissue-dependent deformation rules may comprise tissue-dependent elasticity rules, specifying rules of tissue-dependent elasticity, and deforming bone tissue may comprise deforming bone tissue using a bone tissue elasticity, and deforming said at least one other tissue may comprise deforming said at least one other tissue using at least one other tissue elasticity.
  • the tissue-dependent elasticity rules may further comprise a pre-dominantly water-containing tissue elasticity, and predominantly fat-containing tissue elasticity, wherein the bone tissue elasticity is lower than the water-containing tissue elasticity that is lower than the fat-containing tissue elasticity.
  • Registering said one or more of the first set of volume images onto the second set of volume images within the method may comprise deforming bone tissue of said one or more of the first set of volume images, prior to deforming predominantly-water containing tissue of said one or more of the first set of volume images, prior to deforming predominantly fat-containing tissue of said one or more of the first set of volume images.

Abstract

L'invention concerne un procédé destiné à l'enregistrement d'images de volume du corps entier qui comprend la récupération (210) d'une première image et d'une seconde image, comprenant des données d'images d'eau et de matière grasse par résonance magnétique d'un corps entier. Les tissus osseux sont identifiés (220), ce qui crée des images d'os respectives. Les composantes d'eau sont séparées (230), ce qui génère des images respectives d'eau fondées sur la teneur absolue en eau. Les composantes de matière grasse sont séparées (240), ce qui génère des images respectives de matière grasse fondées sur la teneur absolue en matière grasse. La première image est enregistrée (250) sur la seconde image en déformant la première image d'os selon une règle de déformation du tissu osseux, en déformant la première image d'eau selon une règle de déformation du tissu aqueux sous la contrainte de la première déformation d'image d'os, et en déformant la première image de matière grasse selon une règle de déformation du tissu adipeux sous la contrainte de la première image d'os et de la déformation de ladite première image d'eau. L'invention concerne également un procédé destiné à analyser les images enregistrées.
PCT/SE2015/051177 2014-11-07 2015-11-06 Procédé d'enregistrement d'image du corps entier et procédé permettant d'analyser des images de celui-ci WO2016072926A1 (fr)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP15856647.1A EP3286727B1 (fr) 2014-11-07 2015-11-06 Procédé d'enregistrement d'image du corps entier et procédé permettant d'analyser des images de celui-ci
US15/524,368 US10083506B2 (en) 2014-11-07 2015-11-06 Whole body image registration method and method for analyzing images thereof
CA2967003A CA2967003C (fr) 2014-11-07 2015-11-06 Procede d'enregistrement d'image du corps entier et procede permettant d'analyser des images de celui-ci
DK15856647.1T DK3286727T3 (da) 2014-11-07 2015-11-06 Helkropsbilledregistreringsfremgangsmåde og fremgangsmåde til analyse af billeder deraf

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
SE14005375 2014-11-07
SE1405375 2014-11-07

Publications (1)

Publication Number Publication Date
WO2016072926A1 true WO2016072926A1 (fr) 2016-05-12

Family

ID=55909505

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/SE2015/051177 WO2016072926A1 (fr) 2014-11-07 2015-11-06 Procédé d'enregistrement d'image du corps entier et procédé permettant d'analyser des images de celui-ci

Country Status (1)

Country Link
WO (1) WO2016072926A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018002265A1 (fr) * 2016-06-30 2018-01-04 Koninklijke Philips N.V. Génération et personnalisation d'un modèle statistique du sein
WO2020132713A1 (fr) * 2018-12-24 2020-07-02 Body Composition Technologies Pty Ltd Analyse d'un corps
EP4080453A1 (fr) 2021-04-21 2022-10-26 Antaros Medical AB Procédé et système d'enregistrement d'image médicale
WO2023232897A1 (fr) 2022-06-02 2023-12-07 Antaros Medical Ab Inférence automatisée d'image à partir d'images médicales de tout le corps

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100067769A1 (en) * 2008-09-12 2010-03-18 Huzefa Neemuchwala Method and apparatus for registration and comparison of medical images
WO2013001399A2 (fr) * 2011-06-27 2013-01-03 Koninklijke Philips Electronics N.V. Imagerie par résonance magnétique d'un tissu osseux
WO2013121379A2 (fr) * 2012-02-16 2013-08-22 Koninklijke Philips N.V. Reconstruction d'image nucléaire spatialement corrigée
US20140270446A1 (en) * 2013-03-15 2014-09-18 Siemens Medical Solutions Usa, Inc. Method and Apparatus for Registration of Multimodal Imaging Data Using Constraints

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100067769A1 (en) * 2008-09-12 2010-03-18 Huzefa Neemuchwala Method and apparatus for registration and comparison of medical images
WO2013001399A2 (fr) * 2011-06-27 2013-01-03 Koninklijke Philips Electronics N.V. Imagerie par résonance magnétique d'un tissu osseux
WO2013121379A2 (fr) * 2012-02-16 2013-08-22 Koninklijke Philips N.V. Reconstruction d'image nucléaire spatialement corrigée
US20140270446A1 (en) * 2013-03-15 2014-09-18 Siemens Medical Solutions Usa, Inc. Method and Apparatus for Registration of Multimodal Imaging Data Using Constraints

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
BLEY T. A. ET AL.: "Fat and Water Magnetic Resonance Imaging", JOURNAL OF MAGNETIC RESONANCE IMAGING, vol. 31, no. 1, 2010, pages 4 - 18, XP007913852, DOI: doi:10.1002/jmri.21895 *
HASEGAWA A. ET AL.: "A Tool for Temporal Comparison of Mammograms: Image Toggling and Dense-Tissue-Preserving Registration", INTERNATIONAL WORKSHOP ON DIGITAL MAMMOGRAPHY (IWDM) 2008, vol. 5116, 20 July 2008 (2008-07-20), pages 447 - 454, XP019092116 *
LELIEVELDT B. P. F. ET AL.: "Towards integrated analysis of longitudinal whole-body small animal imaging studies", INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PRODESSING (ICASSP) 2011, 22 May 2011 (2011-05-22), pages 5768 - 5771, XP032001996, DOI: doi:10.1109/ICASSP.2011.5947671 *
MAINTZ J. B. A. ET AL.: "A Survey of Medical Image Registration", MEDICAL IMAGE ANALYSIS, vol. 2, no. 1, 1998, pages 1 - 37, XP001032679, DOI: doi:10.1016/S1361-8415(01)80026-8 *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018002265A1 (fr) * 2016-06-30 2018-01-04 Koninklijke Philips N.V. Génération et personnalisation d'un modèle statistique du sein
CN109414233A (zh) * 2016-06-30 2019-03-01 皇家飞利浦有限公司 统计乳房模型的生成和个性化
US11213247B2 (en) 2016-06-30 2022-01-04 Koninklijke Philips N.V. Generation and personalization of a statistical breast model
CN109414233B (zh) * 2016-06-30 2023-09-12 皇家飞利浦有限公司 统计乳房模型的生成和个性化
WO2020132713A1 (fr) * 2018-12-24 2020-07-02 Body Composition Technologies Pty Ltd Analyse d'un corps
EP4080453A1 (fr) 2021-04-21 2022-10-26 Antaros Medical AB Procédé et système d'enregistrement d'image médicale
WO2022223584A1 (fr) 2021-04-21 2022-10-27 Antaros Medical Ab Procédé et système d'enregistrement d'image médicale
WO2023232897A1 (fr) 2022-06-02 2023-12-07 Antaros Medical Ab Inférence automatisée d'image à partir d'images médicales de tout le corps

Similar Documents

Publication Publication Date Title
CA2967003C (fr) Procede d'enregistrement d'image du corps entier et procede permettant d'analyser des images de celui-ci
JP6378778B2 (ja) 組織種類分離の助けを借りて磁気共鳴画像に基づき１つ以上のコンピュータ断層撮影画像を生成する方法及び装置
US10580159B2 (en) Coarse orientation detection in image data
CN114503159A (zh) 通过对象检测定位的医学图像的三维对象分割
US9741131B2 (en) Anatomy aware articulated registration for image segmentation
US10881321B2 (en) Automatic tract extraction via atlas based adaptive connectivity-based clustering
Zacharaki et al. Non-diffeomorphic registration of brain tumor images by simulating tissue loss and tumor growth
US20120281900A1 (en) Method and apparatus for processing medical images
US10803354B2 (en) Cross-modality image synthesis
Calmon et al. Automatic measurement of changes in brain volume on consecutive 3D MR images by segmentation propagation
Alam et al. Challenges and solutions in multimodal medical image subregion detection and registration
WO2016072926A1 (fr) Procédé d'enregistrement d'image du corps entier et procédé permettant d'analyser des images de celui-ci
Attyé et al. TractLearn: A geodesic learning framework for quantitative analysis of brain bundles
CN115830016A (zh) 医学图像配准模型训练方法及设备
Wang et al. Multi-atlas segmentation with joint label fusion of osteoporotic vertebral compression fractures on CT
Bloy et al. White matter atlas generation using HARDI based automated parcellation
Bekkouch et al. Multi-landmark environment analysis with reinforcement learning for pelvic abnormality detection and quantification
Kazemifar et al. Automated algorithm to measure changes in medial temporal lobe volume in Alzheimer disease
Li et al. Ddmm-synth: A denoising diffusion model for cross-modal medical image synthesis with sparse-view measurement embedding
Rastgarpour et al. The problems, applications and growing interest in automatic segmentation of medical images from the year 2000 till 2011
Galdames et al. Registration of renal SPECT and 2.5 D US images
Karthik et al. Three-dimensional segmentation of the scoliotic spine from MRI using unsupervised volume-based MR-CT synthesis
CN103514607A (zh) 动态对比度增强核磁共振图像检测方法
Li et al. Consistent 4D cortical thickness measurement for longitudinal neuroimaging study
Hanaoka et al. Whole vertebral bone segmentation method with a statistical intensity-shape model based approach

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15856647

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 15524368

Country of ref document: US

ENP Entry into the national phase

Ref document number: 2967003

Country of ref document: CA

NENP Non-entry into the national phase

Ref country code: DE

REEP Request for entry into the european phase

Ref document number: 2015856647

Country of ref document: EP