EP2720613A1 - Method, a system and a computer program product for registration and identification of diagnostic images - Google Patents

Method, a system and a computer program product for registration and identification of diagnostic images

Info

Publication number
EP2720613A1
EP2720613A1 EP12730680.1A EP12730680A EP2720613A1 EP 2720613 A1 EP2720613 A1 EP 2720613A1 EP 12730680 A EP12730680 A EP 12730680A EP 2720613 A1 EP2720613 A1 EP 2720613A1
Authority
EP
European Patent Office
Prior art keywords
image
later
earlier
diagnostic
region
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP12730680.1A
Other languages
German (de)
French (fr)
Inventor
Richard MENDES
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Radiology Morphological Solutions BV
Original Assignee
Radiology Morphological Solutions BV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Radiology Morphological Solutions BV filed Critical Radiology Morphological Solutions BV
Priority to EP12730680.1A priority Critical patent/EP2720613A1/en
Publication of EP2720613A1 publication Critical patent/EP2720613A1/en
Withdrawn legal-status Critical Current

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/52—Devices using data or image processing specially adapted for radiation diagnosis
    • A61B6/5211—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data
    • A61B6/5229—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data combining image data of a patient, e.g. combining a functional image with an anatomical image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/73—Deblurring; Sharpening
    • G06T5/75—Unsharp masking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G06T7/337—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving reference images or patches
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/46—Arrangements for interfacing with the operator or the patient
    • A61B6/461—Displaying means of special interest
    • A61B6/463—Displaying means of special interest characterised by displaying multiple images or images and diagnostic data on one display
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing

Definitions

  • the invention relates to a method for registration of transmission radiography diagnostic images taken at different time moments for the same patient.
  • the invention further relates to a system for registration of transmission radiography diagnostic images taken at different time moments for the same patient.
  • the invention still further relates to a computer program product for registration of transmission radiography diagnostic images taken for different time moments for the same patient.
  • transmission X-ray images also referred to as transmission radiography images
  • lungs and surrounding tissue such as ribs
  • a plurality of different diagnostic apparatus having different operational settings may be used for acquiring the earlier and the later diagnostic image(s) under consideration.
  • the patient may have changed dimensions, because of age, operation or due to a weight change.
  • human errors may occur during visual inspection of the images.
  • the method according to the invention comprises the steps of:
  • any suitable image registration method may be used for practicing the invention. For example, in a region of interest one or more image segments may be identified. This identification may be carried out automatically or manually. Then, for example, linear image warping may be used for the identified segments making use of suitable patterns. Next, a homography matrix composed of rotation, shift and scale may be generated for the individual parts of the identified patterns. It will be appreciated that a great plurality (up to 100) segments may be identified within a region of interest. These segments may be individually warped and corrected for rotation, lateral difference in position and anterior-posterior difference in position.
  • the step of superpositioning is carried out using identified locations of the static reference points in the image.
  • identified locations of the static reference points in the image For example, specific structures or regions, such as lung top, lung border, bony structures, such as ribs may be used for that purpose.
  • bony structures such as ribs
  • Those skilled in the art will readily appreciate which per se known algorithms may be used for enabling automatic delineation of the bony structures, such as ribs, in the diagnostic images under consideration.
  • image intensities may be subtracted yielding the corresponding sought difference. This difference may be used as an aid for making a diagnosis. It is found that in order to distinguish relevant medical differences from signals caused by noise and/or misplacements in the superpositioning process a correlation algorithm may be advantageously applied on the found differences after the steps of matching and subtraction carried out for the earlier diagnostic image and the later diagnostic image.
  • the earlier diagnostic image and the later diagnostic image comprise respective thorax images are selected.
  • a substantially non-variable reference mark such as a portion of a bone.
  • the method according to the invention comprises the steps of identification of a region of interest in the earlier image and in the later image; elimination of structures from the earlier image and the later image not corresponding to the selected set of marks and the region of interest. It is found that by eliminating the structures not part of the image of interest data processing may be further improved and accelerated. More details on this particular embodiment will be given with reference to Figure 3.
  • An embodiment of a system arranged for detecting changes in a radiograph is known from WO 99/05641.
  • a normalization algorithm is applied on the full image without suitably adjusting the region of interest. It will be appreciated that, in case the images are acquired using different equipment (different vendors, or the same vendor, but having different settings), different patient positioning, post processing or changes of patient's shape, there will be a substantial variation in lighting and/or contrast of the images. Accordingly, by applying the normalization step for the whole image without correcting the region of interest, a substantial distortion of data corresponding to the region of interest may occur. In particular, undesirable smoothing of the intensity data corresponding to lungs may be expected. Accordingly, very small
  • the known system has a still further disadvantage in that only a top portion of the lung is taken into account when carrying out segmentation of the image data.
  • such approach leads to substantial errors due to a difference in movement if the diaphragm and the lung.
  • segmentation of the left lung may be inaccurate in the known system because of the beating heart, which forms part of the image. Accordingly, with these limitations of the known system it appears that the known system uses only about 40% of the image data for carrying out segmentation. Such approach leads to substantial problems when matching the remaining parts of the image.
  • the known system is arranged to provide an image which results in a subtraction of the earlier and the later image.
  • this subtraction may be highly inaccurate.
  • some existing patterns within the region of interest, such as diaphragm and surroundings, may be poorly matched.
  • many clinical problems are occurring near the abdominal diaphragm. Accordingly, accurate
  • the method further comprises a step of applying a filter to the earlier diagnostic image and/or the later diagnostic image for further post-processing of the images.
  • a filter comprises a low pass smoothing filter based on data convolution. Those skilled in the art shall readily appreciate how such low pass filter may be defined.
  • the method further comprises the step of adjusting the image contrast prior to image registration.
  • the contrast may be adjusted locally or dynamically. It is found to be particularly advantageous to carry out contrast adjustment of the diagnostic images under consideration, for example to eliminate influence of accidental bright areas surrounding the actual image. This step is found advantageous because contrast adjustments may affect the identification of very small/low intensity differences between the images. This embodiment will be explained in more detail with reference to Figure 2.
  • the method further comprises the step of visualization of the area corresponding to a difference between the earlier and the later diagnostic image.
  • the detected differences are quantified and color coded for indicating the suspected medical relevance. More preferably, an image representing the color coded differences is superimposed on the later diagnostic image.
  • the computer program product for registration of diagnostic images according to the invention comprises instructions for causing a processor to carry out the steps of the method as is discussed in the foregoing.
  • the system for registration of diagnostic images taken for different time moments comprises:
  • system further comprises a display arranged to display the earlier diagnostic image, the later diagnostic image and the area or areas corresponding to a difference between the earlier diagnostic image and the later diagnostic image superimposed on the later diagnostic image.
  • system according to the invention further comprises a display arranged to display the earlier diagnostic image, the later diagnostic image and the area corresponding to a difference between the earlier diagnostic image and the later diagnostic image.
  • Figure 1 presents in a schematic way an embodiment of a method according to the invention.
  • Figure 2 presents in a schematic way an embodiment of a contrast adjustment step.
  • Figure 3 presents in a schematic way an embodiment of a region of interest identification step.
  • Figure 4 presents in a schematic way an embodiment of a further embodiment of a region of interest identification step.
  • Figure 5 presents an embodiment of an earlier diagnostic image.
  • Figure 6 presents an embodiment of a later diagnostic image.
  • Figure 7 presents an embodiment of a diagnostic image comprising color coded data on a difference between the earlier diagnostic image and the later diagnostic image.
  • FIG. 1 presents in a schematic way an embodiment of a method according to the invention. It will be appreciated that the given sequence of steps is not exhaustive, nor binding.
  • a pyramidal image decomposition may be carried out at step 2. For example, for each analyzed image a lowpass pyramid may be generated, for example by first smoothing the image with an appropriate smoothing filter and then sub- sampling the smoothed image by a factor of two along each coordinate direction. As this process proceeds, the result will be a set of gradually more smoothed images, wherein in addition the spatial sampling density is decreasing.
  • the low pass filter may be realized with a two dimensional convolution, implemented using two successive one dimensional convolutions along the rows and columns of the image.
  • step 4 suitable image processing (trimming) may take place. This step will be explained in more detail with reference to Figure 2.
  • an image central point may be established.
  • ROI region of interest
  • suitable identification of the region of interest may be carried out at step 8.
  • suitable landmarks may be identified.
  • the image comprising bony structures it may be preferable to use bones as suitable markers for purposes of image registration between the earlier diagnostic image and the later diagnostic image.
  • other structures or organs may be used for this purpose as well.
  • identification of markers use may be made of delineation of other areas and borders at the respective steps 9a, 9b. The borders may be obtained using knowledge on the expected structures in the diagnostic image. For example, for thorax images, ribs may be delineated using Gaussian filters 11.
  • the method according to the invention proceeds to steps of superpositioning of the earlier diagnostic image with the later diagnostic image at step 12, adjustment of the images, notably the regions of interest, at step 12a and subtraction of data in the images (or regions of interest) at step 12b.
  • Figure 2 presents in a schematic way an embodiment of a contrast adjustment step. It is found that during the processing of diagnostic images, such as radiographs, bright lines may appear in areas surrounding the actual image and cause undesirable interference with the feature detection algorithm used for detection of patterns in the diagnostic image under consideration.
  • the exemplary steps 12 - 18 may be applied to mitigate this problem.
  • duplication of low resolution image may be carried out.
  • a suitable auto contrast algorithm may be applied.
  • the auto contrast procedure expands the dynamic range of the image intensity by mapping the lightest and darkest pixels to maximum (for example, 255) and minimum (0) possible values.
  • maximum for example, 255
  • minimum minimum (0) possible values.
  • histogram clipping may be used.
  • the maximum value of the image intensity (lightest pixels) may be determined so that only 0.5% of the image pixels have the intensity higher than the maximum. For the minimum the same levels are applied, only 0.5% of the pixels have the intensity lower than the maximum. It will be appreciated, however, that any pre-determined value may be used for setting the respective lower and upper cut-off maximum/minimum intensities.
  • a search may be carried out for the first 20% of the image from each side; top, bottom, left and right.
  • the first row with a mean intensity value above 10 may be treated as signal level, the respective boundary of the image. All rows towards the image border may be duplicated with the intensity values found in this row.
  • the upper boundary may be set.
  • additional steps may be performed.
  • Figure 3 presents in a schematic way an embodiment of a region of interest identification step.
  • Thorax X-ray examination may contain a great number of informational structures.
  • the region of interest (ROI) for thorax are lung areas. It is found that for enabling accurate assessment of lung areas it may be preferable to eliminate other structures from the image, except for markers used for image registration.
  • the Gaussian of image A may be calculated as the convolutions of image intensity I(x,y) with the
  • the lowest intensity is determined.
  • the Gaussian(25) may be divided into two equal sides. For each side a search may be carried out on the top half of the image with the exclusion of a
  • an image region may be calculated.
  • An image region is a connected image area containing pixels with the same close value of intensity.
  • a list of image regions may be calculated using the intensity GmmLeft, GmmRight for both sides of image G(25). Regions with width and height more than 1 ⁇ 4 of image size may be excluded from the list.
  • G(25) one may find a central region which is closest to the center of the image. The central region may be used to determine coordinates of centre of both lungs in a thorax image, (Xi e ft, Yieft) and (X ng ht, Y ng ht) as
  • thoracic central point may be calculated as follows:
  • Figure 4 presents in a schematic way an embodiment of a further embodiment of a region of interest identification step.
  • the identification of the region of interest (ROI) may be done using Gaussian filters. This technique results in a clear separation between the ROI and structures outside this region.
  • the image region corresponding to the lung area and it's bounding rectangle may be stored.
  • unsharp masking may be carried out.
  • a Gaussian G(3) of image A may be formed with sigma 3
  • a Gaussian G(40) of image G(3) may be calculated with sigma 40. It will be appreciated that different values for sigma may be used as well.
  • An unsharp mask image may be calculated using G(3) and G(40) with an amplification factor of 5. It will be appreciated that these examples are not limiting.
  • step 34 combining of lung regions is carried out.
  • the Gaussian filter used at step 32 results in a clear separation between the lung and outside area. Within the lung area the collarbone may cause a complete region separation between the top part of the bottom part of the lung.
  • 1 image region of the total ROI for the left and right side lines may be drawn from the weighted center upwards with the following restrictions and settings:
  • dl length is restricted to distance between the weighted centre and dc/2;
  • a threshold adaptation algorithm may be applied/ Adaptation may be done iterative ly with thresholds decreasing from 50 till 27 with steps of 3 in intensity levels. However, other values may be used.
  • a binary image image region may be constructed from USM threshold.
  • the image regions created during the threshold adaptation algorithm may be tested against the following rules:
  • outer positions of image region are not less than a predetermined number of pixels (for example, 5) away from the image boundary;
  • Xieft, Yieft, Xright, Rright coordinates are located inside the image region
  • X and Y coordinates of top most pixel for both sides may be annotated as lung top coordinates (ptLeft, ptRight) he distance between X ce nter and x coordinate for lung top is not allowed to be greater than the distance between Xcenter and xieft;
  • step 38 all the above requirements are checked. Should they be met, the image region is treated as the ROI area. If only one region is found after adaptation iterations it is excluded and mirroring of coordinates from other side around X ce nter is used. It will be appreciated that use can be made of suitable structures, such as lung top, lung border or the like.
  • Figure 5 presents an embodiment of an earlier diagnostic image.
  • a thorax image is used.
  • the relevant structures may relate lung structures 51a and 51b.
  • the images are registered. However, it may be necessary to carry out the step of image registration prior to image warping. Because relevant differences are to be investigated with respect to the present state of the patient, it is found advantageous to warp the earlier diagnostic image to the later unrectified diagnostic image.
  • the procedure of image warping is known per se and may comprise the following steps:
  • Figure 6 presents an embodiment of a later diagnostic image 60.
  • the relevant structures are the ribs 62a, 62b and the lung 61a, 61b.
  • Figure 7 presents an embodiment of a later diagnostic image 70 comprising color coded data 71, 72, 74 representing the difference data, overlaid on the later diagnostic image.
  • the image may be appended with a suitable legend comprising segments 73a, 73b, 73c.
  • the difference image may comprise positive areas and negative areas. Interpretation of the difference data may be dependent upon a clinical case.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Biophysics (AREA)
  • Optics & Photonics (AREA)
  • Pathology (AREA)
  • Radiology & Medical Imaging (AREA)
  • Biomedical Technology (AREA)
  • High Energy & Nuclear Physics (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Apparatus For Radiation Diagnosis (AREA)

Abstract

The invention relates to a method for registration of diagnostic images taken for different time moments, comprising the steps of retrieving an earlier diagnostic transmission radiography image; retrieving a later diagnostic transmission radiography image; identifying a set of reference marks in the earlier diagnostic transmission radiography image and the later diagnostic transmission radiography image; superpositioning the said earlier image and the said later image by using a set of reference marks; identification of a region of interest in the earlier image and in the later image; elimination of structures from the earlier image and the later image not corresponding to the said of marks and the region of interest; automatically registering the earlier image and the later image; automatically identifying differences between the earlier image and the later image; automatically marking an area corresponding to a difference between the earlier image and the later image on the later image. The invention further relates to a system and a computer program for registration of transmission radiography diagnostic images of the same patient taken for different time moments.

Description

Title: Method, a system and a computer program product for registration and identification of diagnostic images
Field of the invention
The invention relates to a method for registration of transmission radiography diagnostic images taken at different time moments for the same patient.
The invention further relates to a system for registration of transmission radiography diagnostic images taken at different time moments for the same patient.
The invention still further relates to a computer program product for registration of transmission radiography diagnostic images taken for different time moments for the same patient.
Background of the invention
In medical practice quite often a patient is being examined for purposes of follow-up examination at different times using substantially the same field of view. For example, for thorax examinations, transmission X-ray images, also referred to as transmission radiography images, of lungs and surrounding tissue, such as ribs, may be taken.
It is a common practice for the radiologists to review the suitable images taken at different moments. These images will be further on referred to as the earlier diagnostic image(s) and the later diagnostic image(s). The procedure of said comparison is generally carried out using light tables or using computer displays, the latter for digitized images.
However, it is found to be quite challenging to carry out accurate assessment of the possible differences between the earlier diagnostic image(s) and the later diagnostic image(s) for a number of reasons. First, the field of view taken for these images may be different. Secondly, an image may be
(slightly) distorted due to organ or patient displacement. Thirdly, a plurality of different diagnostic apparatus having different operational settings may be used for acquiring the earlier and the later diagnostic image(s) under consideration. Fourth, the patient may have changed dimensions, because of age, operation or due to a weight change. And finally, human errors may occur during visual inspection of the images.
Summary of the invention
It is an object of the invention to provide a method for registration of diagnostic transmission radiography images taken for different time moments for the same patient, which is accurate and which allows for quantification of the differences between the earlier and the later diagnostic image.
To this end the method according to the invention comprises the steps of:
- retrieving an earlier diagnostic transmission radiography image; - retrieving a later diagnostic transmission radiography image;
- identifying a set of reference marks in the earlier diagnostic transmission radiography image and the later diagnostic transmission radiography image;
- superpositioning the said earlier image and the said later image by using a set of reference marks;
- identification of a region of interest in the earlier image and in the later image;
- elimination of structures from the earlier image and the later image not corresponding to the said of marks and the region of interest;
- automatically registering the earlier image and the later image;
- automatically identifying differences between the earlier image and the later image;
automatically marking an area corresponding to a difference between the earlier image and the later image on the later image. It is found that with a suitable image processing, drawbacks of the prior art may be mitigated to a substantial extent. For example, when the step of image registration is carried out, misalignments and distortion artefacts can be eliminated. It will be appreciated that any suitable image registration method may be used for practicing the invention. For example, in a region of interest one or more image segments may be identified. This identification may be carried out automatically or manually. Then, for example, linear image warping may be used for the identified segments making use of suitable patterns. Next, a homography matrix composed of rotation, shift and scale may be generated for the individual parts of the identified patterns. It will be appreciated that a great plurality (up to 100) segments may be identified within a region of interest. These segments may be individually warped and corrected for rotation, lateral difference in position and anterior-posterior difference in position.
It will be appreciated that although the invention may be carried out using two-dimensional X-ray transmission images, it is applicable to other imaging modalities and more dimensional images, such as 3D, 4D or 5D images. Those skilled in the art will readily appreciate which three-dimensional image registration algorithms may be applied for the 3D images.
Preferably, the step of superpositioning is carried out using identified locations of the static reference points in the image. For example, specific structures or regions, such as lung top, lung border, bony structures, such as ribs may be used for that purpose. Those skilled in the art will readily appreciate which per se known algorithms may be used for enabling automatic delineation of the bony structures, such as ribs, in the diagnostic images under consideration.
When the earlier image and the later diagnostic image are registered, image intensities may be subtracted yielding the corresponding sought difference. This difference may be used as an aid for making a diagnosis. It is found that in order to distinguish relevant medical differences from signals caused by noise and/or misplacements in the superpositioning process a correlation algorithm may be advantageously applied on the found differences after the steps of matching and subtraction carried out for the earlier diagnostic image and the later diagnostic image.
Preferably, for the earlier diagnostic image and the later diagnostic image comprise respective thorax images are selected. For the thorax images it is found advantageous to use a substantially non-variable reference mark, such as a portion of a bone.
The method according to the invention comprises the steps of identification of a region of interest in the earlier image and in the later image; elimination of structures from the earlier image and the later image not corresponding to the selected set of marks and the region of interest. It is found that by eliminating the structures not part of the image of interest data processing may be further improved and accelerated. More details on this particular embodiment will be given with reference to Figure 3.
An embodiment of a system arranged for detecting changes in a radiograph is known from WO 99/05641. In is a disadvantage of the known system that a normalization algorithm is applied on the full image without suitably adjusting the region of interest. It will be appreciated that, in case the images are acquired using different equipment (different vendors, or the same vendor, but having different settings), different patient positioning, post processing or changes of patient's shape, there will be a substantial variation in lighting and/or contrast of the images. Accordingly, by applying the normalization step for the whole image without correcting the region of interest, a substantial distortion of data corresponding to the region of interest may occur. In particular, undesirable smoothing of the intensity data corresponding to lungs may be expected. Accordingly, very small
abnormalities, such as tumors, may be missed due to a substantial reduction of their detectability due to smoothing. Another disadvantage of the known system is that it is adapted for correcting for lateral inclinations only. However, it will be appreciated that the patient images may be quite different, especially when they are taken with a considerable time span in between. In particular, it is found that the earlier and the later images usually have anterior-posterior differences in position as well as a rotational displacement. It will be appreciated that both these displacement need to be corrected for prior to matching the earlier and the later images.
The known system has a still further disadvantage in that only a top portion of the lung is taken into account when carrying out segmentation of the image data. However, such approach leads to substantial errors due to a difference in movement if the diaphragm and the lung. Similarly,
segmentation of the left lung may be inaccurate in the known system because of the beating heart, which forms part of the image. Accordingly, with these limitations of the known system it appears that the known system uses only about 40% of the image data for carrying out segmentation. Such approach leads to substantial problems when matching the remaining parts of the image.
Further, the known system is arranged to provide an image which results in a subtraction of the earlier and the later image. However, keeping in mind that the known system uses only about 40% of the available data, this subtraction may be highly inaccurate. In particular, some existing patterns within the region of interest, such as diaphragm and surroundings, may be poorly matched. However, it will be appreciated that many clinical problems are occurring near the abdominal diaphragm. Accordingly, accurate
visualization and data analysis of this particular region is highly needed.
In an embodiment of the method according to the invention the step of automatically identifying differences between the earlier diagnostic image and the later diagnostic image comprises the step of:
- transformation of the earlier diagnostic image; - subtraction of intensity signals between the registered earlier diagnostic image and the later diagnostic image.
It is found to be particular advantageous to allow the earlier diagnostic image to be suitably transformed, as presenting data on basis of the true later image may have a higher clinical relevance.
In a further embodiment of the method according to the invention, the method further comprises a step of applying a filter to the earlier diagnostic image and/or the later diagnostic image for further post-processing of the images. This embodiment is explained in further detail with reference to Figure 1. Preferably, the filter comprises a low pass smoothing filter based on data convolution. Those skilled in the art shall readily appreciate how such low pass filter may be defined.
In a still further embodiment of the method according to the invention the method further comprises the step of adjusting the image contrast prior to image registration.
The contrast may be adjusted locally or dynamically. It is found to be particularly advantageous to carry out contrast adjustment of the diagnostic images under consideration, for example to eliminate influence of accidental bright areas surrounding the actual image. This step is found advantageous because contrast adjustments may affect the identification of very small/low intensity differences between the images. This embodiment will be explained in more detail with reference to Figure 2.
In a still further embodiment of the method according to the invention, the method further comprises the step of visualization of the area corresponding to a difference between the earlier and the later diagnostic image.
Preferably, the detected differences are quantified and color coded for indicating the suspected medical relevance. More preferably, an image representing the color coded differences is superimposed on the later diagnostic image. The computer program product for registration of diagnostic images according to the invention comprises instructions for causing a processor to carry out the steps of the method as is discussed in the foregoing.
The system for registration of diagnostic images taken for different time moments comprises:
- a processor adapted to :
i. retrieving an earlier diagnostic transmission radiograph image;
ii. retrieving a later diagnostic transmission radiograph image iii. identifying a set of reference marks in the earlier diagnostic transmission radiograph image and the later diagnostic transmission radiograph image;
iv. superpositioning the said earlier image and the said later image by using a set of reference marks;
v. identifying of a region of interest in the earlier image and in the later image;
vi. eliminating of structures from the earlier image and the later image not corresponding to the said of marks and the region of interest;
vii. automatically registering the earlier image and the later image;
viii. automatically identifying differences between the earlier image and the later image;
ix. automatically marking an area corresponding to a difference between the earlier image and the later image on the later image.
In an embodiment of the system according to the invention it further comprises a display arranged to display the earlier diagnostic image, the later diagnostic image and the area or areas corresponding to a difference between the earlier diagnostic image and the later diagnostic image superimposed on the later diagnostic image. Preferably, the system according to the invention further comprises a display arranged to display the earlier diagnostic image, the later diagnostic image and the area corresponding to a difference between the earlier diagnostic image and the later diagnostic image.
These and other aspects of the invention will be explained in more detail with reference to figures. In the figures like elements are depicted using like reference signs. It will be appreciated that the figures are given for illustrative purposes only and may not be used for limiting the scope of the appended claims.
Brief description of the drawings
Figure 1 presents in a schematic way an embodiment of a method according to the invention.
Figure 2 presents in a schematic way an embodiment of a contrast adjustment step.
Figure 3 presents in a schematic way an embodiment of a region of interest identification step.
Figure 4 presents in a schematic way an embodiment of a further embodiment of a region of interest identification step.
Figure 5 presents an embodiment of an earlier diagnostic image.
Figure 6 presents an embodiment of a later diagnostic image.
Figure 7 presents an embodiment of a diagnostic image comprising color coded data on a difference between the earlier diagnostic image and the later diagnostic image.
Detailed description of the drawings
Figure 1 presents in a schematic way an embodiment of a method according to the invention. It will be appreciated that the given sequence of steps is not exhaustive, nor binding. In accordance with an embodiment of the method of the invention a pyramidal image decomposition may be carried out at step 2. For example, for each analyzed image a lowpass pyramid may be generated, for example by first smoothing the image with an appropriate smoothing filter and then sub- sampling the smoothed image by a factor of two along each coordinate direction. As this process proceeds, the result will be a set of gradually more smoothed images, wherein in addition the spatial sampling density is decreasing.
The low pass filter may be realized with a two dimensional convolution, implemented using two successive one dimensional convolutions along the rows and columns of the image.
At step 4 suitable image processing (trimming) may take place. This step will be explained in more detail with reference to Figure 2.
At step 6 the image is suitably processed, an image central point may be established. This step is explained in more detail with reference to Figure 3. Next, suitable identification of the region of interest (ROI) may be carried out at step 8. Those skilled in the art will readily appreciate which per se known method may be used for this purpose. More particular details on the ROI identification step are given with reference to Figure 4. Afterwards, at step 10 suitable landmarks may be identified. In case when the image comprising bony structures it may be preferable to use bones as suitable markers for purposes of image registration between the earlier diagnostic image and the later diagnostic image. However, other structures or organs may be used for this purpose as well. For example, for identification of markers use may be made of delineation of other areas and borders at the respective steps 9a, 9b. The borders may be obtained using knowledge on the expected structures in the diagnostic image. For example, for thorax images, ribs may be delineated using Gaussian filters 11.
After the set of markers is established at step 10, the method according to the invention proceeds to steps of superpositioning of the earlier diagnostic image with the later diagnostic image at step 12, adjustment of the images, notably the regions of interest, at step 12a and subtraction of data in the images (or regions of interest) at step 12b.
Figure 2 presents in a schematic way an embodiment of a contrast adjustment step. It is found that during the processing of diagnostic images, such as radiographs, bright lines may appear in areas surrounding the actual image and cause undesirable interference with the feature detection algorithm used for detection of patterns in the diagnostic image under consideration. The exemplary steps 12 - 18 may be applied to mitigate this problem.
First, at step 12 duplication of low resolution image may be carried out. At step 14 a suitable auto contrast algorithm may be applied. The auto contrast procedure expands the dynamic range of the image intensity by mapping the lightest and darkest pixels to maximum (for example, 255) and minimum (0) possible values. To determine the lightest and darkest pixels, histogram clipping may be used. The maximum value of the image intensity (lightest pixels) may be determined so that only 0.5% of the image pixels have the intensity higher than the maximum. For the minimum the same levels are applied, only 0.5% of the pixels have the intensity lower than the maximum. It will be appreciated, however, that any pre-determined value may be used for setting the respective lower and upper cut-off maximum/minimum intensities.
Next, at step 16 a search may be carried out for the first 20% of the image from each side; top, bottom, left and right. The first row with a mean intensity value above 10 may be treated as signal level, the respective boundary of the image. All rows towards the image border may be duplicated with the intensity values found in this row.
At step 18 the upper boundary may be set. For the top boundary additional steps may be performed. The row with the minimum intensity is searched within the predetermined level, say 5% of the image. All rows located above this minimum intensity row may be normalized by using I (x) = I (x) Smin/S, wherein I(x) refers to the intensity of the pixel at location x. Smm refers to the intensity sum of found minimum row and S refers to the sum of all pixel intensities of current row.
Figure 3 presents in a schematic way an embodiment of a region of interest identification step. Thorax X-ray examination may contain a great number of informational structures. The region of interest (ROI) for thorax are lung areas. It is found that for enabling accurate assessment of lung areas it may be preferable to eliminate other structures from the image, except for markers used for image registration.
In this exemplary embodiment at step 22 the Gaussian of image A may be calculated as the convolutions of image intensity I(x,y) with the
Gaussian 2nd kernel function. Preferably sigma = 25 is used yielding G(25).
At step 24 the lowest intensity is determined. For this purpose the Gaussian(25) may be divided into two equal sides. For each side a search may be carried out on the top half of the image with the exclusion of a
predetermined portion (20% for example) of the image boundaries for the lowest intensity signal of G > GmmLeft, GmmRight.
At step 26 an image region may be calculated. An image region is a connected image area containing pixels with the same close value of intensity. A list of image regions may be calculated using the intensity GmmLeft, GmmRight for both sides of image G(25). Regions with width and height more than ¼ of image size may be excluded from the list. In each side of image G(25) one may find a central region which is closest to the center of the image. The central region may be used to determine coordinates of centre of both lungs in a thorax image, (Xieft, Yieft) and (Xnght, Ynght) as
X— (Xmin+Xmax)/2; Y = (Ymin+Ymax)/2.
Xmin - minimum horizontal coordinate of pixels from central image region;
Xmax - maximum horizontal coordinate of pixels from central image region; Ymin - minimum vertical coordinate of pixels from central image region;
Ymax - maximum vertical coordinate of pixels from central image region.
It will be appreciated that this example is given for one side of the lungs of a thorax photo taken as an exemplary embodiment of the diagnostic image. In practice the calculation may be done for both sides.
At step 28 a central point may be identified. For example, thoracic central point may be calculated as follows:
Xcenter = (Xleft + Xright)/2
Ycenter— (Yleft Yright)/2
Next, identification of lung areas may be carried out, which is described with reference to Figure 4.
Figure 4 presents in a schematic way an embodiment of a further embodiment of a region of interest identification step. The identification of the region of interest (ROI) may be done using Gaussian filters. This technique results in a clear separation between the ROI and structures outside this region. The image region corresponding to the lung area and it's bounding rectangle may be stored.
At step 32 unsharp masking may be carried out. For the unsharp masking algorithm a Gaussian G(3) of image A may be formed with sigma 3, for example, a Gaussian G(40) of image G(3) may be calculated with sigma 40. It will be appreciated that different values for sigma may be used as well. An unsharp mask image may be calculated using G(3) and G(40) with an amplification factor of 5. It will be appreciated that these examples are not limiting.
At step 34 combining of lung regions is carried out. The Gaussian filter used at step 32 results in a clear separation between the lung and outside area. Within the lung area the collarbone may cause a complete region separation between the top part of the bottom part of the lung. In order to achieve 1 image region of the total ROI for the left and right side lines may be drawn from the weighted center upwards with the following restrictions and settings:
lines drawn with a 3 pixel width;
- dl line is drawn from the weighted centre coordinate till height coordinate image hc/7;
dl length is restricted to distance between the weighted centre and dc/2;
drawn lines in left and right side are inclined to centre of image by dl/8.
At step 36 a threshold adaptation algorithm may be applied/ Adaptation may be done iterative ly with thresholds decreasing from 50 till 27 with steps of 3 in intensity levels. However, other values may be used. For each step a binary image image region may be constructed from USM threshold.
The image regions created during the threshold adaptation algorithm may be tested against the following rules:
outer positions of image region are not less than a predetermined number of pixels (for example, 5) away from the image boundary;
Xieft, Yieft, Xright, Rright coordinates are located inside the image region;
X and Y coordinates of top most pixel for both sides may be annotated as lung top coordinates (ptLeft, ptRight) he distance between Xcenter and x coordinate for lung top is not allowed to be greater than the distance between Xcenter and xieft;
Difference in Y coordinates of lung top coordinates is not allowed to be greater than distance/const, for example const = 10.
It will be appreciated that different approaches for implementing testing may be used. At step 38 all the above requirements are checked. Should they be met, the image region is treated as the ROI area. If only one region is found after adaptation iterations it is excluded and mirroring of coordinates from other side around Xcenter is used. It will be appreciated that use can be made of suitable structures, such as lung top, lung border or the like.
Figure 5 presents an embodiment of an earlier diagnostic image. In this particular embodiment a thorax image is used. In the thorax image 50 the relevant structures may relate lung structures 51a and 51b.
When the earlier diagnostic image and the later diagnostic image are processed as described above, the images are registered. However, it may be necessary to carry out the step of image registration prior to image warping. Because relevant differences are to be investigated with respect to the present state of the patient, it is found advantageous to warp the earlier diagnostic image to the later unrectified diagnostic image.
The procedure of image warping is known per se and may comprise the following steps:
checking and adjustment of the feature detection points
(markers);
normalization of point coordinates;
estimation of widths of specific structures, such as bony structures (ribs);
calculation of a central point;
selection/normalization of points in the surroundings (ribcage points);
labeling;
filtering for displacement;
correlation refinement for motion correction;
recalculation of reference points in the surroundings (rib cage points);
calculation of displacement quality measures; warping the image.
Figure 6 presents an embodiment of a later diagnostic image 60. In this image the relevant structures are the ribs 62a, 62b and the lung 61a, 61b. Figure 7 presents an embodiment of a later diagnostic image 70 comprising color coded data 71, 72, 74 representing the difference data, overlaid on the later diagnostic image. For the convenience, the image may be appended with a suitable legend comprising segments 73a, 73b, 73c. It will be appreciated that the difference image may comprise positive areas and negative areas. Interpretation of the difference data may be dependent upon a clinical case.
It is found that provision of such quantified overlaid image comprising accurate information on the difference in volumes of regions of interest of the same patient over time has substantial added value. It will be further appreciated that because the invention is based on analysis of actual images its results are more robust than results obtained using the methods known in the art, such as applications based on pattern recognition and modeling.
It will be appreciated that the invention is not limited to the embodiments which have been just described. It will be appreciated that the invention may be practiced otherwise than as described.

Claims

Claims
Method for registration of diagnostic transmission radiography images taken for different time moments, comprising the steps of:
- retrieving an earlier diagnostic transmission radiography image;
- retrieving a later diagnostic transmission radiography image;
- identifying a set of reference marks in the earlier diagnostic transmission radiography image and the later diagnostic transmission radiography image;
- superpositioning the said earlier image and the said later image by using a set of reference marks;
- identification of a region of interest in the earlier image and in the later image;
- elimination of structures from the earlier image and the later image not corresponding to the said of marks and the region of interest;
- automatically registering the earlier image and the later image;
- automatically identifying differences between the earlier image and the later image;
- automatically marking an area corresponding to a difference between the earlier image and the later image on the later image.
The method according to claim 1, wherein the step of automatically identifying differences between the earlier diagnostic image and the later diagnostic image comprises the step of:
- transformation of the earlier diagnostic image;
- subtraction of intensity signals between the registered earlier diagnostic image and the later diagnostic image. The method according to claim 1 or 2, further comprising a step of applying a filter to the earlier diagnostic image and/or the later diagnostic image for post-processing of the images.
The method according to claim 3, wherein the filter comprises a low pass smoothing filter based on data convolution.
The method according to any one of the preceding claims, further comprising the step of adjusting the image contrast prior to image registration.
The method according to any one of the preceding claims, wherein the earlier diagnostic image and the later diagnostic image comprise an image of a displaceable organ.
The method according to claim 6, wherein the earlier diagnostic image and the later diagnostic image comprise a thorax image.
The method according to any one of the preceding claims, wherein for the at least one reference mark a portion of a bone is selected.
The method according to any one of the preceding claims, further comprising the step of visualization of the area corresponding to a difference between the earlier and the later diagnostic image.
The method according to claim 9, wherein said visualization is carried out using a color code. System and a computer program product for registration of diagnostic transmission radiograph images taken for different time moments, comprising:
- a processor adapted to :
i. retrieving an earlier diagnostic transmission radiograph image;
ii. retrieving a later diagnostic transmission radiograph image iii. identifying a set of reference marks in the earlier diagnostic transmission radiograph image and the later diagnostic transmission radiograph image;
iv. superpositioning the said earlier image and the said later image by using a set of reference marks;
v. identifying of a region of interest in the earlier image and in the later image;
vi. eliminating of structures from the earlier image and the later image not corresponding to the said of marks and the region of interest;
vii. automatically registering the earlier image and the later image;
viii. automatically identifying differences between the earlier image and the later image;
ix. automatically marking an area corresponding to a difference between the earlier image and the later image on the later image.
The system according to claim 11, further comprising a display arranged to display the earlier diagnostic image, the later diagnostic image and the area corresponding to a difference between the earlier diagnostic image and the later diagnostic image. Computer program product for registration of diagnostic images comprising instructions for causing a processor to carry out the steps of the method as is claimed in claims 1 - 10.
EP12730680.1A 2011-06-14 2012-06-12 Method, a system and a computer program product for registration and identification of diagnostic images Withdrawn EP2720613A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP12730680.1A EP2720613A1 (en) 2011-06-14 2012-06-12 Method, a system and a computer program product for registration and identification of diagnostic images

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
EP11169814A EP2535001A1 (en) 2011-06-14 2011-06-14 Method, a system and a computer program product for registration and identification of diagnostic images
EP12730680.1A EP2720613A1 (en) 2011-06-14 2012-06-12 Method, a system and a computer program product for registration and identification of diagnostic images
PCT/NL2012/050407 WO2012173470A1 (en) 2011-06-14 2012-06-12 Method, a system and a computer program product for registration and identification of diagnostic images

Publications (1)

Publication Number Publication Date
EP2720613A1 true EP2720613A1 (en) 2014-04-23

Family

ID=44898248

Family Applications (2)

Application Number Title Priority Date Filing Date
EP11169814A Withdrawn EP2535001A1 (en) 2011-06-14 2011-06-14 Method, a system and a computer program product for registration and identification of diagnostic images
EP12730680.1A Withdrawn EP2720613A1 (en) 2011-06-14 2012-06-12 Method, a system and a computer program product for registration and identification of diagnostic images

Family Applications Before (1)

Application Number Title Priority Date Filing Date
EP11169814A Withdrawn EP2535001A1 (en) 2011-06-14 2011-06-14 Method, a system and a computer program product for registration and identification of diagnostic images

Country Status (2)

Country Link
EP (2) EP2535001A1 (en)
WO (1) WO2012173470A1 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014155346A2 (en) * 2013-03-29 2014-10-02 Koninklijke Philips N.V. Image registration
FR3007962B1 (en) * 2013-07-04 2015-06-26 X Nov Ip GRAPHICAL SELECTION OF BONE ANCHOR PROSTHESIS
EP3061073B1 (en) * 2013-10-22 2019-12-11 Koninklijke Philips N.V. Image visualization

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH08103439A (en) * 1994-10-04 1996-04-23 Konica Corp Alignment processor for image and inter-image processor

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5359513A (en) * 1992-11-25 1994-10-25 Arch Development Corporation Method and system for detection of interval change in temporally sequential chest images
US5982915A (en) * 1997-07-25 1999-11-09 Arch Development Corporation Method of detecting interval changes in chest radiographs utilizing temporal subtraction combined with automated initial matching of blurred low resolution images
US6901277B2 (en) * 2001-07-17 2005-05-31 Accuimage Diagnostics Corp. Methods for generating a lung report
EP1817744A2 (en) * 2004-11-22 2007-08-15 Koninklijke Philips Electronics N.V. Improved data representation for rtp
US8788012B2 (en) * 2006-11-21 2014-07-22 General Electric Company Methods and apparatus for automatically registering lesions between examinations
US8355552B2 (en) * 2007-06-20 2013-01-15 The Trustees Of Columbia University In The City Of New York Automated determination of lymph nodes in scanned images

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH08103439A (en) * 1994-10-04 1996-04-23 Konica Corp Alignment processor for image and inter-image processor

Also Published As

Publication number Publication date
WO2012173470A1 (en) 2012-12-20
EP2535001A1 (en) 2012-12-19

Similar Documents

Publication Publication Date Title
JP5337845B2 (en) How to perform measurements on digital images
US7545965B2 (en) Image modification and detection using massive training artificial neural networks (MTANN)
CN109124662B (en) Rib center line detection device and method
JP2001511570A (en) A method for detecting interval changes in radiographs
US7593762B2 (en) System and method for automatically segmenting bones in computed tomography angiography data
JPH05176919A (en) Method and apparatus for image treatment
Marias et al. A registration framework for the comparison of mammogram sequences
US20060110022A1 (en) Automatic image contrast in computer aided diagnosis
US8229189B2 (en) Visual enhancement of interval changes using temporal subtraction, convolving, and non-rigid transformation field mapping
WO2012173470A1 (en) Method, a system and a computer program product for registration and identification of diagnostic images
Yoshida Local contralateral subtraction based on bilateral symmetry of lung for reduction of false positives in computerized detection of pulmonary nodules
US8224046B2 (en) Visual enhancement of interval changes using rigid and non-rigid transformations
US8229190B2 (en) Visual enhancement of interval changes using temporal subtraction and pattern detector
US10307124B2 (en) Image display device, method, and program for determining common regions in images
Gooßen et al. A stitching algorithm for automatic registration of digital radiographs
JP6642048B2 (en) Medical image display system, medical image display program, and medical image display method
Iakovidis et al. Robust model-based detection of the lung field boundaries in portable chest radiographs supported by selective thresholding
Schmidt A method for standardizing MR intensities between slices and volumes
Mustra et al. Nipple detection in craniocaudal digital mammograms
Ionescu et al. Comparative study of contour detection methods for intestinal sessile polyps
Ding et al. Automatic segmentation of cortical vessels in pre-and post-tumor resection laser range scan images
Wu et al. Automatic quantitative assessment of the small bowel motility with cine-MRI sequence analysis
Piekar et al. Segmentation of images using gradient methods and polynomial approximation
WO2021044358A1 (en) Method and system for image normalisation
Fischer et al. Bundle adjustment for marker-based rigid MR/X-ray registration

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20140113

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

DAX Request for extension of the european patent (deleted)
17Q First examination report despatched

Effective date: 20190227

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20190710