Classifications

• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B21/00—Microscopes
  • G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
  • G02B21/365—Control or image processing arrangements for digital or video microscopes
  • G02B21/367—Control or image processing arrangements for digital or video microscopes providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison

Definitions

• the present invention relates generally to microscopy.
• the present invention relates to image processing of microscopy images to improve image 5 contrast.
• Microscopy as a technique for the imaging of biological cells has been in use for many hundreds of years, over which period many techniques have been developed to improve the quality of images that are obtained of such biological cells when using 10 various microscopes .
• microscopy techniques have seen an evolution from complex intricate optics-based solutions for image enhancement [1-6] towards more recent developments that apply various image processing techniques [7-9] to images obtained in order to enhance images.
• an imaging system for enhancing microscopic images of unstained cells.
• the imaging system comprises a light source for producing light, a sample holder for containing cells to be imaged, a condenser for focussing the light at a focal plane within the sample holder on the cells to be imaged a translation mechanism for moving the focal plane of the light relative to the sample holder and a detector system configured to acquire a plurality of images at respective focal planes within the sample holder near to the best focus and process the plurality of images to provide an enhanced processed imaged.
• a method for enhancing microscopic images of unstained cells comprises acquiring a plurality of images at respective focal planes within the sample holder and processing the plurality of images to provide an enhanced processed imaged.
• Such an imaging system and method provides for automated image acquisition and enhancement for cellular imaging.
• it also allows non-experts to use aspects of the present invention in high throughput screening assays with better accuracy in identifying cellular changes that are indicative of an effect of potential interest.
• potentially useful drug compounds effecting various types of cell can be identified from automated imaging of those cells.
• these cells may also be accurately imaged in-vivo ensuring that there is no requirement for them to be stained or tagged, e.g. using fluorescent markers or radio-isotopes, in order to obtain the enhanced processed images.
• Figure 1 shows an imaging system for enhancing microscopic images of unstained cells in accordance with an embodiment of the present invention
• Figure 2 shows a method for enhancing microscopic images of unstained cells in accordance with an embodiment of the present invention
• Figure 3 shows part of an optical system for use with various embodiments of the present invention
• Figure 4 shows dark contrast provided by cell lensing
• Figure 5 shows original and processed images obtained in accordance with a method according to the present invention.
• Figure 6 shows the images of Figure 5 with a threshold filter applied to the processed images obtained in accordance with the method according to the present invention.
• FIG 1 shows an imaging system 100 for enhancing microscopic images of unstained cells in accordance with an embodiment of the present invention.
• the imaging system 100 which is illustrated schematically for clarity, comprises a light source 102 for producing light 120a.
• the light source 102 provides bright- field Kohler illumination.
• the light 120a is focussed by a condenser 104 onto a sample holder 109.
• the sample holder 109 can be used to contain cells to be imaged, and the condenser 104 can focus the light 120b at a focal plane within the sample holder 109.
• One or more sample holders 109 can be provided as a matrix in a sample holder tray 108.
• sample holders may be provided as spots on a consumable spot plate having an array of such sample holders, each respective spot being provided with one or more cell- types suitable for identifying a particular biochemical reaction or set of reactions.
• the imaging system 100 also contains a detector system 112 and a translation mechanism (not shown).
• the translation mechanism is configured to move the focus of the light 120b relative to the sample holder 109 (e.g. by moving the sample holder tray 108 depth- wise in the ⁇ z-directions). This enables a plurality of images to be acquired at various respective focal planes (perpendicular to the z-direction) from within various sample holders 109.
• the translation mechanism may be operable to move the sample holder tray 108 in various x-y planes shown in Figure 1 in either one or two dimensions, for example, to bring various sample holders 109 into focus.
• the detector system 112 may be used to identify a best focus within the sample holder 109.
• the detector system 112 is operable to acquire a plurality of images at respective focal planes (e.g. z-positions) within the sample holder 109 (e.g. near to the best focus), and to process the plurality of images to provide an enhanced processed imaged.
• focal planes e.g. z-positions
• process the plurality of images e.g. near to the best focus
• An aperture stop 106 is provided between the light source 102 and the detector system 112, the size of which may be variable. For example, various differently sized movable apertures may be rotated into position or a continuously variable iris-type diaphragm may be provided. Image contrast can be controlled by changing the aperture setting of the aperture stop 106.
• Focussed light 120b passing through the aperture stop 106 passes through the sample holder tray 108 in a transmission imaging mode.
• Emergent light 120c modulated with image information relating to any cells held with a respective sample holder 109 is collected by an objective lens 110 and focussed 12Od onto the detector system 112.
• processing of the plurality of images is performed by a processor (not shown).
• the processor is operable to control the detector system 112 and to acquire, store and process images obtained by the detector system. Additionally, the processor can be configured to control the translation mechanism to move the focus of the light source 102 relative to the sample holder 109 (i.e. the depth- wise position in the z- direction) and optionally to identify the best focus (i.e. optimal z-position setting).
• the processor may, for example, be provided as part of a computer system appropriately programmed to perform such tasks.
• the processor is operable to apply a pixel-by-pixel minimisation operation to each of the plurality of images to identify a so-called dark image, Uj.
• a non-linear top-hat (NTH) transform is then applied to the dark image Uj to obtain an intermediate image U 2 and the intermediate image U 2 is processed to produce a resultant enhanced image
• the basis for the NTH segmentation is the pixel-wise image transform:
• NTH(U; ⁇ ⁇ K) U' ⁇ U > " , where K > 1 - Equation (1)
• the scale parameter ⁇ (the detection scale) relates to an object's size of interest.
• the transform described by Equation (1) implements the following heuristic formula of local enhancement: a pixel is enhanced if it is bright, its close vicinity (e-sized) is bright, and its far vicinity (K ⁇ -sizG ⁇ ) is dim.
• the intermediate image is defined by:
• inv[..] is the image inversion operation.
• U T defines a threshold image.
• U T may be set to C/ ⁇ 1 for all pixel values.
• contrast is proportional to the amount of defocusing.
• the Applicant has applied the theory that certain cells act as lenses to show that the cell lensing effect can be used to enhance image contrast by removing bright feature information and using the dark feature information.
• this criterion is met, various advantages are obtained as outlined below.
• Suitable cells that can give rise to lensing effect may be substantially lenticular in shape, such as certain angiogenic cells, blood cells, sperm cells, keratocytes, stem cells, or various other cells having the shape of a sack with prevalent positive curvature.
• various cell nuclei as well as cell bodies can be imaged and/or numerous z-plane sections may be collated into a single image for easy automated, or semi- automated, analysis. Rapid and accurate identification of various cellular features of interest can thus be provided, thereby making various embodiments of the present invention particularly well-suited to applications involving HTS.
• Figure 2 shows a method 200 for enhancing microscopic images of unstained cells in accordance with various aspects and embodiments of the present invention.
• the method 200 comprises acquiring and processing a set of images to provide an enhanced processed image, and may be used for enhancing microscopy images of unstained cells by obtaining a plurality of images in the z-direction using axial defocusing.
• an image is obtained at a first focal plane.
• this image can be obtained using an imaging system of the type described in connection with Figure 1 , above.
• a further step of setting an aperture stop prior to obtaining the image can be performed, for example, in order to enhance the contrast of the image.
• the image is stored at step 204.
• a decision is then made to determine whether or not any further images are to be acquired.
• a z-stage translation is made to modify the position of the focal plane within a sample at step 208.
• the method 200 then moves back to step 202 and a further image is obtained.
• the further image is then stored at step 204 and the decision step 206 repeats the z-stage translation, image acquisition and storage steps until a set numbering k images is obtained (where k is an integer > 2). Once the plurality of k images has been obtained, the method 200 moves on to processing step 210.
• Processing step 210 involves applying a pixel-by-pixel minimisation operation to each of the plurality of images to identify a dark image Uj, then applying a non-linear top- hat transform to the inverted dark image U / to obtain an intermediate image U 2 , and finally processing the intermediate image U 2 to produce a resultant segmented image U r .
• the values of Uj, U 2 , and U r are determined in accordance with respective of Equations 2 to 4, defined above.
• Figure 3 shows part of an optical system for use with various embodiments of the present invention.
• the partial optical system shows an upper objective lens 110 for focussing light on to a camera forming part of a detector system 112 and a lenticular cell 150 of diameter D ce ⁇ in various positions along the z-axis of the central optical axis of an imaging system 100.
• the cell 150 is displaced from a focal plane F of the objective lens 110 by a distance p ce u along the z-axis in a direction away from the objective lens 110. This produces a dark image (shown on the lower left of Figure 3) at the camera.
• the cell 150 is displaced from a focal plane F of the objective lens 110 by a distance p ce u along the z-axis in a direction towards the objective lens 110. This produces a bright field image (shown on the lower right of Figure 3) at the camera.
• biological cells can represent themselves as micron-sized sacks filled by cytosole and other biological material, and therefore, from the viewpoint of geometric optics, the majority of cells have a shape resembling a convex lens.
• Cells in the sample might be distributed along a z-axis (e.g. in a thick sample), so they do not all appear simultaneously in focus. Therefore, multiple z-planes may be imaged to gather complementary information from throughout the depth of the sample.
• the medium in which cells are immersed is implied to be uniform and transparent, and n c > n m .
• Equation (5) follows immediately from the lens formula and lens maker equations. Signs (+) and (-) correspond to the "bright” and “dark” field cases. It can be shown that at small ⁇ n ⁇ , a Taylor expansion of arc tangent in (4) in the vicinity o ⁇ tg(cc) gives:
• the herein described procedure results in the creation of an effective "projection" image that may be used to gather information from different depths inside a thick sample.
• the procedure is similar to known "best focus" ("extended focus") algorithms applied for fluorescence modality (and may, for example, use the same GUI) such as, for example, those defined using Equations 7 and 8 as follows:
• a pixel-wise transform FocusMeasure[J] maybe defined as:
• the result is the weighted sum of images with focus measure:
• Figure 5 shows original 502, 504 and processed images 512, 514 obtained in accordance with a method according to the present invention.
• Uppermost original image 502 is derived from a sample containing non-stimulated angiogenic cells.
• the lower original image 504 is for a sample containing angiogenic cells stimulated by vascular endothelial growth factor (VEGF) and tissue necrosis factor ⁇ (TNF ⁇ ).
• VEGF vascular endothelial growth factor
• TNF ⁇ tissue necrosis factor ⁇
• V, mm k ⁇ V ⁇ Z k )Y
• a nonlinear top-hat transform is parameterised by a characteristic size ⁇ , which is close to a typical cell size.
• the nonlinear top-hat transform was applied in this case for shading elimination since expression i), relating to intensity ratio, only provides a localised estimate.
• Enhanced grey scale processed images 512, 514 are thus produced by the application of this method according to various aspects of the present invention which was designed to detect dark features present in the images due to the cell lensing effect at
• Figure 6 shows the original images 502, 504 of Figure 5 with a threshold filter applied to the processed images 512, 514 thereof giving rise to thresholded processed images 522, 524.
• Thresholded processed images 522, 524 are thus binary (i.e. black and white) images U r provided in accordance with:
• a computer program product may be provided that is operable to configure an imaging system to implement one or more method steps of various algorithms according to embodiments of the present invention.
• Various embodiments may also include one or more of software, hardware and/or firmware components.
• conventional imaging systems might be upgraded by using software components transmitted to those systems, for example, via the Internet in order to enhance their functionality in accordance with the present invention.
• processing of a plurality of images comprises identifying intensity changes between respective of the plurality of images caused by defocusing of light by cells in the sample and looking for dark intensity or light field intensity changes.
• One advantage of such embodiments is that it enables rapid size-tuned detection of cells to be performed, either automatically or otherwise. For example, a yes/no answer can be rapidly obtained to indicate the presence or absence of certain cells whose properties are known in advance.
• Certain aspects and embodiments of the present invention may use unstained cells that have a low refractive index mismatch with their surroundings.
• High-throughput screening e.g. for drug screening
• Various aspects and embodiments of the present invention may also be used as part of an automated microscope, e.g. in a GE EvT Cell Analyzer 1000TM that is commercially available from GE Healthcare Life Sciences, Little Chalfont, Buckinghamshire, U. K.
• Such an automated microscope is easy to use and can be used by non-expert users, for example, to analyse multiple cell samples. Additions of various aspects and embodiments of the present invention to such an automated microscope can thus not only make these even easier to use but also provide enhanced resultant images in the process.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Physics & Mathematics (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Chemical & Material Sciences (AREA)
• Analytical Chemistry (AREA)
• General Physics & Mathematics (AREA)
• Optics & Photonics (AREA)
• Microscoopes, Condenser (AREA)
• Investigating Or Analysing Materials By Optical Means (AREA)

Priority Applications (3)

Applications Claiming Priority (2)

Publications (1)

Family

ID=39767536

Family Applications (1)

Country Status (5)

Cited By (2)

Families Citing this family (8)

Citations (2)

Family Cites Families (17)

• 2008
  • 2008-08-05 GB GBGB0814297.8A patent/GB0814297D0/en not_active Ceased
• 2009
  • 2009-07-30 US US13/056,824 patent/US8760506B2/en active Active
  • 2009-07-30 JP JP2011521539A patent/JP2011530094A/ja active Pending
  • 2009-07-30 EP EP09781299A patent/EP2321689A1/en not_active Withdrawn
  • 2009-07-30 WO PCT/EP2009/059881 patent/WO2010015563A1/en not_active Ceased

Patent Citations (2)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events