WO2018064251A1

WO2018064251A1 - Methods of aligning multiple images of a biological sample

Info

Publication number: WO2018064251A1
Application number: PCT/US2017/053874
Authority: WO
Inventors: Chao-Ting Wu
Original assignee: President And Fellows Of Harvard College
Priority date: 2016-09-28
Filing date: 2017-09-28
Publication date: 2018-04-05

Abstract

A method of image registration or alignment of multiple images of a three-dimensional sample is provided including scoring a detectable pattern on the three-dimensional sample, creating multiple images of the three-dimensional sample with the detectable pattern, and aligning the detectable pattern of each image of the multiple images to align the multiple images.

Description

METHODS OF ALIGNING MULTIPLE IMAGES OF A BIOLOGICAL SAMPLE

RELATED APPLICATION DATA

This application claims priority to U.S. Provisional Application No. 62/400,670 filed on September 28, 2016 which is hereby incorporated herein by reference in its entirety for all purposes.

STATEMENT OF GOVERNMENT INTERESTS

This invention was made with government support under Grant No. DP1GM106412 awarded by National Institutes of Health. The government has certain rights in the invention.

FIELD

Aspects of the present disclosure generally relate to image registration of biological images.

BACKGROUND

Biological image analysis may use several image registration techniques including TrakEM2, SURF + affine transformation, Unwarpj, bUnwarpJ, CLAHE or BrainAligner. See also Makela et al., A Review of Cardiac Image Registration Methods, IEEE Trans. Med. Imag. 21, 1011-21 (2002), Oliveira et al., Medical Image Registration: A Review, Comput. Methods Biomech. Biomed. Engin, 17, 79-93 (2014) and Zitova et al., Image Registration Methods: A Survey, Imag. Vis. Comput. 21 977-1000 (2003). Conventionally, multiple images of a biological sample are taken and registered or aligned to identify the relative location of features of the biological sample using image fusion or serial image comparison. See Tan et al., 3D Reconstruction from 2D Images with Hierarchical Continuous Simplices, Visual Comput. 23, 905-914 (2007) and Peng et al., V3D Enables Real-Time 3D Visualization and Quantitative Analysis of Large Scale Biological Image Data Sets, Nature Biotechnology 28, 348-353 (2010). Serial slides can be manually aligned by setting up a number of pairs of corresponding control points to the same (x, y) location for consecutive images and Zi+i , and the pairs of images and the paired sets of control points are then given to semi-automatic software for image alignment. See Fiala, Reconstruct: A Free Editor for Serial Section Microscopy, /. Microscopy 218, 52-61 (2005). For small objects or samples, a technique called expansion microscopy (ExM) has been developed to "inflate" the object before imaging, such that it becomes big enough for standard microscopy. The ExM procedure starts like a standard immuno-fluorescence, but before imaging the sample, it is infused with a swellable polymer network. During polyerization, the fluorophore that is on the secondary antibody becomes covalently linked to the polymer. All proteins are then digested away and the polymer - now bearing an imprint of the sample in the form of bound fluorophores - is dilated to an extended conformation by desalting, leading to an isotropic enlargement of the imprint. The imprint can then be imaged at superresolution using a standard microscope. Such technique is however, limited to fixed samples. See Chen F, Tillberg PW, Boyden ES, Optical imaging. Expansion microscopy, Science, 347(6221):543-8 (2015).

The processing of a biological sample can produce deformation problems such as morphological deformation caused by cutting and fixation, stain variations, stain artifacts, rotation, translation and missing sample components. Additional negative issues with image alignment include distortions associated with technical limitations of the imaging process, sample movement, movement of objects within the sample, nonuniform movement of objects within the sample, and drift correction when aligning images at various depths of the sample.

These effects can make image registration difficult. Accordingly, methods are needed to overcome the problems associated with the processing and imaging of a biological sample and their effect on image registration.

SUMMARY

The disclosure provides methods for aligning multiple images of a sample to accurately depict the relative positions of objects in the sample. According to one aspect, the sample is scored or cut to mark the surface of the sample using a suitable design such as a grid formed from parallel lines. The scoring produces a pattern of detectable lines or planes on, in or through the sample. The scoring may continue through the sample to a desired depth. The scoring may continue through the entire sample from one end of the sample to the other. Multiple images of the sample are taken. Reconstruction of the patterns of lines or planes from the multiple images aligns the images of the sample and the objects or features within the sample. According to one aspect, the sample is scored or cut using a laser of suitable wavelength and frequency. The laser may generate a line when it pierces the sample at its surface or the laser may generate a plane when the laser is used to cut through the sample.

According to the present disclosure, a method of image registration or alignment of multiple images of a three-dimensional sample is provided including scoring a detectable pattern on the three-dimensional sample, creating multiple images of the three-dimensional sample with the detectable pattern, and aligning the detectable pattern of each image of the multiple images to align the multiple images. In one embodiment, the images are digital images. In one embodiment, the three-dimensional sample is a biological sample. In another embodiment, the three-dimensional sample is a three-dimensional matrix including a biological sample. In one embodiment, the pattern is scored on the three-dimensional sample using a laser.

In one embodiment, the multiple digital images are created using a digital imaging apparatus. In one embodiment, the detectable pattern is a plurality of scored lines or planes. In certain embodiments, the detectable pattern is a plurality of parallel or nonparallel scored lines or parallel or nonparallel planes. In one embodiment, the three-dimensional sample with the detectable pattern is fixed before imaging. In another embodiment, the three-dimensional sample with the detectable pattern is fixed immediately after creating of the scored pattern. In yet another embodiment, the three-dimensional sample with the detectable pattern is fixed after a certain amount of time has passed after creating of the scored pattern. In still another embodiment, the multiple digital images are processed using software to align the detectable patterns of the multiple digital images into a one coordinate system.

Further features and advantages of certain embodiments of the present invention will become more fully apparent in the following description of embodiments and drawings thereof, and from the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee. The foregoing and other features and advantages of the present embodiments will be more fully understood from the following detailed description of illustrative embodiments taken in conjunction with the accompanying drawings in which: FIG. 1 depicts a schematic showing a laser cutting a grid onto a cell sample. The cell sample can be imaged multiple times with the grid of each image being aligned. While the figure example has only depicted the use of lasers to create slice (plane) of laser damage, a skilled in the art can readily appreciate that nonmoving pulses of laser can be applied to create lines/tunnels of damage through the cell sample.

DETAILED DESCRIPTION

The disclosure provides a method of aligning or registering multiple images of a three-dimensional sample. The three-dimensional sample is scored or cut to produce a detectable pattern in or on or through the three-dimensional sample. Multiple images, such as digital images, are taken of the three-dimensional sample with the detectable pattern. The multiple images are aligned by aligning the detectable pattern of each image.

A biological sample according to the present disclosure includes a tissue sample, a group of cells such as mammalian cells, a group of live cells such as mammalian cells, a single cell such as a mammalian cell, a single live cell such as a mammalian cell or other biological samples known to those of skill in the art. The biological sample may be of human, animal, mammal, plant or bacterial origin. The biological sample may include any kind of cells. For example, the cells can include but are not limited to human cell lines such as HeLa, IMR-90, WI-38, HEK293T, and U20S. The biological samples can also be sections from tissues or organisms. See Table 1 for a non-exhaustive list of exemplary organisms as biological samples.

Table 1. A List of Exemplary Organisms as Biological Samples.

Viruses • Phage lambda

• Phi X 174

• SV40 • T4 phage

• Tobacco mosaic virus

• Herpes simplex virus

Prokaryotes • Escherichia coli (E. coli)

• Bacillus subtilis

• Caulobacter crescentus

• Mycoplasma genitalium

• Aliivibrio fischeri

• Synechocystis,

• Pseudomonas fluorescens,

Eukaryotes

Protists • Chlamydomonas reinhardtii

• Dictyostelium discoideum

• Tetrahymena thermophila

• Emiliania huxleyi

• Thalassiosira pseudonana

Fungi • Ashbya gossypii

• Aspergillus nidulans

• Coprinus cinereus

• Cryptococcus neoformans

• Neurospora crassa

• Saccharomyces cerevisiae

• Schizophyllum commune - model for mushroom formation. ^[5]

• Schizosaccharomyces pombe

• Ustilago maydis

Plants • Arabidopsis thaliana

• Boechera

• Selaginella moellendorffii

• Brachypodium distachyon

• Setaria viridis

• Lotus japonicus

• Lemna gibba

• Maize

• Medicago truncatula

• Mimulus guttatus

• Nicotiana benthamiana

• Tobacco BY-2 cells

• Rice

• Physcomitrella patens

• Marchantia polymorpha

• Populus

Animals

Invertebrates • Amphimedon queenslandica

• Arbacia punctulata

• Aplysia • Branchiostoma floridae

• Caenorhabditis elegans

• Caledia captiva (Orthoptera)

• Callosobruchus maculatus

• Chorthippus parallelus

• Ciona intestinalis

• Daphnia spp.

• Coelopidae

• Diopsidae

• Drosophila

• Euprymna scolopes

• Galleria mellonella

• Gryllus bimaculatus

• Hydra

• Loligo pealei

• Macrostomum lignano

• Mnemiopsis leidyi

• Nematostella vectensis

• Oikopleura dioica

• Oscarella carmela

• Parhyale hawaiensis

• Platynereis dumerilii

• Podisma spp.

• Pristionchus pacificus

• Scathophaga stercoraria

• Schmidtea mediterranea

• Stomatogastric ganglion

• Strongylocentrotus purpuratus

• Symsagittifera roscoffensis

• Tribolium castaneum

• Trichoplax adhaerens

• Tubifex tubifex

Vertebrates • Bombina bombina and Bombina variegata

• Carolina anole

• Cat (Felis sylvestris catus)

• Chicken {Gallus gallus domesticus)

• Cotton rat (Sigmodon hispidus)

• Dog (Canis lupus familiaris)

• Golden hamster (Mesocricetus auratus)

• Guinea pig (Cavia porcellus)

• Little brown bat (Myotis lucifugus)

• Medaka (Oryzias latipes, or Japanese ricefish)

• Mouse (Mus musculus)

• Naked mole-rat, (Heterocephalus glaber)

• Nothobranchius furzeri

• Pigeon (Columba livia domestica),

• Poecilia reticulata, the guppy • Rat (Rattus norvegicus)

• Rhesus macaque (or rhesus monkey) (Macaca mulatto)

• Sea lamprey

• Takifugu (Takifugu rubripes, a pufferfish)

• Three-spined stickleback

• Xenopus tropicalis and Xenopus laevis (African clawed

frog)

• Zebra finch (Taeniopygia guttata)

• Zebrafish (Danio rerio, a freshwater fish)

Human • Homo Sapiens

In an exemplary embodiment, live cells are first scored with lasers. The cells are allowed waiting time from as little as a few seconds to several hours to generate a response to the broken DNA, involving the localization of repair proteins to the sites of broken DNA before they are fixed. A skilled in the art can determine the length of waiting time based on the particular experiment. For example, the waiting time can be but are not limited to from about 1 second to about 24 hours, from about 10 seconds to about 12 hours, from about 30 seconds to about 6 hours, from about 1 minute to about 3 hours, from about 15 minutes to about 2 hours and from about 30 minutes to about 1 hour. In an alternative embodiment, the image registration technique as herein disclosed can avoid dependence on the localization of repair proteins by, for example, applying polymerases to extend the broken ends of DNA in fixed tissues with, for instance, labeled nucleotides or nucleotides that have been modified such that they can be immunolabeled. This strategy may even enable the laser treatment to be applied after fixation. The biological sample can be adhered to a substrate using a suitable medium using methods known to those of skill in the art. Live cells in their usual culture medium are put onto the slide, to which they adhere, and, after the cells are adhered, they are washed in fixation solutions. Suitable substrates include a glass or polystyrene slide or like materials known to a skilled in the art. A slide or plate with chambers added to it can also be used. Exemplary culture media include but are not limited to Eagle's Basal (EBM) and Minimal Essential (MEM) culture media, Dulbecco's modification of MEM (DMEM), Ham's media (F-10 and F-12), RPMI 1640 and RPMI 199, Leibovitz L-15, supplemented with serum or amino acids and vitamins.

The slides or plates are available from commercial vendors such as Mo Bi Tec or Ibidi. Very often slides are treated to enhance adherence of the cells, with treatments including a coating with 1% (vol/vol) poly-L-lysine, gelatin, collagen, fibronectin, laminin or any other substrate known to those of skill in the art suitable for use in imaging a biological sample. Suitable mediums used to fix a biological sample include formaldehyde or any other fixation agent known to those of skill in the art suitable for use in fixing a biological sample. For example, alcohol or acetone, or formaldehyde (such as 4% formaldehyde, although the percentage of formaldehyde can vary from protocol to protocol).

According to one aspect, the biological sample is provided with a detectable pattern in the form of a scoring or cutting of the biological sample. The scoring or cutting provides the biological sample with static or constant landmark or fiducial features which may be aligned between multiple images of the sample. The detectable pattern provides a constant or static registration feature on or in or through the biological sample that can be aligned among multiple images taken of the biological sample. The detectable pattern can be any pattern of lines or shapes that can be produced on or in or through the biological sample. The detectable pattern is detectable visually, such as by the human eye, or by an imaging apparatus. The detectable pattern may be a straight line, a curved line, a grid produced by intersecting lines, a combination of parallel lines, a grid of parallel lines, a combination of nonparallel lines, a grid of nonparallel lines, a shape, a symbol, or any other suitable pattern. The width of the line or plane produced by scoring may be a function of the laser or laser parameters or other scoring device or conditions or parameters used to create the line or plane. According to one aspect, the width of the line or plane created by the scoring device may be minimized by selection of suitable parameters or conditions used to create the line or plane, such as a suitable beam width of a laser light beam. According to one aspect, the width of the line or plane created by the scoring device may be minimized by adjusting the time between scoring and the fixing of the biological sample. For example, the time between scoring and fixing of the sample may allow the two portions of the sample created by the line or plane to separate. Fixing the sample shortly after creation of the line or plane will reduce any separation of the two portions that otherwise may occur. According to one aspect, the scoring or cutting of the sample using a laser does not result in the sample portions separating. Instead, the biological sample remains together joined by forces within the sample but with the line or plane or other suitable pattern being detectable. In an exemplary embodiment, live cells with broken DNA will recruit repair proteins to sites of broken DNA. If these repair proteins are labeled, such as fused with fluorescent proteins including but not limited to GFP, RFP or YFP, etc. the lines will be labeled and detectable. The lines can also be detected with tagged proteins by, for example, hemagglutinin, and then visualized by immunofluorescence.

The scoring or cutting of the biological sample is carried out using any suitable apparatus or scoring device. An exemplary apparatus is a laser having a suitable frequency, wavelength and energy to score or cut the detectable pattern on or in or through the biological sample. Such lasers are known to those of skill in the art and include ultra violet lasers described in Izhar et al., Cell Rep. 2015 Jun 9;11(9), pp. 1486-1500 hereby incorporated by reference in its entirety. The imaging apparatus may include a microscope operatively connected to a digital camera and suitable software. An exemplary imaging apparatus includes widefield microscopes, confocal microscopes, and super-resolution microscopes, or any other imaging apparatus known to those of skill in the art of imaging a biological sample. The detectable pattern of each of the multiple images is aligned using methods known to those of skill in the art which can include alignment by eye or by using suitable imaging software, including those using Imaris and ImageJ.

It is to be understood that the embodiments of the present invention which have been described are merely illustrative of some of the applications of the principles of the present invention. Numerous modifications may be made by those skilled in the art based upon the teachings presented herein without departing from the true spirit and scope of the invention. The contents of all references, patents and published patent applications cited throughout this application are hereby incorporated by reference in their entirety for all purposes.

The following examples are set forth as being representative of the present invention. These examples are not to be construed as limiting the scope of the invention as these and other equivalent embodiments will be apparent in view of the present disclosure, figures and accompanying claims.

EXAMPLE I

Sample Alignment Using a Detectable Pattern

A biological sample, such as a cell, is irradiated using a laser of suitable wavelength, frequency and laser energy, such as a PALM MicroBeam with fluorescence illumination (Zeiss) at about 40 to 46% laser energy and 40% speed and wavelength of 355 nM, to produce a detectable pattern of intersecting parallel lines through the cell. The cell may be sensitized to UV-A laser radiation by pre-incubation with about ΙΟμΜ BrdU for about 12 to 48 hours. In an exemplary embodiment, the cell has been genetically enabled to produce labeled and/or tagged proteins that are recruited to positions of DNA damage and then detected via their label and/or tag. The cell was then fixed with a suitable fixation medium, such as 3.7% formaldehyde, on a substrate, such as a glass slide. The cell may then be further processed as desired. The cell with the detectable pattern may then be imaged multiple times at the surface or at various depths of the sample using a suitable imaging device. An exemplary imaging device is an Axio Imager.M2 Zeiss microscope with a ORCA-R² Hamamatsu digital camera and Axiovision 4.5-4.8 software. The images are aligned using software by aligning the detectable pattern of each image thereby aligning the objects or features within the sample.

Claims

Claims:

1. A method of image registration or alignment of multiple images of a three- dimensional sample comprising

scoring a detectable pattern on the three-dimensional sample,

creating multiple images of the three-dimensional sample with the detectable pattern, and

aligning the detectable pattern of each image of the multiple images to align the multiple images.

2. The method of claim 1 wherein the images are digital images.

3. The method of claim 1 wherein the three-dimensional sample is a biological sample.

4. The method of claim 1 wherein the three-dimensional sample is a three- dimensional matrix including a biological sample.

5. The method of claim 1 wherein the pattern is scored on the three-dimensional sample using a laser.

6. The method of claim 1 wherein the multiple digital images are created using a digital imaging apparatus.

7. The method of claim 1 where the detectable pattern is a plurality of scored lines or planes.

8. The method of claim 1 wherein the detectable pattern is a plurality of parallel or nonparallel scored lines or parallel or nonparallel planes.

9. The method of claim 1 wherein the three-dimensional sample with the detectable pattern is fixed before imaging.

10. The method of claim 1 wherein the three-dimensional sample with the detectable pattern is fixed immediately after creating of the scored pattern.

11. The method of claim 1 wherein the three-dimensional sample with the detectable pattern is fixed after a certain amount of time has passed after creating of the scored pattern.

12. The method of claim 1 wherein the multiple digital images are processed using software to align the detectable patterns of the multiple digital images into a one coordinate system.