EP3504577A1 - Microscope à balayage utilisant un filtre de balayage en mosaïque - Google Patents

Microscope à balayage utilisant un filtre de balayage en mosaïque

Info

Publication number
EP3504577A1
EP3504577A1 EP17842477.6A EP17842477A EP3504577A1 EP 3504577 A1 EP3504577 A1 EP 3504577A1 EP 17842477 A EP17842477 A EP 17842477A EP 3504577 A1 EP3504577 A1 EP 3504577A1
Authority
EP
European Patent Office
Prior art keywords
detector array
filters
specimen
array
filter
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP17842477.6A
Other languages
German (de)
English (en)
Other versions
EP3504577A4 (fr
Inventor
Arthur Edward Dixon
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huron Technologies International Inc
Original Assignee
Huron Technologies International Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huron Technologies International Inc filed Critical Huron Technologies International Inc
Priority claimed from PCT/CA2017/000196 external-priority patent/WO2018035597A1/fr
Publication of EP3504577A1 publication Critical patent/EP3504577A1/fr
Publication of EP3504577A4 publication Critical patent/EP3504577A4/fr
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G02B21/365—Control or image processing arrangements for digital or video microscopes
    • G02B21/367—Control or image processing arrangements for digital or video microscopes providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/0004—Microscopes specially adapted for specific applications
    • G02B21/002—Scanning microscopes
    • G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
    • G02B21/0036—Scanning details, e.g. scanning stages
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G02B21/361—Optical details, e.g. image relay to the camera or image sensor
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B5/00—Optical elements other than lenses
    • G02B5/20—Filters
    • G02B5/201—Filters in the form of arrays
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules

Definitions

  • This invention relates to the fields of scanning microscope imaging of large specimens with particular emphasis on RGB brightfield imaging, as well as fluorescence and spectrally-resolved imaging.
  • Applications include imaging tissue specimens, genetic microarrays, protein arrays, tissue arrays, cells and cell populations, biochips, arrays of biomolecules, fluorescent nanoparticles, semiconductor materials and devices, and many others.
  • a mosaic scan filter array for use with a detector array comprises a plurality of identical tiles, each tile having N rows and M columns, where N is equal to or greater than 1 and M is equal to or greater than 2.
  • One row of tiles extending across a width of the detector array constitutes a base pattern covering N rows of pixels of the detector array in a direction perpendicular to a scan direction.
  • the base pattern is repeated M times and laterally offset in one direction by one pixel width for each repetition of the base pattern.
  • the base pattern and the laterally offset repetitions constitute a repeat pattern, which covers at least an active area of the detector array.
  • the active area of the detector array contains at least one repeat pattern of the mosaic scan filter array.
  • Each identical tile of the mosaic scan filter array has MxN filters selected from the group of one or more of the following: red filters, green filters, blue filters, white filters, any white filters being clear filters, fluorescence emission filters and a series of narrow spectral band filters covering a continuous spectral range.
  • a scanning microscope for obtaining an image of at least a portion of a large microscope specimen comprising:
  • an illumination system to illuminate a part of the specimen being scanned
  • the two dimensional detector array having a mosaic scan filter array with a plurality of identical tiles, each tile comprised of N rows and M columns, where N is equal to or greater than 1 and M is equal to or greater than 2.
  • N is equal to or greater than 1
  • M is equal to or greater than 2.
  • One row of tiles across a width of a detector array constitutes a base pattern covering N rows of pixels of the detector array in a direction perpendicular to a scan direction.
  • the base pattern is repeated M times and laterally offset in one direction by one pixel width for each repetition of the base pattern.
  • the base pattern and the laterally offset repetitions constitute a repeat pattern which covers at least an active area of the detector array.
  • the active area of the detector array contains at least the repeat pattern of the mosaic scan filter array;
  • a computer to control the detector array to capture sequential substantially overlapping frame images of the specimen each time that an optical image of the specimen is moved a distance relative to the detector array that is equal to the distance between adjacent rows of the detector array.
  • Image data from each frame image is translated into computer memory to match a motion of an optical image across the detector array and added to or averaged with any data previously stored in that pixel position to generate an image of a strip across the specimen.
  • the computer capturing of frame images continues until the specimen is moved a relative distance where all object points in that strip have been exposed a number of times equal to a chosen number of active rows in the detector array. There is one strip generated for each distinctive colour of the mosaic scan filter array; and
  • a scanning microscope for obtaining an image of a single-field-of-view of microscope optics comprising:
  • an illumination system to illuminate a part of the specimen being scanned
  • the two dimensional detector array having a mosaic scan filter array with a plurality of identical tiles, each tile comprised of N rows and M columns, where N is equal to or greater than 1 and M is equal to or greater than 2.
  • N is equal to or greater than 1
  • M is equal to or greater than 2.
  • One row of tiles extends across a width of a detector array and constitutes a base pattern covering N rows of pixels of the detector array in a direction perpendicular to a scan direction.
  • the base pattern is repeated M times and laterally offset in one direction by one pixel width for each repetition of the pattern.
  • the base pattern laterally offset repetitions constituting a repeat pattern which is repeated over an active area of the detector array, where the active area of the detector array is substantially the whole area of the detector array;
  • a computer controls the detector array to capture sequential substantially overlapping frame images of the specimen each time that an optical image of the specimen is moved a distance relative to the detector array that is equal to the distance between adjacent rows of the detector array.
  • the computer captures n frame images in sequence where n equals MxN. Image data from each frame image is translated into computer memory to match a motion of the optical image across the detector array and added to any data previously stored to generate a single-frame image of the specimen.
  • Each pixel of the single-frame image contains information from each distinctive colour of the mosaic scan filter array such that the final single-field-of-view image has full colour information in each pixel.
  • a scanning camera for obtaining an image of a single-field-of-view of an optical system comprising:
  • an illumination system to illuminate a part of the specimen being scanned
  • the two dimensional detector array having a mosaic scan filter array with a plurality of identical tiles, each tile comprised of N rows and M columns, where N is equal to or greater than 1 and M is equal to or greater than 2.
  • One row of tiles across a width of the detector array constitutes a base pattern covering N rows of pixels of the detector array in a direction perpendicular to a scan direction.
  • the base pattern is repeated M times and laterally offset in one direction by one pixel width for each repetition of the base pattern.
  • the base pattern and the laterally offset repetitions constituting a repeat pattern which is repeated over the active area of the detector array.
  • the active area of the detector array is substantially the whole area of the detector array.
  • a computer controls the detector array to capture sequential substantially overlapping frame images of the specimen each time that the detector array is moved a distance that is equal to the distance between adjacent rows of the detector array.
  • the computer captures n frame images in sequence where n equals MxN.
  • Image data from each frame image is translated in computer memory to match a motion of the optical image across the detector array and added to any data previously stored to generate a single- frame image of the specimen.
  • Each pixel of the final single-frame image contains information from each distinctive colour of the mosaic scan filter array such that the final single-field-of-view image has full colour information in each pixel.
  • a method of using a mosaic scan filter array comprises a plurality of identical tiles, each tile having N rows and M columns where N is equal to or greater than 1 and M is equal to or greater than 2.
  • One row of tiles extends across a width of a detector array constituting a base pattern covering N rows of pixels of the detector array in a direction perpendicular to a scan direction.
  • the base pattern repeated M times and laterally offset in one direction by one pixel width for each repetition of the base pattern, the base pattern and the laterally offset repetitions of the base pattern constitutes a repeat pattern which covers at least an active area of the detector array.
  • the active area of the detector contains at least one repeat pattern of the mosaic scan filter array.
  • Each identical tile of the mosaic scan filter array has MxN filters selected from the group of one or more of the following: red filters, green filters, blue filters, white filters, any white filters being clear filters, fluorescence emission filters and a series of narrow spectral band filters covering a continuous spectral range.
  • the method comprises using the mosaic scan filter with the detector array to image at least part of a specimen and producing one or more final images of the specimen where each pixel of the one or more final images has full colour information for each distinctive colour of the mosaic scan filter array.
  • an illumination system to illuminate a part of the specimen being scanned; b) at least one lens that focuses light from the specimen onto a two dimensional detector array, the specimen being mounted on a support that is movable relative to a two dimensional detector array;
  • a motion of the support relative to the detector array being controlled by a computer, the motion of the support relative to the detector array being in a direction perpendicular to rows of the two dimensional detector array;
  • the two dimensional detector array having a mosaic scan filter array with a plurality of identical tiles, each tile comprised of N rows and M columns, where N is equal to or greater than 1 and M is equal to or greater than 2, one row of tiles across a width of a detector array constituting a base pattern covering N rows of pixels of the detector array in a direction perpendicular to a scan direction, the base pattern repeated M times and laterally offset in one direction by one pixel width for each repetition of the base pattern, the base pattern and the laterally offset repetitions constituting a repeat pattern which covers at least an active area of the detector array, and where the active area of the detector contains at least one repeat pattern of the mosaic scan filter array;
  • a computer to control the detector array to capture sequential substantially overlapping frame images of the specimen each time that an optical image of the specimen is moved a distance relative to the detector array that is equal to the distance between adjacent rows of the detector array, image data from each frame image translated into computer memory to match a motion of an optical image across the detector array and added to or averaged with any data previously stored to generate an image of a strip across the specimen, the computer capturing of frame images continuing until the specimen is moved a relative distance where all object points in that strip have been exposed a number of times equal to a chosen number of active rows in the detector array, there being one strip generated from each distinctive colour of the mosaic scan filter array;
  • the method comprising activating the scanning microscope or scanning camera to obtain one or more final images of the specimen resulting from the portion of the specimen being scanned, each pixel of the one or more final images containing information from each of the distinctive colours of the mosaic scan filter array.
  • a "macroscopic specimen” (or “large microscope - specimen”) is defined as one that is larger than the field of view of a compound optical microscope containing a microscope objective that has the same Numerical Aperture (NA) as that of the scanner described in this document.
  • NA Numerical Aperture
  • TDI or Time Delay and Integration is defined as a method and detectors used for scanning moving objects, usually consisting of a CCD-based detector array in which charge is transferred from one row of pixels in the detector array to the next in synchronism with the motion of the real image of the moving object. As the object (and its image) moves, charge builds up and the result is charge integration just as if a longer exposure were used in a stationary imaging situation. When the image (and integrated charge) reaches the last row of the array, that line of pixels is read out.
  • One example of such a camera is the DALSA Piranha TDI camera.
  • CMOS TDI imagers have also been developed. CCD TDI imagers combine signal charges, while CMOS TDI imagers combine voltage signals.
  • image acquisition includes all of the steps necessary to acquire and produce the final image of the specimen, including some of but not limited to the following: the steps of preview scanning, instrument focus, predicting and setting gain for imaging each fluorophore, image adjustments including demosaicing (where required), scan linearity adjustment, field flattening (compensating for fluorescence intensity variation caused by excitation intensity and detection sensitivity changes across the field of view), correction of fluorescence signal in one channel caused by overlap of fluorescence from adjacent (in wavelength) channels when two or more fluorophores are excited simultaneously, dynamic range adjustment, butting or stitching together adjacent image strips (when necessary), storing, transmitting, assembling and viewing the final image.
  • image adjustments including demosaicing (where required), scan linearity adjustment, field flattening (compensating for fluorescence intensity variation caused by excitation intensity and detection sensitivity changes across the field of view), correction of fluorescence signal in one channel caused by overlap of fluorescence from adjacent (in wavelength) channels when two or more fluorophores are excited simultaneously, dynamic range adjustment, but
  • a "frame grabber” is any electronic device that captures individual, digital still frames from an analog video signal or a digital video stream or digital camera. It is often employed as a component of a computer vision system, in which video frames are captured in digital form and then displayed, stored or transmitted in raw or compressed digital form.
  • This definition includes direct camera connections via USB, Ethernet, IEEE 1394 (“FireWire”) and other interfaces that are now practical.
  • Moving Specimen Image Averaging is defined as the method and technology for acquiring digital strip images (image strips) across a large microscope specimen by capturing sequential overlapping frame images of a moving specimen where a new image frame is captured each time the specimen has moved a distance that causes the image of that specimen formed on a two-dimensional detector array to move a distance equal to the distance between rows of detectors in the detector array, image data from the new frame is translated (moved) in computer memory to match the motion of the optical image across the detector array, and is added to (or averaged with) the data previously stored to generate an image of a strip across the specimen, such procedure being continued until the specimen has moved a distance such that all object points in that strip have been exposed a number of times equal to the number of active rows in the detector array (usually chosen by defining a "detector area of interest" that has the width of the detector but a smaller number of rows than the detector array contains), or the number of rows of data chosen for processing from each frame image.
  • MSIA Moving Specimen Image Averaging
  • the image strip that results has increased signal-to-noise ratio because of pixel averaging, where the increased signal-to-noise ratio is equal to the square root of the number of times each pixel has been averaged to produce the final MSIA strip image, and increased dynamic range.
  • a frame image and image frame are identical to one another and are used interchangeably throughout this patent document.
  • Fluorescence includes fluorescence from naturally-occurring sources inside the specimen and fluorescent dyes and markers (including quantum dots) that may be added to the specimen, as well as fluorescence from the substrate or a layer above the specimen.
  • Spectral imaging is the method and technology for acquiring images in which each pixel is represented by its spectrum.
  • Hyperspectral imaging is the method and technology for acquiring images in which each pixel is represented by a spectrum composed of narrow spectral bands over a continuous spectral range.
  • Imaging spectroscopy is the acquisition and processing of hyperspectral images.
  • Multispectral imaging is the method and technology for acquiring multiple images of an object, each image representing a range of wavelengths.
  • each image could represent the emission range (or part of the emission range) of a particular fluorophore.
  • each pixel in the final multispectral image does not contain a spectrum of the fluorescence emitted by the specimen at that position, but contains information about the signal detected from each fluorophore at that pixel position.
  • a "mosaic scan filter array” is defined as a mosaic filter array that is designed for high resolution imaging using MSIA and Single-Field-of- View scanning where the resulting image has full colour information at each pixel position without demosaicing or interpolation.
  • MSIA scanning is used to produce an image of a large microscope specimen which is comprised of an RGB colour image and/or a greyscale (panchromatic) image and/or a multispectral fluorescence image using a mosaic scan filter array where no interpolation is required to produce the final RGBW image which contains full colour information (RGBW
  • MSIA scanning is used to produce an image of a large microscope specimen which is comprised of an RGB colour image and/or a greyscale (panchromatic) image and/or a hyperspectral image using a mosaic scan filter array where no interpolation is required to produce the final RGBW image which contains full colour information (RGBW) at each pixel
  • Figure 1 A shows a schematic view of a mosaic scan filter for RGB imaging
  • Figure 1 B shows a schematic view of a mosaic scan filter for RGBW imaging
  • Figure 2 is a schematic view of a 4000 column by 3000 row detector array covered by a mosaic scan colour filter array; - -
  • Figure 3A is a schematic view of data collection during MSIA scanning using a mosaic scan colour filter array for brightfield RGB imaging after a first exposure at the top and after a second exposure at the bottom;
  • Figure 3B is a schematic view of data collection during MSIA scanning using a mosaic scan colour filter array for brightfield RGB imaging after a third exposure at the top and a second exposure at the bottom;
  • Figure 3C is a schematic view of data collection during MSIA scanning using a mosaic scan colour filter array for brightfield RGB imaging after a fifth exposure at the top and after a second exposure at the bottom;
  • Figure 4 shows a schematic view of an MSIA scanner using a mosaic scan filter array
  • Figure 5 shows a schematic view of a scanner for MSIA and single-Field-of-View scanning
  • Figure 6A is a schematic view of data collection during single-Field-of-View scanning using a mosaic scan colour filter array for brightfield RGB imaging after a first exposure;
  • Figure 6B is a schematic view of data collection during single-Field-of-View scanning using a mosaic scan colour filter array for brightfield RGB imaging after a second exposure;
  • Figure 6C is a schematic view of data collection during single-Field-of-View scanning using a mosaic scan colour filter array for brightfield RGB imaging after a third exposure;
  • Figure 6D is a schematic view of data collection during single-Field-of-View scanning using a mosaic scan colour filter array for brightfield RGB imaging after a fourth exposure;
  • Figure 7 is a schematic view of a mosaic scan filter array comprised of 4x4 pixel tiles where each tile is comprised of 2x2 pixel sub-tiles;
  • Figure 8A is a schematic representation of a Mosaic Scan Filter array for fluorescence (multispectral) and brightfield imaging comprised of three-row by three-column tiles;
  • Figure 8B is a schematic representation of a Mosaic Scan Filter Array for fluorescence imaging of six fluorophores comprised of two-row by three-column tiles;
  • Figure 8C is a schematic representation of a Mosaic Scan Filter Array for fluorescence imaging of six fluorophores comprised of one-row by six-column tiles;
  • Figure 9A is a schematic representation of a mosaic scan filter array comprised of three- row by three-column tiles for hyperspectral and brightfield imaging.
  • Figure 9B is a schematic representation of a mosaic scan filter array for hyperspectral - - imaging comprised of three-row by three-column tiles;
  • Figure 9C is a schematic representation of a Mosaic Scan Filter Array for hyperspectral imaging comprised of one-row by six-column tiles;
  • Figure 10 is a schematic view of a combination MSIA and FOV scanner, where the FOV scanner uses a moving detector array that includes a mosaic scan colour filter array;
  • Figure 11 shows a schematic view of a digital scanning FOV camera using a mosaic scan colour filter array
  • Figure 12 shows a schematic view of a digital scanning FOV camera mounted on a microscope with a manual stage.
  • FIG 1A shows a schematic representation of a mosaic scan filter for brightfield imaging that is a first embodiment of this invention.
  • This example is a scan filter for MSIA imaging with a base pattern comprised of two row by two column tiles extend across the entire mosaic scan filter.
  • the tiles in each repetition of the base pattern required to form a Repeat pattern must be displaced laterally a distance equal to the distance between pixels in one direction so that each column of pixels detected during MSIA scanning contains all three colours (R,G ad B).
  • the R, G and B first repetition of the base pattern is identical to the base pattern except for the one pixel with displacement relative to the base pattern.
  • the Repeat Pattern has 10 columns and 4 rows in a direction perpendicular to a scan direction when covering an area detector array (not shown in Figure 1 A).
  • the Repeat Pattern can be repeated any number of times depending on the size of the area detector, there being at least one repeat pattern in the mosaic scan filter. Repetitions of the base pattern can be displaced by one pixel width either to the right or to the left relative to the base pattern and will result in the identical Repeat Pattern. However, if the first repetition of the base pattern is displaced to the right and any additional repetitions of the base pattern must also be displaced to the right. The displacement relative to the base pattern must always be in the same direction (ie. one direction) for each repetition.
  • Each column contains equal numbers of R and B filter components, and there are twice as many G filter components as there are R and B in each column. Also note that the R, G and B filters in each tile can be in any position, as long as all tiles in the mosaic scan filter are identical and the tiles are arranged with a one-pixel-width displacement from one row of tiles to the next. - -
  • FIG. 1 B An RGBW mosaic scan filter is shown in Figure 1 B, which is the second embodiment of this invention, where again the position of R, G, B and W elements in each tile is not important, as long as all tiles are identical and are arranged in a base pattern across the mosaic scan filter which has 10 columns and 4 rows.
  • With the second row of tiles displaced one pixel width to the right when compared to the first row of tiles and that pattern is repeated across the entire area of the detector array.
  • the base pattern and the Repeat Pattern extends across the entire area of the detector array.
  • This arrangement of tiles ensures that all colours in the mosaic scan filter array are present in each and every column of the filter array, which means that no interpolation of data between columns is required to produce a final scanned image containing full colour information at each pixel, which results in higher resolution than would be present in the final image if interpolation were required.
  • Mosaic scan filter arrays comprising tiles larger than 2x2 (3x3 or 4x4 for example) also work well as long as each subsequent row of tiles is displaced one pixel width to the right of the previous row, such that all colours are present in each and every column in the active area of the detector array.
  • a single row of tiles is called a "base pattern", where in this example the base pattern covers two rows of pixels in the detector array.
  • a "repeat pattern” is the pattern of rows of tiles that is repeated across the active area of the detector array (usually across the entire area of the detector array).
  • Figure 2 shows a schematic view of a detector array that is (in this example) 4000 pixels wide and 3000 rows long, where the entire area of the detector array is covered with a mosaic scan filter array with a pattern of rows that is repeated many times, and where the repeat pattern is small compared to the number of rows in the array.
  • a mosaic scan filter array with a pattern of rows that is repeated many times, and where the repeat pattern is small compared to the number of rows in the array.
  • the pattern of the mosaic scan filter array that has been fabricated on top of the pixels in the detector array is repeated every 4 rows of pixels.
  • every pixel position in the repeat pattern will be exposed 64 times during the scan, so the Signal/Noise ratio in the final MSIA image strip is increased by MSIA averaging by a factor of the square root of 64 (a factor of 8).
  • Figure 3 shows how data is collected and strip images are assembled during an MSIA scan using a mosaic scan colour filter example in which the repeat pattern is 4 rows, and the active area of the detector has been selected to be 4 rows by 4000 columns.
  • Fig. 3A shows a schematic view of the filter on the left (although shown as only 4 columns wide in this diagram, this is meant to represent a filter and detector array that is 4000 columns wide (in this example).
  • three image strips, one for each colour in the filter array also represent image strips that are 4000 pixels wide.
  • frame grabber 450 see Fig.
  • Fig. 3A transfers image data from all 4 rows of the detector array (this is the first frame image) to computer 406 which stores data in three image strips, one for each colour in the mosaic scan filter array, as shown on the right side of Fig. 3A.
  • computer 406 As the scanning stage moves a distance that moves the image of the specimen (projected by the microscope optics onto the detector array) a distance equal to the distance between rows of pixels in the detector array, a second exposure is made and this image data is passed to computer 406 by frame grabber 450, and is added to the data already stored in the image strips, as shown in Fig. 3A (bottom).
  • Fig. 3B shows the result after the third exposure (top) and fourth exposure (bottom).
  • FIG. 3C shows the result after the fifth exposure - note that the data in Image Strips Row 4 has not changed since that shown after the fourth exposure (Fig. 3B (bottom) and Row 5 is also now completely exposed.
  • Row 4 in the strip image has been completed after 4 exposures, and a new row in the image strips has been completed after the fifth exposure. After 4 exposures, a new row will be completed in the strip images after each subsequent exposure.
  • a more complicated example is more useful for MSIA.
  • the first row in the strip images that has been completely exposed is row number 64 (after exposure number 64) , and it contains data resulting from measuring each R and B pixel 16 times, and each G pixel in that strip image row 32 times.
  • the signal/noise ratio of the R and B pixels is increased by the square root of 16 (a factor of 4) and of the G pixels by the square root of 32 (a factor of 5.6).
  • a new row is completed in the strip images when each exposure is made after exposure number 64.
  • FIG 4 is a schematic representation of an MSIA scanner using a mosaic scan filter array that is a third embodiment of this invention.
  • Light from light source 410 illuminates from below an area of the surface of specimen 402, which is mounted on specimen holder 401 on moving microscope stage 405. This kind of illumination, where the light illuminating the specimen comes from below and is transmitted through the specimen, is commonly used for brightfield imaging.
  • the motion of Microscope stage 405 is controlled by computer 406 through wired or wireless connection 407.
  • Motion of the microscope stage is in a direction perpendicular to rows in the detector array ⁇ data is read out from rows in the detector array, usually the long dimension of the array (for example see Hamamatsu's ORCA-flash 4.0 camera, or PCO's pco.edge camera, both of which use Scientific CMOS (sCMOS) detector arrays) ⁇ .
  • Light from the specimen is collected by microscope objective 415 which is focused on the specimen by piezo positioner 420 (or other focusing mechanism) and is focused by tube lens 425 onto detector array 41 1 , which is covered by a mosaic scan colour filter array 412 like that shown in Fig. 1.
  • Detector array 41 1 and colour filter array 412 are enclosed in camera 430.
  • Data from the detector array 4 1 is read out by frame grabber 450 and passed to computer 406 where an image strip is assembled for each colour in the mosaic scan imaging filter.
  • Image data for each exposure is passed by the frame grabber to the computer where it is added to or averaged with data already present in the lengthening image strips in the Moving Specimen Image Averaging (MSIA) process.
  • MSIA Moving Specimen Image Averaging
  • the W (clear) filter allows all of the incident light through to the detector pixel underneath. If this causes blooming when doing RGB imaging, a neutral density filter can be used instead of the W (clear) filter in the mosaic scan filter array to match the intensity of signal from the W pixels to that from the R, G and B pixels.
  • Figure 5 shows a schematic representation of an MSIA scanner that can also acquire Single-Field-of-View images (an SFOV scanner) using a mosaic scan filter like those described in this document (designed for RBG or RBGW imaging, and sometimes for multi-spectral or hyperspectral imaging as well) which is a fourth embodiment of this invention.
  • light from light source 410 illuminates specimen 402 mounted on microscope slide or specimen holder 401 , which is mounted on computer-controlled single-axis scanning stage 510, which is mounted on computer-controlled dual-axis scanning stage 405.
  • Computer 560 controls scanning stage 405 through wired or wireless connection 407, and controls scanning stage 510 through wired or wireless connection 502.
  • Scanning stage 510 moves in a direction perpendicular to rows in the detector array, the same direction that scanning stage 405 moves in when it is scanning a strip across the specimen (as shown by the horizontal (left-right) arrows to the left of each scanning stage in the diagram).
  • light from light source 413 illuminates an area of the surface of microscope specimen 402, which is mounted on microscope slide or specimen holder 401 on scanning microscope stage 510.
  • This kind of illumination where the light illuminating the specimen comes from above the specimen, is called epi-illumination (epi-illumination can also be provided using other arrangements that are well known in microscopy).
  • Image data for each exposure is passed by the frame grabber to the computer where it is added to or averaged with data already present in the lengthening image strips in the Moving Specimen Image Averaging (MSIA) process, as described earlier in this document.
  • MSIA Moving Specimen Image Averaging
  • scanning stage 405 moves in a direction perpendicular to the scan direction (shown by the arrow pointing into and out of the diagram to the left of scanning stage 405) to a position centered on an adjacent strip to be imaged on the specimen, and a second strip is scanned. This procedure is continued until the entire specimen has been scanned (or the area of a region of interest on the specimen has been scanned).
  • scanning stage 405 is controlled by computer 560 to move the feature of interest to the centre of the field of view of microscope objective 415, and scanning stage 405 is held stationary at this position. At this position, an image of the portion of the specimen inside the field of view of the microscope optics is projected onto detector array 41 1 that is covered by scanning mosaic colour filter array 520.
  • a spectrally-resolved single-frame image of that portion of the specimen can be acquired as follows: With scanning stage 405 held in a stationary position, and using the entire area of the detector - - array as an active area, a first image of the specimen is acquired by opening and closing the shutter.
  • FIG. 6A This image contains rows of data that match the rows of the mosaic scan filter array, repeated across the whole field of view of the microscope.
  • FIG. 6A The left side of Figure 6A shows the repeat pattern of a scanning mosaic array detector which is covered by R, G and B transmission filters.
  • R, G and B transmission filters In this example (which describes RGB imaging), rows of 2x2 pixel tiles (covering two rows of pixels in the array) cover the entire width of the detector array, with each row of tiles followed by a subsequent row of tiles that is displaced a distance equal to the distance between pixels in the direction along the row.
  • This repeat pattern of 2 rows of tiles (4 rows of pixels) is repeated to cover the entire surface of the detector array.
  • the left strip shows only two pixels in each row, however this represents rows that are the entire width of the detector.
  • a detector array that is 4000 pixels wide (rows are 4000 pixels long) and 3000 pixels high (there are 3000 rows), and the entire detector array is covered with a scan filter array that has 3000 rows that are 4000 pixels long.
  • a scan filter array that has 3000 rows that are 4000 pixels long.
  • Each frame image is represented by a strip that is shown in Fig. 6 as two pixels wide, however each of the image frames is 4000 pixels wide, and 3000 pixels long, and there are three image frames (one for each colour in the mosaic scan colour filter array).
  • the first exposure is made of the specimen by opening and closing the shutter without moving stage 510.
  • the frame grabber transfers data from the entire image to computer 560 and data from each pixel in the detector is stored in the image frame row and column that corresponds with that pixel's colour filter.
  • the Scan Direction arrow (top left) shows the relative motion of the projected image across the detector array.
  • This diagram shows the detector array moving downward while the frame images are stationary, however it is also possible to represent the process by showing a stationary detector array with the six image frames moving upwards on the diagram.
  • the RG row at the top of the detector on the left side of the diagram to be the first row of pixels at the top of the detector, and the rows below it on the diagram are the first few rows of the 3000 rows on the detector.
  • Stage 510 moves specimen 402 a distance equal to the distance between pixel positions on the specimen (object pixel positions as opposed to image pixel positions) and stops, and a second exposure is made. Data acquired during the second exposure is passed by frame grabber 450 to computer 560 which then stores this data in the three image frames (see Fig. 6B). Stage 510 again moves specimen 402 a distance equal to the distance between pixel positions and stops. A third exposure is made and the data stored in the three image frames (see Fig. 6C). - -
  • the specimen is moved a third time, a fourth exposure is made and the data is stored in the image frames, as shown in Fig. 6D.
  • the first frame (the R frame in the diagram) contains a full field of view image of the specimen (minus 3 rows at the top and 3 rows on the bottom, or 2994 rows).
  • Each of the other images is the same, and all three images are perfectly registered with one another after the top and bottom three rows are discarded.
  • each Single-Field-of-View scan will require N-1 steps, with N exposures, and there will be one frame image for each colour in the scanning colour filter array, each of which will result in a Single-Field-of-View image filtered through the transmission filter for each colour in the scan colour filter array.
  • Scanning microscopes are often designed to have a resolution such that the "actual pixel" resolution shown on the computer screen is 0.25 microns or better. For example, that resolution can be achieved in a scanner using a 20X microscope objective with a numerical aperture of 0.75 or better (to achieve the 0.25 micron resolution on the specimen) and a working distance of 1 mm (so focus changes do not cause the objective to hit the specimen during scanning). With a 20X objective, this matches a detector array with 5 micron pixels, which are readily available.
  • each step motion of stage 510 is 0.25 microns, and for very high resolution imaging the accuracy and repeatability of motion should be better than .025 microns, which is a stringent requirement.
  • the range of motion can be quite small, because even for Repeat Patterns as large as 256 rows, the range of motion is only 64 microns, which is less than 0.1 mm.
  • One type of stage that meets these requirements is the piezo stage, which has a very small range of motion, but both the range of motion and the accuracy and repeatability meet these requirements.
  • Motor- driven stages generally have a much larger range of motion, but do not have the accuracy and repeatability necessary for this application.
  • stage 405 can be used for both MSIA and Single-Field-of-View scanning.
  • the separate image types in the example shown in Fig. 8A, for example
  • R,G,B,W and F1 ,F2,F3,F4,F5 (where F1 ,F2,F3,F4, and F5 represent emission filters for five fluorophores) can be acquired separately by making two scans from the same starting position of stage 510, - - one for R,G,B,W and one for F1 ,F2,F3,F4,F5 using white-light epi or transmission illumination for R,G,B imaging and a narrow band epi-illumination wavelength for fluorescence imaging (separate repeat scans can be made for each fluorophore using different excitation wavelengths if required).
  • stage 510 stops at each position, exposure time can be increased when the signal strength is low (for example for fluorescence imaging when compared to brightfield RGB imaging).
  • the same camera with an RGB and spectral imaging mosaic scan filter can be used to image RGB specimens in both MSIA scanning and Single-Field-of-View scanning (by discarding information from the detector pixels dedicated to spectral imaging) and can be used for imaging fluorescence or photoluminescence specimens in both MSIA scanning and Single- Field-of-View scanning by discarding information from RGB pixels in the detector.
  • each of the stacked 3D images contains perfectly registered RGB and spectrally-resolved images in each image plane in the stack.
  • High Dynamic Range SFOV images can be acquired by combining multiple SFOV images of the same field of view which have been acquired using different exposure times.
  • Figure 7 shows a mosaic RGB Scan Filter Array that uses 4x4 pixel tiles (each comprised of four 2x2 pixel sub-tiles), instead of the 2x2 pixel tiles used in Figure 1A, that is a fifth embodiment of this invention.
  • the mosaic scan filter described in Figure 7 has a base pattern of 4 rows and 10 columns and has a repeat pattern that is 8 rows of pixels long and 10 columns.
  • the Repeat Pattern is comprised of tiles that are made by combining four small 2x2 pixel tiles into a single large 4x4 pixel tile. This pattern is repeated across the entire surface of the detector array.
  • the large tiles in each subsequent row of large tiles must be displaced laterally a distance equal to the distance between pixels in the direction of rows of pixels so that each column of pixels detected during MSIA scanning will contain all of the colours contained in the filter array, which allows the final MSIA strip images to contain data in every pixel position in the strip image columns each repetition of the base pattern must be displaced laterally by one pixel width relative to the immediately adjacent pattern.
  • the mosaic array described in Fig. 7 requires eight exposures to acquire an entire RGB single field-of-view scanned image.
  • FIG. 8A shows a mosaic scan filter array for RGBW imaging that also includes transmission filters that act as emission filters for different fluorophores that is a sixth embodiment of this invention.
  • This mosaic filter is comprised of 3x3 pixel tiles extending across the mosaic scan filter array to form a base pattern.
  • the base pattern is highlighted at least once in a bold border in each of Figures 8A to 9C.
  • There are two repetitions of the base pattern and each repetition is displaced laterally by one pixel width in one direction relative to the immediately preceding pattern.
  • the first repetition following the base pattern is displaced laterally by one pixel width relative to the base pattern and the second repetition is displaced laterally by one pixel width relative to the first repetition.
  • the base pattern and the three repetitions form a repeat pattern having 9 rows and 9 columns.
  • Each column of pixels detected during MSIA scanning will contain - - all of the colours contained in the filter array.
  • RGBW RGBW is a clear filter that results in a bright, panchromatic image
  • fluorescence imaging using MSIA several scans are usually necessary to collect all of the data.
  • a first scan is often used to collect RGBW data, with transmission illumination provided by white light source 410. Data collected during this scan is stored in four strip images, one each for R, G, B and W.
  • the computer has access to all of the image data in each frame image collected during the MSIA scan process, and can record data from the RGBW channels while not recording data from the fluorescence channels during the first scan.
  • a second scan acquires data from fluorescence channels that are excited by the epi-illumination wavelengths chosen to illuminate the specimen for the second scan using epi-illumination light source 413 (or other epi-illumination arrangement (not shown)). See Figure 4.
  • the computer acquires data and sets up fluorescence strip images collected by detector pixels covered by transmission filters that match the emission wavelengths of one or more of the fluorophores excited by the chosen illumination wavelengths for this scan.
  • the illumination wavelength of the epi light source 413 is then changed to match the excitation wavelength of one or more other fluorophores in the specimen, another scan is performed and data is acquired to be added to an additional fluorescence strip images containing data for those fluorophores. This is repeated until strip images for each of the fluorophores in the specimen have been acquired, and a final strip image of the specimen is assembled that contains R, G, B and W information and fluorescence intensity for each of several fluorophores for each pixel position in that strip across the specimen. Stage 405 is then moved to the start position to for a second strip across the specimen, and the sequence of scans is repeated to collect R, B, G and W information and fluorescence intensity for a second strip image. Finally, all of the combined brightfield and Fluorescence strip images are assembled to produce an area image of the specimen that contains R, B, G and W as well as Fluorescence intensity information for every pixel position in the scanned area of the specimen.
  • this mosaic scan filter array covers the entire area of the detector array and the entire array is active.
  • Nine exposures are made to ensure that data will be acquired for each pixel position in the final FOV image, for each imaging modality (brightfield and fluorescence) and for each fluorophore.
  • the specimen is moved back to the start position and the stage is moved to acquire an additional 9 exposures using an excitation wavelength for one or more fluorophores using epi-illumination light source 413 (or other epi-illumination light source, not shown).
  • the Scans are repeated using different excitation wavelength and emission filter combinations until all of the fluorophores in the specimen have been imaged, and the data acquired in each scan is added to the single-field-of-view image.
  • the final single-field-of-view image contains full colour information for each pixel (R, G, B, W plus information for each fluorophore in the specimen).
  • Figure 8B shows a mosaic scan filter array for fluorescence imaging of six fluorophores having a base pattern comprised of two-row by three-column tiles extending across the mosaic scan filter array.
  • the tiles in each repetition of the base pattern required to form a repeat pattern must be displaced laterally a distance equal to the distance between pixels in one direction so that each column of pixels detected during MSIA scanning will contain all of the colours from all six fluorophores contained in the filter array.
  • the repeat pattern must be 6 rows long to ensure that each column in the MSIA image strips will contain different colour information at every pixel position, and all colours will be included.
  • the mosaic scan filter array shown in Figure 8B covers the entire area of the detector array and the entire array is active.
  • six exposures are required in the SFOV scan to ensure that full colour data will be acquired for each pixel position in the final SFOV image, for each fluorophore.
  • Up to 6 scans (of 6 exposures each) using excitation wavelengths for one or more fluorophores will be required using epi-illumination light source 413 (or other epi-illumination light source, not shown). Scans are repeated using different excitation wavelengths until all of the fluorophores in the specimen have been imaged, and the data acquired in each scan is added to the single-field-of- view image.
  • Each pixel position in the final single-field-of-view image contains full colour information for each fluorophore in the specimen.
  • Figure 8C shows a mosaic scan filter array for fluorescence imaging of six fluorophores comprised of one-row by six-column tiles extending across the filter array to form a base pattern.
  • each repetition of the base pattern required to form the repeat pattern is displaced laterally a distance equal to the distance between pixels in one direction relative to immediately proceeding patterns so that each column of pixels detected the tiles in each subsequent row of tiles after the first row of tiles must be displaced a distance equal to the distance between pixels along the direction of rows of pixels so that each column of pixels detected during MSIA scanning will contain all of the colours contained in the filter array.
  • the repeat pattern must have 6 rows to ensure that each column in the MSIA image strips will contain different colour - - information at every pixel position, and all colours will be included.
  • the repeat pattern has 12 columns.
  • the mosaic scan filter array shown in Figure 8C covers the entire area of the detector array (and the entire array is active), and requires 6 exposures to be made in the SFOV scan to ensure that full colour data will be acquired for each pixel position in the final SFOV image, for each fluorophore.
  • Up to 6 scans using excitation wavelengths for one or more fluorophores will be required using epi-illumination light source 413 (or other epi-illumination light source, not shown). Scans are repeated using different excitation wavelengths until all of the fluorophores in the specimen have been imaged, and the data acquired in each scan is added to the single-field-of-view image.
  • the final single-field-of-view image contains full colour information for each fluorophore in the specimen.
  • FIG. 9A shows a schematic representation of a mosaic scan filter for both Hyperspectral and RGBW imaging that is a seventh embodiment of this invention.
  • This example is a mosaic scan filter for MSIA imaging comprised of 3x3 pixel tiles that extend across the mosaic scan filter array to form a base pattern.
  • the base pattern is repeated twice to form a repeat pattern and each repetition of the base pattern is laterally offset in one direction by one pixel width from the immediately preceding pattern.
  • the first repetition is laterally displaced from the base pattern by one pixel width in one direction and the second repetition is laterally offset from the first repetition by one pixel width in the same direction.
  • the repeat pattern has 9 rows and 9 columns and each column of pixels detected in the repeat pattern during MSIA scanning contains all of the colours contained in the filter array.
  • bandpass filters that transmit a narrow spectral range (represented in the diagram by C1 ,C2, C3.C4 and C5).
  • the bandwidth of each filter is the same, and the filters cover a continuous spectral range.
  • the entire bandwidth (the continuous spectral range) of the hyperspectral filter and the number of different filters is chosen to match the application, usually covering a range of wavelengths in the visible, but sometimes including wavelengths in the near UV or the IR.
  • tiles can be designed that change the number of measured components in a spectrum.
  • a 4x3 pixel tile can be designed that contains RGBW filters as well as eight spectral filters for measuring a spectrum with eight spectral components. In this case (4x3 pixel tiles) the repeat pattern will be 12 rows.
  • RGBW is a clear filter that results in a bright, panchromatic image
  • MSIA hyperspectral imaging using MSIA
  • a first scan is used to collect RGBW data, with transmission illumination provided by white light source 410 or using reflected light from epi-illumination source 413. Data collected during this scan is stored in four strip images, one each for R, G, B and W.
  • the computer has access to all of the image data in each frame image collected during the MSIA scan process, and can record data from the RGBW channels while not recording data from the spectral channels during the first scan.
  • a second scan acquires data from spectral channels that are excited by the illumination wavelengths chosen to illuminate the specimen for the second scan.
  • the computer acquires data and sets up strip images collected by detector pixels covered by transmission filters that transmit the narrow spectral bands that together cover a continuous spectral range.
  • a strip image containing the spectrum composed of narrow spectral bands over a continuous spectral range has been acquired, a final strip image of the specimen is assembled that contains R, G, B and W information and spectral information for each pixel position in that strip across the specimen.
  • Stage 405 is then moved to the start position to for a second strip across the specimen, and the sequence of scans is repeated to collect R, B, G and W information and spectral intensity for a second strip image.
  • all of the combined brightfield and spectrally-resolved strip images are assembled to produce an image of the specimen that contains R, B, G and W as well as Spectral intensity information for every pixel position in the scanned area of the specimen.
  • the mosaic scan filter array When used for single-field-of-view scanning, the mosaic scan filter array covers the entire area of the detector array, and the entire array is active.
  • the mosaic scan filter array shown in Fig. 9A requires 9 exposures to be made in an SFOV scan to ensure that data from each filter colour present in a mosaic tile will be acquired for each pixel position in the final SFOV image.
  • the specimen After the first 9 exposures are completed using white light source 410 to acquire R, G, B and W data, the specimen is moved back to the start position and the stage is moved to acquire an additional 9 exposures using epi illumination or reflected-light illumination to acquire data describing the spectrum of light transmitted through or reflected from the specimen, using pixels covered by filters C1 through C5 (in the filter array described in Fig. 9).
  • the final single-field-of- view image contains full colour information for each pixel (R, G, B, W plus information on the spectrum of light reflected from or transmitted through the specimen).
  • Figure 9B shows a mosaic scan filter array for hyperspectral imaging comprised of three- row by three-column tiles that extend across the mosaic scan filter array to form a base pattern. - -
  • the repeat pattern has 9 rows and 9 columns with each column of pixels detected during MSIA scanning containing all of the colours contained in the filter array.
  • the mosaic scan filter array covers at least the active area of the detector array.
  • the mosaic scan filter array When used for single-field-of-view scanning, the mosaic scan filter array covers the entire area of the detector array, and the entire array is active.
  • the mosaic scan filter array shown in Fig. 9B requires 9 exposures to be made in an SFOV scan to ensure that data from each narrow spectral band filter in a mosaic tile will be acquired for each pixel position in the final SFOV image.
  • the final single-field-of-view image contains full colour information (C1 through C9) at each pixel position.
  • Figure 9C shows a mosaic scan filter array for hyperspectral imaging comprised of one- row by six-column tiles that extend across the mosaic scan filter array to form a base pattern.
  • the base pattern is repeated five times to form a repeat pattern, the repeat has 6 rows and 12 columns.
  • the mosaic scan filter array covers at least the active area of the detector array.
  • the mosaic scan filter array When used for single-field-of-view scanning, the mosaic scan filter array covers the entire area of the detector array, and the entire detector array is active.
  • the mosaic scan filter array shown in Fig. 9C requires 6 exposures to be made in an SFOV scan to ensure that data from each narrow spectral band filter present in a mosaic tile will be acquired for each pixel position in the final SFOV image.
  • the stage is moved to acquire 6 exposures to acquire data describing the spectrum of light transmitted through, reflected by or emitted from the specimen, using pixels covered by filters C1 through C6 (in the filter array described in Fig. 9C).
  • the final single-field- of-view image contains full colour information for each pixel (R, G, B, W plus information on the spectrum of light reflected from or transmitted through the specimen).
  • each tile contains nine transmission filters that transmit narrow spectral bands that together cover a continuous spectral range. In this case, the base pattern is still one row, but the repeat pattern is nine rows. More generally, if a spectral range requires M narrow spectral band filters, each single-row tile contains M filters and the repeat pattern is M rows.
  • mosaic scan filter arrays designed for hyperspectral imaging are photoluminescence imaging of semiconductor materials and devices.
  • the specimen is usually epi-illuminated with UV light, and a detector array covered with a hyperspectral mosaic scan filter array is used for MSIA or SFOV imaging to acquire spectrally- resolved photoluminescence images of the specimen.
  • mosaic scan filter arrays designed for hyperspectral imaging for imaging fluorescent specimens.
  • the specimen is epi-illuminated with short - - wavelength light (often UV) and a spectrally-resolved image of the fluorescence from the specimen is collected using MSIA or SFOV scanning. Since the emission spectra of common fluorophores are well known, the overlapping spectra can be deconvolved using data in the spectrally-resolved image.
  • FIG. 10 shows a schematic representation of an MSIA scanner for spectrally-resolved imaging that can also acquire Single-Field-of-View images using a mosaic spectral imaging scan filter like those described in this document (designed for multi-spectral or hyperspectral imaging and sometimes including R, G and B and/or W rows) that is an eighth embodiment of this invention.
  • Detector array 41 1 with spectral imaging scan filter 1015 is mounted on a scanning stage 1020 inside digital camera 1030.
  • stage 1020 is held in a fixed position while computer 1060 controls scanning of stage 405 in a direction shown by the horizontal left-right arrow to the left of stage 405. This scan direction is perpendicular to the rows in detector array 41 1 .
  • an active area is defined in detector array 41 1 and data is passed to computer 1060 to assemble MSIA strip images (one for each filter colour in the mosaic scan filter array) as described earlier.
  • scanning stage 405 is controlled by computer 1060 to move the feature of interest to the centre of the field of view of microscope objective 415, and scanning stage 405 is held stationary at this position. At this position, an image of the portion of the specimen inside the field of view of the microscope optics is projected onto detector array 41 1 that is covered by a mosaic scan colour filter array 015.
  • a spectrally-resolved image of that portion of the specimen can be acquired as follows: With scanning stage 405 held in a stationary position, and using the entire area of the detector array as an active area, an image of the specimen is acquired by opening and closing the shutter.
  • This image contains rows of data that match the rows of the mosaic scan filter array, repeated across the whole field of view of the microscope.
  • a first frame image is acquired and transferred to the nine image frames required for a 3x3 pixel filter.
  • the detector array 41 1 covered with mosaic spectral imaging scanning colour filter array 1015 is moved a distance equal to the distance between rows of detector pixels in the array to a new position where a second frame image is acquired and passed to computer 1060.
  • the difference between the instrument shown in Fig. 10 and that shown in Fig. 5 is that in the instrument shown in Fig.
  • each step motion of stage 1020 is 5 microns, and for very high resolution imaging the accuracy and repeatability of motion should be better than .5 microns, which is a not nearly as stringent a requirement as before.
  • the range of motion is larger than before, because for a large Repeat Patterns of 256 rows, the range of stage motion required is 1280 microns, or 1.28mm.
  • stage motion required is 1280 microns, or 1.28mm.
  • motorized stages include piezoelectric stages and stages with linear motors, stepping motors and others. Moving the stage to the next position, stopping and exposing an image while the detector array is stationary will provide the best resolution, but it is also possible to move the stage at constant speed, opening and closing the shutter and transferring image data to the computer in the time it takes for the stage to move a distance less than the distance between pixels (5 microns in this example), but this may result in some motion blur in the image.
  • Moving the detector array instead of moving the specimen when acquiring Single-Field-of- View scanned images has several advantages. First, the requirements for motion of the detector and repeatability are much less stringent than for moving the specimen, so several choices are available for moving stages and cost for the stage will be less. Second, because the available stages have better specifications than the minimum required, performance will likely be more robust over time.
  • An alternative method of moving the detector array with respect to the stationary image when acquiring single-field-of-view scanned images is to mount digital camera 430 onto a - - moving stage that is external to the camera, instead of placing the moving stage 1020 inside the camera 1030.
  • Figure 11 shows a schematic representation of an digital scanning Single-Field-of-View camera 1 101 for use on a microscope (or other optical instrument where single-field-of-view images are acquired) that acquires Single-Field-of-View images using a mosaic spectral imaging scan filter like those described in this document (designed for multi-spectral or hyperspectral imaging and sometimes including R,G and B and/or W rows) where the detector array 41 1 with spectral imaging scan filter 1010 is mounted on a scanning stage 1020 inside digital camera 1030 which is a ninth embodiment of this invention.
  • a microscope or other optical instrument where single-field-of-view images are acquired
  • a mosaic spectral imaging scan filter like those described in this document (designed for multi-spectral or hyperspectral imaging and sometimes including R,G and B and/or W rows)
  • the detector array 41 1 with spectral imaging scan filter 1010 is mounted on a scanning stage 1020 inside digital camera 1030 which is a ninth embodiment of this invention.
  • Computer 1 160 is programmed to control scanning stage 1020 through wired or wireless connection 1040, acquire frame images using frame grabber 1050, and as the scan proceeds, assemble frame images for each filter in the repeat pattern, and when the scan is complete to display these spectrally-resolved images on a computer monitor. It is expected that this scanning Single-Field-of-View camera, including the computer and frame grabber, can be assembled as a single package that can be mounted directly on the camera port of a microscope or other optical instrument.
  • Figure 12 shows a schematic representation of a microscope with digital Single-Field-of- View camera attached.
  • Microscope specimen 402 on microscope slide 401 is mounted on manual X-Y positioning microscope stage 1201 and is illuminated by transmission light source 410 or epi-illumination source 413, as required.
  • Light emitted by or reflected from specimen 402 is collected by microscope objective 415 which is focused on the specimen by manual focus mount 1220 and this light is focused on detector array 41 1 which includes a mosaic spectral- imaging scan filter 1015.
  • the specimen can be viewed through the microscope's eyepieces (not shown) for focusing and to find a feature of interest to display, or the digital scanning Single-field- of-view camera can be programmed to continuously display images of the specimen on a computer monitor while focusing and finding an area of interest on the specimen.
  • computer 1 160 can be programmed to display a spectrally-resolved image of the specimen, as well as registered RGB and Greyscale images of the same area (depending on which mosaic spectral-imaging scan filter is present on detector array 41 1 ).
  • Scanning Single-Field-of-View camera 1101 enables this basic microscope to be used for RGB and/or Greyscale and multi-fluorophore imaging, or RGB and/or Greyscale and hyperspectral imaging, depending on which mosaic spectral-imaging scan filter is present on detector array 41 1 , and each of these combinations can be achieved using only one camera, and all images are perfectly registered with each other.

Abstract

L'invention concerne un réseau de filtres de balayage en mosaïque comprenant une pluralité de tuiles identiques, chaque tuile ayant n rangées et m colonnes, n étant égal ou supérieur à 1 et m étant égal ou supérieur à 2. Une rangée de tuiles s'étend sur une largeur d'un réseau de détecteurs et constitue un motif de base. Le motif de base est répété M fois et chaque répétition est latéralement décalée, dans une direction, d'une largeur de pixel. Le motif de base et les répétitions décalées latéralement constituent un motif de répétition. Il existe MxN filtres sélectionnés dans le groupe comprenant un ou plusieurs des éléments suivants : des filtres RGBW, des filtres d'émission de fluorescence et une série de filtres à bande spectrale étroite couvrant une plage spectrale continue. Les réseaux de filtres de balayage en mosaïque peuvent être utilisés avec un microscope à balayage ou une caméra à balayage.
EP17842477.6A 2016-08-26 2017-08-25 Microscope à balayage utilisant un filtre de balayage en mosaïque Pending EP3504577A4 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201662380224P 2016-08-26 2016-08-26
PCT/CA2017/000196 WO2018035597A1 (fr) 2016-08-26 2017-08-25 Microscope à balayage utilisant un filtre de balayage en mosaïque

Publications (2)

Publication Number Publication Date
EP3504577A1 true EP3504577A1 (fr) 2019-07-03
EP3504577A4 EP3504577A4 (fr) 2020-04-01

Family

ID=66632197

Family Applications (1)

Application Number Title Priority Date Filing Date
EP17842477.6A Pending EP3504577A4 (fr) 2016-08-26 2017-08-25 Microscope à balayage utilisant un filtre de balayage en mosaïque

Country Status (1)

Country Link
EP (1) EP3504577A4 (fr)

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4217493B2 (ja) * 2003-01-27 2009-02-04 オリンパス株式会社 画素シフトデジタルカメラの調整方法、画素シフトデジタルカメラシステム及び顕微鏡画素シフトデジタルカメラシステム
US9575304B2 (en) * 2012-06-25 2017-02-21 Huron Technologies International Inc. Pathology slide scanners for fluorescence and brightfield imaging and method of operation
US9398196B2 (en) * 2014-04-28 2016-07-19 Lynx System Developers, Inc. Methods for processing event timing images

Also Published As

Publication number Publication date
EP3504577A4 (fr) 2020-04-01

Similar Documents

Publication Publication Date Title
CA2877902C (fr) Dispositifs de balayage de lames d'anatomopathologie pour imagerie par fluorescence et en fond clair et procede de fonctionnement
US11269171B2 (en) Spectrally-resolved scanning microscope
US8896918B2 (en) Pathology slide scanner
US9632301B2 (en) Slide scanner with a tilted image
US20140125776A1 (en) 3d pathology slide scanner
US11106026B2 (en) Scanning microscope for 3D imaging using MSIA
WO2019230878A1 (fr) Dispositif d'observation de fluorescence et procédé d'observation de fluorescence
US10114206B2 (en) Microscopy slide scanner with variable magnification
US20180373017A1 (en) Msia scanning instrument with increased dynamic range
US11143855B2 (en) Scanning microscope using pulsed illumination and MSIA
CA3035109C (fr) Microscope a balayage utilisant un filtre de balayage en mosaique
US20190227291A1 (en) Fluorescence microscope
US10877255B2 (en) High resolution pathology scanner with improved signal to noise ratio
EP3504577A1 (fr) Microscope à balayage utilisant un filtre de balayage en mosaïque

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20190326

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
A4 Supplementary search report drawn up and despatched

Effective date: 20200228

RIC1 Information provided on ipc code assigned before grant

Ipc: G02B 5/20 20060101ALI20200224BHEP

Ipc: G02B 21/26 20060101ALI20200224BHEP

Ipc: G02B 26/10 20060101ALI20200224BHEP

Ipc: H01L 31/0216 20140101ALI20200224BHEP

Ipc: H01L 27/146 20060101ALI20200224BHEP

Ipc: G02B 21/36 20060101AFI20200224BHEP

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20210423

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

RIC1 Information provided on ipc code assigned before grant

Ipc: H04N 23/60 20230101ALI20230817BHEP

Ipc: H04N 23/10 20230101ALI20230817BHEP

Ipc: G02B 21/00 20060101ALI20230817BHEP

Ipc: H01L 27/146 20060101ALI20230817BHEP

Ipc: H01L 31/0216 20140101ALI20230817BHEP

Ipc: G02B 5/20 20060101ALI20230817BHEP

Ipc: G02B 26/10 20060101ALI20230817BHEP

Ipc: G02B 21/26 20060101ALI20230817BHEP

Ipc: G02B 21/36 20060101AFI20230817BHEP

INTG Intention to grant announced

Effective date: 20230919