EP1368782A1 - Calibrage coordonne pour systemes de balayage - Google Patents

Calibrage coordonne pour systemes de balayage

Info

Publication number
EP1368782A1
EP1368782A1 EP02713408A EP02713408A EP1368782A1 EP 1368782 A1 EP1368782 A1 EP 1368782A1 EP 02713408 A EP02713408 A EP 02713408A EP 02713408 A EP02713408 A EP 02713408A EP 1368782 A1 EP1368782 A1 EP 1368782A1
Authority
EP
European Patent Office
Prior art keywords
data
calibration data
image
specimens
panel
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP02713408A
Other languages
German (de)
English (en)
Other versions
EP1368782A4 (fr
Inventor
Carl S. Brown
Raymond H. Kraft
John Timothy Strom
Mark D. Cavelero
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Global Life Sciences Solutions USA LLC
Original Assignee
Applied Precision Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Applied Precision Inc filed Critical Applied Precision Inc
Priority claimed from PCT/US2002/001082 external-priority patent/WO2002057999A1/fr
Publication of EP1368782A1 publication Critical patent/EP1368782A1/fr
Publication of EP1368782A4 publication Critical patent/EP1368782A4/fr
Withdrawn legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras

Definitions

  • This invention relates generally to detection of high content field information such as arrays of small biological specimens, and more specifically concerns calibration of the camera and stage to optimize performance.
  • Sequential processing techniques have resulted in important discoveries in a variety of biologically related fields, including, among others, genetics, biochemistry, immunology and enzymology. Historically, sequential processing involved the study of one or two biologically relevant molecules at the same time. These original sequential processing methods, however, were quite slow and tedious. Study of the required number of samples (up to tens of thousands) was time consuming and costly.
  • a plurality of samples are arranged in arrays, referred to herein as microarrays, of rows and columns into a field, on a substrate slide or similar member.
  • the specimens on the slide are then biochemically processed in parallel.
  • the specimen molecules are fluorescently marked as a result of interaction between the specimen molecule and other biological material.
  • the present invention calibrates a scanning system to correct for panel misalignments errors.
  • a reference slide or data point is used to obtain a series of measurements with the scanning system. These measurements are compared with the expected results to determine alignment errors in the scanning system.
  • a model is created to correct the alignment errors during the scanning process, thus providing a plurality of more accurate scans. The plurality of scans may then be assembled to create a perfectly registered, complete image of the scan area.
  • Figure 1 is a diagram showing the arrangement of the scanning system of the present invention.
  • Figure 2a is a diagram showing calibration data obtained using the stepping data technique according to the present invention.
  • Figure 2b is a diagram showing calibration data obtained using the slide data technique according to the present invention.
  • Figure 2c is a diagram showing calibration data obtained using the sub-spot data technique according to the present invention.
  • Figure 2d is a diagram showing calibration data obtained using the absolute data technique according to the present invention.
  • Figure 3 illustrates the process for calibrating the image system according to the present invention.
  • Figure 4 is a diagram showing the arrangement of data using the system of the present invention.
  • FIG 5 is a more complete arrangement of the data produced by the system of the present invention for a icroarray.
  • Figure 6 is a flow chart showing a portion of the software for the present invention. *
  • Figure 7 is a flow chart showing another portion of the software for the present invention.
  • the present invention uses a high content material, such as a microarray extending over a relatively large area (up to 2-1/2 inches square) which is accurately scanned with high resolution as shown in Figure 1.
  • An objective lens 30, with high resolution and high light collection efficiency characteristics, is used to detect the data in successive small portions (panels) of the microarray field 32 present on substrate 34.
  • An example of such a lens is a Nikon 4X objective with a 0.2 NA. Illumination for each panel, typically 1/10 inch
  • a conventional white light (broad spectrum) source 36 The light (illumination) is directed obliquely to the array as shown in Figure 3. This eliminates direct reflection of the illumination off the slide, although it is not necessary to the invention.
  • the light from source 36 is applied to a filter 37 and then past a photosensor 44 before reaching the microarray 32. Photosensor 44 is used to measure the total amount of illumination delivered to the small target area (panel) of the microarray during each exposure of the camera. The photosensor measurement is used during a later processing step to correct small variations in light intensity from panel to panel, which typically amount to approximately 5%.
  • Excitation filter 37 is one of a plurality of filters held in a filter wheel by which a number of different excitation wavelengths can be chosen under software control.
  • the filter wheel may be easily changed; each wheel holds four separate filters.
  • the illumination is provided through a fiber optic cable, which results in a highly consistent pattern of illumination.
  • Illumination of the array results in fluorescence from the biological specimens in area 33 on slide 34 which is then collected by objective lens 30.
  • Panel 42 encompasses an area in which a total of nine biological specimens are located. The fluorescence data from these nine individual specimens is directed through lens 30, then through an emission filter 35, and then to the CCD camera 38, which detects an image of the array.
  • Emission filter 35 is one of a plurality of filters held in a filter wheel. As with the illumination filter, emission filter 35 may be selected through software control. In the embodiment shown, the emission filter wheel is easily changeable and may hold up to four emission filter sets.
  • the system response i.e. the sensitivity and offset
  • area 33 may not be absolutely uniform.
  • Each pixel in the image detected by the camera is compensated with gain and offset to produce a uniform response across the image.
  • the response of each pixel is determined by an exposure series. Linear regression analysis of the exposure series data results in gain-offset values for each pixel. This is a common digital microscopy technique and results in all the pixels having the same light intensity, so that all areas of all panels have the same intensity.
  • Images from the CCD camera and illumination information from the photosensor are applied to a processor 47, which will arrange all of the resulting pictures together, as discussed in more detail below.
  • the light travels from its source 36, through filter 37 and photosensor 44 to the specimens. Fluorescent emissions are collected by the objective lens 30 and passed through filter 35, on their way to the CCD camera 38.
  • Such an optical system is generally conventional and therefore not discussed in detail.
  • the general configuration of such systems, with the exception of oblique illumination, is present in fluorescence microscopes, such as available from Olympus and Nikon, or the assignee of the present invention.
  • the substrate with the microarray 32 is then moved successively by a precise moving system or stage 48.
  • the initial position of the scanner system relative to the microarray is in one corner of the array referred to by x,y coordinates o,o. It should be understood, however, that the image system could alternatively be moved by a stage, with the array remaining stationary.
  • Stage 48 is moved successively in the embodiment shown, such that eventually all of the information in the array is obtained, in the form of successive panels, each of which has an identifying set of stage coordinates.
  • the panels are then put together to form a single, unitary image of the complete array by processor 47.
  • the images can be joined together to form the image of the entire array with minimal or no mathematical processing to achieve alignment. If a geometric alignment of the staging process is performed, it is not necessary to in any way further align the data between adjacent panels or to use computation techniques to string or connect the images together based on particular features of adjacent panels.
  • the complete array thus can be constructed purely on the recorded position of the stage at each collection point, providing coordinate points for each panel are known.
  • the x,y axes of the stage are not exactly parallel with the pixel rows and columns in the camera. If the rotation angle between the stage and the camera is known, the camera can be rotated appropriately relative to the stage. The rotation angle can be determined, for instance, by adjusting the rotation angle until adjacent panels are aligned perfectly. The rotation angle, alternatively, can be used in the processing of the images, as explained below.
  • the staging area In addition to the camera and stage rotation, there are other factors that may cause the staging to be misaligned.
  • the exact magnification of the image system may be unknown.
  • a mechanical device moves the staging area, it is possible that the movements and mechanisms are not completely accurate.
  • the stage perpendicularity may be off slightly. This may result when the angle between the x and y axis is not exactly 90 degrees.
  • a panel may be designed to be nominally 1 mm square. However, during the manufacturing process of the array, the panel may turn out to be 1mm by 0.9mm. Without correction, the staging area will leave a gap of 0.1mm between each panel.
  • a positioning error may be introduced as the staging area is moved in either the x or y direction.
  • the staging area is moved with a lead screw system. The lead screw revolves to move the staging area.
  • the lead screw is highly accurate, the revolution of the screw may introduce a very slight sinusoidal pattern into the movement.
  • a slight x direction sinusoidal error may be introduced in both the x and y directions.
  • a slight y direction sinusoidal error may be introduced in the x and y directions.
  • These errors are referred to as a "ripple" error.
  • the combination of each of the possibilities of panel misalignment creates a situation where the actual panel locations are moved away from the expected panel locations.
  • the present invention calibrates the errors prior to imaging and uses the results of this calibration to adjust the panel locations prior to acquiring the panel images and combining or "stitching" the panels together.
  • FIG. 2a is a diagram showing calibration data obtained using the "stepping data" technique according to the present invention.
  • stepping data a small, bright spot 200 is positioned in a corner of the camera image, and the XY location is recorded based on the intensity center.
  • the bright spot may be a data point from a gauge slide 225, or it may be any bright spot the camera can find, such as a speck of dust.
  • the spot is then scanned along a series of points along the X axis 205, 210 with the field of view of the camera.
  • the recorded locations contain information required to solve for the calibration algorithm.
  • FIG. 2b is a diagram showing calibration data obtained using the "slide data" technique according to the present invention.
  • a gauge slide 225 is positioned in the staging area having a series of bright spots 230-235 at predetermined locations.
  • one of the bright spots 230 is positioned at a predetermined location in the camera image. The location is then calculated from the stage motor coordinates.
  • FIG. 2c is a diagram showing calibration data obtained using the "sub-spot" data technique according to the present invention.
  • a gauge slide 225 is positioned in the staging area having a series of bright spots 240-248 at predetermined locations.
  • Sub-spot data can be collected when more than one gauge slide spot is visible within a single camera image. Such data are affected by the gauge slide, optical, and camera pixel scale, as well as the rotation angle between the camera and the gauge slide. For every slide data point 240-248, the sub-spot data may be collected for solution of the sub-spot scaling factor. The data may be measured relative to a base sub-spot, such as sub- spot 240.
  • FIG. 2d is a diagram showing calibration data obtained using the "absolute data" technique according to the present invention.
  • Absolute data may be generated by combining the stepping data and the slide data, and possibly even the sub-spot data.
  • a gauge slide 225 is positioned in the staging area.
  • the stepping data is used because the gauge slide scaling and rotation are not involved.
  • the dependencies that do exist for stepping data are appropriate for the stage mapping project that is designed to collect panels at well controlled locations.
  • the process divides by the gauge slide scale, rotates by the negative gauge slide angles, and multiples by the optical scale.
  • the absolute data may be used to solve for the ripple parameters.
  • Figure 3 illustrates a process 300 for adjusting the imaging system based on the alignment issues.
  • the process begins in a START block 305. Proceeding to block 310, the calibration data is obtained from the imaging system. As described above, there are many different techniques for obtaining calibration data, and any or all of these techniques may be used.
  • the panel alignment is determined from the calibration data.
  • the panel alignment may be calculated using the calibration data.
  • the following mathematical solution may be used.
  • a sample solution is presented, it can be understood that a solution may be obtained from the calibration data using multiple techniques, and the present invention is not intended to be limited by the solution presented.
  • Using a virtual coordinate inversion technique where the virtual coordinates are converted to the motor coordinates, or vise- versa, may solve the cross-coupled ripple error.
  • the calibration can be calculated using the following equation and solution.
  • Y c Sy * (Xs*Sin[ ⁇ ] - Ys*Cos [ ⁇ ] ) , where Xr and Yr are the ripple coordinates along the respective axis;
  • Ax and Ay are the amplitude
  • Fx and Fy are the frequency; and Xs and Ys are intermediate variable used to simplify the equations.
  • the equations are then solved for ⁇ Xm, Ym ⁇ as a function of ⁇ Xv, Yv ⁇ .
  • the equations may be solved using an iterative solution or other estimate technique. Proceeding to block 320, a solution model is created based on the calibration data.
  • the solution model may be an algorithm which coverts the desired positioning information into the actual positioning information by using the calibration data.
  • One of skill in the art is able to create a solution model after obtaining the calibration data, and thus will not be described herein.
  • the movement of the staging area is adjusted based on the solution model.
  • the staging area is moved under software control, where the software contains the solution model and automatically adjusts the desired ("virtual") positioning information into actual (“motor”) positioning information. This process may be transparent to a user.
  • panel data is collected for each panel in the panel array. Because the calibration data is used to adjust the staging position, the coordinate positions of each panel will be more precise. Details of the panel data collection are described below.
  • a complete image is created by "stitching" together each panel of the panel array.
  • the stitching is described below.
  • the process 300 then terminates in and END block 340.
  • the "stitching" together of the panels is illustrated in Figure 4, a nine panel array comprising 3 columns and 3 rows.
  • Panels 51, 53 and 55 comprise an upper row 57; panels 59, 61 and 63 comprise a middle row 65; and panels 67, 69 and 71 comprise a lower row 73.
  • Each panel has specific x,y coordinates indicating its position.
  • the individual panels, imaged by the CCD camera, are arranged together by processor 47 to form a complete image 75 of the array field 32.
  • FIG. 6 which shows the steps in acquiring the data
  • the pixel size of the information which, after calibration, is known and previously stored (block 80) , approximately 5 microns in the embodiment shown, is used to calculate the size of the panels (block 82) . In the embodiment shown, this would be approximately 2-1/2 X 2-1/2 mm (1/10 inch) , although it should be understood that other panel sizes could be used.
  • the accurate determination of pixel size is important to accomplish the arrangement of the various images into a single picture.
  • the number of rows and columns of the camera images and the size of the pixel determine the exact area of a panel. Where a single panel image comprises 500 X 500 pixels, the pixel size must be accurate to within 0.1% in order to limit placement errors of panels to less than 1/2 pixel.
  • the pixel size can be stored for use by the processor.
  • the user provides the coordinates (block 84) for the array on the slide or other substrate.
  • the coordinates in effect identify the actual physical boundaries and thus the size of the array.
  • the total number of panels which will comprise the scanned array is then determined, as shown at block 86.
  • the particular manner in which the slide is maneuvered by the stage assembly to obtain (scan) the entire array is determined, as shown at block 87. For instance, successive images can be obtained in the direction of successive rows, either in one direction, or back and forth, or by successive columns, or some combination thereof.
  • the calculation of the panel size and total number of panels involves conversion between the virtual and motor coordinates following calibration. For a particular scan area on a given slide, the location and size of each portion of the area covered by a single image must be determined, as well as the number of portions to cover the entire area. The size of the scan area, the pixel size of the detector, the magnification in the image, and the dimensions of the detector array determine this .
  • the x,y coordinates (virtual and motor) for each successive panel are then determined, as shown at block 88.
  • the stage is then moved to the x,y motor coordinates of the first panel as shown at block 92, and the image at that position is acquired (block 94), as discussed above.
  • the stage is arranged so that it only moves in x and y directions. It does not move in the z (height) dimension, so as to preserve correct focus over the array.
  • each panel image comprising nine individual biological specimens in the embodiment shown has very high resolution.
  • This first panel image (coordinates xi yi in Figure 5) is then saved as well as the virtual coordinates, as shown at block 96.
  • the filter wheels 35a and 37a are changed to the appropriate excitation/emission filter pair and a new image is acquired and stored having the same coordinates as the first panel. This process may be repeated for any wavelengths that are selected.
  • the stage 48 does not move when the filter pairs are changed so as to minimize chromatic shift in the final, complete image of the microarray.
  • the net effect of this scanning technique is that each panel position may have data with multiple wavelengths, with substantially zero lateral shift between filter (wavelength) pairs.
  • the software determines whether the panel just obtained is the last panel in the array, shown at block 98. If not, the stage is moved to the next panel location, as established in the acquisition strategy table. The image is acquired for that panel and that information and its virtual coordinates saved, shown at block 96. This repetitive process continues until all of the panels in the array have been imaged and saved, i.e. until panel x n y n in array 95 of Figure 5, for instance, has been obtained and saved. At this point, the file is closed, as shown at block 100, the acquisition process having been completed.
  • Figure 7 shows the processing of the acquired data to produce the whole "stitched together" image of the complete array.
  • the file created by the software portion in Figure 6 is opened, shown at block 102.
  • the light intensities of the panels are normalized, as shown at block 104, to provide uniform values of intensity for each panel relative to each other. This is accomplished with information from the photosensor. Also, conventional techniques of correcting uniformity of illumination, pixel by pixel with gain/offset, known as “flat-fielding", are carried out, as well as making the background intensity patterns of the panels the same, which is known as “panel flattening”. These techniques are disclosed in co-owned U.S. Patent Application No.: 09/771,343, filed January 26, 2001, which is incorporated by reference herein in its entirety.
  • the images are normalized over each separate image portion, such as a panel, and also normalized over the entire area being scanned, comprising all of the images.
  • These techniques eliminate any resulting "patched" look for the final, complete image.
  • the virtual x,y coordinates of each panel are then obtained from the file, as shown at block 106.
  • the panels are then assembled according to their specific coordinates, until the complete array image is produced, as shown at block 108. This is repeated for all filter/wavelength pairs collected for that sample.
  • the assembled plurality of panels is then displayed, as shown at block 110.
  • the complete image, with all of the wavelength information is also saved, as shown at block 112.
  • the individual separate panels each comprising a small portion of the array, are simply put together on the basis of their coordinate values and are not mathematically aligned or otherwise altered to fit together. This is because of the precise, calibrated- movement capability (with no movement in height) of the stage and the software which makes minor adjustments to illumination intensity and background over each image and over all the images and then assembles the individual panels of data into a complete image.
  • the present invention is significant in the scanning of biological arrays in that it is quite different from laser scanning methods, which are presently preferred.
  • a full spectrum illumination source is used, along with a conventional scientific grade, cooled CCD camera, with its superior linearity and efficiency.
  • a succession of individual panel images of the complete array of the various wavelengths are produced, with the panels then being pieced together based on the panel x,y coordinates into a complete image of the array.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

Selon l'invention, un système de balayage est calibré afin de corriger d'éventuelles erreurs d'alignement de panneaux. Un point de données ou curseur de référence permet d'obtenir une série de mesures au moyen du système de balayage. Ces mesures sont comparées aux résultats attendus afin de déterminer les erreurs d'alignement systématiques dans le système de balayage. Un modèle est créé afin de corriger les erreurs d'alignement pendant le processus de balayage, produisant ainsi une pluralité de balayages plus précis. Ces balayages peuvent ensuite être assemblés afin d'obtenir une image complète de la zone de balayage.
EP02713408A 2001-01-16 2002-01-15 Calibrage coordonne pour systemes de balayage Withdrawn EP1368782A4 (fr)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US26200001P 2001-01-16 2001-01-16
US262000P 2001-01-16
PCT/US2002/001082 WO2002057999A1 (fr) 2001-01-16 2002-01-15 Calibrage coordonne pour systemes de balayage

Publications (2)

Publication Number Publication Date
EP1368782A1 true EP1368782A1 (fr) 2003-12-10
EP1368782A4 EP1368782A4 (fr) 2008-08-27

Family

ID=29549731

Family Applications (1)

Application Number Title Priority Date Filing Date
EP02713408A Withdrawn EP1368782A4 (fr) 2001-01-16 2002-01-15 Calibrage coordonne pour systemes de balayage

Country Status (1)

Country Link
EP (1) EP1368782A4 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5038035A (en) * 1988-08-05 1991-08-06 Mitsubishi Jukogyo Kabushiki Kaisha Automatic structure analyzing/processing apparatus
US5581487A (en) * 1994-02-23 1996-12-03 Science Applications International Corporation Method and apparatus for microscopic screening of cytological samples
US5644512A (en) * 1996-03-04 1997-07-01 Advanced Surface Microscopy, Inc. High precision calibration and feature measurement system for a scanning probe microscope
EP1008956A1 (fr) * 1998-12-08 2000-06-14 Synoptics Limited Système de montage automatique d'image

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5038035A (en) * 1988-08-05 1991-08-06 Mitsubishi Jukogyo Kabushiki Kaisha Automatic structure analyzing/processing apparatus
US5581487A (en) * 1994-02-23 1996-12-03 Science Applications International Corporation Method and apparatus for microscopic screening of cytological samples
US5644512A (en) * 1996-03-04 1997-07-01 Advanced Surface Microscopy, Inc. High precision calibration and feature measurement system for a scanning probe microscope
EP1008956A1 (fr) * 1998-12-08 2000-06-14 Synoptics Limited Système de montage automatique d'image

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of WO02057999A1 *

Also Published As

Publication number Publication date
EP1368782A4 (fr) 2008-08-27

Similar Documents

Publication Publication Date Title
US7062091B2 (en) Coordinate calibration for scanning systems
US7864369B2 (en) Large-area imaging by concatenation with array microscope
CA2390540C (fr) Appareil et procede d'utilisation de reperes d'alignement sur un substrat de jeux ordonnes de microechantillons
US20030048933A1 (en) Time-delay integration imaging of biological specimen
KR101885939B1 (ko) 마이크로어레이의 해석 방법 및 판독 장치
JP4806630B2 (ja) 多軸統合を用いて三次元対象物の光学的画像データを取得する方法
EP2758825B1 (fr) Scanner de diapositives ayant un plan image incliné
US20030133009A1 (en) System and method for detecting with high resolution a large, high content field
US8848051B2 (en) Method of scanning biochip and apparatus for performing the same
US11614363B2 (en) Digital pathology color calibration and validation
KR20070107760A (ko) 기판 검사 장치
CN110140129A (zh) 使用双光学路径和单成像传感器的低分辨率载片成像和载片标签成像以及高分辨率载片成像
US20100259805A1 (en) Methods and systems for reducing scanner image distortion
KR20190062439A (ko) 오브젝트 조명 및 이미징을 위한 디바이스, 시스템 및 방법
JP2020046670A (ja) 調整可能な角度付照明を備えたハイスループット光シート顕微鏡
EP0990896A2 (fr) Dispositif d'analyse d'une image à grande surface
JP2012514758A (ja) 顕微鏡法
EP1368782A1 (fr) Calibrage coordonne pour systemes de balayage
US12085703B2 (en) Physical calibration slide
JP2024534332A (ja) 周期的にパターン化された表面のためのシーケンサ焦点品質メトリック及び焦点追跡

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20030811

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE TR

AX Request for extension of the european patent

Extension state: AL LT LV MK RO SI

A4 Supplementary search report drawn up and despatched

Effective date: 20080725

RIC1 Information provided on ipc code assigned before grant

Ipc: G06T 7/00 20060101ALI20080721BHEP

Ipc: G06K 9/32 20060101AFI20020729BHEP

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: APPLIED PRECISION, INC.

17Q First examination report despatched

Effective date: 20090105

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20090716