JP2008529082A - Compensation scanning optical microscope - Google Patents

Abstract

The compensated scanning optical microscope 10 is for obtaining images from different parts of the object plane and forming a preferably curved image surface having at least some aberrations that vary as a function of the part of the object plane from which the image is obtained. A scanning lens assembly 16 is provided. The steering mirror 14 selects the field of view and steers the light from the object 12 along the optical path from the object plane to the final image plane. The compensating optics 18 receives the steered light from the object 12 and compensates for optical aberrations dependent on the field position, and the auxiliary optics 20 moves from the steering mirror 14 through the compensating optics 18 to the final image plane. Is arranged along at least a portion of the optical path to condition and focus the light.
[Selection] Figure 1

Description

  The present invention relates generally to the field of optical microscopy, and in particular, a new and useful compensated scan that addresses and improves the common reciprocity problem between resolution and field of view common to known optical microscopes. The present invention relates to an optical microscope.

  This application claims priority to US Provisional Patent Application No. 60 / 647,572, filed Jan. 27, 2005, which is hereby incorporated by reference.

  In the US patent application Ser. No. 10 / 525,422, filed Feb. 25, 2005, the inventors presented an early approach to solving the problem of reciprocity between resolution and field of view. This application claims priority from US Provisional Patent Application No. 60 / 411,038 and International Application PCT / US2003 / 029332 published as WO2004 / 025331, all of which are also incorporated herein by reference. Incorporated in the book.

  For a wide range of applications (eg, observation and manipulation for microassembly, biological observation, biotechnology and medical diagnostics, manufacturing and inspection, etc.), optical microscopes are still intended for observation below the threshold of the human naked eye. One of the most important tools. However, in its conventional form, the optical microscope suffers from the reciprocal relationship between resolution and field of view as described above. The present invention is a new optical microscope design that combines a scanning lens, steering mirror, adaptive optics, adaptive optics (AO) conditioning optics, and imaging optics to expand the field of view while retaining the resolution in the acquired image. This device has the ability to operate at high image acquisition rates with increased throughput or to facilitate certain spatial-temporary observations.

  With recent advances in biotechnology and micro-electro-mechanical systems (MEMS) and industry trends towards miniaturization, observed and interacted with a scale below the human naked eye threshold, and The need for inspection is increasing. By meeting this need, the optical microscope has become re-interesting and continues to be an important tool as these fields evolve. However, the essential optical design and working principle has not changed significantly in the last century, and the optical microscope is still plagued by the well-known inherent reciprocal relationship between the field of view and resolution of the imaging system. ing.

  The present invention achieves high resolution by integrating image processing techniques using active optics, motion control, and conventional stationary optics in a closely coupled manner, as will be more fully described herein. To achieve an expanded field of view.

  The motivation to expand the field of view initially originated from our experience in microassembly and accurate manufacturing. Visually guided microfabrication often observes features at very distant parts almost simultaneously with micron or sub-micron resolution (eg, multiple critical edges of micromirrors and optical sensors assembled on a substrate) To do). Since a single microscope cannot provide a sufficiently wide field of view at the required resolution, it provides an off-the-shelf solution where multiple microscopes and / or moving stages are readily available. However, the limitations on movement per second and sample swaying by the moving stage, as well as the considerable effort required to reposition and calibrate multiple microscopes for each new assembly operation, This suggests the necessity of designing a new optical microscope to deal with these problems.

For the same reason, such microscopes are also desirable for biological and medical imaging performed using mechanical vision and industrial manufacturing and inspection. In the above-mentioned US, PCT, and provisional patent applications described above, the first design of the present inventors disclosed in Non-Patent Document 1 is called “Scanning Optical Mosaic Scope (SOMS)”. The advantage of combining a fast post-objective scanning system used for assembly and biological imaging with real-time mosaic construction technology is demonstrated. The previously disclosed optical arrangement for SOMS was initially shown by a machine made for laser annealed shape memory alloys (Non-Patent Document 2). This approach shares the concept of a post-objective two-dimensional scanning mirror. This configuration is also used in some commercial products, but in its basic form it has a limited field of view due to off-axis aberrations in the scanning lens. The present invention addresses this problem to provide a larger field of view.
B. Potsaid, Y. Bellouard and JT Wen: Scanning optical mosaic scope for micro-manipulation, Int. Workshop on Micro-Factories (IWMF02), edited by R. Holris and B. J. Nelson, pp. 85-88, 2002 M. Hafez, Y. Bellouard, T. Sidler, R. Clavel and R.-P. Salathe: Local annealing of shape memory alloys using laser scanning and computer vision, Laser Precision Microfabrication, I. Miyamoto, K. Sugioka and T. Sigmon Edit, Proc. SPIE 4088, pp.160-163, 2000

  A wide-field and high-resolution microscope imaging system is derived from consideration of (1) image extraction problems and (2) imaging performance problems. First, an imaging system with optics that is nearly perfect (ie, optical aberrations are well below the diffraction limit) should be considered. Such a system would imagine two point sources that are separated by a distance d, such as two overlapping Airy patterns in one image plane. As the distance between the two point sources decreases, the critical distance will reach a distance r at which the two point sources can no longer be individually identified. According to the Rayleigh criterion, this critical distance, called resolution, occurs when the center of one Airy disk falls over the other first smallest ring, and at the numerical aperture NA of the system and the wavelength of light λ. It is related. The NA of the system is a function of the refractive index n of the medium and the half angle of the cone of light from the focused object.

  Digital cameras must sample at 2 pixels per Airy center radius to avoid aliasing according to the Nyquist sampling standard. This observation gives the maximum theoretical object field width Wo with respect to the number k of sensor array pixels per edge and the resolution r.

  While microscopic imaging systems are often designed as having a resolution in the range of ¼ μm to several μm, the lower practical limit on the pixel size of a CCD camera is about 6 μm due to noise effects. Therefore, the optical system must expand the Airy pattern to allow proper sampling using the minimum magnification factor M required for a given sensor pixel size s. The corresponding image size Wi at this critical magnification is Wi = ks. Imaging optics to achieve this may be considered a comprehensive black box. The optical design task is to specify the design of the imaging system, i.e. to fill the aforementioned black box in detail with specific lens or mirror group geometry, glass type and spacing.

  An intuitive approach to designing wide-field and high-resolution imaging systems takes existing microscope arrangements and redesigns the optics to achieve a larger field of view while simplifying the camera pixel count It seems that it is to increase. While this approach may certainly be possible, it is generally impractical because the requirements for field size, flat image plane, and numerical aperture quickly approach those of a lithographic lens. A 1998 Nikon lithography lens (see, for example, US Pat. No. 5,805,344) has a NA of 0.65 with field sizes of 93.6 mm and 23.4 mm for mask and wafer images, respectively. .

  Lithographic lenses require near perfect manufacturing and very tight assembly tolerances (often requiring an interferometric assembly process) and can cost millions of dollars. Also, a negative magnification element is required and is placed at the location of the fine magnification region where the light beam is thick and the thin light beam area of the microscope and lithography lens. This design technique is used to achieve a flat image surface (small Petzval sum) and results in increased lens count and optical complexity. Another problem to consider is the size of the image sensor in that a large commercial CCD camera has only about 9216 × 9216 pixels (eg, Fairchild Imaging CCD595). Mosaic with the advantage that data can be read from parallel imaging chips to achieve a larger number of pixels (the data rate for obtaining image data from the chip can be a limiting factor in determining the maximum refresh rate) Smaller CCD arrays can be assembled, but at a cost that requires more precise assembly. Even with the latest technology and manufacturing capabilities, wide-field and high-resolution imaging systems based on purely static optical designs are very costly, large, tight assembly tolerances, and optical complexity It seems to be limited application only.

A summary of some of the recent alternative approaches to resolving the reciprocal relationship between field size and resolution is given in Table 1. For comparison, Table 1 also shows the performance of the present invention. The first five methods (multiple confocal objectives with multiple microscopes) are well established and quite normal. In this table, the “basic post-objective scanning” method refers to a commercially available unit that is limited to a very low numerical aperture in terms of system layout and is problematic in terms of significant off-axis aberrations. The array microscope sold by Dmetrix is of particular interest. Dmetrix is protected by several patents, such as US Pat. No. 6,958,464 for equalization for multi-axis imaging systems, US Pat. No. 6,950,241 for miniature microscope objectives for array microscopes, air bearings U.S. Pat. No. 6,905,300 for a slide feeder with a conveyor, U.S. Pat. No. 6,842,290 for a multi-axis imaging system with individually adjustable elements. This system uses an array of 80 miniature microscopes (each of three aspheric design elements) that work in parallel to obtain images quickly. Large scale composite images can be constructed by slowly advancing the array of microscopes along the extent of the microscope slide. Given the parallel imaging trajectory, this is the fastest surface scanning technique we currently know provides a medical diagnostic grade image of a stationary object (225 mm ² at 0.47 microns per pixel). Scan the area in 58 seconds and store it compressed). One related technique is a line scanning system that sweeps a specimen (often projected through a microscope objective) through a linear array of sensor pixels. The main drawback of this line scanning technique is that the image is obtained as a single line (n × 1 pixel) rather than as a single line (n × n pixel) as in a more typical surface-based image sensor. is there. The result is that to obtain high throughput, line scanning systems generally require very short exposure times and / or bright illumination, which is a biology that must take into account photodamage, bleaching and fluorescence. In many cases it is not possible in a typical application.

  Due to the parallel image acquisition and relatively slow relocation speed, Dmetrix excels in stationary and high fill factor applications. Fill factor is the percentage of the total observable area that is of interest and must be absolutely imaged or sensitized for the application under consideration. Since the ASOM of the present invention continuously obtains images for very fast repositioning rates, the ASOM will be superior in dynamic and / or low fill factor applications. Low fill factor applications include biological imaging of rare events on large cell populations, tracking of multiple moving organisms, and tissue extracted by needle extraction placed on a microscope slide Including medical diagnosis. Most manufacturing applications require a low fill factor only in certain critical areas where it is necessary to observe or inspect the frequently required features during dynamic tracking or assembly of the object.

  More generally, the ASOM of the present invention is particularly suitable for difficult spatial-temporary viewing tasks that require both a wide field of view and high resolution. Discussion of these issues motivated and contributed to the design of ASOM.

  Compensation optics with deformable mirrors have been used to enable high resolution imaging inside the human eye (H. Hoffer, L. Chen, G. Y. Yoon). , B. Singer, Y. Yamauchi and D. R. Williams, “Improvement in retinal image quality with dynamic correction of the eye's aberrations,” Op. Express 8, 631-543, 2001, http: //www.opticsexpress. see org / abstract.cfm? URI = OPEX-8-11-631). It is particularly difficult because of the time to change the aberration of the eye lens. Similarly, deformable mirrors have been used to correct off-axis aberrations and to correct wavefront disturbances caused by samples in confocal microscopes. The expansion of the field of view in the imaging system has also been shown previously by liquid crystal spatial light modulators to create a forbidden imaging system (D. Wick, T. Martinez, S. Restaino and B. Stone, “ Foveated imaging demonstration, “Opt. Express 10, 60-65, 2002, see http://www.opticsexpress.org/abstract.cfm?URI=OPEX-10-1-60).

  Known designs of confocal microscopes use a pinhole screen located in a plane that joins the object plane. This pinhole rejects light that is not at the same depth as the focal plane. The pinhole further rejects light that is not in the center of the image plane. Therefore, the confocal microscope extracts an object for each point. The image is constructed point by point and has means for scanning the position of the imaging point on the specimen. A basic introduction to confocal microscopy can be found at http://www.physics.emory.edu/~weeks/confocal.

  The ASOM of the present invention uses a finite image to obtain an image (ie, a complete two-dimensional image is exposed all at once rather than building an image point by point). The requirements for performing a finite image impose certain requirements on the optical system that are not necessary for point extraction techniques (confocal microscopy). Some of the advantages of ASOM's finite image-based approach are that multiple regions of the object are imaged in parallel, resulting in faster acquisition times. This is particularly important for low lighting conditions or when the object is moving. Furthermore, the lighting requirements are also advantageous. However, systems based on finite images do not provide the sample with the ability for vertical “cross-sections” such that confocal systems are possible.

  U.S. Patent No. 6,771,417 discloses a non-confocal arrangement that includes adaptive optics. See US Pat. No. 6,555,826 for confocal placement including adaptive optics and US Pat. No. 6,381,074 for adaptive optics in a scanning confocal microscope to aid in aberration control and precise focusing. US Pat. No. 6,483,641 discloses a spatial light modulator for use in a microscope.

  It is an object of the present invention to provide a compensated scanning optical microscope having improved performance over the prior art in a wide variety of fields.

  While the ASOM design of the present invention shares the scanning and mosaic construction principles of SOMS (see again US patent application Ser. No. 10 / 525,422), the ASOM of the present invention is an order that allows aberrations that do not reach the diffraction limit. It differs from related post-objective systems that perform SOMS and finite images in that the ASOM incorporates an adaptive optics that solves off-axis aberrations by introducing a designed scanning lens. In addition, ASOM simplifies the scanning lens by relaxing the flat image surface requirement and works in that it uses a steering mirror to project a significantly curved intermediate image surface that rotates around its own center. It is different from existing technology. Using these ideas to simplify the optical complexity and reduce manufacturing and assembly requirements, the basic concept of the new ASOM is to form a scanning lens and imaging optics to form a post-objective scanning arrangement. A few and very fast steering mirrors are used.

  An image is obtained at each scan position and a large composite image of the object can be quickly constructed by image mosaic techniques. The advantages of such an arrangement are a high resolution large effective field of view, no disturbance to the sample, and the ability to obtain many motions / images per second. However, such an arrangement also raises important design and implementation requirements due to off-axis imaging, which are further solved by the present invention by:

  1. Incorporating field curvature explicitly in the design to significantly reduce the complexity of the scanning lens.

  2. One image is obtained in one mode of operation, and the associated light is a compensating optical element (eg, variable mirror, spatial light modulator, optical phase array, variable lens or It travels along an optical path including a similar optical element.

  3. Image processing that removes image distortion.

  The ASOM design of the present invention excels in applications requiring high throughput, relatively low illumination conditions, and / or important spatial-temporary observations, but is virtually unlimited in connection with moving stages. It does not provide a field of view.

  Biological applications where ASOM would be attractive are dynamic cell events (mitosis, virus attachment, motility, cellular response to drug application) across many populations of living cells. This includes observing and observing a selected region of interest on the sample. ASOM may also be useful to quickly obtain images from well plates or to provide visual feedback in microinjection or manipulation activity. By illuminating the light path and inserting an appropriate filter, the additional illumination mode will allow fluorescence imaging. By inserting a phase plate near the aperture, phase contrast imaging for the observation of dominant phase objects common in biology will be achieved.

  Industrially, ASOM allows for visually guided microassembly, processing and rapid part inspection with the potential for higher product throughput. For medical diagnosis, ASOM allows for rapid imaging of biological samples. For example, in the case of a biopsy obtained by needle extraction, the sample is placed on the slide randomly and occupies only a small portion of the slide area. ASOM can perform very fast background scans and subsequent high quality scans of only the biopsy site of the region of interest. The high speed of rapid background scanning is obtained by imaging without stopping the movement of the steering mirror. These images will be slightly blurred but allow the tissue sample location to be identified. The ASOM then plans a scan trajectory to capture the region of interest. A high quality image of the sample is then obtained by obtaining the image using a steering mirror that is completely stopped and placed for each exposure.

  Therefore, yet another object of the present invention is to obtain images from different parts of the object plane and to form a preferably curved image plane with at least some aberrations that vary as a function of the portion of the object plane from which the image is obtained. A scanning lens assembly, a steering mirror that steers light from the image plane along the optical path from the object plane to the final image plane, and steered light from the object plane to the image plane Compensation optics for dynamically compensating for aberrations and at least a path of the light path for directing, conditioning and further focusing the light so that it travels from the steering mirror through the compensation optics to the final image plane. It is to provide a compensation scanning optical microscope composed of auxiliary optics provided along a part.

  Various novel features that characterize the invention are pointed out with particularity in the claims appended hereto and forming a part hereof. For a better understanding of the invention, its operating advantages, and specific problems achieved by its use, reference is made to the accompanying drawings and the description in which preferred embodiments of the invention are described.

  Referring to the drawings in which like reference numerals are used to refer to the same or similar elements, FIG. 1 takes a sequence of images that are continuously replaced from object 12 by a small space, and then large 1 shows a compensated scanning optical microscope or ASOM 10 that operates by assembling a scaled composite image (mosaic) or an image of a scene that has been disassembled into several and possibly overlapping.

  The general concept of expanding the field of view while maintaining resolution through mosaic construction is well known, and biological imaging (J. Zemek, C. Monks, and B. Freiberg, “Discovery through automation,” Biophotonics International 10). , 54-57, 2003) and industrial imaging (C. Guestrin, F. Cozman, and S. Godoy, “Industrial applications of image mosaicing and stabilization,” in Proceedings of IEEE International Conference on Knowledge-Based Intelligent Intelligent. Electronic Systems—Institute of Electrical and Electrical Engineers, New York, 1998, vol.2, pp.174-183), but instead of a general moving stage, the mechanism and scanning of the present invention The principle is that a specially designed scanner lens assembly 16, adaptive optics (AO) element 18 (example) For example, a variable mirror, spatial light modulator, optical phase array, variable lens, or similar optical element) and a high-speed two-dimensional steering mirror 14 operating in cooperation with the auxiliary imaging optics 20 are included. The image is eventually extracted by a sensor 22 such as a digital camera, spectrometer, or other light sensitive device of appropriate performance and speed.

  The imaging optics 20 includes a forward AO (compensating optics) conditioning or eyepiece 32, an inverted AO conditioning optics or eyepiece 34 and a final imaging optics 36, each of which can be comprised of one or more elements.

  FIG. 1 further illustrates at least one electronic system 15 for controlling the position of the steering mirror and at least one electronic system 17 for controlling the actuation signal to the adaptive optics. The at least one electronic system 19 also performs at least one of displaying, processing and / or storing light acquisition data at the final image plane to read data from a sensor (eg, camera 22). Provided for.

  FIG. 2 shows a conjugate image and an aperture plane of the ASOM of the present invention, and the optical elements are divided into a scanning lens 16, a front eyepiece 32, an inverted eyepiece 34, and a final imaging optics 36. The three optical assemblies or element groups behind these form the auxiliary imaging optics 20. The scanning lens 16 collects light from the object 12 or the object plane 1 while aiming the actual intermediate image projected by the steering mirror 14 positioned on the image of the pupil. A first image A1 of the aperture is produced as a result of the SLA 16 continuing in the optical path close to the first intermediate image plane 2. Acting like a known eyepiece in a conventional optical microscope, the eyepiece 32 in the ASOM 10 extracts the first intermediate image 2 and projects the external pupil where the deformable mirror 18 is located.

  The forward AO conditioning optics 32 in the initial design of the present invention is similar to Huygens eyepieces in that the intermediate image 2 is located between the negative field lens 33 and the positive eyepiece 35. A significant difference is the use of a negative field lens 33. This has the effect of extending the relief optical element relief (the distance between the eyepiece 35 and the compensation optical element such as the deformable mirror 18), but at the cost of a larger eyepiece 35. A second image A2 of the aperture is generated as a result of the eyepiece 35. The inverted AO conditioning optics 34 is similar to a Kelner eyepiece, but has a negative field lens 37 following the positive lens and the second image A2 of the aperture. The negative field lens 37 also helps to contribute to the negative Petzval sum in the imaging optics and establishes a second intermediate image plane 3.

  In order to investigate the effectiveness of the present invention, the inventor actually used a configuration similar to Huygens and Kelner eyepieces. Nonetheless, there are many eyepiece configurations that work well in the present invention. The experimental device of the present invention assembled in the laboratory uses an eyepiece configuration including up to seven lens elements. What matters is not the type of eyepiece, but what is important in defining ASOM is the function of the eyepiece. For this reason, the front and inverted eyepieces of the present invention are described in more detail as front and inverted AO conditioning optics.

  Also important here is that there are many different ways of constructing the front eyepiece pupil imaging optics, the inverted eyepiece pupil imaging optics, and the final imaging optics. For example, the simulated design discussed in detail herein uses two lens elements for the eyepiece group and one lens element for the final imaging optics. The laboratory setup assembled in the laboratory uses 7 elements for the front eyepiece, 3 elements for the inverted eyepiece, and 7 elements for the final imaging optics. Those with ordinary skill in the field of optics will be able to assemble other embodiments of the present invention once they understand the principles of the present invention.

  The final imaging optics 36 relays the second intermediate image 3 to a sensor (eg, the science camera 22, see FIG. 1) located at the final image plane 4 with an appropriate magnification to prevent aliasing. The system aperture stop delimits the beam bundle received by the imaging system. Eventually, the working area of the sensor will be provided for the field stop, but an additional field stop is added to the first and second intermediate image planes to reduce stray and unwanted light in the system. Can be done. Other mechanisms that reduce undirected stray light, such as machined grooves in the mechanical housing and black coating applied to the surface, can be preferably used throughout the ASOM optical path.

  The present invention also advantageously uses a curved field scan arrangement that is different from a microscope objective or lithography lens.

  Referring to FIG. 3, in range (c), the scanning lens 16 of the ASOM 10 of the present invention is designed to exhibit a pronounced field curvature C having a relatively large Petzval sum. As shown in the range (a) for the thin lens and the range (b) for the human eye in FIG. 3, the “natural” action of the lens is to form an image using a curved image plane. Relaxation of the image plane requirement provides the advantage that a very simplified optical design can be made using very few lens elements.

  Since the positive lens element contributes to the positive Petzval sum and the negative lens element contributes to the negative Petzval sum, the design of the flat image plane imaging system will achieve a Petzval sum of the entire system that is nearly close to zero. It requires careful use of both positive and negative lens elements. The non-single magnification is obtained by arranging the negative lens element in a region where the light beam diameter is thin and arranging the positive lens element in a region where the light beam diameter is thick. Compare the relatively simple ASOM scanning lens shown in FIG. 3 that allows for a curved image plane with a flat field microscope objective and a lithographic lens. In addition, the advantages of curved field designs have been recognized for aerospace applications as providing significant weight savings and design simplicity (J. M. Rodgers, “Curved Focal Surfaces: Design Optimization Through Symmetry, Not Complexity, (See “Photonics, Tech Briefs-Online, 2003, http://www.ptbmagazine.com/content/040103ora.html”).

  Additional characteristics that are not typical optical design objectives for ASOM scanning systems include:

  1. The center of field curvature, the center of rotation of the two-dimensional steering mirror, the surface of the mirror, and the optical pupil plane all coincide with each other.

  2. The shape of the projected image surface is almost spherical, rather than the more typical parabolic surface associated with field curvature. This is achieved through aberration control to a greater extent.

  Under the above mentioned conditions, changing the angle of the steering mirror causes the projected curved image surface to rotate around its own center, as shown in FIG. When the angle of the steering mirror is changed, the stationary imaging optics with a matching negatively curved image surface operate with a frame stop to extract a portion of the image surface, providing an image scanning and selection mechanism.

  This arrangement is advantageous because it eliminates the need for a large and flat image plane imaging system. Instead, as shown in FIG. 5, the system exhibits (1) a large positively curved image surface associated with the scanning lens and (2) a small negatively curved image surface associated with the imaging optics; This avoids the significant difficulties associated with designing and manufacturing a large continuous flat image plane imaging system as discussed above. In fact, the imaging optics are low numerical aperture, small field size and mainly used on-axis, so we have a medium size sensor array (512 × 512 pixels) It has been found that off-the-shelf optics can provide sufficient aberration correction for diffraction limited performance when used. Larger sensor arrays will require custom imaging optics.

  Next, the compensation optical element wavefront correction according to the present invention will be described in detail with reference to FIG.

  Although the scanning lens 16 and the overall system arrangement are clearly designed to handle field curvature, other off-axis aberrations (eg, coma, astigmatism and other higher order aberrations) are still present. While conventional solutions would add lens elements to balance these residual aberrations, with such intense off-axis imaging that occurs in ASOM, the fully compensated lens assembly is It would require an excessively large number of lenses.

  The present invention avoids this problem by designing a “good” scanning lens with significant wavefront aberrations (up to several waves in the optical path difference), as well as aberrations on the specific field of choice. In order to compensate for this, a variable mirror is used as the compensating optical element 18. Variations in aberrations are tolerated between individual field positions throughout the scan range. However, if the deformable mirror can only achieve one specific shape at a time, the rate of change in aberrations between field positions allows diffraction limited imaging performance across all selected partial fields. It must be small enough to do.

  This is similar to the concept of atmospheric isoplanatic (coma-free) patches that are widely recognized in the field of adaptive optics telescopes. By analogy in ASOM, the scan lens isoplanatic patch must be larger than the selected partial field of view. Otherwise, the image will be blurred at the end of each partial field.

  The simulation results below are based on highly reproducible ZEMAX simulations and demonstrate that ASOM can effectively provide an extended field of view while maintaining resolution when compared to existing microscopy techniques. is there. Table 2 lists the performance specifications for the specific ASOM design described herein, but with appropriate modifications to this design, the viewing area and numerical aperture can be adapted to the observation task under consideration. it can. In general, however, the achievable NA will decrease due to physical and practical limitations as the viewing area increases.

  FIG. 7 shows the observable field of view of an ASOM with a stationary microscope with a 4096 × 4096 camera (considering a full field camera with a standard microscope objective) and a more general 1024 × 1024 camera. To compare. ASOM provides a diffraction limit (Strehl ratio> 0.8) at all field positions based on highly reproducible simulations. The field size for the fixed microscope design assumes perfect imaging and λ with respect to the wavelength of the light (green light is relatively non-destructive and desirable for imaging of living biological cells) = 0.510 μm and a numerical aperture of 0.21.

  The partial field of view provided by the 512 × 512 camera used in this ASOM device is also shown in FIG. In this design, relatively simple imaging optics limit the camera sensor size to about 6.0 mm in diameter for diffraction limited performance. This also shows the performance of a single specific device of ASOM. By changing the lens geometry, the lens spacing and the number of pixels of the digital camera, the field size and numerical aperture can be adapted to the observation task under consideration. In general, however, there will eventually be a reciprocal relationship between the maximum observable field size and the numerical aperture of the system.

  With appropriate design changes in imaging optics, the diffraction-limited field size of imaging optics can be expanded to use higher pixel count cameras. However, even with a small 512 × 512 camera, the scan times listed in Table 3 can compete with existing technologies. This table shows the estimated scan times for 100, 250 and 500 frame per second camera speeds and 100%, 50% and 10% fill factors. These calculations assume that the total number of scanning operations is given by the number of scans = total effective field area / partial field area.

  FIG. 8 illustrates the different modes of operation of the ASOM of the present invention, such as detection of rare events, tracking moving objects in time, imaging only the region of interest, and full area coverage.

  FIG. 9 shows how the DM corrects the specific wavefront aberration associated with each field position. The Strehl ratio over the entire field of view and for all field positions is well above the diffraction limit of 0.8, resulting in almost complete imaging. Range (a) shows five different field positions. Range (b) shows five corresponding optimal variable mirror shapes for each field position, and range (c) gives the Strehl ratio extracted on the selected field of view.

  All the results presented here are based on idealized simulations ignoring the reality that lenses and optical housings always obey manufacturing and assembly tolerances.

  In order to demonstrate the basic principles of scanning and image mosaic construction of the present invention, the following experimental device was fabricated as a first generation prototype called a scanning optical mosaic scope (SOMS). No formal optimization of this design was done, and the prototype device was a standard catalog lens available from ThorLabs, Sony XC-77BB CCD camera, Matrox Meteor II frame grabber, Cambridge technologies galvanization Built using a meter and servo driver and a DSP board based on TI products.

  The more advanced ASOM designs proposed here are: (1) simplified optical placement, (2) no variable mirrors or adaptive optics, (3) all lenses are It is different in that it is available as a standard catalog product and (4) the scanning lens is a single standard achromatic doublet. These results of SOMS are included here to demonstrate possible modes of operation, functionality and capabilities that can be performed using ASOM, but better performance is achieved by using ASOM design. Is demonstrated.

  The microassembly demonstration of the present invention is based on a shape memory alloy microgripper that moves between two fixed objects in the workspace. To track the motion of the gripper tip, a basic correlation based image matching algorithm and a Kalman filter are used. A 3x3 tile mosaic images the gripper and the scan pattern is automatically adjusted to keep the gripper tip within the center tile. The scan pattern also includes two stationary objects in the workspace, demonstrating SOMS 'ability to observe multiple stationary and moving objects in the workspace almost simultaneously.

  A sequence of video foot numbers is employed especially for living biological cells (telomerase-immortalized hTERT-RPE1). The 3x3 tile image mosaic monitors large cell populations without damaging cells that continue to survive in temperature-controlled nutrient solutions. It can be seen that several events of mitosis (cell division) occur throughout the entire field of view. ASOM not only offers the possibility of automatically detecting the onset of mitosis and other events, but can also be easily programmed to track and record multiple events simultaneously. Although automated quantitative cell analysis using a moving stage has recently been proposed, the overall system bandwidth is still limited by the responsiveness of the stage and the sensitivity to movement of the cell specimen.

  The ASOM of the present invention solves both of these problems.

  We have also created a second generation experimental prototype.

  The purpose of this experimental ASOM device was to demonstrate the essential optical aspects of the ASOM design, but at a low cost and with a short development time.

  For this reason, off-the-shelf optics were used exclusively to avoid the considerable cost of custom optics and take advantage of the existing inventory of catalog-available parts that are shipped within a few days. However, many off-the-shelf lenses are designed to be used in a particular manner (eg, infinite conjugate) for general applications and are offered within a coarse range of focal length, lens diameter and glass selection. Given the anomalous imaging characteristics of the scanning lens, an experimental ASOM design using only off-the-shelf optics is far from optimal, and is therefore 0.1 over a nominal field size of 20 mm. This indicates a significantly large number of lenses in order to achieve NA. However, even when using only off-the-shelf optics, the experimental setup includes a curved field optical scanning approach and wavefront correction optics using a deformable mirror as the compensating optical element, demonstrating important optical properties that define ASOM. Designed carefully to do. In this device, the steering mirror is manually operated, limited to a microscope for observation of stationary or slowly moving objects. Commercial versions of compensated scanning optical microscopes are custom built to fully realize the potential of the ASOM concept to achieve higher numerical apertures and larger workspaces, as well as incorporating motion-controlled high-speed steering mirrors. Will take advantage of the optics that have been made.

  FIG. 10 shows the optical arrangement for this experimental apparatus. This first prototype uses a transmitted light scheme and uses a 510 nm wavelength notch filter to remove much of the light spectrum below 500 nm and above 520 nm because the current design is very sensitive to chromatic aberration To do. The light is transmitted through the objective contrast pattern and then focused by a telecentric 12-element scanning lens assembly, which projects an image of the object onto a spherically curved image plane. A manually actuated steering mirror having the motility to pivot the mirror about its own silver front surface is located behind the scanning lens assembly and operates in cooperation with the field stop in the wavefront correction optics; Select which part of the spherically curved image plane passes through the system to form an image with the camera. This scanning mechanism effectively enables partial field operations in the work space. However, the light at this point exhibits significant wavefront aberrations as a result of insufficient optical correction of the scanning lens (allowing insufficient correction is a feature of the ASOM design, which is the complexity of the scanning lens assembly. And note that the number of lenses is significantly reduced).

  This poorly corrected light from the steering mirror that has passed through the field stop then continues to travel into the wavefront correcting optics. A MEMS deformable mirror is used in this embodiment of the compensated scanning optical microscope. By precisely controlling the shape of the mirror's reflective surface to be opposite (but half the amplitude) of the wavefront error shape, the deformable mirror can correct the wavefront aberration to within the diffraction limit. Therefore, the light emitted from the deformable mirror will be sufficiently corrected and will form a complete diffraction limited image on the camera that is almost indistinguishable. A three-layer MEMS deformable mirror available from Boston Micromachines was used in this prototype. This mirror has 32 electrostatic actuators with 400 μm actuator spacing and 2.5 μm actuator stroke and has an actively controlled area of 2.0 mm in diameter. The 2.5 μm stroke can correct some waves of aberrations, enables high imaging performance even for off-axis field positions, and allows for a greatly expanded field of view in ASOM. Also, a preconditioning stage for the compensation optics and a postconditioning stage for the compensation optics to condition the light to match the effective diameter of 2.0 mm of the compensation optics are provided for the positive and negative lens elements. It should be noted in this design that it is configured by an appropriate combination. The preconditioning stage for the adaptive optics element forms an image of the aperture so that the steering mirror is located near the aperture image. By arranging the steering mirror at or near the position of the aperture image, the diameter of the steering mirror can be reduced so as to reduce the inertia of the steering mirror for faster dynamic characteristics. Furthermore, placing the steering mirror at or near the position of the aperture image facilitates symmetrical use of the scanning lens during scanning (ie, the chief rays for all field positions are aperture planes). It is optically desirable because it starts from the same position inside). The optical advantages of using a single steering mirror are well known, but for a variety of reasons including cost and dynamic performance, the single steering mirror described herein is two single axis steering mirrors. It is conceivable that can be replaced by. Doing so is less desirable from an optical standpoint, but often meets the requirements and is very often done in practice. An ASOM can be constructed using two or more rotating mirrors.

  Similar to the steering mirror, the adaptive optics is located near the image of the aperture. By placing the compensating optical element at or near the position of the image of the aperture, all of the beam bundles overlap at this position (ie, the chief ray intersects the optical axis), so that the active area of the compensating optical element Can be used effectively. Placing adaptive optics at or near the chief rays also crosses each instantaneous field of view as well so that one adaptive optical wavefront correction (eg, variable mirror shape) is included in each ray bundle. It is possible to effectively correct the aberration.

  We have also performed calibration and online optimization on next generation prototypes using image-based performance metrics and parallel probabilistic gradient descent optimization algorithms.

  The final imaging performance of the compensated scanning optical microscope depends on the magnitude and shape of the residual wavefront aberration in the system. Further, the scanning lens introduces aberrations specific to each field position (steering mirror angle). Therefore, if the influence of the compensating optical element on the wavefront shape is controlled by the control signal and has a direct effect on the wavefront aberration, an initial calibration of the compensating scanning optical microscope should be performed. The ultimate goal of this calibration is to find a set of control signals that minimize wavefront aberrations at different field positions. Once calibrated, the optimal control signal can be recalled from the lookup table using interpolation methods during normal operation. This calibration can compensate for manufacturing and assembly errors, tolerances, or other variations in manufacturing, and to compensate for environmental temperature changes, optical component variations or changes, or other sources of aberrations in the system. Repeated regularly. In operation, the steering mirror and the compensating optics will be adjusted by an electronic device for effective compensation of optical aberrations during imaging.

  There are many potential ways to obtain an optimal adaptive optical control signal, including using wavefront sensors, wavefront evaluating experimental methods and algorithms, interferometer based methods or other image based techniques. In addition, compensation scanning that updates the control signal of the compensation optical element during execution using real-time measurement of wavefront aberrations and feedback control similar to that performed in an adaptive optics telescope using a guide star as a reference wavefront An optical microscope system is conceivable.

Performance metrics and numerical optimization algorithms were used for this experimental prototype. In general, the performance metric Q (u) is defined as a nonlinear function of the adaptive optics control signal u, and Q (u) is defined as decreasing with improved imaging performance. The resulting optimization problem is also subject to upper and lower limits in the adaptive optics control signal. It has been demonstrated to effectively calibrate the system by combining a metric based on high frequency image content and a parallel stochastic-gradient-descent (PSGD) optimization algorithm. In general, adaptive optics control signal optimization requires two parts where there are various possible options and specific combinations.
1. 1. Metric Q (u) for representing imaging performance, and An optimization algorithm for minimizing Q (u).

  Additional improvements to the ASOM described above include using a spectrometer instead of a camera. Introducing a phase plate near the aperture and / or using adaptive optics to introduce a phase perturbation in the wavefront will allow the ASOM to perform phase contrast imaging. It is also possible to inject light into the ASOM as means for irradiating an object by introducing a beam splitter in the optical path.

  In conclusion, the present invention can achieve high resolution and a large effective field of view simultaneously, and provides several advantages over current techniques for observing certain spatial-temporal events. It is a new microscope concept. This design relies heavily on the synergies of optical, mechanical, motion control, and image processing design. ZEMAX optical simulations show diffraction-limited imaging performance over a greatly expanded field of view, while calculations show the potential for fast movement and image acquisition operations. A reduced proof-of-concept prototype was built to demonstrate the basic effectiveness of the mirror-based scanning approach, and we have demonstrated it in both micro-assembly and biological observation work.

  The scanning lens assembly, forward and inverted conditioning optics, and final imaging optics of the present invention each comprise one or more glass lens elements; synthetic resin lens elements; GRIN (graduated index of refraction) Element; Diffractive lens element; Spherical optical element; Non-spherical optical element; External pupil, telecentric behavior, non-telecentric behavior element or collection of elements; uniform numerical aperture for all field positions, no for different field positions Uniform numerical aperture; f-θ strain map, f-cos θ strain map, element or collection of elements substantially following f-sin θ strain map; curved image plane at first intermediate image plane and substantially spherical And an element or a group of elements that project an image plane curved at the first intermediate image plane and a substantially radial image plane curved at the first intermediate image plane. It may be.

  The image steering mirror can be generalized as any image steering means or equivalent for performing the steering function, which includes galvanometer, voice coil actuator, piezoelectric actuator, electrostatic actuator, gimbal mechanism, parallel It may include a steering means having at least one of a mechanism, a flexible mechanism, and a magnetic levitation. The steering mirror can be at least one of a flat reflective surface, a curved reflective surface, a curved reflective surface that is substantially spherical, a substantially aspherical curved reflective surface, or a rotating prism. The sensor for receiving light at the final image plane can be at least one of a digital camera, a charge coupled device, a CMOS sensor, a spectrometer, or an eyepiece for viewing with the human eye.

  While specific embodiments of the invention have been shown and described in detail to illustrate the application of the principles of the invention, it is to be understood that the invention may be embodied in others without departing from such principles. It will be understood.

1 is a schematic conceptual drawing of a compensation scanning optical microscope or an ASOM according to the present invention. FIG. 3 is a composite diagram illustrating various conjugate images and aperture planes of the present invention. (A) is the shape of the image plane for a thin lens, (b) is the curved surface of the retina (image sensor) that allows a very simple lens in the human eye, and (c) is a curved image plane. It is explanatory drawing of several parts which shows the ASOM scanning lens of this invention simplified by doing. It is a compound explanatory view of the curved image surface of the scanning lens assembly of the present invention. It is a compound explanatory drawing of the field curvature of the scanning lens aggregate of this invention and imaging optics. It is explanatory drawing which shows the initial design of ASOM of this invention. Shown is a 40 mm practical field of view of the ASOM of the present invention compared to the field of view presented by a conventional microscope using 1024 × 1024 and 4096 × 4096 cameras (all systems operating at 0.21 NA). Also shown is an ASOM 0.38 mm size partial field of view using a 512 × 512 camera that requires a lot of scanning motion to cover the entire 40 mm field of view. 2 is a diagram illustrating some of the different modes of operation of the ASOM of the present invention. (A) is a different field position of the invention, (b) is the optimal deformable mirror shape for each particular field position, and (c) is a compound that explains the Strehl ratio extracted on the selected field of view. FIG. Fig. 2 is an optical arrangement of our latest experimental device to demonstrate the principle of the present invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Object plane 2 1st intermediate image plane 3 2nd intermediate image plane 4 Final image plane A1 First image A2 of opening A2 Second image of opening 10 Compensation scanning optical microscope or ASOM
12 Object 14 Steering mirror 15 Electronic system 16 for controlling the position of the steering mirror 16 Scanning lens assembly 17 Electronic system 18 for controlling the actuation signal to the compensating optical element 18 Compensating optical element (eg variable mirror)
19 To read the data from the sensor,
Electronic system 20 for performing at least one of displaying, processing and / or storing light acquisition data at the final image plane Auxiliary imaging optics 22 Sensor (eg camera)
32 Front AO Conditioning Optics or Eyepieces 33 Negative Field Lens 34 Inverted AO Conditioning Optics or Eyepieces 35 Positive Eyepiece 36 Final Imaging Optics 37 Negative Field Lenses

Claims (29)

  A scanning lens assembly for forming a finite image from the selected field positions of different objects so that optical aberrations change as a function of the selected field position, forming a single image plane A body, image steering means for steering light from each field position and having a wavefront along an optical path from the object to a final image plane, and at least some of the optical aberrations depending on the field position. A compensation optical element that compensates to affect the shape of the wavefront in the optical path and introduces a selected shape to the wavefront of the light along the optical path; light for the compensation optical element and light for the final image plane Compensating scanning optical microscope comprising: auxiliary imaging optics along at least a portion of the optical path for conditioning and final imaging optics for projecting an image onto the final image plane.   The compensation scanning optical microscope according to claim 1, wherein the image plane formed by the scanning lens assembly is curved.   2. The compensation scanning optical microscope according to claim 1, wherein the image steering means includes a steering mirror and a means for steering the mirror.   The adaptive scanning optical microscope according to claim 1, wherein the adaptive optical element includes a variable mirror.   The compensating optical element that conditions the light along the optical path to adapt the aperture size of the compensating optical element to the optical requirements of the scanning lens assembly to use the active area of the compensating optical element And a preconditioning stage for the adaptive optics to condition the light along the optical path to adapt the aperture size of the adaptive optics to the requirements of the final imaging optics The compensation scanning optical microscope according to claim 1, wherein the imaging optical includes.   6. The adaptive scanning optical microscope according to claim 5, wherein both the preconditioning stage for the compensating optical element and the postconditioning stage for the compensating optical element include at least one negative lens and at least one positive lens.   The compensated scanning optical microscope of claim 1 including an aperture stop for demarcating the bundle of light received by the apparatus and projected onto the final image plane.   Including the final imaging optics to provide additional magnification or reduction to achieve a selected system magnification and to project a final image onto the final image plane, and a light sensor for sensing the final image. The compensation scanning optical microscope according to claim 1.   The scanning lens assembly is a glass lens element; a synthetic resin lens element; a GRIN lens element; a diffractive lens element; a spherical optical element; an aspherical optical element; at least one element exhibiting an external pupil, telecentric behavior, and non-telecentric behavior Uniform numerical aperture for all field positions, non-uniform numerical aperture for different field positions; at least one element substantially following the f-θ distortion map, f-cos θ distortion map, f-sin θ distortion map; Projecting an image plane curved at the intermediate image plane, an image plane curved at the first intermediate image plane that is substantially spherical, and an image plane curved at the first intermediate image plane that is substantially radial. 2. Compensated scanning light according to claim 1, comprising an optical mechanical assembly of one or more lenses or mirror elements including at least one of at least one element. Microscope.   The image steering means has a steering mirror and a means for steering the steering mirror, and the means for steering the steering mirror is at least one galvanometer, voice coil actuator, piezoelectric actuator, electrostatic actuator, gimbal mechanism, parallel The steering mirror includes a flat reflective surface, a curved reflective surface, a substantially spherical curved reflective surface, substantially 2. The compensation scanning optical microscope according to claim 1, comprising at least one of an aspherical curved reflecting surface and a rotating prism.   The compensation scanning optical microscope according to claim 1, wherein the compensation optical element includes at least one of a variable mirror, a spatial light modulator, an optical phase array, a variable lens, and an electro-optical element.   Forward conditioning for the compensating optical element, wherein the auxiliary imaging optics comprises at least one of a glass lens element, a synthetic resin lens element, a GRIN lens element, a diffractive lens element, a spherical lens element, and a non-spherical lens element Optics: Glass lens element, synthetic resin lens element, GRIN lens element, diffractive lens element, spherical lens element, inverted conditioned optics having at least one of aspherical lens element; and glass lens element, synthetic resin lens element, The compensation scanning optical microscope according to claim 1, comprising the final imaging optical element having at least one of a GRIN lens element, a diffractive lens element, a spherical lens element, and an aspherical lens element.   A sensor for receiving light at the final image plane, the sensor having at least one of a digital camera, a charge coupled device, a CMOS sensor, a spectrometer, and an eyepiece for viewing by the human eye The compensation scanning optical microscope according to claim 1.   A scanning lens assembly for obtaining an image from different parts of the object plane and forming a curved image plane having at least some aberrations varying as a function of the off-axis region of the object plane from which the image is obtained; A steering mirror for steering light from the image plane along the optical path from the object plane to the final image plane; receiving the steered light from the object to the image plane and at least some aberrations; A variable mirror for dynamically compensating; and along at least a portion of the optical path for guiding the light so that the light travels from the steering mirror through the variable mirror to the final image plane Auxiliary imaging optics, a front eyepiece between the steering mirror and the variable mirror along the optical path, and the variable mirror along the optical path. Inverted eyepiece between the system aperture of the microscope, and the compensation scanning optical microscope with an auxiliary imaging optics having a final imaging optics between the final image plane and the inverted eyepiece.   The front eyepiece includes at least one negative field lens and at least one positive lens; the inverted eyepiece includes at least one negative field lens and includes at least one positive lens; The compensation scanning optical microscope according to claim 14, which has at least one positive lens.   The front eyepiece includes at least one negative field lens and at least one positive lens; the inverted eyepiece includes at least one negative field lens and includes at least one positive lens; 15. The compensation scanning optical microscope of claim 14, comprising at least one positive lens, the microscope including a camera for receiving light at the final image plane.   A scanning lens assembly for obtaining an image from a different part of the object plane and forming a curved image plane having at least some aberrations varying as a function of the area of the object plane from which the image is obtained; A steering mirror for steering the light from the object plane along the optical path from the object plane to the final image plane; receiving the steered light from the object to the image plane and dynamically at least some aberrations An auxiliary imaging along at least a portion of the optical path for guiding the light such that the light travels from the steering mirror through the variable mirror to the final image plane. A front eyepiece between the steering mirror and the variable mirror along the optical path, and the system opening of the variable mirror along the optical path and the microscope. And an auxiliary imaging optics having a final imaging optics between the inverted eyepiece and the final image plane; and a science camera for receiving light at the final image plane Compensating scanning optical microscope.   The front eyepiece includes at least one negative field lens and at least one positive lens; the inverted eyepiece includes at least one negative field lens and includes at least one positive lens; The compensation scanning optical microscope according to claim 17, wherein at least one positive lens is included.   Using a scanning lens assembly to obtain and enlarge a finite image from different parts of the object plane and further form an image plane with at least some aberrations that vary as a function of the part of the object plane from which the image is obtained Steering the light from the scanning lens assembly along the optical path from the object plane to the final image plane; using deformations to the wavefront of the steered light from the object to the image plane Dynamically compensating for at least some aberrations; and using auxiliary imaging optics along at least a portion of the optical path so that the light travels from the steering phase through the deformation phase to the final image plane. A method of compensating scanning the object plane for an image to be magnified, which comprises focusing the light onto the surface.   20. The method of claim 19, wherein the image plane formed by the scanning lens assembly is curved.   A front eyepiece following a steering mirror for performing the steering step along the optical path, a variable mirror for performing the deformation step, an inverted eyepiece along the optical path between the variable mirror and a system opening of the microscope, The method of claim 19, wherein the auxiliary imaging optics has a final imaging optics between an inverted eyepiece and the final image plane.   The method of claim 21, wherein at least one of the eyepieces comprises a negative lens.   Including a system aperture of the microscope along the optical path and final imaging optics between the inverted eyepiece and the final image plane, both the front and inverted eyepieces having a negative lens and a positive lens. Item 23. The method according to Item 22.   Scanning lens assembly for forming a finite image from different selected field positions of an object such that optical aberrations vary as a function of the selected field position, forming a first intermediate image An image steering means for steering light from a first image of an aperture for each field position and having a wavefront and a first intermediate image plane along an optical path from the object to a final image plane; A compensating optical element that compensates for at least some of the optical aberrations depending on the field position to affect the shape of the wavefront in the optical path and introduces a selected shape to the wavefront of the light along the optical path Compensating optical element located in the second image of the aperture and the aperture size of the compensating optical element to effectively use the active area of the compensating optical element and the optical requirements of the scanning lens assembly A preconditioning stage for the adaptive optics to condition the light along the optical path to be adapted, and the adaptive optics to the requirements for final imaging optics and for projecting light at a second intermediate image plane A post-conditioning stage for the adaptive optics that conditions the light along the optical path to match the aperture size of the element, and a final result for projecting an image from the second intermediate image plane at the final image plane. A compensation scanning optical microscope having image optics.   25. A compensated scanning optical microscope according to claim 24 including an aperture stop for demarcating the bundle of light received through the imaging system and projected onto the final image plane.   25. The adaptive scanning optical microscope according to claim 24, further comprising at least one sensor for extracting the final image and converting optical information into a measurable amount.   27. The adaptive scanning optical microscope according to claim 26, comprising at least one electronic system for controlling the position of the steering mirror and controlling the actuation signal to the adaptive optics.   28. A compensated scanning optical microscope according to claim 27, comprising at least one electronic system for reading data from the sensor.   29. The compensation scanning optical microscope according to claim 28, comprising at least one electronic system for displaying, processing and storing light data obtained in the final image plane.

Images