JP2018520337A - Method for determining spatially resolved height information of a sample using a wide field microscope and a wide field microscope - Google Patents

Abstract

The present invention relates to a wide field microscope and a method for determining spatially resolved height information of a sample (14) using a wide field microscope. In the wide-field microscope, an irradiation source (1, 52, 53) arranged in the irradiation beam path and a wide-field image in the observation beam path of the sample (14) irradiated in the sample surface (P) are displayed. A detector (17, 33) for capturing, a modulator for color modulation in a direction perpendicular to the sample plane (P) of the illumination beam path or the observation beam path, and each of the wide-field images And an evaluation unit for calculating chromatic confocal height information within the pixel. The method comprises using any broadband illumination source (1) to illuminate the sample (14) in an illumination beam path, color modulating the illumination beam path or detection beam path, and the detection beam path. Capturing at least one wide-field image from sample light having a chromatic confocal component reflected or emitted from the sample in the image, and from the wide-field image, the chromatic confocal component of the detection beam path and the color modulation And determining various height information of the sample for each pixel.

Description

  The present invention relates to a method and a wide field microscope for determining spatially resolved height information of a sample using a wide field microscope.

  The method for determining the spatially resolved height information of the sample is also called a light cutting method. Especially in microscopy, such light cutting methods have come to be used to determine the topography of a sample or to measure various surface properties of a sample, such as roughness. Yes.

  In order to characterize various technical surfaces, confocal microscopy is now introduced as a standard technique. There, in most cases, the sample is scanned in all three spatial directions, i.e. this is a point scanning system, in which the rays are x along the surface of the sample. / Y direction. In order to derive the height information, it is necessary to move the sample relative to the detector unit (in the z direction). From the relationship between the maximum intensity and the z position, height information and thus topography can be derived for each xy coordinate point.

  This method has the disadvantage that, among other things, the long time required by raster scanning to obtain 3D topography is a disadvantage. Furthermore, during an xy scan in which the geometrical arrangement between the sample body and the optical sensor is fixed, a relative movement that is not under the control of the sensor head with respect to the sample body may occur due to external shock or vibration. This may cause the measurement result to be bent.

  To avoid this z-raster scan, the chromatic confocal principle is used. Typically, any polychromatic light source is introduced here to spectrally code z-information by illuminating the sample of interest through some refractive and / or diffractive element that exhibits a chromatic effect. At this time, if the spectrum is measured behind the confocal pinhole in the detection path, height information can be derived therefrom. In addition, although it is possible to use some variable light source in which confocal detection is performed in a sequential manner, although it takes time, a spectrum can be obtained similarly.

  Kim et al. (Non-Patent Document 1) describe a point scanning chromatic confocal array with a 50:50 beam splitter in the detection beam path behind the pinhole detector. The sample light is captured accordingly using two photomultiplier tubes (PMT), in which a filter is placed in front of one PMT. From the intensity ratio of both PMTs, the transmittance of this filter is determined, the wavelength detected thereby is determined, and the height information is finally determined therefrom.

  Various confocal wide-field systems have already existed for quite some time to avoid this xy raster scan shortcoming, but it is customary to introduce line scan cameras there. . An example of this is the spinning disk method using a Nipkow disk. There, several points are detected almost simultaneously based on the confocal principle. Here again, it is required to reach a plurality of different z positions in order to determine the tomographic image.

  Furthermore, various confocal wide-field systems based on structured illumination are known. There, for each z value, a confocal tomographic image is calculated from various images taken using structured illumination given by an arbitrary grid, for example. In that case, a wide-field image can also be acquired normally. Various height information is acquired from the sample while fully utilizing the polarization of illumination light and various color characteristics. For example, Patent Document 1 describes such a system, in which two illumination patterns are projected onto a sample.

  Similarly used is the aperture correlation method. There, an optical tomographic image is calculated from two images taken in a parallel or sequential manner using continuously changing structured illumination, but one of the two images has a focal point. It can be considered that the confocal image is inferior in focus and includes an out-of-focus component, and the other is a pure wide-field image or an image in which the out-of-focus component is dominant. The advantage of the method based on structured illumination is that, in parallel with the confocal image, a wide-field image can be acquired with a single shutter shot.

  Finally, common to all systems based on structured illumination, vibrations can interfere with measurement results while the phase pattern is changing and / or when moving the sample or sensor in the z direction Sometimes.

  There are still other wide-field techniques utilized to enable optical cross sections to be generated. For example, focus variation (focus movement method) can be used. In this case, the relationship between the sharpness of the image and the z-coordinate is evaluated, and from there, the maximum value is calculated, similar to the confocal case. doing. At that time, various spatial information of the system is also used.

  With respect to vibration susceptibility, the same problem as the above method arises.

German Patent Application No. 102007018048

Kim et al., “Chromatic confocal microscopy with novel wavelength detection technique, ES method 86 EPR 86 S. 21, no. 5

  It is an object of the present invention to present a microscope and method for generating spatially resolved height information of a sample that can avoid various disturbing movements acting on the microscope.

  This problem is solved by a method according to claim 1 and a wide field microscope according to claim 7.

1 is a diagram of a first preferred embodiment of a wide field microscope according to the present invention. FIG. The figure of the deformation | transformation structure form provided with the parallel detector. The figure of the deformation | transformation structure form provided with the parallel detector and the filter in the detection beam path | route. The figure of the deformation | transformation structural form provided with the switching element in the detection beam path | route. The figure of the deformation | transformation structure form provided with the chip | tip splitter detector (Chip-Splitter-Detektor). FIG. 3 is a diagram of a second preferred embodiment of a microscope according to the present invention. FIG. 3 is a diagram of a third preferred embodiment of a microscope according to the present invention. FIG. 4 is a diagram of an advantageous modified configuration of an irradiation beam path with a switching element. FIG. 4 is a diagram of an advantageous variant configuration of an illumination beam path with two identical illumination sources. FIG. 4 is a diagram of an advantageous variant configuration of an illumination beam path with two illumination sources with different spectra.

  According to the present invention, the chromatic confocal principle is applied to and adapted to any optical wide-field tomography. This is achieved in particular by encoding various wavelengths in the illumination beam path or the detection beam path. In an example of a preferred embodiment, various wavelength dependent filters are used in the detection beam path.

  In the method according to the invention, the sample is irradiated in the irradiation beam path using any broadband irradiation source. This illumination beam path is color modulated based on the chromatic confocal principle. Furthermore, at least one wide-field image is captured by capturing sample light reflected or emitted from the sample in the detection beam path.

  This wide-field image may have only a purely confocal component of the sample light (for example, if a Nipow disk is used), or else it may be out of focus with the confocal component of the sample light. It may be synthesized from the above ingredients.

  Within this observation beam path and / or within the illumination or excitation beam path, at least one wavelength-dependent filter function or spectral distribution is introduced to provide a different filter or spectral distribution for each pixel in the xy direction. In use, at least two measurement steps are performed. These measurement steps may be performed in a parallel manner (when several image sensors are used) or in a sequential manner.

  For example, when the ratio is obtained from the intensity of at least two measurement steps in each pixel of the wide-field image, the wavelength at which the sample light has the maximum intensity in each pixel is determined, and thereby the sample The height value can be determined.

In particular, this is done well, basically irrespective of the spectral reflectance of the sample, and even regardless of the spectral characteristics of the light source and / or device.
In general, the intensity signal in the i th measurement step of the method and microscope according to the invention is obtained by the following equation:

here,
P (x, y, λ ′): Spectral characteristics of the light source and device, possibly also a function of the x, y coordinates R (x, y, λ ′): Spectral reflectance of the sample T _i (x, y, λ) '): Filter function or spectral distribution in the i-th measurement process, or color modulation (including beam splitter, etc.)
g _λmax (λ _max −λ ′): device spectral response function with λ _max as a parameter λ _max = λ [z (x, y)]: maximum reflection wavelength at points x and y according to the height function z (x, y): height function of the sample What is obtained here is the height function z (x, y) regarding the sample to be investigated. Confocal or quasi-confocal detection is expressed in particular by the function g _λmax (λ _max −λ). This parameter display regarding the wavelength suggests that this function has a spectrum variation depending on the nature of the color deviation. For further discussion, for convenience, assume that this parameter representation is negligible and that this function g is a delta function that is localized when it is zero. Then, the above equation is simplified and becomes the following equation.

If P, R and T are sufficiently known, basically z (x, y) can already be derived from this, but in any case this is a kind of absolute measurement, There will be an associated calibration cost.

  However, when using at least two detection steps i = 1,2, the following formula:

Based on this, the ratio can be determined.

  P and R no longer make sense. The same can be said for the case where P and R are stationary over the integration range predetermined by the form of g. From the right hand side, based on various known filter functions or spectral distributions, it is relatively simple to determine the value λ associated with a given intensity ratio and thereby determine the value z (x, y). be able to.

In a special case, T ₂ is not a function of wavelength (T _s (λ) = stationary). This case can be realized, for example, by using two detectors of the same type and a beam splitter, where a wavelength dependent filter is introduced only in one of the beam paths. Will be.

  Yet another possibility is to use only one channel and two measurements, once with a wavelength-dependent filter, not once, or two different wavelength dependencies. However, in any case, these two measurements can be performed without a gap so that it can be said to be a kind of one-shot measurement. (Total measurement time <100 ms).

Alternatively, a filter can be used in the excitation path, but with the same light source, the measurement is alternated with and without the filter.
In yet another special case, two bandpass filters whose spectra are offset relative to each other are used in the excitation and / or detection paths. Again, in the detection path, a parallel array other than the sequential array can be used. As an example of the parallel arrangement, a Bayer pattern color camera capable of selecting two color channels is used.

As another special case, for example, two detection channels and a dichroic beam splitter are used so that T ₂ = 1−T ₁ holds.
In the following, various preferred embodiments of the present invention will be described in detail with reference to the drawings.

  In the following description of the drawings, the same reference numerals are used for the same elements. The functional description regarding each element can be said also about the figure or embodiment in which these elements are not explicitly mentioned.

  FIG. 1 shows a first preferred embodiment of a wide field microscope according to the present invention. Any polychromatic illumination source 1 (eg broadband laser, halogen lamp, super luminescent diode,...), Where in this embodiment a plurality of different spectral distributions can be sorted by the sorting element 2 It has become. The sorting element 2 may be, for example, any AOTF (acousto-optic tunable filter), any prism, any grating, or any other filter sorting unit. This illumination light can be subsequently deflected in different directions by the deflection unit 3. The deflection unit 3 is an arbitrary switching unit based on, for example, an arbitrary high-speed switching mirror (for example, a galvo mirror), an arbitrary AOD (acousto-optic deflector), or a rotation of a polarization plane (Faraday rotation).

  The structured element 4 is arranged at a position conjugate with the sample surface P in the plane A. In the simplest case, this structured element 4 has some 1D or 2D lattice structure exhibiting transparency. This structure is projected here into the sample space via a plurality of elements 6, 7 (objective lenses) that induce axial chromatic aberration, leading to refractive and / or diffractive effects, here constant in the z direction Color splitting will occur, i.e. the focus will be shifted in the z direction depending on the wavelength.

  It is advantageous if a plurality of optical fibers 9 are arranged in the observation beam path. However, in another embodiment, a simple free space beam guide based on multiple mirrors may be used for this purpose. As an option, polarization filtering can be performed using these optical fibers 9.

  The deflecting unit 3 can irradiate the structured element 4 from the two sides in a sequential manner using the collimator lens 11 (indicated by broken lines). For this purpose, the structured element 4 has a reflection specification, so that two grating phases can be projected into the sample space or the sample plane P. If the structured element 4 is not of a reflective specification, neither the deflection unit 3 nor the optical system indicated by a broken line is present.

  A beam splitter 12 is used to integrate the transmitted beam path and the reflected beam path. In that case, the illumination light is further guided to the sample 14 located in the sample space P by the beam splitter 13. In this case, the beam splitter 13 is specifically used as a polarizing beam splitter. It is advantageous if Specifically, a quarter lambda plate 16 is preferably disposed in the beam path, and thereby, illumination light directed to the sample 14 and reflected or emitted from the sample 14 are detected objects. The sample beams having the planes of polarization rotated by 90 ° with respect to each other can be well separated from each other in the beam splitter 13.

  Furthermore, when such a configuration is used for the beam path, disturbing reflection not caused by the sample 14 caused by a plurality of optical elements of the detection unit 17 can be suppressed. For this purpose, a polarizing filter 18 is preferably arranged in front of the detector unit 17 in the beam path. In the case where the spectral distribution is selected only through the sorting element 2 in the irradiation path or the excitation path, the detection unit 17 may be any simple camera equipped with a corresponding imaging optical system. Furthermore, in addition to or instead of it, any one of the variant embodiments according to FIGS.

  In the arrangement described by way of example in FIG. 2, two detection channels, each containing an imaging optical system 21 and a camera 22, are used by directing the sample light through the color splitter 19 first. I try to do it.

This arrangement substantially corresponds to the special case where T2 = 1−T1 described above.
In FIG. 3, compared to FIG. 2, the color splitter is replaced by a beam splitter 23, which initially does not provide wavelength dependent filtering. In any case, this will be accomplished by filter 24 in channel I. Optionally, a filter 26 may also be placed in channel II (in which case T2 will not be stationary; if there is no filter 26, T2 will be stationary).

  FIG. 4 depicts the sequential detection scheme already described above. A plurality of filter functions are sequentially switched using the switching element 27. This switching element 27 may here be, for example, any high speed filter wheel, or any AOTF, or any suitable beam splitter arrangement with a switching mirror arrangement.

  A special arrangement similar in operation to the arrangement described in FIG. 2 or FIG. 3 is shown in FIG. Here, a so-called chip splitter 28 is introduced, for example as sold by the company Optosplit, ie the same camera chip of the camera 22 is used for two measuring steps, Thereby, these measurement steps can be performed in parallel.

  It is therefore possible to project and detect different grating phases or structures on the sample using any one device according to the diagram described above, and in so doing, the conventional structured illumination method As in, an “optical tomographic image” can be formulated, but this image is significant only if the sample surface at this point is in focus for any given pixel in xy coordinates. Will have a good signal. There, as a rule, for this case, a brave wavelength will always be found unless the surface topology significantly exceeds the measurement range associated with the color deviation in the dynamic range at that height.

  The evaluation of the image data is performed so that the wavelength that maximizes the optical tomographic image signal is determined for each pixel. From there, the function z (x, y) or surface topology can be estimated directly. This is done, for example, through the evaluation of the filter function as described above, but in the simplest case, the intensity ratio is evaluated from at least two measurement steps and the wavelength is estimated directly therefrom. It is like that. Of course, in this case, multiple measurements are significant in order to obtain a better signal-to-noise ratio. In addition, it may be significant to perform multiple measurements at different absolute heights by displacing the sample relative to the sensor. This is advantageous when the sample is markedly stained and exhibits different reflection behavior in different spectral bands.

  Similarly, HDR imaging, eg, by multiple measurements with different exposure times, is significant for some samples to limit the noise associated with each pixel, which appears as substantial shot noise. is there. Since calibrations unrelated to function g as well as function P may sometimes be insufficient, both of these functions need to be further incorporated as device features. The wavelength is then determined by applying a kind of iterative method rather than directly in some circumstances.

  If the surface topography is such that the dynamic range of its height exceeds the measurement range associated with the color deviation, the same type of surface topography may be used while varying the distance between the sensor and the sample, possibly by z-stitching. It is necessary to perform measurements and subsequently link the measurement results together.

The filter function is significantly adapted to the color deviation so that the height can be determined with similar sensitivity over the entire wavelength range.
To achieve increased sensitivity for height determination, it is sometimes beneficial to apply a filter function and perform several measurements with different filter functions both in the illumination path and in the detection path. May be.

  Considering FIG. 1, the switching of the different grating phases can also be realized by simply switching the beam path, for example by using any electro-optic modulator (EOM) or any acousto-optic. A modulator (AOD) may be introduced. In that case, in the illustrated case, for example, a phase shift occurs in the structured element 4 projected in the transmission path and the reflection path. Besides that, for example, any EOM, or any AOD, or any other galvo mirror is adapted to provide a kind of fast grating switching through direct modulation of the pupil plane of the objective lens 7. Various arrangements are also conceivable. An arbitrary grating is depicted here as a dot pattern substantially corresponding to the individual diffraction orders of the grating by means of a Fourier transform. With some phase angle modulation, for example, fast switching between different grating positions may be possible.

  In another variant, the structuring element 4, which may be formed by a 2D pinhole array, is moved to a plurality of different positions and the image is recorded using the detector unit 17 In that case, however, only one light channel of illumination is used each time. Since this detector unit 17 can be used as a digital PH-, it is possible to acquire an image that is truly confocal through the calculation and synthesis of the respective images captured at individual positions.

The wavelength-related evaluation is performed as described above.
In yet another variant, the structure 4 is completely removed and only a sharpness function indicating the relationship between sharpness and wavelength is calculated for each local image region. This can be done, for example, by M. Rahlves, J. Seewig, “Opticals Messsen technischer Oberflachen”, Beeth Verlag GmbH, Berlin, 200, Berlin. This applies to focus variations as described, and in the case of a well-structured sample, this is sufficient to obtain various height information.

  A structure that operates based on the principle of the thin layer oblique illumination (HILO) method has a similar structure. The structured element 4 also makes it possible to implement an element for introducing a speckle pattern that can be completely removed from the beam path as intended.

  FIG. 6 shows yet another embodiment of a chromatic wide-field microscope, which corresponds to a combination of aperture correlation principles with chromatic confocal technology. Here, a rotary disk 31 having a mirror structure 32 is arranged on the intermediate imaging plane Z. The sample light (detection beam path) bounced or emitted from the sample 14 is detected by two camera channels in the illustrated example. The sample light transmitted through 31 (confocal component) is captured, and the second sample 34 captures reflected sample light (a wide-field image having an out-of-focus component) emitted from the mirror structure 32. Yes. As an option, here again, a plurality of polarizing filters 18 may be arranged in the detection beam path.

  From the images of both detectors 33, 34, both a wide field image and a confocal image can be calculated depending on the wavelength. Again, from the intensity, for each detected pixel, the required height information is obtained as a function of wavelength. Of course, the acquired color image can also be used directly to depict a color image with extended depth of field information. Similarly, various configurations are possible where both channels are located only on the camera chip.

  FIG. 7 depicts yet another embodiment in which, for example, a pinhole array 41 or a nipo disk is introduced. Capture of the entire sample surface is achieved by moving (rotating, sliding) the pinhole array 41 and / or by the optional scanner unit 42. If this pinhole array 41 is an arbitrary nipou disk and is implemented as such with a plurality of structured sectors and a plurality of unstructured sectors, this embodiment To realize a special case of aperture correlation where confocality is evaluated through computation of structured or unstructured illumination images recorded in sequential or parallel fashion. Become.

An optional interferometer element 43 is optionally provided to improve measurement accuracy. This may also be present in all other embodiments.
FIGS. 8-10 show various possible variations for introducing various filter functions or different spectral distributions in the illumination beam path.

  However, FIG. 8 shows a polychromatic light source 1 that is capable of directing its light to different channels using a fast switching element 44, which is also similar to that discussed above. This is comparable to the two measurement steps. Here, different filters 46 and 47 are preferably arranged in both channels. However, basically, only one of the filters 46 and 47 is in time. In any case, beam combining is performed at the beam combiner element 48. In this case, the beam combiner element 48 is configured to be sensitive to polarization, for example, as a polarization beam splitter. Good thing. If any color splitter is introduced instead of this beam combiner element 48, the filters 46, 47 can optionally be abandoned, but this is explained in the overall effect with reference to FIG. It corresponds to the case of the sequential specification.

FIG. 9 shows a modified configuration of the present invention in which two light sources 1 of the same type are used, each having filters 46 and 47 that are switched at high speed.
FIG. 10 reproduces yet another advantageous variant configuration. Here, two different irradiation sources 51, 52 are introduced, which differ with respect to their spectral characteristics. The beam combining element 53 is now implemented as a pure beam combiner. Its spectral characteristics already provide the desired filter function. For example, the spectra of the illumination sources 51, 52 may be Gaussian waveforms that are slightly offset with respect to each other. In that case, the wavelength can be immediately derived from the intensity ratio of the two measurement steps performed by being coupled to either one of these irradiation sources 51 and 52.

01 Irradiation source 02 Sorting element 03 Deflection unit 04 Structured element 05-
06 Element that induces longitudinal chromatic aberration 07 Objective lens 08 Color division 09 Optical fiber 10 −
11 Collimator lens 12 Beam splitter 13 Beam splitter 14 Sample 15-
16 λ / 4 plate 17 Detector unit 18 Polarizing filter 19 Color splitter 20 −
21 Imaging optical system 22 Camera 23 Beam splitter 24 Filter 25-
26 Filter 27 Switching element 28 Chip splitter 29-
30-
31 Disc 32 Mirror structure 33 First detector unit 34 Second detector unit 35-
41 Pinhole array 42 Scanner unit 43 Interferometer 44 Switching element 45-
46 Filter 47 Filter 48 Beam coupling element 49 −
50-
51 Irradiation source 52 Irradiation source 53 Beam coupling element A Field of view Z Field of view

Claims (12)

A method for determining spatially resolved height information of a sample (14) using a wide field microscope, comprising:
Irradiating the sample (14) in an irradiation beam path with an arbitrary broadband irradiation source (1);
Color modulating the illumination beam path or the detection beam path;
Capturing at least one wide field image from sample light having a chromatic confocal component reflected or emitted from the sample in the detection beam path;
Determining, from the wide field image, various height information of the sample for each pixel by evaluating a relationship between a chromatic confocal component of the detection beam path and the color modulation. Capturing a first image using a first color modulation setting;
Capturing the second image using the second color modulation setting simultaneously with the capture of the first image or at different times; and
Determining the ratio of the intensity signals of both images for each pixel;
The method of claim 1, further comprising: determining a height value z (x, y) of the sample (14) for each pixel. The intensity signal is
Defined by
Here, P (x, y, λ ′) is a spectral characteristic of any one of the light source and the device,
R (x, y, λ ′) is the spectral reflectance of the sample,
T _i (x, y, λ ′) is the color modulation,
g _λmax (λ _max −λ) is a device spectral response function with λ _max as a parameter,
λ _max = λ [z (x, y)] is the maximum reflected wavelength at any point x, y according to the height function,
The method of claim 2, wherein z (x, y) is the height function of the sample. The ratio of the intensity signals of both images for each pixel is
The method according to claim 2, which is obtained based on   The method according to claim 1, wherein the color modulation of the irradiation beam path is performed by switching a filter in the irradiation beam path or in the observation beam path in a sequential manner.   6. A method according to any one of claims 2 to 5, wherein the acquisition of the first wide field image and the second wide field image is performed in two separate detection channels. A wide field microscope,
An irradiation source (1, 52, 53) disposed in the irradiation beam path;
A first detector (17, 33) that captures a wide field image in the observation beam path of the sample (14) irradiated in the sample plane (P);
A modulator for color modulation in a direction perpendicular to the sample plane (P) of the illumination beam path or the observation beam path;
A wide-field microscope comprising: an evaluation unit that calculates chromatic confocal height information in each pixel of the wide-field image.   The wide field microscope according to claim 7, further comprising a second detector of the same type as the first detector. A beam splitter is disposed between the first and second detectors;
The wide-field microscope according to claim 8, wherein a wavelength-dependent filter is arranged as a modulator in any one of the plurality of observation beam paths. A first detector in the observation beam path is placed after the chromatic confocal component capture mechanism;
The wide field microscope of claim 8, wherein the second detector is arranged to capture a defocused component in the observation beam path.   The wide-field microscope according to claim 7, wherein a switching element is disposed in the irradiation beam path.   The wide-field microscope according to any one of claims 7 to 11, wherein the color modulator is a filter.