JP2019204002A - Microscope, image analysis device, observation method, and analysis program - Google Patents

Abstract

To evaluate a correction amount of aberration.SOLUTION: A microscope includes: an imaging unit 4 for taking an image of a sample; an aberration correction unit 8 for correcting the aberration of the image of the sample; a radiation unit 3 for radiating a stripe pattern to the sample; a calculation unit 41 for calculating a contrast value of the stripe pattern on the basis of a photographic image captured by the imaging unit photographing the sample to which the stripe pattern is radiated and a spatial frequency of the stripe pattern; and an analysis unit 42 for analyzing a correction amount of aberration by an aberration correction unit on the basis of the contrast value calculated by the calculation unit.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a microscope, an image analysis device, an observation method, and an analysis program.

  The microscope forms an image of the sample and is used for observing the sample. The sample image is affected by aberrations generated by the optical system of the microscope. Therefore, a technique for correcting the aberration of the microscope has been proposed (see, for example, Patent Document 1 below). It is desirable that the microscope can evaluate the aberration correction amount so that the aberration correction amount can be appropriately set, for example.

JP 2010-243838 A

  According to the first aspect of the present invention, the imaging unit that captures the image of the sample, the aberration correction unit that corrects the aberration of the image of the sample, the irradiation unit that irradiates the sample with the fringe pattern, and the fringe pattern are irradiated. A calculation unit that calculates the contrast value of the fringe pattern based on the captured image obtained by capturing the sample by the imaging unit and the spatial frequency of the fringe pattern, and the aberration by the aberration correction unit based on the contrast value calculated by the calculation unit A microscope including an analysis unit for analyzing the correction amount.

  According to the second aspect of the present invention, the calculation unit that calculates the contrast value of the fringe pattern based on the captured image obtained by capturing the image of the sample irradiated with the fringe pattern and the spatial frequency of the fringe pattern; An image analysis apparatus is provided that includes an analysis unit that analyzes the correction amount of the aberration of the sample image based on the contrast value calculated by the unit.

  According to the third aspect of the present invention, an image of the sample is captured, the aberration of the sample image is corrected, the sample is irradiated with the fringe pattern, and the sample irradiated with the fringe pattern is imaged. The contrast value of the fringe pattern is calculated based on the captured image and the spatial frequency of the fringe pattern, the correction amount of the aberration is analyzed based on the contrast value, and the correction amount of the aberration is calculated based on the analysis. An observation method is provided.

  According to the fourth aspect of the present invention, the computer calculates the contrast value of the fringe pattern based on the captured image obtained by capturing the image of the sample irradiated with the fringe pattern and the spatial frequency of the fringe pattern. And an analysis program for executing the analysis of the correction amount of the aberration of the image of the sample based on the contrast value.

It is a figure which shows the microscope which concerns on 1st Embodiment. It is a figure which shows the fringe pattern which concerns on 1st Embodiment, and the process by an image process part. It is a figure which shows the process of the calculation part which concerns on 1st Embodiment. It is a figure which shows the process of the analysis part which concerns on 1st Embodiment. It is a flowchart which shows the observation method which concerns on 1st Embodiment. It is a figure which shows the microscope which concerns on 2nd Embodiment. It is a figure which shows the process by the fringe pattern which concerns on 2nd Embodiment, and an image process part. It is a figure which shows the process by the fringe pattern which concerns on 2nd Embodiment, and an image process part. It is a figure which shows the process by the fringe pattern which concerns on 2nd Embodiment, and an image process part. It is a figure which shows the process by the fringe pattern which concerns on 2nd Embodiment, and an image process part.

[First Embodiment]
Next, embodiments will be described with reference to the drawings. FIG. 1 is a diagram illustrating a microscope 1 according to the first embodiment. The microscope 1 includes a stage 2, an irradiation unit 3, an imaging unit 4 (detection unit), a control device 5, a display device 6 (display unit), and a storage device 7 (storage unit). The microscope 1 is a fluorescent microscope, for example, and is used for observing a sample X including cells that have been fluorescently stained in advance.

  The microscope 1 may be an optical microscope other than a fluorescence microscope. The microscope 1 may be a microscope (for example, a Raman microscope) that uses incoherent interaction between light and a sample. The microscope 1 may be a microscope that can be used for both observation using fluorescence and observation using no fluorescence. In the following description, it is assumed that the microscope 1 is an epi-illumination microscope that uses epi-illumination. The microscope 1 according to the embodiment may be a microscope (eg, transmission microscope) that detects light (eg, fluorescence, Raman scattered light) emitted from the sample X on the opposite side of the illumination light incident side with respect to the sample X. .

  In general, an image of a sample acquired by a microscope is affected by aberration (eg, Seidel aberration) of an optical system (eg, objective lens, illumination optical system, etc.). The aberration includes, for example, at least one of spherical aberration, coma, astigmatism, curvature of field, and distortion. The microscope 1 according to the embodiment includes an aberration correction unit 8 that corrects the aberration of the image of the sample X. The microscope 1 also includes an image processing unit 9 that analyzes the correction amount of aberration. For example, the image processing unit 9 outputs an optimal value or a recommended value of the correction amount of aberration as an analysis result. The user can observe the image of the sample X in a state where the aberration is appropriately corrected using the analysis result of the correction amount of the aberration. Hereinafter, each part of the microscope 1 will be described.

  Stage 2 holds sample X. The stage 2 includes, for example, an XY stage or an XYZ stage. The stage 2 is movable with respect to the objective lens 28 (described later) while holding the sample X. The stage 2 may not move with respect to the objective lens 28 (it may be fixed).

  The irradiation unit 3 (illumination unit, illumination device) irradiates the sample X placed on the stage 2 with illumination light. The irradiation unit 3 can vary the spatial distribution of the light intensity of the illumination light in the sample X. When observing the sample X, the irradiating unit 3 irradiates the sample X with illumination light having a substantially uniform spatial distribution of light intensity. Further, when analyzing the correction amount of aberration, the irradiation unit 3 irradiates the sample X with illumination light in which the spatial distribution of light intensity is set to a predetermined pattern (predetermined distribution). The predetermined pattern is set, for example, to a stripe pattern (shown later in FIG. 2) in which bright portions and dark portions are arranged periodically (repeatedly). In the microscope 1 according to the present embodiment, the irradiation unit 3 irradiates illumination light that forms a stripe pattern on the sample X, and light (eg, fluorescence) emitted from the sample X and passing (transmitting) through an objective lens described later. Light is received by the imaging unit 4 and a detection result (a captured image, a captured result, a received light result, a received light signal) is obtained.

  The irradiation unit 3 includes a light source LS and an illumination optical system 11. The light source LS includes a solid light source such as a laser diode (LD) or a light emitting diode (LED). In the following description, light emitted from the light source LS is referred to as illumination light as appropriate. When performing fluorescence observation with the microscope 1, the wavelength of the illumination light is set to a wavelength band that includes the excitation wavelength of the fluorescent substance contained in the sample X.

  Note that the irradiation unit 3 may not include at least a part of the light source LS. For example, the microscope 1 may be provided without the light source LS. The light source LS may be attached to the microscope 1 when the microscope 1 is used. The light source LS may be replaceable (attachable or removable) with respect to the irradiation unit 3.

  The illumination optical system 11 irradiates the sample X with illumination light from the light source LS. The illumination optical system 11 includes, for example, a rotationally symmetric optical member (eg, a lens member) such as a spherical lens or an aspheric lens. In the following description, the rotationally symmetric axis of the optical member included in the illumination optical system 11 is referred to as the optical axis 11a of the illumination optical system 11 as appropriate. The illumination optical system 11 may include a free-form surface lens.

  The illumination optical system 11 includes a lens 12, a light guide member 13, a lens 14, and a branching unit 15 in order from the light source LS toward the sample X. The light guide member 13 is an optical fiber, for example. The lens 12 condenses the illumination light from the light source LS on the light incident side end face of the light guide member 13. The light guide member 13 guides the illumination light from the lens 12 to the lens 14. The lens 14 is a collimator, for example, and converts the illumination light emitted from the light guide member 13 into parallel light.

  The branching unit 15 is used for generating a stripe pattern. The branching unit 15 includes a diffraction grating, for example, and branches the illumination light into a plurality of diffracted lights. The illumination optical system 11 divides the illumination light into a plurality of diffracted lights by the branching unit 15 and illuminates the sample X with a fringe pattern (interference fringes) formed by interference of the plurality of diffracted lights. For example, the illumination optical system 11 forms a fringe pattern by interference of two light beams (two-beam interference) among a plurality of diffracted lights. FIG. 1 shows 0th-order diffracted light (shown by a solid line), + 1st-order diffracted light (shown by a broken line), and -1st-order diffracted light (shown by a two-dot chain line) among a plurality of light beams. In the following description, when referring to one or both of the + 1st order diffracted light and the −1st order diffracted light among the diffracted lights, they are simply expressed as 1st order diffracted light.

  The branching unit 15 has, for example, a one-dimensional periodic structure in a plane intersecting the optical axis 11 a of the illumination optical system 11. This periodic structure may be a structure in which the density (transmittance) changes periodically or a structure in which the step (phase difference) changes periodically. When the branching portion 15 is a phase type, the diffraction efficiency of ± first-order diffracted light is high, and the loss of light quantity can be reduced when ± 1st-order diffracted light is used to form a fringe pattern. The branching unit 15 may use, for example, a spatial light modulator (SLM) instead of the diffraction grating (described later in FIG. 6).

  The branching portion 15 is arranged at a position optically conjugate with the surface to be observed in the sample X (for example, a visual field conjugate position). The branching unit 15 may be arranged at a position shifted from the visual field conjugate position (in the vicinity of the visual field conjugate position) within a range in which the contrast value of the fringe pattern is allowed. The branch part 15 can be inserted into and retracted from the optical path of the illumination light. At the time of analyzing the correction amount of the aberration, the branching unit 15 is inserted (arranged) in the optical path of the illumination light and branches the illumination light into a plurality of light beams. Further, when observing the sample X, the branching portion 15 is disposed at a position retracted from the optical path of the illumination light. The branching unit 15 may be inserted into or retracted from the optical path of the illumination light by a driving unit such as an actuator, or may be inserted or retracted manually by the user.

  The illumination optical system 11 includes a lens 21, a mask 22, a lens 23, an illumination field stop 24, a lens 25, a filter 26, a dichroic mirror 27, and an objective lens 28 in order from the branching unit 15 toward the sample X.

  The illumination light branched by the branching unit 15 enters the lens 21. For example, the lens 21 is arranged so that the focal plane on the same side as the light source LS coincides with the branching section 15. For example, the lens 21 is disposed so that the focal plane on the same side as the sample X coincides with the pupil conjugate plane P1. The pupil conjugate plane P1 is an optically conjugate plane with the pupil plane P0 of the objective lens 28.

  The mask 22 transmits diffracted light used for forming the stripe pattern and blocks diffracted light not used for forming the stripe pattern. In this embodiment, the diffracted light used for forming the fringe pattern is the first order diffracted light (+ 1st order diffracted light, −1st order diffracted light), and the diffracted light not used for forming the fringe pattern is the 0th order diffracted light and the second or higher order. Diffracted light. The mask 22 allows the 1st-order diffracted light to pass through and blocks the 0th-order diffracted light and the second-order or higher-order diffracted light. The mask 22 is disposed at a position where the optical path of the first-order diffracted light is separated from the optical paths of other diffracted lights, for example, at the pupil conjugate plane P1. As described above, in the mask 22, the portion where the 0th-order diffracted light is incident is a light shielding portion (non-transmissive portion) that blocks the illumination light, and the portion where the first-order diffracted light is incident is an opening (transmitting portion) through which the diffracted illumination light passes. It is.

  The mask 22 can be inserted into and retracted from the optical path of the illumination light. At the time of analyzing the correction amount of the aberration, the mask 22 is inserted (arranged) in the optical path of the illumination light to block diffracted light that is not used for forming the fringe pattern. Further, when observing the sample X, the mask 22 is retracted from the optical path of the illumination light. The mask 22 may be inserted into the optical path of the illumination light by a driving unit such as an actuator, or may be retracted by a driving unit such as an actuator. The mask 22 may be inserted manually by the user with respect to the optical path of the illumination light, or may be retracted manually by the user. The mask 22 may be a shutter that changes the position of the light-shielding portion or the amount of transmitted light when the sample X is observed, and may not be retracted from the optical path of the illumination light. Further, the microscope 1 may not include the mask 22.

  The illumination light that has passed through the mask 22 enters the lens 23. The lens 23 is disposed so that the focal plane 23a opposite to the light source LS is optically conjugate with the branching portion 15. The illumination field stop 24 is disposed at or near the focal plane 23a. The illumination field stop 24 defines a range (illumination field, illumination area) in which illumination light is irradiated from the illumination optical system 11 to the sample X in a plane perpendicular to the optical axis 11 a of the illumination optical system 11. Note that the microscope 1 may not include the illumination field stop 24. The illumination light that has passed through the illumination field stop 24 enters the lens 25. The lens 25 is arranged such that the focal plane on the same side as the sample X is positioned at the pupil plane P0.

  The illumination light that has passed through the lens 25 enters the filter 26. The filter 26 is an excitation filter, for example, and has an optical characteristic that light in a wavelength band including the excitation wavelength of the fluorescent material included in the sample X selectively passes. For example, the filter 26 blocks at least a part of the illumination light other than the excitation wavelength, stray light, external light, and the like. The light that has passed through the filter 26 enters the dichroic mirror 27. The dichroic mirror 27 has an optical characteristic in which light in a wavelength band including the excitation wavelength of the fluorescent substance contained in the sample X is reflected, and light (for example, fluorescence) in a predetermined wavelength band out of the light from the sample X passes. The light from the filter 26 is reflected by the dichroic mirror 27 and enters the objective lens 28.

  The focal plane 28a of the objective lens 28 corresponds to the surface to be observed (for example, the object plane). The focal plane 28 a is a focal plane opposite to the light source LS with respect to the objective lens 28. The focal plane 28a is disposed at a position optically conjugate with the branch portion 15. The irradiation unit 3 irradiates the sample X with a fringe pattern via the objective lens 28. A stripe pattern (shown later in FIG. 2) corresponding to the periodic structure of the branching portion 15 is formed on the focal plane 28a. When observing the sample X, a part of the sample X is disposed on the focal plane 28a. The fringe pattern formed on the sample X has a periodic distribution of light intensity in a direction perpendicular to the optical axis 11 a of the illumination optical system 11.

  The imaging unit 4 captures an image of the sample X via the objective lens 28. The imaging unit 4 includes an imaging optical system 31 and an imaging element 32. The imaging optical system 31 includes an objective lens 28, a dichroic mirror 27, a filter 33, and a lens 34 in order from the sample X toward the image sensor 32. The objective lens 28 and the dichroic mirror 27 are shared by the illumination optical system 11 and the imaging optical system 31. Light emitted from the sample X by irradiation of illumination light (hereinafter referred to as observation light) enters the objective lens 28 and is converted into parallel light, and enters the filter 33 through the dichroic mirror 27.

  The filter 33 is, for example, a fluorescent filter. The filter 33 has a characteristic that light in a predetermined wavelength band (eg, fluorescence) of observation light from the sample X selectively passes. In this case, for example, the filter 33 blocks illumination light, external light, stray light, and the like reflected from the sample X. The light that has passed through the filter 33 enters the lens 34. The lens 34 is arranged so that the focal plane on the same side as the sample X coincides with the pupil plane P 0 of the objective lens 28. The imaging optical system 31 forms a surface (image plane) optically conjugate with the focal plane 28a (object plane) of the objective lens 28.

  The image sensor 32 includes a two-dimensional image sensor such as a CCD image sensor or a CMOS image sensor. The imaging element 32 has, for example, a structure having a plurality of pixels arranged two-dimensionally and a photoelectric conversion element such as a photodiode disposed in each pixel. Hereinafter, the surface on which the photoelectric conversion element is arranged in the image sensor 32 is appropriately referred to as a light receiving surface. The image sensor 32 is arranged so that the light receiving surface is optically conjugate with the focal plane 28a of the objective lens 28. For example, the imaging element 32 reads out the electric charge generated by the irradiation of the observation light to the photoelectric conversion element by the reading circuit. The image sensor 32 converts the read charges into digital data (for example, 8-bit gradation value), and outputs digital image data in which pixel positions and gradation values are associated with each other. The imaging element 32 outputs captured image data to the control device 5 as an imaging result.

  The aberration correction unit 8 corrects the aberration of the sample image. The aberration correction unit 8 corrects the aberration of the image of the sample X formed on the light receiving surface of the image sensor 32. The aberration correction unit 8 is provided in the objective lens 28, for example. The aberration correction unit 8 includes an optical member (eg, a lens member) that can move along an optical path (eg, an optical axis) in the objective lens 28. The aberration correction unit 8 corrects the aberration by moving the optical member included in the objective lens 28 along the optical path (for example, the optical axis 11a) in the objective lens 28.

  The correction amount of the aberration by the aberration correction unit 8 may be expressed based on the movement amount of the optical member. The aberration correction unit 8 includes an operation member that can be operated by the user, such as a correction ring. The aberration correction amount by the aberration correction unit 8 may be represented by the movement amount of the operation member. When the correction member is a correction ring, a mark such as a scale indicating the movement amount (rotation angle) of the correction ring is formed, and the user can set the rotation angle obtained from the scale or the like as an aberration correction amount (aberration correction level). Can be used as The aberration correction unit 8 may be configured to move the optical member by a driving unit such as an actuator. The aberration correction unit 8 may be configured to acquire the movement amount of the optical member with an encoder or the like.

  The aberration correction unit 8 may include a wavefront adjustment unit that adjusts the wavefront of light emitted from the sample. The wavefront adjusting unit is disposed at or near the pupil plane P0 of the objective lens 28, or at a position optically conjugate with the pupil plane P0. When the aberration correction unit 8 includes a wavefront adjustment unit, the correction amount of the aberration may be expressed using, for example, a Zernike polynomial coefficient based on the adjustment amount of the wavefront by the wavefront adjustment unit. The wavefront adjustment unit includes, for example, a spatial light modulator (SLM) or a deformable mirror.

  The spatial light modulator includes, for example, a liquid crystal element using ferroelectric liquid crystal, and can control the refractive index for each region (for example, pixel) where light enters. When the aberration correction unit 8 includes a spatial light modulator as the wavefront adjustment unit, the aberration correction unit 8 adjusts the wavefront by adjusting the refractive index of each region in the spatial light modulator. The deformable mirror includes, for example, an array of reflective elements (micromirrors) such as a MEMS mirror array, and the surface of the deformable mirror (e.g., reflective element) is inclined for each area (e.g., reflective element). The reflection angle and the direction of the reflection element can be controlled. When the aberration correction unit 8 includes a deformable mirror as the wavefront adjustment unit, the aberration correction unit 8 adjusts the wavefront by controlling the inclination of the surface of each region of the deformable mirror.

  As a method for evaluating the correction amount of aberration, for example, a method for evaluating the contrast value of a desired structure in the image of the sample X can be considered. In this case, the evaluation value is influenced by the structure of the sample X, and it is difficult to evaluate the aberration correction amount with high accuracy because the structure of the sample X varies depending on the position. In the present embodiment, the microscope 1 illuminates the sample X with a fringe pattern, and analyzes the correction amount of the aberration based on the contrast value of the fringe pattern. In this case, the influence of the structure of the sample X on the contrast value is reduced, and the aberration correction amount can be analyzed and evaluated with high accuracy.

  The image processing unit 9 includes a calculation unit 41 and an analysis unit 42. In FIG. 1, the image processing unit 9 (image processing device) is provided in the control device 5, but may be provided in a device different from the control device 5. The calculation unit 41 calculates the contrast value of the fringe pattern based on the captured image (imaging result, detection result) obtained by imaging the sample X irradiated with the fringe pattern and the spatial frequency of the fringe pattern. For example, the image processing unit 9 performs calculation using the captured image and the spatial frequency of the stripe pattern, and calculates the contrast value of the stripe pattern. The analysis unit 42 analyzes and calculates the correction amount of the aberration by the aberration correction unit 8 based on the contrast value calculated by the calculation unit 41. Hereinafter, each unit of the image processing unit 9 will be described.

  FIG. 2 is a diagram illustrating processing of the calculation unit according to the present embodiment. In FIG. 2, symbol Im is a captured image in which the imaging unit 4 images the sample X irradiated with the stripe pattern LP. In the captured image Im, the symbol Dx is the horizontal scanning direction, and the symbol Dy is the vertical scanning direction. The stripe pattern LP is a pattern in which a first portion (eg, bright portion LPa) and a second portion (eg, dark portion LPb) having lower luminance than the first portion are repeatedly arranged in a predetermined direction. In the following description, a direction parallel to a line connecting the same luminance positions in the stripe pattern LP is referred to as a line direction, and a direction perpendicular to the line direction is referred to as a periodic direction.

  The calculation unit 41 calculates the contrast value of the stripe pattern LP based on the spatial frequency of the stripe pattern LP. The calculation unit 41 calculates a contrast value based on the frequency spectrum obtained by Fourier transform of the captured image Im and the spatial frequency of the fringe pattern LP. The frequency spectrum (spatial frequency spectrum) represents the distribution of the intensity of frequency components with respect to the frequency. The frequency spectrum is an amplitude spectrum. In the amplitude spectrum, the intensity of each frequency component is represented by the amplitude of each frequency component. The amplitude spectrum is obtained by calculating the square root of the power of each frequency component using a power spectrum obtained by performing two-dimensional fast Fourier transform on the captured image Im. The power spectrum may be used as the frequency spectrum. In this case, the intensity of each frequency component is represented by power (the square of the amplitude of the frequency component). The calculation unit calculates a contrast value based on the intensity of the frequency component in a region corresponding to the spatial frequency of the fringe pattern in the frequency spectrum.

  The calculation unit 41 selects a region corresponding to the fringe pattern LP in the frequency spectrum based on the spatial frequency of the fringe pattern LP. The intensity of the frequency component corresponding to the fringe pattern LP corresponds to the luminance value of the fringe pattern LP in the captured image Im. The calculating unit 41 calculates the contrast value of the fringe pattern LP based on the intensity of the frequency component in the region selected as the region corresponding to the fringe pattern LP.

  In FIG. 2, a symbol Ti is a converted image obtained by Fourier transform of the captured image Im. A symbol Di is a first axis direction representing a spatial frequency in the horizontal scanning direction Dx. A symbol Dj is a second axis direction representing a spatial frequency in the vertical scanning direction Dy. The converted image Ti in FIG. 2 is a grayscale image, and the intensity of the frequency component with respect to the spatial frequency is represented by the luminance value (eg, gradation value, pixel value) of the pixel at the position corresponding to the spatial frequency. In the converted image Ti, a relatively bright (white) region is a region having a relatively high frequency component intensity, and a relatively dark (black) region is a region having a relatively low frequency component intensity. The converted image Ti can be used as information representing a frequency spectrum. In the following description, the calculation unit 41 appropriately uses the converted image Ti as the frequency spectrum, and uses the luminance value of each pixel of the converted image Ti as the intensity of the frequency component.

  A symbol TX in the converted image Ti is an area including the origin T0 of the frequency space and the periphery of the origin T0. The origin T0 is a point where the coordinate in the first axis direction Di is 0 and the coordinate in the second axis direction Dj is 0, and corresponds to a DC component. Further, the reference symbol TL in the converted image Ti indicates a region corresponding to the stripe pattern LP. The region TL includes a region TLa and a region TLb. The region TLb is arranged at a position symmetrical to the region TLa with respect to the origin T0.

  The code Q1 and the code Q2 in the converted image Ti are positions corresponding to the spatial frequency of the fringe pattern LP. The position Q2 is a point symmetric with respect to the position Q1 with respect to the origin T0. Here, the spatial frequency of the stripe pattern LP in the horizontal scanning direction Dx of the captured image Im is represented by fx. The spatial frequency of the stripe pattern LP in the vertical scanning direction Dy of the captured image Im is represented by fy. The position Q1 is represented by coordinates (fx, fy) in a frequency space with the first axis direction Di and the second axis direction Dj as axes. The position Q2 is represented by coordinates (−fx, −fy) in the frequency space.

  The region TLa is a region including the position Q1 and the periphery of the position Q1. The region TLa is a single pixel in which the position Q1 is arranged in the converted image Ti, or a plurality of pixels including a pixel in which the position Q1 is arranged in the converted image Ti and surrounding pixels. The region TLb is a region including the position Q2 and the periphery of the position Q2. The region TLb is a single pixel in which the position Q2 is arranged in the converted image Ti, or a plurality of pixels including a pixel in which the position Q2 is arranged in the converted image Ti and surrounding pixels.

  FIG. 3 is a diagram illustrating processing of the calculation unit according to the first embodiment. A symbol LQ is a line connecting the region TLa and the region TLb corresponding to the stripe pattern LP in the converted image Ti. The symbol PS is a frequency spectrum on the line LQ (frequency component intensity distribution with respect to frequency). In the frequency spectrum PS, the horizontal axis represents the position on the line LQ, and the vertical axis represents the intensity of the frequency component (eg, the amplitude of the frequency component). The intensity of the frequency component of the frequency spectrum PS corresponds to the distribution of the luminance values of the pixels on the line LQ. The frequency spectrum PS has a symmetrical distribution with respect to the origin, and FIG. 3 shows a portion on one side with respect to the origin.

  The calculation unit 41 sets a region TLa and a region TLb corresponding to the fringe pattern LP based on positions (eg, position Q1, position Q2) in the frequency spectrum PS corresponding to the spatial frequency of the fringe pattern LP. The calculating unit 41 calculates the integrated value of the intensity of the frequency component as a contrast value for one or both of the region TLa and the region TLb. For example, when the frequency spectrum PS is represented by the converted image Ti as illustrated in FIG. 2, the calculation unit 41 calculates the sum of the luminance values of the pixels included in the region TL in the converted image Ti as a contrast value.

  The calculation unit 41 uses a set value as the spatial frequency of the stripe pattern LP. The set value of the spatial frequency of the fringe pattern LP is a value determined by the configuration of the irradiation unit 3. For example, when the branching section 15 is a diffraction grating, the set value is determined by the period of the periodic structure of the branching section 15, the magnification of the illumination optical system 11, the order of the diffracted light that generates the fringe pattern LP, and the like. The spatial frequency of the fringe pattern LP can be obtained in advance based on the configuration of the irradiation unit 3. The set value of the spatial frequency of the fringe pattern LP is stored in the storage device 7 in advance. The calculation unit 41 reads the set value of the spatial frequency of the stripe pattern LP from the storage device 7 and sets the region TL used for calculating the contrast value.

  The calculation unit 41 may use a measurement value measured from the captured image Im as the spatial frequency of the stripe pattern LP. For example, the calculation unit 41 searches for a region excluding the region TX in the frequency spectrum PS (transformed image Ti), and at a position where the intensity (eg, luminance value) of the frequency component is equal to or greater than a predetermined value (eg, maximum value). Coordinates may be derived, and the coordinates may be used as a measurement value of the spatial frequency of the fringe pattern LP. Further, the calculation unit 41 searches the frequency spectrum PS (transformed image Ti) for a position corresponding to the set value of the spatial frequency of the fringe pattern LP and its surroundings, and the intensity (eg, luminance value) of the frequency component is a predetermined value. The coordinates of the position that is the above (for example, the maximum value) may be derived, and this coordinate may be used as a measurement value of the spatial frequency of the fringe pattern LP.

  The size of the region TL (for example, the radius of the region TLa, the radius of the region TLb) is set in advance, for example. For example, the region TLa may be one pixel in which the position Q1 corresponding to the spatial frequency of the fringe pattern LP is arranged in the converted image Ti, or a plurality of pixels centered on this pixel (eg, 3 rows and 3 columns). Pixel matrix).

  Note that the size of the region TL (eg, the radius of the region TLa, the radius of the region TLb) may be set using the captured image Im. The calculation unit 41 includes a position Q1 corresponding to the spatial frequency of the fringe pattern LP in the frequency spectrum PS (eg, a position where the frequency component intensity is maximum) and a position corresponding to the DC component (eg, the intensity of the frequency component) in the frequency spectrum PS. The size of the region TLa may be set on the basis of the local maximum position and the origin T0). For example, the calculation unit 41 sets the radius R1 of the region TLa to a value that is a predetermined ratio (eg, 10%, 20%) with respect to the distance DL between the position Q1 and the origin T0, so that the region TLa You may set the size.

  Further, the calculation unit 41 may set the size of the region TLa based on the intensity of the frequency component corresponding to the spatial frequency (position Q1) of the fringe pattern LP in the frequency spectrum PS. In FIG. 3, reference symbol PW <b> 1 is the intensity (e.g., local maximum value) of the frequency component corresponding to the spatial frequency (e.g., measured value, set value) of the fringe pattern LP. The symbol PW2 is a frequency component intensity value (hereinafter referred to as a reference value) having a predetermined ratio with respect to the maximum value PW1. The predetermined ratio is an arbitrary value (eg, 10%, 50%, 90%) set in advance. The calculation unit 41 may set the size (eg, radius R1) of the region TLa by setting a region (range) in which the intensity of the frequency component is equal to or greater than the reference value PW2 in the region TLa. .

  The analysis unit 42 analyzes the aberration correction amount by calculating an evaluation value of the aberration correction amount by the aberration correction unit 8 based on the contrast value calculated by the calculation unit 41. The analysis unit 42 calculates an evaluation value indicating a level (degree) at which the aberration is reduced by calculation, and analyzes the correction amount of the aberration using the evaluation value. First, the analysis unit 42 calculates an evaluation value using the contrast value calculated by the calculation unit 41. Hereinafter, the evaluation value will be described. The frequency spectrum of the captured image Im (hereinafter referred to as the captured image spectrum) is represented by the following formula (1).

In Expression (1), k is a spatial frequency and I (k) is a captured image spectrum. OTF (k) is an optical transfer function, and O (k) is an object spectrum based on the structure of the sample X. c and c ^* are fringe contrasts and have a complex conjugate relationship with each other. k ₀ is the spatial frequency of the fringe pattern. The region TX shown in FIGS. 2 and 3 is a region where the value of O (k) is relatively large (region where O (k) is dominant) among the three terms on the right side of Equation (1). . The region TLa is a region where the value of O (k−k ₀ ) is relatively large among the three terms on the right side of the equation (1) (region where O (k−k ₀ ) is dominant). In the captured image spectrum, the intensity of the frequency component having a spatial frequency of k ₀ is expressed by the following equation (2) by setting k = k ₀ in equation (1).

The analysis unit 42 calculates a value obtained by normalizing the contrast value as the evaluation value. Here, the reference value of the intensity of the frequency component in the captured image spectrum is defined as I (0), and the evaluation value (| I (k ₀ ) / I (0) |) is defined by the following equation (3). | I (k ₀ ) / I (0) | is the ratio of I (0), which is a DC component in the frequency spectrum, to I (k ₀ ), which is a spatial frequency component of the fringe pattern in the frequency spectrum. | I (k ₀ ) / I (0) | is the absolute value of the ratio between the amplitude of the direct current component and the amplitude of the frequency component having the frequency k ₀ .

O (0) is a direct current component of the object spectrum, and its absolute value is sufficiently larger than O (k ₀ ) and O (2k ₀ ), which are other frequency components of the object spectrum. The relationship 4) holds.

Using equation (4) and OTF (0) = 1, the evaluation value (| I (k ₀ ) / I (0) |) of equation (3) is approximated as equation (5) below: it can. Since the evaluation value approximated in this way does not include the term of the object spectrum, it is not affected by the object spectrum when the aberration correction amount is changed. Since | OTF (k ₀ ) | is larger as the aberration is smaller, the aberration is minimized under the condition that the evaluation value is maximized.

  FIG. 4 is a diagram illustrating processing of the analysis unit according to the present embodiment. The imaging unit 4 (see FIG. 1) executes imaging under a plurality of conditions (for example, conditions that vary depending on the aberration to be corrected) having different aberration correction amounts, and acquires a plurality of captured images. Further, the calculation unit 41 calculates a contrast value for each of the plurality of conditions. The calculation unit 41 calculates the contrast value for each of the plurality of captured images, thereby calculating the contrast value under a plurality of conditions with different aberration correction amounts. The analysis unit 42 calculates an evaluation value for each of a plurality of conditions having different aberration correction amounts by calculating the evaluation value for each of the contrast values calculated by the calculation unit 41.

  The analysis unit 42 determines whether the evaluation value is an extreme value (for example, a local maximum value, a minimum value, or the like) based on the relationship between the evaluation value under a plurality of conditions with different aberration correction amounts and the aberration correction amount. The correction amount of the aberration that is a neighborhood value is derived. In the present embodiment, the analysis unit 42 sets the aberration correction amount (indicated by reference sign CA in FIG. 4) at which the evaluation value is a maximum in the evaluation value for the aberration correction amount as a target value (eg, optimum value, recommended value). ). For example, the analysis unit 42 obtains a fitting curve with respect to a plurality of data points using data obtained by combining the aberration correction amount and the evaluation value as a data point, and calculates a correction amount target value from the fitting curve. . The analysis unit 42 may set the aberration correction amount of the data point having the maximum evaluation value among the plurality of data points as the target value.

  The analysis unit 42 outputs the calculated target value of the correction amount of aberration to, for example, the outside of the image processing unit 9. For example, the user can perform observation while the aberration is appropriately corrected by setting the correction amount of the aberration by the aberration correction unit 8 to the target value of the correction amount of the aberration calculated by the analysis unit 42. . The microscope 1 may automatically set the aberration correction amount by the aberration correction unit 8 to the aberration correction amount calculated by the analysis unit 42. For example, the control unit 43 may set the aberration correction amount by the aberration correction unit 8 to a target value by controlling the aberration correction unit 8 according to an input instruction from the outside or automatically.

  The control unit 43 controls each unit of the microscope 1. The control unit 43 controls the irradiation unit 3 to irradiate the sample X with illumination light. The control unit 43 controls the imaging unit 4 to image the sample X. The control unit 43 controls the aberration correction unit 8 to set (eg, adjust) the correction amount of the aberration. The control unit 43 controls the image processing unit 9 to analyze the correction amount of the aberration.

  Next, the observation method according to the embodiment will be described based on the configuration of the microscope 1 described above. As for the configuration of the microscope 1, FIG. In the observation method according to the embodiment, the aberration correction amount is analyzed prior to the observation of the sample X, and the sample X is imaged by the imaging unit 4 in a state where the aberration correction amount is set based on the analysis result. Get a statue of. FIG. 5 is a flowchart showing an observation method (image analysis method) according to the embodiment.

  In this embodiment, the microscope 1 images the sample X irradiated with the fringe pattern LP under a plurality of conditions with different aberration correction amounts, and analyzes the aberration correction amount. The plurality of conditions are set in advance. Information representing a plurality of conditions (eg, first correction amount, second correction amount,...) Is stored in the storage device 7 in advance.

  In step S <b> 1, the irradiation unit 3 irradiates the sample X with the fringe pattern LP via the objective lens 28. In step S2, the aberration correction unit 8 sets an aberration correction amount. The aberration correction unit 8 sets the aberration correction amount to a preset value (eg, the first correction amount, the initial setting value). For example, the aberration correction unit 8 sets the correction amount of the aberration by manually operating the correction ring by the user. For example, the image processing unit 9 acquires information indicating the first correction amount from the storage device 7 and causes the display device to display an image indicating the first correction amount. The user operates the aberration correction unit 8 (for example, a correction ring) so that the correction amount of the aberration by the aberration correction unit 8 becomes the first correction amount shown in the image displayed on the display device. The aberration correction unit 8 may automatically set the aberration correction amount by being controlled by the control unit 43. In step S <b> 3, the imaging unit 4 images the sample X irradiated with the stripe pattern LP via the objective lens 28.

  In step S4, the image processing unit 9 (eg, the analysis unit 42) determines whether or not to change the aberration correction amount. The image processing unit 9 determines the aberration correction amount when the imaging in step S3 is not completed for a part of the plurality of scheduled conditions (eg, first correction amount, second correction amount,...). Determine to change. When the image processing unit 9 determines that the aberration correction amount is changed (step S4; Yes), the microscope 1 repeats the processing from step S2 to step S4. In step S2, the aberration correction unit 8 sets the aberration correction amount to the following value (eg, second correction amount). In step S <b> 3, the imaging unit 4 images the sample X irradiated with the fringe pattern LP through the objective lens 28 with the aberration correction amount set to the following value (for example, the second correction amount). To do.

  In step S4, the image processing unit 9 determines the aberration when the imaging in step S3 is completed for all of a plurality of scheduled conditions (eg, first correction amount, second correction amount,...). It is determined that the correction amount is not changed. When the image processing unit 9 determines that the aberration correction amount is not changed (step S4; No), the irradiation unit 3 stops the irradiation of the fringe pattern LP started in step S1. In step S5, the calculation unit 41 calculates the contrast value of the fringe pattern LP based on the captured image Im and the spatial frequency of the fringe pattern. The calculation unit 41 calculates the contrast value of the stripe pattern LP for each of the plurality of conditions.

  In step S6, the analysis unit 42 calculates an evaluation value based on the contrast value calculated in step S5. The analysis unit 42 calculates a contrast value for each of the plurality of conditions. In step S7, the analysis unit 42 derives the relationship between the aberration correction amount and the evaluation value (see FIG. 4). For example, the analysis unit 42 derives a function (eg, evaluation function) indicating the relationship between the aberration correction amount and the evaluation value. The function may be expressed by a mathematical expression or a table format. The analysis unit 42 derives the function by deriving a coefficient of a mathematical expression that approximates the relationship between the evaluation value and the aberration correction amount. In step S8, the analysis unit 42 derives an aberration correction amount at which the evaluation value becomes an extreme value. For example, the analysis unit 42 calculates, as a target value of the aberration correction amount, the aberration correction amount corresponding to the point where the slope of the tangent is 0 in the above function.

  Note that the image processing unit 9 may determine the conditions after this time based on one or both of the captured image captured under the current condition and the captured image captured under the previous condition. For example, the image processing unit 9 obtains a contrast value obtained from a captured image corresponding to three past conditions (e.g., the previous two conditions, the previous condition, and the current condition) or a value calculated from the contrast value (e.g., A candidate for the next condition may be calculated by a golden section method using an evaluation value. In this case, when the next condition candidate satisfies a predetermined condition, the image processing unit 9 determines in step S4 that the aberration correction amount is to be changed, and determines the next condition candidate as the next condition (for example, Adopt). The predetermined condition may be, for example, a condition in which a difference between a next condition candidate (eg, next correction amount candidate) and a current condition (eg, current correction amount) is equal to or greater than a threshold value.

  Based on the aberration correction amount derived as described above, the sample is observed. In step S9, the aberration correction unit 8 sets the aberration correction amount to the value derived in step S8. In the process of step S <b> 9, the correction amount may be set manually by a user operation, or the correction amount may be automatically set by the control of the control unit 43, as in the process of step S <b> 2. In step S10, the imaging unit 4 images the sample to be observed. The imaging unit 4 images the sample X in a state where the aberration correction amount is set to the value derived in step S8.

  The captured image captured by the imaging unit 4 in step S10 is used for observation of the sample X. In step S10, the irradiation unit 3 may illuminate the sample X with illumination light with uniform illuminance. Further, when the microscope 1 is a microscope that uses structured illumination light such as a structured illumination microscope (SIM), the irradiation unit 3 may illuminate the sample X with the fringe pattern LP in step S10. . Further, the sample to be observed in step S10 may be the same as or different from the sample in step S1 to step S3. For example, the sample in steps S1 to S3 may be an analysis sample used for analysis of the aberration correction amount.

  In general, there are aberrations in the optical system of a microscope. If there is an aberration in the optical system, the amplitude of the optical transfer function (OTF) decreases. The spatial frequency spectrum of the image depends on the object, but if the object does not change, the aberration can be corrected so that the spectral amplitude of the image is maximized. In general, since an object changes every time an image is acquired, aberration correction using the spectrum of the image is difficult. The variation of the object includes temporal variation of the object itself, stage drift of the microscope, and the like.

  In the present embodiment, pattern-like illumination (eg, striped illumination, structured illumination) is given to an object. When the contrast of the pattern (eg, fringe) is equal to or higher than a predetermined level, the ratio of the amplitude of the frequency component of the fringe to the amplitude of the DC component of the image spectrum is determined by only the fringe contrast and the OTF, regardless of the object. In this embodiment, by measuring the above ratio, it is possible to analyze (eg, evaluate) the amount of aberration correction by reducing the influence of object fluctuation, and to correct aberration robustly against object fluctuation. Is possible.

[Second Embodiment]
A second embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment are appropriately denoted by the same reference numerals, and the description thereof is omitted or simplified. FIG. 6 is a diagram illustrating a microscope according to the second embodiment. In the microscope 1 according to the present embodiment, the feature amount of the fringe pattern LP (shown later in FIGS. 7 to 10) irradiated by the irradiation unit 3 is variable. The feature amount of the fringe pattern LP includes, for example, a fringe direction (period direction, line direction), a phase, and a spatial frequency (period). The irradiation unit 3 can change at least one of the feature amounts of the stripe pattern LP.

  The branch unit 15 according to the present embodiment includes a polarization separation element 51 and a spatial light modulator 52. The polarization separation element 51 is, for example, a polarization beam splitter prism. The polarization separation element 51 includes a polarization separation film 53 that is inclined by, for example, 45 ° with respect to the light emission direction of the light source LS. The polarization separation film 53 has a characteristic that P-polarized light with respect to the polarization separation film 53 is transmitted and S-polarized light with respect to the polarization separation film 53 is reflected. Here, the light source LS emits P-polarized light with respect to the polarization separation film 53 as illumination light, and the illumination light passes through the polarization separation film 53 and enters the spatial light modulator 52. In the following description, P-polarized light with respect to the polarization separation film 53 is simply referred to as P-polarization, and S-polarization with respect to the polarization separation film 53 is simply referred to as S-polarization.

  The spatial light modulator 52 (SLM) includes, for example, a ferroelectric liquid crystal (FLC). The spatial light modulator 52 has a plurality of pixels and can control the azimuth angle of the liquid crystal molecules in the ferroelectric liquid crystal for each pixel. The spatial light modulator 52 can control the polarization state of light incident on each pixel by controlling the azimuth angle of the liquid crystal molecules. The spatial light modulator 52 can maintain the polarization state of P-polarized light that has passed through the polarization separation film 53 for each pixel, and can convert the polarization state into S-polarized light for each pixel.

  P-polarized light emitted from the spatial light modulator 52 passes through the polarization separation film 53 and deviates from the optical path toward the sample X. The S-polarized light emitted from the spatial light modulator 52 is reflected by the polarization separation film 53 and travels toward the lens 21. The irradiation unit 3 forms a fringe pattern LP by the interference of the S-polarized light. The irradiation unit 3 changes the feature amount of the fringe pattern LP by controlling the pixel that converts the polarization state in the spatial light modulator 52.

  FIG. 7 is a diagram illustrating the fringe pattern and the processing by the image processing unit according to the second embodiment. The irradiation unit 3 irradiates the sample X with the first stripe pattern LP1 and the second stripe pattern LP2 as the stripe pattern LP. The first stripe pattern LP1 periodically changes in luminance in the first direction D1. The brightness of the second stripe pattern LP2 changes periodically in the second direction intersecting the first direction D1.

  The irradiation unit 3 irradiates the sample X with the first stripe pattern LP1 and the second stripe pattern LP2 at different timings. The irradiation unit 3 irradiates the sample X with the first stripe pattern LP1 in the first period (first timing), and applies the second stripe pattern LP2 to the sample X in the second period (second timing) that does not overlap with the first period. Irradiate. The imaging unit 4 captures the sample X irradiated with the first fringe pattern LP1 and acquires a first captured image Im1. The imaging unit 4 captures the sample X in the first period and acquires the first captured image Im1. In addition, the imaging unit 4 captures the sample X irradiated with the second stripe pattern LP2, and acquires the second captured image Im2. The imaging unit 4 captures the sample X in the second period and acquires the second captured image Im2.

  The calculation unit 41 calculates a first contrast value as a contrast value based on the spatial frequency of the first stripe pattern LP1 corresponding to the first direction D1. For example, the calculation unit 41 performs a Fourier transform on the first captured image Im1 to generate a first frequency spectrum (for example, a first converted image Ti1). Based on the intensity of the frequency component of the region TL1 corresponding to the first stripe pattern LP1 (eg, the luminance value of the pixel) in the first frequency spectrum (eg, the first converted image Ti1), the calculation unit 41 performs the first contrast. Calculate the value. The region TL1 includes a region TL1a and a region TL1b. The calculation unit 41 calculates the first contrast value by using one or both of the intensity of the frequency component in the region TL1a and the intensity of the frequency component in the region TL1b.

  Further, the calculation unit 41 calculates the second contrast value as the contrast value based on the spatial frequency of the second stripe pattern LP2 corresponding to the second direction D2. For example, the calculation unit 41 performs a Fourier transform on the second captured image Im2, and generates a second frequency spectrum (for example, the second converted image Ti2). The calculation unit 41 calculates the second contrast based on the intensity of the frequency component (eg, the luminance value of the pixel) in the region TL2 corresponding to the second stripe pattern LP2 in the second frequency spectrum (eg, the second converted image Ti2). Calculate the value. The region TL2 includes a region TL2a and a region TL2b. The calculating unit 41 calculates the second contrast value using one or both of the intensity of the frequency component in the region TL2a and the intensity of the frequency component in the region TL2b.

  The irradiation unit 3, the imaging unit 4, and the calculation unit 41 each execute the above-described process under a plurality of conditions with different aberration correction amounts. The analysis unit 42 calculates an evaluation value based on the first contrast value and the second contrast value. For example, the analysis unit 42 calculates an evaluation value by using an average value of the first contrast value and the second contrast value as the contrast value. The analysis unit 42 calculates the evaluation value under a plurality of conditions with different aberration correction amounts. As shown in the lower part of FIG. 7, the analysis unit 42 derives the aberration correction amount at which the evaluation value is an extreme value as the target value of the aberration correction amount.

  In FIG. 7, the first period in which the sample X is irradiated with the first stripe pattern LP1 and the second period in which the sample X is irradiated with the second stripe pattern LP2 do not overlap, but at least one of the second periods. The part may overlap with the first period. For example, the irradiation unit 3 may irradiate the second stripe pattern LP2 simultaneously with the irradiation of the first stripe pattern LP1. Moreover, the irradiation part 3 may irradiate the stripe pattern LP which synthesize | combined 1st stripe pattern LP1 and 2nd stripe pattern LP2.

  FIG. 8 is a diagram illustrating the fringe pattern and the processing by the image processing unit according to the second embodiment. In FIG. 8, the irradiation unit 3 irradiates the sample X with a fringe pattern LP obtained by synthesizing the first fringe pattern LP1 and the second fringe pattern LP2. The first stripe pattern LP1 periodically changes in luminance in the first direction D1. The brightness of the second stripe pattern LP2 changes periodically in the second direction D2. The stripe pattern LP obtained by synthesizing the first stripe pattern LP1 and the second stripe pattern LP2 periodically changes in luminance in each of the first direction D1 and the second direction D2.

  The imaging unit 4 captures the sample X irradiated with the stripe pattern LP, and acquires a captured image Im. The calculation unit 41 calculates a first contrast value as a contrast value based on the spatial frequency of the first stripe pattern LP1 corresponding to the first direction D1. For example, the calculation unit 41 performs a Fourier transform on the captured image Im to generate a frequency spectrum (for example, a converted image Ti). The calculation unit 41 calculates the first contrast value based on the intensity (eg, the luminance value of the pixel) of the frequency component in the region TL1 corresponding to the first stripe pattern LP1 in the frequency spectrum (eg, the converted image Ti). . The region TL1 includes a region TL1a and a region TL1b. The calculation unit 41 calculates the first contrast value by using one or both of the intensity of the frequency component in the region TL1a and the intensity of the frequency component in the region TL1b.

  Further, the calculation unit 41 calculates the second contrast value as the contrast value based on the spatial frequency of the second stripe pattern LP2 corresponding to the second direction D2. The calculation unit 41 calculates the second contrast value based on the intensity of the frequency component (eg, the luminance value of the pixel) in the region TL2 corresponding to the second stripe pattern LP2 in the frequency spectrum (eg, the converted image Ti). . The region TL2 includes a region TL2a and a region TL2b. The calculating unit 41 calculates the second contrast value using one or both of the intensity of the frequency component in the region TL2a and the intensity of the frequency component in the region TL2b. Then, the analysis unit 42 calculates an evaluation value based on the first contrast value and the second contrast value. This process is the same as the process described in FIG.

  As described above, when the fringe pattern LP has a plurality of periodic directions (the first direction D1 and the second direction D2), the analysis unit 42 can analyze the correction amount in consideration of the axially asymmetric aberration. The number of stripe patterns LP in the periodic direction is two (first direction D1 and second direction D2) in FIGS. 7 and 8, but may be three or four or more.

  As shown in FIG. 7, when the first period during which the first stripe pattern LP1 is irradiated onto the sample X and the second period during which the second stripe pattern LP2 is irradiated onto the sample X do not overlap, the irradiation unit 3 May be switched between the first period and the second period in the exposure period (eg, light reception period, one frame period) by the imaging unit 4. In this case, the imaging unit 4 can acquire a captured image Im similar to the state in which the sample X is irradiated with the stripe pattern LP obtained by combining the first stripe pattern LP1 and the second stripe pattern LP2.

  Next, a modification of the fringe pattern and the processing by the image processing unit according to the second embodiment will be described with reference to FIGS. 9 and 10. 9 and 10 are diagrams showing the fringe pattern and the processing by the image processing unit according to the second embodiment. The irradiation unit 3 irradiates the sample X with the third stripe pattern LP3 and the fourth stripe pattern LP4 as the stripe pattern LP. The third stripe pattern LP3 and the fourth stripe pattern LP4 have different phases. The phase of the third stripe pattern LP3 and the phase of the fourth stripe pattern LP4 indicate that the time average value of the luminance of the illumination light (third stripe pattern LP3, fourth stripe pattern LP4) at each point on the sample X is the sample X. It is set to be spatially uniform above.

  When N types of fringe patterns having different phases are used, the phases of the N types of fringe patterns are set to (360 ° / N × m), for example. N is an arbitrary natural number of 2 or more, and m is a natural number of 0 or more and less than N (0, 1, 2,... (N−1)). m = 0 corresponds to the reference phase. Here, N = 2 and the phase of the fourth fringe pattern LP4 corresponding to m = 1 is shifted by 180 ° from the phase of the third fringe pattern LP3 corresponding to m = 0. Note that N is arbitrarily selected from two or more natural numbers. For example, when three types of stripe patterns are used as the stripe pattern LP, N = 3, and the phases of the three types of stripe patterns are shifted from the reference phase (m = 0) by 120 ° from the reference phase (m = 1) and a phase shifted by 240 ° from the reference phase (m = 2).

  The irradiation unit 3 irradiates the sample X with the third stripe pattern LP3 in the third period. Then, the imaging unit 4 images the sample X irradiated with the third fringe pattern LP in the third period, and acquires a third captured image Im3. In addition, the irradiation unit 3 irradiates the sample X with the fourth stripe pattern LP4 in a fourth period that does not overlap at least part of the third period. The fourth stripe pattern LP4 corresponds to a stripe pattern in which the bright portion LPa and the dark portion LPb of the third stripe pattern LP3 are interchanged. Then, in the fourth period, the imaging unit 4 captures the sample X irradiated with the fourth stripe pattern LP4, and acquires a fourth captured image Im4.

  The calculation unit 41 calculates a contrast value for one or both of the third stripe pattern LP3 and the fourth stripe pattern LP4. The calculation unit 41 calculates a third contrast value as a contrast value based on the spatial frequency of the third stripe pattern LP3. For example, the calculation unit 41 performs a Fourier transform on the third captured image Im3 to generate a third frequency spectrum (eg, a third converted image Ti3). The calculation unit 41 calculates the third contrast based on the intensity of the frequency component (eg, the luminance value of the pixel) in the region TL3 corresponding to the third stripe pattern LP3 in the third frequency spectrum (eg, the third converted image Ti3). Calculate the value. The region TL3 includes a region TL3a and a region TL3b. The calculation unit 41 calculates the third contrast value by using one or both of the intensity of the frequency component in the region TL3a and the intensity of the frequency component in the region TL3b.

  Further, the calculation unit 41 calculates the fourth contrast value as the contrast value based on the spatial frequency of the fourth stripe pattern LP4. For example, the calculation unit 41 performs a Fourier transform on the fourth captured image Im4 to generate a fourth frequency spectrum (eg, a fourth converted image Ti4). In FIG. 9, the spatial frequency of the fourth stripe pattern LP4 is the same as the spatial frequency of the third stripe pattern LP3. The calculation unit 41 calculates the fourth contrast based on the intensity of the frequency component (eg, the luminance value of the pixel) of the region TL4 corresponding to the fourth stripe pattern LP4 in the fourth frequency spectrum (eg, the fourth converted image Ti4). Calculate the value. The region TL4 includes a region TL4a and a region TL4b. The calculation unit 41 calculates the fourth contrast value by using one or both of the intensity of the frequency component in the area TL4a and the intensity of the frequency component in the area TL4b. The analysis unit 42 calculates an evaluation value based on the third contrast value and the fourth contrast value. This process is the same as the process described in FIG.

  Thus, when using the fringe pattern LP having two or more types of phases, the microscope 1 can reduce the spatial variation of the time average value of the luminance at each position on the sample X, and the fluorescent material in the sample X Spatial variation in the level of fading can be reduced.

  Note that the imaging unit 4 may not image the sample X in a state where the third stripe pattern LP3 is irradiated on the sample X or in a state where the fourth stripe pattern LP4 is irradiated on the sample X. For example, the imaging unit 4 may image the sample X irradiated with the third stripe pattern LP3 and may not image the sample X irradiated with the fourth stripe pattern LP4. In this case, the calculation unit 41 calculates the contrast value of the third stripe pattern LP3 based on the third captured image Im3. Further, the analysis unit 42 calculates a target value for the aberration correction amount based on the contrast value of the third fringe pattern LP3 calculated by the calculation unit 41. As described above, even when the imaging and the subsequent processing regarding the third stripe pattern LP3 or the fourth stripe pattern LP4 are not performed, the spatial distribution of the time average value of the luminance in the sample X can be made uniform.

  In FIG. 10, the irradiation unit 3 irradiates the sample X with the fifth stripe pattern LP5 and the sixth stripe pattern LP6 as the stripe pattern LP. The fifth stripe pattern LP5 and the sixth stripe pattern LP6 have different spatial frequencies. In FIG. 10, the spatial frequency of the fifth fringe pattern LP5 is higher than the spatial frequency of the sixth fringe pattern LP6 (the fringe period is short and the fringe period is different from the sixth fringe pattern).

  The irradiation unit 3 irradiates the sample X with the fifth stripe pattern LP5 in the fifth period. Then, the imaging unit 4 captures the sample X irradiated with the fifth stripe pattern LP5 in the fifth period, and acquires a fifth captured image Im5. In addition, the irradiation unit 3 irradiates the sample X with the sixth stripe pattern LP6 in a sixth period that does not overlap with at least a part of the fifth period. Then, the imaging unit 4 captures the sample X irradiated with the sixth striped pattern LP6 in the sixth period, and acquires a sixth captured image Im6.

  The calculation unit 41 calculates a contrast value for each of the fifth stripe pattern LP5 and the sixth stripe pattern LP6. The calculation unit 41 calculates a fifth contrast value as a contrast value based on the spatial frequency of the fifth stripe pattern LP5. For example, the calculation unit 41 performs a Fourier transform on the fifth captured image Im5 to generate a fifth frequency spectrum (eg, a fifth converted image Ti5). Based on the intensity of the frequency component of the region TL5 corresponding to the fifth stripe pattern LP5 (eg, the luminance value of the pixel) in the fifth frequency spectrum (eg, the fifth converted image Ti5), the calculation unit 41 performs the fifth contrast. Calculate the value. The region TL5 includes a region TL5a and a region TL5b. The calculation unit 41 calculates the fifth contrast value by using one or both of the intensity of the frequency component in the region TL5a and the intensity of the frequency component in the region TL5b.

  Further, the calculation unit 41 calculates a sixth contrast value as a contrast value based on the spatial frequency of the sixth stripe pattern LP6. For example, the calculation unit 41 performs a Fourier transform on the sixth captured image Im6 to generate a sixth frequency spectrum (eg, a sixth converted image Ti6). Based on the intensity (eg, pixel luminance value) of the frequency component of the region TL6 corresponding to the sixth striped pattern LP6 in the sixth frequency spectrum (eg, sixth transformed image Ti6), the calculation unit 41 calculates the sixth contrast. Calculate the value. The region TL6 includes a region TL6a and a region TL6b. The calculating unit 41 calculates the sixth contrast value by using one or both of the intensity of the frequency component in the region TL6a and the intensity of the frequency component in the region TL6b. The analysis unit 42 calculates an evaluation value based on the fifth contrast value and the sixth contrast value. This process is the same as the process described in FIG.

  The irradiating unit 3 irradiates the sample X with a plurality of stripe patterns having two or more types of feature amounts (a plurality of types of feature amounts (stripe direction, phase, spatial frequency)) as the stripe pattern LP. Also good. For example, the irradiation unit 3 may irradiate the sample X with a plurality of types of stripe patterns having different stripe directions and spatial frequencies as the stripe pattern LP.

  The irradiation unit 3 changes the periodic direction of the fringe pattern LP by rotating the branching unit 15 (diffraction grating) described in the first embodiment about the optical axis 11a of the illumination optical system 11 as a rotation center. Also good. Further, the irradiation unit 3 changes the phase of the fringe pattern LP by moving the branching unit 15 (diffraction grating) described in the first embodiment in a direction intersecting the optical axis 11a of the illumination optical system 11. Also good. The irradiation unit 3 includes, as the branching unit 15, a first diffraction grating and a second diffraction grating having different periods of the periodic structure, and the first diffraction grating is disposed in the optical path of the illumination optical system 11 and the first diffraction grating. The spatial frequency of the fringe pattern LP may be changed by switching the state where the two diffraction gratings are arranged.

  In the above-described embodiment, the control device 5 includes, for example, a computer system. The control device 5 reads a program stored in the storage device 7 and executes various processes according to the program. The program includes, for example, an analysis program (image analysis program). The analysis program causes the computer to check the fringe pattern LP based on the captured image Im obtained by imaging the sample X irradiated with the fringe pattern LP via the objective lens 28 via the objective lens and the spatial frequency of the fringe pattern LP. And calculating the correction amount of the aberration of the image of the sample X based on the contrast value. This analysis program may be provided by being recorded on a computer-readable storage medium.

  The technical scope of the present invention is not limited to the aspects described in the above-described embodiments. One or more of the requirements described in the above embodiments and the like may be omitted. In addition, the requirements described in the above-described embodiments and the like can be combined as appropriate. In addition, as long as it is permitted by law, the disclosure of all documents cited in the above-described embodiments and the like is incorporated as a part of the description of the text.

DESCRIPTION OF SYMBOLS 1 ... Microscope, 3 ... Irradiation part, 4 ... Imaging part, 8 ... Aberration correction part, 9 ... Image processing part (image processing apparatus), X ... Sample, 41 ... Calculation unit, 42 ... analysis unit, LP ... fringe pattern, PS ... frequency spectrum

Claims (19)

An imaging unit for capturing an image of the sample;
An aberration correction unit for correcting aberration of the image of the sample;
An irradiation unit for irradiating the sample with a stripe pattern;
A calculation unit that calculates a contrast value of the fringe pattern based on a captured image obtained by the imaging unit imaging the sample irradiated with the fringe pattern and a spatial frequency of the stripe pattern;
A microscope comprising: an analysis unit that analyzes a correction amount of the aberration by the aberration correction unit based on the contrast value calculated by the calculation unit. The calculation unit calculates the contrast value based on a frequency spectrum obtained by Fourier transform of the captured image and a spatial frequency of the fringe pattern.
The microscope according to claim 1. The calculation unit calculates the contrast value based on the intensity of a frequency component in a region corresponding to the spatial frequency of the fringe pattern in the frequency spectrum.
The microscope according to claim 2. The calculation unit sets the position of the region based on the position where the intensity of the frequency component becomes maximum corresponding to the spatial frequency of the fringe pattern in the frequency spectrum.
The microscope according to claim 3. The calculation unit sets the size of the region based on the maximum value of the intensity of the frequency component corresponding to the spatial frequency of the fringe pattern in the frequency spectrum.
The microscope according to claim 3 or 4. The calculation unit includes a position where the intensity of the frequency component is maximized corresponding to the spatial frequency of the fringe pattern in the frequency spectrum, and a position where the intensity of the frequency component is maximized corresponding to the DC component in the frequency spectrum; Set the size of the area based on
The microscope according to claim 3 or 4. The calculation unit sets the position of the region based on a preset spatial frequency of the fringe pattern,
The microscope according to any one of claims 3 to 6. The calculation unit uses a converted image obtained by Fourier transforming the captured image as the frequency spectrum, and uses a luminance value of a pixel of the converted image as an intensity of a frequency component in the frequency spectrum.
The microscope according to any one of claims 2 to 7. The analysis unit calculates a target value of the correction amount of the aberration by using a ratio of a direct current component in the frequency spectrum and a spatial frequency component of the fringe pattern in the frequency spectrum as an evaluation value;
The microscope according to any one of claims 2 to 8. The imaging unit executes imaging under a plurality of conditions with different correction amounts of the aberration,
The calculation unit calculates the contrast value for each of the plurality of conditions,
The analysis unit calculates a correction amount of the aberration at which the evaluation value based on the contrast value is an extreme value;
The microscope according to any one of claims 1 to 9. The irradiation unit includes a first stripe pattern whose luminance periodically changes in a first direction and a second stripe pattern whose luminance periodically changes in a second direction intersecting the first direction, as the stripe pattern. To the sample,
The calculation unit calculates a first contrast value as the contrast value based on a spatial frequency of the first stripe pattern corresponding to the first direction, and a spatial frequency of the second stripe pattern corresponding to the second direction. A second contrast value is calculated as the contrast value based on
The analysis unit analyzes the correction amount of the aberration based on the first contrast value and the second contrast value calculated by the calculation unit;
The microscope according to any one of claims 1 to 10. The irradiation unit irradiates the sample with a third stripe pattern and a fourth stripe pattern having different phases as the stripe pattern,
The calculation unit calculates the contrast value for one or both of the third stripe pattern and the fourth stripe pattern,
The microscope according to any one of claims 1 to 11. The phase of the fourth stripe pattern is 180 ° out of phase with the third stripe pattern.
The microscope according to claim 12. The irradiation unit irradiates the sample with a fifth stripe pattern and a sixth stripe pattern having different spatial frequencies as the stripe pattern,
The calculation unit calculates the contrast value for each of the fifth stripe pattern and the sixth stripe pattern;
The microscope according to any one of claims 1 to 13. The aberration correction unit includes an optical member movable along an optical path of light emitted from the sample,
The correction amount of the aberration is expressed based on the movement amount of the optical member.
The microscope according to any one of claims 1 to 14. The aberration correction unit includes a wavefront adjustment unit that adjusts a wavefront of light emitted from the sample,
The correction amount of the aberration is expressed based on the adjustment amount of the wavefront by the wavefront adjustment unit,
The microscope according to any one of claims 1 to 15. A calculation unit that calculates a contrast value of the fringe pattern based on a captured image obtained by capturing an image of a sample irradiated with the fringe pattern and a spatial frequency of the fringe pattern;
An image processing apparatus comprising: an analysis unit that analyzes a correction amount of an aberration of an image of the sample based on the contrast value calculated by the calculation unit. Taking an image of the sample;
Correcting aberrations in the sample image;
Irradiating the sample with a stripe pattern;
Calculating a contrast value of the stripe pattern based on a captured image obtained by imaging the sample irradiated with the stripe pattern and a spatial frequency of the stripe pattern;
Analyzing the correction amount of the aberration based on the contrast value;
Setting an amount of correction of the aberration based on the analysis. On the computer,
Calculating a contrast value of the fringe pattern based on a captured image obtained by capturing an image of the sample irradiated with the fringe pattern and a spatial frequency of the fringe pattern;
Analyzing the amount of correction of the aberration of the image of the sample based on the contrast value.