JP6264715B2 - Microscope system - Google Patents

Description

  The present invention relates to a microscope system that derives and forms an intensity distribution of illumination light suitable for observation.

  In the bright field microscope, the intensity distribution of the illumination light is adjusted by changing the circular aperture. In some cases, the shape of the diaphragm is selected and used at the discretion of the observer. In the phase contrast microscope, the ring diaphragm and the phase ring form an intensity distribution of illumination light.

  Since the intensity distribution of the illumination light has a large effect on the observation image of the subject, improvements have been made to improve the observation image of the subject by improving the circular diaphragm, ring diaphragm, phase ring, etc. Yes. For example, in Patent Document 1, a modulation unit is provided so as to surround a ring region provided in a ring shape of a phase ring, and the contrast is obtained by forming the modulation unit and the region other than the modulation unit so that the directions of the transmission axes are different. A continuously variable phase contrast microscope is shown.

JP 2009-237109 A

  A microscope system according to a first aspect includes an illumination light source that irradiates a subject with illumination light, an imaging optical system that forms an image of transmitted light or reflected light of the subject, and illumination light at a conjugate position of a pupil of the imaging optical system. An illumination optical system that irradiates the subject with light from the illumination light source, an image sensor that detects light via the imaging optical system, and a first space Each time the intensity distribution of the illumination light is changed by the light modulation element, the output data detected by the image sensor after changing the intensity distribution of the illumination light and detected by the image sensor before changing the intensity distribution of the illumination light. A calculation unit that sequentially compares the output data and obtains the intensity distribution of illumination light suitable for observation of the subject.

1 is a schematic configuration diagram of a microscope system 100. FIG. (A) is a schematic block diagram in case the 1st spatial light conversion element 90 is the liquid crystal panel 93. FIG. FIG. 6B is a schematic configuration diagram when the first spatial light conversion element 90 is a digital micromirror device (DMD) 94. It is a flowchart of the hill-climbing method for finding a suitable illumination area 91 4 is a diagram of an area setting unit 22 and a parameter setting unit 23 of the display unit 21. FIG. It is a figure of the display part 21 in the case of setting the illumination area 91 of the 1st spatial light conversion element 90. FIG. 6 is a schematic plan view of an illumination area 91 formed by a first spatial light conversion element 90. FIG. It is a flowchart using a genetic algorithm. (A) is a schematic plan view of the first spatial light conversion element 90 having a circular illumination area 91 having a large diameter. (B) is a schematic plan view of the first spatial light conversion element 90 having a circular illumination region 91 having a small diameter. (C) is a schematic plan view of the first spatial light conversion element 90 having an annular illumination area 91. (D) is a schematic plan view of a first spatial light conversion element 90 having an illumination area 91 in which four small circular illumination areas 91 are arranged axially symmetrically with respect to the optical axis. (E) is a schematic plan view of a first spatial light conversion element 90 having two rectangular illumination areas 91 and having illumination areas 91 arranged symmetrically about the optical axis. (F) is a schematic plan view of the first spatial light conversion element 90 in which the illumination region 91 is formed non-axisymmetrically. (A) is the figure which showed the example of a combination of Fig.7 (a) and FIG.7 (b). (B) is the figure which showed the example of a combination of Fig.7 (a) and FIG.7 (d). 3 is a flowchart of a method 1 for estimating phase information of a subject 60; It is a figure of the object information acquisition screen 23d shown by the parameter setting part 23 of the display part 21. FIG. 4 is a flowchart of a method for estimating fine structure information of a subject 60. 5 is a flowchart illustrating a method for estimating information on properties of a subject 60 with respect to the wavelength of illumination light. 10 is a flowchart of a method 2 for estimating phase information of a subject 60; It is the figure of the display part 21 by which the schematic of the 1st spatial light conversion element 90 was shown. 1 is a schematic configuration diagram of a microscope system 200. FIG. It is a flowchart of the spatial frequency information detection method of an object. (A) is the figure of the display part 21 in which the image of the image of the pupil 273 detected by the 2nd image sensor 280 in case the test subject 60 is an integrated circuit (IC) is shown. (B) is the figure of the display part 21 in which the image of the image of the pupil 273 detected by the 2nd image sensor 280 in case the subject 60 is a biological body is shown. (C) is the figure of the display part 21 in which the image of the image of the pupil 273 in each wavelength of red, blue, and green detected by the 2nd image sensor 280 when the subject 60 is a living body is shown. (A) is a schematic block diagram of the microscope system 300. FIG. FIG. 7B is a plan view of the first spatial light modulator 390. (C) is a plan view of the second spatial light modulator 396.

In a microscope such as Patent Document 1, the shape of the diaphragm is determined to some extent, and there is a limitation in adjusting the intensity distribution of illumination light. Further, since the diaphragm shape is selected based on the judgment or experience of the observer, the shape of the diaphragm is not necessarily a shape that allows the image of the object being observed to be observed in the best condition. Further, in the phase contrast microscope, since the positions of the ring stop and the phase ring are fixed, the shape cannot be freely selected, and it is difficult to observe the image of the object under observation in an optimum state.
Therefore, a microscope system that derives and forms an intensity distribution of illumination light suitable for observing a subject is provided.
(First embodiment)
As a first embodiment, a microscope system that can freely change the shape of the aperture and automatically adjust the intensity distribution of illumination light that is suitable for observing the image of the object under observation in good condition. 100 will be described.

<Microscope system 100>
FIG. 1 is a schematic configuration diagram of a microscope system 100. The microscope system 100 mainly includes an illumination light source 30, an illumination optical system 40, a stage 50, an imaging optical system 70, an image sensor 80, and a calculation unit 20. Hereinafter, the central axis of the light beam emitted from the illumination light source 30 will be referred to as the Z-axis direction, and the directions perpendicular to the Z-axis and perpendicular to each other will be described as the X-axis direction and the Y-axis direction.

  The illumination light source 30 irradiates the subject 60 with white illumination light, for example. The illumination optical system 40 includes a first condenser lens 41, a wavelength filter 44, a first spatial light modulator 90, and a second condenser lens 42. The imaging optical system 70 includes an objective lens 71. The stage 50 is capable of moving in the XY axis direction by placing a subject 60 having an unknown structure such as a cell tissue. The imaging optical system 70 causes the image sensor 80 to form an image of the transmitted light or reflected light of the subject 60.

  The first spatial light modulation element 90 of the illumination optical system 40 is disposed at a position conjugate to the position of the pupil of the imaging optical system 70 in the illumination optical system 40, for example. The intensity distribution of illumination light at the conjugate position can be varied. Further, the first spatial light modulator 90 has an illumination area 91 whose shape and size can be freely changed, and the intensity distribution of illumination light can be arbitrarily changed by changing the size or shape of the illumination area 91. Can be varied. The wavelength filter 44 limits the wavelength of the transmitted light beam within a specific range. As the wavelength filter 44, for example, a band pass filter that transmits only light in a specific range of wavelengths is used. The wavelength filter 44 is detachable, and the wavelength of the light transmitted through the wavelength filter 44 can be controlled by preparing and replacing a plurality of band pass filters that transmit light of different wavelengths.

  The calculation unit 20 receives the output data detected by the image sensor 80 and displays it on the display unit 21 such as a monitor. Further, the output data is analyzed, and the intensity distribution of illumination light suitable for observation of the subject 60 is calculated. In addition, the calculation unit 20 can control and drive the illumination area 91 of the first spatial light modulator 90 and control the wavelength range of the light beam that passes through the wavelength filter 44.

  In FIG. 1, the light emitted from the illumination light source 30 is indicated by a dotted line. The light LW11 emitted from the illumination light source 30 is converted into parallel light LW12 by the first condenser lens 41. The light LW 12 passes through the wavelength filter 44, has a wavelength range specified, and enters the first spatial light modulator 90. The light LW 13 that has passed through the illumination area 91 of the first spatial light modulator 90 passes through the second condenser lens 42 to become light LW 14, and travels toward the stage 50. The light LW15 that has passed through the stage 50 passes through the imaging optical system 70 to become light LW16, and forms an image of the subject 60 on the image sensor 80.

  The image of the subject 60 detected by the image sensor 80 is sent to the calculation unit 20 as output data. The calculation unit 20 estimates the structure of the subject 60 based on the output data obtained from the image sensor 80, the transmission wavelength of the wavelength filter 44, and the shape data of the illumination region 91 formed by the first spatial light modulator 90. The illumination shape suitable for observation of the subject 60, that is, the intensity distribution of the illumination light is calculated. Then, the calculated shape suitable for observation of the subject 60 is transmitted to the first spatial light modulator 90, and the illumination area 91 is shaped into an illumination shape suitable for observation of the subject 60. Similarly, the wavelength of illumination light suitable for observing the subject 60 is also calculated by the calculation unit 20, and a bandpass filter optimal for observing the subject 60 is selected as the wavelength filter 44.

<Illumination optical system 40>
As the first spatial light conversion element 90, a liquid crystal panel 93, a digital micromirror device (DMD) 94, or the like can be used. The first spatial light conversion element 90 using these will be described with reference to FIG.

  FIG. 2A is a schematic configuration diagram when the first spatial light conversion element 90 is a liquid crystal panel 93. The liquid crystal panel 93 includes, for example, a liquid crystal film 93a, a first polarizing film 93b, and a second polarizing film 93c. The liquid crystal film 93a is filled with a liquid crystal material to form an electrode such as a thin film transistor (TFT), and a voltage can be applied to any part of the liquid crystal film 93a. The light LW11 emitted from the illumination light source 30 is converted into parallel light LW12 by the first condenser lens 41, the wavelength range is specified by the wavelength filter 44, and the light LW12a polarized in one direction by the first polarizing film 93b Limited to only. The light LW12a is a liquid crystal film 93a, and is converted into light LW12c that is polarized by 90 degrees by applying a voltage to the liquid crystal film 93a and light LW12b that is not polarized without being applied to the liquid crystal film 93a. The second polarizing film 93c is disposed so as to transmit only the 90-degree polarized light of the light transmitted through the first polarizing film 93b. Therefore, only the light LW12c is transmitted through the second polarizing film 93c and the light LW13 is transmitted through the second polarizing film 93c. In the liquid crystal panel 93, the illumination area 91 is formed in an arbitrary shape by controlling the position where the voltage of the liquid crystal film 93a is applied.

  FIG. 2B is a schematic configuration diagram when the first spatial light conversion element 90 is a digital micromirror device (DMD) 94. The surface of the DMD 94 is formed by an aggregate of a plurality of small movable reflecting mirrors (not shown), and each mirror can be moved independently. The light LW11 emitted from the illumination light source 30 is converted into parallel light LW12 by the first condenser lens 41, the wavelength range is specified by the wavelength filter 44, and the entire DMD 94 is irradiated. When the movable reflecting mirror disposed in the region 94a of the DMD 94 is directed in the direction in which the light LW11 is reflected by the subject 60, the light LW13a is irradiated to the subject 60. When the DMD 94 is used for the first spatial light conversion element 90, the object 60 can be irradiated with light in an arbitrary shape by controlling which position of the movable reflection mirror is moved. This corresponds to the illumination area 91 of the first spatial modulation element 90 shown in FIG. 1 being formed in an arbitrary shape.

  An electrochromic element may be used for the first spatial light conversion element 90. The electrochromic element mainly has a laminated structure in which a transparent electrode such as a TFT and an electrochromic layer are combined. When a voltage is applied to an electrochromic layer, an electrolytic oxidation or reduction reaction occurs reversibly in a region where the voltage is applied, and the state of transmitting light and the state of not transmitting light can be reversibly changed. it can. Therefore, in the electrochromic element, the illumination region 91 is formed in an arbitrary shape by controlling the position where the voltage of the electrochromic layer is applied. Details of the structure and operation of the electrochromic element are disclosed in, for example, Japanese Patent Application Laid-Open No. 8-220568.

  The first spatial light conversion element 90 includes an optical element having a plurality of spaces in which an electroactive material whose specific optical characteristics such as transmittance are changed by application of electrical stimulation is encapsulated and electrodes such as TFTs are formed. It may be used. This optical element has cells that are hermetically sealed and formed in an array, and each cell is filled with an electroactive material. An electrode is formed in each cell so that a voltage can be applied independently for each cell. By controlling the voltage applied to each cell, the state where light is transmitted through the cell and the state where light is not transmitted are reversible. Can be changed. In this optical element, the illumination region 91 is formed in an arbitrary shape by controlling which cell is applied with voltage. Details of the structure and operation of this optical element are disclosed in, for example, JP-T-2010-507119.

  Although the wavelength filter 44 is disposed between the first condenser lens 41 and the first spatial light modulator 90 in FIG. 1, the purpose is to detect light of a specific wavelength by the image sensor 80, so that the illumination light source 30 and the image sensor 80 may be arranged at any position.

  Further, instead of using a light source for irradiating the wavelength filter 44 and the illumination light source 30 with white illumination light, an LED (light emitting diode) or the like may be used for the illumination light source 30. When the illumination light source 30 is comprised by LED, it can be comprised by the combination of LED which emits the light of each wavelength of red, blue, and green, for example. The wavelength of the light emitted from the illumination light source 30 is controlled by controlling the lighting and extinguishing of the LED of each wavelength by the calculation unit 20. In this way, the LED light source can be substituted for the combination of the wavelength filter 44 and the white illumination light source. Furthermore, an image sensor having a plurality of light receiving elements having different light receiving wavelengths, such as a CCD and a CMOS, may be used as the image sensor 80. In this case, for example, by extracting only a signal from a light receiving element that receives light having a red wavelength, transmitted light having a red wavelength that has passed through the subject 60 can be obtained.

<< Method for deriving illumination shape (intensity distribution of illumination light) >>
A method for calculating an illumination shape suitable for observation of the subject 60 will be described below. There are several calculation methods, such as annealing method and tabu search. Hereinafter, two methods, a hill-climbing method (steepest gradient method) and a method using a genetic algorithm will be described.

<Mountain climbing method>
In the hill-climbing method, the illumination shape that was initially set is changed little by little, and the output data of the image is acquired for each change, and the condition that this output data is closest to the condition set by the observer is searched. It is a way to go. This will be described below with reference to FIG.

FIG. 3 is a flowchart of a hill-climbing method for finding a suitable illumination shape by gradually changing the illumination shape.
In step S101, first, the illumination area 91 of the first spatial light conversion element 90 is set to an initial size and shape. For example, the default illumination area 91 is circular and has a maximum aperture. In this state, an image of the subject 60 is detected by the image sensor 80. The purpose of detecting the image of the subject 60 is to acquire a reference image before adjusting the shape and size of the illumination area 91. The output data of the image of the subject 60 detected by the image sensor 80 is sent to the calculation unit 20, and the image of the subject 60 is displayed on the display unit 21 such as a monitor connected to the calculation unit 20.

  In step S102, the observation region 24 (see FIG. 4A) of the observation image is set by the region setting unit 22 (see FIG. 4A) on the display unit 21. In step S101, the region setting unit 22 displays an image of the subject 60. Details of the region setting unit 22 and the observation region 24 will be described with reference to FIG. 4A.

  In step S103, parameters for forming an observation image of the subject 60 are set. In the parameter setting unit 23 (see FIG. 4A), the observer can set parameters for inputting the observer's desires and allowable observation conditions regarding the observation image of the subject 60. Hereinafter, a display example of the display unit 21 will be described with reference to FIG. 4A.

  FIG. 4A is a diagram of the area setting unit 22 and the parameter setting unit 23 of the display unit 21. The display unit 21 is preferably formed by a GUI (graphical user interface) that performs input using a mouse, a touch pad, or the like. The GUI is easy to operate because the observer can operate it sensuously. The display unit 21 can display, for example, an area setting unit 22 and a parameter setting unit 23. An image of the subject 60 is displayed on the region setting unit 22 and an observation region desired by the observer can be set. Further, the parameter setting unit 23 can input setting of observation conditions desired by the observer. For example, the parameter setting unit 23 displays an observation condition setting item display screen 23a. On the observation condition setting item display screen 23a, items such as region setting, spatial frequency band setting, wavelength band setting, and the like are displayed. When the region setting is selected, the screen of the display unit 21 is switched to the region setting screen 22a. When the spatial frequency band setting and the wavelength band setting are selected, the screen of the display unit 21 is switched to the spatial frequency band setting screen 23b and the wavelength band setting screen 23c, respectively.

  The area setting unit 22 is represented as an area setting screen 22a, for example. The area setting unit 22 displays an image of the subject 60 detected in step S101. The observer can set the observation area 24 of the observation image of the subject 60. For example, the observer may set the entire subject 60 as the observation region 24 or may set only a part of the subject 60. Further, the region setting unit 22 may set two or more observation regions 24 at a time. In addition, the region setting unit 22 may display the subject 60 in an enlarged manner, not optically, so that the observer can easily set the observation region 24, or reduce the image of the subject 60. The entire image of the subject 60 may be displayed. The observation area 24 is set as an area that reflects the parameters set by the parameter setting unit 23.

  The observation area 24 may be automatically set by the calculation unit 20. For example, the contrast of the image output data acquired in step S101 is obtained, and a region having a high contrast and a region having a low contrast are roughly classified. Then, in the region setting screen 22a of FIG. 4A, a region with low contrast is automatically set as the observation region 24, and the observation conditions can be optimized in this observation region 24. In this example, the contrast is roughly divided into a high region and a low region. However, the present invention is not limited to these two, and a threshold value for contrast may be set as appropriate to divide the region into three or more regions including a medium contrast region. good. In this example, an area with low contrast is set as the observation area 24, but instead, an area with high contrast or an intermediate area may be set. Furthermore, the setting of the observation area 24 is not limited to a method based on contrast. For example, the observation region 24 may be automatically set based on a spatial frequency or the like derived by a method for detecting spatial frequency information of an object described later.

  On the spatial frequency band setting screen 23b, the spatial frequency band of the subject 60 desired by the observer can be set. As shown in FIG. 4A, the spatial frequency band may be set by the observer inputting a numerical value, or the observer may select a desired spatial frequency band from a plurality of options. . In the wavelength band setting screen 23c, the wavelength band of light that the observer wants to use or wants to observe can be set. For example, when a wavelength suitable for observation of the subject 60 is estimated in the object information estimation method 1 described later, the wavelength can be set on the wavelength band setting screen 23c. As shown in FIG. 4A, the wavelength band may be set by the observer inputting a numerical value, or the wavelength band desired by the observer from among a plurality of options, for example, red, green, blue, etc. May be selected.

  If the observer does not want to set the observation area 24, step S102 may be skipped. In this case, the entire image of the subject 60 detected by the image sensor 80 is set as the observation region 24.

  The parameters are set by the observer using the parameter setting unit 23 of the display unit 21 as shown in FIG. 4A. The parameter to be set includes, for example, a specific location of the subject 60 to be observed with high contrast or a specific spatial frequency region of the subject 60. For example, when it is desired to observe the observation region 24 set in step S102 with light and shade (light / dark) or to observe the observation region 24 clearly up to a fine image, the observer sets the wavelength band and the spatial frequency band. Can be set.

  Furthermore, the observer can also initially set the illumination area 91 as one parameter. For example, the shape of the illumination area 91 used first in step S104 described later can be initialized. In step S101, when the shape of the illumination area 91 preferable for the subject 60 can be estimated, the shape of the illumination area 91 may be used as an initial setting. By initializing the shape of the illumination area 91, the time until the shape of the illumination area 91 is finally determined can be shortened. An example in which the observer initially sets the illumination area 91 will be described with reference to FIG. 4B.

  FIG. 4B is a diagram of the display unit 21 when the illumination area 91 of the first spatial light conversion element 90 is initially set. In FIG. 4B, parameter setting units 23 are formed at two locations on the right side and the left side of the display unit 21. In the parameter setting unit 23 on the right side, a plan view of the first spatial light conversion element 90 is displayed with a dotted line. In FIG. 4B, the light shielding portion 92 and the illumination area 91 formed in the light shielding portion 92 are indicated by oblique lines. Further, FIG. 4B shows a coordinate line 97 so that the central axis of the first spatial light conversion element 90 can be recognized. The observer can freely initialize the shape of the illumination area 91 of the first spatial light conversion element 90. For example, the illumination area 91 may be formed by displaying a shape sample of the illumination area 91 in the parameter setting unit 23 on the left side of FIG. 4B and selecting a desired illumination area 91 from the shape sample. 91 may be freely drawn and formed. Further, the illumination area 91 does not need to be arranged on the central axis of the first spatial light conversion element 90. That is, the observer may set the illumination region 91 at a position away from the central axis as shown in FIG. 4B. Furthermore, two or more illumination areas 91 may be set at a time.

  The screen shown in FIG. 4A or 4B is selectively displayed on the display unit 21 in a window display. Further, the display unit 21 may display only the region setting unit 22 or the parameter setting unit 23.

  Returning to FIG. 3, in step S <b> 104, the calculation unit 20 changes the size of the illumination area 91 of the first spatial light conversion element 90. When the observer sets the illumination area 91 in step S103, the calculation unit 20 changes the size of the illumination area 91 using the set illumination area 91 as an initial setting value. When the illumination area 91 is not set in step S103, the calculation unit 20 slightly changes the size of the illumination area 91 having the initial setting value set in step S101. That is, the intensity distribution of the illumination light is slightly changed.

  With reference to FIG. 5, a change in the intensity distribution of the illumination light will be described. FIG. 5 is a schematic plan view of the first spatial light conversion element 90. In FIG. 5, a circular illumination area 91 is formed at the center of the light shielding portion 92 of the first spatial light conversion element 90. FIG. 5 shows an example in which the illumination area 91 is initially set as a circle having a diameter of W13 at the center of the first spatial light conversion element 90 in step S103.

  The diameter of the light shielding portion 92 is W11, and the diameter of the default illumination area 91 is W13. In step S104, the size of the illumination area 91 is slightly changed, and the diameter of the illumination area 91 is W12. In the example of FIG. 5, the calculation unit 20 changes the diameter of the illumination region 91 from the diameter W13 to a diameter W12 slightly larger than the diameter W13, and changes the intensity distribution of the illumination light while being similar.

  In step S105, the image sensor 80 detects an image of the subject 60. For example, in FIG. 5, the image of the subject 60 is detected by the image sensor 80 under the condition of the illumination region 91 changed to the diameter W12, and output data is sent to the calculation unit 20.

  In step S106, it is determined whether or not the output data sent to the calculation unit 20 this time is worse than the previous output data. For example, it is assumed that the observation region 24 is set by the region setting unit 22 of the display unit 21 illustrated in FIG. 4A, and the parameter setting unit 23 is set to increase the contrast of the observation region 24. Contrast calculated based on output data obtained this time (for example, when the illumination area 91 has a diameter W12) is calculated based on output data obtained last time (for example, when the illumination area 91 has a diameter W13). Compare what is worse. If not, the process returns to step S104, the diameter of the illumination area 91 is changed, and the output data is detected (step S105). That is, since the contrast of the observation area 24 is increased, the process returns to step S104 to further change the size of the illumination area 91. On the other hand, if the contrast is worse than the previous time, the contrast is highest when the diameter of the illumination area 91 is the previous time. Therefore, the process proceeds to the next step S107.

  In step S107, an illumination shape suitable for observation of the subject 60 is selected. That is, the illumination shape used immediately before the contrast of the observation region 24 is deteriorated is used for observation of the subject 60, assuming that the illumination shape is suitable for observation of the subject 60.

  In step S104 of the flowchart, the size of the illumination area 91 changed in a similar shape. However, it may be performed not only by changing to a similar shape but also by changing the shape itself. For example, the circular illumination area 91 can be formed little by little so that it finally becomes a triangle, or the circular illumination area 91 can be formed little by little so that it finally becomes an annular shape with a predetermined width.

<Method using genetic algorithm>
Next, a method using a genetic algorithm will be described. The genetic algorithm is a method in which image data is acquired for each of a plurality of illumination shapes prepared in advance, and the illumination shape is searched for by combining the illumination shapes suitable for observation of the subject 60 therein. is there.

FIG. 6 is a flowchart using a genetic algorithm.
First, in step S201, the illumination area 91 of the first spatial light conversion element 90 is first set to an initial size and shape. For example, the default illumination area 91 is circular and has a maximum aperture. In this state, an image of the subject 60 is detected by the image sensor 80.

  In step S202, the region setting unit 22 sets the observation region 24 of the observation image. By step S201, the image of the subject 60 is displayed on the region setting unit 22.

  In step S203, parameters for forming an observation image of the subject 60 are set. In the parameter setting unit 23, the observer can set parameters for inputting the observer's desire regarding the observation image of the subject 60 and allowable observation conditions. Similar to the parameters shown in FIG. 4A, the set parameters include, for example, a specific location of the subject 60, a spatial frequency band and a wavelength band of the subject 60, and the like. However, as shown in FIG. 4B, the observer does not set the illumination area 91 of the first spatial light conversion element 90. The calculation unit 20 arbitrarily sets the illumination area 91 arbitrarily.

  In step S <b> 204, the image sensor 80 detects an image of the subject 60 using two or more initial illumination shapes. Then, the calculation unit 20 obtains each output data of an image of an image obtained by measuring the subject 60 using a plurality of illumination shapes. An example of a plurality of illumination shapes will be described with reference to FIG.

FIG. 7 shows various illumination shapes of the first spatial light conversion element 90. In FIG. 7, a white portion is shown as an illumination area 91 and a shaded area is shown as a light shielding area 92.
FIG. 7A is a schematic plan view of the first spatial light conversion element 90 having a circular illumination shape with a large diameter. As shown in FIG. 7A, the illumination shape of the first spatial light conversion element 90 can be a circle that is axially symmetric with respect to the optical axis of the illumination optical system. FIG. 7B is a schematic plan view of the first spatial light conversion element 90 showing a circular illumination shape with a small diameter. FIG. 7B shows a circular shape that is axisymmetric with respect to the optical axis of the illumination optical system 40 that differs from FIG. 7A only in the size of the illumination shape. The illumination shape of the first spatial light conversion element 90 may include a figure having a shape similar to other figures as shown in FIG. FIG. 7C is a schematic plan view of the first spatial light conversion element 90 having a large annular illumination shape. In FIG. 7C, the central portion is shielded from light in the large circular illumination shape of FIG.

  FIG. 7D is a schematic plan view of a first spatial light conversion element 90 having an illumination shape in which four circular illumination areas 91 having a small diameter are arranged symmetrically with respect to the optical axis. FIG. 7E is a schematic plan view of a first spatial light conversion element 90 having two rectangular illumination areas 91 and having an illumination shape arranged symmetrically with respect to the optical axis.

  FIG. 7F is a schematic plan view of the first spatial light conversion element 90 in which the illumination area 91 is formed non-axisymmetrically with respect to the optical axis. In FIG.7 (f), the illumination shape of the 1st spatial light conversion element 90 is formed in the crescent moon shape, and is non-axisymmetric with respect to the optical axis. Normally, the non-axisymmetric illumination area 91 is often inclined illumination, so that the contrast of the subject 60 can be increased. However, the non-axisymmetric illumination area 91 becomes a biased image by increasing the contrast of only a part of the object, and may not be suitable for the entire observation of the subject 60. Therefore, it is desirable for the genetic algorithm to be able to select whether or not a non-axisymmetric opening is used in the parameter setting unit 23 of the display unit 21 shown in FIG.

  Returning to FIG. 6, in step S205, the output data of the image of the subject 60 obtained in step S204 are compared, and the first illumination light intensity distribution formed by the illumination shape suitable for the most set parameter among these is set. And the second illumination light intensity distribution formed by the second most suitable illumination shape are selected.

  In step S206, the calculation unit 20 forms an illumination shape having a next-generation illumination light intensity distribution from the first illumination intensity distribution and the second illumination intensity distribution using a genetic algorithm crossover or mutation technique. An example of forming an illumination shape having a next-generation illumination light intensity distribution will be described with reference to FIG.

  FIG. 8A is a diagram showing a combination example of the first spatial light conversion elements 90 in FIGS. 7A and 7B. In step S205, the first illumination light intensity distribution is a circular illumination shape having a large diameter as shown in FIG. 7A, and the second illumination light intensity distribution is a circle having a small diameter as shown in FIG. 7B. This is the case of the illumination shape. The calculation unit 20 can form a plurality of new illumination shapes by crossing (combining) the two. The formed illumination shapes are, for example, a first spatial light conversion element 90a having an illumination region 91 having a slightly smaller diameter than the illumination region 91 in FIG. 7A, and slightly more than the illumination region 91 in FIG. 7B. There are a first spatial light conversion element 90b having an illumination area 91 with a large diameter, a first spatial light conversion element 90c in which the illumination area 91 is formed in an elliptical shape, and the like.

  FIG. 8B is a diagram showing a combination example of the first spatial light conversion elements 90 in FIGS. 7A and 7D. In step S205, the first illumination light intensity distribution has a circular shape with a large diameter shown in FIG. 7A, and the second illumination light intensity distribution has four small circular shapes shown in FIG. 7D. This is a case where the illumination region 91 has an illumination shape arranged symmetrically with respect to the optical axis. The calculation unit 20 crosses (combines) the two so that, for example, a first spatial light conversion element 90d having an illumination region 91 in which a part of four circular rings are arranged in axial symmetry with respect to the optical axis, illumination The first spatial light conversion element 90e or the like in which the region 91 is formed in the shape of “x” can be formed.

  FIG. 8A and FIG. 8B are examples of combinations. Actually, since the shape of the first spatial light conversion element 90 is randomly formed, there are an infinite number of newly formed illumination regions 91. Any number of the illumination areas 91 may be newly formed. It is also possible to combine them by other methods. For example, the first spatial light conversion element 90 may be divided into a plurality of fine regions, and operations such as recombination and mutation may be performed on each region. Also, an original function may be created and combined according to the function.

  Returning to FIG. 6, in step S207, the first illumination intensity distribution that is newly optimal for observation of the subject 60 and the second optimum illumination intensity are newly determined from the first illumination intensity distribution, the second illumination intensity distribution, and the next-generation illumination intensity distribution. A second illumination intensity distribution is selected.

  In step S208, it is determined whether crossover or mutation has been performed up to a predetermined generation, for example, 1000 generations. If crossover or the like has not been performed until a predetermined generation, the process returns to step S206 to search for an illumination intensity distribution suitable for observation of the subject. If crossover or the like is performed up to a predetermined generation, the process proceeds to step S209.

  In step S209, an illumination shape of a generation close to the condition desired by the observer is selected from the illumination area 91 obtained by crossing up to a predetermined generation, for example, up to 1000 generations. Thereafter, the first spatial light conversion element 90 having the illumination shape of that generation is used for observation of the subject 60.

<< Object Information Estimation Method 1 >>
When the structure or property of the subject 60 is unknown, it is desirable to obtain information on the structure or property of the subject 60 before deriving an optimal illumination shape for the subject 60. This is because the optimum observation condition can be reliably obtained in a shorter time by using the structure or property of the subject 60 as a reference when estimating the optimum observation condition. Hereinafter, a method for estimating the phase information of the subject 60, the information on the fine structure, and the property information with respect to the wavelength of the illumination light will be described.

<Method 1 for Estimating Object Phase Information>
By observing the subject 60 by changing the shape of the illumination area 91 of the first spatial light conversion element 90, it is estimated whether the subject 60 is a high-contrast intensity object or a low-contrast phase object. Can do. Whether the subject 60 is a phase object or an intensity object can be estimated by irradiating the subject 60 with light having different coherence factor (σ) values. The σ value is defined by σ = NA ′ / NA. NA ′ is the numerical aperture of the illumination optical system 40, and NA is the numerical aperture of the objective lens 71. The numerical aperture NA ′ of the illumination optical system 40 can be controlled by changing the shape of the illumination area 91 of the first spatial light modulator 90. NA ′ can be regarded as having a σ value of 0 by making the illumination area 91 point-shaped (hereinafter referred to as a point light source). In addition, NA ′ is 1 when the illumination area 91 of the first spatial light modulator 90 is circular with a large diameter.

FIG. 9A is a flowchart of the method 1 for estimating the phase information of the subject 60.
First, in step S301, the observer selects phase information on the object information acquisition screen 23e shown on the display unit 21 in FIG. 9B.

  FIG. 9B is a diagram of the area setting unit 22 and the parameter setting unit 23 of the display unit 21 when performing the object information estimation method. First, an object information acquisition screen 23 d is displayed on the parameter setting unit 23 of the display unit 21. The observer selects object information detection 1 when performing object phase information estimation method 1 on the object information acquisition screen 23d, and selects object information detection 2 when performing object phase information estimation method 2 described later. When both the object phase information estimation method 1 and the object phase information estimation method 2 are performed, batch measurement is selected. When the object phase information estimation method 1 is selected, the screen is switched to the object information acquisition screen 23e, and when the object phase information estimation method 2 is selected, the screen is switched to the object information acquisition screen 23f. Furthermore, the object information acquisition screen 23g is a screen that is switched from the object information acquisition screen 23e.

  On the object information acquisition screen 23e, items of phase information 1, fine structure, wavelength property and batch measurement are displayed. Here, when the phase information 1 is selected, the calculation unit 20 performs the estimation method 1 of the phase information of the object, and when the fine structure is selected, the calculation unit 20 performs the estimation method of the information of the fine structure of the object. When the property with respect to the wavelength is selected, the calculation unit 20 performs a property information estimation method with respect to the wavelength of the object. If batch measurement is selected, all of these items are estimated by the calculation unit 20. After each selection item is selected, information on the automatically selected item is acquired.

  Returning to FIG. 9A, in step S302, the shape of the illumination area 91 of the first spatial light modulator 90 is formed as a point light source (σ≈0), and an image of the subject 60 by illumination of the point light source is image sensor 80. Is detected. When the irradiation light is in a coherent state, contrast is observed on the subject 60 regardless of whether the subject 60 is a phase object or an intensity object.

  Next, in step S303, the shape of the illumination area 91 of the first spatial light modulator 90 is formed into a circular shape with a large diameter (σ≈1), and the image of the subject 60 by the large circular illumination is an image sensor. 80 detected. When the irradiation light is incoherent, the subject 60 can be observed with contrast when the subject 60 is an intense object, but the subject 60 has contrast when the subject 60 is a phase object. Cannot be observed.

  Next, in step S304, it is estimated whether the subject 60 is a phase object or an intensity object. If there is no change between the image by the coherent light detected in step S302 and the image by the incoherent light detected in step S303, the subject 60 is estimated to be an intense object. If the image by the coherent light detected in step S302 is different from the image by the incoherent light detected in step S303, it is estimated that the subject 60 is a phase object.

  Next, in step S305, the shape of the illumination region 91 suitable for observation of the subject 60 is estimated. For the phase object, the calculation unit 20 sets the shape of the illumination area 91 of the first spatial light modulator 90 to a small or inclined illumination (see, for example, FIG. 7F). This is because when the subject 60 is a phase object, the shape of the illumination region 91 is reduced or inclined illumination is suitable for observation of the subject 60. When the subject 60 is an intense object, the calculation unit 20 increases the diameter of the circle of the illumination area 91 of the first spatial light modulator 90. This is because a stronger object is easier to observe when the amount of light is larger.

<Method for estimating fine structure information of an object>
Whether or not the subject 60 includes a fine structure can also be estimated by observing the subject 60 while changing the shape of the illumination region 91 of the first spatial light conversion element 90.

  FIG. 9C is a flowchart of a method for estimating fine structure information of the subject 60. First, in step S311, the observer selects fine structure information on the object information acquisition screen 23e shown on the display unit 21 in FIG. 9B.

  Next, in step S <b> 312, the shape of the illumination area 91 of the first spatial light modulator 90 is formed as a point light source, and the image of the subject 60 is detected by the image sensor 80. When the shape of the illumination area 91 of the first spatial light modulator 90 is a point light source (σ = 0), even if the subject 60 includes a fine structure, the fine structure does not appear in the image of the subject 60. .

  Next, in step S313, the shape of the illumination region 91 of the first spatial light conversion element 90 is formed in an annular shape, and the image of the subject 60 is detected by the image sensor 80. At this time, it is preferable that the outer shape of the annular shape is large. When the shape of the illumination area 91 is an annular shape, the fine structure is detected when the subject 60 includes a fine structure.

  Next, in step S314, it is estimated whether or not the subject 60 includes a fine structure. The calculation unit 20 determines that the subject 60 does not include a fine structure when the image of the subject 60 does not change between when the illumination region 91 is a point light source and when the illumination region 91 is an annular shape. On the other hand, the calculation unit 20 detects the image of the subject 60 with the illumination region 91 having an annular shape, depending on whether the illumination region 91 is a point light source or an annular shape. If so, it is determined that the subject 60 includes a fine structure.

  Next, in step S315, the shape of the illumination area 91 suitable for observation of the subject 60 is estimated. For example, when the subject 60 has a fine structure, the illumination area 91 preferably has an annular shape.

<Method for estimating property information with respect to wavelength of illumination light of object>
The subject 60 may show different output data due to the structure and properties of the subject 60 when the wavelength of the illumination light is changed. Therefore, it is desirable to grasp the property of the subject 60 with respect to the wavelength of the illumination light.

FIG. 9D is a flowchart illustrating a method for estimating property information with respect to the wavelength of illumination light of the subject 60.
First, in step S321, the observer selects property information with respect to the wavelength on the object information acquisition screen 23e shown in the display unit 21 of FIG. 9B.

  Next, in step S <b> 322, the illumination light applied to the subject 60 is changed to monochromatic light, and the image of the subject 60 is detected by the image sensor 80. For example, a description will be given assuming that the illumination light source 30 is an LED having three color light sources of red, blue, and green. In this case, for example, only the green LED is turned on, and the LEDs of other wavelengths are turned off. Then, the image of the subject 60 is detected by the image sensor 80.

  Next, in step S323, it is determined whether images of the subject 60 have been detected at all wavelengths. For example, if an image of the subject 60 is detected at each of red, blue, and green wavelengths, the process proceeds to step S325. If there is a wavelength for which an image of the subject 60 has not yet been detected, the process proceeds to step S324.

  Next, in step S 324, a wavelength for which the image of the subject 60 has not yet been acquired is selected as the wavelength of the illumination light source 30, and the image of the subject 60 is detected by the image sensor 80. For example, when only the image of the subject 60 with a green wavelength has been acquired, only one of red and blue is turned on, and the LEDs of other wavelengths are turned off. In this state, an image of the subject 60 is detected by the image sensor 80. Thereafter, the process returns to step S323 to check whether or not the image of the subject 60 has been detected at all wavelengths.

  In step S325, the property of the subject 60 with respect to the wavelength of the illumination light is estimated. The images of the subject 60 detected in steps S322 and S324 are compared. For example, if the image of the subject 60 detected at the blue wavelength has a better contrast than the image of the subject 60 at other wavelengths, the calculation unit 20 has a better contrast with respect to the blue wavelength. Judge that.

  Next, in step S <b> 326, the calculation unit 20 estimates illumination light having a wavelength that is optimal for observation of the subject 60. For example, when the subject 60 is desired to be observed with the highest contrast, when the blue wavelength is used in step S325, the subject 60 was observed with a contrast higher than that of the image of the subject 60 detected at other wavelengths. In this case, the calculation unit 20 determines that the illumination light with the blue wavelength is suitable for observing the subject 60.

  In this method of estimating the property information with respect to the wavelength of the illumination light of the object, the shape of the illumination region 91 may be any shape, but together with the above-described method 1 of estimating the phase information of the object and the method of estimating the fine structure information of the object In some cases, the phase information and fine structure information of the object can be estimated more reliably. In this case, after the phase information or the fine structure information is selected on the object information acquisition screen 23e shown in FIG. 9B, the wavelength is changed so as to switch to the object information acquisition screen 23g shown in FIG. 9B. It may be possible to select whether or not to change.

  By performing the object information estimation method 1 shown above before the illumination shape deriving method described above, the illumination shape deriving time can be shortened.

<< Object Information Estimation Method 2 >>
In the flowcharts shown in FIGS. 3 and 6, parameters for forming an observation image of the subject 60 are set in steps S103 and S203. If the object information of the subject 60 is obtained in advance, such as the line width of a semiconductor integrated circuit or the like, the observer can set parameters from that information. However, when the subject 60 is a living body or the like, the object information of the subject 60 is often not obtained, and the observer may not know how to set the parameters. Further, the information obtained by the object information estimation method 1 may still be insufficient. In such a case, the object information may be examined in more detail before determining the illumination shape. Hereinafter, the method 2 for estimating the phase information of the object and the method for detecting the spatial frequency information of the object will be described.

  The object information estimation method 2 is performed when the observer selects the object information detection 2 on the object information acquisition screen 23d in FIG. 9B. After the object information detection 2 is selected, the screen is switched to the object information acquisition screen 23f. On the object information acquisition screen 23f, items of phase information 2, spatial frequency information, and batch measurement are displayed. Here, when the phase information 2 is selected, the object phase information estimation method 2 is performed, and the object spatial frequency information detection method is performed. In addition, when batch measurement is selected, all of these items are estimated. After each selection item is selected, information on the automatically selected item is acquired.

<Object Phase Information Estimation Method 2>
The phase information of the object is estimated by measuring the subject 60 by setting the size to a minimum so that the illumination area 91 of the first spatial light modulator 90 becomes a point light source and using the illumination light as monochromatic light. Can do. As for the wavelength of this monochromatic light, when the optimum wavelength for observation of the subject 60 is estimated by the object information estimation method 1 or the like, it is desirable that the wavelength be the wavelength of the illumination light.

  The phase information estimation method 2 of the object is performed by selecting the phase information 2 on the object information acquisition screen 23f of FIG. 9B. Hereinafter, the phase information estimation method 2 of the subject 60 will be described with reference to FIG. 10A.

FIG. 10A is a flowchart of the phase information estimation method 2 of the subject 60.
First, in step S401, the observer selects phase information 2 on the object information acquisition screen 23f of FIG. 9B. Thereafter, the display unit 21 is switched to a screen shown in FIG. 10B described later.

  First, in step S402, the observer designates the number of point light sources and the formation position of each point light source through the screen shown in the display unit 21 of FIG. 10B. Below, with reference to FIG. 10B, the number of point light sources and the example of the formation position of each point light source are shown.

  FIG. 10B is a diagram of the display unit 21 in which a schematic diagram of the first spatial light conversion element 90 is shown. FIG. 10B shows a state in which the light transmitted through the first spatial light conversion element 90 has a maximum aperture by a circle 31 indicated by a dotted line in the region setting unit 22 of the display unit 21. The point light source (σ≈0) used for estimating the phase information of the subject 60 is shaped to a size that can be regarded as a point light source or a point light source. That is, the calculation unit 20 causes the first spatial light conversion element 90 to form the dotted illumination region 91 inside the circle 31 indicated by the dotted line. In FIG. 10B, the number of point light sources measured by the parameter setting unit 23 of the display unit 21 and the wavelength of light to be used can be set. When the number of point light sources is five, for example, a total of five point light sources are formed at the center point (on the optical axis) of the circle 31 and the point where the positive and negative values of the X and Y axes are taken. Further, the wavelength of light can be input. Only one wavelength may be input, or when a plurality of wavelengths are input, five point light sources are measured for each wavelength.

  Point light source 32a is the point light source at the center of the circle 31, point light sources are the point light sources 32b and 32d that take the maximum positive and negative values on the X axis, and point light sources are the point light sources that take the maximum positive and negative values on the Y axis. The light sources 32c and 32e are indicated by black dots. The point light source preferably includes a point light source formed near the outermost peripheral portion of the circle 31. This is because coherent light can be incident on the subject 60 from various angles, and diffracted light during oblique illumination can be obtained.

  Instead of setting the number of point light sources and the position of each point light source by the observer, the calculation unit 20 may automatically set the number of point light sources shown in FIG. 10B to 5 points. .

  Next, in step S403, the wavelength of the illumination light is changed to a single color, and the first spatial light conversion element 90 forms a point light source at a predetermined position. The monochromatic illumination light is formed by a wavelength filter 44 that transmits only light of a specific wavelength to the white illumination light source 30, for example. Further, the first spatial light conversion element 90 is formed with an illumination area 91 having the size of one point light source as shown in FIG. 10B. Since the point light source and monochromatic wavelength light are coherent and have high coherence, it is convenient for estimating the phase information of the subject 60.

In step S <b> 404, the image sensor 80 detects an image of the subject 60 with a point light source and monochromatic wavelength light.
In step S405, the calculation unit 20 determines whether images with all point light source positions and light having a monochromatic wavelength have been acquired. For example, if measurement has not been performed with all of the five point light sources shown in FIG. 10B, the process proceeds to step S406. If all the specified point light sources have been measured, the process proceeds to step S407.

  In step S406, the calculation unit 20 changes the position of the illumination area 91 serving as a point light source. For example, when the point light source 32a of FIG. 10B is measured in step S404 and the point light source 32b is not measured, the position of the illumination area 91 is formed only at the position of the point light source 32b. Thereafter, the process proceeds to step S404.

  In step S407, the calculation unit 20 analyzes the diffracted light of the subject 60. For example, from the analyzed diffracted light information, it is possible to know the distribution of diffracted light that causes a specific spatial frequency component in the observation region 24, and thereby to find an illumination shape suitable for observation more efficiently.

  In step S408, a more suitable illumination condition is set by the calculation unit 20. The analyzed diffracted light information is displayed on the display unit 21, and the observer estimates phase information of the subject 60 with reference to the diffracted light information. The observer can set the observation region 24 and parameters in steps S102 and S103 in FIG. 3 and steps S202 and S203 in FIG. 6 from the estimated phase information.

  In the above-described method 2 for estimating the phase information of the subject 60, a plurality of wavelengths may be measured. Therefore, a step of confirming whether or not all the wavelengths set in FIG. 10B have been measured may be formed in the flowchart shown in FIG. 10A.

<Detection method of spatial frequency information of object>
The spatial frequency indicates a repetition cycle of the unit length of the subject 60. That is, there is a high possibility that similar structures are gathered at a place where the same spatial frequency is gathered. Therefore, the spatial frequency information of the subject 60 is a reference for setting the observation region 24 and parameters in steps S102 and S103 in FIG. 3 and steps S202 and S203 in FIG. Detection of spatial frequency information of the subject 60 is performed by acquiring output data of a pupil image of the imaging optical system 70. In addition, the subject 60 is measured using a monochromatic wavelength point light source as described in the method 2 for estimating the phase information of the subject 60. Hereinafter, a method for detecting spatial frequency information of the subject 60 will be described with reference to FIGS. 11 and 12.

  FIG. 11 is a schematic configuration diagram of the microscope system 200. Hereinafter, the same members as those in the microscope system 100 described in FIG. 1 are denoted by the same reference numerals, and the description of the members is omitted.

  In the microscope system 200, a beam splitter 272 is disposed at or near the position of the pupil 273 of the microscope system 100. In addition, the microscope system 200 includes a second image sensor 280 disposed at a position conjugate to the position of the relay lens 243 and the pupil 273 that relays the branched light LW21. The beam splitter 272 branches the light from the imaging optical system 70. The branched light LW21 enters the second image sensor 280 through the relay lens 243. The light LW21 passes through the relay lens 243 to become the light LW22, and the light LW22 forms an image of the pupil 273 on the second image sensor 280. Information on the image of the pupil 273 formed on the second image sensor 280 is sent to the calculation unit 20 and analyzed.

FIG. 12 is a flowchart of a method for detecting spatial frequency information of the subject 60.
First, in step S501, the observer selects spatial frequency information on the object information acquisition screen 23f of FIG. 9B. Thereafter, the display unit 21 is switched to the screen shown in FIG. 10B.

Next, in step S502, the observer designates the number of point light sources (σ = 0) and the formation position of each point light source through the display unit 21.
Next, in step S503, the wavelength of the illumination light is changed to a single color, and an opening having a size close to a point light source is formed at a predetermined position.

  Steps S502 and S503 are the same as steps S402 and S403 in FIG. 10A. Further, an example in which five point light sources are formed as in the case of FIG. 10B will be described.

  In step S <b> 504, the image of the subject 60 on the pupil 273 is detected by the second image sensor 280. For example, if the point light source 32a shown in FIG. 10B is designated in step S503, the second image sensor 280 detects the image of the subject 60 using only the point light source 32a as an illumination light source.

  FIG. 13A is a diagram of the display unit 21 on which an image of the image of the pupil 273 detected by the second image sensor 280 when the subject 60 is an integrated circuit (IC). For example, the image is displayed on the area setting unit 22 of the display unit 21. A circle 233 indicated by a dotted line in FIG. 13A is assumed to be a range through which light can pass. Image data detected by the second image sensor 280 is a light intensity distribution in the pupil 273. The light intensity distribution can be shown as a point 234 in FIG. 13, for example. The position of the point 234 is a signal detection position, and its size reflects the size of the point light source. In FIG. 13A, the black point 234 has a strong detected signal, the white point 234 has a weak detected signal, and the gray point 234 has a detected signal intermediate between the black point 234 and the white point 234. It shows strength. The point 234 is actually displayed small and has almost no size. However, in FIG. 13A, for the purpose of explanation, the point 234 has a size, and the color of the point 234 is changed to indicate the intensity of the signal. In FIG. 13A, black dots 234 are gathered in the upper right area of the image, and white dots 234 are gathered in the lower left part of the screen. This indicates that the spatial frequency in the upper right part of the screen is large and the spatial frequency in the lower left part is small. Further, since the IC has a periodic structure, the points 234 detected by the second image sensor 280 are easily detected periodically.

  FIG. 13B is a diagram of the display unit 21 on which an image of the image of the pupil 273 detected by the second image sensor 280 when the subject 60 is a living body. In FIG. 13B, a point 234 detected by the second image sensor 280 is shown on the display unit 21. In FIG. 13B, the point 234 is shown as a point having no size. A point 234 shown in FIG. 13B has a signal having a different intensity at each point as in FIG. When the subject 60 is a living body, as indicated by a point 234 in FIG. 13B, the point 234 does not have periodicity and is often shown randomly. This is because the living body has fewer structures having periodicity than the IC having the periodic structure shown in FIG.

  In step S505, it is determined whether or not the information of the positions of all point light sources, for example, the light intensity information of five points, has been acquired. If information on the light intensity of all point light sources has not been acquired, the process proceeds to step S506. If the information on the light intensities of all point light sources has been acquired, the process proceeds to step S507.

  In step S506, the position of the illumination area 91 to be a point light source is changed. For example, when the point light source 32a of FIG. 10B is measured in step S504 and the point light source 32b is not measured, the position of the illumination area 91 is formed only at the position of the point light source 32b. Thereafter, the process proceeds to step S504.

  In step S507, the Fourier spectrum of the subject 60 is measured, and the spatial frequency distribution of the subject 60 is obtained. The spatial frequency distribution may be shown as a light intensity distribution as shown in FIG. 13, or may be shown on the display unit 21 by converting the light intensity distribution into a spatial frequency. The periodicity of the structure of the subject 60 is calculated from the spatial frequency distribution of the subject 60. As shown in FIG. 13B, when the spatial frequency distribution of the subject 60 is random, the periodicity of the structure of the subject 60 cannot be calculated.

  In step S508, the calculation unit 20 sets illumination conditions suitable for observation of the subject 60. Further, the result may be displayed on the display unit 21. The observer can set parameters or set the observation region 24 on the display unit 21 based on the analyzed spatial frequency information of the subject 60 (see FIG. 4). In step S102 and step S103 in FIG. 3, and in step S202 and step S203 in FIG. 6, the observer sets parameters and sets the observation region 24. For example, a collection of specific spatial frequencies may indicate the same structure. When it is desired to observe only this structure, the parameter setting unit 23 of the display unit 21 sets the spatial frequency of the structure to be observed, thereby adjusting the observation image of the subject 60 according to the spatial frequency. it can.

  Although the subject 60 information is detected by the method as described above, the calculation unit 20 can automatically set an illumination shape suitable for observation of the subject, but various modifications are added to the embodiment. Can do.

  For example, in the microscope system 200 shown in FIG. 11, two information of the image of the imaging plane and the image of the pupil 273 can be acquired simultaneously by using two image sensors at the same time. By inserting a detachable relay lens between the sensor 80 and forming a conjugate image on the pupil 273 on the image sensor 80, the spatial frequency information of the subject 60 can be acquired by using only one image sensor. be able to.

  Further, in the microscope system 200, it is also possible to check the amplitude information of the pupil and acquire the phase information of the subject 60 by constructing an interferometer in the microscope system 200 to acquire an interference image of the pupil. It is. The interferometer obtains the Fourier spectrum of the object by causing the second image sensor 280 to measure the interference image by causing the object light transmitted through the subject 60 and the reference light not transmitted through the subject 60 to interfere with each other. Therefore, the phase information of the object can be acquired. In the case of forming an interferometer, it is desirable to use a laser or the like for the illumination light source 30. By using a laser, strong light of a single color can be obtained, so that the size of the point light source can be further reduced. Further, it is also possible to form a three-dimensional image of the subject 60 by detecting an interference image between the diffracted light of the object light and the reference light from a plurality of irradiation directions by the image sensor 80. Details of a microscope using an interferometer are disclosed, for example, in Table 2008/123408.

  Further, the point light source shown in FIG. 10B may be formed using a mask having a point opening instead of the first spatial light modulator 90.

  Further, the point light sources shown in FIG. 10B are formed more along the outer periphery of the circle 31 so that diffracted light or spatial frequency information when the subject 60 is illuminated obliquely from more directions is formed. Can be obtained. Further, the plurality of point light sources shown in FIG. 10B may be measured at a plurality of single wavelengths, for example, red, blue, and green wavelengths. A case where each point light source shown in FIG. 10B is measured in red, blue, and green will be described with reference to FIG.

FIG. 13C is a diagram of the display unit 21 on which images of the pupil 273 at red, blue, and green wavelengths detected by the second image sensor 280 when the subject 60 is a living body are shown. is there. FIG. 13C is a diagram in the case where the subject 60 is measured using, for example, the red, blue, and green wavelengths with the point light source 32b of FIG. 10B. In actual measurement, the image of the pupil 273 is measured with all of the point light sources 32a to 32e. In FIG. 13C, an image 234a detected with a red wavelength, an image 234b detected with a blue wavelength, and an image 234c detected with a green image are shown on the same screen. Since the spatial frequency is inversely proportional to the wavelength, the spatial frequency distribution of the subject 60 can be examined more accurately by measuring the image of the subject 60 on the pupil 273 for each wavelength of light. In FIG. 13C, the region where the image 234a detected at the red wavelength is shown has a relatively small spatial frequency, and the region where the image 234b detected at the blue wavelength is shown is a relatively spatial frequency. The existence of a large structure is estimated.
According to the first embodiment, there is provided a microscope system that derives and forms an intensity distribution of illumination light suitable for observing an image of an object under observation in a good state.

(Second embodiment)
In the first embodiment, the microscope system 100 having a bright field microscope has been described. In the second embodiment, a microscope system 300 having a phase contrast microscope will be described.

<Microscope system 300>
FIG. 14A is a schematic configuration diagram of the microscope system 300. The microscope system 300 is an optical microscope system for observing the subject 60. The microscope system 300 mainly includes an illumination light source 30, an illumination optical system 40, an imaging optical system 70, an image sensor 80, and a calculation unit 20. The illumination optical system 40 includes a first condenser lens, a wavelength filter 44, a first spatial light modulation element 390, and a second condenser lens 42. The imaging optical system 70 includes an objective lens 71 and a second spatial light modulation element. 396. A stage 50 is disposed between the illumination optical system 40 and the imaging optical system 70, and the subject 60 is disposed on the stage 50.

  The second spatial light modulator 396 is disposed at or near the pupil position of the imaging optical system 70. Further, the first spatial light modulation element 390 is disposed at a position conjugate with the pupil of the imaging optical system 70 in the illumination optical system 40. The first spatial light modulator 390 is an element that can arbitrarily change the intensity distribution of transmitted light, and is configured by a liquid crystal panel, DMD, or the like. The second spatial light modulator 396 is configured by a liquid crystal panel or the like that is an element capable of changing the phase. Further, it is desirable that the second spatial light modulation element has a configuration in which the light intensity distribution can be freely changed with the phase.

  In FIG. 14A, the light emitted from the illumination light source 30 is indicated by a dotted line. The illumination light LW31 emitted from the illumination light source 30 is converted into light LW32 by the first condenser lens 41. The light LW32 is incident on the first spatial light modulator 390. The light LW33 that has passed through the first spatial light modulator 390 passes through the second condenser lens 42 to become the light LW34, and travels toward the subject 60. The light LW35 that has passed through the subject 60 passes through the objective lens 71 to become light LW36 and enters the second spatial light modulation element 396. The light LW 36 passes through the second spatial light modulator 396 and becomes the light LW 37 and forms an image on the image sensor 80. Output data of the image formed on the image sensor 80 is sent to the calculation unit 20. In the calculation unit 20, based on the output data of the image obtained from the image sensor 80, the shape data of the opening 391 formed by the first spatial light modulator 390, and the shape data of the second spatial light modulator 396. The optimal illumination shape for the subject 60 is calculated. Then, the calculated illumination shape suitable for observation of the subject 60 is transmitted to the first spatial light modulation element 390 and the second spatial light modulation element 396. When the wavelength filter 44 is disposed, only light having a specific wavelength is transmitted through the wavelength filter 44 and is incident on the first spatial light modulator 390.

  FIG. 14B is a plan view of the first spatial light modulator 390. In the first spatial light modulator 390, a light transmission region (illumination region) 391 is formed in a ring shape, and a region other than the transmission region 391 is a light shielding region 392.

  FIG. 14C is a plan view of the second spatial light modulator 396. A phase modulation region 397 is formed in a ring shape in the second spatial light modulation element 396, and the phase of light transmitted through the phase modulation region 397 is shifted by a quarter wavelength. The light transmitted through the diffracted light transmission region 398 other than the phase modulation region 397 has the same phase. The phase modulation region 397 is formed so as to be conjugate with the transmission region 391 of the first spatial light modulation element 390.

  The zero-order light (transmitted light) of the microscope system 300 passes through the transmission region 391 of the first spatial light modulation element 390, passes through the phase modulation region 397 of the second spatial light modulation element 396, and reaches the image sensor 80. Further, the diffracted light emitted from the subject 60 passes through the diffracted light transmission region 398 of the second spatial light modulator 396 and reaches the image sensor 80. Then, the zero-order light and the diffracted light form an image on the image sensor 80. In general, since the 0th-order light has a higher light intensity than the diffracted light, it is desirable to form a filter for adjusting the light intensity in the phase modulation region 397. This filter includes, for example, an optical element that can freely vary the spatial distribution of transmittance as shown in an electrically controllable optical element (for example, Japanese Translation of PCT International Application No. 2010-507119) including an array of cells in the second space. It can be formed by adding to the modulation element 396.

The first spatial light modulation element 390 and the second spatial light modulation element 396 can freely change the size and shape of the transmission region 391 and the phase modulation region 397. For example, when the diameter of the transmission region 391 of the first spatial light modulator 390 is increased, the numerical aperture of the transmitted light is increased, so that the resolution can be increased. Further, the shape of the transmission region 391 of the first spatial light modulator 390 may be optimized by using the illumination shape deriving method shown in the first embodiment. The ring-shaped region 397 of the second spatial light modulator 396 is always formed so as to be conjugate with the transmission region 391 of the first spatial light modulator 390. Therefore, it is desirable that the transmission region 391 and the ring-shaped region 397 change in shape in synchronization.
According to the second embodiment, a microscope system that derives and forms an intensity distribution of illumination light suitable for observing an image of an object under observation in a good state is provided.

  The optimum embodiment of the present invention has been described above, but as will be apparent to those skilled in the art, the present invention can be implemented with various modifications within the technical scope thereof.

DESCRIPTION OF SYMBOLS 20 ... Calculation part 21 ... Display part 22 ... Area setting part 23 ... Parameter setting part 23a-23g ... Object information acquisition screen 24 ... Observation area 30 ... Illumination light source 32 ... Point light source 40 ... Illumination optical system 41 ... 1st condenser lens, DESCRIPTION OF SYMBOLS 42 ... 2nd condenser lens 44 ... Wavelength filter 50 ... Stage 60 ... Subject 70 ... Imaging optical system, 71 ... Objective lens 80 ... Image sensor 90, 390 ... 1st spatial light modulator 91 ... Illumination area 92 ... Light-shielding part 93 ... Liquid crystal panel 94 ... Digital micromirror device (DMD)
DESCRIPTION OF SYMBOLS 100, 200, 300 ... Microscope system 242 ... Relay lens 272 ... Beam splitter 273 ... Pupil 280 ... 2nd image sensor 396 ... 2nd spatial light modulation element 397 ... Phase modulation area | region 398 ... Diffracted light transmission area | region

Claims (5)

An illumination light source for illuminating the subject with illumination light;
An imaging optical system that forms an image of transmitted light or reflected light of the subject;
An illumination optical system having a first spatial light modulation element that changes an intensity distribution of the illumination light at a conjugate position of a pupil of the imaging optical system, and irradiating the subject with light from the illumination light source;
An image sensor for detecting light via the imaging optical system;
After the intensity distribution of the illumination light is changed to a point light source (σ = 0) and an annular shape by the first spatial light modulator, the intensity distribution of the point light source (σ = 0) is changed to the image. Comparing the output data detected by the sensor with the output data detected by the image sensor after changing to the intensity distribution of the annular shape, determining whether the subject includes a fine structure , A calculation unit for calculating the intensity distribution of the illumination light suitable for the subject ;
A microscope system comprising: The microscope system according to claim 1 , wherein when the object is estimated to include a fine structure, the calculation unit makes the intensity distribution of the illumination light an annular shape . The calculation unit outputs the output data detected by the image sensor after changing to the intensity distribution of the point light source (σ = 0) and the output detected by the image sensor after changing to the intensity distribution of the annular shape. When the data is different, it is estimated that the subject includes a fine structure, the output data detected by the image sensor after changing to the intensity distribution of the point light source (σ = 0), and the annular shape The microscope system according to claim 1 or 2, wherein when the output data detected by the image sensor after changing to an intensity distribution is the same, the object is estimated not to include a fine structure. Said computing unit, the microscope system according to any one of claims 1 to 3, further wherein the subject to determine the nature versus wavelength.
The illumination light source can irradiate illumination light of a first wavelength and illumination light of a second wavelength different from the first wavelength,
The image sensor detects the subject irradiated with illumination light of the first wavelength and illumination light of the second wavelength,
The calculation unit calculates a contrast between output data detected by the image sensor by irradiating the illumination light of the first wavelength and output data detected by the image sensor by irradiating the illumination light of the second wavelength. 5. The microscope system according to claim 4 , wherein the subject determines that the contrast of the illumination light of the first wavelength is good and estimates that the illumination light of the first wavelength is suitable for the subject .