JP2020071393A - Image simulation device, and image simulation method - Google Patents

Abstract

To provide an image simulation device with which it is possible to generate an image of a sample with a small amount of calculation.SOLUTION: An image simulation device 1 is of an imaging optical system having an illumination optical system and an observation optical system arranged sandwiching a sample with the illumination optical system, the image simulation device 1 including a processor 2. The processor 2 executes a process comprising the steps of: calculating a plurality of first wavefronts emitted from a plurality of light sources in which the pupil intensity distribution of the illumination optical system is modeled; calculating a plurality of second wavefronts after the plurality of first wavefronts have passed through the sample; calculating, from the plurality of second wavefronts, a plurality of third wavefronts at the focus position on the sample side of the observation optical system; calculating a plurality of fourth wavefronts at the imaging position of the observation optical system using the plurality of third wavefronts and the pupil function of the observation optical system and calculating a plurality of first intensity distributions from the plurality of fourth wavefronts; and calculating the intensity distribution of optical image of the sample from the plurality of first intensity distributions.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image simulation device and an image simulation method.

  In recent years, cell masses have been drawing attention in the fields of regenerative medicine and drug discovery. In order to effectively use the cell mass, quality control of the cell mass is important. The quality control of the cell mass is performed by, for example, an optical microscope.

  In order to control the quality of cell clusters with an optical microscope, a technique for simulating cell cluster microscopic images is required. A cell mass is formed by a plurality of cells, and light is diffracted multiple times while passing through the cell mass.

  As a method for simulating an optical image of a sample that is diffracted multiple times, there is a method using an FDTD (finite-difference time-domain) method. The FDTD method can handle multiple times of diffraction. Non-Patent Document 1 discloses that a simulation is performed using the FDTD method.

Numerical simulation of partially coherent broadband optical imaging using the finite-difference time-domain method, Optics Letters Vol. 36, Issue 9, pp. 1596-1598 (2011)

  In the image simulation using the FDTD method, the voxel size is reduced to about 1/10 of the wavelength. In this case, since the number of voxels increases, the amount of calculation becomes huge and the calculation time becomes long.

  As the number of voxels increases, the size of memory required for calculation also increases. Therefore, the image simulation using the FDTD method cannot handle a large sample due to the memory size constraint.

  The present invention has been made in view of such problems, and an object thereof is to provide an image simulation apparatus and an image simulation method that can generate an image of a sample with a small amount of calculation.

In order to solve the above-mentioned problems and achieve the object, an image simulation apparatus according to at least some embodiments of the present invention is
An image simulation apparatus for generating a simulation image of an imaging optical system having an illumination optical system, and an observation optical system that is arranged so as to face the illumination optical system with a sample in between,
Equipped with a processor,
The processor is
Calculating a plurality of first wavefronts emitted from a plurality of light sources modeling the intensity distribution of the pupil of the illumination optical system,
Calculating a plurality of second wavefronts after the plurality of first wavefronts has passed through the sample;
Calculating a plurality of third wavefronts at a focus position on the sample side of the observation optical system from the plurality of second wavefronts,
A plurality of fourth wavefronts at the imaging position of the observation optical system are calculated using the plurality of third wavefronts and the pupil function of the observation optical system, and the plurality of fourth wavefronts are each squared to obtain a plurality of first intensity distributions. The step of calculating
Calculating the intensity distribution of the optical image of the sample by adding a plurality of first intensity distributions;
It is characterized by performing a process including.

An image simulation method according to at least some embodiments of the present invention comprises
An image simulation method for generating a simulation image of an imaging optical system having an illumination optical system and an observation optical system that is arranged to face the illumination optical system with a sample sandwiched between them.
Calculating a plurality of first wavefronts emitted from a plurality of light sources that model the intensity distribution of the pupil of the illumination optical system,
Calculating a plurality of second wavefronts after the plurality of first wavefronts has passed through the sample,
Calculating a plurality of third wavefronts at the focus position on the sample side of the observation optical system from the plurality of second wavefronts,
A plurality of fourth wavefronts at the imaging position of the observation optical system are calculated using the plurality of third wavefronts and the pupil function of the observation optical system, and the plurality of fourth wavefronts are each squared to obtain a plurality of first intensity distributions. And calculate
It is characterized in that the intensity distribution of the optical image of the sample is calculated by adding a plurality of first intensity distributions.

  According to the present invention, it is possible to provide an image simulation apparatus and an image simulation method that can generate an image of a sample with a small amount of calculation.

It is a figure which shows the image simulation apparatus of this embodiment. It is a figure which shows the example of an imaging optical system. It is a flow chart of processing performed by a processor. It is a figure which shows the state of a sample and a wave front. It is a figure which shows the mode of the wavefront in two planes. It is a figure which shows the optical system used for DIC observation. It is a figure which shows the optical system used for IVC observation. It is a figure which shows a mode that the pupil of the illumination optical system was divided. It is a figure which shows the image in IVC observation. It is a figure which shows the optical system used for bright field observation. It is a figure which shows a mode that the pupil of the illumination optical system was divided. It is a figure which shows the image in bright field observation. It is a figure which shows the image in phase contrast observation. It is a figure which shows the image in DIC observation. It is a figure which shows the example of PCF. It is a figure which shows the optical system used for observation of PCF. It is a figure which shows the image in bright field observation. It is a figure which shows a mode that the pupil of the illumination optical system was divided.

  Prior to the description of the examples, the operation and effect of the embodiment according to an aspect of the present invention will be described. In addition, when concretely explaining the operation and effect of the present embodiment, a concrete example will be shown. However, as in the case of the examples described below, those exemplified modes are merely a part of the modes included in the present invention, and there are many variations in the modes. Therefore, the present invention is not limited to the illustrated embodiments.

  The image simulation apparatus according to the present embodiment is an image simulation apparatus that generates a simulation image of an imaging optical system that includes an illumination optical system and an observation optical system that is arranged opposite to the illumination optical system with a sample interposed therebetween. , A processor, the processor calculating a plurality of first wavefronts emitted from a plurality of light sources modeling the intensity distribution of the pupil of the illumination optical system, and the processor after the plurality of first wavefronts pass through the sample. A step of calculating a plurality of second wavefronts, a step of calculating a plurality of third wavefronts at a focus position on the sample side of the observation optical system from the plurality of second wavefronts, a plurality of third wavefronts and a pupil function of the observation optical system Calculating a plurality of fourth wavefronts at the image forming position of the observation optical system by using, and calculating a plurality of first intensity distributions by squaring the plurality of fourth wavefronts, and a plurality of first intensity distributions. By adding the fabric, and carrying out the steps of calculating the intensity distribution of an optical image of the sample, the process comprising.

  The image simulation device of this embodiment is shown in FIG. The image simulation device 1 includes a processor 2. The processor 2 generates a simulation image of the image pickup optical system. The generated simulation image of the imaging optical system can be displayed on the display device 16, for example.

  An example of the imaging optical system is shown in FIG. The imaging optical system 3 has an illumination optical system 4 and an observation optical system 5. The observation optical system 5 faces the illumination optical system 4 with the sample 6 interposed therebetween.

  The illumination optical system 4 has a lens 7, a lens 8 and a condenser lens 9. The observation optical system 5 has an objective lens 10 and an imaging lens 11.

  The imaging optical system 3 is used together with the light source 12. The light emitted from the light source 12 is applied to the sample 6 via the illumination optical system 4. Thereby, the sample 6 is illuminated. The light from the sample 6 is condensed by the observation optical system 5. Thereby, the optical image 13 is formed.

  The lens 7 and the lens 8 form a light source image 12 'at the position of the pupil 14 of the illumination optical system. Thus, the light source 12 and the pupil 14 of the illumination optical system are conjugate.

  The condenser lens 9 and the objective lens 10 form an image of the pupil 14 of the illumination optical system at the position of the pupil 15 of the observation optical system. In this way, the pupil 14 of the illumination optical system and the pupil 15 of the observation optical system are conjugate.

  The sample 6 is arranged at the front focus position of the objective lens 10. The front focus position is the focus position of the objective lens 10 on the sample 6 side. The front focus position of the objective lens 10 can be regarded as the front focus position of the observation optical system 5. The pupil of the objective lens 10 can be regarded as the pupil 15 of the observation optical system 5.

  In the imaging optical system 3, for example, bright field observation, phase difference observation, differential interference observation (hereinafter referred to as “DIC observation”), or inversion contrast observation (hereinafter referred to as “IVC observation”) can be performed.

  In bright-field observation, a diaphragm having a circular transmitting portion is arranged at the position of the pupil 14 of the illumination optical system. In phase difference observation or IVC observation, a diaphragm having an annular transmission portion is arranged at the position of the pupil 14 of the illumination optical system. Thus, the configuration of the imaging optical system 3 differs depending on the observation method.

  Therefore, the processor 2 generates a simulation image of the imaging optical system 3 (hereinafter referred to as “simulation image”).

  The simulation image is an image when the optical image 13 is calculated. The outline of the calculation of the optical image 13 will be described using the wavefront. For ease of understanding, description will be given from the optical image 13 side.

As shown in FIG. 2, the optical image 13 is formed at the position P _img . The optical image 13 is formed by the wavefront W _img . Therefore, if the wavefront W _img can be known, the optical image 13 can be calculated. The wavefront W _img is the wavefront at the position P _img .

Position P _img is conjugate with position P _fo . Therefore, the wavefront W _img can be known from the wavefront W _fo . The wavefront _Wfo is the wavefront at the position _Pfo .

The wavefront W _fo is a wavefront in which the wavefront W _out has propagated to the position P _fo . Therefore, the wavefront W _fo can be known from the wavefront W _out . The wavefront _Wout is the wavefront at the position _Pout .

The wavefront W _out is a wavefront after the wavefront W _in has passed through the sample 6. Therefore, the wavefront W _out can be known from the wavefront W _in . The wavefront W _in is the wavefront at the position P _in .

Wavefront _{W in} is a wavefront of the illumination light. The wavefront of the illumination light is the wavefront of the light emitted from the light source image 12 ′. Therefore, the wavefront W _in can be obtained from the light source image 12 ′.

In this way, if the wavefront W _in , the wavefront W _out , the wavefront W _fo , and the wavefront W _img can be known, the optical image 13 can be calculated. That is, a simulation image can be generated.

  Multiple steps are performed in the generation of the simulation image. The plurality of steps include a step of calculating a plurality of first wavefronts emitted from a plurality of light sources modeling the intensity distribution of the pupil of the illumination optical system and a plurality of first wavefronts after the plurality of first wavefronts have passed through the sample. Using two steps, a step of calculating a plurality of second wavefronts, a step of calculating a plurality of third wavefronts at a focus position on the sample side of the observation optical system from the plurality of second wavefronts, and a plurality of third wavefronts and a pupil function of the observation optical system. And calculating a plurality of fourth wavefronts at the imaging position of the observation optical system and squaring the plurality of fourth wavefronts to calculate a plurality of first intensity distributions, and adding a plurality of first intensity distributions. Accordingly, the step of calculating the intensity distribution of the optical image of the sample is provided.

  The optical characteristics of the optical image are affected by the temporal coherence of the illumination light. When a light source with low temporal coherence such as a halogen lamp or an LED lamp is used for illumination, a high-resolution optical image can be obtained without the occurrence of peckles, which is a problem with light sources with high temporal coherence such as lasers. .. Therefore, the simulation assumes that the sample is illuminated by a light source with low temporal coherence.

  In the following description, it is assumed that the focal length of the condenser lens, the focal length of the objective lens, and the focal length of the imaging lens are standardized to 1, and that an aplanatic lens is used as the lens.

  A flowchart of the processing performed by the processor is shown in FIG. As shown in FIG. 3, the plurality of steps includes steps S1 to S5.

  In step S1, a plurality of first wavefronts emitted from a plurality of light sources modeling the intensity distribution of the pupil of the illumination optical system are calculated.

  The sample 6 is illuminated with the light emitted from the light source 12. Therefore, in order to generate the simulation image, it is necessary to know the wavefront of the light emitted from the light source image 12.

  Since the light source 12 and the pupil 14 of the illumination optical system are conjugate, the light source image 12 'is formed at the position of the pupil 14 of the illumination optical system. Therefore, in order to generate the simulation image, it is sufficient that the wavefront can be calculated for each point of the light source image 12 '.

  However, if the wavefront is calculated for each point of the light source image 12 ', the amount of calculation becomes enormous. Therefore, the light source image 12 'is divided into a plurality of minute regions. Then, one minute region is regarded as one minute light source. By doing so, the amount of calculation can be reduced.

  The light source image 12 'is formed at the position of the pupil 14 of the illumination optical system. Therefore, if the pupil 14 of the illumination optical system is divided into a plurality of minute areas, the light source image 12 'can be divided into a plurality of minute areas. By thus dividing the pupil 14 of the illumination optical system into a plurality of minute regions, the intensity distribution in the pupil 14 of the illumination optical system can be replaced with an aggregate of minute light sources. That is, the intensity distribution in the pupil 14 of the illumination optical system can be modeled using a plurality of light sources.

  A plurality of light sources modeling the intensity distribution of the pupil of the illumination optical system emit a plurality of first wavefronts. Therefore, a plurality of first wavefronts are calculated.

As described above, one micro area can be regarded as one micro light source. The amplitude distribution of light at the point (x, y, z ₀ ) emitted from the minute light source at the illumination pupil position (ξ _s , η _s ) is represented by the following formula (I).
(I)
here,
u (x, y, z ₀ , ξ _s , η _s ) is the amplitude distribution of the light at the point (x, y, z ₀ ) emitted from the minute light source at the illumination pupil position (ξ _s , η _s ),
k = 2 × n ₀ / λ,
n ₀ is the refractive index of the medium around the sample,
λ is the wavelength of light used for simulation,
Is.

The amplitude distribution of light represents the wavefront of light. Therefore, u (x, y, z ₀ , ξ _s , η _s ) represents the first wavefront. Thus, the first wavefront can be calculated from equation (I).

  In step S2, a plurality of second wavefronts after the plurality of first wavefronts have passed through the sample are calculated.

  A plurality of first wavefronts are incident on the sample 6. The plurality of second wavefronts are generated by the plurality of first wavefronts passing through the sample 6. Therefore, a plurality of second wavefronts can be calculated using a plurality of first wavefronts. The calculation of the second wavefront will be described later.

  In step S3, a plurality of third wavefronts at the focus position on the sample side of the observation optical system are calculated from the plurality of second wavefronts.

  The sample 6 is a thick sample. Further, it is assumed that the front focus of the observation optical system 5 is located inside the sample 6. In this case, the position of the light after passing through the sample 6 is away from the front focus position of the observation optical system 5.

  It is assumed that an empty space extends from the position of light after passing through the sample 6 to the front focus position of the observation optical system 5. In this case, light propagates by Fresnel propagation in this space.

When the light at the position z _N reaches the position z _f by Fresnel propagation, the amplitude distribution of the light at the position z _f is represented by the following formula (II).
(II)
here,
u (x, y, z _f , ξ _s , η _s ) is the amplitude distribution of light at the position (x, y, z _f ),
u (x, y, z _N , ξ _s , η _s ) is the amplitude distribution of light at the position (x, y, z _N ),
k _x = k ξ _s , the wave number in the x direction,
k _y = k η _s , the wave number in the y direction,
Is.

The position z _f is the front focus position of the observation optical system 5. In this case, u (x, y, z _f , ξ _s , η _s ) represents the amplitude distribution of light at the front focus position of the observation optical system 5. Further, the position z _N is the position of the light after passing through the sample 6. In this case, u (x, y, z _N , ξ _s , η _s ) represents the amplitude distribution of light after passing through the sample 6.

As described above, the amplitude distribution of light represents the wavefront of light. _{Thus, u (x, y, z} f, ξ s, η s) represents a third wave front. Thus, the third wavefront can be calculated from equation (II).

U (x, y, z _N , ξ _s , η _s ) on the right side of Expression (II) represents the second wavefront. The formula (II) includes the second wavefront. Therefore, it is possible to calculate the plurality of third wavefronts at the focus position on the sample side of the observation optical system from the plurality of second wavefronts.

  In step S4, the plurality of fourth wavefronts at the image forming position of the observation optical system are calculated using the plurality of third wavefronts and the pupil function of the observation optical system, and the plurality of fourth wavefronts are respectively squared to obtain a plurality of them. Calculate the first intensity distribution.

The amplitude distribution of light at the image forming position of the optical system is expressed by the following equation (III).
(III)
here,
u _image (x, y, ξ _s , η _s ) is the amplitude distribution of light at the imaging position (x, y),
u (x, y, z _f , ξ _s , η _s ) is the amplitude distribution of light at the position (x, y, z _f ),
P (ξ, η) is the pupil function of the observation optical system,
Is.

As described above, the wavefront of light represents the amplitude distribution of light. Therefore, u _image (x, y, ξ _s , η _s ) represents the fourth wavefront. Thus, the fourth wavefront can be calculated from equation (III).

U on the right-hand side of Equation (III) (x, y, z f, ξ s, η s) represents a third wave front. Equation (III) includes the third wavefront. Therefore, it is possible to calculate the plurality of fourth wavefronts at the imaging position of the observation optical system 5 by using the plurality of third wavefronts and the pupil function of the observation optical system 5.

  Further, P (ξ, η) is a pupil function of the observation optical system 5. Therefore, the pupil function of the observation optical system 5 is used for the calculation of the fourth wavefront.

The light intensity distribution at the image forming position of the optical system is represented by the following equation (IV).
(IV)
here,
i (x, y) is the intensity distribution of light at the imaging position (x, y),
u _image (x, y, ξ _s , η _s ) is the amplitude distribution of light at the imaging position (x, y),
S (ξ _s , η _s ) is the intensity distribution of the light source,
Is.

U _image (x, y, ξ _s , η _s ) on the right side of Expression (IV) represents the fourth wavefront. Since the equation (IV) includes the fourth wavefront, the intensity distribution can be calculated using the fourth wavefront. As can be seen from the equation (IV), the fourth wavefront is squared in the intensity distribution calculation. Therefore, the plurality of fourth wavefronts are each squared to calculate the plurality of first intensity distributions.

  In step S5, the intensity distribution of the optical image of the sample is calculated by adding the plurality of first intensity distributions.

  By executing step S5, the intensity distribution of the optical image is calculated. The calculated intensity distribution of the optical image represents the intensity distribution of the optical image of the sample 6. Therefore, the intensity distribution of the optical image of the sample 6 can be calculated.

  In the image simulation apparatus of this embodiment, it is preferable to calculate the plurality of second wavefronts by applying the beam propagation method to each of the plurality of first wavefronts.

  The sample 6 can be represented by a plane continuous in the optical axis direction. Information about the sample 6 is included in one plane. The information includes, for example, the shape, the size, and the refractive index at each point. Depending on the case, the color at each point and the absorption rate at each point are also included in the information on the sample 6.

  In the calculation of the wavefront inside the sample 6, the wavefront on the incident side and the wavefront on the exit side are calculated for one plane. Therefore, as the number of planes increases, the amount of calculation increases.

  The number of planes can be reduced by providing a space between two adjacent planes. In this case, the light emitted from one plane reaches the other plane by Fresnel propagation in the space.

  The state of the sample and the wavefront is shown in FIG. In FIG. 4, the sample 6 is a virtual sample. The virtual specimen is replaced by 11 planes. Of the 11 planes, 2 are virtual planes. The virtual surface is placed where the specimen does not physically exist. The eleven planes are arranged at equal intervals, with the distance between adjacent planes being dz.

The object side sample surface S _ob is located on the condenser lens side. The image side sample surface S _im is located on the objective lens side. The object-side sample surface S _ob , the image-side sample surface S _im , and the seven planes are given a thickness so that the refractive index distribution on each plane can be seen.

A wavefront W _in is incident on the sample 6 toward the object-side sample surface S _ob along the arrow. The wavefront at the position Z ₀ is the wavefront represented by the above formula (I).

The wavefront W _{in at the} position Z ₀ passes through the virtual surface and reaches the object-side sample surface S _ob . From the position Z ₀ to the object-side sample surface S _ob , the wavefront propagates through Fresnel. Therefore, the wavefront at the object-side sample surface S _ob is different from the wavefront W _in . The wavefront incident on the object-side sample surface S _ob passes through some planes and reaches the plane S _k−1 .

The wavefronts on the two planes are shown in FIG. The wavefront W _k-1 is incident on the plane S _k-1 . When the plane S _k-1 has a refractive index distribution, the wavefront W _k-1 is affected by the refractive index distribution when passing through the plane S _k-1 . Therefore, the wavefront W _'k-1 different from the wavefront _{W k-1} is emitted from the plane _{S k-1.}

An empty space extends from the plane S _k-1 to the plane S _k . In this case, as described above, light propagates in this space by Fresnel propagation. The wavefront W ′ _k−1 emitted from the plane S _k−1 propagates by Fresnel propagation and reaches the plane S _k . The wavefront on the plane S _k becomes the wavefront W _k .

Amplitude distribution of light incident on the plane S _k-1 is influenced by the refractive index distribution of the sample in the plane S _k-1. The wavefront is then propagated by Fresnel propagation and reaches the plane S _k . The amplitude distribution of light at the position z _k is expressed by the following equation (V).
(V)
here,
u (x, y, z _k = z _k-1 + dz, ξ _s , η _s ) is the amplitude distribution of light immediately before passing through the sample at the position z _k ,
u (x, y, z _k−1 , ξ _s , η _s ) is the amplitude distribution of light immediately before passing through the sample at the position z _k−1 ,
dn (x, y, z _k-1 ) is the difference between the refractive index at position (x, y, z _k-1 ) and n ₀ ,
n ₀ is the refractive index of the medium around the sample,
Is.

As described above, the wavefront of light represents the amplitude distribution of light. Therefore, u (x, y, z _k = z _k-1 + dz, ξ _s , η _s ) represents the wavefront at the position z _k . Further, u (x, y, z _k−1 , ξ _s , η _s ) represents the wavefront at the position z _k−1 .

From the position z ₀ to the position z _k−1 and from the position z _k to the position z _N , the process of Fresnel propagation is repeated after being influenced by the refractive index distribution of the sample in each plane, and light is transmitted. Propagate. Therefore, the wavefront during this period can also be calculated from the formula (V). The wavefront at the position z _N is the second wavefront. Therefore, the second wavefront can be obtained from the equation (V).

  Expression (V) is an expression used in the beam propagation method. As described above, in the image simulation apparatus of this embodiment, the beam propagation method is used to calculate the second wavefront.

In the image simulation apparatus of this embodiment, the illumination optical system has a light source and a condenser lens, the observation optical system has an objective lens and an imaging lens, and the following conditional expression (1) is satisfied. Is preferably satisfied.
d <0.5 × Rob / β (1)
here,
d is the minimum spacing of multiple light sources,
Rob is the radius of the pupil of the objective lens,
β is the magnification obtained by dividing the focal length of the objective lens by the focal length of the condenser lens,
Is.

  As described above, the intensity distribution in the pupil of the illumination optical system can be modeled using a plurality of light sources. In this modeling, the light sources may be arranged at equal intervals or may not be arranged at equal intervals.

  When the light sources are arranged at equal intervals, all the intervals are the minimum intervals. When the light sources are not arranged at equal intervals, the narrowest interval among the intervals between the adjacent light sources is the minimum interval.

  When the value exceeds the upper limit value of the conditional expression (1), the distance between the adjacent light sources becomes wide in the plurality of modeled light sources. Therefore, the calculation accuracy deteriorates.

It is preferable that the following conditional expression (1 ′) is satisfied instead of conditional expression (1).
d <0.3 × Rob / β (1 ′)
Further, it is preferable that the following conditional expression (1 ″) is satisfied instead of conditional expression (1).
d <0.15 × Rob / β (1 ″)

In the image simulation apparatus of this embodiment, it is preferable that the illumination optical system has an aperture member, the aperture member has a light shielding portion and a transmission portion, and satisfies the following conditional expression (2).
d <0.5 × (R1-R0) (2)
here,
d is the minimum spacing of multiple light sources,
R0 is the distance from the optical axis of the illumination optical system to the inner edge of the transmission part,
R1 is the distance from the optical axis of the illumination optical system to the outer edge of the transmission part,
Is.

  In phase difference observation or IVC observation, a diaphragm having an annular transmission portion is arranged at the position of the pupil 14 of the illumination optical system. Therefore, the transmissive portion has an inner edge and an outer edge. A circular light shield is located inside the ring. An annular light-shielding portion is located outside the annular ring.

  When the value exceeds the upper limit value of the conditional expression (2), the distance between adjacent light sources becomes large in the plurality of modeled light sources. Therefore, the calculation accuracy deteriorates.

It is preferable that the following conditional expression (2 ′) is satisfied instead of conditional expression (2).
d <0.3 × (R1-R0) (2 ')
Further, it is preferable that the following conditional expression (2 ″) is satisfied instead of conditional expression (2).
d <0.15 x (R1-R0) (2 '')

In the image simulation apparatus of this embodiment, the observation optical system has an objective lens and an imaging lens, and in the calculation of the second wavefront, a virtual sample replaced with a plurality of planes is used, and a plurality of planes are used. Are arranged at equal intervals and preferably satisfy the following conditional expression (3).
dz <0.5 × 1.2 × λ / NAob ² (3)
here,
dz is the distance between adjacent planes,
NAob is the numerical aperture of the objective lens,
λ is the wavelength of light used for simulation,
Is.

  As described above, the sample 6 can be represented by a plane continuous in the optical axis direction of the observation optical system. In addition, one plane contains information about the sample 6. Therefore, these planes can be used to form a virtual sample.

  The virtual sample has a plurality of planes. In the virtual sample, the plurality of planes are arranged at equal intervals along the optical axis of the observation optical system. Each of the planes is one section when the sample is sliced. That is, each of the planes is one of the planes continuous in the optical axis direction of the observation optical system. The virtual sample is the sample 6 sampled on a plurality of virtual planes.

  In the virtual sample, virtual surfaces are discretely arranged along the optical axis of the observation optical system. Therefore, the amount of calculation related to the wavefront can be reduced as compared with the case where the virtual surfaces are continuously arranged. Therefore, the virtual sample is used in the calculation of the second wavefront.

  When the value exceeds the upper limit of conditional expression (3), the sampling frequency in the optical axis direction of the observation optical system becomes smaller than the cutoff frequency of the observation optical system. Therefore, accurate simulation cannot be performed.

In the image simulation apparatus of this embodiment, the observation optical system has an objective lens and an imaging lens, and in the calculation of the second wavefront, a virtual sample replaced with a plurality of planes is used, and the following conditions are satisfied. It is preferable to satisfy the expression (4).
dx <0.5 × 1.2 × λ / NAob (4)
here,
dx is the sampling interval in the plane,
NA is the numerical aperture of the objective lens,
λ is the wavelength of light used for simulation,
The sampling interval is the interval between two adjacent regions when the plane is divided into a plurality of regions,
Is.

  The virtual sample is as described above. The sampling interval is an interval in the direction orthogonal to the optical axis of the observation optical system.

  When the value exceeds the upper limit of conditional expression (4), the sampling frequency in the direction orthogonal to the optical axis of the observation optical system becomes smaller than the cutoff frequency of the observation optical system. Therefore, accurate simulation cannot be performed.

In the image simulation apparatus of this embodiment, it is preferable that the imaging optical system has a differential interference prism and satisfies the following conditional expression (5).
dx <Δshear (5)
here,
dx is the sampling interval in the plane,
Δshear is the amount of shear on the sample,
Is.

  The imaging optical system has a differential interference prism. Therefore, DIC observation is possible in the imaging optical system.

  An optical system used for DIC observation is shown in FIG. The optical system 20 has an illumination optical system 21 and an observation optical system 22. In DIC observation, the differential interference prism 23 is arranged in the illumination optical system 21, and the differential interference prism 24 is arranged in the observation optical system 22.

  A polarizing plate 25 is arranged in the illumination optical system 21. Light emitted from a light source (not shown) passes through the polarizing plate 25. Linearly polarized light is emitted from the polarizing plate 25.

  The linearly polarized light enters the differential interference prism 23. The linearly polarized light incident on the differential interference prism 23 is split into two by the differential interference prism 23. Therefore, the light Lp and the light Ls are emitted from the differential interference prism 23.

  The light Lp and the light Ls are both linearly polarized light. The polarization direction of the light Lp is orthogonal to the polarization direction of the light Ls.

  The light Lp and the light Ls enter the condenser lens 26. The light Lp and the light Ls are emitted from the condenser lens 26 in parallel. The light Lp and the light Ls emitted from the condenser lens 26 enter the sample 27.

  Since the light Lp and the light Ls are parallel, the incident position of the light Lp and the incident position of the light Ls are different at the position of the sample 27. The light Lp and the light Ls emitted from the sample 27 enter the objective lens 28.

  The light Lp and the light Ls emitted from the objective lens 28 enter the differential interference prism 24. The light Lp and the light Ls incident on the differential interference prism 24 are combined into one by the differential interference prism 24. Therefore, the combined light is emitted from the differential interference prism 24. The combined light enters the polarizing plate 29.

  As described above, at the position of the sample 27, the incident position of the light Lp and the incident position of the light Ls are different. In the DIC observation, when the phase amount at the position where the light Lp passes and the phase amount at the position where the light Ls passes are different, the difference in the phase amount can be detected as the light intensity.

  When the value exceeds the upper limit value of the conditional expression (5), the second sampling interval becomes larger than the shear amount on the sample. The virtual surface is divided into a plurality of areas. Therefore, when the second sampling interval becomes larger than the shear amount on the sample, the light Lp and the light Ls are incident on the same region. Therefore, accurate simulation cannot be performed.

  Hereinafter, a simulation image obtained by the image simulation apparatus of this embodiment will be described. As the simulation image, an image in IVC observation, an image in bright field observation, an image in phase difference observation, and an image in DIC observation are shown.

<Image in IVC observation>
An optical system used for IVC observation is shown in FIG. The optical system 30 has an illumination optical system 31 and an observation optical system 32. The illumination optical system 31 has a condenser lens 33 and a diaphragm 34. The observation optical system 32 has an objective lens 36 and an imaging lens 39. The objective lens 36 has a diaphragm 37. An optical image of the sample 35 is formed at the image position 40 of the observation optical system 32.

  The diaphragm 34 is arranged at the pupil position of the condenser lens 33. The diaphragm 34 has a circular light-shielding portion 34a, an annular light-transmitting portion 34b, and an annular light-shielding portion 34c. The optical axis of the illumination optical system 31 intersects with the center of the light shielding portion 34a. The transmissive portion 34b is located outside the light shielding portion 34a. The light shielding portion 34c is located outside the transmission portion 34b.

  The diaphragm 37 has a circular transmitting portion 37a and an annular light shielding portion 37b. The optical axis of the observation optical system 32 intersects with the center of the transmission part 37a. The light shielding portion 37b is located outside the transmission portion 37a. A circular boundary 38 is formed between the transmissive portion 37a and the light shielding portion 37b.

  The diaphragm 37 can be regarded as the pupil of the objective lens 36. In this case, the boundary 38 represents the edge of the pupil of the objective lens 36.

  The pupil position of the condenser lens 33 is conjugate with the pupil position of the objective lens 36. An image of the diaphragm 34 is formed at the position of the diaphragm 37. Therefore, an image of the transmission part 34b is formed at the position of the diaphragm 37.

  The transmissive portion 34b has an inner edge and an outer edge. At the position of the diaphragm 37, an image of the inner edge of the transmissive portion 34b (hereinafter referred to as "inner edge image") and an image of the outer edge of the transmissive portion 34b (hereinafter referred to as "outer edge image") are formed. In the optical system 30, the inner edge image is located inside the boundary 38, and the outer edge image is located outside the boundary 38.

  In the optical system 30, the position of the inner edge image and the position of the outer edge image change with respect to the boundary 38 according to the amount of tilt on the surface of the sample 35. The inner edge of the transparent portion 34b represents the outer circumference of the light shielding portion 34a. The change of the position of the inner edge image represents the change of the position of the image of the light shielding portion 34a.

  When the surface of the sample 35 is flat, or when the amount of inclination is small, the image of the light shielding portion 34 a is located inside the boundary 38. While the image of the light shield 34a is located inside the boundary 38, the amount of light passing through the diaphragm 37 does not change.

  When the amount of inclination of the surface of the sample 35 increases, a part of the image of the light shielding portion 34a protrudes outside the boundary 38. The larger the tilt amount, the larger the protruding amount. As a result, the amount of light passing through the diaphragm 37 changes. As described above, in the IVC observation, the change in the tilt amount on the surface of the sample can be grasped as the change in the light amount.

  FIG. 8 shows a state in which the pupil of the illumination optical system is divided. The diaphragm 34 is used for IVC observation. The diaphragm 34 is arranged at the pupil position of the condenser lens 33, that is, the pupil position of the illumination optical system 31.

  In the diaphragm 34, the light from the light source passes through the transmitting portion 34b. Therefore, when the pupil of the illumination optical system 31 is divided into a plurality of minute regions, the intensity distribution in the pupil of the illumination optical system 31 is modeled by distributing a plurality of light sources in an annular shape, as shown in FIG. ..

  FIG. 9 shows a simulation image and an actual image in the IVC observation. In FIG. 9, (a), (b), and (c) show the in-focus position with respect to the sample. (D), (e), and (f) show simulation images. (G), (h), and (i) show actual images. In (j), (k), and (l), the solid line shows the simulation, and the dotted line shows the intensity profile of the actual image.

  The specimen is a bead. The beads are held in a container with the liquid. The sample, measurement conditions, and focus position are as follows.

<Sample>
Bead diameter 72 μm
Refractive index of beads 1.59
Retention of beads Liquid Refractive index of liquid 1.516

<Measurement conditions>
Objective lens magnification 10 times Objective lens numerical aperture 0.25
Light source LED
Light source wavelength 550nm
Width of transmission part of condenser lens Min: 0.242, Max: 0.278
The width of the transparent portion of the condenser lens is represented by the numerical aperture.

<Focus position>
Position 1 Position 2 μm downward from position 2 Position 2 Center of beads Position 3 Position 32 μm upward from position 2

  In (a), (d), (g), and (j), the position 1 is in focus. In (b), (e), (h), and (k), the position 2 is in focus. In (c), (f), (i), and (l), the position 3 is in focus.

  From the simulation image, the actual image, and the state of the cross section, it can be seen that the simulation image is in good agreement with the actual image.

<Image in bright field observation>
An optical system used for bright field observation is shown in FIG. The same components as those of the optical system 50 are designated by the same reference numerals and the description thereof will be omitted.

  The optical system 50 has an illumination optical system 51 and an observation optical system 32. The illumination optical system 51 has a condenser lens 33 and a diaphragm 52. In the optical system 50, the diaphragm 52 has a circular transmitting portion and an annular light shielding portion.

  FIG. 11 shows how the pupil of the illumination optical system is divided. The diaphragm 52 is used for bright-field observation. The diaphragm 52 is arranged at the pupil position of the condenser lens 33, that is, the pupil position of the illumination optical system 51.

  At the diaphragm 52, the light from the light source passes through the circular transmitting portion. Therefore, when the pupil of the illumination optical system 51 is divided into a plurality of minute regions, the intensity distribution in the pupil of the illumination optical system 31 is modeled by distributing a plurality of light sources in a circle, as shown in FIG.

  FIG. 12 shows a simulation image and an actual image in bright-field observation. In FIG. 12, (a), (b), and (c) show simulation images. (D), (e), and (f) show actual images. In (g), (h), and (i), the solid line shows the simulation, and the dotted line shows the intensity profile of the actual image.

  The sample and measurement conditions are the same as in the IVC observation. The focus position is the position 2.

  In (a), (d), and (g), the numerical aperture of the illumination optical system is 0.05. In (b), (e), and (h), the numerical aperture of the illumination optical system is 0.125. In (c), (f), and (i), the numerical aperture of the illumination optical system is 0.275.

  From the simulation image, the actual image, and the state of the cross section, it can be seen that the simulation image is in good agreement with the actual image.

<Image in phase contrast observation>
Illustration of the optical system used for the phase difference observation is omitted. In the phase difference observation, as in the IVC observation, a diaphragm having an annular transmission part is used. Therefore, similarly to FIG. 8, the intensity distribution in the pupil of the illumination optical system is modeled by distributing a plurality of light sources in an annular shape.

  In the phase difference observation, the phase plate is arranged at the pupil position of the objective lens. The phase plate has an annular phase film. An absorption film is provided on the phase film.

  The phase film is located inside the pupil of the objective lens. In the diaphragm of the illumination optical system, the annular transmission part is conjugated with the phase film. Therefore, the image of the annular transmission part is located inside the pupil of the objective lens.

  In IVC observation, the image of the annular transmission part is located inside and outside the pupil of the objective lens. As described above, the diaphragm used for the phase difference observation is different from the diaphragm used for the IVC observation in the width and position of the annular transmission part.

  FIG. 13 shows a simulation image and an actual image in the phase difference observation. In FIG. 13, (a) shows a simulation image. (B) shows an actual image. In (c), the solid line shows the simulation and the dotted line shows the intensity profile of the actual image.

The sample and measurement conditions are the same as in the IVC observation. The focus position is the position 2. The diaphragm and the phase plate are as follows. The width of the transmission part and the width of the phase film are expressed by numerical aperture.
Width of transmission part Min: 0.13, Max: 0.15
Width of phase film Min: 0.125, Max: 0.155
Absorption film transmittance 14%

  From the simulation image, the actual image, and the state of the cross section, it can be seen that the simulation image is in good agreement with the actual image.

  As described above, in the phase difference observation, the phase plate is arranged at the pupil position of the objective lens. Therefore, this point must be taken into consideration in the simulation. In the calculation of the light amplitude distribution at the image forming position, the pupil function of the optical system is used as shown in Expression (III).

In the phase difference observation, the following formula (VI) is used as the pupil function of the optical system.
(VI)
here,
NA _{phase_film1} is the numerical aperture of the inner edge of the phase film,
NA _{phase_film2} is the numerical aperture of the outer edge of the phase film,
Is.

  P (ξ, η) = 0 represents the pupil function outside the pupil of the objective lens. P (ξ, η) = √t × (−i) and P (ξ, η) = 1 represent the pupil function inside the pupil of the objective lens. P (ξ, η) = √t × (−i) represents the pupil function in the phase film. P (ξ, η) = 1 represents the pupil function in a place other than the phase film.

<Image in DIC observation>
For DIC observation, the optical system shown in FIG. 6 is used. In the DIC observation, as in the bright field observation, a diaphragm having a circular transmission part is used. Therefore, similarly to FIG. 11, the intensity distribution in the pupil of the illumination optical system is modeled by distributing a plurality of light sources in a circle.

  FIG. 14 shows a simulation image and an actual image in DIC observation. In FIG. 14, (a) shows a simulation image. (B) shows an actual image. In (c), the solid line shows the simulation and the dotted line shows the intensity profile of the actual image. Brightness and darkness occur in the direction of the shear.

  From the simulation image, the actual image, and the state of the cross section, it can be seen that the simulation image is in good agreement with the actual image.

In DIC observation, an image of the sample is formed by the light Lp and the light Ls. Therefore, the light intensity distribution at the image forming position is represented by the following formula (VII).
(VII)
here,
θ is the relative retardation amount of the light Lp and the light Ls,
Is.

  In the above example, beads are used for the sample. An example using another sample is shown below. The sample is a photonic crystal fiber (hereinafter referred to as "PCF").

  An example of PCF is shown in FIG. The PCF 60 has a columnar member 61 and a through hole 62. In the PCF 60, a plurality of through holes 62 are formed inside the columnar member 61. The through hole 62 has a cylindrical shape and is formed along the generatrix of the columnar member 61.

  Bright field observation is used for the observation. FIG. 16 shows an optical system used for observing the PCF. The same components as those of the optical system 50 are designated by the same reference numerals and the description thereof will be omitted. In the optical system 50, a PCF 60 is arranged as a sample.

  FIG. 17 shows a simulation image and an actual image in bright-field observation. In FIG. 17, (a), (b), and (c) show the in-focus position with respect to the sample. (D), (e), and (f) show simulation images. (G), (h), and (i) show actual images. In (j), (k), and (l), the solid line shows the simulation, and the dotted line shows the intensity profile of the actual image.

  The sample is PCF. The PCF is held in a container with the liquid. The measurement conditions are the same as the IVC observation. The sample and the focus position are as follows. The numerical aperture of the illumination optical system is 0.125.

<Sample>
PCF diameter 230 μm
Hole diameter 6 μm
Refractive index of PCF 1.458
Holding of PCF Liquid Refractive index of liquid 1.424
<Focus position>
Position 4 Position 30 μm downward from position 5 Position 5 Center of PCF Position 6 Position 30 μm upward from position 5

  In (a), (d), (g), and (j), the position 4 is in focus. In (b), (e), (h), and (k), the position 5 is in focus. In (c), (f), (i), and (l), the position 6 is in focus.

  From the simulation image, the actual image, and the state of the cross section, it can be seen that the simulation image is in good agreement with the actual image.

  The image simulation apparatus according to the present embodiment can also use observation by polarized illumination. FIG. 18 shows how the pupil of the illumination optical system is divided. In the observation by the polarized illumination, the intensity distribution in the pupil of the illumination optical system is modeled by distributing a plurality of light sources in a substantially half-moon shape.

  The image simulation method of the present embodiment is an image simulation method for generating a simulation image of an imaging optical system that includes an illumination optical system and an observation optical system that is arranged to face the illumination optical system with a sample in between. , Calculating a plurality of first wavefronts emitted from a plurality of light sources modeling the intensity distribution of the pupil of the illumination optical system, and calculating a plurality of second wavefronts after the plurality of first wavefronts have passed through the sample, A plurality of third wavefronts at the focal point on the sample side of the observation optical system are calculated from the plurality of second wavefronts, and a plurality of third wavefronts at the imaging position of the observation optical system are calculated using the plurality of third wavefronts and the pupil function of the observation optical system. The fourth wavefront of is calculated, each of the plurality of fourth wavefronts is squared to calculate a plurality of first intensity distributions, and the plurality of first intensity distributions are added together to calculate an intensity distribution of the optical image of the sample. To do.

  In the image simulation apparatus of this embodiment, information regarding the observation method and information regarding the sample are input to the processor.

  As described above, the observation methods include bright field observation, phase difference observation, DIC observation, and IVC observation. These observation methods differ in the shape and size of the aperture of the diaphragm, the magnification of the condenser lens, the numerical aperture of the condenser lens, the type of the objective lens, the numerical aperture of the objective lens, the magnification of the objective lens, and the like.

  Even with one observation method, the shape and size of the aperture of the diaphragm, the magnification of the condenser lens, the numerical aperture of the condenser lens, the type of the objective lens, the numerical aperture of the objective lens, and the magnification of the objective lens, depending on the observation purpose. , Etc. are different.

  Information relating to the observation method is input to the processor according to the observation method. An input unit for these pieces of information can be provided in the processor. Further, an input unit independent of the processor may be provided and information may be input from the input unit to the processor.

  The image simulation device of this embodiment can be connected to a microscope. Bright field observation, phase difference observation, DIC observation, and IVC observation can be performed with a microscope. A diaphragm, a condenser lens, and an objective lens are appropriately selected according to the observation method. Based on this selection, information regarding the observation method may be input to the processor.

  As described above, the present invention is suitable for an image simulation apparatus and an image simulation method that can generate an image of a sample with a small amount of calculation.

1 Image Simulation Device 2 Processor 3 Imaging Optical System 4 Illumination Optical System 5 Observation Optical System 6 Specimen 7, 8 Lens 9 Condenser Lens 10 Objective Lens 11 Imaging Lens 12 Light Source 12 'Light Source Image 13 Optical Image 14 Illumination Optical System Pupil 15 Pupil of observation optical system 16 Display device 20, 30, 50 Optical system 21, 31, 51 Illumination optical system 22, 32 Observation optical system 23, 24 Differential interference prism 25, 29 Polarizing plate 26, 33 Condenser lens 27, 35 Sample 28 , 36 Objective lens 34, 37, 52 Diaphragm 34a, 34c, 37b Light-shielding part 34b, 37a Transmitting part 38 Boundary 39 Imaging lens 40 Image position 60 PCF (photonic crystal fiber)
61 cylindrical member 62 through hole Ls, Lp light

Claims (8)

An image simulation apparatus for generating a simulation image of an imaging optical system having an illumination optical system, and an observation optical system arranged to face the illumination optical system with a sample interposed therebetween,
Equipped with a processor,
The processor is
Calculating a plurality of first wavefronts emitted from a plurality of light sources modeling the intensity distribution of the pupil of the illumination optical system;
Calculating a plurality of second wavefronts after the plurality of first wavefronts have passed through the sample;
Calculating a plurality of third wavefronts at the sample-side focal position of the observation optical system from the plurality of second wavefronts;
A plurality of fourth wavefronts at the image forming position of the observation optical system are calculated using the plurality of third wavefronts and the pupil function of the observation optical system, and the plurality of fourth wavefronts are squared to obtain a plurality of square wavefronts. Calculating a first intensity distribution,
Calculating the intensity distribution of the optical image of the sample by adding the plurality of first intensity distributions;
An image simulation apparatus, characterized by performing a process including:   The image simulation apparatus according to claim 1, wherein the plurality of second wavefronts are calculated by applying a beam propagation method to each of the plurality of first wavefronts. The illumination optical system has a light source and a condenser lens,
The observation optical system has an objective lens and an imaging lens,
The image simulation apparatus according to claim 1, wherein the following conditional expression (1) is satisfied.
d <0.5 × Rob / β (1)
here,
d is the minimum distance between the plurality of light sources,
Rob is the radius of the pupil of the objective lens,
β is a magnification obtained by dividing the focal length of the objective lens by the focal length of the condenser lens,
Is. The illumination optical system has an aperture member,
The opening member has a light shielding portion and a transmission portion,
The image simulation apparatus according to claim 1, wherein the following conditional expression (2) is satisfied.
d <0.5 × (R1-R0) (2)
here,
d is the minimum distance between the plurality of light sources,
R0 is the distance from the optical axis of the illumination optical system to the inner edge of the transmission part,
R1 is the distance from the optical axis of the illumination optical system to the outer edge of the transmission part,
Is. The observation optical system has an objective lens and an imaging lens,
In the calculation of the second wavefront, a virtual sample replaced by a plurality of planes is used,
The plurality of planes are arranged at equal intervals,
The image simulation apparatus according to any one of claims 1 to 4, wherein the following conditional expression (3) is satisfied.
dz <0.5 × 1.2 × λ / NAob ² (3)
here,
dz is the interval between the adjacent planes,
NAob is the numerical aperture of the objective lens,
λ is the wavelength of light used for simulation,
Is. The observation optical system has an objective lens and an imaging lens,
In the calculation of the second wavefront, a virtual sample replaced by a plurality of planes is used,
The image simulation apparatus according to claim 1, wherein the following conditional expression (4) is satisfied.
dx <0.5 × 1.2 × λ / NAob (4)
here,
dx is a sampling interval in the plane,
NAob is the numerical aperture of the objective lens,
λ is the wavelength of light used for simulation,
The sampling interval is an interval between two adjacent regions when the plane is divided into a plurality of regions,
Is. The imaging optical system has a differential interference prism,
The image simulation apparatus according to claim 6, wherein the following conditional expression (5) is satisfied.
dx <Δshear (5)
here,
dx is the sampling interval in the plane,
Δshear is the amount of shear on the sample,
Is. An image simulation method for generating a simulation image of an imaging optical system having an illumination optical system and an observation optical system that is arranged to face the illumination optical system with a sample interposed therebetween,
Calculating a plurality of first wavefronts emitted from a plurality of light sources modeling the intensity distribution of the pupil of the illumination optical system,
Calculating a plurality of second wavefronts after the plurality of first wavefronts has passed through the sample,
Calculating a plurality of third wavefronts at the focus position on the specimen side of the observation optical system from the plurality of second wavefronts,
A plurality of fourth wavefronts at the image forming position of the observation optical system are calculated using the plurality of third wavefronts and the pupil function of the observation optical system, and the plurality of fourth wavefronts are squared to obtain a plurality of square wavefronts. Calculate the first intensity distribution,
An image simulation method for calculating an intensity distribution of an optical image of the sample by adding the plurality of first intensity distributions together.