JP2009300589A - Microscope apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a microscope apparatus acquiring an accurate sample image. <P>SOLUTION: This microscope 1 is provided with: an illumination optical system 10 that has a diffraction grating 16 for spatially modulating the light emitted from a light source 11 and irradiates a sample 5 with the illumination light acquired by spatial modulation by the diffraction grating 16; an image forming optical system 30 for forming a modulation image from the sample 5 irradiated with the illumination light; an imaging element 35 for imaging the modulation image; a diffraction grating driving device 25 for modulating the phase of the illumination light; and an image processor 41 for forming the image of the sample 5 from a plurality of modulation images imaged by the imaging element 35 whenever the phase of the illumination light is modulated. The image processor 41 corrects the intensity of the image signal outputted from the imaging element 35 in the pixel unit of the imaging element 35, and converts it into the light quantity of the modulation image. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a microscope apparatus including a structured illumination microscope.

  The resolution of an optical microscope is determined by the numerical aperture and wavelength of the objective lens, and it has been generally considered that there is no way to increase the resolution, rather than shortening the wavelength or increasing the numerical aperture. On the other hand, there is a demand for observing a sample with a higher resolution than that determined by the numerical aperture and wavelength.

As one of the techniques to meet this requirement, for example, a modulation unit that modulates the spatial frequency of illumination light is provided in the vicinity of a specimen to be observed in the illumination optical system, and a plurality of photographings are performed while modulating the spatial frequency of the illumination light by the modulation unit There has been proposed a structured illumination microscope that acquires an image and demodulates a plurality of acquired captured images by linear calculation to obtain a high-resolution sample image (see, for example, Patent Document 1).
Japanese Patent Laid-Open No. 11-242189

  In general, an image sensor composed of a plurality of light receiving elements is made so that all the light receiving elements have the same sensitivity and background noise level. However, in practice, variations in sensitivity of ± several percent or background noise may occur due to manufacturing errors or the like. Therefore, conventional microscopes do not take into account fixed pattern noise caused by variations in sensitivity and background noise in each light receiving element of the image sensor used for image acquisition, and therefore there is an error in the image obtained by demodulation. May occur.

  In particular, the structured illumination microscope acquires a plurality of images while changing the conditions of structured illumination, separates diffraction components therefrom, and then rearranges them in the original wave number space to synthesize (ie, demodulate). The sample image is acquired. For this reason, the signal intensity variation in the pixel before demodulation is transmitted to a high spatial frequency component, and there is a possibility that an accurate sample image cannot be acquired.

  The present invention has been made in view of such problems, and an object thereof is to provide a microscope apparatus that can acquire a more accurate specimen image.

  In order to achieve such an object, according to the first aspect of the present invention, the diffraction grating that spatially modulates the light emitted from the light source is provided and spatially modulated by the diffraction grating. An illumination optical system that irradiates the specimen with illumination light, an imaging optical system that forms a modulated image from the specimen irradiated with the illumination light, an imaging device that captures the modulated image, and a phase of the illumination light A phase modulation unit that modulates the phase of the illumination light, and an image processing device that generates an image of the sample from the plurality of modulation images captured by the imaging device each time the phase of the illumination light is modulated using the phase modulation unit. The image processing apparatus includes a correction unit that corrects the intensity of the image signal output from the image sensor in units of pixels of the image sensor and converts the intensity into a light amount of the modulated image. A microscope apparatus is provided.

  According to the present invention, it is possible to calculate an accurate incident light amount from the output signal of the image sensor and obtain a more accurate specimen image that reflects the incident light amount as accurately as possible.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 shows a first embodiment of a structured illumination microscope that is a microscope apparatus according to the present invention. The structured illumination microscope 1 according to the first embodiment mainly includes an illumination optical system 10, an imaging optical system 30, an image sensor 35, a control device 40 incorporating an image processing device 41, and a monitor 50. Is done.

  The imaging optical system 30 includes, in order from the light source side, the lamp light source 11, the collector lens 12, the relay lens 13, the aperture stop 14, the field lens 15, the diffraction grating 16, the condenser lens 17, and the folding mirror 18. And the relay lens 19, the field stop 20, the second objective lens 21, the half mirror 9, and the objective lens 7. The light from the lamp light source 11 is condensed on the opening of the aperture stop 14 via the collector lens 12 and the relay lens 13, and is parallel light substantially parallel to the optical axis of the imaging optical system 30 by the field lens 15. And is incident on the diffraction grating 16. The light emitted from the diffraction grating 16 is collected by the condenser lens 17 and deflected by the folding mirror 18 disposed at the conjugate position of the pupil plane 8 of the objective lens 7, and the relay lens 19, the field stop 20, and the first lens. 2 It reaches the half mirror 9 via the objective lens 21. The light that has reached the half mirror 9 is reflected by the half mirror 9 toward the objective lens 7 so as to be condensed on the pupil plane 8 of the objective lens 7, and is irradiated on the sample 5 through the objective lens 7.

  The imaging optical system 30 includes an objective lens 7, a half mirror 9, and an imaging lens 31 in order from the object (sample 5) side. The light from the sample 5 is collected by the objective lens 7, passes through the half mirror 9, and forms an image on the image sensor 35 through the imaging lens 31.

  The imaging element 35 captures an image formed by the imaging optical system 30 and outputs an image signal to the image processing device 41 of the control device 40. The image processing device 41 performs a predetermined calculation process to generate a sample image that is an image of the sample 5. The specimen image generated by the image processing device 41 is stored in a memory (not shown) of the control device 40 and is displayed on the monitor 50 and observed by the user. The diffraction grating 16 is connected to a diffraction grating driving device 25 that moves the diffraction grating 16 in the direction in which the grating patterns constituting the diffraction grating 16 are arranged, and the control device 40 controls the operation of the diffraction grating driving device 25. .

  The image processing apparatus 41 is mainly configured by a circuit board (not shown) equipped with an MPU (Micro Processing Unit), and as shown in FIG. 2, an input unit 42, an internal memory 43, an arithmetic processing unit 44, And an output unit 45. When the image signal from the image sensor 35 is input from the input unit 42 and stored in the internal memory 43, the arithmetic processing unit 44 generates a sample image from the data of the image signal stored in the internal memory 43, and generates the sample image The image data of the sample image is output from the output unit 45 to the monitor 50 and the memory (not shown) of the control device 40.

  Hereinafter, image acquisition by the structured illumination microscope 1 of the first embodiment will be described. The diffraction grating 16 of the first embodiment has a grating pattern having a one-dimensional transmittance distribution in the direction in the plane of the paper perpendicular to the optical axis of the imaging optical system 30, and diffracted light indicated by a broken line in FIG. Give rise to It should be noted that not only such a diffraction grating but also a phase diffraction grating having an optical path difference of ½ wavelength with respect to the wavelength of the lamp light source 11 may be used.

  In order to obtain a high spatial resolution according to the present embodiment, it is desirable that the diffraction grating 16 has a fine period within a range in which illumination light spatially modulated (by the diffraction grating 16) can be projected onto the sample 5. At this time, it is only necessary to provide a sufficiently fine sinusoidal illumination to the specimen 5 in order to generate moire, so that oblique illumination is applied to a grating having a period near the imaging limit, or ± 1 It is preferable to arrange a light shielding plate on a plane conjugate with the pupil plane 8 of the objective lens 7 so that only the ± first-order light can pass through the diffraction grating 16 having a period in which the diffraction of the next-order light can be imaged.

  In the present embodiment, a diffraction grating 16 is used in which ± first-order light interferes on the specimen 5 to provide illumination with a period near the resolution limit. The diffraction grating 16 separates the ± first-order light so as to pass through the peripheral portion of the field lens 17, and the + 1st-order light and the −1st-order light are spotted at the positions of the peripheral portion of the folding mirror 18 placed on the pupil plane. make. Here, the aperture stop 14 is adjusted so that the ± first-order light does not overlap on the folding mirror 18. When monochromatic light is used as the light source 11, it is easy to set the illumination condition so that the ± primary light does not overlap. Therefore, the aperture stop 14 may not be provided.

  Since zero-order light is generated depending on the type of the diffraction grating 16 and the illumination conditions, a light shielding plate (not shown) for blocking the zero-order light is arranged on the optical axis of the surface of the folding mirror 18 and is not necessary for sinusoidal illumination. It is desirable to remove 0th order light. In FIG. 1, since the folding mirror 18 is placed on the pupil plane, a light shielding plate (not shown) for removing the 0th-order light is provided on the folding mirror 18. However, when the folding mirror 18 is not used, If it is placed at a position different from the pupil plane, a light shielding plate (not shown) may be provided at a position conjugate with the pupil plane 8 in the illumination optical system 30 instead of on the folding mirror 18.

  The ± first-order light deflected by the folding mirror 18 passes through the peripheral portion of the pupil plane 8 of the objective lens 7 and interferes with each other in the vicinity of the sample 5 to become illumination light having a fringe structure and is irradiated on the sample 5. The reflected light from the specimen 5 irradiated with such (spatial-modulated) illumination light is guided to the image sensor 35 by the imaging optical system 30, and an image of the specimen 5 irradiated with the spatially-modulated illumination light. (Hereinafter referred to as a modulated image) is formed.

  The modulated image is picked up by an image pickup device 35 such as a CCD or CMOS. At this time, the diffraction grating driving device 25 acquires N images while accurately shifting the diffraction grating 16 by 1 / N of the period in the direction in which the grating patterns are arranged. However, N is 3 or more and is desirably as large as possible.

  Image signals of a plurality of modulated images captured and acquired by the image sensor 35 are sent from the input unit 42 of the image processing device 41 to the internal memory 43, and an arithmetic processing unit is applied to the image signal data stored in the internal memory 43. A sample image (image data) is generated by performing predetermined arithmetic processing 44. Therefore, the contents of the calculation for generating the sample image are shown below.

In a microscope optical system having a point image intensity distribution P (r) for a point r = (x, y) on an image, when illumination having an intensity distribution on the sample O (r) is given, the diffracted light in the sample is spatial Undergo modulation. In the case of sinusoidal illumination having a single spatial frequency component vector K, the spatial modulation components are the zero-order and ± first-order three modulation components. When these spatial modulations are expressed as m _l exp (ilKr + ilφ) (where l = 1, 0, −1), the image of the sample subjected to spatial modulation (incident light quantity I (r) to the image sensor 35) is It can be expressed as the following equation (1).

  Here, r is a position vector on the sample, Kr represents an inner product of the vectors, and * represents a convolution integral. Φ is the modulation phase.

When imaging is performed while shifting the diffraction grating 16 as described above, N images having the same modulation frequency and modulation amplitude and different only in the modulation phase φ are obtained. At this time, the j-th image signal intensity (imaging device) The amount of light incident on 35) I _j (r) is expressed by the following equation (2) where the structured illumination phase of the j-th image is φ _j, and N equations are obtained.

  Further, the incident light quantity I (r) is calculated from the measured data obtained by measuring the relationship between the incident light quantity to the image sensor 35 and the output signal intensity (image signal intensity) from the image sensor 35 in advance. Is approximated by the following equation (3).

  Here, a (r) represents the sensitivity of each light receiving element constituting the image sensor 35, and b (r) represents the background noise of each light receiving element. The prior measurement data is preferably stored in a memory (not shown) of the control device 40 as a table.

  FIG. 3 is an example of an image of the sensitivity a (r) of each light receiving element constituting the imaging element 35. In FIG. 3, a bright area indicates that the sensitivity of the light receiving element is high, and a dark area indicates that the sensitivity of the light receiving element is low. FIG. 4 is an example of an image of the background noise b (r) of each light receiving element. In FIG. 4, a bright area indicates that the background noise of the light receiving element is large, and a dark area indicates that the background noise of the light receiving element is small.

  Sensitivity a (r) and background noise b (r) for each light receiving element as shown in FIGS. 3 and 4 are obtained as follows. First, uniform illumination is performed on the image sensor 35. If the image sensor 35 (CCD camera) is connected to the microscope 1, uniform illumination can be performed on the image sensor 35 by placing a diffusion plate (not shown) on the objective pupil plane of the microscope 1 or in the vicinity thereof. desirable. At this time, the intensity of the light source (of the diffusion plate) is sufficiently weakened so that the image sensor 35 is saturated with an exposure time of about several seconds if possible.

  Next, while changing the exposure time from 0 until a part of the camera output is saturated, an image is acquired by the image sensor 35 at every predetermined exposure time. At this time, the exposure time is set as equal as possible, and images are acquired for several or more exposure times.

  Then, linear fitting is performed for each output signal intensity S (r) at each pixel (corresponding to each light receiving element) of the image sensor 35, and the sensitivity that satisfies the following expression (4) with respect to the exposure time t. a (r) and background noise b (r) are obtained.

  In addition, more reliability can be obtained by using the least square approximation for the linear fitting. At that time, it is preferable to use measurement data in a region with good linearity in which the output signal is not saturated. The image signal output from the image sensor 35 for obtaining the sensitivity a (r) and the background noise b (r) is input from the input unit 42 of the image processing device 41 to the internal memory 43, and the arithmetic processing unit 44 Sensitivity a (r) and background noise b (r) are obtained based on a predetermined program. A calculation unit for obtaining the sensitivity a (r) and the background noise b (r) may be provided separately.

  Further, the relationship between the incident light quantity I (r) and the output signal intensity S (r) varies depending on the temperature of the image sensor 35, the internal gain, and the like. Therefore, it is necessary to acquire and approximate data in accordance with the temperature and internal gain of the image sensor 35 actually used, and the sensitivity a (r) and background noise b (r) for each temperature and internal gain. It is desirable to obtain the data and store it in a memory (not shown) of the control device 40 as a table. Therefore, by operating the operation unit 49 (see FIG. 1) such as a keyboard connected to the control device 40, the control device 40 is operated to obtain the sensitivity a (r) and the background noise b (r). It comes to let you. This makes it possible to obtain sensitivity a (r) and background noise b (r) when necessary.

  The correction between the incident light amount I (r) and the output signal intensity S (r) in the image sensor 35 can be performed immediately after reading from the image sensor 35, but is performed simultaneously with the diffraction component separation calculation. Is advantageous in terms of processing speed by an MPU (Micro Processing Unit). Many MPUs are equipped with a cache memory, which is a technique that achieves high performance while avoiding a large increase in cost by utilizing the localization of processing found in normal data processing.

  Therefore, when image processing is performed by an MPU having such an architecture, it is desirable to perform operations for the same pixel as much as possible. Therefore, it is desirable to perform the process of correcting the output signal intensity S (r) together with the separation of diffraction components. From equations (2) and (3), N equations as shown in the following equation (5) can be obtained.

  In the equation (5), O (r) exp (ilKr) is an unknown, and these can be obtained by acquiring an image of N ≧ 3. When three of the N equations (5) are used, the following equation (6) is obtained.

  By solving this equation, O (r) exp (ilKr) * P (r) can be obtained.

  Or you may obtain | require from the image of N> = 4 using the least squares method by a well-known method. Noise can be reduced by using the least square method. In these, l = 0 corresponds to a normal microscopic image that is not super-resolution, and l = ± 1 corresponds to a super-resolution component.

  Each of O (r) exp (ilKr) * P (r) obtained above corresponds to Ok (k) Pk (r + lK) in the wave number space (ie, for l = -1, 0, 1 respectively). . Here, Ok (k) is the Fourier transform of the distribution of the illuminant in the specimen, and Pk (k) is a transfer function (OTF: Optical Transfer Function) of the microscope optical system when illumination without an intensity distribution is given. It is.

  The normal transfer function when illumination with no intensity distribution is given is −2 NA / λ ≦ k ≦ 2 NA / λ with respect to the NA of the objective lens 7 when the wavelength of light is λ. The obtained sample information O (r) exp (ilKr) * P (r) is −2NA / λ−K ≦ k ≦ 2NA / λ−K for l = −1 and −2NA / for l = 0. It can be seen that the information of −2 NA / λ + K ≦ k ≦ 2 NA / λ + K is included for λ ≦ k ≦ 2 NA / λ and l = 1. Therefore, since O (r) exp (ilKr) * P (r) as a whole includes information up to −2NA / λ−K ≦ k ≦ 2NA / λ + K, a microscopic image having a high resolution, that is, a sample image, is included. Obtainable.

  As a result, according to the structured illumination microscope 1 of the first embodiment, the output signal intensity S (r) from the image sensor 35 is corrected for each pixel of the image sensor 35 and the incident light amount I (r) to the image sensor 35 is corrected. ) (That is, the amount of light of the modulated image), the accurate amount of incident light can be calculated from the output signal of the image sensor 35 from the previously measured sensitivity of the image sensor 35 and variations in background noise. It is possible to acquire a more accurate specimen image that reflects as accurately as possible. In particular, by moving the diffraction grating 16 in the direction in which the grating patterns are arranged, a high effect can be obtained in the structured illumination microscope 1 configured to modulate the phase of the illumination light.

  In addition, the modulation incident on the image sensor 35 is performed using a function (Equation (3)) indicating the correlation between the amount of light I (r) incident on the image sensor 35 and the output signal intensity S (r) from the image sensor 35. If the light quantity of the image is obtained, the output signal intensity S (r) can be corrected by a relatively simple calculation.

  Further, according to the observation condition, it has an arithmetic processing unit 44 that obtains a function (Equation (3)) based on an image signal output from the image sensor 35 in a state where the image sensor 35 is irradiated with uniform light. Thus, the function (equation (3)) can be properly used, and a more accurate sample image can be acquired.

  The image obtained based on the correction as described above has a high resolution only in the one-dimensional direction in which spatial modulation is performed, but the direction in which spatial modulation is performed is changed to at least two directions, and 1 in each direction. A microscope having a high resolution in the two-dimensional direction can be configured by performing the same processing as the dimension.

  At this time, three directions are desirably applied to the spatial modulation. In the case of only two directions, the super-resolution effect is not so much obtained in the middle direction between the two directions, but in the three directions, the degree of super-resolution effect is not inferior to that in the direction between the three directions. Super-resolution effect up to can be obtained. When the number of directions is four or more, the effect of suppressing the decrease in super-resolution between the illumination directions is reduced instead of increasing the number of images.

  Next, a second embodiment of the structured illumination microscope will be described with reference to FIG. The structured illumination microscope 101 of the second embodiment is different from the structured illumination microscope 1 of the first embodiment in the configuration of the diffraction grating and the diffraction grating driving device, and the image processing method by the image processing device, Since other configurations are the same, the same members are denoted by the same reference numerals and detailed description thereof is omitted.

  The diffraction grating 116 configured in the illumination optical system 110 of the second embodiment is a diffraction grating having sinusoidal transmittance distributions orthogonal to each other in a plane perpendicular to the optical axis of the illumination optical system 110. The diffraction grating 116 is formed with two grating patterns arranged in two directions in a plane perpendicular to the optical axis, and each grating pattern does not necessarily have to be orthogonal, but the arithmetic processing described below is simple. Therefore, it is desirable to be orthogonal. Further, the diffraction grating 116 is connected to a diffraction grating driving device 125 that moves the diffraction grating 116 in the direction in which the two grating patterns of the diffraction grating 116 are arranged. The control device 40 controls the operation of the diffraction grating driving device 125. The

  The diffraction grating 116 generates diffracted light indicated by a broken line in FIG. 5 by a grating pattern extending along a direction perpendicular to the paper surface. In addition, diffracted light (not shown) passing through the top and bottom of the paper surface is generated by the lattice pattern extending along the direction in the paper surface. In order to obtain high spatial resolution according to the present embodiment, it is desirable that the diffraction grating 116 has a fine period within a range in which illumination light spatially modulated (by the diffraction grating 116) can be projected onto the sample 5. For this reason, only the ± first-order light has to be taken into consideration among the diffracted light. It should be noted that the period of the lattice pattern in the two directions is preferably the same, but if the resolution in the two directions is not a problem, the period in the two directions may be different.

  In addition, since it is sufficient if the specimen 5 can be sufficiently finely sine-wave illuminated to generate moire, oblique illumination is performed on a grating having a period near the imaging limit, or ± 1st order. It is preferable to dispose a light shielding plate on a plane conjugate with the pupil plane 8 of the objective lens 7 so that only the ± first-order light can pass through the diffraction grating 116 having a period capable of imaging light diffraction. Note that the diffraction grating 116 needs to have a sufficiently small 0th-order light component. Therefore, the diffraction grating 116 is not limited to the diffraction grating as described above, but a phase diffraction grating having an optical path difference of ½ wavelength with respect to the wavelength of the lamp light source 11. May be used.

  The ± first-order light is separated by the diffraction grating 116 so as to pass through the periphery of the field lens 17, and the + 1st-order light and the −1st-order light form spots at the positions of the periphery of the folding mirror 18. Here, the aperture stop 14 is adjusted so that the ± first-order light does not overlap on the folding mirror 18. Note that when monochromatic light is used as the light source 11, the ± first-order light does not overlap, and thus the aperture stop 14 need not be provided.

  The ± first-order light deflected by the folding mirror 18 passes through the peripheral portion of the pupil plane 8 of the objective lens 7 and interferes with each other in the vicinity of the sample 5 to become illumination light having a fringe structure and is irradiated on the sample 5. The reflected light from the specimen 5 irradiated with such (spatial-modulated) illumination light is guided to the image sensor 35 by the imaging optical system 30, and the sample 5 irradiated with the spatially-modulated illumination light is modulated. An image is formed.

  The imaging of the modulated image is performed by the image sensor 35. Since the diffraction grating 116 has a grating pattern in two directions orthogonal to each other, the sample 5 is illuminated with light based on the grating pattern in two directions orthogonal to each other. An image is acquired by the diffraction grating driving device 125 while shifting the diffraction grating 116 by exactly 1 / N of the period in the direction in which the grating patterns are arranged in two directions orthogonal to each other.

  That is, N images are acquired while shifting the position by 1 / N in the direction in which one of the two lattice patterns is arranged (this is the y direction), and then the direction in which the other lattice pattern is arranged ( The position is shifted by 1 / N in the x direction), and N images are acquired again while shifting the position by 1 / N in the y direction. Thereafter, this is repeated, and (N × N) images are acquired by shifting the position N times in the x direction.

  Image signals of a plurality of modulated images captured and acquired by the image sensor 35 are sent to the image processing device 141, and the arithmetic processing unit in the image processing device 141 performs predetermined arithmetic processing to generate a sample image (image data). To do. Therefore, the contents of the calculation for acquiring the specimen image by the structured illumination microscope 101 of the second embodiment are shown below.

When imaging is performed while shifting the diffraction grating 116 as described above, N images having the same phase in one grating pattern can be acquired, but these images have different modulation amplitudes depending on the other grating pattern. That is, if the lattice pattern directions are the x direction and the y direction, respectively, the structured illumination intensity I _ri can be generally expressed as the following equation (7).

Therefore, if the phase φ _x is shifted in the x direction so that φ _x = 0,..., 2πj / N _x ,..., 2π (N _x −1) / N _x , illumination in the y direction at a specific x The amplitude has a value proportional to cos (2πj / N _y ). That is, images with different modulation amplitudes are obtained.

Specifically, (7) to illuminate the sample in a structured illumination shown by the formula, and fixing the φ _y = 0, φ _x and _{φ x = 0, ..., 2πj} / N x, ..., 2π (N x - 1) Capture N _x specimen images while changing to / N _x . If the image signal at that time is S _j (r), the equation (4) can be used similarly to the one-dimensional case in the first embodiment, and the equation (8) similar to the equation (5) can be obtained.

Create three equations based on the equation (8) and solve them in the same manner as the equation (6), or use a method using a known least square method from an image of N ≧ 4 to obtain O (r) exp (il _x Kr) * P (r) is obtained (where l _x = -1, 0, 1), and this is defined as O _x (r) exp (il _x K _x x) * P (r).

Next, while changing these to φ _y = 2π / N _y ,..., 2πj / N _y ,..., 2π (N _y −1) / N _y , N _y sets are repeated including the case of φ _y = 0. _{When, O x (r) exp (} il x K x x) * P a (r) may be calculated N _y pairs.

Finally, when three groups are selected from N _y groups of O _x (r) exp (il _x K _x x) * P (r) obtained by changing φ _y , this corresponds to equation (6) (9 ) Can be made one for each of l _x = -1, 0, 1.

By solving this equation (9) for each _{l x, O (r) exp} (il x K x x + il y K y y) * can be calculated P a (r) (However, l _x = -1 , 0, 1 and l _y = -1, 0, 1).

Among these, the components of l _x = 0 and l _y = 0 correspond to the normal microscope image, but the other components are components corresponding to the super-resolution information, and thus are the same as in the case of the first embodiment. A super-resolution image can be obtained by combining.

  Although the case where the structured illumination phase is changed at equal intervals has been described here, the image can be demodulated by the correction calculation method described above even at unequal intervals. A sample image is created by performing a correction calculation on the image processing apparatus. The calculation process may be applied by extending the technique described in the first embodiment in two dimensions.

  As described above, according to the structured illumination microscope 101 of the second embodiment, the same effects as those of the first embodiment can be obtained.

  The above-described embodiments are merely examples, and are not limited to the above-described configuration and shape, and can be appropriately modified and changed within the scope of the present invention.

  In each of the above-described embodiments, a modulated image incident on the image sensor 35 using a function (equation (3)) indicating a correlation between the amount of light incident on the image sensor 35 and the output signal intensity from the image sensor 35. However, the present invention is not limited to this, and modulation that is incident on the image sensor 35 using a data table that indicates the amount of light incident on the image sensor 35 corresponding to the output signal intensity from the image sensor 35 is not limited thereto. You may make it obtain | require the light quantity of an image. Even if it does in this way, the effect similar to the above-mentioned case can be acquired. When such a data table is used, the amount of incident light can be obtained more accurately if interpolation calculation is performed.

  At this time, similarly to the case described above, a data table can be generated by an arithmetic processing unit or the like based on an image signal output from the image sensor 35 in a state where the image sensor 35 is irradiated with uniform light. The operation for obtaining such a data table is performed by the operation unit 49. Thereby, the same effect as the above-mentioned case can be acquired.

  In each of the above-described embodiments, a plurality of images obtained by modulating the phase of the illumination light are acquired. However, the present invention is not limited to this, and a plurality of images may be acquired by changing the modulation amplitude of the illumination light. It may be.

It is a schematic block diagram of the structured illumination microscope concerning 1st Embodiment. It is a control block diagram of an image processing apparatus. It is a figure which image-forms and shows the variation in the sensitivity of an image pick-up element. It is a figure which image-forms and shows the variation of the background noise of an image sensor. It is a schematic block diagram of the structured illumination microscope concerning 2nd Embodiment.

Explanation of symbols

1 Structured illumination microscope (first embodiment)
5 Specimen 10 Illumination Optical System 11 Lamp Light Source 16 Diffraction Grating 25 Diffraction Grating Drive Device (Phase Modulation Unit)
Reference Signs List 30 imaging optical system 35 image sensor 41 image processing device 44 arithmetic processing unit (correction unit and arithmetic unit)
49 Operation Unit 101 Structured Illumination Microscope (Second Embodiment)
110 Illumination optical system 116 Diffraction grating 125 Diffraction grating driving device (phase modulation unit)
141 Image processing apparatus

Claims (10)

An illumination optical system that has a diffraction grating that spatially modulates light emitted from a light source, and that irradiates a specimen with illumination light that is spatially modulated by the diffraction grating;
An imaging optical system that forms a modulated image from the sample irradiated with the illumination light;
An image sensor that captures the modulated image;
A phase modulation unit for modulating the phase of the illumination light;
An image processing device that generates an image of the sample from the plurality of modulated images captured by the imaging device each time the phase of the illumination light is modulated using the phase modulation unit,
The image processing apparatus includes a correction unit that corrects the intensity of an image signal output from the image sensor in units of pixels of the image sensor and converts the intensity into a light amount of the modulated image. Microscope device.   The correction unit uses the function indicating the correlation between the amount of light incident on the image sensor calculated in units of pixels and the intensity of the image signal output from the image sensor to modulate the incident light to the image sensor. The microscope apparatus according to claim 1, wherein the light quantity of the image is obtained.   The said image processing apparatus has a calculating part which calculates | requires the said function based on the image signal output from the said image pick-up element in the state in which the uniform light was irradiated to the said image pick-up element. Microscope device.   The microscope apparatus according to claim 3, further comprising an operation unit that performs an operation for irradiating the image sensor with uniform light to obtain the image signal from the image sensor. The image processing apparatus has a data table indicating the amount of incident light on the image sensor corresponding to the intensity of the image signal output from the image sensor, which is obtained in units of pixels.
The microscope apparatus according to claim 1, wherein the correction unit obtains a light amount of the modulated image incident on the image sensor using the data table.   The microscope apparatus according to claim 5, wherein the correction unit obtains a light amount of the modulated image incident on the imaging element by performing interpolation calculation from data in the data table.   The said image processing apparatus has a calculating part which produces | generates the said data table based on the image signal output from the said image pick-up element in the state in which the uniform light was irradiated to the said image pick-up element. 6. The microscope apparatus according to 6.   The microscope apparatus according to claim 7, further comprising an operation unit that performs an operation for irradiating the image sensor with uniform light to obtain the image signal from the image sensor. The diffraction grating has a grating pattern arranged in a direction perpendicular to the optical axis of the illumination optical system,
The microscope according to any one of claims 1 to 8, wherein the phase modulation unit modulates the phase of the illumination light by moving the diffraction grating in a direction in which the grating patterns are arranged. apparatus. The diffraction grating has two grating patterns arranged so as to be orthogonal to each other in a plane perpendicular to the optical axis of the illumination optical system,
The microscope according to any one of claims 1 to 8, wherein the phase of the illumination light is modulated by moving the diffraction grating in a direction in which one of the two grating patterns is aligned. apparatus.

Images