JP2015084062A - Device, method, and program for capturing microscopic images - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a device, method, and program for capturing microscopic images, which allow for capturing images using light having a pattern without deteriorating image quality.SOLUTION: Patterned measurement light is produced by a projection light modulation element 112 and is irradiated on an object S under measurement. A spatial phase of a produced pattern is sequentially shifted on the object S under measurement by predetermined increments by the light modulation element 112. A light receiving unit 120 receives light from the object S under measurement and outputs a light reception signal that represents received light intensity. A plurality of sets of image data generated at a plurality of phases of the pattern using the light reception signal from the light receiving unit 120 is used to generate sectioning image data representing an image of the object S under measurement. Based on periodicity of non-light-guiding sections between multiple modulation-pixels of the light modulation element 112, portions of the image data or sectioning image data corresponding to the non-light-guiding sections are removed by a filtering process.

Description

  The present invention relates to a microscope imaging apparatus, a microscope imaging method, and a microscope imaging program.

  A microscope imaging apparatus that generates an image that can be used to create a three-dimensional image by optical sectioning (cutting) has been proposed (see Patent Document 1).

  In the microscope imaging apparatus described in Patent Document 1, a mask is provided in a light source so that an object is illuminated with a spatially periodic pattern. Thereby, a mask pattern is projected on the sample. In order to adjust the spatial phase of the mask pattern, the mask is moved to at least three positions by the carriage. By analyzing three or more images of the object illuminated at at least three positions of the mask, a three-dimensional image of the object is generated.

Special table 2000-506634

  By using a light modulation element instead of a mechanical mechanism such as a carriage, it is possible to take an image using light having an arbitrary pattern. However, the image quality of an image obtained using a light modulation element may be lower than the image quality of an image obtained using a mechanical mechanism.

  An object of the present invention is to provide a microscope imaging apparatus, a microscope imaging method, and a microscope imaging program capable of imaging using light having an arbitrary pattern without degrading the image quality.

  (1) A microscope imaging apparatus according to a first aspect of the present invention includes a light projecting unit that emits light, and a plurality of second light sources configured to generate measurement light having an arbitrary pattern from the light emitted by the light projecting unit. A light modulation element including one pixel, a light projecting optical system that irradiates the measurement object with measurement light generated by the light modulation element, and receives light from the measurement object and outputs a received light signal indicating the amount of light received A light receiving unit, an image data generating unit that generates image data based on a light reception signal output from the light receiving unit, an image processing unit that performs image processing on the image data generated by the image data generating unit, and a pattern The light modulation element is controlled so that the spatial phase of the generated pattern is sequentially moved on the measurement object by a predetermined amount while generating the measurement light, and the plurality of image data generated at the plurality of phases of the pattern are generated. Based on measurement A control unit that controls the image data generation unit so as to generate sectioning image data indicating an image of the elephant, and the light modulation element includes a periodic non-light-guiding unit between the plurality of first pixels. The image processing unit executes a filter process for removing a portion corresponding to the non-light-guiding unit in the image data or the sectioning image data based on the periodicity of the non-light-guiding unit of the light modulation element.

  In this microscope imaging apparatus, measurement light having an arbitrary pattern is generated from light emitted from the light projecting unit by a light modulation element including a plurality of first pixels. Measurement light generated by the light modulation element is irradiated onto the measurement object by the light projecting optical system. The spatial phase of the generated pattern is sequentially moved on the measurement object by a predetermined amount by the light modulation element. Light from the measurement object is received by the light receiving unit, and a light reception signal indicating the amount of light received is output. Based on the plurality of image data generated at the plurality of phases of the pattern based on the light reception signal output from the light receiving unit, sectioning image data indicating the image of the measurement object is generated.

  Here, the light modulation element includes a periodic non-light-guiding unit in the plurality of first pixels. No measurement light is generated from the non-light guide portion of the light modulation element. Therefore, a shadow pattern corresponding to the non-light guide portion of the light modulation element is reflected in the image represented by the generated image data. Even in such a case, based on the periodicity of the non-light guide portion of the light modulation element, the portion corresponding to the non-light guide portion in the image data or the sectioning image data is removed by the filtering process. Thereby, imaging using light having an arbitrary pattern can be performed without degrading the image quality of the image.

  (2) The microscope imaging apparatus further includes a first instruction unit for instructing whether or not to execute the filter process, and the control unit is configured to execute the filter process when instructed to execute the filter process by the first instruction unit. May execute the filtering process, and may control the image processing unit not to execute the filtering process when the first instructing unit does not instruct the execution of the filtering process.

  In this case, whether or not to execute the filter process is instructed by the first instruction unit. Accordingly, it is possible to execute the filter process only when a shadow pattern corresponding to the non-light guide portion of the light modulation element does not appear in the image. As a result, it is possible to prevent the filtering process from being performed on image data or sectioning image data representing an image in which no shadow pattern is reflected.

  (3) The microscope imaging apparatus further includes a second instruction unit for instructing the first operation mode or the second operation mode, and the light modulation element has an arbitrary pattern from the light emitted by the light projecting unit. And the control unit is configured to selectively generate the measurement light having a pattern and the measurement light having the pattern when the first operation mode is instructed by the second instruction unit. The light modulation element is controlled so that the spatial phase of the generated pattern is sequentially moved on the measurement target by a predetermined amount, and sectioning is performed based on a plurality of image data generated at a plurality of phases of the pattern. When the image data generation unit is controlled to generate image data and the second instruction mode is instructed by the second instruction unit, the light modulation element is generated so as to generate measurement light having no pattern. Controls may control the image data generation unit to generate the normal image data representing the image of the measuring object when no measuring light pattern is irradiated.

  In this case, the first operation mode or the second operation mode is instructed by the second instruction unit. In the first operation mode, measurement light having a pattern is generated by the light modulation element and irradiated onto the measurement object. Further, the spatial phase of the generated pattern is sequentially moved on the measurement object by a predetermined amount by the light modulation element. Sectioning image data is generated based on a plurality of image data generated at a plurality of phases of the pattern.

  On the other hand, in the second operation mode, measurement light having no pattern is generated by the light modulation element and irradiated onto the measurement object. Thereby, normal image data indicating an image of the measurement object when the measurement light having no pattern is irradiated is generated.

  As described above, by using the light modulation element, it is possible to irradiate the measurement object with measurement light having an arbitrary pattern and measurement light having no pattern by a common light projecting optical system. Thereby, sectioning image data when using measurement light having an arbitrary pattern and normal image data when using measurement light having no pattern can be obtained.

  Further, a portion corresponding to the non-light guide portion in the image data, sectioning image data, or normal image data is removed by the filtering process. Therefore, in the first and second operation modes, it is possible to generate image data indicating an image in which deterioration in image quality is prevented.

  (4) The filtering process may include a spatial filtering process. In this case, the portion corresponding to the non-light guide portion of the light modulation element in the image data or the sectioning image data can be easily removed.

  (5) The image processing unit may sequentially perform a waveform conversion process, a spatial filter process, and an inverse conversion process of the waveform conversion process as the filter process.

  In this case, the part of the image data or sectioning image data corresponding to the non-light-guiding part of the light modulation element is separated from the other parts by the waveform conversion process. Thereby, the part equivalent to the non-light guide part of the light modulation element in the image data or the sectioning image data can be efficiently removed by the spatial filter processing. As a result, it is possible to obtain an image from which a portion corresponding to the non-light guide portion of the light modulation element is removed by the inverse conversion process.

  (6) The waveform conversion process may include discrete Fourier transform or discrete wavelet transform.

  In this case, based on the spatial frequency corresponding to the non-light guide portion of the light modulation element, the portion corresponding to the non-light guide portion of the light modulation element in the image data or the sectioning image data is separated with high accuracy from the other portions. Can do.

  (7) The spatial filter process may include a blur filter process.

  In this case, the image is blurred by the blur filter process. As a result, a portion corresponding to the non-light guide portion of the light modulation element in the image data or the sectioning image data can be easily removed without significantly reducing the image quality.

  (8) The blur filter process may include a Gaussian filter process.

  In this case, the portion corresponding to the non-light-guiding portion of the light modulation element is blurred with a greater weight than other portions of the image by the Gaussian filter processing. As a result, the portion corresponding to the non-light guide portion of the light modulation element in the image data or sectioning image data can be easily removed without substantially reducing the image quality.

  (9) The light modulation element, the light receiving unit, and the light projection optical system may be configured such that the resolution of the light projection optical system is lower than the width of the gap between the plurality of first pixels of the light modulation element.

  In this case, since the resolution of the light projecting optical system is lower than the width of the gap between the plurality of first pixels of the light modulation element, an image based on the generated sectioning image data is displayed on the non-light guide portion of the light modulation element. The corresponding shadow pattern is not reflected. This makes it possible to generate sectioning image data indicating an image that hardly deteriorates in image quality.

  (10) One of the light modulation element and the light receiving unit has a plurality of light modulation elements whose resolution is higher than the other of the light modulation element and the light reception unit from a position having a conjugate relationship via the light projection optical system. It may be arranged at a position shifted until it becomes lower than the width of the gap between the first pixels.

  In this case, the resolution of the light projecting optical system can be easily made lower than the width of the gap between the plurality of first pixels of the light modulation element.

  (11) The value of the modulation transfer function of the light projecting optical system may be equal to or less than a predetermined threshold at a spatial frequency corresponding to the width of the gap between the plurality of first pixels of the light modulation element. .

  In this case, the resolution of the light projecting optical system can be easily made lower than the width of the gap between the plurality of first pixels of the light modulation element.

  (12) On the plane including the light receiving surface of the light receiving unit, the light projecting optical system may be configured so that the irradiation range of light from the measurement object to the light receiving unit includes the light receiving surface of the light receiving unit.

  Due to the misalignment of the light projecting optical system, the irradiation range of light from the measurement object to the light receiving unit may deviate from the range corresponding to the light receiving surface of the light receiving unit. When light is not irradiated to a part of the light receiving surface of the light receiving unit, a part of the image based on the generated sectioning image data is missing. According to said structure, the irradiation range of the light from a measuring object to a light-receiving part includes the light-receiving surface of a light-receiving part. Therefore, most of the light enters the light receiving surface of the light receiving unit. Thereby, sectioning image data indicating an image in which a loss due to the deviation of the arrangement of the projection optical system is prevented can be generated.

  (13) The light receiving unit includes a plurality of second pixels arranged in first and second directions intersecting each other, and each of the first light modulation elements on the plane including the light receiving surface of the light receiving unit. The area below the area including the three second pixels arranged in the first direction and the three second pixels arranged in the second direction in the portion of the light from the measurement object corresponding to the pixel to the light receiving unit The projection optical system may be configured to irradiate the light.

  When the light irradiation range from the measurement object to the light receiving part is too large than the light receiving surface of the light receiving part, the minimum width of the measurement light pattern by the light modulation element is increased, so that the details of the measurement object are highly accurate The image capturing performance (sectioning performance) decreases. According to said structure, the irradiation range of the part of the light corresponding to each 1st pixel of a light modulation element is located in a line with the 3rd 2nd pixel arranged in a 1st direction, and a 2nd direction The area is limited to an area equal to or less than an area including three second pixels. Thereby, the minimum width | variety of the pattern of the measurement light by a light modulation element can be made small. As a result, it is possible to prevent the sectioning performance from being lowered.

  (14) In the microscope imaging method according to the second invention, an arbitrary pattern is formed by the step of emitting light by the light projecting unit and the light modulation element including a plurality of first pixels from the light emitted by the light projecting unit. Generating measurement light having a step, irradiating a measurement object with measurement light generated by a light modulation element by a projection optical system, and a spatial phase of the generated pattern by a predetermined amount by the light modulation element A step of sequentially moving on the measuring object, a step of receiving light from the measuring object by the light receiving unit, outputting a light receiving signal indicating the amount of received light, and a plurality of patterns based on the light receiving signal output from the light receiving unit Generating sectioning image data indicating an image of the measurement object based on the plurality of image data generated at the phase of the non-light-guiding unit between the plurality of first pixels of the light modulation element Based on the period property, it is intended to include a step of removing a portion corresponding to the non-light unit in the image data or sectioning image data.

  According to this microscope imaging method, measurement light having an arbitrary pattern is generated from the light emitted from the light projecting unit by the light modulation element including the plurality of first pixels. Measurement light generated by the light modulation element is irradiated onto the measurement object by the light projecting optical system. The spatial phase of the generated pattern is sequentially moved on the measurement object by a predetermined amount by the light modulation element. Light from the measurement object is received by the light receiving unit, and a light reception signal indicating the amount of light received is output. Based on the plurality of image data generated at the plurality of phases of the pattern based on the light reception signal output from the light receiving unit, sectioning image data indicating the image of the measurement object is generated.

  Here, the light modulation element includes a periodic non-light-guiding unit in the plurality of first pixels. No measurement light is generated from the non-light guide portion of the light modulation element. Therefore, a shadow pattern corresponding to the non-light guide portion of the light modulation element is reflected in the image represented by the generated image data. Even in such a case, based on the periodicity of the non-light guide portion of the light modulation element, the portion corresponding to the non-light guide portion in the image data or the sectioning image data is removed. Thereby, imaging using light having an arbitrary pattern can be performed without degrading the image quality of the image.

  (15) A microscope imaging program according to a third invention is a microscope imaging program that can be executed by a processing device, and includes a plurality of first processes based on a process of emitting light by a light projecting unit and light emitted by the light projecting unit. A process for generating measurement light having an arbitrary pattern by a light modulation element including one pixel, a process for irradiating a measurement object with measurement light generated by the light modulation element by a light projection optical system, and a generated pattern A process of sequentially moving the spatial phase of the object on the measurement object by a predetermined amount by the light modulation element, a process of receiving light from the measurement object by the light receiving unit, and outputting a light reception signal indicating the amount of received light; Processing for generating sectioning image data indicating an image of a measurement object based on a plurality of image data generated at a plurality of phases of a pattern based on a light reception signal output from the unit, and light modulation The processing device executes processing for removing a portion corresponding to the non-light guide portion in the image data or the sectioning image data based on the periodicity of the non-light guide portion between the plurality of first pixels of the child. is there.

  According to this microscope imaging program, measurement light having an arbitrary pattern is generated from the light emitted from the light projecting unit by the light modulation element including the plurality of first pixels. Measurement light generated by the light modulation element is irradiated onto the measurement object by the light projecting optical system. The spatial phase of the generated pattern is sequentially moved on the measurement object by a predetermined amount by the light modulation element. Light from the measurement object is received by the light receiving unit, and a light reception signal indicating the amount of light received is output. Based on the plurality of image data generated at the plurality of phases of the pattern based on the light reception signal output from the light receiving unit, sectioning image data indicating the image of the measurement object is generated.

  Here, the light modulation element includes a periodic non-light-guiding unit in the plurality of first pixels. No measurement light is generated from the non-light guide portion of the light modulation element. Therefore, a shadow pattern corresponding to the non-light guide portion of the light modulation element is reflected in the image represented by the generated image data. Even in such a case, based on the periodicity of the non-light guide portion of the light modulation element, the portion corresponding to the non-light guide portion in the image data or the sectioning image data is removed. Thereby, imaging using light having an arbitrary pattern can be performed without degrading the image quality of the image.

  According to the present invention, imaging using light having an arbitrary pattern can be performed without degrading the image quality.

It is a block diagram which shows the structure of the microscope imaging device which concerns on one embodiment of this invention. It is a schematic diagram which shows the structure of the measurement part and measurement light supply part of the microscope imaging device of FIG. It is a schematic diagram which shows the optical path in the measurement part of the microscope imaging device of FIG. It is a block diagram which shows the structure of CPU. It is a figure which shows the example of the measurement light radiate | emitted by a pattern provision part. It is a figure which shows the normal image of a measuring object. It is a figure which shows an example of a normal image and a conversion image. It is a figure which shows an example of the pattern measurement light used for calculation of the spatial frequency of a shadow pattern. It is a figure for demonstrating the adjustment method of projection resolution size. It is a figure which shows the relationship between the irradiation range of the fluorescence on the plane containing a light-receiving surface, and a light-receiving surface. FIG. 4 is an enlarged view of a part of a light irradiation surface and a fluorescence irradiation range. It is a typical enlarged view of a light-receiving part.

(1) Configuration of Microscope Imaging Device FIG. 1 is a block diagram showing a configuration of a microscope imaging device according to an embodiment of the present invention. FIG. 2 is a schematic diagram illustrating the configuration of the measurement unit and the measurement light supply unit 300 of the microscope imaging apparatus 500 of FIG. FIG. 3 is a schematic diagram showing an optical path in the measurement unit of the microscope imaging apparatus 500 of FIG.

  Hereinafter, a microscope imaging apparatus 500 according to the present embodiment will be described with reference to FIGS. As shown in FIG. 1, the microscope imaging apparatus 500 includes a measurement unit 100, a PC (personal computer) 200, a measurement light supply unit 300, and a display unit 400.

  As shown in FIG. 2, the measurement light supply unit 300 includes a power supply device 310, a light projecting unit 320, and a light guide member 330. In this example, the light guide member 330 is a liquid light guide. The light guide member 330 may be, for example, a glass fiber or a quartz fiber. The power supply device 310 supplies power to the light projecting unit 320 and supplies power to the measurement unit 100 via a power cable (not shown).

  The light projecting unit 320 includes a measurement light source 321, a light reduction mechanism 322, and a light shielding mechanism 323. The measurement light source 321 is, for example, a metal halide lamp. The measurement light source 321 may be another light source such as a mercury lamp or a white LED (light emitting diode). Hereinafter, the light emitted from the measurement light source 321 is referred to as measurement light.

  The dimming mechanism 322 includes a plurality of ND (Neutral Density) filters having different transmittances. The dimming mechanism 322 is arranged such that any one of the plurality of ND filters is positioned on the optical path of the measurement light emitted from the measurement light source 321. By selectively switching the ND filter located on the optical path of the measurement light, the intensity of the measurement light passing through the dimming mechanism 322 can be adjusted. The dimming mechanism 322 may include an optical element such as an optical modulator capable of adjusting the light intensity instead of the plurality of ND filters.

  The light shielding mechanism 323 is, for example, a mechanical shutter. The light shielding mechanism 323 is disposed on the optical path of the measurement light that has passed through the dimming mechanism 322. When the light shielding mechanism 323 is in the open state, the measurement light passes through the light shielding mechanism 323 and is input to the light guide member 330. On the other hand, when the light shielding mechanism 323 is in the closed state, the measurement light is blocked and is not input to the light guide member 330. The light blocking mechanism 323 may include an optical element such as an optical modulator that can switch between passage and blocking of light.

  The measuring unit 100 is, for example, a fluorescence microscope, and includes a fluorescent illumination lens 101, a pattern applying unit 110, a light receiving unit 120, a transmitted light supply unit 130, a stage 140, a filter unit 150, a lens unit 160, and a control board 170. The light receiving unit 120, the filter unit 150, the lens unit 160, the stage 140, and the transmitted light supply unit 130 are arranged in this order from the bottom to the top.

  On the stage 140, the measuring object S is placed. In this example, the measuring object S is a biological specimen containing various proteins. The measuring object S is coated with a fluorescent reagent that fuses with a specific protein. Fluorescent reagents include, for example, GFP (green fluorescent protein), Texas Red (Texas Red), and DAPI (diamidinophenylindole).

  GFP absorbs light near 490 nm and emits light with a wavelength near 510 nm. Texas Red absorbs light near a wavelength of 596 nm and emits light near a wavelength of 620 nm. DAPI absorbs light near a wavelength of 345 nm and emits light near a wavelength of 455 nm.

  Two directions perpendicular to each other in a plane (hereinafter referred to as a placement surface) on the stage 140 on which the measurement object S is placed are defined as an X direction and a Y direction, and the directions perpendicular to the placement surface are defined as the directions. It is defined as the Z direction. In the present embodiment, the X direction and the Y direction are horizontal directions, and the Z direction is a vertical direction. In the present embodiment, stage 140 is an XY stage, and is arranged so as to be movable in the X direction and the Y direction by a stage drive unit (not shown).

  The measurement unit 100 includes an optical system that guides measurement light emitted from the measurement light supply unit 300 to the measurement object S, an optical system that guides light emitted from the transmitted light supply unit 130 to the measurement object S, and the measurement object. An optical system for guiding the light from S to the light receiving unit 120 is configured.

  The pattern providing unit 110 includes a light output unit 111, a light modulation element 112, and a plurality (two in this example) of mirrors 113. The light output unit 111 outputs the measurement light input to the light guide member 330. The light output from the light output unit 111 is reflected by the plurality of mirrors 113 and enters the light modulation element 112.

  The light modulation element 112 is, for example, a DMD (digital micromirror device). The DMD is composed of a plurality of micromirrors arranged two-dimensionally. The light modulation element 112 may be LCOS (Liquid Crystal on Silicon: reflection type liquid crystal element) or LCD (liquid crystal display). The light incident on the light modulation element 112 is converted into a preset pattern and preset intensity (brightness) by a pattern generation unit 212 described later, and is emitted to the filter unit 150 through the fluorescent illumination lens 101. .

  The filter unit 150 includes a plurality (three in this example) of filter cubes 151 and filter turrets 152. The plurality of filter cubes 151 correspond to a plurality of types of fluorescent reagents applied to the measurement object S. As shown in FIG. 3, each filter cube 151 includes a frame 151a, an excitation filter 151b, a dichroic mirror 151c, and an absorption filter 151d. The frame 151a is a cubic member that supports the excitation filter 151b, the dichroic mirror 151c, and the absorption filter 151d.

  The excitation filter 151b in FIG. 2 is a band-pass filter that passes light in the first wavelength band. The absorption filter 151d is a band-pass filter that passes light in a second wavelength band different from the first wavelength band. The dichroic mirror 151c is a mirror that reflects light in a wavelength band including the first wavelength band and passes light in a wavelength band including the second wavelength band. The first and second wavelength bands are different for each filter cube 151 depending on the absorption wavelength and emission wavelength of the fluorescent reagent.

  The filter turret 152 has a disk shape. In the present embodiment, the filter turret 152 is provided with four filter cube mounting portions 152a at approximately 90 ° intervals. Each filter cube attaching part 152a is an opening formed so that the filter cube 151 can be attached.

  In the present embodiment, three filter cubes 151 are attached to the three filter cube attachment parts 152a, respectively, and the filter cube 151 is not attached to the remaining one filter cube attachment part 152a. Therefore, it is possible to perform bright field observation without using the filter cube 151 by arranging the filter cube attachment portion 152a to which the filter cube 151 is not attached on the optical path of the measurement light. In the example of FIG. 2, two filter cubes 151 are attached to the filter turret 152.

  The filter turret 152 is rotatably arranged at a predetermined angular interval (90 ° interval in this example) around an axis parallel to the Z direction by a filter turret driving unit (not shown). The user selects the filter cube 151 used for measuring the measurement object S by rotating the filter turret 152 by operating the operation unit 250 of the PC 200 described later.

  The selected filter cube 151 is attached to the filter unit 150 so that the measurement light enters the excitation filter 151b. As shown in FIG. 3, when the measurement light is incident on the excitation filter 151b, only the component having the first wavelength band in the measurement light passes through the excitation filter 151b. The measurement light that has passed through the excitation filter 151b is reflected by the dichroic mirror 151c toward the upper lens unit 160 (FIG. 2).

  The lens unit 160 includes a plurality (six in this example) of objective lenses 161, a lens turret 162, and a focal position adjustment mechanism 163. The plurality of objective lenses 161 have different magnifications. The lens turret 162 has a disk shape. In the present embodiment, six objective lens mounting portions 162 a are provided on the lens turret 162 at approximately 60 ° intervals. Each objective lens attachment portion 162a is an opening formed so that the objective lens 161 can be attached.

  In the present embodiment, six objective lenses 161 are attached to the six objective lens attachment portions 162a, respectively. In the example of FIG. 2, three objective lenses 161 are attached to the lens unit 160.

  The lens turret 162 is disposed so as to be rotatable at a predetermined angular interval (60 ° interval in this example) around an axis parallel to the Z direction by a lens turret driving unit (not shown). The user selects the objective lens 161 used for measuring the measuring object S by operating the operation unit 250 of the PC 200 described later and rotating the lens turret 162. The selected objective lens 161 overlaps the selected filter cube 151. Thereby, as shown in FIG. 3, the measurement light reflected by the dichroic mirror 151 c of the filter cube 151 passes through the selected objective lens 161.

  The focal position adjusting mechanism 163 in FIG. 2 is arranged so that the lens turret 162 can be moved in the Z direction by a focal position adjusting mechanism driving unit (not shown). Thereby, the relative distance between the measuring object S on the stage 140 and the selected objective lens 161 is adjusted. The stage 140 has an opening at a substantially central portion. The measurement light that has passed through the objective lens 161 passes through the opening of the stage 140 while being condensed and is applied to the measurement object S.

  The measurement object S irradiated with the measurement light absorbs the measurement light and emits fluorescence in a wavelength band including the second wavelength band. The fluorescence emitted below the measuring object S passes through the selected objective lens 161 and the dichroic mirror 151c and the absorption filter 151d of the selected filter cube 151. Thereby, a component having the second wavelength band in the fluorescence enters the light receiving unit 120.

  In the present embodiment, the measurement unit 100 is a fluorescence microscope capable of observing fluorescence from the measurement object S, but is not limited thereto. The measurement unit 100 may be, for example, a reflection microscope. In this case, a half mirror is attached to the filter cube attachment portion 152 a of the filter turret 152 instead of the filter cube 151.

  The transmitted light supply unit 130 is used for bright field observation, phase difference observation, differential interference observation, dark field observation, declination observation, or polarization observation of the measurement object S. The transmitted light supply unit 130 includes a transmitted light source 131 and a transmitted optical system 132. The transmissive light source 131 is, for example, a white LED. The transmissive light source 131 may be another light source such as a halogen lamp. Hereinafter, the light emitted from the transmissive light source 131 is referred to as transmitted light.

  The transmission optical system 132 includes optical elements such as an aperture stop, a phase difference slit, a relay lens, a condenser lens, and a shutter. The transmitted light emitted from the transmissive light source 131 passes through the transmissive optical system 132 and is applied to the measurement object S on the stage 140.

  The transmitted light passes through the measuring object S and passes through the objective lens 161. Thereafter, the transmitted light passes through the filter cube attachment portion 152a of the filter turret 152 to which the filter cube 151 is not attached, and enters the light receiving portion 120.

  The light receiving unit 120 includes a camera 121, a color filter 122, and an imaging lens 123. The camera 121 is, for example, a CCD (charge coupled device) camera including an image sensor. The image sensor is, for example, a monochrome CCD. The image sensor may be a color CCD or another image sensor such as a CMOS (complementary metal oxide semiconductor) image sensor. In the case where the image sensor is a color CCD, the light filter 120 is not provided with the color filter 122.

  The fluorescence or transmitted light incident on the light receiving unit 120 is collected and imaged by the imaging lens 123 and then received by the camera 121 through the color filter 122. Thereby, the image of the measuring object S is obtained. From each pixel of the image sensor of the camera 121, an analog electric signal (hereinafter referred to as a light reception signal) corresponding to the amount of received light is output to the control board 170.

  On the control board 170, an A / D converter (analog / digital converter) and a FIFO (First In First Out) memory (not shown) are mounted. The light reception signal output from the camera 121 is sampled at a constant sampling period by the A / D converter and converted into a digital signal based on control by the PC 200. Digital signals output from the A / D converter are sequentially stored in the FIFO memory. The digital signals stored in the FIFO memory are sequentially transferred to the PC 200 as pixel data.

  The control board 170 controls operations of the pattern applying unit 110, the light receiving unit 120, the transmitted light supply unit 130, the stage 140, the filter unit 150, and the lens unit 160 based on control by the PC 200. Further, the control board 170 controls the operation of the light projecting unit 320 of the measurement light supply unit 300 based on the control by the PC 200.

  As shown in FIG. 1, the PC 200 includes a CPU (Central Processing Unit) 210, a ROM (Read Only Memory) 220, a RAM (Random Access Memory) 230, a storage device 240 and an operation unit 250. The operation unit 250 includes a keyboard and a pointing device. A mouse or a joystick is used as the pointing device.

  The display unit 400 is configured by, for example, an LCD panel or an organic EL (electroluminescence) panel. In the example of FIG. 2, the PC 200 and the display unit 400 are realized by a single notebook personal computer.

  The ROM 220 stores a system program. The RAM 230 is used for processing various data. The storage device 240 is composed of a hard disk or the like. The storage device 240 stores an image processing program and a microscope imaging program. The storage device 240 is used for storing various data such as pixel data provided from the measurement unit 100.

  FIG. 4 is a block diagram showing the configuration of the CPU 210. As illustrated in FIG. 4, the CPU 210 includes an image data generation unit 211, a pattern generation unit 212, a control unit 213, and an image processing unit 214. The image data generation unit 211 generates image data based on the pixel data given from the measurement unit 100. Image data is a set of a plurality of pixel data. The image processing unit 214 performs image processing on an image representing the image data generated by the image data generation unit 211. Details will be described later.

  The pattern generation unit 212 generates a pattern to be irradiated to the measurement object while sequentially moving the spatial phase by a predetermined amount as the measurement light pattern emitted from the light modulation element 112 in FIG. The control unit 213 irradiates the measurement object S with measurement light having a predetermined pattern by controlling the light modulation element 112 via the control board 170 in FIG. 2 based on the pattern generated by the pattern generation unit 212. While shifting the phase of the pattern.

  The control unit 213 controls operations of the light receiving unit 120, the transmitted light supply unit 130, the stage 140, the filter unit 150, the lens unit 160, and the light projecting unit 320 via the control board 170. Further, the control unit 213 performs various processes on the generated image data using the RAM 230 and causes the display unit 400 to display an image based on the image data.

  In the measurement unit 100, the position of the focal point of the objective lens 161 with respect to the measurement target S (hereinafter, the focal position of the objective lens 161) is changed by changing the relative distance between the measurement target S and the objective lens 161 of FIG. Changes). The measuring object S is irradiated with measurement light while the focal position of the objective lens 161 is changed. Thereby, image data of the measuring object S at each focal position is generated.

  In the microscope imaging apparatus 500 according to the present embodiment, it is possible to perform epi-illumination observation of the measurement object S using the light projecting unit 320 of FIG. 2 and transmission observation of the measurement object S using the transmitted light supply unit 130. It can be performed.

  As epi-illumination observation, sectioning observation using patterned measurement light and normal observation using uniform measurement light described below can be performed.

(2) Sectioning Observation and Normal Observation In sectioning observation, the phase of the pattern is moved by a certain amount while irradiating the measuring object S with measuring light having a one-dimensional or two-dimensional pattern. The measurement light having a one-dimensional pattern has an intensity that periodically changes in one direction (for example, the Y direction) on the XY plane. The measurement light having a two-dimensional pattern has an intensity that periodically changes in two directions (for example, the X direction and the Y direction) intersecting each other on the XY plane.

  Hereinafter, measurement light having a pattern is referred to as pattern measurement light. In particular, measurement light having a one-dimensional pattern is called a one-dimensional pattern measurement light, and measurement light having a two-dimensional pattern is called a two-dimensional pattern measurement light. Further, measurement light having a uniform intensity is referred to as uniform measurement light.

  The pattern of the pattern measurement light is controlled by the light modulation element 112. Hereinafter, the pattern of the pattern measuring light will be described. Here, the portion of the pattern measurement light having an intensity equal to or higher than a predetermined value is called a bright portion, and the portion of the pattern measurement light having an intensity smaller than the predetermined value is called a dark portion. FIG. 5 is a diagram illustrating an example of measurement light emitted by the pattern imparting unit 110.

  FIG. 5A shows an example of the one-dimensional pattern measurement light. The one-dimensional pattern measurement light in FIG. 5A is referred to as rectangular wave measurement light. The cross section of the rectangular wave-shaped measurement light includes a plurality of linear bright portions that are parallel to one direction (for example, the X direction) and arranged at substantially equal intervals in another direction (for example, the Y direction) orthogonal to the one direction. A plurality of linear dark portions are included between the light portions.

  FIG. 5B shows another example of the one-dimensional pattern measurement light. The one-dimensional pattern measurement light in FIG. 5B is referred to as a one-dimensional sinusoidal measurement light. The cross section of the one-dimensional sinusoidal measurement light includes, for example, a pattern that is parallel to the X direction and whose intensity changes sinusoidally in the Y direction.

  FIG. 5C shows an example of the two-dimensional pattern measurement light. The two-dimensional pattern measurement light in FIG. 5C is referred to as dot measurement light. The cross section of the dot-shaped measuring light includes a plurality of dot-shaped bright portions arranged at substantially equal intervals in the X direction and the Y direction.

  As another example of the two-dimensional pattern measurement light, the pattern measurement light may be a two-dimensional sinusoidal measurement light. The cross section of the two-dimensional sinusoidal measurement light includes a pattern in which the intensity changes sinusoidally in the X direction and the Y direction. As still another example of the two-dimensional pattern measurement light, the pattern measurement light may have a lattice pattern or a checker pattern (checkered pattern).

  In sectioning observation, the pattern measurement light is emitted by the measurement object S while moving the phase of the pattern measurement light by a certain amount so that the bright part of the pattern measurement light is irradiated at least once over the entire irradiation range of the measurement light. Detect fluorescence. Thereby, a plurality of image data of the measuring object S is generated.

  Hereinafter, the image data obtained when the measurement object S is irradiated with the pattern measurement light is referred to as pattern image data. An image based on the pattern image data is called a pattern image.

  In each pattern image data, pixel data corresponding to a bright portion of the pattern measurement light has a high value (luminance value), and pixel data corresponding to a dark portion of the pattern measurement light has a low value (luminance value). Therefore, in each pattern image, the pixel corresponding to the bright part of the pattern measurement light is bright, and the pixel corresponding to the dark part of the pattern measurement light is dark.

  A component (hereinafter referred to as an in-focus component) representing the degree of contrast is calculated from the plurality of pattern image data using the values of the plurality of pixel data for each pixel. Image data generated by connecting pixels having a focusing component is referred to as sectioning image data. An image based on the sectioning image data is called a sectioning image.

  In pattern image data generated using rectangular wave-shaped measurement light or dot-shaped measurement light, the focus component is, for example, the difference between the maximum value (maximum luminance value) and minimum value (minimum luminance value) of pixel data, or This is the standard deviation of the pixel data value. In the pattern image data generated using the one-dimensional sinusoidal measurement light or the two-dimensional sinusoidal measurement light, the focus component is, for example, the amplitude (peak-to-peak) of the pixel data.

  In the simplest method, it is possible to generate sectioning image data by selecting pixel data having the maximum value from a plurality of pattern image data for each pixel and connecting the selected pixel data for all pixels. It is.

  Here, each pattern image is affected by stray light. As a result, the pattern of the pattern image is blurred. A component caused by stray light in each pattern image data is called a blur component. The blur component includes a blur component due to stray light generated in each bright portion of the pattern measurement light and a blur component due to stray light from another bright portion adjacent to each bright portion of the pattern measurement light.

  Therefore, in order to remove the influence of stray light, the difference between the pixel data of the pattern image data when the bright portion and the dark portion of the pattern measurement light are irradiated is calculated for each pixel. Sectioning image data is generated by connecting the calculated differences for all pixels. For each pixel, the value of the pixel data of the pattern image data when the dark portion of the pattern measurement light is irradiated corresponds to a blur component. Therefore, sectioning image data from which the influence of stray light is removed can be obtained.

  As an example of a method for generating sectioning image data, in this embodiment, for each pixel, a difference between a maximum value (maximum luminance value) and a minimum value (minimum luminance value) of a plurality of pixel data of a plurality of pattern image data is calculated. calculate. Sectioning image data is generated by connecting the calculated differences for all pixels. The sectioning image may be generated based on a plurality of pixel data of the plurality of pattern image data by another method.

  For example, for each pixel, the standard deviation of the values of the plurality of pixel data of the plurality of pattern image data is calculated. Sectioning image data may be generated by connecting standard deviations for all the calculated pixels.

  In sectioning observation, when the one-dimensional pattern measurement light is used, the phase of the pattern measurement light is moved in the Y direction, for example, so that sectioning image data from which the blur component in the Y direction has been removed is generated. Further, since the phase of the pattern measurement light does not need to be moved in the X direction, for example, the number of imaging is reduced. Therefore, a sectioning image having a relatively high image quality can be obtained at high speed.

  On the other hand, in the sectioning observation, when the two-dimensional pattern measurement light is used, the phase of the pattern measurement light is moved in the X direction and the Y direction, so that the sectioning image data from which the blur component in the X direction and Y direction is removed. Generated. Therefore, although the number of times of imaging is increased as compared with the case where the one-dimensional pattern measurement light is used, a sectioning image having a very high image quality can be obtained.

  In particular, when the measurement unit 100 is a fluorescence microscope, the fluorescent reagent of the measurement object S may be faded by irradiating the measurement object S with a number of times of pattern measurement light. Therefore, depending on the measurement object S, it may be important to reduce the number of times of imaging.

  In the present embodiment, the one-dimensional pattern measurement light and the two-dimensional pattern measurement light can be switched easily and at high speed by controlling the light modulation element 112. Therefore, the user can select the pattern measurement light used for sectioning observation in consideration of the time required for generating the sectioning image data (number of imaging) and the image quality of the obtained sectioning image.

  FIG. 5D shows an example of uniform measurement light. The uniform measurement light has a uniform intensity distribution. That is, the uniform measurement light is measurement light composed of only a bright part. In normal observation, the uniform measurement light in FIG. 5D is irradiated to all parts of the measurement object S, and the fluorescence emitted by the measurement object S is detected. Thereby, the image data of the measuring object S is generated. Image data obtained in normal observation is called normal image data, and an image based on the normal image data is called a normal image.

(3) Image processing The light modulation element 112 includes a plurality of pixels arranged in a two-dimensional shape. Hereinafter, in order to distinguish from the pixel of the camera 121, the pixel of the light modulation element 112 is referred to as a modulation pixel. When the light modulation element 112 is a DMD, the modulation pixel is a micromirror, and when the light modulation element 112 is an LCOS or LCD, the modulation pixel is a liquid crystal pixel.

  Between the plurality of modulation pixels of the light modulation element 112, there is a minute gap as a non-light guide portion. The measurement light is not reflected or transmitted by the non-light guide portion of the light modulation element 112. Therefore, measurement light is not emitted from the non-light guide portion of the light modulation element 112. Therefore, the portion of the measurement object S corresponding to the non-light guide portion of the light modulation element 112 is not irradiated with the bright portions of the pattern measurement light and the uniform measurement light.

  FIG. 6 is a view showing a normal image of the measuring object S. As shown in FIG. Note that the measurement object S in the example of FIG. 6 is a blank sheet. FIG. 6A is a normal image when the measurement object S is irradiated with uniform measurement light without using the light modulation element 112. FIG. 6B is a normal image when the measurement object S is irradiated with the uniform measurement light using the light modulation element 112. FIG.6 (c) is an enlarged view of the A section of FIG.6 (b).

  When the light modulation element 112 is not used, all portions of the measurement object S are irradiated with the bright portion of the uniform measurement light. Therefore, as shown in FIG. 6A, an entire normal image of the measuring object S is obtained. On the other hand, when the light modulation element 112 is used, the portion of the measurement object S corresponding to the non-light guide portion between the modulation elements is not irradiated with the bright portion of the uniform measurement light. Therefore, as shown in FIGS. 6B and 6C, a grid-like shadow pattern is reflected in the obtained normal image.

  When the shadow patterns in FIGS. 6B and 6C are reflected in the sectioning image or the normal image, the image quality of the sectioning image or the normal image is deteriorated. Therefore, in the present embodiment, the image processing unit 214 of the CPU 210 executes the following image processing to remove the shadow pattern from the sectioning image or the normal image. In the following description, the procedure for removing the shadow pattern from the normal image will be described, but the procedure for removing the shadow pattern from the sectioning image is the same.

  First, waveform conversion processing is performed on normal image data. In this example, the waveform conversion process is, for example, DFT (Discrete Fourier Transform). The waveform conversion process may be another waveform conversion process such as DWT (Discrete Wavelet Transform). Thereby, the image data subjected to the waveform conversion process is generated. Hereinafter, image data that has undergone waveform conversion processing is referred to as converted image data, and an image based on the converted image data is referred to as a converted image.

  FIG. 7 is a diagram illustrating an example of a normal image and a converted image. FIG. 7A shows a normal image obtained using the light modulation element 112. The horizontal axis in FIG. 7A is the position in the X direction, and the vertical direction is the position in the Y direction. A grid-like shadow pattern is reflected in the normal image in FIG.

  FIG. 7B shows a converted image. The horizontal axis in FIG. 7B is the spatial frequency in the X direction, and the vertical axis is the spatial frequency in the Y direction. The pixel data indicating the pixel in the center portion of the converted image in FIG. 7B corresponds to a low frequency component (DC component) in the waveform conversion process and has a large value. That is, the central pixel of the converted image is displayed brightly.

  Since the non-light-guiding portions between the modulation pixels exist at substantially equal intervals, the shadow pattern transferred to the normal image has periodicity. Accordingly, the pixel data corresponding to the shadow pattern has a high spatial frequency and a relatively large value. In the example of FIG. 7B, pixel data indicating pixels in four regions surrounded by a frame is pixel data corresponding to the shadow pattern. Pixels in these areas are displayed relatively brightly. In the converted image of FIG. 7B, the brightness is displayed logarithmically in order to make it easier to visually recognize the brightness of the pixels in the area surrounded by the frame.

  Next, a filtering process is performed on the converted image data. As the filter process, a known filter process such as a low-pass filter process or a band cutoff filter process can be used. The parameter of the filter process is determined so that the value of the pixel data corresponding to the shadow pattern among the pixel data of the converted image data becomes 0 or is reduced. Thereby, converted image data in which the value of the pixel data corresponding to the shadow pattern becomes 0 or reduced is generated.

  Thereafter, inverse conversion processing of waveform conversion processing is executed on the converted image data on which the filtering processing has been executed. When the waveform transformation process is DFT, the inverse transformation process is IDFT (Inverse Discrete Fourier Transform), and when the waveform transformation process is DWT, the inverse transformation process is IDWT (Inverse Discrete Wavelet Transformation). is there. Thereby, normal image data is generated.

  In the converted image data, the value of the pixel data corresponding to the shadow pattern is 0 or reduced. Therefore, in the normal image data obtained by the inverse conversion process, the value of the pixel data corresponding to the shadow pattern is 0. Become or be reduced. Thereby, the shadow pattern is removed from the normal image based on the normal image data obtained by the inverse transformation process.

  Instead of the image processing described above, blur filter processing may be performed on the normal image data. As the blur filter process, a known blur filter process such as a Gaussian filter process or a moving average filter process can be used. The parameter of the blur filter processing is determined so that the pixel data value corresponding to the shadow pattern in the pixel data of the normal image data is 0 or reduced. In this case, the shadow pattern is removed from the normal image based on the normal image data obtained by the blur filter process.

  The parameter of the filter process to be performed on the converted image data or the parameter of the blur filter process to be performed on the normal image data is determined based on periodicity such as the spatial frequency of the shadow pattern. The spatial frequency of the shadow pattern depends on the spacing between the plurality of modulation pixels of the light modulation element 112, the spacing between the plurality of pixels of the camera 121, the magnification of the fluorescent illumination lens 101, and the magnification of the imaging lens 123. Therefore, the spatial frequency of the shadow pattern can be calculated based on the values of these optical elements.

  In the present embodiment, the objective lens 161 is used in common for the measurement light traveling from the pattern imparting unit 110 toward the measurement object S and the fluorescence traveling from the measurement object S toward the light receiving unit 120. Therefore, the spatial frequency of the shadow pattern does not depend on the magnification of the objective lens 161.

  Here, the spatial frequency of the shadow pattern calculated by the above procedure may be slightly different from the spatial frequency of the actually measured shadow pattern. This is caused by an error in the spacing between the modulation pixels, an error in the spacing between the pixels, an error in the magnification of the lens, an error in the assembly of the measurement unit 100, an error in the inclination of the light receiving unit 120 with respect to the light modulation element 112, and the like. Therefore, in the present embodiment, the spatial frequency of the shadow pattern is calculated by the following procedure.

  FIG. 8 is a diagram illustrating an example of pattern measurement light used for calculation of the spatial frequency of the shadow pattern. FIG. 8A shows pattern measurement light emitted from the pattern applying unit 110, and FIG. 8B shows pattern measurement light received by the light receiving unit 120. In FIG. 8, the bright part of the pattern measurement light is represented by white and the dark part is represented by a hatching pattern.

  In the example of FIG. 8A, the pattern measurement light emitted from the pattern imparting unit 110 has two bright portions separated by a distance D1 in the X direction. The unit of the distance D1 is the number of modulation pixels (modulator pixel). The pattern measurement light emitted from the pattern applying unit 110 is received by the light receiving unit 120 through a plurality of mirrors and lenses.

  As shown in FIG. 8B, in the pattern measurement light received by the light receiving unit 120, the two bright portions are separated by a distance D2, and the straight line connecting the two bright portions has an angle θ with respect to the X direction. Eggplant. The unit of the distance D2 is the number of pixels (image pixel).

  The spatial frequencies in the X and Y directions are given by D2 × cos θ / (n × D1) and D2 × sin θ / (n × D1), respectively. Here, n is the binning number. The binning number means the number of pixel data combined in a binning process in which a plurality of pixel data are combined in a pseudo manner and handled as one pixel data. For example, when 3 × 3 binning processing is performed on pixel data, the spatial frequencies in the X direction and Y direction are D2 × cos θ / (3 × D1) and D2 × sin θ / (3 × D1), respectively.

  When the Gaussian filter process is executed as the blur filter process, the standard deviation of the Gaussian curve is determined based on the value of D2 / (n × D1). In this example, the standard deviation of the Gaussian curve is given by k × D2 / (n × D1). Here, k is an appropriate proportional coefficient, for example, 0.5.

  In the present embodiment, a portion of the pattern image data, sectioning image data, or normal image data that corresponds to the non-light guiding portion of the light modulation element 112 is removed by automatically performing the filtering process. Here, depending on the conditions of the optical system, the shadow pattern may not be reflected in the pattern image, the sectioning image, or the normal image. For example, when the magnification and numerical aperture of the objective lens 161 are large, no shadow pattern is reflected in the pattern image, sectioning image, or normal image.

  In such a case, when the filter process is executed, the image quality of the pattern image, the sectioning image, or the normal image is degraded. For this reason, the filtering process may not be automatically performed, but may be performed based on a user instruction. Alternatively, the user may select whether to execute the filtering process. In this case, the user can use the operation unit 250 in FIG. 1 to instruct or select whether to execute the filter process.

(4) Adjustment of Projection Resolution Size When the measurement object S is irradiated with the pattern measuring light by the light modulation element 112, the light or dark portion of the pattern measuring light that can be recognized together with the measurement object S by the light receiving unit 120 Is referred to as the projected resolution size d of the light modulation element 112. The shadow pattern can be removed from the normal image by adjusting the projection resolution size d.

  When the light modulation element 112 is used for a long time, a defective pixel may occur in the modulation pixel of the light modulation element 112 due to deterioration. In this case, the portion corresponding to the defective pixel is lost from the bright portion of the measurement light. In addition, a bright portion of the measurement light may be generated using a plurality of modulation pixels as a set of modulation pixels. Here, when a defective pixel occurs in a plurality of modulation pixels of the modulation pixel group, a portion corresponding to the defective pixel is largely lost from the bright portion of the measurement light. Even when a part of the bright part of the measurement light is lost due to the defective pixel of the modulation pixel or the modulation pixel group, the missing part can be removed from the normal image by adjusting the projection resolution size d.

  FIG. 9 is a diagram for explaining a method of adjusting the projection resolution size d. As shown in FIG. 9, in the measurement unit 100, a plurality of positions D, C, B, A, B ′, C ′, and D ′ are arranged in this order as positions for disposing the light modulation element 112. Provided. Here, the dimension (width) of the gap between the plurality of modulation pixels of the light modulation element 112 is a1, the modulation pixel dimension is a2, and the modulation pixel group dimension is a3. The dimension a3 is larger than the dimension a2, and the dimension a2 is larger than the dimension a1.

  The projection resolution size d of the light modulation element 112 is adjusted by arranging the light modulation element 112 at any of a plurality of positions D, C, B, A, B ′, C ′, and D ′. The position A is a position where the resolution of the optical system including the fluorescent illumination lens 101 and the imaging lens 123 is maximized. That is, the position A is a position where the projection resolution size d of the light modulation element 112 is minimum, and is a position having a conjugate relationship with the camera 121 with the fluorescent illumination lens 101 and the imaging lens 123 interposed therebetween.

  The positions B and B ′ are positions where the resolution of the optical system is lower than the dimension a1 and is equal to or larger than the dimension a2. The position C and the position C ′ are positions where the resolution of the optical system is lower than the dimension a2 and equal to or larger than the dimension a3. The positions D and D ′ are positions where the resolution of the optical system is lower than the dimension a3.

  Therefore, by disposing the light modulation element 112 at the position B or the position B ′, the projection resolution size d of the light modulation element 112 can be adjusted to be larger than the dimension a1 and smaller than or equal to the dimension a2. Thereby, the shadow pattern can be removed from the normal image.

  By disposing the light modulation element 112 at the position C or the position C ′, the projection resolution size d of the light modulation element 112 can be adjusted to be larger than the dimension a2 and smaller than or equal to the dimension a3. As a result, the shadow pattern can be removed from the normal image, and the missing portion of the modulation pixel due to the defective pixel can be removed.

  By disposing the light modulation element 112 at the position D or the position D ′, the projection resolution size d of the light modulation element 112 can be adjusted to be larger than the dimension a3. Accordingly, it is possible to remove the shadow pattern from the normal image, remove the missing portion due to the defective pixel of the modulation pixel, and remove the missing portion due to the defective pixel of the modulation pixel group.

  In the present embodiment, the light modulation element 112 is arranged at a position slightly different from the position (position A) in a conjugate relationship with the camera 121, but the present invention is not limited to this. When the transmitted light supply unit 130 is not provided in the measurement unit 100, the position where the position of the camera 121 is conjugate with the light modulation element 112 in a state where the light modulation element 112 is disposed at the position A is slightly different. You may arrange | position in a different position.

  The projection resolution size d of the light modulation element 112 may be adjusted by another method instead of the above method or in combination with the above method. For example, the fluorescent illumination lens 101 or the imaging lens 123 may be configured such that the value of MTF (modulation transfer function) is smaller (preferably 0) than a preset threshold value at a desired spatial frequency. . Thereby, the projection resolution size d of the light modulation element 112 is adjusted. In this case, the light modulation element 112 may be disposed at a position A in FIG.

  When the desired spatial frequency is 1 / (2 × a1), the shadow pattern can be removed from the normal image. When the desired spatial frequency is 1 / (2 × a2), it is possible to further remove a missing portion due to a defective pixel of the modulation pixel from the normal image. When the desired spatial frequency is 1 / (2 × a3), a missing portion due to a defective pixel in the modulation pixel group can be further removed from the normal image.

(5) Configuration of optical system When measurement light is irradiated onto measurement object S, fluorescence is emitted from measurement object S. The intensity of the fluorescence emitted from the measuring object S is proportional to the intensity of the measuring light applied to the measuring object S. Therefore, fluorescence is emitted from the portion of the measurement object S irradiated with the measurement light and irradiated to the camera 121 of the light receiving unit 120. The intensity of fluorescence is about 10 ⁻⁶ times the intensity of measurement light.

  It is preferable that the fluorescence irradiation range of the camera 121 coincides with the light receiving surface of the camera 121. Specifically, it is preferable that the dimension (projection size) and the center position of the fluorescence irradiation range to the camera 121 respectively coincide with the dimension (observation field size) and the center position of the light receiving surface of the camera 121.

  However, the measurement unit 100 has various optical system misalignments. The deviation in the arrangement of the optical system includes the inclination of the filter turret 152 with respect to the optical axis of fluorescence. Further, the displacement of the arrangement of the optical system further includes a displacement of attachment of the excitation filter 151b, the dichroic mirror 151c, and the absorption filter 151d with respect to the frame 151a in the filter cube 151 of FIG.

  Due to the misalignment of these optical systems, it is difficult to make the fluorescence irradiation range coincide with the light receiving surface. When fluorescence is not irradiated to a part of the light receiving surface, a part of the image based on the generated image data is missing. Therefore, in the present embodiment, the optical system is configured so that the fluorescence irradiation range is slightly larger than the dimension of the light receiving surface, and most of the fluorescence is irradiated on all the light receiving surfaces.

  FIG. 10 is a diagram showing the relationship between the fluorescence irradiation range and the light receiving surface on the plane including the light receiving surface 120S. FIG. 11 is an enlarged view of a part of the fluorescence irradiation range R and the light receiving surface 120S of FIG. FIG. 11A shows part B of FIG. 10, and FIG. 11B shows part C of FIG. The dimensions used in the following description are all equivalent dimensions in the light receiving surface 120S.

  In the example of FIG. 10, the fluorescence irradiation range R has a rectangular shape composed of two sides parallel to the X direction and two sides parallel to the Y direction. Similarly, the light receiving surface 120S has a rectangular shape composed of two sides parallel to the X direction and two sides parallel to the Y direction. As shown in FIG. 10, the fluorescence irradiation range R is larger than the light receiving surface 120S, and the light receiving surface 120S is located at the approximate center of the fluorescence irradiation range. In the following description, the dimension and number in the X direction of the optical system will be described, but the same applies to the dimension and number in the Y direction.

  The number of modulation pixels of the light modulation element 112 is N1, the interval between the plurality of modulation pixels of the light modulation element 112 is L1, and the focal length of the fluorescent illumination lens 101 is F1. Further, the number of pixels on the light receiving surface is N2, the interval between the plurality of pixels on the light receiving surface is L2, and the focal length of the imaging lens 123 is F2. The interval L1 between the modulation pixels is the sum of the gap interval between the plurality of modulation elements a1 and the dimension of the modulation element a2. When the focal length of the objective lens 161 is F0, the light projection magnification M1 by the fluorescent illumination lens 101 is M1 = F0 / F1, and the light projection magnification M2 by the imaging lens 123 is M2 = F2 / F0.

  As shown in FIG. 11A, the fluorescence irradiation range R includes a plurality of enlarged modulation pixels p1, and the size of each modulation pixel p1 is L1 × M1 × M2. That is, the dimension of each modulation pixel p1 is L1 × F2 / F1. On the other hand, as shown in FIG. 11B, the size of each pixel p2 on the light receiving surface 120S is L2.

  Since the plurality of modulation pixels p1 are arranged in the fluorescent irradiation range R, the size of the fluorescent irradiation range R is N1 × L1 × F2 / F1. Since N2 pixels p2 are arranged on the light receiving surface 120S, the size of the light receiving surface 120S is N2 × L2. A margin from one end of the light receiving surface 120S in the X direction to one end of the fluorescence irradiation range R and a margin from the other end of the light receiving surface 120S in the X direction to the other end of the fluorescence irradiation range R are denoted by Δ, respectively. In this case, the margin Δ is given by Δ = (N1 × L1 × F2 / F1−N2 × L2) / 2.

  Due to the displacement of the optical system, the measurement light to the measurement object S enters the objective lens 161 at an angle inclined with respect to the Z axis, or the fluorescence from the measurement object S is inclined with respect to the Z axis. When the light is incident on the light receiving unit 120 at an angle, the fluorescence irradiation range R is shifted in the X direction or the Y direction with respect to the light receiving surface 120S. Even in this case, the optical system is configured such that most of the light receiving surface 120S is irradiated with the majority of the fluorescence.

  FIG. 12 is a schematic enlarged view of the light receiving unit 120. In the example of FIG. 12, the fluorescence from the measuring object S is inclined by an angle φ with respect to the Z axis. In this case, the deviation δ of the fluorescence irradiation position in the X direction on the light receiving surface 120S is given by δ = F2 × sinφ.

  In this example, a shift of 4 minutes may occur as an individual difference in the attachment of the dichroic mirror 151c to the frame 151a in the filter cube 151. Further, as the reproducibility of attachment of the filter cube 151 to the filter turret 152, a deviation of 3 minutes may occur. Further, as the reproducibility of switching the filter cube 151 due to the rotation of the filter turret 152, a deviation of 3 minutes may occur.

  As shown in FIG. 3, the optical axis from the light modulation element 112 to the measuring object S is bent by 90 ° in the filter cube 151 by the dichroic mirror 151c. In this case, the inclination of the measurement light to the measurement object S with respect to the Z axis is twice the sum of the angular deviations of the installed dichroic mirror 151c. Therefore, the angle φ is twice the sum of the deviations. That is, the angle φ is about φ = 2 × 10 minutes = 0.008 radians. Thus, since the angle φ is very small, the irradiation position shift δ can be approximated to δ = F2 × φ.

If the irradiation position deviation δ is equal to or less than the margin Δ, even when the fluorescence from the measurement object S enters the light receiving unit 120 at an angle inclined with respect to the Z axis, most of the fluorescence is present on the entire light receiving surface 120S. Irradiated. Therefore, the relationship between the irradiation position deviation δ and the margin Δ is given by the following equation (1).

On the other hand, when the number of pixels p2 that overlap one enlarged modulation pixel p1 becomes too large, the minimum width of the pattern of measurement light by the light modulation element 112 is increased, so that details of the measurement object can be obtained with high accuracy. The imaging performance (sectioning performance) decreases. Therefore, an upper limit value α of the number of pixels p2 that overlaps one enlarged modulation pixel p1 is set. Therefore, the relationship between the number of pixels p2 overlapping one enlarged modulation pixel p1 and the upper limit value α is given by the following equation (2). In this example, the value α is, for example, 1 or more and 3 or less.

By arranging the equations (1) and (2), the relationship of the following equation (3) is given. The focal length F1 of the fluorescent illumination lens 101 is set within the range given by equation (3). As a result, even when the measuring light to the measuring object S is tilted with respect to the Z-axis or when the fluorescence from the measuring object S is tilted with respect to the Z-axis, it is received within a range that does not degrade the sectioning performance. All of the surface 120S can be irradiated with the majority of the fluorescence. As a result, it is possible to generate image data in which deterioration in image quality and loss of image portions are prevented.

(6) Effect In the microscope imaging apparatus 500 according to the present embodiment, pattern measurement light or uniform measurement light is generated by the light modulation element 112 from the light emitted from the light projecting unit 320. In sectioning observation, the measurement object S is irradiated with the pattern measurement light generated by the light modulation element 112. Further, the spatial phase of the generated pattern is sequentially moved on the measurement object S by a predetermined amount by the light modulation element 112. Sectioning image data is generated based on a plurality of pattern image data generated at a plurality of phases of the pattern.

  On the other hand, in normal observation, uniform measurement light is generated by the light modulation element 112 and irradiated onto the measurement object S. Thereby, normal image data indicating an image of the measurement object S when the measurement light having no pattern is irradiated is generated.

  As described above, by using the light modulation element 112, it is possible to irradiate the measurement object S with the pattern measurement light and the uniform measurement light through the common fluorescent illumination lens 101, the filter cube 151, and the objective lens 161. Further, the fluorescence from the measuring object S can be received through the common objective lens 161 and the imaging lens 123. Thereby, sectioning observation and normal observation of the measuring object S can be performed.

  Here, the light modulation element 112 includes a periodic non-light-guiding unit in a plurality of modulation pixels. Measurement light is not generated from the non-light guide portion between the plurality of modulation pixels of the light modulation element 112. Therefore, a shadow pattern corresponding to a non-light guiding portion between a plurality of modulation pixels of the light modulation element 112 is reflected in the sectioning image or the normal image. Even in such a case, a portion corresponding to the non-light guide portion of the light modulation element 112 in the sectioning image data or the normal image data is removed by the filtering process. As a result, sectioning observation and normal observation can be performed without degrading the image quality of the image.

(7) Correspondence between each component of claim and each part of embodiment The following describes an example of the correspondence between each component of the claim and each part of the embodiment. It is not limited.

  In the above embodiment, the light projecting unit 320 is an example of the light projecting unit, the modulation pixel p1 is an example of the first pixel, the pixel p2 is an example of the second pixel, and the light modulation element 112 is It is an example of a light modulation element, and the measuring object S is an example of the measuring object. The fluorescent illumination lens 101, the imaging lens 123, the filter cube 151, and the objective lens 161 are examples of a light projecting optical system, the light receiving unit 120 is an example of a light receiving unit, and the control unit 213 is an example of a control unit and a processing unit. The operation unit 250 is an example of first and second instruction units. The image data generation unit 211 is an example of an image data generation unit, the image processing unit 214 is an example of an image processing unit, the light receiving surface 120S is an example of a light receiving surface, the irradiation range R is an example of an irradiation range, The microscope imaging device 500 is an example of a microscope imaging device.

  As each constituent element in the claims, various other elements having configurations or functions described in the claims can be used.

  The present invention can be effectively used for various microscope imaging apparatuses, microscope imaging methods, and microscope imaging programs.

DESCRIPTION OF SYMBOLS 100 Measurement part 101 Fluorescent illumination lens 110 Pattern provision part 111 Light output part 112 Light modulation element 113 Mirror 120 Light receiving part 120S Light receiving surface 121 Camera 122 Color filter 123 Imaging lens 130 Transmitted light supply part 131 Transmitted light source 132 Transmitted optical system 140 Stage 150 Filter unit 151 Filter cube 151a Frame 151b Excitation filter 151c Dichroic mirror 151d Absorption filter 152 Filter turret 152a Filter cube mounting part 160 Lens unit 161 Objective lens 162 Lens turret 162a Objective lens mounting part 163 Focus position adjustment mechanism 170 Control board 200 PC
210 CPU
211 Image Data Generation Unit 212 Pattern Generation Unit 213 Control Unit 214 Image Processing Unit 220 ROM
230 RAM
DESCRIPTION OF SYMBOLS 240 Memory | storage device 250 Operation part 300 Measurement light supply part 310 Power supply device 320 Light projection part 321 Measurement light source 322 Dimming mechanism 323 Shading mechanism 330 Light guide member 400 Display part 500 Microscope imaging device p1 Modulation pixel p2 Pixel R Irradiation range S Measurement object object

Claims (15)

A light projecting unit that emits light;
A light modulation element including a plurality of first pixels configured to generate measurement light having an arbitrary pattern from the light emitted by the light projecting unit;
A projection optical system for irradiating a measurement object with measurement light generated by the light modulation element;
A light receiving unit that receives light from the measurement object and outputs a light reception signal indicating the amount of light received;
An image data generation unit that generates image data based on a light reception signal output from the light reception unit;
An image processing unit that performs image processing on the image data generated by the image data generation unit;
The light modulation element is controlled so as to generate measurement light having a pattern and to sequentially move the spatial phase of the generated pattern on the measurement object by a predetermined amount, and is generated with a plurality of phases of the pattern. A control unit that controls the image data generation unit so as to generate sectioning image data indicating an image of the measurement object based on a plurality of image data.
The light modulation element has a periodic non-light guide portion between the plurality of first pixels,
The image processing unit executes a filtering process for removing a portion corresponding to the non-light-guiding unit in image data or sectioning image data based on the periodicity of the non-light-guiding unit of the light modulation element. . A first instruction unit for instructing whether to perform the filtering process;
The control unit executes the filter process when the execution of the filter process is instructed by the first instruction unit, and executes the filter process when the execution of the filter process is not instructed by the first instruction unit. The microscope imaging apparatus according to claim 1, wherein the image processing unit is controlled so as not to perform filter processing. A second instruction unit for instructing the first operation mode or the second operation mode;
The light modulation element is configured to selectively generate measurement light having an arbitrary pattern and light having no pattern from light emitted from the light projecting unit,
The controller is
When the first operation mode is instructed by the second instruction unit, the measurement light having a pattern is generated, and the spatial phase of the generated pattern is sequentially set on the measurement object by a predetermined amount. Controlling the light modulation element to move, controlling the image data generation unit to generate sectioning image data based on a plurality of image data generated at a plurality of phases of the pattern,
When the second operation mode is instructed by the second instruction unit, the light modulation element is controlled to generate measurement light having no pattern, and measurement light having no pattern is irradiated. The microscope imaging apparatus according to claim 1, wherein the image data generation unit is controlled so as to generate normal image data indicating an image of the measurement object at the time. The microscope imaging apparatus according to claim 1, wherein the filtering process includes a spatial filtering process. The microscope imaging apparatus according to claim 4, wherein the image processing unit sequentially performs a waveform conversion process, the spatial filter process, and an inverse conversion process of the waveform conversion process as the filter process. The microscope imaging apparatus according to claim 5, wherein the waveform conversion processing includes discrete Fourier transform or discrete wavelet transform. The microscope imaging apparatus according to claim 4, wherein the spatial filter processing includes blur filter processing. The microscope imaging apparatus according to claim 7, wherein the blur filter process includes a Gaussian filter process. The light modulation element, the light receiving unit, and the light projecting optical system are configured such that a resolution of the light projecting optical system is lower than a width of a gap between the plurality of first pixels of the light modulation element. The microscope imaging apparatus according to any one of claims 1 to 8. One of the light modulation element and the light receiving unit has a resolution of the light projecting optical system from the position that is conjugate to the other of the light modulation element and the light receiving unit via the light projecting optical system. The microscope imaging apparatus according to claim 9, wherein the microscope imaging apparatus is arranged at a position shifted until it becomes lower than a width of a gap between the plurality of first pixels of the modulation element. The value of the modulation transfer function of the light projecting optical system is equal to or less than a predetermined threshold at a spatial frequency corresponding to a width of a gap between the plurality of first pixels of the light modulation element. The microscope imaging apparatus according to 9 or 10. The light projecting optical system is configured such that an irradiation range of light from the measurement object to the light receiving unit includes the light receiving surface of the light receiving unit on a plane including the light receiving surface of the light receiving unit. Item 12. The microscope imaging apparatus according to any one of Items 1 to 11. The light receiving unit includes a plurality of second pixels arranged in first and second directions intersecting each other,
On the plane including the light receiving surface of the light receiving unit, light portions from the measurement object corresponding to each first pixel of the light modulation element to the light receiving unit are arranged in the first direction. The microscope according to claim 12, wherein the light projecting optical system is configured to irradiate a region equal to or smaller than a region including three second pixels and three second pixels arranged in the second direction. Imaging device. Emitting light by the light projecting unit;
Generating measurement light having an arbitrary pattern from a light emitted from the light projecting unit by a light modulation element including a plurality of first pixels;
Irradiating the measuring object with the light projecting optical system generated by the light modulation element; and
Sequentially moving the spatial phase of the generated pattern on the measurement object by a predetermined amount by the light modulation element;
Receiving light from the measurement object by a light receiving unit and outputting a light reception signal indicating the amount of light received;
Generating sectioning image data indicating an image of the measurement object based on a plurality of image data generated at a plurality of phases of the pattern based on a light reception signal output from the light receiving unit;
Removing a portion corresponding to the non-light-guiding unit in image data or sectioning image data based on the periodicity of the non-light-guiding unit between the plurality of first pixels of the light modulation element. Imaging method. A microscope imaging program executable by a processing device,
A process of emitting light by the light projecting unit;
A process of generating measurement light having an arbitrary pattern from the light emitted from the light projecting unit by a light modulation element including a plurality of first pixels;
A process of irradiating the measurement object with the measurement light generated by the light modulation element by the light projecting optical system;
A process of sequentially moving a spatial phase of the generated pattern on the measurement object by a predetermined amount by the light modulation element;
A process of receiving light from the measurement object by a light receiving unit and outputting a light reception signal indicating the amount of received light;
Processing for generating sectioning image data indicating an image of the measurement object based on a plurality of image data generated at a plurality of phases of the pattern based on a light reception signal output from the light receiving unit;
A process of removing a portion corresponding to the non-light guide portion in image data or sectioning image data based on the periodicity of the non-light guide portion between the plurality of first pixels of the light modulation element;
A microscope imaging program to be executed by the processing apparatus.

Images