JP2015084057A - Device, method, and program for capturing microscopic images - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a device, method, and program for capturing microscopic images, which enable users to recognize it when light reception signal strength is saturated.SOLUTION: Measurement light having a predetermined pattern is produced by a pattern imparting unit 110 and is irradiated on an object S under measurement. A spatial phase of the produced pattern is sequentially shifted on the object S under measurement by predetermined increments by the pattern imparting unit 110. A light receiving unit 120 receives light from the object S under measurement and outputs a light reception signal that represents received light intensity. Based on a plurality of sets of patterned image data generated at a plurality of phases of the pattern using the light reception signal, sectioning image data representing an image of the object S under measurement is generated. Assessment is made as to whether the light reception signal is saturated or not based on at least one of the plurality of sets of patterned image data. A user is notified if the light reception signal is saturated.

Description

  The present invention relates to a microscope imaging apparatus, a microscope imaging method, and a microscope imaging program.

  In the microscope imaging apparatus, light is reflected or emitted by the measurement object by irradiating the measurement object with light. The light reflected or emitted by the measurement object is detected by the detector, thereby generating image data of the measurement object. Here, when stray light is present around the measurement object, the image of the measurement object is blurred based on the generated image data. Patent Document 1 describes a microscope apparatus configured to reduce the influence of stray light.

  In the microscope apparatus of Patent Document 1, signal light including stray light is reflected by the specimen when the specimen is irradiated with light. The signal light reflected by the sample is detected by a detector, thereby forming an image of the sample. Further, the intensity of the received light signal including only stray light is separately detected. By subtracting the intensity corresponding to the stray light from the intensity of the received light signal corresponding to the signal light reflected by the sample, an image of the sample with reduced influence of the stray light is obtained.

JP 2012-212018 A

  When the intensity of the light reception signal corresponding to the stray light is subtracted from the intensity of the light reception signal corresponding to the signal light in order to reduce the influence of the stray light as in Patent Document 1, the intensity of the light irradiated to the measurement object is sufficient. The image of the measurement object may become dark despite being large. Therefore, the user cannot recognize that the received light signal is saturated even when viewing the image of the measurement object.

  An object of the present invention is to provide a microscope imaging apparatus, a microscope imaging method, and a microscope imaging program that allow a user to easily recognize that the intensity of a received light signal is saturated.

  (1) A microscope imaging apparatus according to a first invention includes a light projecting unit that emits light, and a measurement light generator configured to generate measurement light having a predetermined pattern from the light emitted from the light projecting unit. , An optical system that irradiates the measurement object with the measurement light generated by the measurement light generation part, a light reception part that receives light from the measurement object, and outputs a light reception signal indicating the amount of light received, and is output from the light reception part An image data generation unit that generates image data based on the received light signal, and a measurement light generation unit that sequentially moves the spatial phase of the generated pattern by a predetermined amount on the measurement target, A control unit that controls the measurement light generation unit to generate sectioning image data indicating an image of the measurement object based on a plurality of image data generated in a plurality of phases, and a sectionin generated by the image data generation unit A display unit configured to display an image based on the image data, and the control unit saturates a light reception signal output by the light reception unit for at least one of the plurality of image data generated by the image data generation unit. If the light reception signal is saturated, the user is notified that the light reception signal is saturated.

  In this microscope imaging apparatus, measurement light having a predetermined pattern is generated by the measurement light generation unit from light emitted by the light projecting unit. The measurement light generated by the measurement light generation unit is irradiated onto the measurement object by the optical system. Further, the spatial phase of the generated pattern is sequentially moved on the measurement object by a predetermined amount by the measurement light generation unit. Light from the measurement object is received by the light receiving unit, and a light reception signal indicating the amount of light received is output.

  Sectioning image data indicating an image of the measurement object is generated based on a plurality of image data generated at a plurality of phases of the pattern based on the light reception signal output from the light receiving unit. An image of the measurement object based on the sectioning image data is displayed on the display unit.

  Here, it is determined whether or not the light reception signal output by the light receiving unit is saturated for at least one of the plurality of generated image data. When the light reception signal is saturated, the user is notified that the light reception signal is saturated. Thereby, the user can easily recognize that the light reception signal is saturated.

  (2) The image data generation unit, for each pixel of the plurality of image data, has a pixel value when a measurement target is irradiated with a first portion having a higher intensity than a certain intensity in the measurement light pattern. From the pattern of measurement light, sectioning image data may be generated by removing the value of the pixel when the measurement object is irradiated with the second portion having a certain intensity or less.

  In this case, the value of each pixel of the sectioning image data is the pixel value when the first portion of the measurement light pattern is irradiated on the measurement object and the second portion of the measurement light pattern is the measurement object. It becomes a difference from the value of the pixel when irradiated. Here, for each pixel in the plurality of image data, the value of the pixel when the measurement object is irradiated with the second portion of the measurement light corresponds to a blur component of the image due to stray light. Therefore, the blur component due to stray light is removed from each pixel of the sectioning image data. Thereby, a clear image of the measurement object can be obtained.

  On the other hand, when the blur component due to stray light is removed from each pixel of the sectioning image data, the image based on the sectioning image data becomes darker than when the blur component due to stray light is not removed from each pixel. Therefore, the user cannot recognize whether or not the received light signal is saturated even when viewing the image.

  Even in such a case, it is determined whether or not the light reception signal is saturated for at least one of the plurality of generated image data. If the light reception signal is saturated, the light reception signal is saturated to the user. Is notified. Thereby, even when the bright part does not exist in the image based on the sectioning image data, the user can easily recognize that the light reception signal is saturated.

  (3) The control unit controls the image data generation unit to generate brightness image data indicating a brightness distribution state of the image as a brightness image based on the plurality of image data generated by the image data generation unit, The display unit may be configured to display a brightness image based on the brightness image data.

  In this case, a brightness image indicating the brightness distribution state of the image is displayed on the display unit. Therefore, the user can recognize the brightness of the image by looking at the brightness image. Thereby, the user can easily recognize that the light reception signal is saturated.

  (4) The control unit may determine whether the light reception signal output from the light receiving unit is saturated based on the brightness image data generated by the image data generation unit.

  The brightness image data indicates the brightness distribution state of the images of the plurality of image data generated by the generation unit. Therefore, it is possible to easily determine whether or not the light reception signal is saturated by using the brightness image data.

  (5) The control unit calculates information related to the brightness of the image based on the brightness image data generated by the image data generation unit, and the display unit is configured to be able to display information related to the brightness calculated by the control unit. May be.

  In this case, information regarding the brightness of the image is displayed on the display unit. The user can recognize the brightness of the image in more detail by viewing the information related to the brightness of the image. As a result, the user can more easily recognize that the received light signal is saturated.

  (6) The information on brightness may include a histogram indicating the relationship between the values of a plurality of pixels of a plurality of image data and the number of pixels having each value.

  In this case, a histogram indicating the relationship between the values of the plurality of pixels of the plurality of image data and the number of pixels having each value is displayed on the display unit. The user can recognize the relationship between the values of the plurality of pixels of the plurality of image data and the number of pixels having each value by looking at the histogram. Thereby, the user can recognize that the received light signal is saturated and the degree of saturation.

  (7) The control unit may correct the brightness of the image by amplifying the values of the plurality of pixels of the sectioning image data, and the information related to the brightness may include an amplification factor of the values of the plurality of pixels.

  In this case, the image based on the sectioning image data is corrected so as to be bright. Thereby, the user can observe a bright image. Further, the amplification factors of the values of the plurality of pixels of the sectioning image data are displayed on the display unit. The user can recognize the brightness of the image before correction by looking at the amplification factor. Thereby, it is possible to recognize that the received light signal is saturated and the degree of saturation.

  (8) When the light receiving signal output from the light receiving unit is saturated, the control unit, the pixel value corresponding to the portion where the light receiving signal is saturated in the sectioning image data generated by the image data generating unit May be amplified larger than the value of the pixel corresponding to the non-saturated portion of the received light signal.

  In this case, in the image based on the sectioning image data, the pixel corresponding to the portion where the light reception signal is saturated is displayed brighter than the pixel corresponding to the portion where the light reception signal is not saturated. Thereby, the user can easily recognize that the received light signal is saturated by viewing the image.

  (9) When the light receiving signal output from the light receiving unit is saturated, the control unit displays the pixel corresponding to the saturated light receiving signal portion in the image displayed on the display unit in an identifiable manner. The display unit may be controlled to do so. In this case, the user can easily recognize that the received light signal is saturated by looking at the image.

  (10) When the generation condition of the plurality of image data by the image data generation unit is changed, the control unit controls the measurement light generation unit so that the measurement object is irradiated with the measurement light, and the generation unit generates The control unit is controlled to update the sectioning image data based on the plurality of image data, and when the generation conditions of the plurality of image data by the generation unit are not changed, the irradiation of the measurement light to the measurement target is stopped In this way, the measurement light generator may be controlled.

  According to this configuration, when the generation conditions of the plurality of image data by the image data generation unit are changed, the measurement object is irradiated with the measurement light. Further, the sectioning image data is updated based on the plurality of image data generated by the image data generation unit. Accordingly, the user can change the generation conditions of the plurality of image data by the image data generation unit so that the light reception signal is not saturated.

  On the other hand, when the generation conditions of the plurality of image data by the image data generation unit are not changed, the irradiation of the measurement light to the measurement object is stopped. Therefore, it is possible to minimize the deterioration of the measurement object due to the measurement object being continuously irradiated with the measurement light.

  (11) When the generation condition of the plurality of image data by the image data generation unit is changed, the control unit is irradiated with measurement light before the generation condition of the plurality of image data by the image data generation unit is changed. The measurement light generation unit may be controlled so that the measurement light is irradiated to a part different from the part of the object.

  According to this configuration, when the generation conditions of the plurality of image data by the image data generation unit are changed, the measurement target irradiated with the measurement light before the generation conditions of the plurality of image data by the image data generation unit are changed Measurement light is irradiated to a part different from the part of the object. In this case, the measurement light is not continuously irradiated only on a specific part of the measurement object. Thereby, it can prevent that only the specific part of a measuring object deteriorates.

  (12) The control unit may change the level of the light receiving signal output from the light receiving unit between the light projecting unit and the light receiving unit so that the level is changed between the first level and a second level lower than the first level. A plurality of image data in a state where the level of the light reception signal is the first level is generated as a plurality of first image data by controlling at least one, and a plurality in the state where the level of the light reception signal is the second level Are generated as a plurality of second image data, and the plurality of generated first image data and the plurality of second image data are combined so that the dynamic range of the light receiving unit is expanded. The third image data may be generated, and the image data generation unit may be controlled to generate sectioning image data based on the plurality of third image data.

  In this case, the dynamic range of the light receiving unit is expanded. Therefore, even when the light reception signal is relatively large or the light reception signal is relatively small, sectioning image data that does not include blackout and overexposure can be generated.

  (13) In the microscope imaging method according to the second invention, a step of emitting light by the light projecting unit, and a step of generating the measurement light having a predetermined pattern by the measurement light generating unit from the light emitted by the light projecting unit Irradiating the measurement object generated by the measurement light generation unit to the measurement object by the optical system, and sequentially moving the spatial phase of the generated pattern on the measurement object by a predetermined amount by the measurement light generation unit. A step of receiving light from the measurement object by the light receiving unit, outputting a light receiving signal indicating the amount of light received, and a plurality of patterns generated at a plurality of phases of the pattern based on the light receiving signal output from the light receiving unit Generating sectioning image data indicating an image of the measurement object based on the image data of the image, and causing the display unit to display an image based on the generated sectioning image data. Determining whether the received light signal output by the light receiving unit is saturated for at least one of the generated plurality of image data; and receiving light to the user if the received light signal is saturated And notifying that the signal is saturated.

  According to this microscope imaging method, the measurement light having a predetermined pattern is generated by the measurement light generation unit from the light emitted by the light projecting unit. The measurement light generated by the measurement light generation unit is irradiated onto the measurement object by the optical system. Further, the spatial phase of the generated pattern is sequentially moved on the measurement object by a predetermined amount by the measurement light generation unit. Light from the measurement object is received by the light receiving unit, and a light reception signal indicating the amount of light received is output.

  Sectioning image data indicating an image of the measurement object is generated based on a plurality of image data generated at a plurality of phases of the pattern based on the light reception signal output from the light receiving unit. An image of the measurement object based on the sectioning image data is displayed on the display unit.

  Here, it is determined whether or not the light reception signal output by the light receiving unit is saturated for at least one of the plurality of generated image data. When the light reception signal is saturated, the user is notified that the light reception signal is saturated. Thereby, the user can easily recognize that the light reception signal is saturated.

  (14) A microscope imaging program according to a third invention is a microscope imaging program that can be executed by a processing device, and a process of emitting light by a light projecting unit and measurement light having a predetermined pattern by the light projecting unit. A process for generating the measurement light from the emitted light, a process for irradiating the measurement target with the measurement light generated by the measurement light generation part using an optical system, and a spatial phase of the generated pattern as a measurement light A process for sequentially moving a predetermined amount on the measurement target by the generation unit, a process for receiving light from the measurement target by the light receiving unit, and outputting a light reception signal indicating the amount of received light, and a light reception signal output from the light reception unit Generating sectioning image data indicating an image of a measurement object based on a plurality of image data generated at a plurality of phases of a pattern based on the pattern, and a generated sectioning image A process of displaying an image based on the data on the display unit, a process of determining whether or not the light reception signal output by the light reception unit is saturated for at least one of the plurality of generated image data, and the light reception signal When saturated, the processing device is caused to execute processing for notifying the user that the received light signal is saturated.

  According to this microscope imaging program, the measurement light having a predetermined pattern is generated by the measurement light generation unit from the light emitted by the light projecting unit. The measurement light generated by the measurement light generation unit is irradiated onto the measurement object by the optical system. Further, the spatial phase of the generated pattern is sequentially moved on the measurement object by a predetermined amount by the measurement light generation unit. Light from the measurement object is received by the light receiving unit, and a light reception signal indicating the amount of light received is output.

  Sectioning image data indicating an image of the measurement object is generated based on a plurality of image data generated at a plurality of phases of the pattern based on the light reception signal output from the light receiving unit. An image of the measurement object based on the sectioning image data is displayed on the display unit.

  Here, it is determined whether or not the light reception signal output by the light receiving unit is saturated for at least one of the plurality of generated image data. When the light reception signal is saturated, the user is notified that the light reception signal is saturated. Thereby, the user can easily recognize that the light reception signal is saturated.

  According to the present invention, the user can easily recognize that the intensity of the received light signal is saturated.

It is a block diagram which shows the structure of the microscope imaging device which concerns on one embodiment of this invention. It is a schematic diagram which shows the structure of the measurement part and measurement light supply part of the microscope imaging device of FIG. It is a schematic diagram which shows the optical path in the measurement part of the microscope imaging device of FIG. It is a block diagram which shows the structure of CPU. It is a figure which shows the example of the measurement light radiate | emitted by a pattern provision part. It is a figure which shows the kind of measurement conditions and a brightness parameter. It is a figure which shows intensity distribution of rectangular wave-shaped measurement light. It is a figure for demonstrating the moving amount | distance of the phase of the pattern of the rectangular wave shaped measurement light of FIG.7 (b). It is a figure which shows intensity distribution of one-dimensional sinusoidal measurement light. It is a figure for demonstrating the moving amount | distance of the phase of the pattern of the one-dimensional sinusoidal measurement light of FIG.9 (b). It is a figure which shows intensity distribution of the rectangular-wave-shaped measurement light when a spatial period is made smaller than the spatial period of FIG. It is a figure which shows intensity distribution of the rectangular-wave-shaped measurement light when a spatial period is made smaller than the spatial period of FIG. It is a figure which shows intensity distribution of the rectangular-wave-shaped measurement light when the width | variety of a bright part is made larger than the width | variety of the bright part of FIG. It is a figure which shows intensity distribution of the rectangular wave shaped measurement light when the width | variety of a bright part is made smaller than the width | variety of the bright part of FIG. It is a flowchart which shows the control process of the light modulation element by CPU. It is a figure which shows the light reception signal output from a light-receiving part. It is a figure which shows the example of a display of a display part. It is a figure which shows the example of a display of a display part. It is a figure which shows an example of the image displayed on the main window in normal display. It is a figure which shows the other example of the image displayed on the main window in normal display. It is a figure which shows an example of the image superimposed and displayed on the main window in normal display. It is a figure which shows the other example of the image superimposed and displayed on the main window in normal display. It is a figure which shows the sectioning image of the measuring object displayed on the main window in the preview display. It is a figure which shows the sectioning image of the measuring object displayed on the main window in the preview display. It is a figure which shows an example of a measurement condition detailed setting window. It is a figure for demonstrating the determination method of the focused pixel data in sectioning observation. It is a figure for demonstrating the determination method of the focused pixel data in normal observation. It is a figure which shows an example of an omnifocal image creation window. It is a flowchart which shows an omnifocal image data generation process. It is a flowchart which shows an omnifocal image data generation process.

(1) Configuration of Microscope Imaging Device FIG. 1 is a block diagram showing a configuration of a microscope imaging device according to an embodiment of the present invention. FIG. 2 is a schematic diagram illustrating the configuration of the measurement unit and the measurement light supply unit 300 of the microscope imaging apparatus 500 of FIG. FIG. 3 is a schematic diagram showing an optical path in the measurement unit of the microscope imaging apparatus 500 of FIG.

  Hereinafter, a microscope imaging apparatus 500 according to the present embodiment will be described with reference to FIGS. As shown in FIG. 1, the microscope imaging apparatus 500 includes a measurement unit 100, a PC (personal computer) 200, a measurement light supply unit 300, and a display unit 400.

  As shown in FIG. 2, the measurement light supply unit 300 includes a power supply device 310, a light projecting unit 320, and a light guide member 330. In this example, the light guide member 330 is a liquid light guide. The light guide member 330 may be, for example, a glass fiber or a quartz fiber. The power supply device 310 supplies power to the light projecting unit 320 and supplies power to the measurement unit 100 via a power cable (not shown).

  The light projecting unit 320 includes a measurement light source 321, a light reduction mechanism 322, and a light shielding mechanism 323. The measurement light source 321 is, for example, a metal halide lamp. The measurement light source 321 may be another light source such as a mercury lamp or a white LED (light emitting diode). Hereinafter, the light emitted from the measurement light source 321 is referred to as measurement light.

  The dimming mechanism 322 includes a plurality of ND (Neutral Density) filters having different transmittances. The dimming mechanism 322 is arranged such that any one of the plurality of ND filters is positioned on the optical path of the measurement light emitted from the measurement light source 321. By selectively switching the ND filter located on the optical path of the measurement light, the intensity of the measurement light passing through the dimming mechanism 322 can be adjusted. The dimming mechanism 322 may include an optical element such as an optical modulator capable of adjusting the light intensity instead of the plurality of ND filters.

  The light shielding mechanism 323 is, for example, a mechanical shutter. The light shielding mechanism 323 is disposed on the optical path of the measurement light that has passed through the dimming mechanism 322. When the light shielding mechanism 323 is in the open state, the measurement light passes through the light shielding mechanism 323 and is input to the light guide member 330. On the other hand, when the light shielding mechanism 323 is in the closed state, the measurement light is blocked and is not input to the light guide member 330. The light blocking mechanism 323 may include an optical element such as an optical modulator that can switch between passage and blocking of light.

  The measurement unit 100 is, for example, a fluorescence microscope, and includes a pattern applying unit 110, a light receiving unit 120, a transmitted light supply unit 130, a stage 140, a filter unit 150, a lens unit 160, and a control board 170. The light receiving unit 120, the filter unit 150, the lens unit 160, the stage 140, and the transmitted light supply unit 130 are arranged in this order from the bottom to the top.

  On the stage 140, the measuring object S is placed. In this example, the measuring object S is a biological specimen containing various proteins. The measuring object S is coated with a fluorescent reagent that fuses with a specific protein. Fluorescent reagents include, for example, GFP (green fluorescent protein), Texas Red (Texas Red), and DAPI (diamidinophenylindole).

  GFP absorbs light near 490 nm and emits light with a wavelength near 510 nm. Texas Red absorbs light near a wavelength of 596 nm and emits light near a wavelength of 620 nm. DAPI absorbs light near a wavelength of 345 nm and emits light near a wavelength of 455 nm.

  Two directions perpendicular to each other in a plane (hereinafter referred to as a placement surface) on the stage 140 on which the measurement object S is placed are defined as an X direction and a Y direction, and the directions perpendicular to the placement surface are defined as the directions. It is defined as the Z direction. In the present embodiment, the X direction and the Y direction are horizontal directions, and the Z direction is a vertical direction. In the present embodiment, stage 140 is an XY stage, and is arranged so as to be movable in the X direction and the Y direction by a stage drive unit (not shown).

  The measurement unit 100 includes an optical system that guides measurement light emitted from the measurement light supply unit 300 to the measurement object S, an optical system that guides light emitted from the transmitted light supply unit 130 to the measurement object S, and the measurement object. An optical system for guiding the light from S to the light receiving unit 120 is configured.

  The pattern providing unit 110 includes a light output unit 111, a light modulation element 112, and a plurality (two in this example) of mirrors 113. The light output unit 111 outputs the measurement light input to the light guide member 330. The light output from the light output unit 111 is reflected by the plurality of mirrors 113 and enters the light modulation element 112.

  The light modulation element 112 is, for example, a DMD (digital micromirror device). The DMD is composed of a plurality of micromirrors arranged two-dimensionally. The light modulation element 112 may be LCOS (Liquid Crystal on Silicon: reflection type liquid crystal element) or LCD (liquid crystal display). The light incident on the light modulation element 112 is converted into a preset pattern and a preset intensity (brightness) by a pattern generation unit 212 described later, and is emitted to the filter unit 150.

  The filter unit 150 includes a plurality (three in this example) of filter cubes 151 and filter turrets 152. The plurality of filter cubes 151 correspond to a plurality of types of fluorescent reagents applied to the measurement object S. As shown in FIG. 3, each filter cube 151 includes a frame 151a, an excitation filter 151b, a dichroic mirror 151c, and an absorption filter 151d. The frame 151a is a cubic member that supports the excitation filter 151b, the dichroic mirror 151c, and the absorption filter 151d.

  The excitation filter 151b in FIG. 2 is a band-pass filter that passes light in the first wavelength band. The absorption filter 151d is a band-pass filter that passes light in a second wavelength band different from the first wavelength band. The dichroic mirror 151c is a mirror that reflects light in a wavelength band including the first wavelength band and passes light in a wavelength band including the second wavelength band. The first and second wavelength bands are different for each filter cube 151 depending on the absorption wavelength and emission wavelength of the fluorescent reagent.

  The filter turret 152 has a disk shape. In the present embodiment, the filter turret 152 is provided with four filter cube mounting portions 152a at approximately 90 ° intervals. Each filter cube attaching part 152a is an opening formed so that the filter cube 151 can be attached.

  In the present embodiment, three filter cubes 151 are attached to the three filter cube attachment parts 152a, respectively, and the filter cube 151 is not attached to the remaining one filter cube attachment part 152a. Therefore, it is possible to perform bright field observation without using the filter cube 151 by arranging the filter cube attachment portion 152a to which the filter cube 151 is not attached on the optical path of the measurement light. In the example of FIG. 2, two filter cubes 151 are attached to the filter turret 152.

  The filter turret 152 is rotatably arranged at a predetermined angular interval (90 ° interval in this example) around an axis parallel to the Z direction by a filter turret driving unit (not shown). The user selects the filter cube 151 used for measuring the measurement object S by rotating the filter turret 152 by operating the operation unit 250 of the PC 200 described later.

  The selected filter cube 151 is attached to the filter unit 150 so that the measurement light enters the excitation filter 151b. As shown in FIG. 3, when the measurement light is incident on the excitation filter 151b, only the component having the first wavelength band in the measurement light passes through the excitation filter 151b. The measurement light that has passed through the excitation filter 151b is reflected by the dichroic mirror 151c toward the upper lens unit 160 (FIG. 2).

  The lens unit 160 includes a plurality (six in this example) of objective lenses 161, a lens turret 162, and a focal position adjustment mechanism 163. The plurality of objective lenses 161 have different magnifications. The lens turret 162 has a disk shape. In the present embodiment, six objective lens mounting portions 162 a are provided on the lens turret 162 at approximately 60 ° intervals. Each objective lens attachment portion 162a is an opening formed so that the objective lens 161 can be attached.

  In the present embodiment, six objective lenses 161 are attached to the six objective lens attachment portions 162a, respectively. In the example of FIG. 2, three objective lenses 161 are attached to the lens unit 160.

  The lens turret 162 is disposed so as to be rotatable at a predetermined angular interval (60 ° interval in this example) around an axis parallel to the Z direction by a lens turret driving unit (not shown). The user selects the objective lens 161 used for measuring the measuring object S by operating the operation unit 250 of the PC 200 described later and rotating the lens turret 162. The selected objective lens 161 overlaps the selected filter cube 151. Thereby, as shown in FIG. 3, the measurement light reflected by the dichroic mirror 151 c of the filter cube 151 passes through the selected objective lens 161.

  The focal position adjusting mechanism 163 in FIG. 2 is arranged so that the lens turret 162 can be moved in the Z direction by a focal position adjusting mechanism driving unit (not shown). Thereby, the relative distance between the measuring object S on the stage 140 and the selected objective lens 161 is adjusted. The stage 140 has an opening at a substantially central portion. The measurement light that has passed through the objective lens 161 passes through the opening of the stage 140 while being condensed and is applied to the measurement object S.

  The measurement object S irradiated with the measurement light absorbs the measurement light and emits fluorescence in a wavelength band including the second wavelength band. The fluorescence emitted below the measuring object S passes through the selected objective lens 161 and the dichroic mirror 151c and the absorption filter 151d of the selected filter cube 151. Thereby, a component having the second wavelength band in the fluorescence enters the light receiving unit 120.

  In the present embodiment, the measurement unit 100 is a fluorescence microscope capable of observing fluorescence from the measurement object S, but is not limited thereto. The measurement unit 100 may be, for example, a reflection microscope. In this case, a half mirror is attached to the filter cube attachment portion 152 a of the filter turret 152 instead of the filter cube 151.

  The transmitted light supply unit 130 is used for bright field observation, phase difference observation, differential interference observation, dark field observation, declination observation, or polarization observation of the measurement object S. The transmitted light supply unit 130 includes a transmitted light source 131 and a transmitted optical system 132. The transmissive light source 131 is, for example, a white LED. The transmissive light source 131 may be another light source such as a halogen lamp. Hereinafter, the light emitted from the transmissive light source 131 is referred to as transmitted light.

  The transmission optical system 132 includes optical elements such as an aperture stop, a phase difference slit, a relay lens, a condenser lens, and a shutter. The transmitted light emitted from the transmissive light source 131 passes through the transmissive optical system 132 and is applied to the measurement object S on the stage 140.

  The transmitted light passes through the measuring object S and passes through the objective lens 161. Thereafter, the transmitted light passes through a filter cube attachment portion 152a (hereinafter referred to as an opening portion of the filter turret 152) of the filter turret 152 to which the filter cube 151 is not attached, and enters the light receiving portion 120.

  The light receiving unit 120 includes a camera 121, a color filter 122, and an imaging lens 123. The camera 121 is, for example, a CCD (charge coupled device) camera including an image sensor. The image sensor is, for example, a monochrome CCD. The imaging device may be another imaging device such as a CMOS (complementary metal oxide semiconductor) image sensor.

  The color filter 122 includes an R (red) filter that passes light of red wavelength, a G (green) filter that passes light of green wavelength, and a B (blue) filter that passes light of blue wavelength. The fluorescence or transmitted light incident on the light receiving unit 120 is collected and imaged by the imaging lens 123 and then received by the camera 121 through the color filter 122. Thereby, the image of the measuring object S is obtained. From each pixel of the image sensor of the camera 121, an analog electric signal (hereinafter referred to as a light reception signal) corresponding to the amount of received light is output to the control board 170.

  Unlike a color CCD, a monochrome CCD does not need to be provided with pixels that receive red wavelength light, pixels that receive green wavelength light, and pixels that receive blue wavelength light. Therefore, the measurement resolution of the monochrome CCD is higher than the resolution of the color CCD. Further, unlike a color CCD, a monochrome CCD does not require a color filter for each pixel. Therefore, the sensitivity of the monochrome CCD is higher than that of the color CCD. For these reasons, the camera 121 in this example is provided with a monochrome CCD.

  In this example, light that has passed through the R filter, G filter, and B filter of the color filter 122 is received by the camera 121 in a time-sharing manner. According to this configuration, a color image of the measuring object S can be obtained by the light receiving unit 120 using a monochrome CCD.

  On the other hand, when the color CCD has sufficient resolution and sensitivity, the image sensor may be a color CCD. In this case, since the camera 121 does not need to receive light that has passed through the R filter, G filter, and B filter in a time-sharing manner, the color filter 122 is not provided in the light receiving unit 120. Thereby, the configuration of the light receiving unit 120 can be simplified.

  On the control board 170, an A / D converter (analog / digital converter) and a FIFO (First In First Out) memory (not shown) are mounted. The light reception signal output from the camera 121 is sampled at a constant sampling period by the A / D converter and converted into a digital signal based on control by the PC 200. Digital signals output from the A / D converter are sequentially stored in the FIFO memory. The digital signals stored in the FIFO memory are sequentially transferred to the PC 200 as pixel data.

  The control board 170 controls operations of the pattern applying unit 110, the light receiving unit 120, the transmitted light supply unit 130, the stage 140, the filter unit 150, and the lens unit 160 based on control by the PC 200. Further, the control board 170 controls the operation of the light projecting unit 320 of the measurement light supply unit 300 based on the control by the PC 200.

  As shown in FIG. 1, the PC 200 includes a CPU (Central Processing Unit) 210, a ROM (Read Only Memory) 220, a RAM (Random Access Memory) 230, a storage device 240 and an operation unit 250. The operation unit 250 includes a keyboard and a pointing device. A mouse or a joystick is used as the pointing device.

  The display unit 400 is configured by, for example, an LCD panel or an organic EL (electroluminescence) panel. In the example of FIG. 2, the PC 200 and the display unit 400 are realized by a single notebook personal computer.

  The ROM 220 stores a system program. The RAM 230 is used for processing various data. The storage device 240 is composed of a hard disk or the like. The storage device 240 stores an image processing program and a microscope imaging program. The storage device 240 is used for storing various data such as pixel data provided from the measurement unit 100.

  FIG. 4 is a block diagram showing the configuration of the CPU 210. As illustrated in FIG. 4, the CPU 210 includes an image data generation unit 211, a pattern generation unit 212, a control unit 213, and a focus detection unit 214. The image data generation unit 211 generates image data based on the pixel data given from the measurement unit 100. Image data is a set of a plurality of pixel data.

  The pattern generation unit 212 generates a pattern to be irradiated to the measurement object while sequentially moving the spatial phase by a predetermined amount as the measurement light pattern emitted from the light modulation element 112 in FIG. The control unit 213 irradiates the measurement object S with measurement light having a predetermined pattern by controlling the light modulation element 112 via the control board 170 in FIG. 2 based on the pattern generated by the pattern generation unit 212. While shifting the phase of the pattern.

  The control unit 213 controls operations of the light receiving unit 120, the transmitted light supply unit 130, the stage 140, the filter unit 150, the lens unit 160, and the light projecting unit 320 via the control board 170. Further, the control unit 213 performs various processes on the generated image data using the RAM 230 and causes the display unit 400 to display an image based on the image data.

  In the measurement unit 100, the position of the focal point of the objective lens 161 with respect to the measurement target S (hereinafter, the focal position of the objective lens 161) is changed by changing the relative distance between the measurement target S and the objective lens 161 of FIG. Changes). The measuring object S is irradiated with measurement light while the focal position of the objective lens 161 is changed. Thereby, image data of the measuring object S at each focal position is generated. The focus detection unit 214 detects a focus position based on a plurality of image data respectively generated by the image data generation unit 211 when a plurality of focus positions of the objective lens 161 are changed by the focus position adjustment mechanism 163.

  In the microscope imaging apparatus 500 according to the present embodiment, it is possible to perform epi-illumination observation of the measurement object S using the light projecting unit 320 of FIG. 2, and normal transmission of the measurement object S using the transmitted light supply unit 130. Observation (transmission observation) can be performed.

  As the epi-illumination observation, sectioning observation using patterned measurement light and normal epi-illumination observation using uniform measurement light described below can be performed. In the following description, transmission normal observation and epi-illumination normal observation are collectively referred to as normal observation.

(2) Sectioning Observation and Epi-illumination Normal Observation In sectioning observation, the phase of the pattern is moved by a certain amount while irradiating the measuring object S with measuring light having a one-dimensional or two-dimensional pattern. The measurement light having a one-dimensional pattern has an intensity that periodically changes in one direction (for example, the Y direction) on the XY plane. The measurement light having a two-dimensional pattern has an intensity that periodically changes in two directions (for example, the X direction and the Y direction) intersecting each other on the XY plane.

  Hereinafter, measurement light having a pattern is referred to as pattern measurement light. In particular, measurement light having a one-dimensional pattern is called a one-dimensional pattern measurement light, and measurement light having a two-dimensional pattern is called a two-dimensional pattern measurement light. Further, measurement light having a uniform intensity is referred to as uniform measurement light.

  The pattern of the pattern measurement light is controlled by the light modulation element 112. Hereinafter, the pattern of the pattern measuring light will be described. Here, the portion of the pattern measurement light having an intensity equal to or higher than a predetermined value is called a bright portion, and the portion of the pattern measurement light having an intensity smaller than the predetermined value is called a dark portion. FIG. 5 is a diagram illustrating an example of measurement light emitted by the pattern imparting unit 110.

  FIG. 5A shows an example of the one-dimensional pattern measurement light. The one-dimensional pattern measurement light in FIG. 5A is referred to as rectangular wave measurement light. The cross section of the rectangular wave-shaped measurement light includes a plurality of linear bright portions that are parallel to one direction (for example, the X direction) and arranged at substantially equal intervals in another direction (for example, the Y direction) orthogonal to the one direction. A plurality of linear dark portions are included between the light portions.

  FIG. 5B shows another example of the one-dimensional pattern measurement light. The one-dimensional pattern measurement light in FIG. 5B is referred to as a one-dimensional sinusoidal measurement light. The cross section of the one-dimensional sinusoidal measurement light includes, for example, a pattern that is parallel to the X direction and whose intensity changes sinusoidally in the Y direction.

  FIG. 5C shows an example of the two-dimensional pattern measurement light. The two-dimensional pattern measurement light in FIG. 5C is referred to as dot measurement light. The cross section of the dot-shaped measuring light includes a plurality of dot-shaped bright portions arranged at substantially equal intervals in the X direction and the Y direction.

  As another example of the two-dimensional pattern measurement light, the pattern measurement light may be a two-dimensional sinusoidal measurement light. The cross section of the two-dimensional sinusoidal measurement light includes a pattern in which the intensity changes sinusoidally in the X direction and the Y direction. As still another example of the two-dimensional pattern measurement light, the pattern measurement light may have a lattice pattern or a checker pattern (checkered pattern).

  In sectioning observation, the pattern measurement light is emitted by the measurement object S while moving the phase of the pattern measurement light by a certain amount so that the bright part of the pattern measurement light is irradiated at least once over the entire irradiation range of the measurement light. Detect fluorescence. Thereby, a plurality of image data of the measuring object S is generated.

  Hereinafter, the image data obtained when the measurement object S is irradiated with the pattern measurement light is referred to as pattern image data. An image based on the pattern image data is called a pattern image.

  In each pattern image data, pixel data corresponding to a bright portion of the pattern measurement light has a high value (luminance value), and pixel data corresponding to a dark portion of the pattern measurement light has a low value (luminance value). Therefore, in each pattern image, the pixel corresponding to the bright part of the pattern measurement light is bright, and the pixel corresponding to the dark part of the pattern measurement light is dark.

  A component (hereinafter referred to as an in-focus component) representing the degree of contrast is calculated from the plurality of pattern image data using the values of the plurality of pixel data for each pixel. Image data generated by connecting pixels having a focusing component is referred to as sectioning image data. An image based on the sectioning image data is called a sectioning image.

  In pattern image data generated using rectangular wave-shaped measurement light or dot-shaped measurement light, the focus component is, for example, the difference between the maximum value (maximum luminance value) and minimum value (minimum luminance value) of pixel data, or This is the standard deviation of the pixel data value. In the pattern image data generated using the one-dimensional sinusoidal measurement light or the two-dimensional sinusoidal measurement light, the focus component is, for example, the amplitude (peak-to-peak) of the pixel data.

  In the simplest method, it is possible to generate sectioning image data by selecting pixel data having the maximum value from a plurality of pattern image data for each pixel and connecting the selected pixel data for all pixels. It is.

  Here, each pattern image is affected by stray light. As a result, the pattern of the pattern image is blurred. A component caused by stray light in each pattern image data is called a blur component. The blur component includes a blur component due to stray light generated in each bright portion of the pattern measurement light and a blur component due to stray light from another bright portion adjacent to each bright portion of the pattern measurement light.

  Therefore, in order to remove the influence of stray light, the difference between the pixel data of the pattern image data when the bright portion and the dark portion of the pattern measurement light are irradiated is calculated for each pixel. Sectioning image data is generated by connecting the calculated differences for all pixels. For each pixel, the value of the pixel data of the pattern image data when the dark portion of the pattern measurement light is irradiated corresponds to a blur component. Therefore, sectioning image data from which the influence of stray light is removed can be obtained.

  As an example of a method for generating sectioning image data, in this embodiment, for each pixel, a difference between a maximum value (maximum luminance value) and a minimum value (minimum luminance value) of a plurality of pixel data of a plurality of pattern image data is calculated. calculate. Sectioning image data is generated by connecting the calculated differences for all pixels. The sectioning image may be generated based on a plurality of pixel data of the plurality of pattern image data by another method.

  For example, for each pixel, the standard deviation of the values of the plurality of pixel data of the plurality of pattern image data is calculated. Sectioning image data may be generated by connecting standard deviations for all the calculated pixels.

  In sectioning observation, when the one-dimensional pattern measurement light is used, the phase of the pattern measurement light is moved in the Y direction, for example, so that sectioning image data from which the blur component in the Y direction has been removed is generated. Further, since the phase of the pattern measurement light does not need to be moved in the X direction, for example, the number of imaging is reduced. Therefore, a sectioning image having a relatively high image quality can be obtained at high speed.

  On the other hand, in the sectioning observation, when the two-dimensional pattern measurement light is used, the phase of the pattern measurement light is moved in the X direction and the Y direction, so that the sectioning image data from which the blur component in the X direction and Y direction is removed. Generated. Therefore, although the number of times of imaging is increased as compared with the case where the one-dimensional pattern measurement light is used, a sectioning image having a very high image quality can be obtained.

  In particular, when the measurement unit 100 is a fluorescence microscope, the fluorescent reagent of the measurement object S may be faded by irradiating the measurement object S with a number of times of pattern measurement light. Therefore, depending on the measurement object S, it may be important to reduce the number of times of imaging.

  In the present embodiment, the one-dimensional pattern measurement light and the two-dimensional pattern measurement light can be switched easily and at high speed by controlling the light modulation element 112. Therefore, the user can select the pattern measurement light used for sectioning observation in consideration of the time required for generating the sectioning image data (number of imaging) and the image quality of the obtained sectioning image.

  FIG. 5D shows an example of uniform measurement light. The uniform measurement light has a uniform intensity distribution. That is, the uniform measurement light is measurement light composed of only a bright part. In the epi-illumination normal observation, the uniform measurement light shown in FIG. 5D is irradiated to all parts of the measurement object S, and the fluorescence emitted by the measurement object S is detected. Thereby, the image data of the measuring object S is generated. The image data obtained in the reflected normal observation is called reflected normal image data, and an image based on the reflected normal image data is called the reflected normal image.

  When the measuring object S has a three-dimensional structure, the objective lens 161 is not focused on the other part of the measuring object S even if the objective lens 161 is focused on a part of the measuring object S. . Therefore, some pixel data of the sectioning image data or the epi-illumination normal image data at a certain focus position is obtained in a state where the measurement object S is in focus, and the other part of the pixel data is obtained from the measurement object S. Obtained when the part is out of focus. Hereinafter, pixel data obtained in a state where a part of the measuring object S is in focus is referred to as focused pixel data.

  Among the plurality of sectioning image data or epi-illumination normal image data obtained at a plurality of focal positions, the image data obtained in a state in which the whole measuring object S is in focus by combining the focused pixel data. Generated. Hereinafter, the image data obtained in a state where the entire measurement object S is in focus is referred to as omnifocal image data. An image based on the omnifocal image data is referred to as an omnifocal image.

(3) Setting of measurement conditions (a) Measurement conditions and brightness parameters Measurement conditions in sectioning observation and brightness parameters in sectioning observation and epi-illumination normal observation will be described. FIG. 6 is a diagram showing types of measurement conditions and brightness parameters.

  As shown in FIG. 6, the measurement conditions in sectioning observation include a pattern type and measurement parameters. The pattern type includes pattern measurement light and uniform measurement light. The pattern measurement light includes one-dimensional pattern measurement light and two-dimensional pattern measurement light. The one-dimensional pattern measurement light includes rectangular wave measurement light and one-dimensional sinusoidal measurement light. The two-dimensional pattern measurement light includes dot-shaped measurement light and two-dimensional sinusoidal measurement light.

  Measurement parameters in sectioning observation include the number of times of imaging, the amount of movement of the phase of the pattern measurement light, the width of the bright and dark portions of the pattern measurement light, and the spatial period of the phase of the pattern measurement light. The number of times of imaging is the number of times pattern image data is generated.

  The brightness parameter includes the exposure time of the light receiving unit 120, the gain of the light receiving unit 120, the number of bins in the pattern image data or the incident normal image data, and the intensity of fluorescence (measurement light). Here, the binning number means the number of pixel data to be combined in a binning process in which a plurality of pixel data is pseudo combined and handled as one pixel data.

  An appropriate measurement parameter in sectioning observation varies depending on the pattern type. The brightness parameter is automatically set appropriately in conjunction with the measurement conditions. Hereinafter, the setting of the measurement parameter according to the pattern type will be described.

(B) Rectangular wave-like measurement light FIG. 7 is a diagram showing the intensity distribution of the rectangular wave-like measurement light. 7A and 7B, the horizontal axis indicates the position of the rectangular wave measuring light (for example, the position in the Y direction), and the vertical axis indicates the intensity of the rectangular wave measuring light. FIG. 7A shows an ideal intensity distribution of the rectangular wave-shaped measurement light. In an ideal rectangular wave measuring light, each bright portion has a substantially rectangular intensity distribution.

  The maximum intensity (intensity of each bright part) of the rectangular wave measuring light is Imax. The minimum intensity (intensity of each dark part) of the rectangular wave measuring light is Imin. As the difference between the maximum intensity Imax and the minimum intensity Imin of the rectangular wave measuring light is larger, the image quality of the pattern image can be improved.

  The average intensity of the rectangular wave measuring light is Iave. The larger the average intensity Iave of the rectangular wave measuring light, the brighter the pattern image. Since the average intensity Iave of the rectangular wave measuring light is relatively small, the pattern image is relatively dark. Therefore, when the rectangular measurement light is selected, the brightness parameter is automatically set to be relatively large. Thereby, the pattern image can be brightened.

  The width of each bright part of the rectangular wave-like measurement light is W1, and the width of each dark part of the rectangular wave-like measurement light is W2. The pattern is repeated with a constant spatial period Ts. The spatial period Ts is the sum of the width W1 of the bright portion and the width W2 of the dark portion.

  When the light modulation element 112 is a DMD, the dimension of each micromirror is 1 unit. The width W1 of each bright portion of the rectangular wave measuring light is 4 units, for example, and the width W2 of each dark portion of the rectangular wave measuring light is 12 units, for example. In this case, the spatial period Ts is 16 units. The unit of the bright part and the dark part varies depending on the configuration of the light modulation element 112. For example, when the light modulation element 112 is an LCD, one unit is the size of one pixel.

  When the spatial period Ts is small, stray light may be irradiated from the other bright part to the part of the measuring object S to be irradiated with one bright part of the rectangular wave-shaped measurement light. In this case, blurring generated in the pattern image increases. Therefore, blurring generated in the pattern image can be reduced by setting the spatial period Ts large.

  Further, when the width W1 of each bright portion of the rectangular wave measurement light is large, the image quality of the pattern image in the details of the measurement target S is deteriorated by irradiating the measurement target S with stray light in each bright portion. Therefore, the image quality of the pattern image in the details of the measuring object S can be improved by setting the width W1 of the rectangular wave bright portion small.

  FIG. 7B shows an actual intensity distribution of the rectangular wave measuring light. In realistic rectangular wave-shaped measurement light, the intensity distribution of each bright part is substantially trapezoidal. Further, the actual maximum intensity Imax of the rectangular wave-shaped measurement light is smaller than the ideal maximum intensity Imax of the rectangular wave-shaped measurement light in FIG. The actual minimum intensity Imin of the rectangular wave-like measurement light is larger than the ideal minimum intensity Imin of the rectangular wave-like measurement light in FIG.

  The movement amount of the phase of the rectangular wave measuring light pattern is set so that the bright portion of the rectangular wave measuring light in FIG. 7B is irradiated at least once over the entire irradiation range of the measuring light. FIG. 8 is a diagram for explaining a phase shift amount of the rectangular wave-shaped measurement light pattern of FIG. 8A and 8B, the horizontal axis indicates the position of the rectangular wave measuring light (for example, the position in the Y direction), and the vertical axis indicates the intensity of the rectangular wave measuring light.

  In the example of FIG. 8A, the measurement target object S is irradiated with the rectangular wave measurement light so that the bright part is located in the part A of the measurement target object S at the first time point. Thereby, the fluorescence emitted from the portion A of the measuring object S is received by the light receiving unit 120. Based on the amount of received fluorescence, the first pattern image data of the measuring object S is generated.

  Thereafter, the phase of the pattern of the rectangular wave measuring light is moved in the Y direction by approximately 1/3 of the spatial period Ts. Next, at a second time point after the first time point, the measurement object S is irradiated with the rectangular wave-shaped measurement light so that the bright part is located at the part B of the measurement object S. Thereby, the fluorescence emitted from the portion B of the measuring object S is received by the light receiving unit 120. Based on the amount of received fluorescence, second pattern image data of the measuring object S is generated.

  Thereafter, the phase of the rectangular wave-shaped measurement light pattern is further moved in the Y direction by approximately 1/3 of the spatial period Ts. Next, at the third time point after the second time point, the measurement object S is irradiated with the rectangular wave-shaped measurement light so that the bright part is located at the part C of the measurement object S. Thereby, the fluorescence emitted from the part C of the measuring object S is received by the light receiving unit 120. Based on the amount of received fluorescence, the third pattern image data of the measuring object S is generated.

  In this way, by irradiating the measuring object S three times with the rectangular wave-shaped measuring light while moving the phase of the pattern, all the portions of the measuring object S are irradiated with the bright part. Sectioning image data is generated based on the generated first to third pattern image data. This sectioning image data is equivalent to the pattern image data generated when the measurement light having the intensity distribution indicated by the thick curve in FIG.

  However, in the example of FIG. 8A, the intensities of the bright portions irradiated near the boundaries of the portions A and B, near the boundaries of the portions B and C, and near the boundaries of the portions C and A are the portions A to C. It is smaller than the intensity of the bright part irradiated near the center. In this case, the pixels of the sectioning image data corresponding to the vicinity of the boundary between the parts A and B, the vicinity of the boundary between the parts B and C, and the vicinity of the boundary between the parts C and A become dark or missing. Therefore, accurate sectioning image data cannot be generated.

  In the example of FIG. 8B, at the first time point, the measurement object S is irradiated with the rectangular wave-shaped measurement light so that the bright part is positioned on the part A of the measurement object S, thereby measuring the measurement object. First pattern image data of the object S is generated. Thereafter, the phase of the pattern of the rectangular wave measuring light is moved in the Y direction by about 1/5 of the spatial period Ts.

  In this state, the emission of the same rectangular wave-shaped measurement light and the movement of the pattern phase are repeated. Thereby, at the second to fifth time points, the rectangular wave-shaped measurement light is irradiated to the measurement object S so that the bright portions are located in the parts B to E of the measurement object S, respectively, and the second to fifth points. Pattern image data is generated.

  In this way, by irradiating the measurement object S with the rectangular wave-shaped measurement light five times while shifting the phase of the pattern, the bright part is irradiated to all the parts of the measurement object S. Sectioning image data is generated based on the generated first to fifth pattern image data. This sectioning image data is equivalent to the pattern image data generated when the measurement light of the intensity distribution indicated by the thick curve in FIG.

  In the example of FIG. 8B, the intensity of the bright part irradiated to all the parts of the measuring object S is substantially uniform. Thereby, the accurate sectioning image data of the measuring object S can be generated.

(C) One-dimensional sinusoidal measurement light FIG. 9 is a diagram showing the intensity distribution of the one-dimensional sinusoidal measurement light. 9A and 9B, the horizontal axis indicates the position of the one-dimensional sinusoidal measurement light (for example, the position in the Y direction), and the vertical axis indicates the intensity of the one-dimensional sinusoidal measurement light.

  FIG. 9A shows an ideal intensity distribution of the one-dimensional sinusoidal measurement light. The maximum intensity of the one-dimensional sinusoidal measurement light is Imax. The minimum intensity of the one-dimensional sinusoidal measurement light is Imin. The average intensity of the one-dimensional sinusoidal measurement light is Iave. The average intensity Iave of the one-dimensional sinusoidal measurement light is larger than the average intensity Iave of the rectangular wave measurement light. Therefore, the pattern image using the one-dimensional sinusoidal measurement light is brighter than the pattern image using the rectangular wave measurement light. Therefore, when the one-dimensional sinusoidal measurement light is selected, the brightness parameter is automatically set to be relatively small.

  The pattern is repeated with a constant spatial period Ts. When the spatial period Ts is large, it is possible to reduce a wide range blur component generated in the sectioning image, but the image quality of the sectioning image in the details of the measurement object S is deteriorated. Therefore, the spatial period Ts is appropriately set by a trade-off between wide-area blurring generated in the sectioning image and the image quality of the sectioning image in the details of the measurement object S.

  FIG. 9B shows a realistic intensity distribution of the one-dimensional sinusoidal measurement light. The actual maximum intensity Imax of the sinusoidal measurement light is smaller than the ideal maximum intensity Imax of the one-dimensional sinusoidal measurement light in FIG. The actual minimum intensity Imin of the one-dimensional sinusoidal measurement light is larger than the ideal minimum intensity Imin of the one-dimensional sinusoidal measurement light in FIG.

  The movement amount of the phase of the pattern of the one-dimensional sinusoidal measurement light is set so that the bright portion of the one-dimensional sinusoidal measurement light in FIG. 9B is irradiated at least once over the entire irradiation range of the measurement light. FIG. 10 is a diagram for explaining the amount of phase shift of the pattern of the one-dimensional sinusoidal measurement light in FIG. 9B. The horizontal axis of FIG. 10 indicates the position of the one-dimensional sinusoidal measurement light (for example, the position in the Y direction), and the vertical axis indicates the intensity of the one-dimensional sinusoidal measurement light.

  In the example of FIG. 10, the measurement target object is irradiated with the one-dimensional sinusoidal measurement light so that the bright part is positioned at the part A of the measurement target object S at the first time point. S first pattern image data is generated. Thereafter, the phase of the pattern of the one-dimensional sinusoidal measurement light is moved in the Y direction by approximately 1/3 of the spatial period Ts.

  In this state, the emission of the same one-dimensional sinusoidal measurement light and the movement of the pattern phase are repeated. Thus, at the second and third time points, the measurement object S is irradiated with the one-dimensional sinusoidal measurement light so that the bright portions are located at the parts B and C of the measurement object S, respectively. The th pattern image data is generated.

  In this way, by irradiating the measuring object S three times with the one-dimensional sinusoidal measuring light while moving the phase of the pattern, the bright part is irradiated on all the parts of the measuring object S. Sectioning image data is generated based on the generated first to third pattern image data.

Specifically, first to 3-th of the respective values of any pixel data in the pattern image data L _A, L _B, when the L _C, values L of the pixel data is calculated by the following equation (1) . By connecting the calculated pixel data for all pixels, sectioning image data using one-dimensional sinusoidal measurement light can be generated.

(D) Dot measurement light The measurement parameter setting using the dot measurement light is the same as the measurement parameter setting using the rectangular wave measurement light except for the following points.

  The pattern of dot-shaped measurement light is repeated, for example, not only in the Y direction but also in the X direction with a constant spatial period Ts. Therefore, when dot-shaped measurement light is used, the pattern phase is moved in the Y direction by the set number of times (in this example, 5 times), and then the set number of times (in this example, 5 times). The phase of the pattern is moved in the X direction.

  In a state where the phase of the pattern is moved, the fluorescence emitted from the measuring object S is received by the light receiving unit 120. Thereby, the 1st to 25th pattern image data of the measuring object S is generated based on the amount of received fluorescence.

  In this way, by irradiating the measurement object S 25 times with the dot-shaped measurement light while moving the phase of the pattern, all the portions of the measurement object S are irradiated with the bright portions. Sectioning image data is generated based on the generated first to 25th pattern image data.

(E) Two-dimensional sinusoidal measurement light The measurement parameter setting using the two-dimensional sinusoidal measurement light is the same as the measurement parameter setting using the one-dimensional sinusoidal measurement light except for the following points.

  The pattern of the two-dimensional sinusoidal measurement light is repeated, for example, not only in the Y direction but also in the X direction with a constant spatial period Ts. Therefore, when two-dimensional sinusoidal measurement light is used, the pattern phase is moved in the Y direction a set number of times (three times in this example). This movement of the phase of the pattern in the Y direction is repeated while moving the phase of the pattern in the X direction a set number of times (in this example, 3 times).

  In a state where the phase of the pattern is moved, the fluorescence emitted from the measuring object S is received by the light receiving unit 120. Thereby, the 1st to 9th pattern image data of the measuring object S is generated based on the amount of received fluorescence.

  In this way, by irradiating the measuring object S nine times with the two-dimensional sinusoidal measuring light while moving the phase of the pattern, the bright part is irradiated on all parts of the measuring object S. Sectioning image data is generated based on the generated first to ninth pattern image data. Specifically, the pixel data value L similar to that in Expression (1) is calculated for all pixels, and sectioning image data using the two-dimensional sinusoidal measurement light is generated by connecting the calculated pixel data. Can do.

(F) Uniform measurement light When sectioning observation is not performed, the incident normal image data can be generated by using the uniform measurement light by the normal incident observation. When the uniform measurement light is used, the bright part is irradiated to all the parts of the measurement object S, so that it is not necessary to move the phase of the uniform measurement light. For this reason, the spatial period Ts is not set. Since the intensity of the uniform measurement light is sufficiently large, the image of the measuring object S using the uniform measurement light is sufficiently bright. Therefore, when the uniform measurement light is selected, the brightness parameter is automatically set sufficiently small. By irradiating the measurement object S once with the uniform measurement light, one incident normal image data is generated.

  Thus, sectioning observation and epi-illumination normal observation can be used properly. In sectioning observation, a pattern type is set. Thereby, sectioning image data having characteristics according to the pattern type can be generated. Measurement parameters are set according to the set pattern type.

  As an example of the measurement parameter, the number of times of imaging is set to 5 to 10 times for the rectangular wave-shaped measurement light, and is set to 3 to 4 times for the one-dimensional sinusoidal measurement light. The number of times of imaging is set to 25 to 100 times for the dot-shaped measurement light, and is set to 9 to 16 times for the two-dimensional sinusoidal measurement light. Note that the number of times of imaging is one for uniform measurement light.

  In this example, the number of times of imaging is automatically set by setting the pattern type, the amount of movement of the pattern phase, and the spatial period Ts. For example, the number of times of imaging is given by the ratio of the spatial period Ts to the movement amount of the pattern phase when using the one-dimensional measurement light, and the space with respect to the movement amount of the pattern phase when using the two-dimensional measurement light. It is given by the square of the ratio of the period Ts.

  The brightness parameter is automatically and appropriately set in conjunction with the ratio of the bright part to the dark part of the pattern measurement light that changes depending on the measurement conditions. As a first example of the brightness parameter, the exposure time of the light receiving unit 120 is set to be relatively long in the rectangular wave measurement light and the dot measurement light. The exposure time of the light receiving unit 120 is set to be relatively short for the one-dimensional sinusoidal measurement light and the two-dimensional sinusoidal measurement light. The exposure time of the light receiving unit 120 is set sufficiently short for uniform measurement light.

  As a second example of the brightness parameter, the intensity of the fluorescence, that is, the intensity of the measurement light may be set to be relatively large in the rectangular wave measurement light and the dot measurement light. The intensity of the measurement light may be set to be relatively small in the one-dimensional sinusoidal measurement light and the two-dimensional sinusoidal measurement light. The intensity of the measurement light may be set sufficiently small for uniform measurement light.

  As a third example of the brightness parameter, the binning number in the pattern image data may be set to be relatively large in the rectangular wave measurement light and the dot measurement light. The number of bins in the pattern image data may be set to be relatively small in the one-dimensional sine wave measurement light and the two-dimensional sine wave measurement light. The binning number in the incident normal image data may be set to be sufficiently small in the uniform measurement light.

  As a fourth example of the brightness parameter, the gain of the light receiving unit 120 may be set to be relatively large in the rectangular wave measurement light and the dot measurement light. The gain of the light receiving unit 120 may be set to be relatively small in the one-dimensional sinusoidal measurement light and the two-dimensional sinusoidal measurement light. The gain of the light receiving unit 120 may be set sufficiently small for uniform measurement light.

  In the pattern measurement light, the spatial period Ts is set according to the tolerance of the blur generated in the pattern image. In addition, in the rectangular wave measurement light and the dot measurement light, the width W1 of the bright portion is set.

  FIG. 11 is a diagram showing the intensity distribution of the rectangular wave measuring light when the spatial period Ts is smaller than the spatial period Ts of FIG. FIG. 11A shows an ideal intensity distribution of the rectangular wave-like measurement light, and FIG. 11B shows an actual intensity distribution of the rectangular wave-like measurement light.

  When the spatial period Ts is small, stray light from another bright part may be irradiated to the part of the measuring object S to be irradiated with one bright part of the rectangular wave-shaped measurement light. In the example of FIG. 11B, for example, the stray light from the bright part a or the bright part c may be irradiated to the part of the measuring object S to be irradiated with the bright part b. In this case, wide-area blur components generated in the sectioning image increase.

  On the other hand, when the spatial period Ts is small, since the interval between the bright portions a to e is small, the irradiation of the bright portions a to e over the entire measurement light irradiation range is performed in a short time. Thereby, the number of times of imaging can be reduced, and sectioning image data can be generated in a short time.

  FIG. 12 is a diagram showing the intensity distribution of the rectangular wave measuring light when the spatial period Ts is larger than the spatial period Ts of FIG. 12A shows an ideal intensity distribution of the rectangular wave-like measurement light, and FIG. 12B shows an actual intensity distribution of the rectangular wave-like measurement light.

  When the spatial period Ts is large, since the interval between the bright portions a to e is large, it takes a long time to finish the irradiation of the bright portions a to e over the entire irradiation range of the measurement light. In this case, the number of times of imaging increases and it takes a long time until sectioning image data is generated.

  On the other hand, when the spatial period Ts is large, the portion of the measuring object S to be irradiated with one bright portion of the rectangular wave-shaped measurement light is hardly irradiated with stray light from another bright portion. As a result, it is possible to reduce a wide range blur component of the sectioning image.

  FIG. 13 is a diagram showing the intensity distribution of rectangular wave-shaped measurement light when the width W1 of the bright portion is larger than the width W1 of the bright portion in FIG. FIG. 13A shows an ideal intensity distribution of the rectangular wave-like measurement light, and FIG. 13B shows an actual intensity distribution of the rectangular wave-like measurement light.

  When the width W1 of the bright part is large, stray light generated in each bright part of the rectangular wave measurement light may be irradiated to the corresponding part of the measurement object S. In the example of FIG. 13B, stray light generated in each of the bright portions a to e may be irradiated to the corresponding portion of the measurement object S. In this case, the image quality of the sectioning image in the details of the measuring object S is degraded.

  On the other hand, when the width W1 of the bright portion is large, the width W2 of the dark portion is small, and therefore, the irradiation of the bright portions a to d to the entire measurement light irradiation range is performed in a short time. Thereby, the number of times of imaging can be reduced, and sectioning image data can be generated in a short time.

  FIG. 14 is a diagram showing the intensity distribution of the rectangular wave-shaped measurement light when the bright portion width W1 is smaller than the bright portion width W1 of FIG. FIG. 14A shows an ideal intensity distribution of the rectangular wave-like measurement light, and FIG. 14B shows an actual intensity distribution of the rectangular wave-like measurement light.

  When the width W1 of the bright part is small, the width W2 of the dark part is large. Therefore, it takes a long time to finish the irradiation of the bright parts a to d over the entire irradiation range of the measurement light. In this case, the number of times of imaging increases and it takes a long time until sectioning image data is generated.

  On the other hand, when the width W1 of the bright part is small, the stray light generated in each bright part of the rectangular wave measurement light is hardly irradiated to the corresponding part of the measurement object S. Thereby, the image quality of the sectioning image in the detail of the measuring object S can be improved.

  The user can arbitrarily set the movement amount of the phase of the pattern. Here, in order to irradiate the entire light irradiation range of the rectangular wave-shaped measurement light at least once, it is preferable that the movement amount of the phase of the pattern is set slightly smaller than the width W1 of the light portion. . The ratio of the spatial period Ts to the set movement amount of the phase of the pattern is the number of times of imaging.

  In the present embodiment, the measurement parameters set corresponding to the pattern type are automatically switched by switching the pattern type. The measurement parameter may be a fixed value corresponding to the pattern type, or may be selected from a plurality of predetermined values. In addition, when some of the plurality of measurement parameters are selected by the user, the other part of the measurement parameters may be automatically determined.

  Other setting items may be added in addition to the above measurement conditions and brightness parameters. For example, in the present embodiment, the user can set an ROI (region of interest) using the operation unit 250 of FIG. In this case, the light modulation element 112 may be controlled so that the measurement light is irradiated only on the portion of the measurement object S corresponding to the ROI and the measurement light is not irradiated on the other portions. Thereby, a plurality of pattern image data or epi-illumination normal image data can be generated at high speed.

  Even when the light receiving unit 120 receives light having uniform intensity, the level of the light reception signal in each pixel output from the light receiving unit 120 may not be uniform. This is because the intensity of light is not inherently uniform, the reflectance of the mirror is not uniform over the entire reflecting surface, or the transmittance of the lens is not uniform over the entire lens. As a result, a shading phenomenon occurs in which the central portion of the image becomes bright and the peripheral portion of the image becomes dark.

  Therefore, even if the light modulation element 112 is controlled to emit intensity measurement light multiplied by the shading correction coefficient in order to uniformly correct the level of the non-uniform received light signal (hereinafter referred to as shading correction). Good. Thereby, a plurality of pattern image data or epi-illumination normal image data can be generated accurately.

  FIG. 15 is a flowchart showing a control process of the light modulation element 112 by the CPU 210. The CPU 210 waits until the measurement conditions are instructed by the user (step S1). When the measurement condition is instructed by the user in step S1, the CPU 210 sets the pattern type based on the instructed measurement condition pattern type (step S2).

  Further, the CPU 210 calculates the number of imaging based on the spatial period Ts of the measurement parameter of the instructed measurement condition and the movement amount of the pattern phase (step S3). Either of the processes of steps S2 and S3 may be executed first. Subsequently, the CPU 210 controls the light modulation element 112 so as to emit the measurement light of the set pattern type (step S4). Thereafter, the CPU 210 determines whether or not the measurement light is emitted by the number of times of imaging calculated by the light modulation element 112 (step S5).

  In step S5, when the light modulation element 112 has not emitted the measurement light by the calculated number of times of imaging, the CPU 210 controls the light modulation element 112 to move the phase of the pattern by the set movement amount (step S5). S6). Thereafter, the CPU 210 returns to the process of step S4. Thereby, the processing of steps S4 to S6 is repeated until the measurement light is emitted by the light modulation element 112 for the calculated number of times of imaging. In step S <b> 5, when the light modulation element 112 emits the measurement light for the calculated number of times of imaging, the CPU 210 ends the control process for the light modulation element 112.

(4) Light reception level adjustment The measurement object S is irradiated with measurement light, whereby fluorescence is emitted from the measurement object S. The intensity of the fluorescence emitted from the measuring object S is proportional to the intensity of the measuring light applied to the measuring object S. Therefore, when the measurement object S is irradiated with the pattern measurement light, the measurement object S emits fluorescence having substantially the same pattern as the pattern measurement light pattern. The intensity of fluorescence is about 10 ⁻⁶ times the intensity of measurement light.

  The fluorescence emitted from the measuring object S is received by the light receiving unit 120. As a result, a light reception signal indicating the amount of received light is output from the light receiving unit 120. Based on the light reception signal output from the light receiving unit 120, the pattern image data of the measuring object S is generated. The level of the received light signal is proportional to the intensity of the fluorescence emitted from the measuring object S, the exposure time of the light receiving unit 120, and the gain of the light receiving unit 120. The intensity of the fluorescence emitted from the measuring object S is proportional to the intensity of the measuring light.

  Therefore, the level of the light reception signal can be adjusted by adjusting at least one of the intensity of the measurement light, the exposure time of the light receiving unit 120, and the gain of the light receiving unit 120. Hereinafter, adjusting the level of the received light signal by adjusting at least one of the intensity of the measurement light, the exposure time of the light receiving unit 120, and the gain of the light receiving unit 120 is referred to as received light level adjustment. If the intensity of the measurement light, the exposure time of the light receiving unit 120 or the gain of the light receiving unit 120 is excessively increased, the light reception signal output from the light receiving unit 120 is saturated.

  In sectioning observation, as described above, the measurement object S is irradiated with the pattern measurement light a plurality of times while shifting the phase of the pattern. Therefore, the fluorescence emitted from the measuring object S is received by the light receiving unit 120 a plurality of times.

  As a result, a plurality of pattern image data is generated based on the level of the light reception signal for each pixel at the time of multiple irradiations. In the present embodiment, the difference between the maximum value of pixel data corresponding to the maximum light reception level and the minimum value of pixel data corresponding to the minimum light reception level is calculated for each pixel based on the generated plurality of pattern image data. Is done. Sectioning image data is generated by connecting the differences calculated for all pixels. Thereby, the influence of stray light is removed from the generated sectioning image data.

  Thus, in order to remove the influence of stray light from the sectioning image data, the difference between the maximum value of the pixel data and the minimum value of the pixel data is calculated for each pixel. Therefore, the sectioning image from which the influence of stray light is removed becomes darker than the sectioning image from which the influence of stray light is not removed.

  In the present embodiment, the contrast of the sectioning image data is corrected by multiplying the value of each pixel data of the sectioning image data by a constant contrast correction amount larger than 1. The contrast correction amount is, for example, the ratio of the upper limit value of the output range of the light receiving unit 120 to the maximum value among a plurality of values of the sectioning image data.

  The contrast correction amount is not the ratio of the upper limit value of the output range of the light receiving unit 120 to the maximum value of the sectioning image data, but, for example, the light reception with respect to the average value of a plurality of values of the sectioning image data having values from the maximum value to the upper few percent. The ratio of the upper limit value of the output range of the unit 120 may be used.

  The sectioning image can be brightened by correcting the contrast. On the other hand, when the contrast is corrected, the brightness of a plurality of sectioning images cannot be compared quantitatively. Therefore, in the present embodiment, when the generated sectioning image data is stored in the storage device 240 of FIG. 1, the sectioning image data before the contrast is corrected and the metadata indicating the contrast correction amount are included. Stored independently.

  In this case, the brightness of a plurality of sectioning images can be quantitatively compared based on the sectioning image data before the contrast is corrected. Further, it is possible to display the sectioning image after the contrast is corrected based on the sectioning image data and the metadata before the contrast is corrected. Note that the storage format of the sectioning image data may be a general storage format or another unique storage format.

  Common storage formats include a TIFF (Tagged Image File) format, a JPEG (Joint Photographic Experts Group) format, a BMP (Windows (registered trademark) bitmap) format, or a PNG (Portable Network Graphics) format. When storing sectioning image data in a general storage format, if the software for displaying the image supports reading of metadata, the sectioning image after the contrast is corrected can be displayed.

  On the other hand, when the software for displaying an image does not support reading of metadata, the sectioned image after the contrast is corrected cannot be displayed. In this case, in addition to the sectioning image data and metadata before the contrast is corrected, the sectioning image data after the contrast is corrected may be separately stored as thumbnail data.

  By displaying the thumbnail of the sectioning image on the browser based on the thumbnail data, the visibility of the sectioning image on the browser can be improved. As a result, the searchability of sectioning image data can be improved.

  The higher the contrast of the sectioning image data before correction, the more the image quality of the sectioning image can be improved. For this reason, it is preferable to appropriately set the level of the light reception signal output from the light receiving unit 120. FIG. 16 is a diagram illustrating a light reception signal output from the light receiving unit 120. 16A to 16C, the horizontal axis indicates the horizontal position of the measuring object S (for example, the position in the Y direction), and the vertical axis indicates. The maximum light receiving level is Lmax, the minimum light receiving level is Lmin, and the upper limit value of the output range of the light receiving unit 120 is Lsat.

  FIG. 16A shows an example where the level of the received light signal is appropriate. In the example of FIG. 16A, the maximum light reception level Lmax is slightly lower than the upper limit value Lsat of the output range, and the minimum light reception level Lmin is slightly higher than the lower limit value of the output range. This increases the contrast Ct.

  FIG. 16B shows an example when the level of the received light signal is too low. When the level of the light reception signal is decreased from the state of FIG. 16A, the minimum light reception level Lmin becomes constant after the level is decreased to the lower limit value of the output range of the light receiving unit 120. On the other hand, when the level of the received light signal is lowered from the state of FIG. 16A, the maximum received light level Lmax remains after the minimum received light level Lmin becomes constant at the lower limit value of the output range of the light receiving unit 120. Can be reduced. For this reason, as shown in FIG. 16B, if the level of the received light signal is too low, the contrast Ct is lowered. In this case, the sectioning image becomes dark.

  Further, since the part of the measuring object S corresponding to the bright part of the measuring light and the part of the measuring object S corresponding to the dark part of the measuring light cannot be distinguished, the sectioning image cannot be generated accurately. . Furthermore, since the sectioning image is dark, the contrast correction amount increases when contrast correction is performed. For this reason, the noise component of the sectioning image also increases.

  FIG. 16C shows an example when the level of the received light signal is too high. When the level of the received light signal is increased from the state of FIG. 16A, the maximum received light level Lmax becomes constant after increasing to the upper limit value Lsat of the output range of the light receiving unit 120. On the other hand, when the level of the received light signal is increased from the state of FIG. 16A, the minimum received light level Lmin is after the maximum received light level Lmax becomes constant at the upper limit value Lsat of the output range of the light receiving unit 120. Can also rise. Therefore, as shown in FIG. 16C, if the level of the received light signal is too high, the contrast Ct is lowered. In this case, the sectioning image becomes dark although the level of the light reception signal is increased.

  When the level of the received light signal is further increased from the state shown in FIG. 16C, the contrast Ct becomes 0 when the minimum received light level Lmin also increases to the upper limit value Lsat of the output range of the light receiving unit 120. Further, since the fluorescence pattern near the maximum light receiving level Lmax is deformed, the sectioning image cannot be generated accurately.

(5) Display Unit (a) Image Display Area FIGS. 17 and 18 are diagrams showing display examples of the display unit 400. FIG. As shown in FIGS. 17 and 18, the display unit 400 is provided with an image display area 410 and a setting display area 420 arranged side by side. The setting display area 420 will be described later. In the image display area 410, a main window 411 and a sub window 412 are displayed. The display and non-display of the sub window 412 can be switched.

  The transmitted light emitted from the transmitted light supply unit 130 of FIG. 2 passes through the measurement object S and is received by the light receiving unit 120, whereby image data of the measurement object S is generated. Hereinafter, the image data of the measuring object S generated using transmitted light is referred to as transmitted normal image data, and an image based on the transmitted normal image data is referred to as transmitted normal image. The transmitted normal image data and the reflected normal image data are collectively referred to as normal image data, and the transmitted normal image and the reflected normal image are collectively referred to as a normal image. In the main window 411, various images such as a pattern image, a sectioning image, a normal image, and an omnifocal image are displayed.

  Specifically, the main window 411 is configured so that normal display and preview display can be executed selectively or simultaneously. The normal display is a method for displaying a pattern image, a sectioning image, a normal image, an omnifocal image, or the like based on already generated image data. The preview display is a method for displaying a pattern image or a sectioning image based on image data generated when a measurement condition is changed. In normal display, a plurality of images can be superimposed and displayed on the main window 411 based on already generated image data.

  In the preview display, the measurement object S is again irradiated with the measurement light only when the measurement condition, the brightness parameter, or the field of view is changed. For example, when the pattern type, the amount of movement of the phase of the pattern measurement light, the width W1 of the bright portion or the spatial period Ts of the phase, the exposure time of the light receiving unit 120 or the number of binning is changed, the measurement light is again applied to the measurement object S. Is irradiated. Alternatively, when the stage 140 is moved in the X direction or the Y direction, or when the focus position adjustment mechanism 163 is controlled, the measurement object S is irradiated with the measurement light again.

  Accordingly, pattern image or sectioning image data is generated based on the received fluorescence, and brightness image data is generated. As a result, the pattern image or sectioning image displayed on the main window 411 is updated.

  On the other hand, when the measurement conditions are not changed, there is no need to irradiate the measurement object S with the measurement light again. Therefore, the CPU 210 controls the light modulation element 112 to irradiate the measurement object S with the measurement light. Shut off. Thereby, unnecessary fading of the fluorescent reagent due to the measurement object S being continuously irradiated with the measurement light can be reduced.

  In the present embodiment, the light modulation element 112 can switch between irradiation of the measurement object S with the measurement light and blocking of the irradiation at high speed. Therefore, a plurality of pattern image data can be generated at high speed. Thereby, sectioning image data generated from a plurality of pattern image data can be generated at high speed. As a result, the sectioning image when the measurement condition is changed can be preview-displayed with high responsiveness.

  In the preview display, for example, it may be set to generate pattern image data by increasing the number of binning. In this case, pattern image data and sectioning image data can be generated at higher speed. Thereby, in the preview display, the pattern image or the sectioning image can be displayed at a higher speed. The user can select an appropriate measurement condition easily and in a short time while viewing the pattern image or sectioning image displayed as a preview.

  FIG. 19 is a diagram illustrating an example of an image displayed on the main window 411 in the normal display. FIG. 19A shows a sectioning image when the measurement object S is irradiated with measurement light having an absorption wavelength of GFP. FIG. 19B shows a sectioning image when the measuring object S is irradiated with measurement light having an absorption wavelength of Texas Red.

  FIG. 20 is a diagram illustrating another example of an image displayed on the main window 411 in the normal display. FIG. 20A shows a sectioning image when the measurement object S is irradiated with measurement light having an absorption wavelength of DAPI. FIG. 20B shows a normal transmission image when the measurement object S is irradiated with transmitted light for phase difference observation.

  FIG. 21 is a diagram illustrating an example of an image superimposed and displayed on the main window 411 in normal display. FIG. 21A shows an image in which two images are superimposed and displayed. The image in FIG. 21 (a) includes the sectioning image in FIG. 19 (a) and the sectioning image in FIG. 19 (b). FIG. 21B shows an image in which three images are superimposed and displayed. The image in FIG. 21B includes the sectioning image in FIG. 19A, the sectioning image in FIG. 19B, and the transparent normal image in FIG. 20B.

  FIG. 22 is a diagram showing another example of an image superimposed and displayed on the main window 411 in normal display. FIG. 22 shows an image in which four images are superimposed and displayed. The image of FIG. 22 includes the sectioning image of FIG. 19A, the sectioning image of FIG. 19B, the sectioning image of FIG. 20A, and the transparent normal image of FIG. 20B.

  In particular, when the measurement object S is a biological specimen, when observing the shape of the cell of the measurement object S, a sectioning image and a transmission normal image (for example, an image obtained by phase difference observation) are displayed in a superimposed manner. Is effective. Thereby, the user can easily recognize the part of the measuring object S where the fluorescence is generated only when the light having a specific wavelength is irradiated by the composition of the protein. As a result, the nucleus, cell membrane or DNA (deoxyribonucleic acid) in the cell can be easily identified.

  When binning processing is performed when generating sectioning image data or normal image data, the size of the sectioning image based on the sectioning image data or the normal image based on the normal image data is changed. Therefore, the size of the sectioning image or the normal image is different from the size of the sectioning image or the normal image based on other image data that is not subjected to the binning process.

  For this reason, when the superimposed display of images is performed, the images are enlarged or reduced so that all displayed images have the same size. Alternatively, the brightness parameter related to the image size, such as binning processing, may be set in common when all image data is generated. In this case, the size of the image based on the generated image data is unified, so that the superimposed display of the image can be easily performed.

  The user can display the sub window 412 in the image display area 410 by designating the sub window display check box 450 of the setting display area 420 in FIG. 18 using the operation unit 250 of the PC 200 in FIG. The sub window 412 displays an image before the contrast is corrected. Hereinafter, an image before the contrast is corrected is referred to as a brightness image, and image data for displaying the brightness image is referred to as brightness image data.

  23 and 24 are diagrams showing a sectioning image of the measuring object S displayed on the main window 411 in the preview display. In the main window 411, a pattern image of the measuring object S may be displayed.

  The exposure time of the light receiving unit 120 when acquiring the sectioning image of FIG. 23 is longer than the exposure time of the light receiving unit 120 when acquiring the sectioning image of FIG. That is, the level of the light receiving signal of the light receiving unit 120 in the example of FIG. 23 is higher than the level of the light receiving signal of the light receiving unit 120 in the example of FIG. Therefore, the light reception signal of the light receiving unit 120 in the example of FIG. 23 is more likely to be saturated than the light reception signal of the light receiving unit 120 in the example of FIG.

  In the example of FIG. 23, the light reception signal corresponding to the pixels in the region R surrounded by, for example, a white circle of the sectioning image is saturated. On the other hand, in the example of FIG. 24, the light reception signal corresponding to the pixels in the region R of the sectioning image is not saturated. However, since the contrast is corrected, the pixels in the region R in the example of FIG. 23 are displayed darker than the pixels in the region R in the example of FIG.

  According to the user's intuition, the higher the level of the received light signal, the brighter the sectioning image. When the level of the light reception signal is increased from a low state, the sectioning image becomes bright until the level of the light reception signal reaches a certain value. However, when the level of the light reception signal exceeds a certain value, the light reception signal in the pixel corresponding to the portion of the measuring object S irradiated with the bright portion of the pattern is saturated. On the other hand, the level of the light reception signal in the pixel corresponding to the part of the measuring object S irradiated with the dark part of the pattern increases. Therefore, the sectioning image based on the sectioning image data generated by the difference between the pixel data values becomes gradually darker.

  Therefore, the user may recognize that the level of the light reception signal of the light receiving unit 120 when obtaining the sectioning image of FIG. 23 is higher than the level of the light reception signal of the light receiving unit 120 when obtaining the sectioning image of FIG. Can not. Further, the user cannot recognize that the received light signal is saturated even when viewing the sectioning image of the main window 411 in FIG.

  Therefore, in the example of FIGS. 23 and 24, the maximum light reception level in each pixel is extracted based on a plurality of pattern image data constituting the sectioning image displayed on the main window 411. Lightness image data is generated based on the maximum received light level for each extracted pixel. A brightness image is displayed in the sub-window 412 based on the generated brightness image data. Further, it is determined whether the light reception signal output from the light receiving unit 120 is saturated based on the brightness image data.

  The brightness image indicates the brightness distribution state of a plurality of pattern images constituting the sectioning image displayed on the main window 411. Therefore, the brightness image in the subwindow 412 in FIG. 23 is displayed brighter than the brightness image in the subwindow 412 in FIG. When there is a pixel in which the light reception signal is saturated, the pixel in the lightness image is displayed in the sub-window 412 so that the pixel can be identified (in a different color in this example). Will be notified.

  The user sees the brightness image so that the level of the light receiving signal of the light receiving unit 120 when obtaining the sectioning image of FIG. 23 is higher than the level of the light receiving signal of the light receiving unit 120 when obtaining the sectioning image of FIG. Can be recognized. Further, the user can recognize that the light reception signal is saturated by looking at the brightness image.

  The user can change the brightness parameter such as the intensity of the measurement light by recognizing that the received light signal is saturated. Here, as described above, in the preview display, the measurement object S is again irradiated with the measurement light only when the measurement condition is changed. Accordingly, pattern image or sectioning image data is generated based on the received fluorescence, and brightness image data is generated. As a result, the pattern image or sectioning image displayed in the main window 411 is updated, and the brightness image displayed in the sub window 412 is updated.

  Note that the measurement light to the measurement object S is blocked at a high speed by controlling the light modulation element 112, but is not limited thereto. The measurement light may be blocked from the measurement object S by controlling the light shielding mechanism 323 of the light projecting unit 320 in FIG.

  Further, when displaying the pattern image on the main window 411, the CPU 210 irradiates the measurement object S with the measurement light whose pattern phase is moved by a certain amount each time the measurement condition is changed. The light modulation element 112 is controlled. In this case, the measurement light is prevented from being irradiated only on a specific portion of the measurement object S. Thereby, it is prevented that only the fluorescent reagent in the specific part of the measuring object S fades.

  In the present embodiment, the brightness image data is generated based on the maximum light reception level in each pixel of the plurality of pattern image data, but is not limited thereto. The brightness image data may be generated based on an average value of the light reception signals in each pixel of the plurality of pattern image data.

  Alternatively, the lightness image data may be any one of the plurality of pattern image data or two or more image data. When the brightness image data is any two or more of the plurality of pattern image data, two or more pattern images may be alternately displayed in the sub-window 412 as the brightness image.

  In the present embodiment, the contrast correction amount is uniformly determined for all the pixels of the sectioning image, but is not limited thereto. The contrast correction amount may be determined so that the pixel in the sectioning image is displayed brightly when there is a pixel in which the light reception signal is saturated.

  In this case, in the sectioning image, the pixel corresponding to the portion where the light reception signal is saturated is displayed brighter than the pixel corresponding to the portion where the light reception signal is not saturated. Thereby, the user can easily recognize that the received light signal is saturated by looking at the sectioning image.

  In this embodiment, when there is a pixel in which the light reception signal is saturated, the pixel in the lightness image is displayed in an identifiable manner, so that the user is notified of the pixel in which the light reception signal is saturated. It is not limited to this. The pixel in which the light reception signal is saturated may be notified to the user by displaying the pixel in the sectioning image or the pattern image in a distinguishable manner instead of the brightness image.

  For example, a painting process for painting a saturated pixel of the sectioning image so as to be identifiable may be performed. The filling process can be applied to both the normally displayed sectioning image and the previewed sectioning image. The value of pixel data to be filled is, for example, the maximum value of pixel data. In this example, since the pixel data has 8 bits, the maximum value of the pixel data is 255.

  The value of the pixel data to be filled may be any one of the maximum value, the average value, the minimum value, the median value, and the mode value of the pixel data of the pixel group around the saturated pixel. In this case, the brightness of the filled pixel (saturated pixel) and the brightness of the pixel group around the pixel do not differ greatly. As a result, it is possible to perform a painting process that does not give the user a sense of incongruity on saturated pixels.

  Alternatively, when there is a pixel in which the light reception signal is saturated, a notification screen indicating that the light reception signal is saturated may be displayed on the display unit 400. Alternatively, when there is a pixel in which the light reception signal is saturated, a notification sound indicating that the light reception signal is saturated may be output from a speaker (not shown).

  Alternatively, when there is a pixel in which the light reception signal is saturated, the following HDR (High Dynamic Range) processing may be automatically performed by the CPU 210. Alternatively, when there is a pixel in which the light reception signal is saturated, a screen that recommends performing HDR processing may be displayed on the display unit 400. Alternatively, when there is a pixel in which the light reception signal is saturated, sound recommending that HDR processing be performed may be output from a speaker (not shown).

  In the HDR process, a plurality of first pattern image data is generated in a state where the light reception level is adjusted so that the overexposed portion is not included in the plurality of pattern images of the measuring object S. In addition, a plurality of second pattern image data is generated in a state where the light reception level is adjusted so that the black pattern is not included in the plurality of pattern images of the measuring object S.

  The plurality of generated first pattern image data and the plurality of second pattern image data are combined. Thereby, a plurality of third pattern image data having an expanded dynamic range is generated. By synthesizing a plurality of third pattern image data, sectioning image data with an expanded dynamic range can be generated.

  In the HDR processing according to the present embodiment, the measurement light is sequentially irradiated onto the measurement object S in a state where the first light reception level adjustment is performed, and then the second light reception level adjustment is performed. Measurement light is sequentially applied to the measurement object S. According to this procedure, unlike the procedure of alternately adjusting the light reception level and irradiating the measurement object S with the measurement light, the measurement light is not continuously irradiated to a specific portion of the measurement object S. . Thereby, it is prevented that only the fluorescent reagent in the specific part of the measuring object S fades.

(B) Setting Display Area As shown in FIGS. 17 and 18, in the setting display area 420, a filter selection field 430, a measurement condition setting field 440, a sub window display check box 450, and an accompanying function setting field 460 are displayed. . In the setting display area 420, a plurality of tabs are displayed. The plurality of tabs include a focus position adjustment tab 470 and a contrast correction setting tab 480. The user can give various instructions to the CPU 210 by operating a GUI (Graphical User Interface) displayed on the display unit 400 using the operation unit 250 of the PC 200 of FIG.

  In the filter selection field 430, a plurality of (three in this example) filter selection buttons 431, 432, 433, and 434 are displayed. The three filter selection buttons 431 to 433 correspond to the three filter cubes 151, respectively, and the filter selection button 434 corresponds to the opening of the filter turret 152.

  Any one of the filter selection buttons 431 to 434 is selected by the user. The CPU 210 drives the filter turret driving unit so that the filter cube 151 or the opening corresponding to the selected filter selection button is positioned on the optical axis of the light receiving unit 120.

  In the measurement condition setting field 440, a pattern selection field 441, a pattern setting bar 442, an exposure time setting bar 443, an exposure time setting button 444, an auto button 445, and a detailed setting button 446 are displayed. From the pattern selection column 441, one of rectangular wave-shaped measurement light, one-dimensional sinusoidal measurement light, dot-shaped measurement light, two-dimensional sinusoidal measurement light, and uniform measurement light is selected by the user. The CPU 210 controls the light modulation element 112 so that the selected measurement light is emitted from the pattern applying unit 110.

  The pattern setting bar 442 has a slider that can move in the horizontal direction. When rectangular wave measurement light or dot measurement light is selected, the slider of the pattern setting bar 442 is moved to set the spatial period Ts of the pattern, the width W1 of the bright portion, and the movement amount of the pattern phase. The

  When the one-dimensional sinusoidal measurement light or the two-dimensional sinusoidal measurement light is selected, the pattern space bar Ts and the phase movement amount are set by moving the slider of the pattern setting bar 442. The CPU 210 controls the light modulation element 112 so as to have the spatial period Ts of the pattern in which the measurement light is set and the width W1 of the bright portion. Further, the CPU 210 controls the light modulation element 112 so that the phase of the pattern moves by the set movement amount.

  The exposure time setting bar 443 has a slider that can move in the horizontal direction. The exposure time of the light receiving unit 120 is set by moving the slider of the exposure time setting bar 443 or operating the exposure time setting button 444. Further, when the auto button 445 is operated, the exposure time of the light receiving unit 120 is automatically set appropriately. The CPU 210 controls the light receiving unit 120 so that the exposure time of the light receiving unit 120 becomes the set exposure time.

  The user can set the measurement conditions in more detail by operating the detailed setting button 446. Details will be described later.

  In the accompanying function setting field 460, an automatic contrast correction check box 461, a contrast correction amount display field 462, and an all-focus button 463 are displayed. When the automatic contrast correction check box 461 is designated, the CPU 210 multiplies the value of each pixel data of all sectioning image data or epi-illumination normal image data by a certain contrast correction amount. As a result, the contrast of the sectioning image or the reflected normal image is corrected.

  Further, the CPU 210 displays the contrast correction amount in the contrast correction amount display field 462. The user can recognize the brightness of the sectioning image before contrast correction by looking at the contrast correction amount displayed in the contrast correction amount display field 462. Thereby, it is possible to recognize that the received light signal is saturated and the degree of saturation. The omnifocal button 463 is operated, for example, when generating omnifocal image data. Details will be described later.

  One of the tabs is selected from the plurality of tabs. As shown in FIG. 17, when the focus position adjustment tab 470 is selected, an objective lens selection field 471, a focus position adjustment field 472, and a stage position adjustment field 473 are displayed in the setting display area 420.

  In the objective lens selection field 471, a plurality (six in this example) of objective lens selection buttons 471a, 471b, 471c, 471d, 471e, 471f are displayed. The six objective lens selection buttons 471a to 471f correspond to the six objective lenses 161, respectively.

  Any of the objective lens selection buttons 471a to 471f is selected. The CPU 210 drives the lens turret driving unit so that the objective lenses 161 corresponding to the selected objective lens selection buttons 471 a to 471 f are positioned on the optical axis of the light receiving unit 120.

  In the focus position adjustment field 472, a focus position adjustment bar 472a, an initial distance button 472b, and an autofocus button 472c are displayed. The focal position adjustment bar 472a has a slider that can move in the vertical direction. The position of the slider of the focal position adjustment bar 472a corresponds to the distance between the measuring object S and the objective lens 161.

  By moving the slider of the focal position adjustment bar 472a, the distance between the measuring object S and the objective lens 161 is adjusted. The CPU 210 controls the focal position adjustment mechanism 163 so that the distance between the measurement object S and the objective lens 161 is the distance adjusted by the slider.

  When the initial distance button 472b is operated, the CPU 210 controls the focal position adjustment mechanism 163 so that the distance between the measurement object S and the objective lens 161 becomes a distance set in advance as an initial condition. When the autofocus button 472c is operated, the CPU 210 controls the focus position adjustment mechanism 163 so that the objective lens 161 is focused on the measurement object S.

  In the stage position adjustment field 473, stage movement buttons 473a, 473b, 473c, 473d and an initial position button 473e are displayed. When the stage moving buttons 473a and 473b are operated, the CPU 210 controls the stage driving unit so that the stage 140 moves in one direction and the opposite direction in the X direction.

  When the stage movement buttons 473c and 473d are operated, the CPU 210 controls the stage driving unit so that the stage 140 moves in one direction and the opposite direction in the Y direction. When the initial position button 473e is operated, the CPU 210 controls the stage driving unit so that the position of the measurement object S moves to a position set in advance as an initial condition.

  As shown in FIG. 18, when the contrast correction setting tab 480 is selected, a histogram display field 481 is displayed in the setting display area 420. The CPU 210 causes the histogram display column 481 to display a histogram image indicating the relationship between the values of the plurality of pixel data of the sectioning image data before the contrast correction is performed and the number of pixels having each value. That is, the histogram image displayed in the histogram display field 481 shows the relationship between the pixel data value (luminance value) and the number of pixels in the brightness image.

  The user can recognize the relationship between the values of the plurality of pixel data of the sectioning image data and the number of pixels having each value by looking at the histogram displayed in the histogram display field 481. Thereby, the user can recognize that the received light signal is saturated and the degree of saturation.

  When the contrast correction setting tab 480 is selected, a plurality of contrast correction amount setting bars, a contrast correction amount input field, and a gamma correction value input field (not shown) are displayed in the setting display area 420. Each contrast correction amount setting bar has a slider that can move in the horizontal direction.

  When the automatic contrast correction check box 461 is not designated, the contrast correction amount is arbitrarily set by moving the slider of each contrast correction amount setting bar or by entering a numerical value in the contrast correction amount input field. When a numerical value is input in the gamma correction value input field, the CPU 210 performs gamma correction on the contrast correction amount based on the input numerical value. The CPU 210 corrects the contrast of the sectioning image by multiplying the contrast of all pixels by the contrast correction amount set or gamma corrected.

  When the detailed setting button 446 is operated, a measurement condition detailed setting window is displayed in the setting display area 420. FIG. 25 is a diagram illustrating an example of a measurement condition detail setting window. As shown in FIG. 25, in the measurement condition detail setting window 4460, a rectangular wave measurement light check box 4461, a dot measurement light check box 4462 and a spatial period setting field 4463 are displayed. The measurement condition detail setting window 4460 displays a bright part width setting field 4464, a phase shift amount setting field 4465, a binning number selection field 4466, and a gain selection field 4467.

  In the example of FIG. 25, the measurement condition detail setting window 4460 also displays an exposure time setting bar 443, an exposure time setting button 444, and an auto button 445 similar to the example of FIG. The functions of the exposure time setting bar 443, the exposure time setting button 444, and the auto button 445 in FIG. 25 are the same as the functions of the exposure time setting bar 443, the exposure time setting button 444, and the auto button 445 in FIG.

  One of the rectangular wave measurement light check box 4461 and the dot measurement light check box 4462 is designated. When the rectangular wave measuring light check box 4461 is designated, the rectangular wave measuring light is emitted from the pattern applying unit 110. On the other hand, when the dot measurement light check box 4462 is designated, the dot measurement light is emitted from the pattern applying unit 110. A spatial period Ts is set by inputting a numerical value in the spatial period setting field 4463.

  By operating the bright part width setting field 4464, the width W1 of the bright part of the pattern is set. Here, the user selects either the auto mode in which the width W1 of the bright portion of the pattern is automatically set appropriately or the manual mode in which the width W1 of the bright portion of the pattern is arbitrarily set. When the manual mode is selected, a numerical value corresponding to the width W1 of the bright portion of the pattern is input to the bright portion width setting field 4464.

  By operating the phase movement amount setting column 4465, the movement amount of the phase of the pattern is set. Here, the user selects either the auto mode in which the movement amount of the pattern phase is automatically set appropriately or the manual mode in which the movement amount of the pattern phase is arbitrarily set. When the manual mode is selected, a numerical value corresponding to the movement amount of the pattern phase is input to the phase movement amount setting field 4465.

  From the binning number selection column 4466, the number of binning in the pattern image data or the epi-illumination normal image data is selected by the user. The CPU 210 generates pattern image data or epi-illumination normal image data with the selected number of binning. Further, the gain value of the light receiving unit 120 is selected by the user from the gain selection field 4467. The CPU 210 controls the light receiving unit 120 so that the gain of the light receiving unit 120 becomes the selected gain value.

  The measurement condition detail setting window 4460 may be provided with an intensity setting field for setting the maximum intensity Imax and the minimum intensity Imin of the measurement light. In this case, the user can set the maximum intensity Imax and the minimum intensity Imin of the measurement light.

  The measurement condition detail setting window 4460 may be provided with an ROI setting field for setting an ROI. In this case, for example, the user can set the ROI by operating the operation unit 250 on the main window 411 and specifying the ROI in the state where the ROI setting field is operated.

(6) Determination of In-Focus Pixel Data (a) Determination Method By irradiating the measurement object S with the measurement light while changing the relative distance between the measurement object S and the objective lens 161, a plurality of focus positions are obtained. A plurality of sectioning image data or normal image data is generated. Out-of-focus pixel data among the generated plurality of sectioning image data or normal image data is determined for each pixel by the focus detection unit 214 of the CPU 210.

  Based on the determination result of the in-focus pixel data, omnifocal image data or data indicating the three-dimensional shape of the measurement object S (hereinafter referred to as three-dimensional shape data) can be generated. Further, autofocus can be executed based on the determination result of the focused pixel data. In the present embodiment, different focused pixel data determination methods are used for sectioning observation and normal observation.

  FIG. 26 is a diagram for explaining a determination method of in-focus pixel data in sectioning observation. The horizontal axis in FIG. 26 indicates the focal position (position in the Z direction) of the pattern measurement light, and the vertical axis indicates pixel data.

  In sectioning observation, it is determined that the pixel data having the maximum value among the plurality of pixel data for the same pixel in the plurality of sectioning image data is the focused pixel data. In the example of the pixel in FIG. 26, the pixel data generated at the position Pa has the maximum value Dmax. Therefore, it is determined that the pixel data generated at the position Pa is in-focus pixel data.

  In particular, when rectangular wave-shaped measurement light or dot-shaped measurement light is used, pixel data having a maximum value can be easily determined as compared with the case of using one-dimensional sine wave-shaped measurement light or two-dimensional sine wave-shaped measurement light. it can. Therefore, it is possible to easily determine the focused pixel data for the pixel data of the measuring object S such as a glass substrate that is transparent and has no pattern.

  FIG. 27 is a diagram for explaining a determination method of in-focus pixel data in normal observation. The horizontal axis in FIG. 26 indicates the focal position (position in the Z direction) of the pattern measurement light, and the vertical axis indicates the local contrast. Here, the local contrast is, for example, the difference between the value of arbitrary pixel data and the value of other pixel data adjacent to the pixel data. The local contrast may be a dispersion between a value of arbitrary pixel data and a value of other pixel data adjacent to the pixel data.

  In normal observation, it is determined that the pixel data having the maximum local contrast among the plurality of pieces of pixel data for the same pixel data in the plurality of pieces of normal image data is the focused pixel data. In the pixel example of FIG. 27, the pixel data generated at the position Pb has the maximum local contrast Cmax. Therefore, it is determined that the pixel data generated at the position Pa is in-focus pixel data. In normal observation, focused pixel data may be determined based on a change in local contrast at the edge portion.

  In the present embodiment, the CPU 210 automatically sets an appropriate in-focus pixel data determination method according to the observation method, but is not limited to this. The CPU 210 may display a setting screen for setting a determination method for focused pixel data on the display unit 400.

(B) Composition of Image Data When generating omnifocal image data, the user operates the omnifocal button 463 in FIGS. 17 and 18 using the operation unit 250 in FIG. As a result, an omnifocal image creation window is displayed in the setting display area 420. FIG. 28 is a diagram illustrating an example of an omnifocal image creation window. As shown in FIG. 28, an omnifocal image creation window 4630 displays a start button 4631 and an image save button 4632.

  The user can instruct the CPU 210 to perform omnifocal image data generation processing for generating omnifocal image data by operating the start button 4631. 29 and 30 are flowcharts showing the omnifocal image data generation processing. Hereinafter, the omnifocal image data generation process will be described with reference to FIGS. 2 and 28 to 30.

  CPU210 determines whether the start button 4631 was operated (step S11). If the start button 4631 is not operated in step S1, the CPU 210 waits until the start button 4631 is operated.

  When the start button 4631 is operated in step S1, the CPU 210 controls the light projecting unit 320 and the pattern providing unit 110 in FIG. 1 so that the measurement object S is irradiated with the measurement light (step S12). Further, the CPU 210 generates sectioning image data or normal image data based on the pixel data given from the control board 170 of FIG. 1 (step S13). The generated sectioning image data or normal image data is stored in the RAM 230 in FIG.

  Here, the CPU 210 extracts pixel data obtained in a state where the focus is closer to a part of the measurement object S from pixel data of one or a plurality of sectioning image data or normal image data generated up to the present time. (Step S14).

  In sectioning observation, the extracted pixel data is pixel data having a maximum value among a plurality of pixel data for the same pixel in one or a plurality of sectioning image data generated up to the present time. In normal observation, the extracted pixel data is pixel data having the maximum local contrast among a plurality of pixel data for the same pixel in one or a plurality of normal image data generated up to the present time.

  The CPU 210 generates image data by combining the extracted pixel data (step S15). The generated image data is stored in the RAM 230. At the time when sectioning image data or normal image data is first generated, all pixel data of the sectioning image data or normal image data is obtained in a state where the focus is closer to a part of the measuring object S. The data is determined and extracted. Therefore, image data generated by combining a plurality of pixel data is the same as the sectioning image data or normal image data.

  Next, the CPU 210 determines whether or not omnifocal image data has been generated (step S16). The CPU 210 determines that the omnifocal image data has been generated when the movement of the focal position is completed within a predetermined range.

  In the sectioning observation, if the CPU 210 determines that all the pixel data having the maximum value is extracted before the movement of the focal position within the predetermined range is completed, the omnifocal image data is generated. You may judge. When the CPU 210 determines that all pixel data having the maximum local contrast is extracted before the movement of the focal position within a predetermined range is completed in normal observation, the all-focus image data is stored. You may determine with having produced | generated.

  In step S16, when the omnifocal image data is not generated, the CPU 210 controls the focal position adjustment mechanism 163 so that the focal position of the objective lens 161 moves by a certain distance (step S17). Thereafter, the CPU 210 returns to the process of step S12. Thereafter, the CPU 210 repeats steps S12 to S17. As a result, the focus position of the objective lens 161 moves, and the image data stored in the RAM 230 is updated each time sectioning image data or normal image data at a new focus position is generated.

  By repeating the above procedure, pixel data obtained in a state where a part of the measuring object S is focused, that is, focused pixel data is extracted. When all the in-focus pixel data is extracted, the image data stored in the RAM 230 is image data obtained in a state where the entire measuring object S is in focus, that is, all-focus image data.

  When the omnifocal image data is generated in step S16, the CPU 210 stores the omnifocal image data stored in the RAM 230 in the storage device 240 of FIG. 1 (step S18). Thereby, the CPU 210 ends the omnifocal image data generation process. Also when the image save button 4632 is operated, the CPU 210 saves the image data stored in the RAM 230 in the storage device 240 and ends the omnifocal image data generation process.

  Further, the CPU 210 can generate a plurality of sectioning image data or normal image data while moving the stage 140 of FIG. 2 in the X direction or the Y direction. By combining the plurality of sectioning image data or normal image data generated in this way, the user can observe the entire sectioning image or normal image of the measuring object S having a larger dimension.

  Furthermore, the CPU 210 can generate sectioning image data or normal image data of the measurement object S at regular intervals (time-lapse imaging). By combining a plurality of sectioning image data or normal image data generated at different points in time, the user can observe the sectioning image or the normal image of the measuring object S that changes with time as a moving image.

  As described above, when measurement conditions are changed in the process of generating a plurality of sectioning image data or normal image data to be synthesized later, the brightness of a part of the synthesized sectioning image or normal image becomes unnatural. Or the uniformity of brightness decreases. In addition, the brightness of the moving image of the measuring object S flickers. Therefore, when generating a plurality of sectioning image data or normal image data to be synthesized later, it is preferable that the measurement conditions are not changed.

  Similarly, even when the contrast correction amount differs between a plurality of sectioning image data to be combined later, the brightness of a part of the combined sectioning image changes unnaturally or the uniformity of the brightness is descend. In addition, the brightness of the moving image of the measuring object S flickers.

  On the other hand, in the present embodiment, the sectioning image data before the contrast is corrected as described above and the metadata indicating the contrast correction amount are stored independently. Therefore, the contrast correction amount can be unified among the plurality of sectioning image data. As a result, it is possible to prevent unnatural changes in brightness, ununiformity in brightness, or flickering in brightness of the combined sectioning image.

  Here, the contrast correction amount may be set to a value at which the overexposed and underexposed portions of the sectioning image are minimized. In this case, the dynamic range of the sectioning image is expanded. Alternatively, the contrast correction amount may be an average value of the contrast correction amounts of the plurality of sectioning image data. In this case, the overall brightness of the sectioning image is optimized. These contrast correction amounts may be switchable according to the characteristics of the sectioning image data.

(C) Stereoscopic shape data and autofocus When the focal position is moved in the generation procedure of the omnifocal image data, the relative distance between the measurement object S and the objective lens 161 is stored in the RAM 230. The CPU 210 calculates the height of the measurement object S based on the relative distance between the part of the measurement object S corresponding to the extracted pixel data and the objective lens 161 stored in the RAM 230.

  The CPU 210 generates solid shape data by synthesizing the heights calculated for all the parts of the measuring object S. The three-dimensional shape data is stored in the RAM 230. When the image save button 4632 is operated, the CPU 210 saves the solid shape data together with the image data stored in the RAM 230 in the storage device 240.

  When the autofocus button 472c in FIGS. 17 and 18 is operated, autofocus is performed. Specifically, the CPU 210 calculates pixel data of the central portion of a plurality of pattern images at a plurality of different focal positions. Or the local contrast of the pixel of the center part of several normal images in several different focus positions is calculated.

  Thereby, the focused pixel data of the central portion of the pattern image or the normal image is determined. Further, the relative distance between the measurement object S and the objective lens 161 when the measurement object S corresponding to the central portion of the pattern image or the normal image is focused is calculated. The CPU 210 controls the focal position adjustment mechanism 163 so that the relative distance between the measurement object S and the objective lens 161 becomes the calculated distance.

  When the ROI is set, the CPU 210 may determine the focused pixel data in the ROI instead of the central portion of the pattern image or the normal image. In this case, autofocus can be performed on the portion of the measuring object S corresponding to the ROI.

  The sectioning observation and the normal observation differ in the determination method of the focused pixel data. Therefore, the sectioning observation and the normal observation have different focal positions when autofocus is performed. The user can determine the optimum focus position by switching between autofocus during sectioning observation and autofocus during normal observation.

  Further, when the spatial period Ts of the rectangular wave measurement light or the dot measurement light is long, the determination of the focused pixel data is completed in a short time. Therefore, autofocus can be performed at high speed by setting the spatial period Ts to be long. On the other hand, when the spatial period Ts of the rectangular wave measurement light or the dot measurement light is short, the pixel data having the maximum value is determined with high accuracy. Therefore, the accuracy of autofocus can be improved by setting the spatial period Ts short.

  When autofocusing is performed, the measurement light irradiation position on the measurement object S is switched each time the measurement light is emitted. Thereby, it is prevented that only the fluorescent reagent in the specific part of the measuring object S fades. Here, if the irradiation position of the measurement light on the measuring object S is switched within the exposure time of the light receiving unit 120, the level of the light reception signal output from the light receiving unit 120 becomes inaccurate, and the accuracy of autofocus is reduced. To do. Therefore, it is preferable that the irradiation position of the measurement light on the measurement object S is switched after the exposure time of the light receiving unit 120 has elapsed.

(7) Effect In the imaging apparatus 500 according to the present embodiment, measurement light having a predetermined pattern is generated by the pattern applying unit 110 from the light emitted by the light projecting unit 320. The measurement light generated by the pattern applying unit 110 is irradiated onto the measurement object S by the filter unit 150. Further, the spatial phase of the generated pattern is sequentially moved on the measuring object S by a predetermined amount by the pattern applying unit 110. Light from the measuring object S is received by the light receiving unit 120, and a light reception signal indicating the amount of light received is output.

  Sectioning image data of the measurement object S is generated based on a plurality of pattern image data generated at a plurality of phases of the pattern based on the light reception signal output from the light receiving unit 120. Specifically, the sectioning image is obtained by removing the pixel data value when the dark portion is irradiated on the measuring object S from the pixel data value when the measuring object S is irradiated with the bright portion of the measuring light. Data is generated.

  For each piece of pixel data in the plurality of pattern image data, the pixel value when the measurement object S is irradiated with the dark portion of the measurement light corresponds to a blur component of the image due to stray light. Therefore, the blur component due to stray light is removed from each pixel of the sectioning image data. Thereby, a clear sectioning image of the measuring object S can be obtained.

  On the other hand, when the blur component due to stray light is removed from each pixel of the sectioning image data, the sectioning image becomes darker than when the blur component due to stray light is not removed from each pixel data. Therefore, even if the user looks at the sectioning image, the user cannot recognize whether or not the light reception signal is saturated.

  Even in such a case, it is determined whether or not the light reception signal output by the light receiving unit 120 is saturated for at least one of the generated plurality of pattern image data. When the light reception signal is saturated, the user is notified that the light reception signal is saturated. Thereby, even when a bright part does not exist in the sectioning image, the user can easily recognize that the light reception signal is saturated.

(8) Correspondence between each component of claim and each part of embodiment The following describes an example of the correspondence between each component of the claim and each part of the embodiment. It is not limited.

  In the above embodiment, the light projecting unit 320 is an example of a light projecting unit, the pattern providing unit 110 is an example of a measurement light generating unit, the measurement object S is an example of a measurement object, and the filter unit 150. Is an example of an optical system. The light receiving unit 120 is an example of a light receiving unit, the image data generation unit 211 is an example of an image data generation unit, the control unit 213 is an example of a control unit and a processing device, and the display unit 400 is an example of a display unit. The microscope imaging apparatus 500 is an example of a microscope imaging apparatus.

  As each constituent element in the claims, various other elements having configurations or functions described in the claims can be used.

  The present invention can be effectively used for various microscope imaging apparatuses, microscope imaging methods, and microscope imaging programs.

DESCRIPTION OF SYMBOLS 100 Measuring part 110 Pattern provision part 111 Light output part 112 Light modulation element 113 Mirror 120 Light receiving part 121 Camera 122 Color filter 123 Imaging lens 130 Transmitted light supply part 131 Transmitted light source 132 Transmitting optical system 140 Stage 150 Filter unit 151 Filter cube 151a Frame 151b Excitation filter 151c Dichroic mirror 151d Absorption filter 152 Filter turret 152a Filter cube attachment part 160 Lens unit 161 Objective lens 162 Lens turret 162a Objective lens attachment part 163 Focus position adjustment mechanism 170 Control board 200 PC
210 CPU
211 Image data generation unit 212 Pattern generation unit 213 Control unit 214 Focus detection unit 220 ROM
230 RAM
240 Storage Device 250 Operation Unit 300 Measurement Light Supply Unit 310 Power Supply Device 320 Light Projection Unit 321 Measurement Light Source 322 Dimming Mechanism 323 Shading Mechanism 330 Light Guide Member 400 Display Unit 410 Image Display Area 411 Main Window 412 Subwindow 420 Setting Display Area 430 Filter selection field 431-434 Filter selection button 440 Measurement condition setting field 441 Pattern selection field 442 Pattern setting bar 443 Exposure time setting bar 444 Exposure time setting button 445 Auto button 446 Detail setting button 450 Sub window display check box 460 Associated function setting field 461 Automatic contrast correction check box 462 Contrast correction amount display field 463 All focus button 470 Focus position adjustment tab 471 Objective lens selection field 472 Focus position adjustment Field 472a Focus position adjustment bar 472b Initial distance button 472c Autofocus button 473 Stage position adjustment field 473a to 473d Stage movement button 480 Contrast correction setting tab 481 Histogram display field 500 Microscope imaging device 4460 Measurement condition detailed setting window 4461 Rectangle wave measurement light check Box 4462 Dot-shaped measuring light check box 4463 Spatial period setting field 4464 Bright part width setting field 4465 Phase shift amount setting field 4466 Binning number selection field 4467 Gain selection field 4630 All-focus image creation window 4630 Start button 4632 Image save button S Measurement object object

Claims (14)

A light projecting unit that emits light;
A measurement light generator configured to generate measurement light having a predetermined pattern from the light emitted by the light projecting unit;
An optical system that irradiates the measurement object with the measurement light generated by the measurement light generation unit; a light reception unit that receives light from the measurement object and outputs a light reception signal indicating the amount of received light;
An image data generation unit that generates image data based on a light reception signal output from the light reception unit;
Based on a plurality of image data generated at a plurality of phases of the pattern by controlling the measurement light generation unit so as to sequentially move a spatial phase of the generated pattern by a predetermined amount on the measurement object. A control unit that controls the image data generation unit to generate sectioning image data indicating an image of the measurement object;
A display unit configured to display an image based on the sectioning image data generated by the image data generation unit,
The control unit determines whether or not a light reception signal output from the light receiving unit is saturated for at least one of the plurality of image data generated by the image data generation unit, and the light reception signal is saturated. In some cases, the microscope imaging apparatus notifies the user that the received light signal is saturated. The image data generation unit, for each pixel of the plurality of image data, the value of the pixel when the measurement object is irradiated with a first portion having a higher intensity than a certain intensity in the measurement light pattern The sectioning image data is generated by removing a value of a pixel when a second portion having an intensity equal to or lower than a certain intensity in the measurement light pattern is irradiated on the measurement object. Microscope imaging device. The control unit controls the image data generation unit to generate brightness image data indicating a brightness distribution state of the image as a brightness image based on the plurality of image data generated by the image data generation unit;
The microscope imaging apparatus according to claim 1, wherein the display unit is configured to be able to display a brightness image based on the brightness image data. The microscope imaging apparatus according to claim 3, wherein the control unit determines whether or not a light reception signal output from the light receiving unit is saturated based on brightness image data generated by the image data generation unit. The control unit calculates information on the brightness of the image based on the brightness image data generated by the image data generation unit,
The microscope imaging apparatus according to claim 3, wherein the display unit is configured to be able to display information related to brightness calculated by the control unit. The microscope imaging apparatus according to claim 5, wherein the information on the brightness includes a histogram indicating a relationship between a plurality of pixel values of a plurality of image data and a number of pixels having each value. The control unit corrects the brightness of the image by amplifying the values of the plurality of pixels of the sectioning image data,
The microscope imaging apparatus according to claim 5, wherein the information on brightness includes amplification factors of a plurality of pixel values. When the light receiving signal output from the light receiving unit is saturated, the control unit is configured to obtain a pixel value corresponding to a portion where the light receiving signal is saturated in the sectioning image data generated by the image data generating unit. The microscope imaging apparatus according to claim 7, wherein A is amplified larger than a value of a pixel corresponding to a portion where the received light signal is not saturated. When the light receiving signal output from the light receiving unit is saturated, the control unit displays a pixel corresponding to the saturated light receiving signal portion in the image displayed on the display unit. The microscope imaging apparatus according to claim 1, wherein the display unit is controlled to do so. The control unit controls the measurement light generation unit so that the measurement object is irradiated with measurement light when the generation conditions of the plurality of image data by the image data generation unit are changed, and the generation unit When the control unit is controlled to update the sectioning image data based on the plurality of image data generated by the method, and the generation conditions of the plurality of image data by the generation unit are not changed, the measurement light to the measurement object The microscope imaging apparatus according to any one of claims 1 to 9, wherein the measurement light generation unit is controlled so as to stop irradiation. When the generation condition of the plurality of image data by the image data generation unit is changed, the control unit is irradiated with measurement light before the generation condition of the plurality of image data by the image data generation unit is changed. The microscope imaging apparatus according to claim 10, wherein the measurement light generation unit is controlled so that measurement light is irradiated to a part different from the part of the object. The control unit includes the light projecting unit and the light receiving unit so that a level of a light reception signal output from the light receiving unit is changed between a first level and a second level lower than the first level. A plurality of image data in a state where the level of the light reception signal is the first level is generated as the plurality of first image data, and the level of the light reception signal is the second level. A plurality of image data in a certain state is generated as a plurality of second image data, and the generated first image data and the plurality of second image data are expanded so that the dynamic range of the light receiving unit is expanded. A plurality of third image data is generated by combining, and the image data generation unit is controlled to generate sectioning image data based on the plurality of third image data That, the shape measuring apparatus according to any one of claims 1 to 11. Emitting light by the light projecting unit;
Generating measurement light having a predetermined pattern from the light emitted from the light projecting unit by the measurement light generating unit;
Irradiating a measurement object by the optical system with the measurement light generated by the measurement light generation unit;
Sequentially moving the spatial phase of the generated pattern on the measurement object by a predetermined amount by the measurement light generation unit;
Receiving light from the measurement object by a light receiving unit and outputting a light reception signal indicating the amount of light received;
Generating sectioning image data indicating an image of the measurement object based on a plurality of image data generated at a plurality of phases of the pattern based on a light reception signal output from the light receiving unit;
Displaying an image based on the generated sectioning image data on a display unit;
Determining whether the light reception signal output by the light receiving unit is saturated for at least one of the plurality of generated image data; and
And a step of notifying the user that the light reception signal is saturated when the light reception signal is saturated. A microscope imaging program executable by a processing device,
A process of emitting light by the light projecting unit;
A process of generating measurement light having a predetermined pattern from the light emitted by the light projecting unit by the measurement light generating unit;
A process of irradiating the measurement object by the optical system with the measurement light generated by the measurement light generation unit;
A process of sequentially moving the spatial phase of the generated pattern on the measurement object by a predetermined amount by the measurement light generation unit;
A process of receiving light from the measurement object by a light receiving unit and outputting a light reception signal indicating the amount of received light;
Processing for generating sectioning image data indicating an image of the measurement object based on a plurality of image data generated at a plurality of phases of the pattern based on a light reception signal output from the light receiving unit;
Processing to display an image based on the generated sectioning image data on the display unit;
A process of determining whether or not a light reception signal output by the light receiving unit is saturated for at least one of the plurality of generated image data;
A process of notifying the user that the received light signal is saturated when the received light signal is saturated,
A microscope imaging program to be executed by the processing apparatus.

Images