WO2023053540A1 - 撮像方法、焦点位置調整方法および顕微鏡システム - Google Patents

撮像方法、焦点位置調整方法および顕微鏡システム Download PDF

Info

Publication number
WO2023053540A1
WO2023053540A1 PCT/JP2022/015946 JP2022015946W WO2023053540A1 WO 2023053540 A1 WO2023053540 A1 WO 2023053540A1 JP 2022015946 W JP2022015946 W JP 2022015946W WO 2023053540 A1 WO2023053540 A1 WO 2023053540A1
Authority
WO
WIPO (PCT)
Prior art keywords
imaging
sample
image
relative position
candidate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/015946
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
耕平 村上
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sysmex Corp
Original Assignee
Sysmex Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sysmex Corp filed Critical Sysmex Corp
Priority to JP2023551052A priority Critical patent/JPWO2023053540A1/ja
Publication of WO2023053540A1 publication Critical patent/WO2023053540A1/ja
Priority to US18/621,162 priority patent/US20240241360A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G02B21/362—Mechanical details, e.g. mountings for the camera or image sensor, housings
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G02B21/361—Optical details, e.g. image relay to the camera or image sensor
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G02B21/365—Control or image processing arrangements for digital or video microscopes
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G02B21/365—Control or image processing arrangements for digital or video microscopes
    • G02B21/367—Control or image processing arrangements for digital or video microscopes providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G02B21/368—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements details of associated display arrangements, e.g. mounting of LCD monitor
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
    • G02B7/28—Systems for automatic generation of focusing signals
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
    • G02B7/28—Systems for automatic generation of focusing signals
    • G02B7/36—Systems for automatic generation of focusing signals using image sharpness techniques, e.g. image processing techniques for generating autofocus signals
    • C—CHEMISTRY; METALLURGY
    • C12—BIOCHEMISTRY; BEER; SPIRITS; WINE; VINEGAR; MICROBIOLOGY; ENZYMOLOGY; MUTATION OR GENETIC ENGINEERING
    • C12M—APPARATUS FOR ENZYMOLOGY OR MICROBIOLOGY; APPARATUS FOR CULTURING MICROORGANISMS FOR PRODUCING BIOMASS, FOR GROWING CELLS OR FOR OBTAINING FERMENTATION OR METABOLIC PRODUCTS, i.e. BIOREACTORS OR FERMENTERS
    • C12M1/00—Apparatus for enzymology or microbiology
    • C—CHEMISTRY; METALLURGY
    • C12—BIOCHEMISTRY; BEER; SPIRITS; WINE; VINEGAR; MICROBIOLOGY; ENZYMOLOGY; MUTATION OR GENETIC ENGINEERING
    • C12M—APPARATUS FOR ENZYMOLOGY OR MICROBIOLOGY; APPARATUS FOR CULTURING MICROORGANISMS FOR PRODUCING BIOMASS, FOR GROWING CELLS OR FOR OBTAINING FERMENTATION OR METABOLIC PRODUCTS, i.e. BIOREACTORS OR FERMENTERS
    • C12M1/00—Apparatus for enzymology or microbiology
    • C12M1/34—Measuring or testing with condition measuring or sensing means, e.g. colony counters

Definitions

  • the present invention relates to an imaging method for imaging a sample in a microscope system, a focal position adjusting method for adjusting the focal position in the microscope system, and a microscope system.
  • Patent Literature 1 describes an example of a focus position adjusting method. Specifically, in a cell observation device using a holographic microscope, a large number of phase images with different focal positions are created in advance. When the observer moves the knob of the slider arranged on the displayed image, the phase image of the focus position corresponding to the position of the knob is displayed in the image display frame on the displayed image. While moving the knob of the slider, the observer confirms whether or not the phase image of the image display frame is formed at each position of the knob. When the observer confirms that the phase image in the image display frame is in the focused state by this operation, the observer operates the enter button on the displayed image to fix the focal position.
  • the present invention relates to an imaging method for imaging a sample in a microscope system (1).
  • the imaging method of the present invention determines a plurality of candidate relative positions based on a plurality of captured images obtained by imaging the sample while varying the relative positions of the sample and the focal point of the light receiving optical system (140).
  • a step (S4), a step (S6) of determining a relative position for performing imaging from a plurality of candidate relative positions, and a step (S10) of performing imaging of the sample at the determined relative position. include.
  • a plurality of candidate relative positions are automatically determined based on a plurality of captured images obtained while varying the relative positions of the sample and the focal point of the light receiving optical system.
  • the user can adjust the focal position simply by selecting a relative position that the user considers appropriate from among a plurality of automatically determined candidate relative positions. Since there is no need to search for an appropriate relative position from a large number of captured images, it is possible to adjust the focal position more easily than before.
  • the present invention relates to a focus position adjustment method for adjusting the relative position between the sample and the focus of the light receiving optical system (140) in the microscope system (1).
  • a plurality of candidate relative positions are determined based on a plurality of captured images obtained by imaging the sample while changing the relative position between the sample and the focus of the light receiving optical system (140).
  • the user can adjust the focal position simply by selecting a relative position that the user considers appropriate from a plurality of automatically determined candidate relative positions. Adjustment is possible. Since there is no need to search for an appropriate relative position from a large number of captured images, it is possible to adjust the focal position more easily than before. As a result, subsequent imaging of the sample can be performed with the sample set at a desired relative position.
  • the present invention relates to a microscope system (1) for imaging a sample.
  • a microscope system (1) of the present invention comprises a sample placement section (12) for placing a sample, an imaging element (138) for capturing an image of the sample via a light receiving optical system (140), and a sample placement section (12).
  • a drive section (129b) for changing the relative position of the focal point of the light receiving optical system (140) with respect to the light receiving optical system (140), and a control section (211).
  • the control unit (211) determines a plurality of candidate relative positions based on a plurality of captured images obtained by imaging the sample with the imaging device (138) while changing the relative position, and determines a plurality of candidate relative positions.
  • a relative position for performing imaging is determined from the relative position, and an imaging device (138) performs imaging of the sample at the relative position for performing imaging.
  • the user can adjust the focal position simply by selecting a relative position that the user considers appropriate from a plurality of automatically determined candidate relative positions. It becomes possible. Since there is no need to search for an appropriate relative position from a large number of captured images, it is possible to adjust the focal position more easily than before.
  • FIG. 1A is a perspective view showing the configuration of a microscope system according to an embodiment
  • FIG. FIG. 1B is a perspective view showing the configuration of the microscope device according to the embodiment
  • FIG. 2 is a diagram schematically showing the internal configuration of the microscope apparatus according to the embodiment
  • FIG. 3 is a block diagram showing the configuration of the microscope system according to the embodiment
  • 4 is a flowchart illustrating processing performed by a control unit of a control device in the microscope system according to the embodiment
  • FIG. FIG. 5 is a diagram illustrating the configuration of a screen displayed on the display unit according to the embodiment
  • FIG. 6 is a diagram illustrating the configuration of a screen displayed on the display unit according to the embodiment
  • FIG. 7 is a diagram illustrating the configuration of a screen displayed on the display unit according to the embodiment;
  • FIG. 8 is a flowchart detailing a process for determining candidate locations, according to an embodiment.
  • FIG. 9 is a schematic diagram for explaining acquisition of captured images, acquisition of indices, and determination of candidate positions according to the embodiment.
  • FIG. 10 is a diagram schematically showing the number of steps, captured images, indexes, and candidate flags stored in the storage unit of the control device according to the embodiment;
  • FIG. 11A is a diagram schematically illustrating a procedure for obtaining an index when using a root mean square according to the embodiment;
  • FIG. 11B is a diagram schematically illustrating a procedure for obtaining an index when standard deviation is used according to the embodiment;
  • FIG. 12A is a schematic diagram of a graph when three candidate positions are determined without dividing the search range into sections, according to a modification.
  • FIG. 12B is a schematic diagram of a graph when the search range is divided into three sections and one candidate position is determined for each section, according to the modification.
  • FIG. 13 is a flowchart detailing a process for displaying candidate locations, according to an embodiment.
  • FIG. 14 is a flowchart illustrating details of a process for displaying an enlarged image, according to an embodiment.
  • FIG. 15 is a schematic diagram for explaining a process of performing imaging of a sample and a process of acquiring a super-resolution image according to the embodiment;
  • FIG. 16A is a flow chart illustrating the process of accepting candidate location selections, according to a variation.
  • FIG. 16B is a flow chart illustrating the process of receiving candidate location selections, according to a variation.
  • the Z-axis direction is the height direction of the microscope system 1 .
  • the XY plane is a plane parallel to the horizontal plane.
  • the positive X-axis direction, positive Y-axis direction, and positive Z-axis direction are the leftward direction, the forward direction, and the upward direction, respectively.
  • FIG. 1A is a perspective view showing the configuration of the microscope system 1.
  • the microscope system 1 includes a microscope device 1a and a control device 1b.
  • the microscope device 1a and the control device 1b are connected to each other by a wire so as to be able to transmit and receive signals to and from each other.
  • the microscope device 1a and the control device 1b may be connected wirelessly.
  • the microscope system 1 is a super-resolution microscope device for imaging a sample and creating and displaying a super-resolution image of the captured sample.
  • a sample is a biological sample taken from a subject (eg, subject).
  • Biological samples include, for example, proteins.
  • a super-resolution microscope is a microscope that observes an object using a microscopy method that reaches a resolution below the diffraction limit of light, and has a resolution of 200 nm or below, which is the limiting resolution of conventional fluorescence microscopes.
  • the microscope apparatus 1a is suitable for observing intracellular aggregated proteins with a size of about several tens of nanometers, abnormalities in cell organelles, and the like.
  • the microscope device 1a has a display unit 21 on the front surface, and the display unit 21 displays an image of the captured sample.
  • the control device 1b receives a user's instruction via the input unit 213 (see FIG. 3), and controls the microscope device 1a according to the user's instruction.
  • the control device 1b processes the image acquired by the microscope device 1a and causes the display unit 21 to display an image of the sample.
  • FIG. 1B is a perspective view showing the configuration of the microscope device 1a.
  • the microscope device 1a includes a base portion 10 and a moving portion 20.
  • a configuration for imaging a sample (see FIG. 2) is housed inside the base portion 10 .
  • a concave portion 11 is formed in the upper portion near the left end of the base portion 10 .
  • a sample placement portion 12 is arranged near the bottom surface of the recess 11 .
  • the sample placement section 12 is a stage for placing a slide glass on which a sample is placed.
  • An objective lens 127 is positioned below the sample placement section 12 .
  • the moving part 20 can move left and right between a state in which the top of the sample placement part 12 is closed as shown in FIG. 1A and a state in which the top of the sample placement part 12 is opened as shown in FIG. 1B. It is supported by the base portion 10 . As shown in FIG. 1B, the user slides the moving part 20 to the right to open the upper part of the sample placement part 12 and place the slide glass on which the sample is placed on the sample placement part 12 . Subsequently, the user slides the moving part 20 leftward to close the upper part of the sample setting part 12 as shown in FIG. 1A. When the top of the sample placement section 12 is closed by the moving section 20 , a later-described cover 22 provided inside the moving section 20 is positioned above the sample placement section 12 . Then, the user starts imaging processing by the microscope system 1 .
  • FIG. 2 is a diagram schematically showing the internal configuration of the microscope device 1a.
  • the microscope apparatus 1a includes a first illumination 110, mirrors 121 and 122, a filter 123, a beam expander 124, a condenser lens 125, a dichroic mirror 126, an objective lens 127, a second illumination 128, and a sample installation.
  • Unit 12 XY-axis drive unit 129a, Z-axis drive unit 129b, cover 22, filter 131, mirror 132, imaging lens 133, relay lens 134, mirrors 135 and 136, and relay lens 137 and an imaging device 138 .
  • a light receiving optical system 140 is composed of the objective lens 127 , the dichroic mirror 126 , the filter 131 , the mirror 132 , the imaging lens 133 , the relay lens 134 , the mirrors 135 and 136 , and the relay lens 137 .
  • the first illumination 110 includes light sources 111 and 112, collimator lenses 113 and 114, a mirror 115, a dichroic mirror 116, and a quarter wave plate 117.
  • the light source 111 emits light of a first wavelength
  • the light source 112 emits light of a second wavelength different from the first wavelength.
  • the light sources 111 and 112 are semiconductor laser light sources.
  • the light sources 111 and 112 may be mercury lamps, xenon lamps, LEDs, or the like.
  • Light from light sources 111, 112 is excitation light that causes fluorescence from fluorochromes bound to the sample.
  • the fluorescent dye preliminarily bound to the sample is a dye that alternates between a luminous state and a quenching state when irradiated with light of the first wavelength, and emits fluorescence when irradiated with light of the first wavelength in the luminous state.
  • the fluorescent dye that is preliminarily bound to the sample alternates between the luminescence state and the quenching state when irradiated with the light of the second wavelength, and the dye that emits fluorescence when the light of the second wavelength is irradiated in the luminescence state.
  • the fluorescent dye a dye is selected that emits fluorescence of a wavelength that passes through a dichroic mirror 116 and a filter 131, which will be described later.
  • Self-flickering refers to the repetition of luminous state and quenching state by the irradiation of excitation light. (Thermo Fisher Scientific) and the like are preferably used.
  • Either one of the light sources 111 and 112 is used for adjusting the focus position and acquiring the super-resolution image, which will be described later, depending on the fluorescent dye bound to the sample.
  • the collimator lenses 113 and 114 collimate the light emitted from the light sources 111 and 112, respectively.
  • Mirror 115 reflects light from light source 111 .
  • Dichroic mirror 116 transmits light from light source 111 and reflects light from light source 112 .
  • the quarter-wave plate 117 converts the linearly polarized light emitted from the light sources 111 and 112 into circularly polarized light. As a result, the light emitted from the light sources 111 and 112 can be evenly absorbed by the sample in any polarization direction.
  • Each part in the first illumination 110 is arranged so that the optical axes of the light from the light sources 111 and 112 emitted from the first illumination 110 are aligned with each other.
  • the mirrors 121 and 122 reflect the light emitted from the first illumination 110 to the filter 123 .
  • Filter 123 cuts unnecessary wavelengths of light reflected by mirror 122 .
  • the beam expander 124 enlarges the beam diameter of the light that has passed through the filter 123 and widens the irradiation area of the light on the slide glass placed on the sample placement section 12 . As a result, the intensity of the light irradiated onto the slide glass can be brought close to a uniform state.
  • the condenser lens 125 collects the light from the beam expander 124 so that the objective lens 127 irradiates the slide glass with substantially parallel light.
  • the dichroic mirror 126 reflects light emitted from the light sources 111 and 112 and condensed by the condensing lens 125 . In addition, the dichroic mirror 126 transmits fluorescence generated from fluorescent dyes bound to the sample and passed through the objective lens 127 . The objective lens 127 guides the light reflected by the dichroic mirror 126 to the sample on the slide glass placed on the sample placement section 12 .
  • the cover 22 is supported by a shaft 22a that extends in the Y-axis direction and that is installed on the moving part 20 (see FIG. 1B).
  • the cover 22 rotates around the shaft 22a when the moving part 20 moves in the X-axis direction.
  • the cover 22 stands up as shown by broken lines in FIGS. 1B and 2 in conjunction with the movement of the moving part 20 to the right (X-axis negative direction).
  • the cover 22 moves in conjunction with this, as indicated by the solid line in FIG. 90 degrees and the cover 22 is parallel to the horizontal plane.
  • a cover 22 covers the upper side of the sample placement section 12 .
  • a second illumination 128 is provided on the surface of the cover 22 facing the sample placement section 12 .
  • the second lighting 128 is an LED light source that emits white light, and has a planar light emitting area. Light from the second illumination 128 is used for capturing brightfield images.
  • the second illumination 128 is obliquely provided on the surface of the cover 22 . As a result, compared to the case where the second illumination 128 is provided parallel to the surface of the cover 22, it is possible to image the sample with enhanced contrast.
  • the structure of the cover 22 that rotates in conjunction with the moving part 20 is disclosed in US Patent Publication 2020-0103347, the disclosure of which is incorporated herein by reference.
  • the sample placement section 12 is supported in the XY plane by the XY-axis driving section 129a, and supported in the Z-axis direction by the Z-axis driving section 129b.
  • the XY-axis drive section 129a includes a stepping motor for moving the sample placement section 12 in the X-axis direction and a stepping motor for moving the sample placement section 12 in the Y-axis direction.
  • the Z-axis driving section 129b includes a stepping motor for moving the sample placement section 12 and the XY-axis driving section 129a in the Z-axis direction.
  • the relative position of the focal point of the light receiving optical system 140 with respect to the sample placement section 12 changes by driving the Z-axis driving section 129b and moving the sample placement section 12 up and down along the Z axis.
  • the relative position of the sample placement section 12 with respect to the objective lens 127 defines the relative position of the focal point of the light receiving optical system 140 with respect to the sample placement section 12 . That is, by driving the Z-axis driving section 129b, the relative position between the sample placement section 12 and the focal point of the light receiving optical system 140 changes, resulting in a plurality of relative positions.
  • the light receiving optical system 140 forms an image within a predetermined viewing angle range including that position on the imaging surface of the image sensor 138 .
  • the filter 131 cuts unnecessary wavelength light from the light transmitted through the dichroic mirror 126 .
  • Mirrors 132 , 135 , and 136 reflect the light that has passed through filter 131 and guide it to imaging element 138 .
  • the imaging lens 133 once forms an image of the light generated from the sample on the optical path with the relay lens 134 and guides the light to the relay lens 134 .
  • the relay lenses 134 and 137 image the light generated from the sample on the imaging surface of the imaging device 138 .
  • the imaging element 138 is, for example, a CCD image sensor or a CMOS image sensor.
  • the imaging device 138 captures an image of light incident on the imaging surface.
  • FIG. 3 is a block diagram showing the configuration of the microscope system 1.
  • the microscope apparatus 1a includes a control unit 201, a laser driving unit 202, an XY-axis driving unit 129a, a Z-axis driving unit 129b, an imaging device 138, a display unit 21, a moving unit driving unit 203, and an interface 204. , provided.
  • the control unit 201 includes a processor such as a CPU or FPGA, and a memory.
  • the control unit 201 controls each unit of the microscope apparatus 1a according to instructions from the control device 1b via the interface 204, and transmits captured images received from the imaging device 138 to the control device 1b.
  • a laser drive unit 202 drives the light sources 111 and 112 under the control of the control unit 201 .
  • the XY-axis drive unit 129a includes a stepping motor, and drives the stepping motor under the control of the control unit 201 to move the sample placement unit 12 within the XY plane.
  • the Z-axis driving section 129b includes a stepping motor, and drives the stepping motor under the control of the control section 201 to move the XY-axis driving section 129a and the sample placement section 12 in the Z-axis direction.
  • the moving part driving part 203 includes a motor, and moves the moving part 20 in the X-axis positive direction and the X-axis negative direction by driving the motor.
  • the imaging device 138 captures light incident on the imaging surface under the control of the control unit 201 and transmits the captured image to the control unit 201 .
  • the display unit 21 is, for example, a liquid crystal display or an organic EL display.
  • the display unit 21 displays various screens according to signals from the control device 1b.
  • the control device 1 b includes a control section 211 , a storage section 212 , an input section 213 and an interface 214 .
  • the control unit 211 is, for example, a CPU.
  • Storage unit 212 is, for example, a hard disk or an SSD.
  • the input unit 213 is, for example, a mouse and keyboard. The user operates the mouse of the input unit 213 to perform an operation such as clicking or double-clicking on the screen displayed on the display unit 21 to input an instruction to the control unit 211 .
  • the display unit 21 and the input unit 213 may be configured by a touch panel type display. In this case, the user taps or double-tap the display surface of the touch panel display instead of clicking or double-clicking.
  • the control unit 211 performs processing based on software stored in the storage unit 212, that is, computer programs and related files. Specifically, the control unit 211 transmits a control signal to the control unit 201 of the microscope apparatus 1a via the interface 214 to control each unit of the microscope apparatus 1a. Also, the control unit 211 receives a captured image from the control unit 201 of the microscope apparatus 1 a via the interface 214 and stores it in the storage unit 212 . Further, the control unit 211 causes the display unit 21 of the microscope device 1a to display a screen 300 (see FIGS. 5 to 7) to adjust the focal position based on the captured image received from the microscope device 1a. Further, the control unit 211 causes the microscope device 1a to perform imaging for generating a super-resolution image at the focal position adjusted via the screen 300, generates a super-resolution image based on the captured image, Displayed on the display unit 21 .
  • a fringe pattern is projected onto an object, and the fringe pattern is imaged while changing the relative position between the objective lens and the stage.
  • Software automatically searches for a position (focus position), and by moving the objective lens or stage to the identified focus position, automatic focus adjustment is performed on the subject.
  • the position can be automatically adjusted. In such a case, the user manually adjusts the focus position without adopting the automatically adjusted focus position. Such work is time consuming. Further, since deterioration of the fluorescent dye may progress due to irradiation with light, it is not preferable to expose the fluorescent dye for a long time in order to adjust the focal position before acquiring the super-resolution image.
  • the software does not automatically determine the focus position at one specific position, but rather identifies a plurality of candidate focus positions based on the index obtained from the image, and the user be presented as selectable.
  • the processing of the control unit 211 of this embodiment will be described in detail below with reference to flowcharts.
  • FIG. 4 is a flowchart showing processing performed by the control unit 211 of the control device 1b in the microscope system 1.
  • the control unit 211 controls each unit of the microscope device 1a via the control unit 201 and the interface 204 of the microscope device 1a.
  • step S1 the control unit 211 causes the display unit 21 to display a screen 300 (see FIG. 5). Screen 300 will be described later with reference to FIGS.
  • step S2 the control unit 211 opens and closes the moving unit 20 according to the user's operation.
  • the control unit 211 drives the moving unit driving unit 203 to move the moving unit 20 rightward, thereby moving the sample.
  • the installation section 12 is exposed.
  • the user sets the sample on the exposed sample placement section 12 .
  • the control section 211 drives the moving section driving section 203 to move the moving section 20 leftward and cover the sample setting section 12 .
  • step S3 the control unit 211 receives the user's operation of the search button 303 (see FIG. 5) via the input unit 213.
  • step S4 the control unit 211 performs a step of determining a plurality of relative positions (candidate positions) that are candidates for capturing a super-resolution image.
  • step S4 the control unit 211 drives the Z-axis driving unit 129b to capture an image of the sample while changing the relative position between the sample and the objective lens 127, and for each of a plurality of captured images obtained by capturing, To obtain a quantitative index for determining whether or not a subject is in focus. Based on the obtained index values, the control unit 211 creates a graph in which the index values corresponding to the respective relative positions are plotted. At least one candidate location for imaging is determined by identifying the relative location of the . Both the relative position and the candidate position are defined by the number of steps from the origin position of the stepping motor of the Z-axis driving section 129b. Details of step S4 will be described later with reference to FIG.
  • step S5 the control unit 211 displays the candidate positions on the display unit 21 in a selectable manner. Details of step S5 will be described later with reference to FIG.
  • step S6 the control unit 211 determines the relative position for executing imaging based on the user's selection of the candidate position displayed in step S5. Step S6 will be described later with reference to FIGS. 5 and 6.
  • step S7 the control section 211 moves the sample placement section 12 to the candidate position determined in step S6.
  • step S8 the control unit 211 performs a step of displaying the enlarged image.
  • step S ⁇ b>8 the control unit 211 displays the enlarged image 315 (see FIG. 7) on the display unit 21 . Details of step S8 will be described later with reference to FIG.
  • step S9 the control unit 211 accepts the operation of the start button 330 (see FIG. 7) via the input unit 213 by the user.
  • step S10 the control unit 211 performs a step of executing imaging of the sample.
  • step S10 the control unit 211 performs imaging of the sample at the relative position determined in step S6 or the position of the objective lens 127 finely adjusted in step S83 (see FIG. 14).
  • the control unit 211 acquires a plurality of fluorescence images with the imaging element 138 while irradiating the sample with the wavelength preset by the user, that is, the first wavelength or the second wavelength.
  • step S11 the control unit 211 acquires a super-resolution image based on the image acquired in step S10. Details of steps S10 and S11 will be described later with reference to FIG.
  • FIG. 5 to 7 are diagrams showing the configuration of the screen 300 displayed on the display unit 21.
  • FIG. 5 is a diagram showing the screen 300 in the initial state
  • FIG. 6 is a diagram showing the screen 300 after the search button 303 is operated
  • FIG. 7 is a diagram showing the screen 300 after the candidate positions are selected. It is a figure which shows.
  • a screen 300 includes a search range setting area 301, a sensitivity slider 302, a search button 303, a graph 311, a position slider 312, a reference image area 313, and fine adjustment setting areas 321 and 322. , and a start button 330 .
  • the search range setting area 301 has two numerical boxes 301a and 301b.
  • the search range is defined by defining the first numerical value input in the numerical box 301a as the upper limit position and the second numerical value input in the numerical box 301b as the lower limit position.
  • the number of steps of the stepping motor of the Z-axis drive unit 129b corresponding to the upper limit position of the distance between the sample (sample placement unit 12) and the objective lens 127 is entered in the numerical box 301a.
  • the number of steps of the stepping motor of the Z-axis drive unit 129b corresponding to the lower limit position between the sample and the objective lens 127 is entered in the numerical box 301b.
  • Two numerical boxes 301a and 301b are used to set the range (search range) of the distance between the sample and the objective lens 127 when acquiring the captured image in the process of determining candidate positions.
  • the sensitivity slider 302 is a slider for setting the interval in the Z-axis direction to acquire captured images in the search range.
  • the captured image acquisition interval in the Z-axis direction is set to be narrower, and when the knob 302a is moved to the right, the captured image acquisition interval in the Z-axis direction is increased. set to be wide.
  • the acquisition interval of captured images in the search range is defined, for example, as the number of steps of the stepping motor of the Z-axis driving unit 129b per captured image, and is stepped within the range of, for example, 1 image/10 steps to 1 image/500 steps. configurable.
  • the imaging element 138 acquires a plurality of captured images while changing the relative position between the sample and the objective lens 127.
  • the relative position between the sample and the objective lens 127 is changed by moving the sample mounting part 12 in one direction along the Z-axis with respect to the objective lens 127 whose position is fixed.
  • the captured image thus acquired is stored in the storage unit 212 of the control device 1b.
  • the sample placement section 12 moves from top to bottom along the Z axis, but the direction of movement may be reversed.
  • the control unit 211 of the control device 1b calculates an index, which will be described later, from each of the acquired captured images.
  • the index is a numerical value that quantifies the definition of an image, which is obtained by image analysis of individual captured images. The higher the numerical value of the index, the clearer the image, and the more likely that the object in the sample is in focus.
  • the search button 303 By operating the search button 303, the screen 300 changes to the state shown in FIG.
  • control unit 211 acquires the captured image, calculates the index, determines the candidate position, and the like again.
  • the graph 311 shows the relationship between the relative position and the value of the index acquired for each captured image corresponding to each relative position.
  • the horizontal axis of the graph 311 indicates the relative position, that is, the number of steps applied to the stepping motor of the Z-axis drive unit 129b.
  • the right end corresponds to the lower limit position of the search range entered in the numeric box 301b.
  • the vertical axis of the graph 311 indicates the value of the index.
  • a mark 311a in the graph 311 has an arrow shape to indicate the positions of points corresponding to the four determined candidate positions.
  • the mark 311 a is displayed so as to be selectable with the mouse of the input unit 213 .
  • the user operates the mouse to place the cursor on the mark 311a and clicks to select the mark 311a.
  • any point on the graph 311 can be selected by a click operation.
  • the user can grasp where the value of the index corresponding to the candidate position occurs in the Z-axis direction from the position of the mark 311a.
  • a reference image area 313 is an area where the four extracted captured images are displayed as reference images 314 .
  • a reference image 314 in the reference image area 313 is displayed so as to be selectable with the mouse of the input unit 213 .
  • the reference image 314 is selected by the user operating the mouse to place the cursor on the reference image 314 and clicking it.
  • the rightmost reference image 314 of the four reference images 314 or the rightmost mark 311a of the four marks 311a is selected
  • the rightmost reference image 314 in the reference image area 313 is displayed.
  • the knob 312a of the position slider 312 is aligned with the step number corresponding to the rightmost reference image 314, and the value in the numeric box 312b is the step number corresponding to the rightmost reference image 314.
  • the reference image area 313, the graph 311 and the position slider 312 are displayed in conjunction with each other.
  • the control unit 211 determines the candidate position corresponding to the selected reference image 314 or mark 311a as the relative position for imaging.
  • the control unit 211 applies the number of steps corresponding to the determined relative position to the Z-axis driving unit 129b, thereby moving the sample placement unit 12 to the determined relative position.
  • the captured image is acquired in real time by the imaging element 138 .
  • An acquired real-time captured image that is, a moving image of the sample is displayed as an enlarged image 315 on the screen 300 .
  • the user After displaying the enlarged image 315 by selecting the reference image 314 or the mark 311a via the reference image area 313 and the graph 311, the user can finely adjust the relative position using the fine adjustment setting areas 321 and 322. can also
  • the fine adjustment setting area 321 has a plurality of buttons for moving the sample placement section 12 in the X-axis direction, Y-axis direction, and Z-axis direction. Two buttons for movement are provided in one direction, the button labeled ">>" (large movement button) is for large movement, and the button labeled ">” is for small movement. It is a button (small movement button) for moving.
  • the fine adjustment setting area 322 is provided with numerical boxes in which the step width as the movement amount corresponding to the large movement button and the step width as the movement amount corresponding to the small movement button can be set. In the example of FIG.
  • the control unit 211 controls the XY-axis driving unit 129a and the Z-axis driving unit 129b according to the number of steps set for each button to move the sample placement unit 12. Move along the XYZ axes. Even when the sample placement section 12 is moved, the imaging element 138 acquires a real-time captured image, and the acquired real-time captured image is displayed as an enlarged image 315 .
  • the user selects a candidate relative position (candidate position) via the reference image 314 and the mark 311a, and adjusts the relative position using the fine adjustment setting areas 321 and 322 as appropriate. If so, the start button 330 is operated. As a result, the relative position of the sample placement unit 12 at the time when the start button 330 is operated is determined as the relative position for imaging, and imaging for obtaining a super-resolution image is performed by the imaging element 138 in this state. .
  • step S4 in FIG. 4 The step of determining candidate positions (step S4 in FIG. 4) will be described with reference to FIGS. 8 to 11B.
  • FIG. 8 is a flowchart showing the details of the process of determining candidate positions (step S4 in FIG. 4).
  • step S41 the control unit 211 of the control device 1b captures images of the sample at intervals set by the sensitivity slider 302 while changing the relative position between the sample and the objective lens 127, and the image sensor 138 captures a plurality of captured images. get.
  • the captured image acquired in step S41 is an image used for adjusting the relative position between the sample and the objective lens 127.
  • the control section 211 drives the Z-axis driving section 129b to move the sample placement section 12 in one direction along the Z-axis.
  • the movement range of the sample placement section 12 in the Z-axis direction is the search range set in the search range setting area 301 shown in FIG. It is the distance corresponding to the sensitivity set by the shown sensitivity slider 302 .
  • the control unit 211 emits light from any one of the light sources 111 and 112 and the second illumination 128 based on the wavelength of the light source selected in advance by the user. Accordingly, when either one of the light sources 111 and 112 is driven, the imaging device 138 captures an image of fluorescence generated from the fluorescent dye bound to the sample. When the second illumination 128 is driven, the light that has passed through the dichroic mirror 116 and the filter 131 of the light that has passed through the sample is imaged by the imaging device 138 .
  • step S41 when a plurality of captured images are acquired in the search range as shown in FIG. 9, the acquired captured images correspond to the relative position of the sample and the objective lens 127 (Z-axis driving unit 129b) is stored in the storage unit 212 in association with the number of steps applied to the stepping motor 129b.
  • the captured image is stored in the storage unit 212 in association with the data file of the captured image, the name of the captured image, and the storage location.
  • step S42 the control unit 211 acquires indices based on pixel values from the captured image acquired in step S41.
  • indices are acquired from all the acquired captured images, and as shown in FIG. remembered.
  • Methods of obtaining indices include a method using root mean square, a method using standard deviation, and a method using contrast.
  • the captured image is equally divided into a predetermined number of divided areas (for example, 36 divided areas consisting of 6 vertically ⁇ 6 horizontally). At this time, one divided area has a height of H and a width of W.
  • the number of divisions of the captured image may be a number other than 36.
  • a sub-area consisting of 3 dots vertically and 3 dots horizontally centered on an arbitrary pixel is set.
  • W ⁇ H N sub-regions are provided in one divided region.
  • the pixel value of the central pixel is T
  • the pixel values of the eight pixels located around this pixel are a1 to a8
  • the sum of the differences between the pixel value T and the pixel values a1 to a8 is Assuming R, the total R is calculated by the following formula (1).
  • RMS is calculated by the following equation (2).
  • the captured image is equally divided into a predetermined number of divided areas (for example, 36 divided areas consisting of 6 vertically ⁇ 6 horizontally). At this time, one divided area has a height of H and a width of W.
  • the number of divisions of the captured image may be a number other than 36.
  • a sub-region consisting of 1 vertical dot and 1 horizontal dot is set in one divided region.
  • the pixel value of the i-th sub-region is x i
  • the average value of the pixel values of all sub-regions is x a
  • the standard deviation in one divided region is ⁇
  • ⁇ is calculated by the following equation (3).
  • the standard deviation ⁇ is similarly obtained based on the above equation (3) for all divided regions in the captured image.
  • ⁇ max be the largest value
  • ⁇ min be the smallest value among the standard deviations ⁇ of all the divided regions
  • step S43 the control unit 211 determines candidate positions based on the index obtained from each captured image in step S42. Specifically, the control unit 211 identifies a plurality of peaks in a graph showing index values with respect to positions on the Z-axis based on all the index values acquired for each captured image, and determines the index values at the respective peaks. (referred to as peak value), Nd (for example, 4) peak values are determined in descending order, and relative positions (referred to as peak positions) corresponding to the determined peak values are determined as candidate positions.
  • the number Nd may be set to a value other than four. However, if the number Nd is too small, the number of candidate positions that can be selected by the user is reduced, and there is a possibility that the position where the distance between the sample and the objective lens 127 is appropriate will not be included in the determined candidate positions. There is also On the other hand, if the number Nd is too large, the number of candidate positions to be determined increases, and the user's burden of selecting candidate positions increases. Therefore, the number Nd is preferably set in advance in consideration of these balances. From such a viewpoint, the number Nd is, for example, preferably 2 or more and 20 or less, more preferably 3 or more and 10 or less.
  • the search range may be divided into a predetermined number of sections, and the peak values of the number Nd may be determined in descending order for each section.
  • the search range may be divided into three sections, and the peak values of the number Nd may be determined in descending order in each of the three sections. In this case, a total of Nd ⁇ 3 peak values are determined, and Nd ⁇ 3 candidate positions are determined.
  • FIG. 12A is a schematic diagram of a graph when three candidate positions are determined without dividing the search range into sections.
  • FIG. 12B is a schematic diagram of a graph when the search range is divided into three sections and one candidate position is determined for each section.
  • a plurality of high peaks appear intensively in a part of the search range, here, near the lower limit position, while the observation object is in focus. may exist in another part of the search range, for example, the peak enclosed by the dashed line.
  • the search range is divided into a plurality of sections at equal intervals along the Z-axis and a fixed number of candidate positions are determined for each section, Since candidate positions are determined from other search ranges without localizing candidate positions in a specific part of the search range, the possibility of properly detecting the observation object increases.
  • step S43 as shown in FIG. 9, the peak values of the number Nd are determined in descending order among the values of all indices, and the relative positions corresponding to the determined peak values are assigned to the candidate positions. It is determined. Subsequently, as shown in FIG. 10, the control unit 211 sets 1 to the candidate flag corresponding to the determined index, and sets 0 to the candidate flag corresponding to the undetermined index. As a result, the relative position with the candidate flag set to 1 becomes the candidate position. Also, the captured image and index corresponding to the candidate position are the captured image and index with the candidate flag set to 1, respectively.
  • FIG. 13 is a flowchart showing the details of the process of displaying candidate positions (step S5 in FIG. 4).
  • the control unit 211 of the control device 1b displays the reference image 314 on the screen 300 in step S51, and displays the graph 311 on the screen 300 in step S52. Specifically, as shown in FIG. 10, candidate positions are defined by candidate flags.
  • the control unit 211 displays the captured image with the candidate flag set to 1 in the reference image area 313 as the reference image 314 . Further, the control unit 211 displays the graph 311 based on the values of all the indices, and displays the mark 311a indicating the candidate position on the peak for which the candidate flag is set.
  • the arrangement of the reference images 314 matches the arrangement of the corresponding peaks in the graph 311.
  • the captured image corresponding to the leftmost peak in the graph 311 is displayed on the leftmost side in the reference image area 313, and the captured image corresponding to the rightmost peak in the graph 311 is displayed on the rightmost side in the reference image area 313. to be displayed. This makes it easy to visually grasp the correspondence relationship between the peaks in the graph 311 and the reference image 314 .
  • the user refers to the reference images 314 arranged in the reference image area 313 and refers to the values of the indices in the graph 311 to determine the most suitable candidate position, that is, the position where the sample is mostly in focus, and where air bubbles are present. Choose a suitable candidate position with less noise.
  • four reference images 314 are displayed corresponding to the four peaks in the graph 311.
  • the rightmost peak shows the highest peak value.
  • the reference image 314 displayed on the rightmost side corresponding to this highest peak value shows the material component, and the other reference images 314 do not show the material component. If the tangible component shown in the rightmost reference image 314 is the user's intended observation object, the user may select this reference image 314 or the mark 311a.
  • a plurality of candidate positions are determined in one search, and a plurality of reference images 314 corresponding to the plurality of candidate positions are displayed in a list so that they can be selected. It is possible to reduce the trouble of moving the knob 312a of the slider 312 to search for an image in focus from among a large number of images. In addition, not only the captured image with the highest peak value but also a plurality of reference images 314 selected in descending order of the peak value are displayed. Even in this case, the user is more likely to be able to select an observation object from the reference image 314 .
  • the target observation object is not shown in the plurality of displayed reference images 314, it means that the current search did not detect a candidate position where the observation object is in focus.
  • the user may manually move the position slider 312 to search for the observation object, change the search conditions using the search range setting area 301 and the sensitivity slider 302, and perform the search again.
  • the search may be performed by moving the XY coordinate position using the adjustment setting area 321 .
  • the graph 311 is displayed as shown in the screen example of FIG. Become.
  • the tangible component of the reference image 314 corresponding to the rightmost peak is not the observed object, there are other peaks in the graph 311 that may represent the observed object. Therefore, even if the position slider 312 is operated, the possibility of finding the observed object is not high. In this case, the user can decide that it is better to change the search conditions and search again.
  • the position slider 312 is operated. or by selecting an arbitrary peak on the graph 311, it is possible to confirm whether or not the object to be observed is displayed.
  • the user's effort to focus on the object to be observed can be reduced, and the work time required for focus adjustment can be shortened.
  • Fluorescent dyes may deteriorate due to exposure to light, but it is also possible to avoid exposing the fluorescent dyes for a long period of time for focus adjustment.
  • FIG. 14 is a flowchart showing the details of the process of displaying an enlarged image (step S8 in FIG. 4).
  • step S81 the control unit 211 of the control device 1b displays on the screen 300 an enlarged image 315 (see FIG. 7) corresponding to the candidate position determined in step S6 of FIG. As described above, at this time, the sample placement section 12 has been moved to the candidate position determined in step S7 of FIG. Therefore, the control unit 211 displays the real-time captured image acquired by the image sensor 138 as the enlarged image 315 .
  • the user refers to the enlarged image 315 and determines whether or not the selected candidate position is the appropriate position of the sample placement section 12 .
  • the user finely adjusts the position of the sample placement section 12 via the fine adjustment setting areas 321 and 322 (see FIG. 7).
  • step S83 the operation of the fine adjustment setting areas 321 and 322 is performed. Accordingly, the Z-axis driving section 129b is driven to move the sample placement section 12 in the Z-axis direction. This changes the relative position between the sample and the objective lens 127 . Further, in step S83, the control section 211 drives the XY-axis driving section 129a to move the sample placement section 12 within the XY plane in accordance with the operation of the fine adjustment setting areas 321 and 322.
  • step S ⁇ b>84 the control unit 211 displays the real-time captured image acquired by the image sensor 138 as the enlarged image 315 .
  • step S82 if the control unit 211 has not received the fine adjustment from the user (step S82: NO), steps S83 and S84 are skipped. Note that the user can repeatedly perform fine adjustment until the start button 330 is operated.
  • step S9 in FIG. 4 the acceptance of the selection of the candidate position is completed, and the position of the sample placement section 12 at the time the start button 330 is operated is the position for imaging. Determined as a position. Then, the imaging in step S10 is performed at the candidate position for which the selection has been completed.
  • FIG. 15 is a schematic diagram for explaining the processing of steps S10 and S11 in FIG.
  • step S10 of FIG. 4 the control unit 211 of the control device 1b drives the laser driving unit 202 with the sample placement unit 12 positioned at the position when the start button 330 was operated, and either of the light sources 111 and 112 Light (excitation light) is emitted from one of them.
  • a user presets the wavelength of the excitation light corresponding to the fluorescent dye bound to the sample via the input unit 213 .
  • the control unit 211 causes one of the light sources 111 and 112 to emit the excitation light corresponding to the wavelength set by the user. Then, the control unit 211 captures an image of fluorescence generated from the fluorescent dye bound to the sample using the imaging device 138 .
  • the fluorescent dye bound to the sample is configured to switch between a luminescent state that produces fluorescence and a quenched state that does not produce fluorescence when the excitation light continues to irradiate.
  • a luminescent state that produces fluorescence
  • a quenched state that does not produce fluorescence when the excitation light continues to irradiate.
  • some of the fluorescent dyes enter a luminescent state and generate fluorescence. Thereafter, when the excitation light continues to irradiate the fluorescent dye, the fluorescent dye flickers on its own, and the distribution of the fluorescent dye in the luminescent state changes over time.
  • the control unit 211 repeatedly captures the fluorescence generated while the fluorescent dye is being irradiated with the excitation light, and acquires several thousand to several ten thousand fluorescence images.
  • fluorescence bright points are extracted by Gaussian fitting for each fluorescence image acquired in step S10.
  • a bright point is a point that can be recognized as a shining point in a fluorescence image.
  • the coordinates of each bright point are obtained on the two-dimensional plane.
  • For each fluorescence area on the fluorescence image when matching with the reference waveform is obtained within a predetermined range by Gaussian fitting, a bright spot area having a width corresponding to this range is assigned to each bright spot.
  • a super-resolution image is created by superimposing the bright spot regions of the bright spots thus obtained for all fluorescence images.
  • a plurality of candidate relative positions are determined based on captured images obtained while changing the relative position (the number of steps of the Z-axis driving unit 129b) between the sample and the focal point of the light receiving optical system 140. (Step S4 in FIG. 4).
  • a relative position for performing imaging is determined from a plurality of candidate relative positions (step S6 in FIG. 4)
  • the sample is imaged at the determined relative position (step S10 in FIG. 4).
  • the user can adjust the relative position only by selecting the relative position that the user considers appropriate from among the plurality of automatically determined candidate relative positions. Since there is no need to search for an appropriate relative position from a large number of captured images, it is possible to adjust the focal position more easily than before.
  • an enlarged image 315 (see FIG. 7) of the sample that is larger than the reference image 314 (see FIG. 7) is displayed.
  • This allows the user to refer to the enlarged image 315 and smoothly determine whether the relative position for performing imaging is appropriate.
  • the user refers to the enlarged image 315 to determine the appropriate relative position for imaging. can be determined more smoothly.
  • the step of displaying candidate positions displays a plurality of reference images 314 (see FIG. 6) of the sample corresponding to a plurality of candidate relative positions (candidate positions) (step S51 in FIG. 13). ). Thereby, when the user selects any one of the candidate positions, the user can refer to the reference image 314 and judge whether or not the candidate position is appropriate.
  • a plurality of reference images 314 are displayed in a selectable manner, and based on the selection of any one of the reference images 314, the selected The relative position corresponding to the reference image 314 is determined as the relative position for performing imaging. Accordingly, the user can smoothly select a relative position for executing imaging by referring to the reference image 314 and making a selection with respect to the reference image 314 .
  • the step of determining the relative position for performing imaging (step S6 in FIG. 4) and the step of displaying an enlarged image (step S8) can be repeatedly performed unless the start button 330 (see FIG. 7) is operated.
  • the step of displaying the enlarged image an enlarged image 315 (see FIG. 7) of the sample at different relative positions is displayed in accordance with the determination of different relative positions as the relative positions for executing imaging. .
  • the user can smoothly refer to the corresponding enlarged image 315 to determine whether another relative position is appropriate. can be judged.
  • step S8 in FIG. 4 an operation to finely adjust the relative position for executing imaging is received via the fine adjustment setting areas 321 and 322 in FIG. Then, the enlarged image 315 (see FIG. 7) is changed (step S84 in FIG. 14). This allows the user to smoothly fine-tune the relative position while referring to the enlarged image 315 .
  • the step of determining a plurality of candidate relative positions (candidate positions) includes, as shown in FIG. Determining a plurality of candidate relative positions (candidate positions) based on the index (step S43). This makes it possible to smoothly acquire the candidate positions from the captured image.
  • the step of displaying candidate positions is a step of displaying a graph 311 (see FIG. 6) showing the relationship between the plurality of relative positions and the index corresponding to each relative position (step S52 in FIG. 13). )including. Thereby, the user can refer to the graph 311 to grasp the relationship between the relative position and the index corresponding to the relative position.
  • step S6 in FIG. 4 based on the selection of the relative position via the graph 311 (see FIG. 6), the selected relative position is used for imaging. Determined relative position to execute. Thereby, the user can smoothly select the relative position while referring to the graph 311 .
  • index is calculated from the pre-indices of the divided areas of .
  • the pre-index can be the root mean square or standard deviation of the values for the pixel values obtained from the multiple sub-regions within the segmented region. According to the index calculated using the root mean square or standard deviation in this way, when a captured image corresponding to a bright-field image is acquired, appropriate candidate positions can be acquired from the captured image.
  • the difference or ratio between the maximum and minimum pixel values based on the captured image is calculated as an index. According to the index calculated using the maximum value and the minimum value (contrast) of the pixel values in this way, when a captured image corresponding to the fluorescence image is acquired, an appropriate candidate position can be acquired from the captured image.
  • step S11 In the step of acquiring a super-resolution image (step S11), as described with reference to FIG. is obtained.
  • Super-resolution images have a resolution that exceeds the diffraction limit of light (about 200 nm). can be observed and highly accurate image analysis can be performed.
  • both the display of the reference image 314 (step S51) and the display of the graph 311 (step S52) are performed.
  • the present invention is not limited to this, and only the reference image 314 may be displayed as shown in FIG. 16A, or only the graph 311 may be displayed as shown in FIG. 16B.
  • candidate positions are selected via the reference image 314 and the marks 311a.
  • the selection is not limited to this, and the candidate positions may be selected only via the reference image 314, or the candidate positions may be selected only via the mark 311a.
  • the candidate position may be selected by operating the knob 312a or the numerical box 312b of the position slider 312. FIG.
  • the relative position between the sample and the objective lens 127 is changed by changing the position in the Z-axis direction of the sample placing portion 12 and the objective lens 127 .
  • the relative position between the sample and the objective lens 127 may be changed by changing the position of the objective lens 127 in the Z-axis direction.
  • the number of steps of the stepping motor of the Z-axis drive section separately provided to drive the objective lens 127 in the Z-axis direction corresponds to the relative position between the sample and the objective lens 127 .
  • the relative positions may be changed by changing the positions of both the sample placement section 12 and the objective lens 127 in the Z-axis direction.
  • the relative position between the sample and the focal point of the light receiving optical system 140 may be adjusted by moving an optical element other than the objective lens 127 in the light receiving optical system 140 .
  • an inner focus lens may be provided in addition to the objective lens 127, and the focus of the light receiving optical system 140 may be changed by moving the inner focus lens.
  • the candidate position determined in step S4 of FIG. 4 is obtained as the position of the inner focus lens.
  • the candidate positions determined in step S4 of FIG. A value that uniquely determines the relative position of .
  • it may be a distance indicating how far the sample placement section 12 has moved in the Z-axis direction from the origin.
  • step S43 of FIG. 8 the index values of the number Nd in descending order among all the peak values are determined as the values corresponding to the candidate positions.
  • the present invention is not limited to this, and an index value equal to or greater than the threshold value Th among all index values may be determined as the value corresponding to the candidate position.
  • step S43 it is possible that no candidate positions can be listed. For example, if the pretreatment of the sample is not appropriate, or if the placement of the sample or the placement of the slide glass is not appropriate, there is no indicator that is equal to or greater than the threshold value Th, and candidate positions cannot be listed. obtain.
  • the control unit 211 obtains indices from all the captured images after obtaining all the captured images, and uses the obtained indices as Candidate positions were determined based on
  • the present invention is not limited to this, and the control unit 211 may acquire the index from the acquired captured image while changing the relative position between the sample and the objective lens 127 while acquiring the captured image. In this case, if the value of the index sequentially acquired according to the captured image is equal to or greater than the threshold value Th, the control unit 211 determines the value of the index as the value corresponding to the candidate position.
  • the enlarged image 315 displayed by selecting the candidate position was a real-time image acquired by the imaging device 138, but is not limited to this and may be a still image.
  • the enlarged image 315 may be an image obtained by enlarging the captured image corresponding to the selected candidate position, in other words, the captured image displayed as the reference image 314 .
  • control unit 211 moves the sample placement unit 12 in step S83 of FIG. 315.
  • the captured image with the candidate flag set to 1 is displayed as the reference image 314.
  • the sample placement unit 12 is moved based on the candidate position, and the captured image corresponding to the candidate position is captured again. may be displayed as the reference image 314 .
  • the reference image 314 displayed in FIGS. 6 and 7 can be selected according to the operation on the reference image 314.
  • the present invention is not limited to this, and the button or check mark attached to the reference image 314 can be selected.
  • a reference image 314 may be selected, such as by a box.
  • indices based on pixel values are obtained from a plurality of captured images, and the relative position between the sample and the objective lens 127 is determined based on the obtained indices.
  • Candidate locations were determined for which distances were considered appropriate.
  • the method of analyzing a plurality of captured images and determining at least one candidate position is not limited to this. For example, by analyzing a plurality of captured images with a deep learning algorithm, the captured images may be selected, and candidate positions corresponding to the selected captured images may be determined. Also in this case, the user can set the relative position of the sample and the objective lens 127 to an appropriate position by selecting any one of the candidate positions determined by the deep learning algorithm.
  • microscope system 12 sample setting section 129b Z-axis drive section (drive section) 138 image sensor 140 light receiving optical system 211 control section 311 graph 314 reference image 315 enlarged image

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Microscoopes, Condenser (AREA)
PCT/JP2022/015946 2021-09-30 2022-03-30 撮像方法、焦点位置調整方法および顕微鏡システム Ceased WO2023053540A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2023551052A JPWO2023053540A1 (https=) 2021-09-30 2022-03-30
US18/621,162 US20240241360A1 (en) 2021-09-30 2024-03-29 Imaging method, focal position adjusting method, and microscope system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021162241 2021-09-30
JP2021-162241 2021-09-30

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/621,162 Continuation US20240241360A1 (en) 2021-09-30 2024-03-29 Imaging method, focal position adjusting method, and microscope system

Publications (1)

Publication Number Publication Date
WO2023053540A1 true WO2023053540A1 (ja) 2023-04-06

Family

ID=85782204

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/015946 Ceased WO2023053540A1 (ja) 2021-09-30 2022-03-30 撮像方法、焦点位置調整方法および顕微鏡システム

Country Status (3)

Country Link
US (1) US20240241360A1 (https=)
JP (1) JPWO2023053540A1 (https=)
WO (1) WO2023053540A1 (https=)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117233921A (zh) * 2023-08-31 2023-12-15 珠海市迪谱医疗科技有限公司 一种用于荧光成像设备的自动对焦方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06186469A (ja) * 1990-12-26 1994-07-08 Hitachi Ltd 顕微鏡装置の自動焦点検出方法及び自動焦点検出装置
JP2013131862A (ja) * 2011-12-20 2013-07-04 Olympus Corp 画像処理システム及びそれを備えた顕微鏡システム
JP2013142769A (ja) * 2012-01-11 2013-07-22 Olympus Corp 顕微鏡システム、オートフォーカスプログラムおよびオートフォーカス方法
JP2015166829A (ja) * 2014-03-04 2015-09-24 富士フイルム株式会社 細胞撮像制御装置および方法並びにプログラム
JP2015227940A (ja) * 2014-05-30 2015-12-17 国立研究開発法人理化学研究所 光学顕微鏡システムおよびスクリーニング装置
JP2016206228A (ja) * 2015-04-15 2016-12-08 キヤノン株式会社 合焦位置検出装置、合焦位置検出方法、撮像装置、撮像システム

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102017111718A1 (de) * 2017-05-30 2018-12-06 Carl Zeiss Microscopy Gmbh Verfahren zur Erzeugung und Analyse eines Übersichtskontrastbildes
GB201907953D0 (en) * 2019-06-04 2019-07-17 Smi Drug Discovery Ltd An optical microscope
JP7505707B2 (ja) * 2020-03-24 2024-06-25 国立大学法人東海国立大学機構 結晶欠陥評価方法及び結晶欠陥評価装置
CN113158996A (zh) * 2021-05-25 2021-07-23 兰波(苏州)智能科技有限公司 一种基于扫描电子显微镜图像和人工智能的硅藻两步识别和分类方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06186469A (ja) * 1990-12-26 1994-07-08 Hitachi Ltd 顕微鏡装置の自動焦点検出方法及び自動焦点検出装置
JP2013131862A (ja) * 2011-12-20 2013-07-04 Olympus Corp 画像処理システム及びそれを備えた顕微鏡システム
JP2013142769A (ja) * 2012-01-11 2013-07-22 Olympus Corp 顕微鏡システム、オートフォーカスプログラムおよびオートフォーカス方法
JP2015166829A (ja) * 2014-03-04 2015-09-24 富士フイルム株式会社 細胞撮像制御装置および方法並びにプログラム
JP2015227940A (ja) * 2014-05-30 2015-12-17 国立研究開発法人理化学研究所 光学顕微鏡システムおよびスクリーニング装置
JP2016206228A (ja) * 2015-04-15 2016-12-08 キヤノン株式会社 合焦位置検出装置、合焦位置検出方法、撮像装置、撮像システム

Also Published As

Publication number Publication date
US20240241360A1 (en) 2024-07-18
JPWO2023053540A1 (https=) 2023-04-06

Similar Documents

Publication Publication Date Title
JP7609804B2 (ja) 無限遠補正顕微鏡のための自己較正および方向集束のシステムおよび方法
JP5698398B2 (ja) スライド全体蛍光スキャナ
US9383569B2 (en) Magnification observation device
US9007452B2 (en) Magnification observation device, magnification observation method, and magnification observation program
JP3888980B2 (ja) 物質同定システム
US12566323B2 (en) AI enabled selective raman spectrometry using microscopy for diagnosis and analysis of biological samples
KR20120003376A (ko) 현미경 및 영역 판정 방법
US20140152800A1 (en) Image quality optimization of biological imaging
JP2019527375A (ja) 顕微鏡によって試料をデジタルで撮影するための方法
US20140295535A1 (en) Measurement apparatus
JP2014523545A (ja) 生体イメージングのための顕微鏡システム及び方法
CN116337755A (zh) 用于定位样品的方法和系统
US20240331100A1 (en) Method for recording a merged microscope image, microscope system and computer program product
US20240241360A1 (en) Imaging method, focal position adjusting method, and microscope system
CN114355600A (zh) 用于显微镜的控制装置
US20230194339A1 (en) Raman microscope
CN116593388A (zh) 红外拉曼显微镜
JP2009002795A (ja) 蛍光ｘ線分析装置
JP6362435B2 (ja) 顕微鏡システム
JP2020086296A (ja) 拡大観察装置
JP2020086295A (ja) 拡大観察装置
KR102602005B1 (ko) 하전 입자선 장치
JP3125124U (ja) 赤外顕微鏡
JP7251957B2 (ja) 拡大観察装置
JP3121902U (ja) 赤外顕微鏡

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22875423

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023551052

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22875423

Country of ref document: EP

Kind code of ref document: A1