JP4936867B2 - Enlarged image observation apparatus and enlarged image observation method - Google Patents

Description

The present invention is an enlarged image observation apparatus such as a fluorescent microscope or a confocal microscope having a function of displaying by capturing a fluorescent image of the sample, to an enlarged image observation how.

  For observation using a microscope, bright field observation, dark field observation, phase difference observation, differential interference observation, fluorescence observation, polarization observation, and the like are used according to the specimen to be observed and the research purpose. In bright field observation, light transmitted through or reflected from a specimen is observed. On the other hand, in fluorescence observation, a fluorescent dye is introduced into a sample such as a specimen or an object in advance, and the sample is irradiated with excitation light of a specific wavelength, and fluorescence emitted from the sample is observed. In general, fluorescence observation is performed to observe the fine structure of cells, the localization of molecules, and the like, and a fluorescence microscope is used. In a fluorescence microscope, a fluorescent molecule that specifically binds to a specific molecule of interest in a sample (called a fluorescent probe or the like, for example, a fluorescent protein covalently bound to an antibody of the protein of interest is used) In addition, we observe the distribution and movement of this molecule.

Not only such a fluorescence microscope, but also various observations and measurements are performed using an enlarged image observation device such as an observation microscope, a measurement microscope, a microscope, etc. including an optical microscope and an electron microscope. In such a case, an image with a wide field of view (wide area image) is displayed at a low magnification, and then a target field of view is searched for and enlarged. Various techniques have been developed to realize such work. For example, in Patent Document 1, when an area is designated on a wide-area image obtained at a low magnification, a visual field that can observe the area, that is, an objective lens having a magnification is automatically set. A technique for inserting and moving the XY stage is disclosed. Patent Document 2 discloses a technique for automatically dividing a rectangular area on an observation image including an entire image of a specimen into a plurality of further rectangular areas.
Japanese Patent No. 2925647 Japanese Patent No. 3321198

  However, in any of these methods, only one low-magnification wide-area image can be displayed and an area can be specified on the low-magnification wide-area image. Limited to the field of view obtained. In other words, there has been a problem that a wide-area image wider than the field of view obtained with the objective lens having the lowest magnification cannot be obtained.

The present invention has been made to solve such conventional problems. A first object of the present invention is enlarged not limited to the field of view obtained the range of the wide area image at the lowest magnification of the objective lens image observation apparatus is to provide an enlarged image observation how.

  On the other hand, as shown in FIG. 1, the wide area image WI and the enlarged image MI are displayed on the same screen, and the corresponding position of the enlarged image MI is displayed on the frame BX on the wide area image WI of FIG. Thus, a function is provided in which the positional relationship between the wide area image WI and the enlarged image MI can be grasped. At this time, as a method for enlarging the enlarged image MI shown in FIG. 1B, there is one having a zoom function by image processing called digital zoom in addition to optical zoom. Since image enlargement by digital zoom is limited to the resolution of the original image, high resolution cannot be maintained when enlarged, unlike optical zoom. When the enlarged image MI shown in FIG. 1B is enlarged by digital zoom, it becomes narrower than the frame BX showing the enlarged image MI displayed on the wide area image WI of FIG. Although the amount of information is reduced as data in use, the field of view is not changed on the actually displayed wide-area image WI, so that the user maintains the same resolution as the optical zoom. There is a possibility of misunderstanding, and the corresponding position on the low-magnification wide-area image WI is not correctly reflected, and there is a problem that an accurate positional relationship is not grasped.

The present invention has been made to further solve such problems. A second object of the present invention, the wide area image and the positional relationship expandable enlarged image observation apparatus capable of matching in accordance with the magnification of the digital zoom of the enlarged image to provide an enlarged image observation how.

Means for Solving the Problems and Effects of the Invention

In order to achieve the above object, an enlarged image observation apparatus according to an embodiment of the present invention is an enlarged image observation apparatus that enlarges and displays an observation object, and optical imaging for acquiring an image of the observation object A display unit having a system, a wide-area image display area for displaying a wide-area image captured at a low magnification, an enlarged image display area for displaying an enlarged image captured at a high magnification, and a plurality of wide-area areas A memory unit for acquiring images in advance and holding them, a sample mounting unit for mounting an observation object, and a moving mechanism for moving the sample mounting unit relative to the optical imaging system And a wide area image switching means for reading and switching the wide area image displayed in the wide area image display area from the memory unit, and the display unit further includes a relative movement for displaying the relative movement range of the sample mounting unit. Has a range display area, The imaging position of a plurality of wide area images can be displayed on the relative movement range while the relative movement range is displayed in the movement range display area, and the wide area image displayed on the relative movement range is selected. Thus, the wide area image displayed in the wide area image display area can be switched and displayed, and a desired area can be designated on the wide area image so that the enlarged image can be displayed in the enlarged image display area of the display unit. Thereby, a plurality of wide area images can be registered, and these can be switched and displayed as appropriate, and the field of view can be searched in a wider range without being limited to the display area of one wide area image. Furthermore, not only can a plurality of wide-area images be registered and switched, but also the positional relationship of these multiple wide-area images can be grasped on the movement range of the sample mounting portion, which is the entire observable area. The user can be guided without losing sight of the position of the wide area image, and the visual field search and area specification can be performed reliably.

The first magnified image observing device is a magnified image observing device that magnifies and displays an observation target, an optical imaging system for acquiring an image of the observation target, and a wide-area image obtained by capturing the observation target at a low magnification. a wide area image display area for displaying, a display unit having a larger image display area for displaying an enlarged image taken at a high magnification, a memory unit for acquired wide band image in advance, held, observed A sample placement unit for placing an object, a moving mechanism for moving the sample placement unit relative to the optical imaging system, and for performing image composition on a wide area image display region accepting designation of a plurality of images, comprising: a wide area image synthesizing means to move the moving mechanism you combining the images of areas adjacent imaged, and on the large area image displayed on the wide area image display region The specified area is enlarged as an enlarged image And displays in the display area, the position where the enlarged image corresponding over a wide area image, it can be displayed in a frame shape in the wide area image display region. This can generate a greater synthetic image by synthesizing a plurality of images, wide synthetic wide image than the wide area image obtained at the lowest magnification of the objective lens is obtained.

Second enlarged image observation apparatus further the combined synthetic image, based on the position obtained by synthesizing the images with each other that by the wide area image synthesizing unit may comprise a correction coordinate position calculating means for correcting a coordinate position. As a result, when the synthesized wide area image is synthesized, even if a coordinate position shift occurs due to, for example, an overlapping portion between adjacent areas of the wide area image, a coordinate position in which the coordinate position is accurately corrected can be set for the synthesized wide area image. Therefore, it is possible to accurately calculate the corresponding position even for the synthesized wide area image.

Third enlarged image observation apparatus, when generating the synthetic image the wide area image synthesizing means to synthesize four or more images, based on the coordinate position of the image correction coordinate position calculating means you located in the upper left The corrected coordinate position can be calculated. Thereby, the reference position for correcting the coordinate position is clarified, and correction by the corrected coordinate position calculation means can be easily performed.
According to the fourth magnified image observation device, the magnified image observation device is a fluorescence microscope, and the magnified image observation device further includes, as an observation mode, a fluorescence observation mode for acquiring a fluorescence image and a bright field image for acquiring a bright field image. Observation mode switching means for switching between the visual field observation modes can be provided.
According to the fifth enlarged image observation apparatus, the display unit is provided with a channel selection screen for selecting an observation method and a channel setting screen for performing setting for each channel. When an observation mode is selected for each channel, an observation condition setting unit that automatically switches the setting of the corresponding channel to a setting corresponding to the selected observation mode is provided and is selected on the display unit. An image captured in the observation mode corresponding to the selected channel can be configured to be displayed in the enlarged image display area.
According to the sixth enlarged image observation device, the enlarged image display area can simultaneously display different images corresponding to the respective channels.
According to the seventh enlarged image observation apparatus, in addition to the setting for each channel, the display unit can display a common setting column for setting common setting items for all channels.
According to the eighth enlarged image observation apparatus, the fluorescence observation mode is a mode for observing a monochrome fluorescence image with a monochrome camera, and has a pseudo color display function for coloring a pseudo color on the monochrome image obtained by the monochrome camera. Can do.
According to the ninth magnified image observation apparatus, the magnified image observation apparatus is a confocal microscope, and the confocal microscope is a color super depth image obtained by synthesizing a color image with a confocal image, an ultra depth image, and an ultra depth image. Alternatively, the height information of the sample can be obtained.
The tenth enlarged image observation apparatus has a digital zoom function capable of displaying an enlarged image by enlarging the image by image processing. Further, when the enlarged image is digitally zoomed, the frame size on the wide area image is displayed after the digital zoom. The image can be reduced or enlarged according to the area displayed in the digital zoom image. Accordingly, the corresponding position on the wide area image is also reduced or enlarged in correspondence with the digital zoom, so that the user can grasp the relationship between the wide area image and the enlarged image that correctly correspond to the data amount of the original image. In particular, since the resolution of an enlarged image obtained by digital zooming is deteriorated, it is useful for notifying the user of the state of digital zooming by reducing the resolution to a range corresponding to the digital zoom.
The eleventh enlarged image observation device displays the position corresponding to the original image before the digital zoom of the enlarged image on the wide area image displayed in the wide area image display area in the first frame shape, and The corresponding size can be displayed in the second frame shape, and the first frame shape and the second frame shape can be displayed in different modes. As a result, the user can compare the position and size before digital zoom with the position and size after digital zoom on the wide-area image display area, and can visually grasp the change of the area due to digital zoom. For example, when the original image is an optical image captured by an optical imaging system, the optical field of view can be displayed in comparison with the field of view that has been digitally zoomed, and the limit of the digital zoom capability can be notified to the user.

The enlarged image observation apparatus according to the embodiment can be configured such that the wide area image display area can be displayed on a separate screen. This makes it possible to display a wide-area image that is generally displayed in a reduced size on a separate screen, and to easily enlarge and display it as necessary, contributing to work such as searching for a visual field as a more detailed display.

In the twelfth enlarged image observation apparatus, the enlarged image observation apparatus includes an imaging system that captures a fluorescence image and a bright field image, and the wide area image to be displayed in the wide area image display area is a bright field image. The enlarged image displayed on the screen can be a fluorescent image. Thereby, while using a color bright field image as a wide-area image, a monochrome fluorescent image capable of acquiring a clear image at a high magnification is used as an enlarged image, so that a field of view can be searched and observed efficiently.

The thirteenth enlarged image observation apparatus can display a digital zoom image of a registered wide area image on the display unit while the sample mounting unit is moving. Thereby, even when it takes time to move the sample mounting portion, a temporary digital zoom image is displayed, so that it is possible to reduce waiting time stress for the user.

The magnified image observation apparatus according to the embodiment can be configured to divide an image display area for displaying an image on the display unit into a plurality of parts and to display a plurality of wide area images held in the memory unit side by side. Thereby, a plurality of registered wide area images can be confirmed on one screen, and visibility can be further enhanced.

An enlarged image observation method according to an embodiment is an enlarged image observation method for displaying an observation object in an enlarged manner, and includes a step of acquiring and storing a plurality of wide-area images captured at a low magnification, and an observation object. The relative movement range in which the sample placement unit to be placed can move relative to each other is displayed in the relative movement range display area of the display unit, and the positions of the plurality of stored wide-area images on the relative movement range are A step of displaying the image over the moving range, and selecting a position of any one of the wide area images on the relative movement range so that the corresponding wide area image is switched to the wide area image display area of the display unit and displayed. And a step of designating a desired area on the wide area image and displaying the enlarged image in the enlarged image display area of the display unit. Thereby, a plurality of wide area images can be registered, and these can be switched and displayed as appropriate, and the field of view can be searched in a wider range without being limited to the display area of one wide area image. Furthermore, not only can a plurality of wide-area images be registered and switched, but also the positional relationship of these multiple wide-area images can be grasped on the movement range of the sample mounting portion, which is the entire observable area. The user can be guided without losing sight of the position of the wide area image, and the visual field search and area specification can be performed reliably.

The magnified image observation method according to the embodiment is a magnified image observation method for magnifying and displaying an observation object, and acquiring and storing a plurality of wide-area images captured at a low magnification, and reducing the observation object. On a wide area image displayed in the wide area image display area in a display unit having a wide area image display area for displaying a wide area image captured at a magnification and an enlarged image display area for displaying an enlarged image captured at a high magnification The area designated in (2) is displayed as an enlarged image in the enlarged image display area, and the position corresponding to the enlarged image on the wide area image is displayed in the first frame shape in the wide area image display area, and the enlarged image is digitally zoomed. Then, while displaying the first frame shape, the second frame shape displayed in a mode different from the first frame shape, and the frame shape on the wide area image according to the area displayed in the digitally zoomed image. big It may include a step of reducing or enlarging the. Accordingly, the corresponding position on the wide area image is also reduced or enlarged in correspondence with the digital zoom, so that the user can grasp the relationship between the wide area image and the enlarged image that correctly correspond to the data amount of the original image. In particular, since the resolution of an enlarged image obtained by digital zooming is deteriorated, it is useful for notifying the user of the state of digital zooming by reducing the resolution to a range corresponding to the digital zoom. In addition, the user can compare the position and size before digital zoom with the position and size after digital zoom on the wide-area image display area, and can visually grasp the change of the area due to digital zoom. For example, when the original image is an optical image captured by an optical imaging system, the optical field of view can be displayed in comparison with the field of view that has been digitally zoomed, and the limit of the digital zoom capability can be notified to the user.

Enlarged image observation method of the first 4 is an enlarged image observation method to enlarge the observed object, it is moved relative to the optical imaging system a specimen mounting portion for mounting the observed object Then, a step of capturing a plurality of wide area images in adjacent areas, and combining the plurality of wide area images on the wide area image display area of the display unit by image processing so that the boundary portions are matched and adjacent to each other a step of creating a larger wide area image region, in the synthesis synthetic wide image may include the step of, based on the position information by connecting the wide area image with each other, to correct the coordinate position of the entire composite image. As a result, a larger combined wide area image can be generated by combining a plurality of wide area images, and a combined wide area image wider than the wide area image obtained at the minimum magnification of the objective lens can be obtained. In addition, when a synthesized wide area image is synthesized, even if a coordinate position shift occurs due to an overlapping portion between adjacent areas of the wide area image, a coordinate position in which this is accurately corrected can be set for the synthesized wide area image. Thus, it is possible to accurately calculate the corresponding position for the synthesized wide area image.

The magnified image observation program according to the embodiment is a magnified image observation program that magnifies and displays an observation object, and includes a function for acquiring and storing a plurality of wide-area images captured at a low magnification, and an observation object. The relative movement range in which the sample placement unit to be placed can move relative to each other is displayed in the relative movement range display area of the display unit, and the positions of the plurality of stored wide-area images on the relative movement range are A function of displaying the image over the moving range, and selecting a position of one of the wide area images on the relative movement range to switch the corresponding wide area image to the wide area image display area of the display unit, The computer can realize a function of designating a desired area from the wide area image and displaying the enlarged image in the enlarged image display area of the display unit. Thereby, a plurality of wide area images can be registered, and these can be switched and displayed as appropriate, and the field of view can be searched in a wider range without being limited to the display area of one wide area image. Furthermore, not only can a plurality of wide-area images be registered and switched, but also the positional relationship of these multiple wide-area images can be grasped on the movement range of the sample mounting portion, which is the entire observable area. The user can be guided without losing sight of the position of the wide area image, and the visual field search and area specification can be performed reliably.

The magnified image observation program according to the embodiment is a magnified image observation program for magnifying and displaying an observation object, and has a function of acquiring and storing a plurality of wide-area images captured at a low magnification, and reducing the observation object. On a wide area image displayed in the wide area image display area in a display unit having a wide area image display area for displaying a wide area image captured at a magnification and an enlarged image display area for displaying an enlarged image captured at a high magnification The area designated in (2) is displayed as an enlarged image in the enlarged image display area, and the position at which the enlarged image corresponds on the wide area image is displayed in the first frame shape in the wide area image display area, and the enlarged image is digitally zoomed. Then, while displaying the first frame shape, the second frame shape displayed in a different form from the first frame shape, and on the wide area image according to the area displayed as the digitally zoomed image It is possible to realize a function to reduce or enlarge the frame-like size to a computer. Accordingly, the corresponding position on the wide area image is also reduced or enlarged in correspondence with the digital zoom, so that the user can grasp the relationship between the wide area image and the enlarged image that correctly correspond to the data amount of the original image. In particular, since the resolution of an enlarged image obtained by digital zooming is deteriorated, it is useful for notifying the user of the state of digital zooming by reducing the resolution to a range corresponding to the digital zoom. In addition, the user can compare the position and size before digital zoom with the position and size after digital zoom on the wide-area image display area, and can visually grasp the change of the area due to digital zoom. For example, when the original image is an optical image captured by an optical imaging system, the optical field of view can be displayed in comparison with the field of view that has been digitally zoomed, and the limit of the digital zoom capability can be notified to the user.

The magnified image observation program according to the embodiment is a magnified image observation program that magnifies and displays an observation object, and moves a sample placement unit for placing the observation object relative to the optical imaging system. And a function for capturing a plurality of wide area images in adjacent areas, a function for combining these multiple wide area images so that the boundary portions match by image processing, and creating a larger wide area image, and a composition In the synthesized wide area image, the computer can realize a function of correcting the coordinate position of the entire synthesized image based on position information obtained by connecting the wide area images. As a result, a larger combined wide area image can be generated by combining a plurality of wide area images, and a combined wide area image wider than the wide area image obtained at the minimum magnification of the objective lens can be obtained. In addition, when a synthesized wide area image is synthesized, even if a coordinate position shift occurs due to an overlapping portion between adjacent areas of the wide area image, a coordinate position in which this is accurately corrected can be set for the synthesized wide area image. Thus, it is possible to accurately calculate the corresponding position for the synthesized wide area image.

The computer-readable recording medium or the recorded device storing the program according to the embodiment stores the program. CD-ROM, CD-R, CD-RW, flexible disk, magnetic tape, MO, DVD-ROM, DVD-RAM, DVD-R, DVD + R, DVD-RW, DVD + RW, Blu-ray (registered) Trademarks), HD DVD and other magnetic disks, optical disks, magneto-optical disks, semiconductor memories and other media that can store programs. The program includes a program distributed in a download manner through a network line such as the Internet, in addition to a program stored and distributed in the recording medium. Further, the recorded devices include general-purpose or dedicated devices in which the program is implemented in a state where it can be executed in the form of software, firmware, or the like. Furthermore, each process and function included in the program may be executed by computer-executable program software, or each part of the process or hardware may be executed by hardware such as a predetermined gate array (FPGA, ASIC), or program software. And a partial hardware module that realizes a part of hardware elements may be mixed.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiments to be described below, an enlarged image observation apparatus for embodying the technical idea of the present invention, intended to illustrate an enlarged image observation how, the invention magnified image observation apparatus, an enlarged image observation methods that do not specific to the following. In particular, the present specification by no means specifies the members shown in the claims as the members of the embodiments. Note that the size, positional relationship, and the like of the members shown in each drawing may be exaggerated for clarity of explanation. Furthermore, in the following description, the same name and symbol indicate the same or the same members, and detailed description thereof will be omitted as appropriate. Furthermore, each element constituting the present invention may be configured such that a plurality of elements are constituted by the same member and the plurality of elements are shared by one member, and conversely, the function of one member is constituted by a plurality of members. It can also be realized by sharing.

Enlarged image observation apparatus herein, the enlarged image observation how the fluorescence microscope of the operation and display, the system itself performs the setting, and the operation of the fluorescence microscope, display, input and output associated with the set or the like, display, computation, communication The present invention is not limited to an apparatus or method that performs other processing in hardware. An apparatus and method for realizing processing by software are also included in the scope of the present invention. For example, by incorporating software, programs, plug-ins, objects, libraries, applets, scriptlets, compilers, modules, macros that run on specific programs, etc. into general-purpose circuits or computers, the image display itself of the fluorescence microscope or related to this a general-purpose or special-purpose computer which enables processing, workstation, terminal, portable electronic devices and other electronic device is also enlarged image observation apparatus of the present invention include at least one of the enlarged image observation how. In this specification, the program itself is included in the fluorescence microscope apparatus. In addition, this program is not limited to the one used alone, but is provided as a service in an environment that functions as a part of a specific computer program, software, service, etc., an aspect that is called and functions when necessary, and an environment such as an OS. It can also be used as a mode, a mode that operates resident in the environment, a mode that operates in the background, and other support programs.

Hereinafter, an example of a fluorescence microscope will be described as an example of the enlarged image observation apparatus. However, the magnified image observation device is not limited to the fluorescence microscope, but other microscopes such as electron microscopes such as optical microscopes, digital microscopes, confocal microscopes, SEM, TEM and other charged particle beam devices, atomic force microscopes, electrostatic forces It can also be applied to a microscope, a near-field microscope, and the like.
(Embodiment 1)

  FIG. 2 shows a block diagram of a fluorescence microscope system according to an embodiment of the present invention. In the following example, a multi-color fluorescence observation is performed using a fluorescent microscope to introduce a plurality of fluorescent dyes (fluorescent dyes) into a sample WK (also referred to as a specimen, specimen, sample, workpiece, etc.), and to stain and express in multiple colors. An example of performing is described. The fluorescent microscope main body 100 shown in this figure is operated by an externally connected computer 58 to display an observation image on the display unit 24 of the computer 58.

  The computer 58 is connected to the fluorescent microscope main body 100 in a wired or wireless manner, or is configured integrally with the computer 58. Various operations and settings are performed by an input device 59 connected to the computer 58. Examples of the input device 59 include various pointing devices such as a mouse, keyboard, slide pad, track point, tablet, joystick, console, jog dial, digitizer, light pen, numeric keypad, touch pad, and accu point. By operating the fluorescence microscope operation program with the input device 59, the fluorescence microscope and its peripheral devices can be operated. Further, the display unit 24 that displays the interface screen uses a touch screen or a touch panel so that the user can directly input or operate the screen by hand, or can use voice input or other existing input means. They can be used or used in combination. In the example of FIG. 2, the computer 58 includes a keyboard and a mouse as the input device 59. The mouse can be used to operate buttons, icons, sliders, and perform various operations such as specifying the position of an image. In this way, the display unit 24 displays an operation menu, settings, and the like together with the image, and by selecting an operation item or performing an operation on the image, the user can accurately grasp the operation content and state and prevent an operation error. It is possible to realize an operational system that is sensuous and easy to operate.

  The fluorescence microscope main body 100 includes an epi-illumination light source 48 that constitutes an excitation light source, a fluorescence epi-illumination collector lens 54, a filter set 1, an objective lens 50, an imaging lens 52, and a fluorescence imaging unit 22 as an optical imaging system. These members are arranged on a certain optical path. The epi-illumination light source 48 emits excitation light that excites the fluorescent dye. Since ultraviolet rays are frequently used as the excitation light, a high-pressure mercury lamp or a high-pressure xenon lamp containing an ultraviolet light component is used as the incident light source 48. They emit a wide range of wavelengths of light. The members such as the incident light source 48 can be unitized, and the units can be combined to constitute a fluorescence microscope system.

  On the front surface of the epi-illumination light source 48, an epi-illumination amount adjusting unit 49 for blocking / transmitting the epi-illumination light is provided. The incident light amount adjustment unit 49 is disposed on the optical path, and adjusts the incident light amount by switching filters having different transmittances. A filter wheel, a slider, or the like is used for switching the filter. In the example of FIG. 2, the transmitted light amount, that is, the excitation light amount can be switched in six stages using the filter wheel, with the transmittance being 0%, 20%, 40%, 60%, 80%, and 100% without the filter. . The incident light amount adjusting unit may be an open / close or movable incident light shutter, and a mechanism for turning on / off the incident light may be used, or a wheel and a shutter may be used in combination. The operation of the incident light amount adjustment unit 49 is controlled by the control unit 26 so that the incident light is transmitted during fluorescence observation and is blocked during bright field observation. In the example of FIG. 2, since an ultrahigh pressure mercury lamp is used as an excitation light source, ON / OFF is physically controlled. However, if an epi-illumination light source that can quickly turn on / off the light source and can change the amount of light by an applied voltage or the like, the epi-illumination light amount adjustment unit can be omitted. Examples of such an incident light source include a semiconductor light emitting element such as a light emitting diode and a semiconductor laser with low power consumption and a small size and high efficiency. Since the light amount can be adjusted by voltage control, continuous light control is possible instead of discrete light amount switching using a mechanical filter or shutter, and more accurate excitation light amount suppression control can be realized. Since a semiconductor light emitting element has a relatively narrow emission wavelength band, a plurality of light emitting diodes having different wavelengths may be arranged in order to cope with a plurality of excitation lights.

  The epi-illumination light transmitted from the epi-illumination light source 48 through the epi-illumination light amount adjusting unit 49 is converted into a substantially parallel light beam by the collector lens 54 and introduced into the filter set 1 as excitation light. As the collector lens 54, a fluorescent epi-illumination lens or the like can be used in the case of epi-illumination. In the following example, the illumination of the sample is the epi-illumination type, but the present invention is also applicable to other illumination methods such as transmission type illumination and total reflection illumination type. In the example of FIG. 2, the fluorescence imaging unit 22 and the eyepiece 29 for visual observation can be switched by being reflected and branched by an observation switching unit 53 using a mirror or the like. Alternatively, the visual mechanism may be omitted, and the image captured by the fluorescence imaging unit 22 may be confirmed only by the display unit 24. Further, a bright field imaging unit 25 that captures a bright field image can be arranged together with or instead of the eyepiece 29.

The filter set 1 is a combination of a single-pass filter and a mirror that selectively transmits light having a wavelength suitable for observation of a specific fluorescent dye, and as shown in FIG. 2, an excitation filter 12, an absorption filter 16, and a dichroic mirror. 14. Furthermore, the filter set 1 is configured so that a plurality of types can be switched. Each filter set 1 includes a combination of an excitation filter 12, an absorption filter 16, and a dichroic mirror 14 corresponding to the fluorescent dye to be used, and by switching these, different monochrome images can be taken. The plurality of filter sets 1 are set in the filter holder 56 and switched by the filter switching unit 18. The excitation light selected by the excitation filter 12 of the filter set 1 is reflected by the dichroic mirror 14, passes through the objective lens 50, and is projected onto the sample WK. The objective lens 50 also serves as a condenser lens. The objective lens 50 can be exchanged according to the purpose of observation, and can be configured to be able to switch between a plurality of objective lenses 50 using a screw type or the like, or a revolver or the like. For example, it can be replaced with a dedicated objective lens for phase difference observation, differential interference observation, bright field observation, dark field observation, etc., oil immersion objective lens, water immersion lens, dry lens, cover slide specimen, non-cover slide For example, an appropriate objective lens can be attached or switched depending on the observation purpose and application.
(Dark room space 46)

  The sample WK is placed on the sample placement unit 28. In general, observation of a faint light sample such as a fluorescent sample is performed in a dark room because it is necessary to exclude disturbance light. The fluorescence microscope main body 100 arranges the optical path of the sample mounting unit 28 and the optical system in a dark room space 46 that is shielded from ambient light, and this dark room space 46 is made into a dark room state, so that fluorescence can be obtained without preparing a dark room. We can observe and improve usability. The sample placement unit 28 can use an XY stage or the like, and can move in the X-axis and Y-axis directions. Further, the sample mounting portion 28 can be moved in the vertical direction (Z-axis direction), so that the focus can be adjusted by changing the relative distance from the optical system (objective lens 50). The sample mounting unit 28 is driven by a moving mechanism 38, and the moving mechanism 38 is controlled by the control unit 26. Alternatively, the focus can be adjusted by moving the objective lens 50. Similarly, in changing the field of view, the field of view can be moved by moving the sample mounting portion 28 in the X-axis and Y-axis directions as well as by moving the objective lens side.

  Among the fluorescent dyes included in the sample WK, a fluorescent dye corresponding to the irradiated excitation light emits fluorescence, and this fluorescence passes through the objective lens 50 and enters the filter set 1 and passes through the dichroic mirror 14. Thus, the dichroic mirror 14 reflects the incident illumination light and transmits the fluorescence. Further, the absorption filter 16 passes the fluorescence, and selectively absorbs light components other than fluorescence such as epi-illumination light. The absorption filter 16 is also called a barrier filter, and is disposed closer to the fluorescent image forming surface than the dichroic mirror 14. The light exiting the filter set 1 passes through the imaging lens 52 and enters the fluorescence imaging unit 22. The fluorescence imaging unit 22 is disposed at a position conjugate with the focal plane of the objective lens 50. The fluorescence imaging unit 22 converts the fluorescence into an electrical signal and sends it to the image processing unit 30. The image processing unit 30 realizes a function of a wide area image composition unit described later. After the image processing unit 30 performs necessary image processing, the image transfer unit 31 transfers the image to the external computer 58. These controls are performed by the control unit 26. The control unit 26 includes an observation mode switching unit 21 that switches between the fluorescence observation mode and the bright field observation mode, and an observation condition setting unit 23 for designating observation conditions such as automatically setting an imaging condition according to the observation mode. And a switching setting unit 20 that controls the filter switching unit 18, a wide-area image switching unit 76 that switches the wide-area image displayed in the wide-area image display area WA, and a coordinate position correction based on the position where the wide-area images are combined in the combined wide-area image. Based on the corrected coordinate position calculation means 77, the reference excitation light amount and the frame rate at the display unit, the necessary excitation light amount and the exposure time of the fluorescence imaging unit are calculated, and the exposure time of the excitation light amount adjustment unit and the fluorescence imaging unit are controlled. The functions of the exposure time / excitation light quantity control means 27 and the like are realized. The control unit 26 includes a gate array such as a microprocessor (MPU), CPU, LSI, FPGA, ASIC, or the like. The computer 58 connected to the fluorescent microscope main body 100 includes a display unit 24 and displays an image sent from the image transfer unit 31 on the display unit 24.

In this specification, an image captured at a low magnification is referred to as a wide area image, and an image captured at a high magnification is referred to as an enlarged image for convenience. Here, the low magnification and the high magnification are relative values. For example, when the low magnification magnification is about 1 to 10 times, the high magnification can be set to about 20 to 100 times.
(Fluorescence imaging unit 22)

The fluorescence imaging unit 22 is composed of an imaging device, and a two-dimensional photodetector arranged in a two-dimensional manner is used. The two-dimensional photodetectors are arranged in a two-dimensional shape, and one screen can be simultaneously imaged without sequentially scanning the screen as in a laser microscope. As the two-dimensional photodetector, a CCD image sensor, a photodiode image sensor, a phototransistor image sensor, and the like can be used in addition to a CCD. In this embodiment, a high-resolution monochrome CCD camera is used. Since the noise characteristics can be improved by cooling the CCD camera, a CCD camera having a mechanism for cooling with a Peltier element or liquid nitrogen may be used. As described above, the fluorescence microscope can automatically switch the single-pass filter set 1 by the filter switching unit 18 and can simultaneously display the monochromatic image captured by each filter set 1 and the superimposed image obtained by superimposing these images.
(Color filter 32)

A color filter 32 is provided in front of the fluorescence imaging unit 22. As the color filter 32, the mechanical filter 33 shown in FIG. 3A or the liquid crystal filter 34 shown in FIG. 3B can be used. The liquid crystal filter 34 is also called a liquid crystal tunable filter, a wavelength tunable filter, or the like, and switches R, G, and B transmission spectral characteristics according to an applied voltage. Since this filter has no mechanical operation, it can be switched stably, can be switched at a higher speed, and can be suitably used.
(Mechanical filter 33)

  However, instead of the liquid crystal filter 34, a mechanical filter 33 using a rotary turret shown in FIG. A rotary turret is a rotating plate having a structure that can generally quickly replace a plurality of filters. The disk-shaped rotating plate is rotatably supported by a central axis, and a plurality of openings are provided around the rotational axis. In the example of FIG. 3A, four openings are provided, which are R, G, B filters and openings without filters, respectively. The turret is arranged so that each opening coincides with the optical axis, and the opening can be quickly switched by rotating the turret. These filters are switched according to an instruction from the control unit 26. These color filters 32 are not limited to the RGB type, and may be a CMY type. It is also possible to use a CCD with a color filter in which a color filter is provided on the CCD itself. In this case, it is not necessary to provide another color filter.

Further, as the color filter, in addition to the liquid crystal filter and the mechanical filter 33 described above, an acoustooptic filter using an acoustooptic element (AOTF), an imaging spectrometer (inspector) combining a prism and a grating, and the like can be used.
(Filter set 1)

The filter set 1 includes a set of an excitation filter 12, an absorption filter 16, and a dichroic mirror 14 in a box-like body generally called a dichroic cube. In the filter set 1, the combination of the excitation filter 12, the dichroic mirror 14, and the absorption filter 16 is determined according to the fluorescent dye introduced into the sample WK. A combination of single-pass bandpass filters is determined so as to extract light of a desired wavelength component and exclude other wavelength components so that only the color expressed by the fluorescent dye can be correctly observed. Therefore, the filter set 1 to be used is determined according to the fluorescent dye to be used. In general, filter sets 1 having different fluorescent colors are used, and combinations of colors according to fluorescent dyes such as RGB and CMY can be appropriately employed. The plurality of filter sets 1 are configured to be switchable by a filter switching unit 18. The plurality of filter sets 1 are set in the filter holder 56, and the filter switching unit 18 causes any one of the plurality of filter sets 1 to be set on the optical path. As the filter switching unit 18, a turret or the like that switches the filter set 1 in a rotating manner or a sliding manner by driving a motor or the like can be used. Control of the filter switching unit 18 is set by the switching setting unit 20 of the control unit 26. A necessary set of excitation filter 12, absorption filter 16, and dichroic mirror 14 can be switched at once using the filter set 1, and the switching operation portion can be integrated into one place to facilitate high-speed operation and maintenance. it can. However, without using a filter set consisting of an excitation filter, an absorption filter, and a dichroic mirror, individual switching means for individually switching multiple excitation filters, absorption filters, and dichroic mirrors are provided, and each switching means is linked. Thus, a predetermined set of excitation filters, absorption filters, and dichroic mirrors may be arranged on the optical path. Furthermore, it is possible to switch the filters and take an image by changing the optical path with a mirror or the like without moving the filters.
(Display unit 24)

The display unit 24 is a display that displays an image captured by the optical system. The display constituting the display unit 24 is a monitor capable of high-resolution display, and a CRT, a liquid crystal panel, or the like is used. The display unit 24 can be incorporated in the fluorescence microscope itself or can be an external monitor.
(Transmission illumination)

Further, the fluorescence microscope of FIG. 2 includes transmission illumination so that bright field observation using transmitted light (transmission illumination) can be performed in addition to fluorescence observation using incident light (bright field illumination). A transmission illumination lens 35 and a transmission light source 36 are disposed on the lower surface of the sample mounting portion 28. The transmissive light source 36 is connected to a transmissive illumination drive power source 37 for turning on / off the transmissive illumination, and lighting is controlled by the control unit 26. As the transmissive light source 36, a halogen lamp or the like is used. Since the light quantity of the halogen lamp can be changed depending on the applied voltage, ON / OFF and light quantity control can be performed without using a mechanical filter or shutter. The transmitted illumination light emitted from the halogen lamp irradiates the sample WK on the sample mounting portion 28 through the condenser lens which is the transmitted illumination lens 35. At this time, in bright field observation, since the image is taken without a filter set for fluorescence observation, the control unit 26 operates the filter switching unit 18 to retract the filter unit from the filter set 1. The bright field image of the sample WK obtained with the transmitted illumination light passes through the objective lens 50, the filter set 1, and the imaging lens 52, is reflected by the observation switching unit 53, is captured by the bright field imaging unit 25, and is displayed on the display unit. 24 is displayed.
(Observation mode switching means 21)

  As described above, the fluorescence microscope of FIG. 2 includes the fluorescence observation mode for performing fluorescence observation and the bright field observation mode for performing bright field observation, and these can be switched by the observation mode switching means 21. Further, according to the observation mode selected by the observation mode switching means 21, the observation condition setting means 23 automatically sets an imaging condition suitable for the observation mode. Thereby, the observation mode can be switched smoothly.

Conventionally, the observation is mainly a monochrome image for fluorescence observation and a color image for bright field observation. To switch to each observation method, the user manually inserts and removes the filter and blocks / opens the excitation light. , Adjusting the focus shift and visual field shift. These operations are complicated, and particularly in a fluorescent microscope, there are many points to be operated, which is extremely troublesome. Therefore, in the present invention, the fluorescence observation mode and the bright field observation mode are selected as the observation mode, and the apparatus is automatically set to the imaging condition corresponding to the selected observation mode. As a result, the user can display a desired observation image on the display unit only by selecting the observation mode, and a very convenient microscope observation is realized.
(Fluorescence microscope operation program)

  Next, FIG. 4 shows a user interface screen of the operation screen 60 of the fluorescence microscope operation program. From the operation screen 60 of FIG. 4, the user can perform desired observations such as visual field search and magnification adjustment on the sample WK. This example is merely an example, and it goes without saying that the layout, shape, display method, size, color scheme, pattern, etc. of each input field and buttons can be changed as appropriate on the user interface screen of the fluorescence microscope operation program. By changing the design, the layout can be made easier to display, easier to evaluate and judge, and easy to operate. For example, the image display area or the detailed setting screen can be displayed in a separate window, or a plurality of screens can be displayed in the same display screen. On the user interface screens of these programs, ON / OFF operations for numerically provided buttons and input fields, designation of numerical values and command inputs, etc. are specified by an input connected to a computer 58 incorporating a fluorescence microscope operation program. Performed by device 59. In this specification, “pressing” includes not only physically touching and operating buttons, but also clicking or selecting with the input device 59 and pressing it in a pseudo manner. Further, items that cannot be selected are displayed grayed out, so that it is easy to distinguish whether the item can be set by the user or not.

  The operation screen 60 shown in this figure has an image display area 61 for displaying an image on the left side of the screen and an operation area 62 on which buttons for performing various operations are arranged on the right side. Here, four channels CH1 to CH4 are prepared for image display, and the image display area 61 is divided into four, each of which is independent and capable of displaying different images. In the example of FIG. 4, a fluorescent image is displayed on CH1 to CH3, and a bright field image is displayed on CH4. Below the image display area 61, buttons for turning on / off excitation, previewing, photographing, recording, and the like are arranged. In addition, a function selection column (Z focus, zoom, exposure time) to be assigned to the mouse wheel, ON / OFF of automatic saving, image size, gradation, and the like are displayed.

  An “observation method selection” column 64 is provided in the upper part of the operation area 62, and the observation mode selected in each channel is displayed as a button-like channel button 65. When the channel button 65 is selected, the image of the selected channel is enlarged and displayed in the image display area 61. FIG. 5 shows a state in which a bright-field image of CH4 is selected and enlarged and displayed as an enlarged image MI. Thereby, a desired image can be selected and observed in more detail. In the image display area 61 of FIG. 5, an exposure time display column for displaying the exposure time, an exposure time adjustment slider, an auto adjustment button, and the like are provided in the lower part of the image display area 61. Further, in the example of FIG. 5, a scale (length corresponding to 200 μm) SC is displayed so as to overlap the image display area 61. The user performs desired magnified observation from this screen.

Further, an observation magnification display field 66 and a “channel setting” button 67 are provided below the channel button 65. In the observation magnification display column 66, the magnification in the current image display area 61 is calculated and displayed. Further, the “channel setting” button 67 is a button for calling a channel setting screen 90 (FIG. 6) for performing setting for each channel such as changing the observation mode of the image displayed in each channel.
(Channel setting screen 90)

FIG. 6 shows an example of a channel setting screen 90 for performing setting for each channel. When the “channel setting” button 67 is pressed from the screen of FIG. 4, the channel setting screen 90 of FIG. 6 is displayed. Here, the change of the observation mode for each channel, the presence or absence of pseudo color display, and the like are performed. Specifically, the setting fields for each channel of CH1 to CH4 are divided into four in correspondence with FIG. 4 and displayed in the pull-down menu 21A constituting the observation mode switching means 21. Select either fluorescence observation or phase difference observation. Depending on the observation mode selected here, the observation condition setting means 23 automatically turns ON / OFF the transmission illumination halogen lamp (transmission light source 36), the fluorescence excitation shutter (the epi-illumination shutter provided in the excitation light amount adjustment unit 49). OPEN / CLOSE) is selected and displayed in the transmitted illumination halogen state display column 91 and the fluorescence excitation shutter state display column 92 provided in the lower part of the pull-down menu 21A. Furthermore, the internal processing method in the image processing unit 30 is also automatically switched to a setting corresponding to the change in these observation modes. For example, when “bright field” is selected, the color filter 32 is automatically inserted as a microscope setting suitable for bright field observation, and the image processing unit 30 switches to a setting for creating a color image. In this way, by eliminating the operation of manually switching each item according to the observation mode selected by the user and automatically performing the necessary switching work, significant labor saving is achieved, and the user can A desired observation image can be quickly observed without being aware of the switching operation.
(Pseudo color display function)

  In addition, a pseudo color display function for coloring a monochrome image obtained by fluorescence observation into a desired color is provided. The pseudo color display is a process of assigning color shades to monochrome gradations, and is also called LUT (lookup table) assignment. Here, by turning on the check box of the pseudo color display column 93 and selecting a desired display color in the display color selection column 94, a pseudo color fluorescent image colored in the selected color is displayed. . Further, ON / OFF of the pseudo color display function can be displayed on the fluorescent image. In the example of FIG. 4, a pseudo color ON / OFF display field 95 is provided in the lower part of the image of each channel in the image display area 61, and “pseudo color” is displayed for those that are displayed in pseudo color, and further colored. It is displayed with the same background color as the color. As a result, the user can visually recognize the channel for which the pseudo color display is selected, and can also know how many colors it has been colored. The pseudo color display function is for coloring a monochrome image and is not applied to a bright field image that is already a color image. For this reason, the pseudo color ON / OFF display column 95 is grayed out when a bright field image is selected to prevent erroneous setting.

  Furthermore, a comment display field 96 is provided, and desired comments such as the name of the sample WK and observation conditions can be displayed in the image display area 61 as text.

  In the example of FIG. 6, CH1 to CH3 select the fluorescence observation mode, and CH4 selects the bright field observation mode. In the bright field observation mode, since the color is originally displayed, the pseudo color display column 93 is grayed out and cannot be selected. In addition, a “use as overlay display” column 97 is provided at the bottom of the CH4 setting column. The overlay display is a mode in which an overlay image obtained by combining the images of CH1 to CH3 is displayed in CH4 in the case of multi-color photographing 4-screen display. As shown in FIG. 6, when the “Use as overlay display” column 97 is checked, overlay display is performed as shown in FIG. 4 regardless of the setting in CH4 (bright field observation mode in the example of FIG. 6). I do. At this time, “overlay” is displayed on the left side of the status display field 95A in the lower part of the CH4 display area in the image display area 61, and it can be confirmed that the overlay display is applied. Further, the status display field 95 can be provided with a composition adjustment field 95B for adjusting the weighting of each image of CH1 to CH3 which is the composition source image of the overlay image. In the example of FIG. 4, the transmittance of each image can be independently adjusted in the range of 0 to 100 as the synthesis adjustment field 95B.

  In addition, common setting fields 98 for setting common setting items for all channels include gain / preview speed, metering method, LUT correction / black balance, real-time filter, white balance, noise reduction, excitation light quantity, etc. ON / OFF can be selected. After these settings are completed, the setting is reflected by pressing the “set” button 99, and the observation condition setting unit 23 executes filter insertion / removal and the like.

Returning to the fluorescence microscope operation program of FIG. 4 and continuing the description, in the operation area 62 provided in the lower part of the “observation method selection” column 64, a plurality of tabs are switched to select operation items. In the example of FIG. 4, the “microscope operation” tab 102 is selected. At this time, a “lens selection” column 68 is provided in the operation area 62, and the objective lens 50 can be selected. In this fluorescent microscope, a plurality of objective lenses 50 are set on an objective lens turret, and the objective lens turret is rotated and switched. The optical magnification can be changed by switching the objective lenses 50 having different magnifications. In the example of FIG. 4, up to six objective lenses can be set. Here, five objective lenses are set. In the “lens selection” column 68, the currently set objective lens 50 is displayed in an icon shape together with its magnification. The position where the objective lens is not set is displayed in a blank, and it can be visually confirmed that the objective lens can be added to this position. When an icon of a desired objective lens 50 is selected, the control unit 26 issues a command, and the objective lens 50 is automatically switched by rotating the objective lens turret. When the objective lens 50 is switched, the objective lens 50 is temporarily retracted so that it does not come into contact with the sample mounting portion 28 when the objective lens turret is rotated.
(Navigation column 70)

  Further, a zoom adjustment slider 69 for adjusting the magnification is provided at the lower part of the “lens selection” column 68. Further, a navigation column 70 is provided at the lower stage. The navigation column 70 forms a relative movement range display area indicating the entire range of the observable range, that is, the relative movement range SI of the sample placement unit 28. In the navigation column 70, the currently observed position is displayed in a cross-shaped grid GD. Thereby, the user can grasp | ascertain where the position currently observed is located in the whole. In this navigation column 70, a wide area image is displayed and selected. When the “navigation” button 71 in the navigation column 70 is pressed, the navigation screen 72 of FIG. 7 opens in a separate window, and the registered wide area image WI is displayed. That is, the navigation screen 72 functions as a wide area image display area WA that displays the wide area image WI. Conventionally, a wide area image often uses a small screen provided beside the enlarged image. However, in this state, there is a problem that it is difficult to determine details. In the present embodiment, by displaying the wide-area image WI in a large size using the navigation screen 72, it is easier to see and detailed display is possible, and the visual field searching operation can be facilitated. Further, an area corresponding to the enlarged image MI currently displayed in the enlarged image display area MA of FIG. 5 is displayed on the wide area image WI as a rectangular first frame BX1. As a result, the part currently being observed can be easily grasped on the wide area image WI, and can be used for moving the visual field. Further, when the first frame BX1 is moved by operating the mouse or the like on the screen of FIG. 7, the display of the enlarged image display area MA of FIG. 5 is updated so as to become the enlarged image MI of the corresponding area. The In this example, both the wide-area image WI and the enlarged image MI are bright-field images. However, by using the wide-area image WI as a bright-field image and the enlarged image MI as a fluorescent image, a high-resolution image is searched for a visual field. An environment suitable for fluorescence observation of magnifying observation with a monochrome image can be realized.

In this example, the wide area image display area WA is a separate window, but it is needless to say that it can be divided and displayed on the same screen as the enlarged display area. For example, in the screen of FIG. 5, the image display area can be divided into two and the enlarged image display area and the wide area image display area can be displayed side by side. Further, in the example of FIG. 5, the relative movement range SI of the sample placement unit 28 is displayed in the navigation column 70, but it is also possible to display a wide area image in a reduced size in the navigation column as in the past. Such setting of the display form can be arbitrarily designed according to user preferences and the like.
(Wide-area image switching means 76)

  Furthermore, a plurality of wide area images can be switched. A channel display field 73, a navigation image magnification display field 74, a history image display field 75, and a wide area image switching means 76 are provided in the upper part of the navigation screen 72 in FIG. The channel display field 73 displays the currently displayed channel number (CH4 in the example of FIG. 7). The navigation image magnification display field 74 displays the magnification of the wide area image WI displayed on the navigation screen 72 (200 times in the example of FIG. 7). The history image display field 75 displays the number of currently registered wide area images and the number of the wide area image being displayed. In the example of FIG. 7, 3/3 is displayed, indicating that the third wide area image WI is displayed among the three registered wide area images. The wide area image WI can be switched by an arrow button constituting the wide area image switching means 76. In the example of FIG. 7, the ← and → buttons are arranged, and by pressing these buttons, the registered wide area images WI are sequentially switched in ascending order and descending order of registration numbers. In this way, by enabling switching of a plurality of wide area images WI, it is possible to select and switch the field of view more flexibly without being limited to only one wide area image as in the past.

  Registration of the wide area image WI is performed by pressing the “image registration” button 78 provided at the lower right of the screen after adjusting to a desired field of view and magnification on the navigation screen 72 of FIG. Thereby, the image currently displayed on the navigation screen 72 is preserve | saved as a wide area image WI in a memory part. The wide-area image WI is recorded together with information on imaging conditions such as the type of observation method such as a bright field image and a fluorescent image, and the magnification at the time of imaging. Further, based on the coordinate information of each wide area image WI, the display of the navigation column 70 indicating the registered position of the wide area image WI in FIG. 5 is also automatically updated.

The wide area image WI is registered in advance and stored in the memory unit. The memory unit for storing a plurality of registered images can be provided in the control unit on the fluorescent microscope main body side, the image processing unit, or the like, or can be provided in the computer side. In the example of FIG. 2, the computer 58, which is a camera control PC, includes a memory unit. For example, by using a hard disk of a computer or the like, the configuration on the fluorescent microscope main body side can be simplified and the cost can be reduced. A computer 58 constituting the control unit performs various controls. Furthermore, the image being displayed on the navigation screen 72 in FIG. 7 can be simply saved without registering it as a wide area image. When a “save” button 79 arranged on the right side of the “image registration” button 78 is pressed, a dialog for naming and saving the image appears, and the image can be saved with a desired name.
(Relative movement range display area)

  As described above, on the screen of the fluorescence microscope operation program shown in FIG. 5, the relative movement range SI of the sample placement unit 28 is displayed in the navigation column 70, and the registered wide-area image is displayed on the relative movement range SI. The position is displayed as a point. When a plurality of wide area images are registered, a point is displayed at a position corresponding to each wide area image. Thereby, it is possible to look down on the position on the sample placement unit 28 where the wide area image is registered, and even when a plurality of wide area images are discretely registered, the mutual positional relationship is grasped and these are obtained. It becomes easy to distinguish.

  In this way, in addition to searching for a field of view of an enlarged image based on only one conventional wide area image, a plurality of wide area images and a relative movement range SI indicating the range of these wide area images are added to FIG. As shown, it enables more flexible visual field movement and visual field search. 8A shows the relative movement range SI (navigation column 70 in FIG. 5), FIG. 8B shows the wide area image WI2 (navigation screen 72 in FIG. 7), and FIG. 8C shows the enlarged image MI (in FIG. 5). Enlarged image display areas MA) are shown respectively. In this example, four wide area images are registered, and four points WI1 to WI4 indicating the positions of the wide area images are displayed in the relative movement range SI of FIG. When one of the points of the wide area images WI1 to WI4 is selected from the screen of FIG. 8A, the corresponding wide area image (WI2 in the example of FIG. 8B) is displayed as shown in FIG. It is displayed on the navigation screen 72. When the enlarged image MI is designated on the navigation screen 72, an enlarged image corresponding to the enlarged image display area MA of FIG. 5 is displayed, and a first rectangular shape is displayed in the corresponding area on the wide area image WI of FIG. The frame shape BX1 is displayed, and the position of the enlarged image MI can be confirmed on the wide area image WI.

  As described above, in order to determine the field of view of the fluorescence microscope, three steps of large, medium, and small regions are prepared and linked to each other, making it easy to grasp the relative positional relationship of the field of view. Thus, the field of view can be moved and selected very easily. That is, as compared with the conventional method in which the field of view is searched only by one wide area image, the field of view can be searched in a wider range by registering and switching a plurality of wide area images. Further, by displaying in the relative movement range display area the position of the plurality of wide area images in the relative movement range of the XY stage, the positional relationship between the plurality of wide area images can be easily organized. Select a wide area image without confusion and search for an appropriate field of view.

A plurality of registered wide area images can be displayed on the display unit. The image display area 61 of the display unit 24 is divided into a plurality of areas (for example, CH1 to CH4 shown in FIG. 4), and a plurality of wide area images are displayed on one screen by displaying each registered wide area image in the divided areas. I can confirm. For example, a plurality of images of a desired region can be extracted and displayed at the same time, as in the case of displaying a plurality of long-time observation images side by side with the time lapse function, thereby improving visibility and contributing to searching for a visual field.
(Digital zoom)

In addition to the optical zoom, the fluorescence microscope has a digital zoom function by image processing. When performing digital zoom, as shown in FIG. 9, an area corresponding to the field of view currently displayed in the enlarged image display area MA is displayed on the wide-area image WI both on the optical field of view and on the field of view zoomed in digitally. That is, when the area of the enlarged image MI is displayed in a rectangular shape on the wide area image WI, the rectangle of the optical image that is the original image is displayed in a broken line (first frame shape BX1) as shown in FIG. On the other hand, the enlarged display area is displayed with a solid line (second frame shape BX2) according to the digital zoom magnification. As a result, the original optical image having a high resolution and the area corresponding to the field of view enlarged by the digital zoom can be compared on the wide-area image WI, and the user can display with the digital zoom, that is, the resolution is lower than that of the optical image. Can recognize that In addition, since the corresponding position on the low-magnification wide-area image WI is correctly reflected, it is possible to confirm how much the resolution is lowered, that is, the relative relationship between the area of the original data and the visual field. When the current position is changed on the wide area image WI, a digital zoom image is displayed while the sample placement unit 28 is moving to the changed position. That is, the movable range of the XY stage is large, and a certain amount of time is required until the stage is physically moved to the designated target position. For this reason, an image corresponding to the visual field of the target position is cut out from the wide area image WI displayed on the navigation screen 72 (for example, a visual field range image displayed in the frame shape BX1), and this visual field range is moved while the stage is moving. A visual field range image obtained by enlarging and displaying the image with digital zoom is temporarily displayed. After the stage movement is completed, the display is switched to the display of the obtained enlarged image MI. Thereby, since the temporary digital zoom image is displayed even during the stage movement which takes time, it is possible to reduce the stress of waiting time for the user.
(Wide-area image composition means)

In addition to discretely registering and switching display of a plurality of wide area images, adjacent wide area images can be connected to be combined into a larger wide area image. A procedure for connecting such images will be described with reference to FIG. In the example of FIG. 10, four wide-area images WI <b> 5 to 8 are connected to generate a single synthetic wide-area image. First, as shown in FIG. 10A, the sample mounting unit 28 is moved, and four pieces of the wide area images WI5 to WI8 are imaged clockwise from the upper left. The wide area image WI is captured as a bright field image using an objective lens having the lowest magnification. At this time, the field of view is set so that adjacent wide area images WI slightly overlap each other. And as shown in FIG.10 (b), based on the overlap part of adjacent wide area image WI, these are connected by image processing. This composition processing is performed by the wide area image composition means of the image processing unit 30. As an algorithm for synthesizing an image, a known method can be used as appropriate, such as extracting a feature point from image data and performing matching. As a result, one large synthesized wide area image can be obtained. Previously, it was not possible to obtain a wide area image wider than the field of view obtained with the objective lens with the lowest magnification, but this method limits the minimum magnification of the objective lens by adding multiple wide area images. It is possible to obtain a wider image that is not performed.
(Correction coordinate position calculation means 77)

  The synthesized wide area image obtained in this way has a problem that the position coordinates of the wide area images are slightly deviated due to overlapping portions for joining. Therefore, the overlap amount when the wide-area image synthesizing unit synthesizes the image is held, and the coordinate position is corrected based on this value. This correction is performed by the corrected coordinate position calculation means 77 of the control unit 26. Specifically, in FIG. 10B, the coordinates of the upper left wide area image WI5 are used as a reference, and the wide area images WI6 to WI8 are corrected to coordinate positions obtained by subtracting the overlap amount with each image. Thereby, the position of the enlarged image MI can be correctly reflected on the synthesized wide area image.

  FIG. 11 shows an example of a user interface screen for performing such a wide area image composition. In the example shown in this figure, six wide images WI9 to WI14 are designated on the navigation screen 72, and the wide image synthesis is executed. While the wide area image composition is being executed, a progress bar is displayed in the progress dialog box 120 to indicate the progress of executing the shooting of each wide area image and the connection of the captured wide area images.

  Returning to the description of the microscope operation program in FIG. 5, the position of the objective lens 50 is displayed in a vertical bar shape on the right side of the navigation column 70 so that the relative position of the objective lens 50 can be visually grasped. Further, a “stage position storage” column 80 is provided in the lower stage. In addition, an illumination light amount adjustment column 82 is provided at the lower stage. In the example of FIG. 4, the halogen lamp that is the transmissive light source 36 is switched ON / OFF by a check box, and the ultrahigh pressure mercury lamp that is the epi-illumination light source 48 can adjust the amount of light by operating the slider.

Also, at the bottom of the operation area 62, a “lens retract” button for retracting the objective lens 50, an “analysis software” button for calling analysis software for performing various analyzes on the image, and when a microscope operation program is started There is provided a “go to initial menu” button for displaying the initial screen.
(Switch to fluorescence observation mode)

  A procedure for the observation condition setting means 23 to switch the imaging conditions will be described with reference to the flowcharts of FIGS. In these flowcharts, FIG. 12 illustrates an observation mode switching procedure when fluorescence observation is selected and FIG. 13 illustrates bright field observation. First, switching to the fluorescence observation mode will be described with reference to FIG. When the user designates fluorescence observation with the observation mode switching means 21 from the screen of FIG. 6 in step S1201, the halogen lamp of the transmissive light source 36 is turned off in step S1202, the fluorescent filter cube is inserted in step S1203, and the CCD in step S1204. The color filter 32 disposed in front of the camera is eliminated. Here, when the color filter 32 is a rotary type, the position of a hole without a filter is selected, and when the color filter is a liquid crystal type, it is mechanically retracted by an electric insertion / extraction mechanism or the like. In step S1205, the fluorescence imaging unit 22 for fluorescence observation is set. For example, peak metering or AE (automatic exposure) OFF is set. In step S1206, the image processing unit 30 is switched to the monochrome image processing mode. Specifically, the processing is such that color interpolation or color synthesis is not performed. In step S1207, the fluorescence excitation shutter is set to the open position. However, useless excitation should not be performed after the observation is completed, for example, when the operation is shifted to another observation condition or when another calculation is performed by a personal computer application.

  On the other hand, as shown in FIG. 13, when the user selects the bright field observation mode in step S1301, the fluorescent excitation shutter is set to the closed position in step S1302 so that unnecessary excitation is not performed. . In step S1303, the halogen lamp that is the transmissive light source 36 is turned on. In particular, a halogen lamp is preferably lit early because the rise of the lamp light amount is poor. In step S1304, the fluorescent filter cube is retracted so that transmitted light does not pass through the fluorescent filter. In step S1305, the color filter 32 is switched to the color mode. In the case of using the mechanical filter 33, the turret is rotated to sequentially switch RGB, and in the case of the liquid crystal filter 34, the filter is inserted into the optical path. In step S1306, the image processing unit 30 is switched to the color image processing mode. In the case of the mechanical filter 33, turret rotation control and color synthesis are linked, or in the case of the liquid crystal filter 34, applied voltage control and color synthesis are linked to synthesize a color image. In step S1307, the bright field imaging unit 25 is set. For example, average photometry of the color CCD camera and setting such as turning on AE are performed. Note that these procedures are merely examples, and the order of some processes can be changed.

Thus, in the fluorescence observation mode, high-sensitivity, high gradation, high-resolution monochrome photography can be performed, while in bright-field observation, high-resolution color photography having color information and comparable to 3CCD can be performed. By changing the observation mode, the user can automatically control the microscope, camera, image processing, color filter 32, etc. by only one operation of selecting the observation mode. This can save a lot of labor. Further, this configuration can be realized at low cost without adding special equipment.
(Phase difference observation mode)

  Furthermore, it goes without saying that this method is not limited to bright field observation but can also be applied to phase difference observation and differential interference observation. According to the phase difference observation, it is possible to observe a transparent sample with a contrast of light and dark using light diffraction and interference. Since it is not necessary to stain the sample as in bright field observation, living cells, cultured cells, blood, etc. can be observed as they are alive.

Hereinafter, the procedure for setting the phase difference observation mode will be described with reference to the flowchart of FIG. First, in step S1401, phase difference observation is designated. Here, the user selects the phase difference observation mode by the observation mode switching means 21. In step S1402, the fluorescence excitation shutter is set to the closed position so that unnecessary excitation is not performed. In step S1403, after turning on the halogen lamp that is the transmissive light source 36, in step S1404, the fluorescent filter cube is retracted so that the transmitted light does not pass through the fluorescent filter. In step S1405, the color filter 32 is switched to the color mode. In step S1406, the image processing unit 30 is switched to the color image processing mode. The processes in steps S1402 to S1406 are almost the same as those in FIG. In step S1407, the phase difference imaging unit is set. For example, the average photometry of the color CCD camera of the bright-field imaging unit 25, settings such as turning on AE, and applying an edge enhancement filter are performed. In step S1408, the fluorescent microscope is switched to a physical configuration for performing phase difference observation. Here, a phase ring stop corresponding to the magnification of the objective lens 50 is inserted. In the present embodiment, there are four types of Ph rings of the phase difference ring, PhL, Ph1, Ph2, and Ph3, which are properly used according to the magnification of the objective lens 50. In this way, in the phase difference observation mode, a phase difference image with high contrast can be obtained by forming a phase difference image based on the phase or optical path difference of the transparent sample or the reflection sample. Note that these procedures are merely examples, and the order of some processes can be changed.
(Differential interference observation mode)

  Differential interference observation is a method in which the difference in refractive index when light passes through an unstained sample and the optical path difference (difference in how light travels) due to the shape of the sample surface are changed to light and dark contrast for observation. . In differential interference observation, polarized light is used to generate parallel light beams that are slightly separated from each other and transmitted through a colorless and transparent sample, and then these light beams are interfered to obtain a three-dimensional image. This differential interference observation, like the phase difference observation, can observe a living sample, and is particularly suitable for a thicker sample than the phase difference observation. Applications include the observation of nerve and muscle fiber structures, cell division spindles, cell nucleus structures, other unstained thick samples, IC wafer surfaces, magnetic head polished surfaces, and crystal growth processes. Is done.

  Hereinafter, the procedure for setting the differential interference observation mode will be described with reference to the flowchart of FIG. First, in step S1501, differential interference observation is designated. Here, the user selects the differential interference observation mode by the observation mode switching means 21. In step S1502, the fluorescence excitation shutter is set to the closed position, the transmission light source 36 is turned on in step S1503, and the fluorescence filter cube is retracted in step S1504. The processes in steps S1502 to S1504 are substantially the same as those in FIGS. 13 and 14 described above. In step S1505, the color filter 32 disposed in the front stage of the CCD camera is eliminated. Here, when the color filter 32 is a rotary type, the position of a hole without a filter is selected, and when the color filter is a liquid crystal type, it is mechanically retracted by an electric insertion / extraction mechanism or the like. In step S1506, the image processing unit 30 is switched to the monochrome image processing mode. Specifically, the processing is such that color interpolation or color synthesis is not performed. In step S1507, an imaging unit for differential interference is set. For example, average metering of the monochrome CCD camera 22A constituting the fluorescence imaging unit 22, setting of turning on AE, applying an emboss filter, and the like are performed. Finally, in step S1508, the fluorescence microscope is switched to a physical configuration for performing differential interference observation. Here, an optical element (prism) and a polarizing plate (polarizer) are inserted on the light path on the light projecting side, and an optical element (prism) and polarizing plate (analyzer) are inserted on the optical path on the imaging side. In this way, differential interference observation is executed after switching to the differential interference observation mode. Note that these procedures are merely examples, and the order of some processes can be changed.

By adding an observation mode for performing such phase difference observation and differential interference observation to the observation mode switching means 21 and making it selectable, switching to these observation modes can be performed smoothly. Alternatively, depending on the configuration of the fluorescence microscope, in addition to differential interference observation, a special condenser is used to illuminate the sample from an oblique direction to observe the light scattered by the sample, and the polarization characteristics of the sample It is also possible to include switching to polarized light observation, etc., in which observation is performed by changing the contrast between light and dark and changing the color.
(Fluorescence synthesis observation mode)

  There is also a need to colorize monochrome images obtained by fluorescence observation. For example, a color image may first be used to confirm the position of cells in a sample before performing fluorescence observation. In addition, in order to judge whether the obtained signal is fluorescence or mere noise, it is possible to judge by looking at the hue of fluorescence, when observing a non-fluorescent specimen, or autofluorescence (background) which is a problem in fluorescence imaging. There are times when you want to take a color image when you want to distinguish it from a fluorescent signal. In such a case, a monochrome CCD camera cannot perform color photography, and adding a color CCD camera is expensive. Further, when only one camera port for connecting the camera is provided, it is necessary to remount the camera one by one. Therefore, a method of colorizing by inserting a color filter into a monochrome camera has been developed. After acquiring the color information of the sample by color imaging in bright field observation, the color information is superimposed on the luminance information of the fluorescent image, which is a monochrome image obtained by fluorescence observation, thereby forming a colored fluorescent composite image. This enables high-sensitivity, high-resolution, high-speed fluorescence observation while discriminating signals. However, in order to form a fluorescence composite image, it is necessary to change the camera image processing as well as the filter insertion / extraction, and it is troublesome to change according to the switching of the observation method. In addition, when a color filter is inserted, a focus shift, a visual field shift, a color shift, or a shift due to an individual difference of lenses occurs. Therefore, these adjustments are also necessary. Therefore, the condition for capturing such a fluorescence composite image is added to the observation mode switching unit 21 as a fluorescence synthesis observation mode, and is automatically set by the observation condition setting unit 23, thereby saving such labor. Thus, it is possible to easily obtain a fluorescence composite image.

  An example of a procedure for obtaining a color fluorescence composite image in the fluorescence synthesis observation mode will be described. First, the halogen lamp of the transmission light source 36 is turned off, the fluorescence filter cube is inserted, and the color filter 32 is switched to the color mode. Further, the fluorescence imaging unit 22 for fluorescence observation is set, the image processing unit 30 is switched to the color image processing mode, and the fluorescence excitation shutter is set to the open position. In this way, a configuration for obtaining a color fluorescence composite image can be obtained. When the fluorescence synthesis observation mode is selected, the observation condition setting unit 23 changes to the above imaging condition.

Furthermore, a plurality of images are taken while moving in the optical axis direction with a fluorescence microscope, that is, the height is changed, and a three-dimensional image (stack image) including three-dimensional information is obtained by combining these images. You can also. The stack image has a depth in the optical axis direction, and the surface state of the sample W can be observed. In particular, in multicolor fluorescence observation (multicolor), scanning images in the Z-axis direction are acquired in time series while automatically selecting the turret that switches the filter set 1 that matches the multi-stained sample. Can be observed three-dimensionally and deeply.
(Automated correction work)

In addition, when the observation condition setting unit 23 changes the imaging condition, various deviations that occur when the color filter 32 is inserted can be automatically corrected. Conventionally, such correction of correction has been performed manually, but by previously registering the correction amount for each filter, correction work such as fine adjustment of the lens position is performed together with switching of the observation mode. It is possible to achieve further labor saving and highly accurate observation.
(Band)

Furthermore, when the observation mode is switched, the data amount can be adjusted in consideration of a frame drop during image transfer. In general, color image data has a larger amount of information than monochrome image data. Therefore, in FIG. 2, the transfer path bandwidth limit, transfer speed, and frame rate when transferring image data from the image transfer unit 31 to the external computer 58 in FIG. It becomes a problem. That is, since the amount of data is relatively small in the case of a monochrome image, the amount of data communication can be suppressed within the range of the band, the image data can be transferred without causing a frame drop, and the display unit 24 can display the image in real time. Even if it has been updated, as soon as it is switched to a color image, the amount of data increases and the amount of data transferred within a limited band may be insufficient, resulting in frame dropping. In order to cope with this, the number of bits of the image data is adjusted according to the bandwidth of the data communication path and the frame transfer speed performed between the image transfer unit 31 and the computer 58. For example, when monochrome shooting is performed, transfer is performed with a high gradation (10 to 16 bits) of 1CH, and during color shooting, the gradation for each RGB is reduced to 8 bits or less to transfer an image, thereby reducing the bandwidth during moving image transfer to a certain amount. Suppressing and preventing frame dropping and lowering. This setting is set in advance on the apparatus or program side. Alternatively, the user can individually set according to the transfer speed or the like.
(Exposure adjustment function)

  Furthermore, in this embodiment, the function of adjusting the exposure time as long as possible according to the frame rate of the fluorescence imaging unit, automatically calculating the minimum necessary excitation light amount, and automatically adjusting the excitation light amount to this value Is provided. Thereby, the damage given to a sample can be reduced by continuing irradiating a strong excitation light to a sample. That is, conventionally, in order for the user to suppress the excitation light amount, the user manually adjusts the exposure time appropriately according to the purpose and conditions of the fluorescence observation to lower the excitation light amount. However, accurate adjustment has not always been made, and there is a problem that this work is very troublesome. Therefore, in the present embodiment, in order to automate such operations and obtain the brightness of the fluorescent image desired by the user, the exposure time / excitation of the control unit 26 is suppressed so as to suppress the excitation light amount as much as possible within the range of conditions. The light quantity control means 27 automatically sets the exposure time. This procedure will be described based on the flowchart of FIG.

  First, in step S1601, the user adjusts to an appropriate brightness under desired conditions. That is, the desired observation mode is set by the observation mode switching procedure, and the brightness of the excitation light source, the shutter speed of the fluorescence imaging unit, and the like are set by the observation condition setting means 23. The excitation light quantity at this time is set as a reference excitation light quantity. In the example of FIG. 3, the excitation light amount can be switched in six steps with a 20% width from 0 to 100% by switching of a filter that is an excitation light amount adjustment unit. Actually, it is not set to 0% (excitation light blocking), so the excitation light quantity is stored in five stages of 20% to 100%. In step S1602, the exposure adjustment function of the exposure time / excitation light quantity control means 27 is turned ON. In step S1603, the current imaging condition of the fluorescence imaging unit is acquired. That is, the frame rate (shutter speed or exposure time), the number of pixels, and the like at the time of capturing a fluorescent image are held as camera photographing conditions of the monochrome CCD camera.

  In step S1604, a frame rate is calculated from the imaging conditions. The frame rate varies depending on the observation mode. Examples of the observation mode include binning, image integration, and fluorescence synthesis observation. Binning is an observation mode that can brighten an image, although the number of pixels is reduced by adding light amounts at a plurality of adjacent pixels and treating them as one pixel. Image integration is an observation mode in which a plurality of fluorescent images taken continuously are overlapped for each pixel to increase the amount of light. The fluorescence synthesis observation is an observation mode in which fluorescence images corresponding to RGB colors are acquired and a color image obtained by combining these fluorescence images is obtained. Further, it varies depending on the presence or absence of readout for imaging only a partial area of the fluorescent image, that is, the number of pixels of the fluorescent image. As an example, Table 1 shows changes in the frame rate when the number of pixels is 1360 pixels × 1024 pixels and the standard frame rate is 15 frames / second.

  These frame rates are calculated based on standard observation. For example, if partial reading is performed, high-speed reading is required as the number of pixels decreases. If the number of pixels is set to 1000 pixels × 400 pixels, (1360 × 1024) ÷ (1000 × 400) times = about 3.48 times. 15 frames / second × 3.48 = 52 frames / second. In this case, if an excitation light quantity is 80% transmitted by partial reading and a sufficient fluorescent image is obtained at an exposure time of 10 ms, the exposure time is 1/52 frames when the exposure adjustment function is executed using this as a reference light quantity. Since /sec=19.1 ms, the amount of excitation light can be reduced from 80% of the standard observation to 80% / (19.1 ms / 10 ms) = 41.8%.

  If 2 × 2 pixels are set in the sub-array (partial area) by binning, 1360 pixels × 1024 pixels of the original image correspond to 680 pixels × 512 pixels, and thus the calculation can be performed in the same manner as the partial reading. Further, in the image integration, the frame rate is lowered by the number of integration, and for example, in the case of 5 image integration, (15 frames / second) / 5 sheets = 3 frames / second. Furthermore, in the fluorescence synthesis observation, as shown in FIG. 3, a color filter is inserted into a monochrome camera to make a color. As shown in FIG. 3 (a), the color filter is inserted in such a manner that R, G, B, and transmission filters are provided in a rotating turret shape, and these are mechanically switched. As shown, there is a method of switching R, G, B and transmission spectral characteristics or switching transmission according to an applied voltage using a liquid crystal RGB filter. In either case, since the fluorescent image for each color of RGB is synthesized, the frame rate is 1/3, and 15 frames / second is 5 frames / second.

  Further, these shooting conditions can be combined, and the frame rate can be calculated by the same calculation as described above. Table 2 shows an example of a change in the frame rate when the observation conditions are combined.

  When the frame rate is determined in this way, in step S1605, the exposure time of the fluorescence imaging unit is adjusted according to the frame rate. That is, the exposure time is extended until the time allowed by the frame rate is full. For example, when the user observes fluorescence with the excitation light amount set to 80% transmission and the exposure time set to 10 ms in the standard observation, and this value is set as the reference excitation light amount assuming that the brightness of the fluorescent image is sufficiently obtained, When the mode is changed to partial readout, the number of pixels is set to 1000 pixels × 400 pixels, and the exposure adjustment function is applied, the frame rate is 52 frames / second as shown in Table 1, so the exposure time is 1/52 = It can be extended to 19.2 ms. With the exposure time so far, the frame rate required for real-time display on the display unit can be sufficiently accommodated. In step S1606, the amount of excitation light is adjusted by adjusting the exposure time. In the above example, by extending the exposure time from 10 ms to 19.2 ms, the amount of excitation light can be reduced from 80% transmission, and specifically 80% / (19.2 / 10) = The excitation light amount can be reduced by the excitation light amount adjustment unit up to 41.7%. In the example of FIG. 2, the excitation light amount can be adjusted in steps of 20%, so it is set to 40% closest to the settable excitation light amount. In this way, the amount of excitation light can be suppressed as much as possible while maintaining the brightness necessary for the fluorescent image, and the load on the sample can be reduced.

  Although the exposure time is set to be long within the range allowed by the frame rate, it is set so as not to exceed the frame rate. For example, when the excitation light quantity is set discretely, such as in increments of 20%, it may not be possible to set an exposure time that matches the calculated exposure time. The discrete value selection method in this case adjusts the excitation light quantity so that the exposure time does not exceed the frame rate. This can avoid frame dropping.

As described above, since the frame rate is determined according to the viewing condition, the frame rate is necessary to maintain the brightness from the reference state set by the user to the desired brightness under the desired viewing condition. By automatically adjusting the excitation light and exposure time required for maintenance, that is, the shutter speed of the CCD camera, and providing a function to reduce the amount of received light accordingly, the amount of excitation light irradiated to the sample can be minimized. This enables stable fluorescence observation.
(Camera settings)

  The above settings such as binning and partial reading (ROI) are performed from the setting screen shown in FIG. FIG. 17 shows a user interface screen in a state where the “camera setting” tab 104 is selected in the operation area 62 of the fluorescence microscope operation program. When the “camera setting” tab 104 is selected, a “color / monochrome setting” field 130, a “camera setting” field 140, an “autofocus setting” field 150, and a “white balance” field 160 are displayed in the operation area 62. The In the “color / monochrome setting” column 130, one of monochrome 8 bit, monochrome 12 bit, and color is selected with a radio button. In addition, as an expansion method for expanding the number of data bits at the time of storage from 12 bits to 16 bits, either top (standard) or bottom alignment can be selected.

  In the “Camera Setting” field 140, a preview mode (ultra-high speed / high speed / standard), gain (0 dB / 6 dB / 12 dB / 24 dB), photometry method (average photometry / peak photometry, full screen / partial), etc. are displayed in a pull-down menu. Selected. Furthermore, “OFF”, “binning (2 × 2/4 × 4/8 × 8)”, and “ROI (small / medium / large)” can be selected as “binning / ROI”. In this example, partial reading (ROI) is selected from any one of small / medium / large specified values, but partial reading may be performed by designating an arbitrary area on the screen.

Furthermore, as the shooting conditions, select the number of full-size pixels (1360 x 1024/680 x 512/2720 x 2048/4080 x 3072) and multi-color & Z-stack shooting order (multi-color → Z-stack / Z-stack → multi-color) it can. In the “auto focus setting” field 150, the focus area display is turned ON / OFF and full screen / partial setting. In the “white balance” field 160, in addition to the push set execution and initialization, the push set area display is turned ON. / OFF, large / medium / small when display is ON, and RGB balance at the time of manual setting can be set.
(XY coordinate storage position display function)

Furthermore, a desired area on the wide area image can be stored as the XY coordinate position of the XY stage and displayed on the wide area screen. A plurality of XY coordinates can be stored. In the example shown in FIGS. 18, 19 and 20, three desired areas are stored, and each storage area is shown on the wide area image WI on the navigation screen 72 by thin dotted frames PX1 to PX3. When storing a plurality of areas, they can be easily distinguished by displaying each dotted frame with a registration number. The display of the XY coordinate storage position is switched by turning ON / OFF the check in the “stage storage position” column 110 provided in the lower stage on the navigation screen 72 of FIGS. In this way, the relative positions of a plurality of registered areas can be confirmed. In particular, it is suitable for position confirmation when sequentially photographing while moving a plurality of preset positions. In these drawings, FIG. 18 shows a state in which the navigation screen 72 is opened and the wide-area image WI is displayed while the enlarged image MI of the bright field image is displayed in the image display area 61 of the operation screen 60, and FIG. FIG. 20 illustrates a state in which the navigation screen 72 is enlarged, and FIG. 20 illustrates a state in which the wide-area image WI of the fluorescent image is displayed as another navigation screen 72.
(Shooting history display function)

  In addition, an area corresponding to an enlarged image displayed in the past can be displayed on the wide area screen. In the example shown in FIG. 18 described above, the enlarged image MI currently displayed in the enlarged image display area MA of the image display area 61 is displayed in a rectangular shape at a corresponding position on the wide area image WI displayed on the navigation screen 72. 1 is displayed as a frame shape BX1, and a region displayed in the past is displayed as a history region in a whitish frame shape HX. The history area HX displayed by the shooting history display function can store and display all the positions of the enlarged images displayed in the enlarged image display area MA, and can also store only the enlarged images registered as images. The shooting history display can be switched by turning ON / OFF the check in the “shooting history” column 112 provided in the lower part of the navigation screen 72 of FIGS. 19 to 20. By switching ON / OFF of the shooting history display in this manner, the past history can be easily confirmed from the wide area screen, and can be used as a reference when searching for a new field of view. You can also delete the latest history or delete all history. In the example of the navigation screen 72 shown in FIGS. 19 to 20, when the “delete latest history” button 114 provided in the lower row is pressed, the history is sequentially deleted from the latest one, and the “delete all history” button 116 is pressed. All the history is deleted.

Further, by changing the frame-shaped line type in addition to or in addition to the frame-shaped line type, visibility is improved, and a plurality of types of frames can be easily distinguished. In the examples shown in FIGS. 18, 19, and 20, the thin dotted frame is displayed in yellow, and the thick solid line is displayed in red.
(Simple scale function)

The enlargement magnification displayed on the navigation screen 72 can be arbitrarily adjusted, and the current magnification is displayed in the navigation image magnification display field 74. The magnification can be arbitrarily designated by the user, or can be automatically adjusted to be displayed according to the magnification of the registered wide area image. In addition, the example of the navigation screen 72 in FIGS. 19 to 20 includes a simple scale function that displays an appropriate scale SC so as to overlap the wide area image on the navigation screen 72. The scale SC is automatically calculated from the field-of-view range of the image display area 61, that is, the magnification of the selected objective lens, and is displayed on the image with an optimal length dimension. In the example of FIGS. 19 to 20, when the “simple scale” column 118 provided in the lower row is checked, the wide area image is displayed on the navigation screen 72 at the optimum scale according to the magnification of the wide area image being selected. The
(Confocal microscope)

Furthermore, the present invention can be used for confocal microscopes as well as optical microscopes and digital microscopes using optical lenses. Since the confocal microscope scans the surface of the measurement object using laser light, it is also called a confocal laser microscope, a confocal scanning microscope, or the like. A confocal image obtained with a confocal microscope is also called a laser image. In a confocal microscope, light from a measurement object such as a sample or workpiece is received by a light receiving element via a confocal optical system, and an ultra-deep image of a sample (an image with a very deep depth of focus) based on the amount of received light. ) And height distribution. When the relative distance between the sample placed on the stage and the objective lens is changed in the direction of the optical axis, the amount of light incident on the light receiving element via the confocal optical system, that is, the amount of received light changes, and the surface of the sample is changed. The amount of light received is maximized when the subject is in focus. Therefore, the height information of the surface of the sample is calculated from the relative distance between the sample and the objective lens when the maximum amount of received light is obtained, and the height distribution of the surface of the sample is acquired by scanning the surface of the sample with light. be able to. The acquired height distribution is displayed on the screen of the display device by, for example, three-dimensional display. Alternatively, the height distribution replaced with a luminance distribution or a color distribution is displayed on the screen. A CRT (Cathode Ray Tube) or LCD (Liquid Crystal Display) is used as a display device, and a confocal microscope is configured by connecting a control controller, a display device, a console, and the like to the confocal microscope. Also, by combining the information on the amount of light received when each point (pixel) on the sample surface is in focus (that is, the maximum luminance information of each pixel), a black and white image of the sample surface with a very deep focal depth can be obtained. Can do. This image is a so-called ultra-deep image. Furthermore, by separating the light from the sample irradiated with white light from the confocal optical system and receiving it with a color imaging device, a color image of the sample surface in the same range as the ultra-deep image can be obtained. Unlike the ultra-deep image, this color image has a shallow depth of focus, but by connecting pixel data at the in-focus position of the color image based on the height distribution information, a color image (color It is also possible to obtain a color image with a deep focal depth by performing a synthesis process that replaces the luminance signal of the color peak image with the luminance signal of the ultra-deep image.
(electronic microscope)

  Furthermore, the present invention can also be applied to an electron microscope using an electron lens. An electron microscope freely refracts the traveling direction of electrons and is an electro-optical design of an imaging system such as an optical microscope. Electron microscopes include a transmission type that forms an image of electrons that have passed through a sample or specimen using an electron lens, a reflection type that forms an image of electrons reflected on the sample surface, and scans the sample surface with a focused electron beam. There are a scanning electron microscope that forms an image using secondary electrons from each scanning point, and a surface emission type (field ion microscope) that forms an image of electrons emitted from a sample by heating or ion irradiation.

  Scanning Electron Microscopy (SEM) is a secondary electron detector or backscattered electron detector that detects secondary electrons and reflected electrons generated when a target sample is irradiated with a thin electron beam (electron probe). And the like, and are displayed on a display screen such as a cathode ray tube or an LCD to mainly observe the surface form of the sample. On the other hand, a transmission electron microscope (TEM) transmits an electron beam through a thin film sample, and at that time, electrons scattered and diffracted by atoms in the sample are obtained as an electron diffraction pattern or a transmission electron microscope image. Can mainly observe the internal structure of the substance. When an electron beam is irradiated onto a solid sample, it is transmitted through the solid by the energy of the electrons. At that time, due to the interaction with the nuclei and electrons that make up the sample, elastic collision, elastic scattering, and energy loss occur. Causes elastic scattering. By inelastic scattering, electrons in the shell of the sample element are excited, X-rays are excited, secondary electrons are emitted, and the corresponding energy is lost. The amount of secondary electrons emitted varies depending on the angle of collision. On the other hand, the reflected electrons scattered back by elastic scattering and emitted again from the sample are emitted in an amount specific to the atomic number. The SEM uses these secondary electrons and reflected electrons. The SEM irradiates a sample with electrons and detects emitted secondary electrons and reflected electrons to form an observation image.

Enlarged image observation apparatus, enlarged image observation how the present invention may, for example, by reacting a patient's serum and cell nuclei were observed anti nuclear anti body with a fluorescence microscope by adding a fluorescent label, positive by fluorescence Anakakuanatai It can be used for fluorescent antibody testing to determine negative.

It is a conceptual diagram which shows a mode that the area | region of an enlarged image is displayed on a wide area image. It is a block diagram which shows the fluorescence microscope system which concerns on one embodiment of this invention. It is a schematic diagram which shows the structural example of the color filter used for a monochrome camera. It is an image figure which shows the user interface of the operation screen of a fluorescence microscope operation program. FIG. 5 is an image diagram showing a state in which a bright-field image of CH4 in FIG. 4 is enlarged and displayed. It is an image figure which shows the channel setting screen which performs the setting for every channel. It is an image figure which shows the example which displays a wide area image on a navigation screen. It is a conceptual diagram which shows the correspondence of the display of a relative movement range, an enlarged image, and a wide area image. It is a conceptual diagram which shows a mode that the area | region of the enlarged image by digital zoom is displayed on a wide area image. It is a conceptual diagram which shows a mode that a several wide area image is synthesize | combined. It is an image figure which shows an example of the user interface screen which performs a wide area image composition. It is a flowchart which shows the procedure which switches an imaging condition when fluorescence observation is selected. It is a flowchart which shows the procedure which switches an imaging condition when bright field observation is selected. It is a flowchart which shows the procedure which performs the setting of phase difference observation mode. It is a flowchart which shows the procedure which performs the setting of differential interference observation mode. It is a flowchart which shows the procedure which adjusts exposure time automatically. It is an image figure which shows the user interface of the setting screen of a fluorescence microscope operation program. It is an image figure which shows the state which opened the navigation screen from the operation screen of the fluorescence microscope operation program. It is an image figure which shows the state which expanded the navigation screen of FIG. It is an image figure which shows the example of another navigation screen.

DESCRIPTION OF SYMBOLS 1 ... Filter set 12 ... Excitation filter; 14 ... Dichroic mirror 16 ... Absorption filter; 18 ... Filter switching part; 20 ... Switching setting part 21 ... Observation mode switching means; 21A ... Pull-down menu 22 ... Fluorescence imaging part; Camera 23 ... Observation condition setting means; 24 ... Display part 25 ... Bright field imaging part; 26 ... Control part; 27 ... Exposure time / excitation light quantity control means 28 ... Sample placement part; 29 ... Eyepiece 30 ... Image processing part; DESCRIPTION OF SYMBOLS 31 ... Image transfer part 32 ... Color filter; 33 ... Mechanical filter; 34 ... Liquid crystal filter 35 ... Transmission illumination lens; 36 ... Transmission light source 38 ... Movement mechanism; 37 ... Transmission illumination drive power supply 46 ... Dark room space; Light source; 49 ... Excitation light quantity adjustment unit 50 ... Objective lens; 52 ... Imaging lens; 53 ... Observation switching means 54 ... Collector lens; 6 ... Filter holder 58 ... Computer; 59 ... Input device 60 ... Operation screen; 61 ... Image display area; 62 ... Operation area 64 ... "Selection of observation method"field; 65 ... Channel button 66 ... Observation magnification display field; "Channel setting" button 68 ... "Lens selection"column; 69 ... Zoom adjustment slider 70 ... Navigation column; 71 ... "Navigation" button 72 ... Navigation screen; 73 ... Channel display column 74 ... Navigation image magnification display column; Image display field 76 ... Wide-area image switching means; 77 ... Correction coordinate position calculation means 78 ... "Image registration"button; 79 ... "Save" button 80 ... "Stage position storage"field; 82 ... Light quantity adjustment field; Screen 91 ... Transmission illumination halogen state display column; 92 ... Fluorescence excitation shutter state display column 93 ... Pseudo color table Column: 94 ... Display color selection column 95 ... Pseudo color ON / OFF display column; 96 ... Comment display column 95A ... Status display column; 95B ... Composite adjustment column 97 ... "Use as overlay display"column; 98 ... Common setting column; 99 ... "Setting" button 100 ... Fluorescence microscope main body 102 ... "Microscope operation"tab; 104 ... "Camera setting" tab 110 ... "Stage storage position"field; 112 ... "Photographing history" field 114 ... "Latest history deletion" button 116 ... "Delete all history" button 118 ... "Simple scale" column 120 ... Progress dialog box; 130 ... "Color / monochrome setting" column 140 ... "Camera setting"column; 150 ... "Auto focus setting" column 160 ... "White balance" column WI, WI1 to WI14 ... wide area image MI ... enlarged image; WA ... wide area image display area; MA ... enlarged image table Indication area SI ... relative movement range; BX ... frame shape; BX1 ... first frame shape; BX2 ... second frame shape PX1 to PX3 ... storage area; HX ... history area WK ... sample; SC ... scale; GD ... grid

Claims (14)

An enlarged image observation apparatus that enlarges and displays an observation object,
An optical imaging system for acquiring an image of an observation object;
A wide area image display area for displaying a wide area image obtained by imaging the observation object at a low magnification, and a display unit having an enlarged image display area for displaying an enlarged image obtained by imaging at a high magnification;
Get the wide area image in advance, a memory unit for holding,
A sample placement unit for placing an observation object;
A moving mechanism for moving the sample mounting portion relative to the optical imaging system;
In the wide area image display region, and the wide area image synthesizing means receives designation of a plurality of images, said moving the moving mechanism you combining the images of areas adjacent captured for executing an image synthesis ,
Equipped with a,
An area designated on the wide area image displayed in the wide area image display area is displayed as an enlarged image in the enlarged image display area, and a position corresponding to the enlarged image on the wide area image is displayed in a frame shape in the wide area image display area. in displayable and to such Rukoto enlarged image observation apparatus according to claim. The enlarged image observation apparatus according to claim 1 , further comprising:
In combined synthetic image, based on the position obtained by synthesizing by that between images on the wide area image synthesizing means, and correcting the coordinate position calculating means for correcting the coordinate position,
An enlarged image observation apparatus comprising: The enlarged image observation apparatus according to claim 2 ,
When generating a synthetic image by synthesizing the wide area image synthesizing means 4 or more images, based on the coordinate position of the image the correction coordinate position calculating means you located in the upper left, the coordinate position corrected An enlarged image observation apparatus characterized by performing an operation.   An enlarged image observation apparatus according to any one of claims 1 to 3,
  The magnified image observation device is a fluorescence microscope,
  The magnified image observation apparatus further includes an observation mode,
    A fluorescence observation mode for acquiring a fluorescence image;
    A bright field observation mode for acquiring a bright field image;
An enlarged image observation apparatus comprising observation mode switching means for switching between the two.   The enlarged image observation device according to claim 4,
  The display unit
    A channel selection screen for selecting the observation method;
    Channel setting screen for setting for each channel
Is provided,
  The magnified image observation apparatus further includes observation condition setting means for automatically switching the setting of the corresponding channel to the setting corresponding to the selected observation mode when the observation mode is selected for each channel. ,
  An enlarged image observation apparatus configured to display an image captured in an observation mode corresponding to a channel selected on the display unit in the enlarged image display area.   The enlarged image observation apparatus according to claim 5,
  The magnified image observation apparatus, wherein the magnified image display area can simultaneously display different images corresponding to each channel.   The enlarged image observation apparatus according to any one of claims 4 to 6,
In addition to the setting for each channel, the display unit displays a common setting column for setting common setting items for all channels.   The enlarged image observation apparatus according to any one of claims 4 to 7,
  The fluorescence observation mode is a mode for observing a monochrome fluorescence image with a monochrome camera,
  An enlarged image observation apparatus comprising a pseudo color display function for coloring a pseudo image to a monochrome image obtained by the monochrome camera.   An enlarged image observation apparatus according to any one of claims 1 to 8,
  The magnified image observation device is a confocal microscope,
  The confocal microscope is configured to obtain a confocal image, an ultra-deep image, a color ultra-deep image obtained by synthesizing a color image with an ultra-deep image, or a height information of a sample. apparatus. An enlarged image observation apparatus according to any one of claims 1 to 9 ,
It has a digital zoom function that can display enlarged images by image processing .
Further, when the enlarged image is digitally zoomed, the frame-like size on the wide area image is reduced or enlarged according to the area displayed in the digital zoom image displayed after the digital zoom. The enlarged image observation apparatus according to claim 10 ,
On the wide area image displayed in the wide area image display area, the position corresponding to the original image before the digital zoom of the enlarged image is displayed in a first frame shape,
The corresponding size of the digital zoomed enlarged image is displayed in the second frame shape,
An enlarged image observation apparatus, wherein the first frame shape and the second frame shape are displayed in different modes. An enlarged image observation apparatus according to any one of claims 1 to 11 ,
The enlarged image observation apparatus includes an imaging system that captures a fluorescent image and a bright field image,
A wide area image to be displayed in the wide area image display area is a bright field image,
An enlarged image observation apparatus, wherein an enlarged image displayed in the enlarged image display area is a fluorescent image. The enlarged image observation apparatus according to any one of claims 1 to 12 ,
A magnification observation apparatus, wherein a digital zoom image of a registered wide-area image is displayed on the display unit during movement of the sample placement unit. An enlarged image observation method for magnifying and displaying an observation object,
A step of moving a sample mounting portion for mounting an observation object relative to the optical imaging system and capturing a plurality of wide-area images of adjacent regions;
Combining a plurality of these wide-area images on the wide-area image display area of the display unit so that the boundary portions are matched by image processing , and creating a larger wide-area image in an adjacent area ;
In the synthesized synthesis image, a step of based on the location information by connecting the wide area image with each other, to correct the coordinate position of the entire composite image,
An enlarged image observation method comprising: