JP5347051B2 - Magnification observation apparatus, magnification observation method, and magnification observation program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a magnification observation device and the like allowing color observation, while picking up an image of excellent resolution feeling. <P>SOLUTION: The magnification observation device includes: a color imaging element (imaging element 12) for obtaining a color image; and synthesizing means (image synthesizing means 85), which obtains a color observation image by synthesizing the color information of the color image obtained from the color imaging element (imaging element 12) and the luminance information of a monochrome image obtained from the pixel information corresponding to the color with the shortest wavelength among pieces of pixel information from the color imaging element (imaging element 12). <P>COPYRIGHT: (C)2013,JPO&amp;INPIT

Description

The present invention relates to a magnification observation apparatus such as a digital microscope or a microscope that captures and displays an enlarged image, a magnification observation method, a magnification observation program, and a computer-readable recording medium.

An optical microscope using an optical lens, a digital microscope, or the like is used as a magnification observation apparatus that magnifies and displays a sample such as a minute object or a subject such as a workpiece. A digital microscope is a CCD, CMOS, or the like that electrically reads reflected light or transmitted light from an observation object fixed to an observation object fixing unit incident through an optical system for each pixel arranged in a two-dimensional shape. An image received by the image sensor and electrically read is displayed on a display unit such as a display (for example, Patent Document 1).

In such a magnification observation apparatus of an optical system, the resolution is in principle proportional to the wavelength of illumination light. For this reason, a magnification observation apparatus has been developed that improves the resolution of the optical system by using short-wavelength light, such as an ultraviolet microscope, and enables finer observation.

Further, the larger the number of pixels of the image sensor, the more detailed display becomes possible. Particularly in recent years, there has been a growing demand for higher image quality, and CCDs of 2 million pixel class or 8 million pixel class, for example, having a large number of pixels are also used for the image sensor. Such an image sensor such as a CCD is a sensor that detects the intensity of light reception, and cannot identify a color. For this reason, in a conventional CCD, color information is obtained by applying a single color filter in advance to the CCD elements constituting each pixel and combining them with signals obtained from adjacent pixels with other color filters. Yes. In such a single plate type, so-called 1CCD system, as shown in FIG. 1, a Bayer arrangement is performed so that a plurality of CCD elements to which filters of different colors are applied are adjacent to each other. In the Bayer array, when attention is paid to four adjacent pixels, the number of RGB pixels is generally 1: 2: 1. In this configuration, since only one color signal can be acquired by the image sensor of each pixel, color information cannot be acquired for each pixel. Therefore, in order to obtain color information for each pixel, (1) a method for calculating other color information for each pixel based on different color information obtained in surrounding pixels, and (2) a small movement of the CCD A so-called pixel shift method (for example, Patent Document 2) that obtains RGB light reception signals in pixel units and obtains color information in pixel units by combining a plurality of images, and (3) RGB in pixel units. A so-called 3CCD system (for example, Patent Document 3) of a three-plate type in which a CCD is arranged is known.
JP 2002-135648 A Japanese Patent Laid-Open No. 58-111580 Japanese Patent Laid-Open No. 1-123580

On the other hand, in the magnifying observation apparatus, observation is performed by changing the wavelength of illumination light applied to the sample. Different observation images can be acquired by changing the wavelength of the irradiation light. For example, when observing a green flexible substrate, if the illumination light is red, the illumination light is transmitted inside, and an observation image like a perspective view is obtained. On the other hand, when illuminated with blue light, the illumination light is not transmitted to the inside, so a scratch pattern on the surface is observed. By changing the illumination light in this way, it is possible to produce a change in appearance such as selectively observing only the inside of the sample or only the surface layer or improving the contrast of the sample.

However, when the observation is performed by changing the wavelength of the illumination light, if the observation is performed using a normal two-dimensional color image pickup device such as a CCD, the effective number of pixels decreases due to the influence of the Bayer arrangement of the image pickup device. There was a problem. As shown in FIG. 1 described above, the ratio of the number of RGB pixels in a normal image sensor is 1: 2: 1. For example, when blue illumination light is used, the effective number of pixels is white. It becomes 1/4 of the time of photographing using illumination light. For this reason, when observation is performed by changing the wavelength of illumination light, only extremely low-resolution images can be obtained as compared with observation using white light. As a result, even if it is attempted to improve the resolution by illuminating with the illumination light having the short wavelength described above, the effective number of pixels of the image pickup device is reduced, and it is difficult to obtain an improvement in the resolution of the entire system.

On the other hand, when observation is performed by changing the wavelength of the illumination light, if the observation is performed using a monochrome imaging device, the above-described reduction in the effective number of pixels can be prevented. However, this method has a problem that only a monochrome image can be obtained even when observed using white light. In order to perform color observation in this case, a method of synthesizing a captured image by switching RGB filters at high speed is conceivable. However, this method has a problem that convenience is lowered as compared with the case where a color image manipulation element is used, for example, the mechanism is complicated and the switching operation is required, so that the frame rate is lowered.

In addition, a method of acquiring a high-resolution color image by performing pixel shift in the so-called 3CCD method and combining RGB images has been proposed (Patent Document 3). However, this method has a problem that it is difficult to correct chromatic aberration. In other words, since the refractive index of the optical material constituting the lens varies depending on the wavelength of light, the position of the image point shifts depending on the wavelength, resulting in a blurred image, or the focal length varies depending on the wavelength, so that it is around the image surface. As it approaches, chromatic aberration occurs such that the image position shifts and color fringing occurs. For this reason, for example, it is necessary to perform correction with a compound lens combining various optical materials having different refractive indexes and wavelength dependencies so as to make the image formation point constant by correcting the chromatic aberration of each color of RGB. There was a problem of requiring. In addition, 3CCD is expensive, and there is a problem that the size of the image sensor increases.

The present invention has been made to solve such conventional problems. A main object of the present invention is to provide an enlargement observation apparatus, an enlargement image observation method, an enlargement image observation program, and a computer-readable recording medium that enable color observation while allowing an image excellent in resolution to be captured. There is.

Means for Solving the Problems and Effects of the Invention

In order to achieve the above object, a first magnification observation apparatus has a plurality of types of pixels corresponding to different wavelength regions by a color filter, and a single-plate color imaging device for obtaining a color image, Color information of a color image obtained from a color image sensor and a monochrome image obtained from pixel information of a pixel corresponding to a short wavelength range by the color filter among the pixel information from the color image sensor by performing pixel shifting Combining means for combining the luminance information and obtaining a color observation image can be provided. Accordingly , a high-resolution color image can be synthesized in a short time while using a camera using a normal color image sensor .

In the second magnification observation apparatus, the illumination light selection unit is arranged between the illumination light source and the sample and includes a plurality of illumination filters that extract light emitted from the illumination light source as illumination light in different wavelength ranges. Filter means and filter selection means for selecting a desired illumination filter from a plurality of illumination filters provided in the filter means can be included. Thereby, the optical path shift control unit controls the optical path shift unit so that the image sensor corresponding to the illumination light in the wavelength band selected by the filter selection unit acquires the light reception signal in units of pixels, and according to the illumination light. The light reception signal can be acquired in units of pixels, and a high-definition observation image can be captured.

Further, in the third magnifying observation device, the filter means includes, as an illumination filter, at least a transmission filter that transmits substantially the entire wavelength range of light emitted by the illumination light source and a blue filter that transmits the wavelength range of the blue component. Can do. As a result, a color observation image can be captured by using a transmission filter, while a high resolution observation image can be captured by using a blue filter because of a narrow band and a short wavelength. Moreover, if these are combined, a high-resolution color observation image can be obtained.

Furthermore, the fourth magnification observation apparatus further includes a plurality of images captured in the same field of view of the sample using display means capable of displaying an observation image captured by the image sensor and a plurality of illumination filters provided in the filter member. From the state in which the observation image is displayed on the display means at the same time, the image selection condition including the image selection means capable of selecting one and the type of illumination filter used for imaging the observation image selected by the image selection means And imaging condition setting means for setting as follows. As a result, the user can automatically set necessary observation conditions in subsequent observations by selecting a desired observation image from among observation images actually captured using a plurality of illumination filters. In particular, even a user who is not familiar with the items to be set as the observation conditions can be set based on the actual image, so that the setting work can be easily performed regardless of the skill level of the operation.

Furthermore, the fifth magnification observation apparatus can further include an image synthesis unit that synthesizes at least two observation images obtained by imaging the same sample using different illumination filters. Thereby, for example, a synthesized image obtained by synthesizing a plurality of observation images can be acquired by synthesizing a color image with a high-resolution monochrome image and synthesizing a high-resolution color image.

Furthermore, in the sixth magnification observation apparatus, at least two observation images synthesized by the image synthesizing unit use a transmission filter in which one observation image transmits substantially the entire wavelength range of light irradiated by the illumination light source. The optical path shift means is operated and the color image of the white wavelength range captured by all the image sensors at each pixel position, and the other observation image is transmitted to the optical path using a blue filter that transmits the wavelength range of the blue component. By operating the shift means, it is possible to obtain a high-resolution observation image in the blue wavelength region captured by the blue image sensor at each pixel position. Thereby, a color image can be combined with a high-resolution single color observation image captured with illumination light having a short wavelength of blue wavelength, and a high-resolution color image can be obtained.

Furthermore, the seventh magnification observation apparatus further selects a blue filter by the filter selection means when the observation image is synthesized by the image synthesis means, and the optical path shift control means operates the optical path shift means to cause blue illumination light. An image of a blue image sensor corresponding to is captured at each pixel position to obtain a high-resolution observation image in the blue wavelength range, a transmission filter is selected by the filter selection means, and the optical path shift control means operates the optical path shift means The color observation image obtained by capturing all image sensors at each pixel position and obtaining the color observation image in the white wavelength range and the image synthesis means combine the color information of the color observation image with the high-resolution observation image, An automatic synthesizing unit that automatically performs an operation of acquiring a resolution image observation image can be provided. As a result, it is possible to automate the operation of acquiring a color high-resolution composite observation image, and the user is freed from appropriate selection and switching operations such as selection of an illumination filter according to illumination light, switching of an image sensor, optical path shift, etc. Troublesome and incorrect operation can be eliminated.

Furthermore, in the eighth magnified observation device, the imaging elements are arranged in a matrix with a Bayer arrangement for each pixel, and the optical path shift means can be switched so as to shift to adjacent 2 × 2 pixel positions. As a result, the image pickup devices with different light receiving characteristics arranged in a Bayer array are shifted by the optical path shifting means so as to make a round for adjacent 2 × 2 pixels of interest, and light reception signals are obtained at all 2 × 2 pixel positions. And a high-resolution observation image can be obtained.

Furthermore, in the ninth magnified observation device, the optical path shift means can include an actuator that moves the selected image sensor in the pixel group of interest by the pixel interval. Thereby, it is possible to perform photographing a plurality of times by moving the selected image sensor vertically and horizontally in pixel units.

Furthermore, in the tenth magnification observation apparatus, the illumination light source includes a plurality of light emitting elements having different wavelength ranges, and the illumination light selecting unit is configured to select a light emitting element having a desired wavelength range among the illumination light sources. Light emitting element selection means can be used. Thereby, the illumination light of a desired wavelength range can be obtained directly with a light emitting element, without using a filter etc. Further, if a semiconductor light emitting element such as an LED is used, a more stable and highly reliable magnification observation apparatus having low power consumption and long life with excellent responsiveness can be obtained.

Furthermore, the eleventh magnification observation method has a plurality of types of pixels corresponding to different wavelength regions by a color filter , and uses a magnification observation device including a single-plate color imaging device for obtaining a color image. Monochromatic image obtained from color information of a color image obtained from the color image sensor and pixel information of a pixel which is pixel-shifted and corresponding to a short color wavelength range by the color filter among the pixel information from the color image sensor And a synthesizing step of synthesizing the luminance information with each other to obtain a color observation image . Accordingly , a high-resolution color image can be synthesized in a short time while using a camera using a normal color image sensor .

Furthermore, the twelfth magnification observation program operates a magnification observation apparatus having a plurality of types of pixels corresponding to different wavelength regions by a color filter and including a single-plate color imaging device for obtaining a color image. A magnification observation program for obtaining a color observation image of a sample, wherein color information of a color image obtained from the color image sensor and pixel information from the color image sensor are shifted and shorter by the color filter. The computer can realize a combining function for combining the luminance information of the monochrome image obtained from the pixel information of the pixel corresponding to the color wavelength range to obtain a color observation image . Accordingly , a high-resolution color image can be synthesized in a short time while using a camera using a normal color image sensor .

A thirteenth computer-readable recording medium stores the above program. Recording media include CD-ROM, CD-R, CD-RW, flexible disk, magnetic tape, MO, DVD-ROM, DVD-RAM, DVD-R, DVD + R, DVD-RW, DVD + RW, Blu-ray, HD , DVD (AOD) and other magnetic disks, optical disks, magneto-optical disks, semiconductor memories, and other media that can store programs. The program includes a program distributed in a download manner through a network line such as the Internet, in addition to a program stored and distributed in the recording medium. Furthermore, the recording medium includes a device capable of recording the program, for example, a general purpose or dedicated device in which the program is implemented in a state where it can be executed in the form of software, firmware, or the like. Furthermore, each process and function included in the program may be executed by computer-executable program software, or each part of the process or hardware may be executed by hardware such as a predetermined gate array (FPGA, ASIC), or program software. And a partial hardware module that realizes a part of hardware elements may be mixed.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiment shown below exemplifies a magnification observation apparatus, a magnification image observation method, a magnification image observation program, and a computer-readable recording medium for embodying the technical idea of the present invention, The present invention does not specify a magnification observation apparatus, a magnification image observation method, a magnification image observation program, and a computer-readable recording medium as follows. Further, the present specification by no means specifies the members shown in the claims to the members of the embodiments. In particular, the dimensions, materials, shapes, relative arrangements, and the like of the component parts described in the embodiments are not intended to limit the scope of the present invention unless otherwise specified, and are merely explanations. It is just an example. Note that the size, positional relationship, and the like of the members shown in each drawing may be exaggerated for clarity of explanation. Furthermore, in the following description, the same name and symbol indicate the same or the same members, and detailed description thereof will be omitted as appropriate. Furthermore, each element constituting the present invention may be configured such that a plurality of elements are constituted by the same member and the plurality of elements are shared by one member, and conversely, the function of one member is constituted by a plurality of members. It can also be realized by sharing. In addition, the contents described in some examples and embodiments may be used in other examples and embodiments.

Magnifying observation apparatus, magnifying image observing method, magnifying image observing program and computer-readable recording medium used in the embodiments of the present invention and a computer for operation, control, display, and other processing connected thereto For example, IEEE1394, RS-232x, RS-422, serial connection such as USB, parallel connection, or network such as 10BASE-T, 100BASE-TX, 1000BASE-T, etc. The communication is performed by electrical, magnetic, or optical connection through the network. The connection is not limited to a physical connection using a wire, but IEEE802. Wireless connection using radio waves such as wireless LAN such as x, Bluetooth (registered trademark), infrared rays, optical communication, or the like may be used. Furthermore, a memory card, a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like can be used as a recording medium for exchanging data or storing settings. In this specification, the term “magnification observation device” is used to include not only the magnification observation device body but also a magnification observation system in which peripheral devices such as a computer and an external storage device are combined.

Further, in this specification, the magnification observation apparatus is not limited to a system that performs magnification observation, and an apparatus or method that performs input / output, display, calculation, communication, and other processes related to imaging in hardware. An apparatus and method for realizing processing by software are also included in the scope of the present invention. For example, a general-purpose circuit or computer that incorporates software, programs, plug-ins, objects, libraries, applets, compilers, modules, macros that operate on specific programs, etc., and enables imaging itself or related processing The system also corresponds to the magnification observation apparatus of the present invention. In this specification, the computer includes a general-purpose or dedicated computer, a workstation, a terminal, a portable electronic device, PDC, CDMA, W-CDMA, FOMA (registered trademark), GSM, IMT2000, fourth generation, and the like. Mobile phones, PHS, PDAs, pagers, smartphones and other electronic devices. Further, in the present specification, the program is not limited to a program that is used alone, an aspect that functions as a part of a specific computer program, software, service, etc., an aspect that is called and functions when necessary, an environment such as an OS, etc. It can also be used as a mode provided as a service, a mode that operates resident in the environment, a mode that operates in the background, and other support programs.

Hereinafter, the magnifying observation apparatus 100 according to an embodiment of the present invention will be described with reference to FIGS. As shown in FIG. 2, the magnifying observation apparatus 100 includes an illuminating unit 60 for illuminating a sample (or workpiece or other subject) S to be observed, and an imaging unit 10 for imaging the sample S illuminated by the illuminating unit 60. The main body 50 includes a display unit 52 that displays an enlarged image captured by the imaging unit 10. The imaging unit 10 is connected to the main body unit 50 via the cable unit 24 as the head unit 15. Further, the magnified observation apparatus 100 in FIG. 2 serves as a sample fixing unit for fixing the sample S, and a reflected light from the stage 30 on which the sample S is placed and the sample S fixed on the stage 30 incident through the optical system 11. Or the image pick-up element 12 which electrically reads transmitted light, and the stage elevator 20 as a focus adjustment part which changes the relative distance in the optical axis direction of the stage 30 and the optical system 11 and adjusts a focus. Furthermore, as shown in FIG. 3, the main body 50 indicates the focal length information regarding the relative distance between the stage 30 and the optical system 11 in the optical axis direction when the focal point is adjusted by the stage elevator 20, substantially perpendicular to the optical axis direction. A memory 53, a display means 52 for displaying an image read by the imaging device 12, a head unit 15, the stage elevator 20, and data are stored as a focal length information storage unit that stores the two-dimensional position information of the sample S in the plane. And an interface 54 for communication. The magnification observation apparatus 100 captures an observation image using the imaging element 12 that electrically reads reflected light or transmitted light from the sample S fixed to the stage 30 incident via the optical system 11, and displays the display unit 52. To display.

Further, the magnification observation apparatus 100 relates to the operation unit 55 as a region setting unit capable of setting a region on the image displayed by the display unit 52 and a part or all of the sample S corresponding to the region set by the region setting unit. Based on the focal length information stored in the memory 53, a control unit 51 is provided that calculates the height in the optical axis direction of the sample S corresponding to the region set by the region setting unit. The magnification observation apparatus 100 can calculate an average height (depth) in the optical axis direction of the sample S corresponding to a region designated by using the imaging element 12.

This control unit 51 is a state in which a plurality of observation images taken for the same field of view of the sample S using a plurality of illumination filters are simultaneously displayed on the display unit 52 using an optical path shift control unit 81 for operating the optical path shift unit 14 described later. From the image selection means 82 that can select one, the imaging condition setting means 83 that sets the image observation condition including the type of the illumination filter used for imaging the observation image selected by the image selection means 82 as the imaging condition, An image synthesizing unit 85 that synthesizes at least two observation images obtained by imaging the same sample S using different illumination filters, and an operation of synthesizing the observation images by the image synthesizing unit 85 to obtain a color high-resolution image observation image. Automatic synthesizing means 84 that performs automatically, and a plurality of different wavelength ranges included in the wavelength range of light emitted from the illumination light source 63 can be selectively switched to any one of the wavelength ranges. To realize the functions such as the illumination light selection means 90. The control unit 51 can be composed of a gate array such as an ASIC or FPGA.

The operation unit 55 is connected to the main body unit 50 or the computer in a wired or wireless manner, or is fixed to the computer. Examples of the general operation unit 55 include various pointing devices such as a mouse, keyboard, slide pad, track point, tablet, joystick, console, jog dial, digitizer, light pen, numeric keypad, touch pad, and accu point. In addition to the operation of the operation program for magnification observation, these operation units 55 can be used for operations of the magnification observation apparatus 100 itself and its peripheral devices. Furthermore, using a touch screen or touch panel on the display itself that displays the interface screen, the user can directly input or operate the screen by hand, or use voice input or other existing input means, Or these can also be used together. In the example of FIG. 2, the operation unit 55 is configured by a pointing device such as a mouse.
(Lighting means 60)

The illumination unit 60 generates illumination light that illuminates the sample S imaged on the image sensor 12. The illumination light source 63 (FIG. 5) of the illumination unit 60 is built in the main body unit 50, and illumination light is transmitted to the illumination unit 60 of the head unit 15 through the optical fiber 61. The illuminating means 60 can be incorporated in the head unit 15 or can be separated from the head unit 15. As the illumination method of illumination light, epi-illumination, transmission illumination, or the like can be used as appropriate. The illuminating means 60 shown in FIG. 2 includes an epi-illumination 60A for irradiating the sample S with epi-illumination and a transmissive illumination 60B for irradiating the transmitted light. These illuminations are connected to the main body 50 via the optical fiber 61. The main body 50 includes a connector 62 for connecting the optical fiber 61 and a built-in illumination light source 63 for sending light to the optical fiber 61 via the connector 62. The epi-illumination 60A is a ring-shaped illumination. The ring-shaped illumination can be switched between all-around illumination and side illumination. In order to realize this, a turret-type mask that cuts a part of the illumination light, a configuration in which a plurality of LEDs are arranged in a ring shape as ring-shaped illumination, and a part of the LEDs are turned on / off can be used.
(Illumination light source 63)

As the illumination light source 63, a halogen lamp, a xenon lamp, an HID lamp, or the like can be used as a white light source that emits white light in a wide wavelength range. A light source capable of irradiating not only visible light but also infrared light is preferable. In particular, a halogen lamp is preferable because the wavelength range of the emission wavelength is wide. In addition to using a single light source, a plurality of light sources can be provided, and these can be turned on simultaneously to make mixed color light illumination light, or can be switched to illuminate. Furthermore, a semiconductor light emitting element such as an LED (Light Emitting Diode) or an LD (Laser Diode) can be used as an illumination light source. For example, an LED having an RGB wavelength range is prepared, and illumination light can be switched to red, green, and blue by turning on each LED, or white light can be obtained by mixing these colors. In particular, since the LED is excellent in ON / OFF responsiveness, there is an advantage that the measurement throughput can be improved. It also has features such as long life, low power consumption, low heat generation, and resistance to mechanical shock. Or it can also be set as the light source using wavelength conversion members, such as the fluorescent substance excited by the ultraviolet-ray of visible light, or visible light. Thereby, even one LED can emit white light. Further, an LED capable of emitting ultraviolet light or infrared light in addition to visible light can be used as a light source. For example, observation with infrared light is useful for analysis of defective products, tissue distribution of living tissue, and the like.
(LED lighting)

As an example of such an illumination light source 63 ′, FIG. 12 shows an example of an annular ring illumination 60 </ b> A ′ in which LEDs, G, R, B, and W capable of emitting green, red, blue, and white are sequentially arranged. As shown in FIG. 2, the ring-shaped illumination 60 </ b> A ′ is disposed around the vicinity of the tip of the head unit 15 having the optical system 11 and the image sensor 12 to illuminate the sample S. The illumination light source 63 ′ is electrically connected to the light emitting element selecting means 91 as the illumination light selecting means 90 for selecting the illumination light, and the selection and lighting of the LEDs are controlled by the light emitting element selecting means 91. That is, when green is selected as the illumination light, only the green LEDs and G are turned on. When red, blue, and white illumination lights are selected, the LED, R, B, and W corresponding to each illumination light are selected. Is selected and the lighting is controlled. Thus, the illumination light source comprised by LED of different luminescent color has the advantage that a luminescent color can be changed easily.

In this example, four types of LEDs, green G, red R, blue B, and white W, are arranged alternately at a constant cycle. However, the arrangement pattern, number, combination of emission colors, etc. can be changed as appropriate. Needless to say. For example, when it is desired to increase the output of the green LED, more green LEDs are arranged, or when white light is obtained by simultaneous lighting of RGB, the white LED can be eliminated. In addition, various emission colors can be obtained without being limited to white light by RGB color mixing. Further, since no filter is used, a mechanical operation such as filter switching is unnecessary, and stable high-speed illumination light switching using only an electric signal is realized. Further, since LEDs have a long life, maintenance work such as replacement of a light bulb can be saved. Furthermore, since the semiconductor light emitting element is smaller than the bulb, there is an advantage that a plurality of types of light emitting elements can be arranged in a space-saving manner. For example, by providing an infrared light emitting element and an ultraviolet light emitting element, it is possible to easily switch the illumination light to not only visible light but also infrared light, ultraviolet light, and the like. In this way, an illumination light source having a plurality of light emitting elements in different wavelength ranges can be controlled by the illumination light selecting means 90, and a light emitting element in a desired wavelength range can be selected and lit to irradiate illumination light.
(Filter means 86; Filter selection means 88)

A filter unit 86 is disposed between the illumination light source 63 and the sample fixing unit as one form of the illumination light selection unit 90. The illumination unit 60 can extract RGB light or white light from the illumination light source 63 as illumination light via the filter unit 86. The filter means 86 includes a plurality of illumination filters, and these are switched by a filter selection means 88. For example, as shown in FIG. 4, a rotary turret is provided with illumination filters of a red filter RF, a blue filter BF, a green filter GF, and a transmission filter PF, and these are mechanically switched by the rotation of the turret. The transmission filter PF transmits all wavelengths of light emitted from the illumination light source 63, and is typically an opening. The blue filter BF, the red filter RF, and the green filter GF are color filters that transmit the wavelength ranges corresponding to the respective colors, and glass or the like that defines the transmitted wavelength ranges can be used. In addition to the three primary colors RGB, these color complementary colors (for example, cyan, magenta, and yellow) can be used as appropriate for the color filter. In addition, a filter that transmits ultraviolet light or infrared light can be used as a filter. The filter selection means 88 can switch the illumination filter by rotating the turret as shown in FIG. The turret is fixed to the rotation shaft of the filter rotation motor 89 and is controlled to be rotatable by the filter rotation motor 89. The turret is provided with a filter position sensor 87, and the output of the filter position sensor 87 and the rotation control input of the filter rotation motor 89 are connected to the filter selection means 88. As a result, the filter selection unit 88 can determine the type of the currently selected illumination filter, and can rotate the turret to a desired posture by rotating the filter rotation motor 89, and the illumination filter selected with respect to the optical axis of the illumination light source 63. Can be placed. The user can irradiate illumination light of a desired wavelength by selecting an illumination filter with the filter selection means 88.

The illumination filter is arranged between the illumination light source 63 and the sample S when the transmitted light of the sample S is observed, and when the reflected light or excitation light of the sample S is observed, the illumination light source 63 and the sample are arranged. S between the objective lens and the eyepiece or the image sensor. In addition, the filter means 86 can include a diffusion filter, a polarizing filter, and the like in addition to the color filter. Thus, the filter means 86 can change not only the wavelength of illumination light but also characteristics such as intensity and polarization state by switching the filter. In addition, a plurality of characteristics such as wavelength, intensity, polarization state, and the like can be changed by using a plurality of filters having two or more stages.

Alternatively, instead of the turret type illumination filter, R, G, B and transmission spectral characteristics can be switched or transmission can be switched by an applied voltage using a liquid crystal RGB filter. In this case, the filter selection unit 88 can select the illumination light by controlling the voltage applied to the liquid crystal RGB filter. Furthermore, instead of white light such as a halogen lamp as an illumination light source, a semiconductor light emitting element such as an LED or LD can be used. For example, by preparing RGB LEDs and turning on each LED, the illumination light can be switched to RGB, and by simultaneously turning on RGB, white light can be obtained by color mixing. Further, the present invention is not limited to the three primary colors of RGB but can use complementary colors as described above.
(Imaging unit 10)

The imaging unit 10 includes an imaging element 12 that electrically reads reflected light incident via the optical system 11 from the sample S to be observed illuminated by the illumination unit 60. In this example, the imaging device 12 uses a CMOS, but other light receiving devices such as a CCD can also be used. This image pickup device 12 is a single plate image pickup device that can receive light of a specific wavelength in the amount of received light with respect to the illumination light irradiated to the sample S from the illumination light source 63 through the filter means 86, and corresponds to different wavelength ranges. A two-dimensional color imaging device having a plurality of types prepared and one element for each pixel is arranged, thereby enabling a color image to be captured. Each image sensor is arranged at a constant pixel interval, and further, adjacent pixels are arranged to have light receiving characteristics of different wavelengths. Preferably, the two-dimensional color imaging device has a Bayer arrangement as shown in FIG. An image acquired by the image sensor 12 is held in the memory 53 as necessary. The imaging element corresponding to the wavelength range of the illumination light is an imaging element having high sensitivity to the wavelength range of the illumination light. For example, when the illumination light is red, the light reception sensitivity is high in the wavelength range of the red light. It refers to an image sensor. When the complementary color of the illumination light is used as the image sensor, the image sensor having a high light reception sensitivity is indicated in the corresponding complementary color region, for example, the light reception sensitivity is given to the green region which is a red complementary color with respect to the red illumination light. The imaging element which has is applicable.
(Optical path shifting means 14)

Furthermore, the magnification observation apparatus 100 includes an optical path shift unit 14 for relatively shifting the detection position of the image sensor 12 and an optical path shift control unit 81 for operating the optical path shift unit 14. Specifically, for three or more target pixel groups, the pixel interval of the image sensor so that the light reception signal makes a round at the position of each pixel of the image sensor constituting the target pixel group and the received light amount is detected at each position. The detection position of one of the image sensors constituting the target pixel group is relatively shifted by the optical path shift means 14 by the amount of displacement corresponding to.
(Optical path shift control means 81)

On the other hand, when the sample S is irradiated with illumination light of a predetermined wavelength via the illumination filter selected by the filter selection unit 88, the optical path shift control unit 81 is an image sensor corresponding to the wavelength region among the plurality of image sensors. The optical path shift means 14 is operated so as to detect the amount of received light. As a result, the selection of the illumination filter and the image sensor and the pixel shift can be linked, and the user can perform high-speed switching without cumbersome switching or illumination light and the combination of the illumination filter and the image sensor corresponding to the illumination light. A resolution observation image can be easily acquired.

In the example of FIG. 3, the optical path shift unit 14 is provided in the imaging unit 10, and a higher resolution than the resolution of the CMOS can be obtained by pixel shifting. The pixel shift is a space between an adjacent element and an element (pixel) by using a piezoelectric element or the like as described in Patent Document 2 for a single plate type and Patent Document 3 for a three-plate type, for example. In addition, by pixel shift that physically shifts the element, for example, an image obtained by shifting the sample S by half the pixel pitch and an image before the shift are combined with each other to increase the resolution. In addition, the color reproducibility is also improved by acquiring RGB data at each pixel by shifting by one pixel pitch. As typical image shifting mechanisms, there are an image sensor driving method in which the image sensor 12 is moved by an actuator AC, an LPF tilt method in which the LPF is tilted, a lens moving method in which the lens is moved, and the like.

When the pixel shifting function is executed, as shown in FIG. 1, in the state in which the image pickup elements are arranged in a matrix in a Bayer array for each pixel, the optical path shifting means 14 has an adjacent 2 × 2 as shown in FIG. It can be switched to shift to the pixel position. As a result, the image pickup devices having different light receiving characteristics arranged in a Bayer array are shifted by the optical path shift means 14 so as to make a round for adjacent 2 × 2 pixels of interest, and light reception signals are transmitted at all 2 × 2 pixel positions. It is possible to obtain a high-resolution observation image. Note that the shift amount by which the image sensor is relatively shifted by the optical path shift means 14 is moved four times counterclockwise, a total of four pixels, as the amount of displacement corresponding to the pixel interval of the image sensor in the example of FIG. However, it is also possible to move only two adjacent pixels such as up and down, left and right, or 3 pixels. Further, the amount of movement is not limited to one pixel of the image sensor, and may be set to 1/2 pixel or 1/3 pixel. By adjusting the amount of movement according to the peak position and range of the light receiving sensitivity of each pixel constituting the image sensor, the amount of received light can be improved even with a moving amount of one pixel or less, so that high resolution can be achieved. Thus, the amount of displacement corresponding to the pixel interval of the image sensor is not limited to the pixel pitch or an integer multiple thereof, and includes fractional multiples such as 1/2 pixel and 1/3 pixel.
(Display means 52)

Further, such image data and setting contents held in the memory 53 can be displayed on the display means 52. As the display means 52, a monitor such as a CRT, a liquid crystal display, or an organic EL can be used. In addition, an operation unit 55 is connected to the control unit 51 for a user to perform various operations. The operation unit 55 is an input device such as a console or a mouse. Also in this example, the display means and the operation unit can be integrated with the main body unit or can be an external member. Furthermore, if the display means is configured by a touch panel, the display means and the operation unit can be configured integrally.

The main body 50 inputs the control data related to the control of the stepping motor 21 to the motor control circuit 22, whereby the light from the stage 30 that is the sample fixing unit and the head unit 15 that includes the optical system 11 and the image sensor 12 is obtained. The relative distance in the axial direction, here, the height in the z direction is changed. Specifically, the main body 50 controls the rotation of the stepping motor 21 by inputting control data necessary for controlling the stage elevator 20 to the motor control circuit 22, and the height z of the stage 30 (in the z direction). Position). The stepping motor 21 generates a rotation signal corresponding to the rotation. The main body 50 stores the height z of the stage 30 as information regarding the relative distance between the sample fixing unit and the optical system 11 in the optical axis direction based on the rotation signal input via the motor control circuit 22. The stage 30 functions as an observation positioning unit that positions the observation position with respect to the sample S. In the present embodiment, the example in which the relative distance in the optical axis direction between the sample fixing unit and the optical system 11 is changed by changing the height of the stage 30 is shown. However, the height of the optical system is fixed by fixing the stage. For example, the height of the camera may be changed. Further, in addition to providing the stage in the main body of the magnifying observation apparatus, the stage can be provided in a head part that is a separate member from the main body, or an imaging part in which the stage is omitted can be provided in the head part. The imaging unit from which the stage is omitted can be mounted on a mounting stand or can be held by the user. Such a head portion is connected to the magnifying observation apparatus main body by a cable.

The image sensor 12 can electrically read the amount of received light for each pixel arranged two-dimensionally in the x and y directions. The image of the sample S formed on the image sensor 12 is converted into an electrical signal in accordance with the amount of received light in each pixel of the image sensor 12, and further converted into digital data in the image sensor control circuit 13. The main body 50 uses the digital data converted by the image sensor control circuit 13 as the received light data D in a plane (x and y directions in FIG. 3) substantially perpendicular to the optical axis direction (z direction in FIG. 3). The pixel arrangement information (x, y) as the two-dimensional position information of the sample S is stored in the memory 53. Here, the in-plane substantially perpendicular to the optical axis direction does not need to be a plane strictly forming 90 ° with respect to the optical axis, and is such that the shape of the sample S can be recognized by the resolution in the optical system and the image sensor. Any observation surface that falls within the range of the inclination of the angle may be used.

In the above description, an example in which the sample S is placed on the stage 30 is shown as an example of the sample fixing unit. However, for example, an arm is attached instead of the stage and the sample S is fixed to the tip. You can also. Further, the head unit 15 can be used by being attached to the camera attachment unit 43, and can be arranged at a desired position and angle by a method such as hand-holding so as to be removable.
(Control unit 51)

The control unit 51 controls the captured observation image to be displayed after being converted to a resolution that can be displayed by the display unit 52. In the magnifying observation apparatus 100 of FIG. 2, the imaging unit 10 displays an observation image obtained by imaging the sample S with the imaging element 12 on the display unit 52. In general, the performance of an image sensor such as a CMOS or CCD often exceeds the display capability of the display means, so in order to display the captured observation image on one screen, the resolution is displayed on one screen by thinning the image. Reduced to the possible size and reduced. Assuming that the reading resolution when reading by the imaging unit 10 is the first resolution, the display unit 52 displays the second resolution lower than the first resolution.

Further, when still images of observation images are continuously captured and displayed on the display unit 52, the captured images may be displayed by switching them continuously to display them as if they were moving images. Such a continuous shooting mode can be switched to normal still image shooting and display mode by the control unit 51. In addition, the pixel shifting function can also be used during such continuous shooting.
(Magnification observation device operation program)

FIG. 7 shows an illumination condition selection screen 200 as an example of the observation condition setting screen. This figure is a user interface screen of a magnification observation apparatus operation program for operating the magnification observation apparatus 100, and the operation screen can be displayed on the display means 52 of the magnification observation apparatus 100 or a monitor of a computer 70 connected externally. The user performs various settings and operations of the magnification observation apparatus 100 from the displayed screen. The magnification observation apparatus operation program is incorporated in the main body unit 50.
(Procedure for switching illumination light)

In the illumination condition selection screen 200 shown in FIG. 7, illumination light can be switched. Specifically, when a “wavelength switching” button is pressed, a dialog for selecting a wavelength band appears. Here, in the pull-down menu format, one of blue illumination, green illumination, red illumination, infrared illumination, high-resolution color image acquisition, and infrared + color image acquisition can be selected. A plurality of illumination filters are prepared in the illumination means 60, and the color filters are switched as shown in FIG. 5 according to the wavelength selected here. Here, when a light other than white light is selected as the illumination light, the pixel shift function is set to always work when displaying an image or storing a still image. By such pixel shifting, a monochrome observation image can be acquired without impairing the resolution.
(Imaging condition selection screen 300)

Furthermore, in this embodiment, since the observation with the illumination light changed can be easily performed, the images acquired with a plurality of different illumination lights are displayed side by side as the imaging condition selection screen 300 for the same field of view of the sample S, When the user selects a desired image, an easy setting function of an observation condition that is determined as an observation condition (such as selection of illumination light, illumination filter, and image sensor) set in the selected image can be realized. As an example of the imaging condition selection screen 300, as shown in FIG. 8, observation images obtained by imaging illumination light as white light, red light, green light, and blue light are respectively displayed in a list by dividing the display means 52 into four. Is available. When the user selects the desired observation image from the screen while using the imaging condition selection screen 300 as the image selection means 82 and contrasts various actually obtained observation images, the illumination used for imaging the selected image. The combination of the filter and the image sensor 12 is read out, and this condition is automatically set by the imaging condition setting unit 83 as an image observation condition in subsequent observations. In addition to the normal observation image, the observation image acquired on the imaging condition selection screen 300 can be a simple observation image that is simply captured in a shorter time, for example, by reducing the frame rate. The simple imaging condition for acquiring the simple observation image is automatically set by the imaging condition setting unit 83 of the calculation unit. Thus, by shortening the imaging time of each observation image, a plurality of simple observation images can be acquired in a short time. Here, by pressing a “preview function” button (not shown), simple observation images using all the illumination filters are acquired, and the imaging condition selection screen 300 is automatically displayed. As a result, the user can acquire a plurality of observation images with different image observation conditions in a short time without performing a switching operation of a plurality of illumination filters, an operation of acquiring or storing an observation image, and the actually obtained image. Since the image observation conditions can be selected sensuously based on the image, desired observation can be easily performed without being bothered by the meaning, correlation, adjustment work, etc. of the plurality of parameters. Although the example of FIG. 8 shows an example in which only the illumination light is changed, an imaging condition selection screen in which other image observation conditions such as a frame rate and the presence / absence of pixel shift are changed can be configured.
(Image composition means 85)

Furthermore, a high-resolution color image can be acquired using the image composition means 85. If observation is performed with illumination light having a short wavelength, an image with high resolution can be obtained. By utilizing this property and using blue illumination light and pixel shifting, a high-resolution monochromatic image as shown in FIG. 9 can be obtained. In particular, by limiting the wavelength range of the illumination light, the influence of chromatic aberration can be suppressed, thereby obtaining the effect of further improving the resolution. However, this image has only blue information and is not a full color image. Therefore, a full color image using white illumination light is obtained separately (FIG. 10), and the image composition means 85 superimposes the color information (chromaticity, saturation) of the full color image on the luminance information of the single color image. A color image with high resolution as shown in FIG. 11 can be obtained. In other words, among single-plate image sensors, an image sensor that can capture illumination light with a short wavelength, specifically, a monochrome high-resolution image is captured using a blue image sensor, and a color observation image taken separately. Can be combined with the high resolution monochrome observation image to obtain a color high resolution observation image.

In the conventional magnification observation apparatus, since color information that can be detected by one pixel is single, in order to obtain a color image, (1) color information that cannot be detected in each pixel is adjacent to the surroundings (the color that cannot be detected). There is a need to predict color information based on color information of pixels (which can detect information) and (2) to use a three-plate type imaging device such as a so-called 3CCD.

On the other hand, in the synthesis method by the image synthesis means 85 described above, a color image having a higher resolution than the colorization by prediction of (1), a sharp edge of the sample can be obtained, and less expensive than 3CCD of (2). Can be configured. Furthermore, the size of the image sensor, that is, the camera can be reduced as compared with 3CCD. In addition to shifting the pixel by one pixel, it is also possible to shift the pixel by 1/2 or 1/3 pixel, and there is an advantage that a higher resolution image can be obtained compared to 3CCD having the same number of pixels. . Furthermore, it is not necessary to shift the pixels for each wavelength of RGB, and it is sufficient to shift the pixels once for a high-resolution image with a short wavelength and to synthesize and synthesize a color image with ordinary white light. The advantage of being able to synthesize a high-resolution color image is also obtained.
(Automatic synthesis means 84)

Furthermore, the automatic synthesizing unit 84 can automatically perform the steps of acquiring a series of images and synthesizing the images without the user manually performing the steps. As a result, the user can easily obtain a high-resolution color image, and high-definition observation is realized. The procedure for automatically acquiring the composite image by the automatic combining means 84 is as follows. When the user presses the “acquire high resolution color image” button on the operation screen of the magnification observation apparatus operation program shown in FIG. (1) An instruction to select the blue filter BF is sent to the filter selection means 88, and the optical path shift means 14 is operated by the optical path shift control means 81 so that the blue image pickup device is imaged at each pixel position and synthesized. Operation to obtain a high-resolution observation image in the blue wavelength range;
(2) Sending an instruction to select the transmission filter PF to the filter selection unit 88 and operating the optical path shift unit 14 to the optical path shift control unit 81 so that all image sensors are imaged at each pixel position. To obtain a color observation image of
(3) The image synthesizing unit 85 automatically performs the operation of synthesizing the color information of the color observation image with the high resolution observation image and synthesizing the color high resolution image observation image. As a result, a color high-resolution image can be acquired with almost one touch, and the user-friendliness is extremely high. Any of the operations (1) and (2) may be performed first. That is, the same result can be obtained by first capturing a color image with white light and then capturing and synthesizing a blue high-resolution image. In the above example, a high-resolution image is captured using blue illumination light. However, green and red illumination light and white light can be combined depending on the observation application. An observation image obtained by illumination with red light or green light is useful for edge detection or the like in dimension measurement applications. For example, it is possible to enhance edge detection by emphasizing the edge portion of a colored workpiece having a low reflectance. In addition to these colors, various illumination lights can be used depending on the application. For example, if the illumination light is yellow, it is possible to measure an image in a yellow room sensitive to light exposure. Furthermore, a high-resolution color image can also be acquired by synthesizing single-color images captured with blue light, green light, and red light. In this case, since three observation images are acquired, it takes a long imaging time. On the other hand, since a luminance signal can be acquired for each wavelength component of RGB, there is an advantage that a clearer image can be acquired.
(How to use the magnifier)

Using the magnification observation apparatus 100 described above, a clear observation image with high resolution can be taken and displayed on the display means 52 for observation. The user irradiates the sample with light in a specific wavelength range selected according to the purpose. For example, blue illumination with a short wavelength is used for improving the resolution, red illumination with a long wavelength is used for improving the transmittance, and infrared illumination is used for observing the inside of a sample that does not transmit visible light. Further, by combining infrared illumination and white light illumination, the inside and outside of the sample can be observed simultaneously. For example, when the sample to be observed is an IC chip, the structure existing in the silicon can be confirmed by the effect of infrared rays, while the character information such as the number marked on the chip surface can be recognized by white light. Thus, when infrared light is used together, there is an advantage that both the inside and the outside of the sample can be observed.

The combination of these illumination lights uses the filter selection means 88 to select an appropriate illumination filter for the light source light from the filter means 86. At the time of image capture, the image sensor control circuit 13 selects only the pixel information of the color filter with the highest sensitivity in the image sensor for the illumination light selected by the filter selection means 88, and acquires an observation image. Further, by the pixel shifting function, a fine actuator AC is connected to the two-dimensional color imaging device as the optical path shifting means 14, and the selected path is moved by the optical path shifting means 14 in the vertical and horizontal directions in units of pixels as shown in FIG. Take multiple shots. By reconstructing the entire image information based on the pixel information obtained in this manner, a high-resolution single color (monochrome) image can be obtained. In this manner, a high-quality image can be created that selectively uses optimal pixel information from among the R, G, and B colors according to the wavelength of irradiation light selected by the user. For example, when the user selects blue light as the irradiation light from the operation screen of the magnification observation apparatus shown in FIG. 7, the pixel of the image sensor with the highest sensitivity is the pixel of the blue color filter. Therefore, red and green pixel information is ignored, and an image is created using only the blue color information by using the pixel shifting function. This makes it possible to obtain an optimal observation image without damaging the resolution and sensitivity of the camera depending on illumination conditions while using a camera using a normal color image sensor.

The magnifying observation apparatus, the magnifying image observing method, the magnifying image observing program, and the computer-readable recording medium of the present invention can be suitably used for a microscope, a reflection / transmission type digital microscope, and a digital camera. In addition, when the present technology is applied to a fluorescence microscope, reflected light or transmitted light from a sample with respect to illumination light can be read as excitation light.

It is a top view which shows the state by which the image pick-up element of a different color is Bayer arrayed. 1 is an external view of a magnification observation apparatus according to an embodiment of the present invention. It is a block diagram of the magnification observation device concerning one embodiment of the present invention. It is a top view which shows a rotary turret provided with an illumination filter. It is a perspective view which shows a mode that the turret of FIG. 4 is rotated. It is a top view which shows a mode that the detection position of an image pick-up element is relatively shifted with an optical path shift means so that the amount of light received may be detected by making a round of the position of the pixel of each image pick-up element about an attention pixel group. It is an image figure which shows the example of an illumination condition selection screen. It is an image figure which shows the example of an imaging condition selection screen. It is an image figure which shows the high resolution observation image imaged with the blue illumination light. It is an image figure which shows the observation image of the color imaged with white illumination light. It is an image figure which shows the synthesized image which synthesize | combined FIG.9 and FIG.10. It is a schematic cross section which shows an example of an illumination light source.

DESCRIPTION OF SYMBOLS 100 ... Magnification observation apparatus 10 ... Image pick-up part 11 ... Optical system 12 ... Image pick-up element; 13 ... Image pick-up element control circuit 14 ... Optical path shift means; 15 ... Head part 20 ... Stage elevator; 21 ... Stepping motor; 24 ... Cable unit 30 ... Stage 43 ... Camera mounting unit 50 ... Main body unit; 51 ... Control unit; 52 ... Display means 53 ... Memory; 54 ... Interface; 55 ... Operation unit 60 ... Illumination unit; Transmitted illumination; 60A '... ring illumination 61 ... optical fiber; 62 ... connector; 63, 63' ... illumination light source 70 ... computer 81 ... optical path shift control means 82 ... image selection means 83 ... imaging condition setting means 84 ... automatic synthesis means 85 ... Image synthesis means 86 ... Filter means 87 ... Filter position sensor 88 ... Filter selection means 89 ... Filter rotation motor 9 Illumination light selection means 91 ... Light emitting element selection means 200 ... Illumination condition selection screen 300 ... Imaging condition selection screen S ... Sample AC ... Actuator RF ... Red filter; BF ... Blue filter; GF ... Green filter; PF ... Transmission filter R ... Red LED; G ... Green LED ;; B ... Blue LED; W ... White LED

Claims (6)

A single- chip color imaging device for obtaining a color image, having a plurality of types of pixels corresponding to different wavelength regions by a color filter ;
Monochromatic image obtained from color information of a color image obtained from the color image sensor and pixel information of a pixel which is pixel-shifted and corresponding to a short color wavelength range by the color filter among the pixel information from the color image sensor And a synthesizing means for obtaining a color observation image by synthesizing with the luminance information. The color image sensor includes a pixel of the red color filter, a pixel of green color filter, and a pixel of the blue color filter, that the monochrome image is obtained by image-containing information of the pixels of the blue color filter The magnification observation apparatus according to claim 1, wherein The magnification observation apparatus further includes:
Optical path shifting means for relatively shifting the detection position of the color image sensor and performing the pixel shift ;
A monochrome image obtained by pixel information corresponding to a color having a short wavelength among pixel information from the color image sensor, wherein the detection position of the color image sensor is relatively shifted by a predetermined pixel pitch by the optical path shift means; Optical path shift control means for synthesizing the monochrome image before the shift to obtain a high-resolution monochrome image,
3. The magnified observation according to claim 1, wherein the synthesizing unit uses a high-resolution monochrome image obtained by the optical path shift control unit as a monochrome image to be synthesized in order to obtain the color image observation image. apparatus. The magnification observation apparatus further includes:
It has an LED light source and a wavelength conversion member excited by the LED light source, and includes a white light source for irradiating an imaging region of the color imaging device. The magnified observation device described. A magnification observation method that obtains a color observation image of a sample using a magnification observation device that has a plurality of types of pixels corresponding to different wavelength regions by a color filter and includes a single-plate color imaging device for obtaining a color image. There,
Monochromatic image obtained from color information of a color image obtained from the color image sensor and pixel information of a pixel which is pixel-shifted and corresponding to a short color wavelength range by the color filter among the pixel information from the color image sensor A synthesis step of synthesizing the luminance information with a color observation image;
A magnified observation method comprising: Magnification for obtaining a color observation image of a sample by operating a magnification observation device that has a single type of color image sensor for obtaining a color image , having multiple types of pixels corresponding to different wavelength ranges by a color filter An observation program,
Monochromatic image obtained from color information of a color image obtained from the color image sensor and pixel information of a pixel which is pixel-shifted and corresponding to a short color wavelength range by the color filter among the pixel information from the color image sensor An enlargement observation program characterized by causing a computer to realize a composition function for obtaining a color observation image by synthesizing with luminance information of the image.