JP6587709B2 - Observation target extraction processing apparatus, method and program, and observation image imaging display system - Google Patents

Description

  The present invention relates to an observation object extraction processing device, method and program, and an observation image imaging display system for generating an image obtained by extracting an observation object from an image obtained by imaging the observation object installed on an observation object support member such as a resin sheet. is there.

  Conventionally, there are cases in which observation objects such as cells and microorganisms, foods, crystals, and the like are placed on a sheet member made of resin or the like and observed with a microscope. Specifically, for example, the cell culture process may be confirmed by arranging and culturing cells on a porous sheet member, and capturing and observing images of the cells with a microscope.

  In this way, when an image of a cell placed on a sheet member is taken with a microscope, an image in which the observation target such as a cell and the hole pattern of the sheet member are mixed is picked up, so that only the observation target is extracted. Must be generated.

  As a method for generating an image in which only the observation target is extracted, for example, when the pattern of the hole of the sheet member is a known regular pattern, a signal representing the periodic structure of the sheet member is stored in advance as a template signal. There is a method of generating an image in which only the observation target is extracted by subtracting the template signal from the captured image signal.

JP 2012-39930 A

  However, for example, when the sheet member has an irregular pattern, the hole pattern changes depending on the imaging field of view, so if the imaging field of view changes, the same template signal cannot be used, and the image to be observed is the image of the sheet member pattern. It is difficult to separate and extract.

  In addition, a method of generating an image in which an observation target is extracted by removing a component of a sheet member that does not change with time using a plurality of time-lapse images is also conceivable. For example, when observing cells, cells such as incubator equipment are used. An apparatus for observing the time change of the above is required, which increases the cost.

  In addition, there is a method of generating an image in which an observation target is extracted by removing the component of the sheet member by frequency image processing. However, when the sheet member has an irregular pattern, the peak in the frequency space is disturbed, and the sheet member There is a problem that the pattern cannot be removed properly.

  In addition, a sheet member is imaged in advance in the absence of cells, and a matching image processing method such as registration is used between the image captured when the cells are cultured to some extent and the image of the sheet member described above. A method of removing the image of the sheet member is also conceivable. However, when the sheet member has an irregular pattern, the matching point used for registration also changes irregularly, and the pattern of the sheet member may not be removed appropriately. is there. Note that it is practically difficult to take an image of the sheet member before culturing the cells and the sheet member at a stage where the cells are installed and cultured to some extent in the same field of view.

  In addition, a plurality of images including an image in which the sheet member is most focused and an image in which the observation target is most focused are captured, and an image in which only the observation target is extracted is generated by enhancing the difference component. Although a method is also conceivable, when the sheet member has an irregular pattern, it is difficult to know an appropriate amount of defocus in advance, and the pattern of the sheet member may not be removed appropriately.

  Patent Document 1 proposes that a plurality of images are obtained by irradiating a dish with illumination light of a plurality of types of stripe patterns, and a medium region is identified by analyzing the plurality of images. However, no method has been proposed for a method of generating an image in which only the cells are extracted by removing the sheet member as described above.

  In view of the above problems, the present invention provides an image obtained by appropriately removing the pattern of the observation target support member and extracting the observation target even when the observation target support member on which the observation target is installed has an irregular pattern. An object of the present invention is to provide an observation object extraction processing apparatus, method, and program that can be generated, and an observation image capturing display system.

  The observation target extraction processing apparatus of the present invention images the observation target support member by irradiating the sheet-like observation target support member supporting the observation target with light and detecting the light emitted from the observation target support member. A correction image acquisition unit that acquires two types of first and second images having different spectral characteristics, a correction coefficient calculation unit that calculates a correction coefficient based on the first and second images, and an observation target are installed Two types of spectral characteristics obtained by irradiating the observation target support member with light and detecting the light emitted from the observation target support member on which the observation target is set, and imaging the observation target support member on which the observation target is set. An observation image acquisition unit that acquires the third and fourth images, and corrects one of the third and fourth images based on the correction coefficient to generate a corrected image. Image of Based on, characterized by comprising an extraction unit for generating an extracted image observation target is extracted.

  In the observation target extraction processing apparatus of the present invention, the first and second images are emitted from the observation target support member by irradiating the observation target support member with two types of light having different spectral characteristics, respectively. It is the image which imaged the observation object support member by detecting each light, and the 3rd and 4th image irradiates two types of light to the observation object support member in which the observation object was installed, respectively, and the observation object It is preferable that the image is an image of the observation target support member on which the observation target is installed by detecting light emitted from the observation target support member on which the observation target is installed.

  In addition, the first and second images are obtained by irradiating the observation target support member with excitation light and detecting fluorescence emitted from the observation target support member, and imaging the observation target support member, and the observation target This is a transmission image obtained by imaging the observation target support member by irradiating the support member with visible light and detecting the transmitted light transmitted through the observation target support member, and the third and fourth images are provided with the observation target. Irradiating the observation target support member with excitation light, and detecting the fluorescence emitted from the observation target support member on which the observation target is installed, and a fluorescence image obtained by imaging the observation target support member on which the observation target is installed; The observation target is installed by irradiating the observation target support member with the observation target with visible light, and detecting the transmitted light transmitted through the observation target support member with the observation target. It is preferable to subject the support member is a transmission image captured.

  In addition, the correction coefficient calculation unit performs sharpness enhancement processing on the fluorescent image of the first and second images, and cultures the observation target on the transmission image of the first and second images. The level correction process for suppressing the image of the medium component to be performed is performed, and the correction coefficient can be calculated based on the fluorescence image subjected to the sharpness enhancement process and the transmission image subjected to the level correction process.

  In addition, the extraction processing unit performs sharpness enhancement processing on the fluorescence image of the third and fourth images, and cultures the observation target on the transmission image of the third and fourth images. Performs level correction processing that suppresses the image of the medium component, generates a corrected image using the fluorescence image that has been subjected to sharpness enhancement processing, and extracts using the corrected image and the transmission image that has been subjected to the level correction processing An image can be generated.

  The first and second images detect light that irradiates light to the observation target support member and passes through two types of filters having different spectral characteristics emitted from the observation target support member. The third and fourth images are images obtained by irradiating light to the observation target support member on which the observation target is set and emitting from the observation target support member on which the observation target is set. Preferably, the image is an image of the observation target support member on which the observation target is set by detecting the light that has passed through two types of filters having different spectral characteristics.

  The first and second images detect the first fluorescent component that has passed through the first filter among the fluorescence emitted from the observation target support member by irradiating the observation target support member with excitation light. A first fluorescent image obtained by imaging the observation target support member, and a second fluorescent image obtained by imaging the observation target support member by detecting the second fluorescent component of the fluorescence that has passed through the second filter. The third and fourth images are the first filter of the fluorescence emitted from the observation target support member on which the observation target is set by irradiating the observation target support member on which the observation target is set with excitation light. A third fluorescent image obtained by imaging the observation target support member on which the observation target is installed by detecting the third fluorescent component that has passed through the second fluorescent component, and a fourth fluorescent component that has passed through the second filter out of the fluorescent light. By detecting It is preferable to observation target support member observation target is placed a fourth fluorescence image captured Te.

  The correction coefficient calculation unit performs sharpness enhancement processing on the first and second fluorescent images, and calculates a correction coefficient using the first and second fluorescent images subjected to the sharpness enhancement processing. can do.

  The extraction processing unit performs sharpness enhancement processing on the third and fourth fluorescent images, and generates a correction image using the third or fourth fluorescent image subjected to the sharpness enhancement processing. The extracted image can be generated using the corrected image and the third or fourth fluorescent image.

  The first and second images are images obtained by imaging the observation target support member by detecting the light emitted from the observation target support member by the imaging units having different spectral sensitivities. The third and fourth images The image is an image obtained by imaging the observation target support member on which the observation target support member is installed by detecting the light emitted from the observation target support member on which the observation target is installed by the imaging units having different spectral sensitivities. Is preferred.

  Further, the first and second images are obtained by imaging the observation target support member by irradiating the observation target support member with visible light and detecting the transmitted light transmitted through the observation target support member by the first imaging unit. 1 is a second transmission image obtained by imaging the observation target support member by detecting the transmitted light and the transmitted light by the second imaging unit, and the third and fourth images are observations where the observation target is installed. A first image of the observation target support member on which the observation target is installed is detected by irradiating the target support member with visible light and detecting the transmitted light transmitted through the observation target support member on which the observation target is installed by the first imaging unit. 3 and a fourth transmission image obtained by imaging the observation target support member on which the observation target is installed by detecting the transmitted light using the second imaging unit.

  Moreover, it is preferable that the spectral sensitivity of the first imaging unit is a band other than the visible light band, and the spectral sensitivity of the second imaging unit is a visible light band.

  Further, the correction coefficient calculation unit performs a level correction process for suppressing the image of the medium component for culturing the observation target on the first and second transmission images, and the first and second subjected to the level correction process. The correction coefficient can be calculated using the two transmission images.

  In addition, the extraction processing unit performs a level correction process for suppressing the image of the medium component for culturing the observation target on the third and fourth transmission images, and the third or fourth subjected to the level correction process. It is possible to generate a corrected image using the transparent image, and generate an extracted image using the corrected image and the third or fourth transparent image.

  The correction coefficient calculation unit can calculate the correction coefficient based on the ratio of the density information of the first image and the second image.

  Further, the extraction processing unit can generate a subtraction image obtained by subtracting the correction image from the other image, and generate an extraction image by performing a morphological process on the subtraction image.

  Moreover, it is preferable that an observation object support member is a sheet | seat which has a pattern in a plane direction.

  The pattern may have irregularity.

  Further, the pattern can have regularity.

  A honeycomb film can be used as the sheet.

  The observation target extraction processing apparatus of the present invention irradiates light to a sheet-like observation target support member that supports the observation target on which the observation target is installed, and emits light emitted from the observation target support member on which the observation target is installed. An observation image acquisition unit that acquires two types of images with different spectral characteristics obtained by imaging the observation target support member on which the observation target is set, and a correction coefficient calculation that calculates a correction coefficient based on the two types of images And one of the two types of images is corrected based on a correction coefficient to generate a corrected image, and an extracted image from which an observation target is extracted is generated based on the corrected image and the other image And an extraction processing unit.

  In the observation target extraction processing method of the present invention, the observation target support member is imaged by irradiating the sheet-like observation target support member supporting the observation target with light and detecting the light emitted from the observation target support member. Obtaining two types of first and second images having different spectral characteristics, calculating a correction coefficient based on the first and second images, irradiating light to the observation target support member on which the observation target is installed, By detecting light emitted from the observation target support member on which the observation target is installed, two types of third and fourth images having different spectral characteristics obtained by imaging the observation target support member on which the observation target is set are acquired. Then, one of the third and fourth images is corrected based on the correction coefficient to generate a corrected image, and the extracted image from which the observation target is extracted based on the corrected image and the other image Generate The features.

  The observation target extraction processing method of the present invention irradiates light to a sheet-like observation target support member that supports the observation target on which the observation target is installed, and is emitted from the observation target support member on which the observation target is installed. By detecting light, the observation target support member on which the observation target is installed is imaged to obtain two types of images having different spectral characteristics, and a correction coefficient is calculated based on the two types of images. One of the images is corrected based on a correction coefficient to generate a corrected image, and an extracted image from which an observation target is extracted is generated based on the corrected image and the other image .

  The observation object extraction processing program of the present invention images the observation target support member by irradiating light to the sheet-like observation target support member that supports the observation target, and detecting light emitted from the observation target support member. A correction image acquisition unit that acquires two types of first and second images having different spectral characteristics, a correction coefficient calculation unit that calculates a correction coefficient based on the first and second images, and an observation target are installed Two types of spectral characteristics obtained by irradiating the observation target support member with light and detecting the light emitted from the observation target support member on which the observation target is set, and imaging the observation target support member on which the observation target is set. An observation image acquisition unit that acquires the third and fourth images, and corrects one of the third and fourth images based on the correction coefficient to generate a corrected image. Based on the image, and characterized by causing a computer to function as extraction processing unit for generating an extracted image observation target is extracted.

  The observation object extraction processing program of the present invention irradiates light onto a sheet-like observation target support member that supports the observation target on which the observation target is installed, and emits light emitted from the observation target support member on which the observation target is installed. An observation image acquisition unit that acquires two types of images having different spectral characteristics by detecting the image, a correction coefficient calculation unit that calculates a correction coefficient based on the two types of images, and one of the two types of images And generating a corrected image based on the correction coefficient, and causing the computer to function as an extraction processing unit that generates an extracted image from which an observation target is extracted based on the corrected image and the other image. To do.

  The observation image imaging display control system of the present invention includes the observation target extraction processing device, an imaging device that captures the first to fourth images, the imaging of the first to fourth images in the imaging device, and the observation target extraction. A control unit that controls generation of an extracted image based on the first to fourth images in the processing device and display of the extracted image; and an imaging start instruction receiving unit that receives an input of an imaging start instruction. In response to an input of an imaging start instruction, imaging of the first to fourth images, generation of an extracted image based on the first to fourth images, and display of the extracted image are automatically performed continuously. Features.

  An observation image capturing display control system according to the present invention includes the observation target extraction processing device, an imaging device that captures two types of images, two types of images captured by the imaging device, and two types of observation target extraction processing devices. A control unit that controls generation of an extracted image based on an image and display of the extracted image, and an imaging start instruction receiving unit that receives an input of an imaging start instruction. Two types of images are captured, generation of an extracted image based on the two types of images, and display of the extracted image are automatically performed continuously.

  According to the observation object extraction processing device, method, program, and observation image imaging display control system of the present invention, the first and second images having different spectral characteristics are obtained by irradiating the observation object support member with light, A correction coefficient is calculated based on the first and second images, and two types of third and fourth images having different spectral characteristics are acquired by irradiating light onto the observation target support member on which the observation target is installed. Then, one of the third and fourth images is corrected based on the correction coefficient to generate a corrected image, and the extracted image from which the observation target is extracted based on the corrected image and the other image For example, even when the observation target support member on which the observation target is installed has an irregular pattern, the extracted image in which the observation target support member pattern is appropriately removed and the observation target is extracted is obtained. Can be generated.

  According to the observation target extraction processing device, method, program, and observation image capturing / display control system of the present invention, two types of images having different spectral characteristics are acquired by irradiating the observation target support member with light. Then, a correction coefficient is calculated based on the two types of images, one of the two types of images is corrected based on the correction coefficient to generate a corrected image, and the corrected image and the other image are generated. Based on this, an extracted image from which the observation target is extracted is generated. For example, even when the observation target support member on which the observation target is installed has an irregular pattern, the image of the observation target support member is appropriately removed. In addition, an extracted image from which the observation target is extracted can be generated.

The figure which shows schematic structure of the microscope system using 1st Embodiment of the observation object extraction processing apparatus of this invention. A plan view showing a specific configuration of the slit plate Plan view showing specific configuration of phase plate The flowchart for demonstrating the effect | action of the microscope system using 1st Embodiment of the observation object extraction processing apparatus of this invention. The figure which showed the 1st gray image and the 2nd gray image typically The figure which showed the 3rd gray image and the 4th gray image typically The figure which shows an example of a subtraction image The figure which shows an example of an extraction image Schematic block diagram showing a modification of the microscope system shown in FIG. The figure which shows schematic structure of the microscope system using 2nd Embodiment of the observation object extraction processing apparatus of this invention. The flowchart for demonstrating the effect | action of the microscope system using 2nd Embodiment of the observation object extraction processing apparatus of this invention. The figure which showed the 1st fluorescence image and the 2nd fluorescence image typically The figure which showed the 3rd fluorescence image and the 4th fluorescence image typically The figure which shows schematic structure of the microscope system using 3rd Embodiment of the observation object extraction processing apparatus of this invention. The flowchart for demonstrating the effect | action of the microscope system using 3rd Embodiment of the observation object extraction processing apparatus of this invention.

  Hereinafter, a microscope system using an embodiment of the observation object extraction processing apparatus of the present invention will be described in detail with reference to the drawings. FIG. 1 is a diagram showing a schematic configuration of a microscope system using the first embodiment of the observation object extraction processing apparatus of the present invention.

  The microscope system according to the first embodiment includes a microscope main body 1, an observation object extraction processing device 50, and a display device 60.

  As shown in FIG. 1, the microscope main body 1 according to the first embodiment includes a visible light irradiation unit 10, an excitation light irradiation unit 20, an imaging optical system 30, and an imaging unit 40.

  A stage 82 is provided between the visible light irradiation unit 10 and the imaging optical system 30, and a culture vessel 80 in which the observation target S is accommodated is installed on the stage 82. The microscope system of this embodiment includes a stage drive mechanism (not shown) that moves the stage 82 in the X direction and the Y direction orthogonal to each other. The X direction and the Y direction are directions orthogonal to each other on a plane parallel to the installation surface of the observation target S.

  As the culture vessel 80, a petri dish, a dish, a well plate, or the like can be used. The culture vessel 80 and the stage 82 are made of a material that transmits visible light emitted from the visible light irradiation unit 10.

  An observation target support member 81 that supports the observation target S is provided on the bottom surface of the culture vessel 80. As the observation target support member 81, a sheet-like porous member can be used. In particular, the observation target support member 81 of the present embodiment is formed of a material that emits fluorescence when irradiated with the excitation light emitted from the excitation light irradiation unit 20. As a material of the observation target support member 81, for example, a phosphor-containing resin containing a phosphor that emits fluorescence may be used, or a resin that emits fluorescence may be used.

  Further, as the observation target support member 81, as described above, it is desirable to use a porous member in which a large number of holes are arranged in a predetermined pattern in the plane direction. It is desirable that a large number of holes be arranged irregularly. In this case, the observation target support member 81 may be formed by, for example, a self-organizing action, or may be formed using a plateless printing technique such as an ink jet printer or a 3D printer. As a manufacturing method by the self-organizing action, for example, a manufacturing method in which a dew phenomenon of water vapor is combined with a hydrophobic solvent-based solution cast film forming method can be used. In addition, the pattern of many holes is not necessarily an irregular pattern, and a resin sheet in which holes are arranged in a regular pattern may be used as the observation target support member 81. Specifically, for example, a honeycomb film or the like may be used as the observation target support member 81.

  The observation object S includes, for example, a group of cells to be cultured. Specifically, for example, pluripotent stem cells such as iPS cells and ES cells, nerves induced to differentiate from stem cells, skin, cardiac muscle or liver cells, or the human body There are skin, retina, heart muscle, blood cells, nerves or organ cells removed from the body. Note that the observation target S is not limited to the cells as described above, but may be microorganisms, food, crystals, or the like.

  Further, in the culture container 80 of the present embodiment, the observation target support member 81 and the observation target S described above and the culture solution C for culturing the observation target S are accommodated.

  The visible light irradiation unit 10 of the microscope main body 1 irradiates the observation target S and the observation target support member 81 accommodated in the culture vessel 80 with illumination light L1 for so-called phase difference measurement. In the embodiment, the ring-shaped illumination light L1 is irradiated as the phase difference measurement illumination light. Specifically, the visible light irradiation unit 10 of the present embodiment has a white light source 11 that emits white light and a ring-shaped slit, and the white light emitted from the white light source 11 is incident on the ring-shaped illumination. The slit plate 12 that emits the light L1 and the ring-shaped illumination light L1 emitted from the slit plate 12 are incident, and the first objective lens 13 that irradiates the observation target S with the incident ring-shaped illumination light L1. And.

  FIG. 2 is a diagram showing a specific configuration of the slit plate 12. As shown in FIG. 2, the slit plate 12 is provided with a ring-shaped slit 12 a that transmits white light to a light blocking plate 12 b that blocks white light emitted from the white light source 11. Passes through the slit 12a to form the ring-shaped illumination light L1.

  The excitation light irradiation unit 20 includes an excitation light source 21 that emits excitation light L2 and a collimator lens 22 that makes the excitation light L2 emitted from the excitation light source 21 parallel light. As the excitation light source 21, for example, a light emitting diode, a laser diode, a high-pressure mercury light source, a halogen light source, or the like can be used. The excitation light L2 emitted from the excitation light source 21 is light having spectral characteristics different from the white light emitted from the white light source 11.

  Specifically, for example, when a light emitting diode is used as the excitation light source 21, light having a peak wavelength between 340 nm and 410 nm can be used. Further, when a light emitting diode or a laser diode is used as the excitation light source 21, heat applied to the observation object S can be suppressed, and irradiation on and off can be stably switched at high speed.

  The imaging optical system 30 includes a second objective lens 31, a phase plate 32, a half mirror 33, and an imaging lens 34. FIG. 3 is a plan view showing a specific configuration of the phase plate 32. As shown in FIG. 3, the phase plate 32 is obtained by forming a phase ring 32a with respect to a transparent plate 32b transparent to the wavelength of the ring-shaped illumination light L1. Note that the size of the slit 12a described above is in a conjugate relationship with the phase ring 32a.

  The phase ring 32a is a ring in which a phase film that shifts the phase of incident light by a quarter wavelength and a neutral density filter that attenuates the incident light are formed. When the direct light incident on the phase ring 32a passes through the phase ring 32a, the phase is shifted by a quarter wavelength, and the brightness is weakened. On the other hand, most of the diffracted light diffracted by the observation object S passes through the transparent plate 32b of the phase plate 32, and its phase and brightness do not change.

  The second objective lens 31 is moved in the Z direction by an imaging optical system driving unit (not shown). Autofocus control is performed by the movement of the second objective lens 31 in the Z direction, and the contrast of the image captured by the imaging unit 40 is adjusted.

  The half mirror 33 transmits the transmitted light L3 (direct light and diffracted light) that has passed through the observation object S and the phase plate 32, and reflects the excitation light L2 emitted from the excitation light irradiation unit 20 in the direction of the observation object S. It has the optical characteristic to do. Further, the half mirror 33 has an optical characteristic of transmitting the fluorescence L4 emitted from the observation target support member 81 when the observation target support member 81 is irradiated with the excitation light L2.

  The imaging lens 34 receives the transmitted light L3 or fluorescence L4 that has passed through the half mirror 33, and forms an image of these lights on the imaging unit 40.

  The imaging unit 40 includes an imaging element that captures a fluorescent image and a transmission image formed by the imaging lens 34. As the image pickup device, a charge-coupled device (CCD) image sensor, a complementary metal-oxide semiconductor (CMOS) image sensor, or the like is used. In the present embodiment, an imaging device provided with RGB color filters is used as the imaging unit 40. However, the image sensor is not limited to the one provided with the color filter, and a monochrome image sensor may be used.

  The observation target extraction processing device 50 is obtained by installing an embodiment of the observation target extraction processing program of the present invention on a computer. The observation target extraction processing device 50 includes a central processing unit, a semiconductor memory, a hard disk, and the like, and an embodiment of the observation target extraction processing program is installed on the hard disk. When this program is executed by the central processing unit, a correction image acquisition unit 51, a correction coefficient calculation unit 52, an observation image acquisition unit 53, an extraction processing unit 54, and a display control unit 55 as shown in FIG. Operate.

  The correction image acquisition unit 51 detects the fluorescence L4 emitted from the observation target support member 81 by irradiating the observation target support member 81 on which the observation target S is not installed with the excitation light L2, thereby observing the observation target support member 81. And a transmitted light transmitted through the observation target support member 81 by irradiation of the ring-shaped illumination light L1 to the observation target support member 81 on which the observation target S is not installed. By detecting L3, a transmission image (hereinafter referred to as a second image) obtained by imaging the observation target support member 81 is acquired.

  The correction coefficient calculation unit 52 calculates a correction coefficient based on the first and second images acquired by the correction image acquisition unit 51. A method for calculating the correction coefficient will be described in detail later.

  The observation image acquisition unit 53 detects the fluorescence L4 emitted from the observation target support member 81 by irradiating the observation target support member 81 on which the observation target S is set with the excitation light L2, so that the observation target S is set. Irradiating the observation target support member 81 with the ring-shaped illumination light L1 onto the observation target support member 81 on which the observation target S is installed , A transmission image (hereinafter referred to as a fourth image) obtained by imaging the observation target support member 81 on which the observation target S is installed by detecting the transmitted light L3 transmitted through the observation target support member 81 and the observation target S. To get.

  The third image and the fourth image are picked up by switching between irradiation with visible light L1 and irradiation with excitation light L2 in the same field of view. On the other hand, the first image and the second image described above are not necessarily captured in the same field of view, and may be images captured in different fields of view.

  The extraction processing unit 54 corrects the third image picked up by detecting the fluorescence based on the correction coefficient to generate a corrected image, and based on the corrected image and the fourth image, the observation target S is detected. An extracted image is generated. The extracted image generation process will be described in detail later.

  The display control unit 55 causes the display device 60 to display the extracted image generated by the extraction processing unit 54.

  A display device 60 is connected to the observation target extraction processing device 50. The display device 60 is composed of a display device such as a liquid crystal display, and displays the extracted image generated by the extraction processing unit 54.

  Next, the operation of the microscope system of this embodiment will be described with reference to the flowchart shown in FIG.

  First, the observation object support member 81 on which the observation object S is not installed and the culture vessel 80 containing the culture solution C are installed on the stage 82. Then, the excitation light L <b> 2 is irradiated from the excitation light irradiation unit 20 toward the observation target support member 81, the fluorescent image of the observation target support member 81 is captured by the imaging unit 40, and the first image is captured by the correction image acquisition unit 51. Obtained (S10).

  Next, the ring-shaped illumination light L <b> 1 is irradiated from the visible light irradiation unit 10 toward the observation target support member 81, and a transmission image of the observation target support member 81 is captured by the imaging unit 40. Are acquired (S12).

  The first image and the second image acquired by the correction image acquisition unit 51 are output to the correction coefficient calculation unit 52. Then, the correction coefficient calculation unit 52 performs gray scale conversion processing on the first image and the second image to make it gray, and generates a first gray image and a second gray image (S14). . FIG. 5I schematically shows the first gray image, and FIG. 5II schematically shows the second gray image.

  Next, the correction coefficient calculation unit 52 calculates the density contrast a of the first gray image and the density contrast b of the second image, and calculates the ratio b / a as the correction coefficient f (S16).

  It should be noted that the edge portion of the fluorescent image of the observation target support member 81 may be blurred. Therefore, in order to suppress this blur, sharpness enhancement processing such as unsharp mask processing may be performed on the first gray image before calculating the correction coefficient f. For example, when a colored medium is used as the culture medium C (corresponding to the medium), the color of the culture medium C may appear in the transmission image of the observation target support member 81. Therefore, a level correction process for suppressing the color of the culture medium C may be performed on the second gray image. Since the color signal of the culture medium C is uniformly contained in the image plane, it can be removed by such level correction processing.

  In this embodiment, when calculating the correction coefficient f as described above, the ratio of the density contrast of the first and second images is calculated. However, the ratio is not limited to the ratio of the density contrast. The ratio of the average value, the maximum value, the minimum value, or the median value of the values of the pixels constituting the pixel may be obtained. Further, the ratio of the corresponding pixel values of the two images may be calculated as the correction coefficient f. In this case, the correction coefficient f is calculated for each pixel.

  The correction coefficient f calculated by the correction coefficient calculation unit 52 is output to the extraction processing unit 54.

  Next, the observation target support member 81 in which the observation target S is installed and the culture vessel 80 in which the culture solution C is stored are installed on the stage 82. Then, the excitation light L2 is irradiated from the excitation light irradiation unit 20 toward the observation target support member 81, the fluorescent image of the observation target support member 81 is captured by the imaging unit 40, and the third image is captured by the observation image acquisition unit 53. Obtained (S18).

  Next, the ring-shaped illumination light L1 is irradiated from the visible light irradiation unit 10 toward the observation target support member 81 on which the observation target S is installed, and transmission images of the observation target support member 81 and the observation target S are captured by the imaging unit 40. The captured image is captured and the fourth image is acquired by the observation image acquisition unit 53 (S20).

  Then, the third image and the fourth image acquired by the observation image acquisition unit 53 are output to the extraction processing unit 54. Similarly to the first image and the second image, the extraction processing unit 54 performs gray scale conversion processing on the third image and the fourth image to make the third gray image and the fourth image gray. Are generated (S22).

  FIG. 6I schematically shows a third gray image, and FIG. 6II schematically shows a fourth gray image. As shown in FIGS. 6I and 6II, in the present embodiment, no fluorescence is emitted from the cell group of the observation target S, and the image of the observation target S does not appear on the third gray image. It is assumed that the image of the observation target S appears only on a certain fourth gray image.

  Note that, like the first gray image described above, sharpness enhancement processing such as unsharp mask processing may be performed on the third gray image. Further, similarly to the second gray image, the level correction process may be performed on the fourth gray image.

  Next, the extraction processing unit 54 multiplies the third gray image by the correction coefficient f described above to generate a corrected image (S24). And the extraction process part 54 produces | generates the subtraction image from which the image of the observation object support member 81 was remove | subtracted by subtracting a correction image from a 4th gray image (S26). FIG. 7 is a diagram schematically showing a subtraction image, and a dotted line shown in FIG. 7 shows a portion of the image of the observation target support member 81 removed from the fourth gray image by the subtraction process described above. .

  As shown in FIG. 7, when the corrected image is subtracted from the fourth gray image, the portion of the observation target support member 81 is removed, and thus the observation target support member 81 and the observation target S overlap with each other. Will be removed. Therefore, the extraction processing unit 54 performs a morphological processing closing process or the like on the subtracted image obtained by subtracting the correction image from the fourth gray image, so that a portion where the observation target support member 81 and the observation target S overlap each other is obtained. Perform the filling process. FIG. 8 is a diagram schematically showing an extracted image generated by performing a closing process on a subtracted image.

  Then, the extracted image generated by the extraction processing unit 54 is output to the display control unit 55, and the display control unit 55 displays the input extracted image on the display device 60 (S28).

  According to the microscope system of the first embodiment, the observation target support member 81 on which the observation target S is not installed is irradiated with visible light and excitation light, and the first and second images of the observation target support member 81 are imaged. Are obtained, the correction coefficient is calculated based on the first and second images, the observation target support member 81 on which the observation target S is installed is irradiated with visible light and excitation light, and the observation target S is installed. The third and fourth images obtained by imaging the observed object support member 81 are acquired. Then, the third image is corrected based on the correction coefficient to generate a corrected image, and based on the corrected image and the fourth image, an extracted image from which the observation target is extracted is generated. Even when the observation target support member 81 has an irregular pattern, the pattern of the observation target support member 81 is appropriately removed, and an extracted image in which the observation target S is extracted can be generated.

  In the microscope system of the first embodiment (corresponding to the observation image capturing / displaying control system), as shown in FIG. 9, the microscope main body 1 (corresponding to the imaging device) and the observation target extraction processing device 50 are provided. You may make it provide the control part 500 which controls automatically, and the imaging start instruction | indication reception part 501 which receives the input of the imaging start instruction | indication by a user. The imaging start instruction receiving unit 501 is provided with an input device such as a keyboard and a mouse.

  Then, in response to the input of the imaging start instruction received by the imaging start instruction receiving unit 501, the control unit 500 captures the first to fourth images described above and the extraction using the first to fourth images. The generation of the image and the display of the extracted image may be automatically performed continuously. At this time, it is desirable to complete an example of the operation from taking the first and second images to displaying the extracted image within one minute.

  In the microscope system of the first embodiment, the correction coefficient is calculated using the first image and the second image obtained by imaging the observation target support member 81 where the observation target S is not installed. However, although there is a possibility that the image quality is somewhat deteriorated, the third image and the third image obtained by imaging the observation target support member 81 on which the observation target S is installed without imaging the first image and the second image. The correction coefficient may be calculated using the four images. Also in this case, the control unit 500 captures the above-described third and fourth images and the third and fourth images according to the input of the imaging start instruction received by the imaging start instruction reception unit 501. The generation of the extracted image using and the display of the extracted image may be automatically performed continuously. Also in this case, it is desirable to complete an example of the operation from the third and fourth image capturing to the display of the extracted image within one minute.

  Next, a microscope system using the second embodiment of the observation target extraction processing apparatus of the present invention will be described. FIG. 10 is a diagram showing a schematic configuration of a microscope system using the second embodiment of the observation object extraction processing apparatus of the present invention.

  The microscope system according to the second embodiment includes a microscope main body 2, an observation target extraction processing device 70, and a display device 60. The microscope system according to the second embodiment is different from the microscope system according to the first embodiment in the configuration of the microscope main body 2 and in the extraction image generation processing in the observation target extraction processing device 70.

  The microscope main body 2 of the second embodiment does not capture a transmission image by irradiation with visible light unlike the microscope main body 1 of the first embodiment, and the observation target S and the observation target support member 81 are irradiated by excitation light. The fluorescence emitted from is detected using two types of filters having different spectral characteristics. In the present embodiment, it is assumed that fluorescence is emitted not only from the observation target support member 81 but also from the observation target S. The observation object S may itself emit fluorescence or may be fluorescently labeled.

  Specifically, the microscope body 2 of the second embodiment includes an excitation light irradiation unit 20, an imaging optical system 300, and an imaging unit 40, as shown in FIG. The culture vessel 80 shown in FIG. 10, the observation target S, the culture medium C, the observation target support member 81, and the stage 82 accommodated in the culture vessel 80 are substantially the same as those in the first embodiment. However, the culture vessel 80 and the stage 82 of the present embodiment are formed of a material that transmits fluorescence emitted from the observation target S and the observation target support member 81 by irradiation of excitation light.

  The excitation light irradiation unit 20 has the same configuration as that of the first embodiment, and includes an excitation light source 21 that emits the excitation light L2 and a collimator lens 22 that uses the excitation light L2 emitted from the excitation light source 21 as parallel light. I have.

  The imaging optical system 300 includes an objective lens 301, a half mirror 302, an imaging lens 303, and a filter unit 304.

  The objective lens 301 is moved in the Z direction by an imaging optical system driving unit (not shown). Autofocus control is performed by the movement of the objective lens 301 in the Z direction, and the contrast of the image captured by the imaging unit 40 is adjusted.

  The half mirror 302 reflects the excitation light L2 emitted from the excitation light irradiation unit 20 in the direction of the observation target S, and transmits the fluorescence emitted from the observation target S and the observation target support member 81 by the irradiation of the excitation light L2. It has the optical characteristic to do.

  The imaging lens 303 is for entering the fluorescence that has passed through the half mirror 302 and imaging the incident fluorescence on the imaging unit 40.

  As shown in FIG. 10, the filter unit 304 is provided between the imaging lens 303 and the imaging unit 40, and includes two types of fluorescent filters having different spectral characteristics. Specifically, as shown in FIG. 10, the filter unit 304 of the present embodiment includes a first fluorescent filter 304a, a second fluorescent filter 304b, and an exchange mechanism (not shown) for exchanging these filters. ing. The first fluorescent filter 304a and the second fluorescent filter 304b have different spectral characteristics as described above. For example, the first fluorescent filter 304a having a transmission peak wavelength between 330 nm and 385 nm is used. As the second fluorescent filter 304b, a filter having a transmission peak wavelength at 460 to 495 nm can be used. However, it is not limited to the combination of fluorescent filters having such spectral characteristics, and other combinations may be used.

  In the present embodiment, the first fluorescent filter 304a and the second fluorescent filter 304b described above are switched in order to acquire two types of fluorescent images having different spectral characteristics. However, the present invention is not limited to such a configuration. Two types of fluorescent mirror units that combine a filter that transmits excitation light, an absorption filter that absorbs fluorescence, and a dichroic mirror may be used, and these fluorescent mirror units may be replaced. As the fluorescent mirror unit, for example, a combination of U-MWIB3 (manufactured by Olympus Corporation) and U-MWU2 (manufactured by Olympus Corporation) can be used. However, the present invention is not limited to these combinations, and other combinations of fluorescent mirror units may be used.

  The imaging unit 40 includes an imaging element that captures a fluorescent image that has passed through the first fluorescent filter 304a and a fluorescent image that has passed through the second fluorescent filter 304b. As the image sensor, for example, an image sensor provided with RGB color filters is used. Note that the image sensor is not limited to the RGB color filter, and other image sensors may be used as long as the image sensor is provided with a plurality of filters having different spectral characteristics.

  Similar to the observation target extraction processing device 50 of the first embodiment, the observation target extraction processing device 70 of the second embodiment is a corrected image acquisition unit 71, a correction coefficient calculation unit 72, an observation image acquisition unit 73, and an extraction process. A unit 74 and a display control unit 75 are provided.

  The corrected image acquisition unit 71 is a fluorescence emitted from the observation target support member 81 by the irradiation of the excitation light L2 to the observation target support member 81 on which the observation target S is not installed, and the first fluorescent filter 304a. Fluorescence image obtained by imaging the observation target support member 81 by detecting the passed fluorescence L5 (hereinafter referred to as first fluorescence image), and fluorescence emitted from the observation target support member 81 due to irradiation with the excitation light L2. Thus, by detecting the fluorescence L6 that has passed through the second fluorescence filter 304b, a fluorescence image obtained by imaging the observation target support member 81 (hereinafter referred to as a second fluorescence image) is acquired.

  The correction coefficient calculation unit 52 calculates a correction coefficient based on the first and second fluorescent images acquired by the correction image acquisition unit 51. A method for calculating the correction coefficient will be described in detail later.

  The observation image acquisition unit 53 is fluorescence emitted from the observation target support member 81 and the observation target S by the irradiation of the excitation light L2 to the observation target support member 81 on which the observation target S is installed, and is the first fluorescence. By detecting the first fluorescence L5 that has passed through the filter 304a, a fluorescence image obtained by imaging the observation target support member 81 on which the observation target S is placed (hereinafter referred to as a third fluorescence image), and irradiation with the excitation light L2 The observation target support member 81 on which the observation target S is installed by detecting the second fluorescence L6 that has been emitted from the observation target support member 81 and the observation target S and has passed through the second fluorescent filter 304b. Is acquired as a fluorescent image (hereinafter referred to as a fourth fluorescent image).

  Note that the third fluorescent image and the fourth fluorescent image are captured by irradiation with the excitation light L2 in the same field of view. In contrast, the first fluorescent image and the second fluorescent image described above do not necessarily have to be captured in the same field of view, and may be images captured in different fields of view.

  The extraction processing unit 74 detects the first fluorescence L5 that has passed through the first fluorescence filter 304a, and based on the correction coefficient, the third fluorescence image obtained by imaging the observation target support member 81 on which the observation target S is installed. Are corrected to generate a corrected image, and based on the corrected image and the fourth fluorescent image, an extracted image from which the observation target S is extracted is generated. The extracted image generation process will be described in detail later.

  The display control unit 75 causes the display device 60 to display the extracted image on which the extraction processing has been performed in the extraction processing unit 74.

  Next, the operation of the microscope system of this embodiment will be described with reference to the flowchart shown in FIG.

  First, the observation object support member 81 on which the observation object S is not installed and the culture vessel 80 containing the culture solution C are installed on the stage 82. Then, the excitation light L2 is irradiated from the excitation light irradiation unit 20 toward the observation target support member 81 in a state where the first fluorescent filter 304a is disposed on the optical path of the fluorescence. A fluorescence image emitted from the observation target support member 81 by the irradiation of the excitation light L2 and having passed through the first fluorescence filter 304a is captured by the imaging unit 40, and the first fluorescence is captured by the correction image acquisition unit 71. An image is acquired (S30).

  Next, the first fluorescent filter 304a is removed from the fluorescent light path by the replacement mechanism of the filter unit 304, and the second fluorescent filter 304b is placed on the fluorescent light path instead. Then, the excitation light L2 is irradiated again from the excitation light irradiation unit 20 toward the observation target support member 81 in a state where the second fluorescent filter 304b is disposed on the optical path of the fluorescence. A fluorescence image emitted from the observation support member 81 by the irradiation of the excitation light L2 and passing through the second fluorescence filter 304b is captured by the imaging unit 40, and the second fluorescence is captured by the correction image acquisition unit 71. An image is acquired (S32). FIG. 12I schematically shows the first fluorescence image, and FIG. 12II schematically shows the second fluorescence image.

  The first fluorescence image and the second fluorescence image acquired by the correction image acquisition unit 71 are output to the correction coefficient calculation unit 72. Then, the correction coefficient calculation unit 72 of the present embodiment converts the R (red) component fluorescence image (hereinafter referred to as the first R component fluorescence image) and the G (green) component from the first fluorescence image that is a color image. A fluorescence image (hereinafter referred to as a first G component fluorescence image) and a B (blue) component fluorescence image (hereinafter referred to as a first B component fluorescence image) are generated. Further, from the second fluorescence image which is a color image, an R (red) component fluorescence image (hereinafter referred to as a second R component fluorescence image) and a G (green) component fluorescence image (hereinafter referred to as a second G component fluorescence). Image) and a B (blue) component fluorescence image (hereinafter referred to as a second B component fluorescence image).

  Then, the correction coefficient calculation unit 72 calculates the density contrast a (R) of the first R component fluorescence image and the density contrast b (R) of the second R component fluorescence image, and the ratio b (R) thereof. / a (R) is calculated as a correction coefficient f (R) (S34). Similarly, a correction coefficient f (G) is calculated based on the density contrast of the first G component fluorescence image and the density contrast of the second G component fluorescence image, and the density contrast of the first B component fluorescence image is calculated. A correction coefficient f (B) is calculated based on the density contrast of the second B component fluorescent image.

  Before calculating the correction coefficients f (R), f (G), and f (B), sharpness enhancement processing such as unsharp mask processing is performed on the first and second fluorescent images. Also good. In the present embodiment, as described above, when calculating the correction coefficients f (R), f (G), and f (B), the ratio of density contrasts is calculated. For example, the ratio of the average value, the maximum value, the minimum value, or the median value of the pixels constituting the image may be obtained. Further, the ratio of the corresponding pixel values of the two images may be calculated as correction coefficients f (R), f (G), and f (B). In this case, correction coefficients f (R), f (G), and f (B) are calculated for each pixel.

  The correction coefficients f (R), f (G), and f (B) calculated by the correction coefficient calculation unit 72 are output to the extraction processing unit 74.

  Next, the observation target support member 81 in which the observation target S is installed and the culture vessel 80 in which the culture solution C is stored are installed on the stage 82. Then, in a state where the first fluorescent filter 304a is again arranged on the fluorescent light path, the excitation light L2 is irradiated from the excitation light irradiation unit 20 toward the observation target support member 81 on which the observation target S is installed. A fluorescence image emitted from the observation target support member 81 and the observation target S by the irradiation of the excitation light L2 and having passed through the first fluorescent filter 304a is captured by the imaging unit 40, and is observed by the observation image acquisition unit 73. A third fluorescent image is acquired (S36).

  Next, the first fluorescent filter 304a is removed from the fluorescent light path by the replacement mechanism of the filter unit 304, and the second fluorescent filter 304b is placed on the fluorescent light path instead. Then, the excitation light L2 is again irradiated from the excitation light irradiation unit 20 toward the observation target support member 81 and the observation target S in a state where the second fluorescent filter 304b is disposed on the fluorescent light path. A fluorescence image emitted from the observation target support member 81 by the irradiation of the excitation light L2 and having passed through the second fluorescence filter 304b is captured by the imaging unit 40, and the fourth fluorescence is captured by the observation image acquisition unit 73. An image is acquired (S38).

  Then, the third fluorescence image and the fourth fluorescence image acquired by the observation image acquisition unit 73 are output to the extraction processing unit 74. FIG. 13I schematically shows a third fluorescence image, and FIG. 13II schematically shows a fourth fluorescence image. As shown in FIGS. 13I and 13II, in this embodiment, since fluorescence is emitted from the cell group of the observation target S, the image of the observation target S appears on the third and fourth fluorescence images.

  Here, similarly to the first and second fluorescent images described above, sharpness enhancement processing such as unsharp mask processing may be performed on the third and fourth fluorescent images.

  Next, the extraction processing unit 74 extracts a fluorescent image of an R (red) component (hereinafter referred to as a third R component fluorescent image) and a fluorescent image of a G (green) component (hereinafter, referred to as a third fluorescent image that is a color image). A third G component fluorescence image) and a B (blue) component fluorescence image (hereinafter referred to as a third B component fluorescence image) are generated. Further, from the fourth fluorescence image which is a color image, an R (red) component fluorescence image (hereinafter referred to as a fourth R component fluorescence image) and a G (green) component fluorescence image (hereinafter referred to as a fourth G component fluorescence). Image) and a B (blue) component fluorescence image (hereinafter referred to as a fourth B component fluorescence image).

  Then, the extraction processing unit 74 multiplies the third R component fluorescence image by the correction coefficient f (R) described above to generate an R component correction image, and the third G component fluorescence image described above. The correction coefficient f (G) is multiplied to generate a G component correction image, and the third B component fluorescence image is multiplied by the correction coefficient f (B) described above to generate a B component correction image. (S40).

  Next, the extraction processing unit 74 generates an R component subtraction image by subtracting the R component correction image from the fourth R component fluorescence image, and subtracts the G component correction image from the fourth G component fluorescence image. Thus, a G component subtraction image is generated, and a B component subtraction image is generated by subtracting the B component correction image from the fourth B component fluorescence image.

  By performing the subtraction process as described above, the R component, G component, and B component images of the observation target support member 81 are removed, but the fluorescence image of the observation target support member 81 and the fluorescence image of the observation target S are R Since the balance of the component, the G component, and the B component is different, the R component subtraction image, the G component subtraction image, and the B component subtraction image are images from which the observation target S is extracted.

Then, similarly to the first embodiment, the extraction processing unit 74 performs a morphological processing closing process on the R component subtraction image, the G component subtraction image, and the B component subtraction image, and thus the observation target support member. The process which fills the part with which 81 and the observation object S overlapped is performed, and R component extraction image, G component extraction image, and B component extraction image are produced | generated (S42).
The R component extracted image, the G component extracted image, and the B component extracted image generated by the extraction processing unit 74 are output to the display control unit 75, and the display control unit 75 converts the input G component extracted image and B component extracted image into the input G component extracted image and B component extracted image. Based on this, the color extraction image is displayed on the display device 60 (S44).

  According to the microscope system of the second embodiment, the first fluorescent image and the first fluorescent image that have passed through the first fluorescent filter 304a by irradiation of the excitation light to the observation target support member 81 on which the observation target S is not installed. The second fluorescent image that has passed through the second fluorescent filter 304b is acquired, a correction coefficient is calculated based on the first and second fluorescent images, and excitation to the observation target support member 81 on which the observation target S is installed By the light irradiation, a third fluorescent image that has passed through the first fluorescent filter 304a and a fourth fluorescent image that has passed through the second fluorescent filter 304b are acquired. Then, the third fluorescent image is corrected based on the correction coefficient to generate a corrected image, and based on the corrected image and the fourth fluorescent image, an extracted image from which the observation target is extracted is generated. Therefore, even when the observation target support member 81 has an irregular pattern, the pattern of the observation target support member 81 is appropriately removed, and an extracted image in which the observation target S is extracted can be generated.

  In the microscope system of the second embodiment (corresponding to the observation image capturing / displaying control system), similarly to the first embodiment, the microscope main body 2 (corresponding to the imaging device) and the observation object extraction processing device 70 and an imaging start instruction receiving unit that receives an input of an imaging start instruction by a user. The control unit responds to an input of an imaging start instruction received by the imaging start instruction receiving unit. Thus, the first to fourth fluorescent images described above, the generation of the extracted image using the first to fourth fluorescent images, and the display of the extracted image may be automatically performed continuously. Good. At this time, it is desirable to complete an example of the operation from taking the first and second fluorescent images to displaying the extracted image within one minute.

  In the microscope system of the second embodiment, the correction coefficient is calculated using the first fluorescent image and the second fluorescent image obtained by imaging the observation target support member 81 where the observation target S is not installed. However, there is a possibility that the image quality may be somewhat deteriorated, but the third image obtained by imaging the observation target support member 81 on which the observation target S is installed without imaging the first fluorescent image and the second fluorescent image. The correction coefficient may be calculated using the fluorescent image and the fourth fluorescent image.

  Also in this case, the control unit captures the third and fourth fluorescent images and the third and fourth fluorescent images according to the input of the imaging start instruction received by the imaging start instruction receiving unit. The generation of the extracted image using and the display of the extracted image may be automatically performed continuously. In this case as well, it is desirable to complete an example of the operation from taking the third and fourth fluorescent images to displaying the extracted image within one minute.

  Next, a microscope system using the third embodiment of the observation object extraction processing apparatus of the present invention will be described. FIG. 14 is a diagram showing a schematic configuration of a microscope system using the third embodiment of the observation object extraction processing apparatus of the present invention.

  The microscope system according to the third embodiment includes a microscope main body 3, an observation object extraction processing device 90, and a display device 60. The microscope system according to the third embodiment differs from the microscope system according to the first embodiment in the configuration of the microscope main body 2 and in the extraction image generation processing in the observation target extraction processing device 90.

  The microscope main body 3 according to the third embodiment does not take a fluorescent image by irradiation with excitation light unlike the microscope main body 1 according to the first embodiment, and the observation target S and the observation target support member 81 by irradiation with visible light. The transmitted light that has passed through is picked up by the image pickup units having different spectral sensitivities.

  Specifically, the microscope main body 3 of the third embodiment includes a visible light irradiation unit 10, an imaging optical system 310, and an imaging unit 400, as shown in FIG. The culture vessel 80 shown in FIG. 14, the observation target S, the culture medium C, the observation target support member 81, and the stage 82 accommodated in the culture vessel 80 are the same as those in the first embodiment.

  The visible light irradiation unit 10 includes the white light source 11, the slit plate 12, and the first objective lens 13 as in the first embodiment, and the observation target S and the observation target accommodated in the culture vessel 80. The support member 81 is irradiated with the ring-shaped illumination light L1.

  The imaging optical system 310 includes a second objective lens 311, a phase plate 312, and an imaging lens 313.

  The second objective lens 311 is moved in the Z direction by an imaging optical system driving unit (not shown). Autofocus control is performed by the movement of the second objective lens 311 in the Z direction, and the contrast of the image captured by the imaging unit 400 is adjusted.

  The phase plate 312 has the same configuration as the phase plate 32 of the first embodiment.

  The imaging lens 313 receives the transmitted light L3 (direct light and diffracted light) that has passed through the phase plate 312, and forms an image of the transmitted light L3 on the imaging unit 400.

  The imaging unit 400 captures a transmission image formed by the imaging lens 313, but the imaging unit 400 of this embodiment includes two types of imaging units having different spectral sensitivities as described above. It is.

  Specifically, as shown in FIG. 14, the imaging unit 400 of the present embodiment includes a first imaging unit 41, a second imaging unit 42, and an exchange mechanism (not shown) that exchanges these imaging units. I have. The first imaging unit 41 and the second imaging unit 42 have different spectral sensitivities as described above. For example, the first imaging unit 41 has spectral sensitivity in a band other than the visible light band. The second imaging unit 42 has spectral sensitivity in the visible light band. As the 1st imaging part 41, what has spectral sensitivity to infrared light or ultraviolet light, for example can be used.

  Note that having a spectral sensitivity in the visible light band is sufficient if the sensitivity to the visible light band is relatively higher than the sensitivity to bands other than the visible light band. Conversely, having spectral sensitivity in a band other than the visible light band only requires that the sensitivity to the band other than the visible light band is relatively higher than the sensitivity to the visible light band.

  Similar to the observation target extraction processing device 50 of the first embodiment, the observation target extraction processing device 90 of the third embodiment is a correction image acquisition unit 91, a correction coefficient calculation unit 92, an observation image acquisition unit 93, and an extraction process. A unit 94 and a display control unit 95 are provided.

  The corrected image acquisition unit 91 causes the first imaging unit 41 to transmit the transmitted light L3 transmitted through the observation target support member 81 by irradiation of the ring-shaped illumination light L1 to the observation target support member 81 where the observation target S is not installed. A transmission image (hereinafter referred to as a first transmission image) obtained by imaging the observation target support member 81 by detection and the transmitted light L3 transmitted through the observation target support member 81 by the irradiation of the ring-shaped illumination light L1 A transmission image obtained by imaging the observation target support member 81 (hereinafter referred to as a second transmission image) is acquired by detection by the imaging unit.

  The correction coefficient calculation unit 92 calculates a correction coefficient based on the first and second transmission images acquired by the correction image acquisition unit 91. A method for calculating the correction coefficient will be described in detail later.

  The observation image acquisition unit 93 performs first imaging of the transmitted light L3 transmitted through the observation target support member 81 and the observation target S by irradiating the observation target support member 81 on which the observation target S is installed with the ring-shaped illumination light L1. The observation target support member 81 is irradiated with the transmission image (hereinafter referred to as a third transmission image) obtained by imaging the observation target support member 81 on which the observation target S is set by detection by the unit 41 and the ring-shaped illumination light L1. And a transmission image (hereinafter referred to as a fourth transmission image) obtained by imaging the observation target support member 81 on which the observation target S is installed by detecting the transmitted light L3 transmitted through the observation target S by the second imaging unit 42. Is something to get.

  Note that the third transmission image and the fourth transmission image are captured by irradiation with the ring-shaped illumination light L1 in the same field of view. On the other hand, the first transmission image and the second transmission image described above do not necessarily have to be captured in the same field of view, and may be images captured in different fields of view.

  The extraction processing unit 94 corrects the third transmission image captured by the first imaging unit 41 based on the correction coefficient to generate a correction image, and based on the correction image and the fourth transmission image, An extraction image from which the observation object S is extracted is generated. The extracted image generation process will be described in detail later.

  The display control unit 95 causes the display device 60 to display the extracted image that has been subjected to the extraction processing in the extraction processing unit 94.

  Next, the operation of the microscope system of this embodiment will be described with reference to the flowchart shown in FIG.

  First, the observation object support member 81 on which the observation object S is not installed and the culture vessel 80 containing the culture solution C are installed on the stage 82. The ring-shaped illumination light L1 is irradiated from the visible light irradiation unit 10 toward the observation target support member 81 in a state where the first imaging unit 41 is installed at the imaging position of the imaging lens 313, and the observation target support is performed. The transmitted light L3 transmitted through the member 81 is detected by the first imaging unit 41, and the first transmitted image is acquired by the corrected image acquisition unit 91 (S50).

  Next, the first imaging unit 41 is removed from the imaging position of the imaging lens 313 by the exchange mechanism of the imaging unit 400, and the second imaging unit 42 is arranged at the imaging position of the imaging lens 313 instead. Is done. The ring-shaped illumination light L1 is irradiated from the visible light irradiation unit 10 toward the observation target support member 81 in a state where the second imaging unit 42 is disposed at the imaging position of the imaging lens 313, and the observation target support is performed. The transmitted light L3 transmitted through the member 81 is detected by the second imaging unit 42, and the second transmitted image is acquired by the corrected image acquisition unit 91 (S52).

  The first transmission image and the second transmission image acquired by the correction image acquisition unit 91 are output to the correction coefficient calculation unit 92. Then, the correction coefficient calculation unit 92 of the present embodiment includes a transmission image of an R (red) component (hereinafter referred to as a first R component transmission image) and a transmission image of a G (green) component (hereinafter referred to as a first transmission image). , First G component transmission image) and B (blue) component transmission image (hereinafter referred to as first B component transmission image). Further, a transmission image of the R (red) component (hereinafter referred to as a second R component transmission image) and a transmission image of the G (green) component (hereinafter referred to as a second G component transmission image) from the second transmission image. A transmission image of the B (blue) component (hereinafter referred to as a second B component transmission image) is generated.

  Then, the correction coefficient calculation unit 92 calculates the density contrast a (R) of the first R component transmission image and the density contrast b (R) of the second R component transmission image, and the ratio b (R) thereof. / a (R) is calculated as a correction coefficient f (R) (S54). Similarly, a correction coefficient f (G) is calculated based on the density contrast of the first G component transmission image and the density contrast of the second G component transmission image, and the density contrast of the first B component transmission image is calculated. A correction coefficient f (B) is calculated based on the density contrast of the second B component transmission image.

  In addition, before calculating the correction coefficients f (R), f (G), and f (B), the level correction process for suppressing the color of the culture medium C described above is performed on the first and second transmission images. You may make it give. In the present embodiment, as described above, when calculating the correction coefficients f (R), f (G), and f (B), the ratio of density contrasts is calculated. For example, the ratio of the average value, the maximum value, the minimum value, or the median value of the pixels constituting the image may be obtained. Further, the ratio of the corresponding pixel values of the two images may be calculated as correction coefficients f (R), f (G), and f (B). In this case, correction coefficients f (R), f (G), and f (B) are calculated for each pixel.

  The correction coefficients f (R), f (G), and f (B) calculated by the correction coefficient calculation unit 92 are output to the extraction processing unit 94.

  Next, the observation target support member 81 in which the observation target S is installed and the culture vessel 80 in which the culture solution C is stored are installed on the stage 82. The ring-shaped illumination light is directed from the visible light irradiation unit 10 toward the observation target support member 81 on which the observation target S is installed in a state where the first imaging unit 41 is installed again at the imaging position of the imaging lens 313. The transmitted light L3 irradiated with L1 and transmitted through the observation target support member 81 and the observation target S is detected by the first imaging unit 41, and a third transmission image is acquired by the observation image acquisition unit 93 (S56).

  Next, the first imaging unit 41 is removed from the imaging position of the imaging lens 313 by the exchange mechanism of the imaging unit 400, and the second imaging unit 42 is arranged at the imaging position of the imaging lens 313 instead. Is done. Then, in a state where the second imaging unit 42 is disposed at the imaging position of the imaging lens 313, the ring-shaped illumination light L1 is directed from the visible light irradiation unit 10 toward the observation target support member 81 where the observation target S is installed. , The transmitted light L3 transmitted through the observation target support member 81 and the observation target S is detected by the second imaging unit 42, and the fourth transmission image is acquired by the observation image acquisition unit 93 (S58).

  Then, the third transmission image and the fourth transmission image acquired by the observation image acquisition unit 93 are output to the extraction processing unit 94. In the present embodiment, the image of the observation target S appears in both the third and fourth transmission images.

  Here, as with the first and second transmission images described above, level correction processing may be performed on the third and fourth transmission images.

  Next, the extraction processing unit 94 transmits an R (red) component transmission image (hereinafter referred to as a third R component transmission image) and a G (green) component transmission image (hereinafter referred to as a third G image) from the third transmission image. Component transmission image) and B (blue) component transmission image (hereinafter referred to as a third B component transmission image). Further, from the fourth transmission image, an R (red) component transmission image (hereinafter referred to as a fourth R component transmission image) and a G (green) component transmission image (hereinafter referred to as a fourth G component transmission image), A transmission image of the B (blue) component (hereinafter referred to as a fourth B component transmission image) is generated.

  Then, the extraction processing unit 94 multiplies the third R component transmission image by the correction coefficient f (R) described above to generate an R component correction image, and the third G component transmission image described above. The correction coefficient f (G) is multiplied to generate a G component correction image, and the third B component transmission image is multiplied by the correction coefficient f (B) described above to generate a B component correction image. (S60).

  Next, the extraction processing unit 94 generates an R component subtraction image by subtracting the R component correction image from the fourth R component transmission image, and subtracts the G component correction image from the fourth G component transmission image. Thus, a G component subtraction image is generated, and a B component subtraction image is generated by subtracting the B component correction image from the fourth B component transmission image.

  By performing the subtraction process as described above, the R component, G component, and B component images of the observation target support member 81 are removed, but the transmission image of the observation target support member 81 and the transmission image of the observation target S are R Since the balance of the component, the G component, and the B component is different, the R component subtraction image, the G component subtraction image, and the B component subtraction image are images from which the observation target S is extracted.

  Then, similarly to the first embodiment, the extraction processing unit 94 performs a morphological closing process on the R component subtraction image, the G component subtraction image, and the B component subtraction image, and thus the observation target support member. The process which fills the part with which 81 and the observation object S overlapped is performed, and an R component extraction image, a G component extraction image, and a B component extraction image are produced | generated (S62).

  The R component extracted image, the G component extracted image, and the B component extracted image generated by the extraction processing unit 94 are output to the display control unit 95, and the display control unit 95 converts the input G component extracted image and B component extracted image into the input G component extracted image and B component extracted image. Based on this, a color extraction image is displayed on the display device 60 (S64).

  According to the microscope system of the third embodiment, the first transmission image captured by the first imaging unit 41 by irradiation of visible light onto the observation target support member 81 on which the observation target S is not installed, and The second transmission image captured by the second imaging unit 42 is acquired, a correction coefficient is calculated based on the first and second transmission images, and the observation target support member 81 on which the observation target S is installed is obtained. The third transmission image captured by the first imaging unit 41 and the fourth transmission image captured by the second imaging unit are acquired by irradiation with visible light. Then, the third transmission image is corrected based on the correction coefficient to generate a correction image, and based on the correction image and the fourth transmission image, an extraction image from which the observation target is extracted is generated. Therefore, even when the observation target support member 81 has an irregular pattern, the pattern of the observation target support member 81 is appropriately removed, and an extracted image in which the observation target S is extracted can be generated.

  In the microscope system of the third embodiment (corresponding to the observation image capturing / displaying control system), similarly to the first embodiment, the microscope main body 3 (corresponding to the imaging device) and the observation object extraction processing device 90, and an imaging start instruction receiving unit that receives an input of an imaging start instruction by a user. The control unit responds to an input of an imaging start instruction received by the imaging start instruction receiving unit. Thus, the first to fourth transmission images, the generation of the extraction image using the first to fourth transmission images, and the display of the extraction image may be automatically performed continuously. Good. At this time, it is desirable to complete an example of operations from capturing the first and second transmission images to displaying the extracted image within one minute.

  In the microscope system of the third embodiment, the correction coefficient is calculated using the first transmission image and the second transmission image obtained by imaging the observation target support member 81 where the observation target S is not installed. However, although there is a possibility that the image quality may be somewhat deteriorated, the third image obtained by imaging the observation target support member 81 on which the observation target S is installed without imaging the first transmission image and the second transmission image. The correction coefficient may be calculated using the transparent image and the fourth transparent image.

  Also in this case, the control unit performs the above-described third and fourth transmission images and the third and fourth transmissions according to the input of the imaging start instruction received by the imaging start instruction receiving unit. The generation of the extracted image used and the display of the extracted image may be automatically performed continuously. In this case as well, it is desirable to complete an example of the operation from taking the third and fourth transmission images to displaying the extracted image within one minute.

  In the microscope systems of the first to third embodiments, a phase difference image is captured as a transmission image. However, the present invention is not limited to this, and a configuration may be employed in which a bright field image is captured.

  In the microscope systems of the first to third embodiments, it is desirable that the observation target extraction processing device has a function of storing the extracted image generated in the extraction processing unit. The location where the extracted image is stored may be in the observation target extraction processing device, or may be stored in a cloud server or the like via a communication network.

  In the microscope systems of the first to third embodiments, a plurality of partial extracted images are obtained by dividing the imaging range of the observation target S installed on the observation target support member 81 into a plurality of ranges and capturing the images. One extracted image may be generated by generating and tiling the plurality of partially extracted images.

1, 2, 3 Microscope body 10 Visible light irradiation unit 11 White light source 12 Slit plate 12a Slit 12b Light shielding plate 13 First objective lens 20 Excitation light irradiation unit 21 Excitation light source 22 Collimator lens 30 Imaging optical system 31 Second objective Lens 32 Phase plate 32a Phase ring 32b Transparent plate 33 Half mirror 34 Imaging lens 40 Imaging unit 41 First imaging unit 42 Second imaging unit 50, 70, 90 Observation target extraction processing device 51, 71, 91 Correction image acquisition Units 52, 72, 92 Correction coefficient calculation units 53, 73, 93 Observation image acquisition units 54, 74, 94 Extraction processing units 55, 75, 95 Display control unit 60 Display device 80 Culture vessel 81 Observation target support member 82 Stage 300 Image optical system 301 Objective lens 302 Half mirror 303 Imaging lens 304 Filter unit 304a First fluorescent filter 3 04b Second fluorescent filter 310 Imaging optical system 311 Second objective lens 312 Phase plate 313 Imaging lens 400 Imaging unit 500 Control unit 501 Imaging start instruction receiving unit

Claims (5)

Light is emitted to an observation target support member that is a sheet-like porous member that supports the observation target on which the observation target is installed, and light emitted from the observation target support member on which the observation target is installed is detected. An observation image acquisition unit for acquiring two types of images with different spectral characteristics obtained by imaging the observation target support member on which the observation target is installed,
A correction coefficient calculation unit that calculates a correction coefficient based on the two types of images;
One of the two types of images is corrected based on the correction coefficient to generate a corrected image, and the observation target support included in the other image is generated based on the corrected image and the other image. An observation target extraction processing apparatus comprising: an extraction processing unit that generates an extraction image in which the observation target is extracted by removing a pattern caused by a plurality of holes of the member . The observation target support member, which is a sheet-like porous member that supports the observation target, on which the observation target is installed is irradiated with light, and light emitted from the observation target support member on which the observation target is installed is detected. In this way, the observation target support member on which the observation target is installed is imaged to obtain two types of images having different spectral characteristics,
A correction coefficient is calculated based on the two types of images,
One of the two types of images is corrected based on the correction coefficient to generate a corrected image, and the observation target support included in the other image is generated based on the corrected image and the other image. An observation object extraction processing method, wherein an extraction image from which the observation object is extracted is generated by removing a pattern caused by a plurality of holes of the member . Light is emitted to an observation target support member that is a sheet-like porous member that supports the observation target on which the observation target is installed, and light emitted from the observation target support member on which the observation target is installed is detected. An observation image acquisition unit that acquires two types of images having different spectral characteristics by:
A correction coefficient calculation unit that calculates a correction coefficient based on the two types of images;
One of the two types of images is corrected based on the correction coefficient to generate a corrected image, and the observation target support included in the other image is generated based on the corrected image and the other image. An observation object extraction processing program that causes a computer to function as an extraction processing unit that generates an extracted image in which the observation object is extracted by removing a pattern caused by a plurality of holes of a member . An observation object extraction processing apparatus according to claim 1;
An imaging device that captures the two types of images;
A control unit that controls imaging of the two types of images in the imaging device, generation of the extracted images based on the two types of images in the observation target extraction processing device, and display of the extracted images;
An imaging start instruction receiving unit that receives an input of an imaging start instruction;
In response to the input of the imaging start instruction, the control unit automatically and continuously performs imaging of the two types of images, generation of the extracted images based on the two types of images, and display of the extracted images. An observation image pickup display control system characterized in that: The observation target support member is composed of a member that emits fluorescence when irradiated with excitation light,
When the observation object does not emit fluorescence, the one image is a fluorescence image photographed by detecting fluorescence emitted from the observation object support member by irradiation of the excitation light, and the other image The image is a transmission image taken by detecting light transmitted through the observation target support member and the observation target,
When the observation target emits fluorescence, the one image is a fluorescence image taken by detecting fluorescence emitted from each of the observation target support member and the observation target by irradiation of the excitation light. The observation object extraction processing apparatus according to claim 1, wherein the other image is the transmission image.