JP7161218B2 - SAMPLE OBSERVATION METHOD, SAMPLE OBSERVATION DEVICE, AND MICROSCOPE - Google Patents

Description

TECHNICAL FIELD The present disclosure relates to a sample observation method in a microscope having an adjustable focus optical system, an apparatus therefor, and a microscope equipped with the same.

Transmission light microscopy is widely used in biomedical, food safety, and many other applications. When viewing a specimen with a microscope, light hits small particles in the specimen and is redirected before reaching the image sensor. As a result, the image sensor will capture the sum of the scattered rays through different paths. This light scattering obscures the viewed image. FIG. 1(b) is an observation image of a transmission electron microscope when the scattered matter is placed on the micrometer (FIG. 1(a)). Light scattering thus blurs the image.

Image obscuring due to scattering is a major challenge in biomedical imaging. For example, in spatio-spectral analysis of biological tissue, the absorption coefficient must be able to be measured accurately at specific points. However, the measured signal contains scattered light from different points than the specific point, and the scattered light contains information for different points within the tissue. For accurate measurements at specific points in living tissue, it is essential to separate direct light from specific points and scattered light from other points.

In the field of computational photography, there are several methods of extracting the direct components of specular reflection and diffuse reflection by removing light such as interreflection, which is a global component, from the reflected light from the sample. Proposed. For example, Non-Patent Document 1 discloses the use of high frequency illumination. High-frequency illumination can separate the direct component and the global component by irradiating the same location with a high-frequency spatial pattern (see, for example, Non-Patent Documents 3 to 8).
On the other hand, in Non-Patent Document 2, unlike the reflection-type technique of Non-Patent Document 1, separation of components in transmissive observation of a translucent object (a direct component that is a single scattered light from an observation point and other A transmissive method that separates the multi-point single-scattered light and multiple-scattered light from a global component has been introduced. This technique attempts to observe transmitted light using high-frequency illumination of parallel light. Therefore, in Non-Patent Document 2, a special lens (telecentric lens) is used to convert the high-frequency illumination into parallel light, and the parallel light is observed.

Nayar, S.; K. , Krishnan, G.; , Grossberg, M.; D. , Raskar, R. : Fast Separation of Direct and Global Components of a Scene Using High Frequency Illumination. ACM Transactions on Graphics 25(3), pp. 935-944 (2006) Tanaka, K.; , Mukaigawa, Y.; , Kubo, H.; , Matsushita, Y.; , Yagi, Y.; : Descattering of Transmissive Observation using Parallel High-frequency Illumination. In: IEEE Conference on Computational Photography, (2013) Lamond, B. , Peers, P.S. , Debevec, P.S. : Fast Image-based Separation of Diffuse and Specular Reflections. In ACM SIGGRAPH sketches, 2007. Gupta, M.; , Tian, Y.; , Narasimhan, S.; G. , Zhang, L.; : A Combined Theory of Defocused Illumination and Global Light Transport. International Journal of Computer Vision 98(2), pp. 146-167 (2012) Achar, S.; , Narasimhan, S.; G. : Multi Focus Structured Light for Recovering Scene Shape and Global Illumination. European Conference on Computer Vision (2014) Reinhard, E. , Khan, E. A. , Akyuz, A.; O.D. , Johnson, G.; : Color Imaging: Fundamentals and Applications. CRC Press (2008) Mukaigawa, Y.; , Yagi, Y.; , Raskar, R. : Analysis of Light Transport in Scattering Media. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 153 {160 (2010) Tanaka, K.; , Mukaigawa, Y.; , Kubo, H.; , Mtsushita, Y.; , Yagi, Y.; : Recovering Inner Slices of Translucent Objects by Multi-frequency Illumination. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 5464-5472. (2015) Salomatina, E. , Jiang, B. , Novak, J.; , Yaroslavsky, A.; N. : Optical Properties of Normal and Cancerous Human Skin in the Visible and Near-infrared Spectral Range. Journal of Biomedical Optics 11(6) (2006)

However, in the case of a microscope with a focus-adjustable optical system, information in the thickness direction of the sample is added. The transmission-type method used cannot separate the light reflected by the sample and the light transmitted through the sample into direct components and global components, which makes it difficult to sharpen images and perform spatial spectrum analysis. Therefore, in order to solve the above problems, the present invention is a sample observation method that enables sharpening of the image of the sample and spatial spectrum analysis even if it has an optical system that can adjust the focus (not parallel light). , a sample observation device, and a microscope.

In order to achieve the above object, a sample observation method and a sample observation device according to the present invention are observed at a certain pixel on the camera side,
When the observation point is positioned in the bright part of the illumination, the direct component (A) by the observation point single scattered light from the observation point in focus, the global component (A) by the multi-point single scattered light from the defocused point B), and from the global component (C) due to multiple scattered light,
By removing the global component (B) and the global component (C) when the observation point is positioned in the dark part of the illumination,
The direct component (A) is taken out by using the difference between the focused state that can be used as pattern illumination for bright and dark areas and the non-focused state that makes the pattern illumination uniform.

Specifically, the sample observation method according to the present invention includes:
a focusing procedure that focuses the photosensor and illumination in the Z direction to the XY plane containing the observation point within the sample;
After performing the focusing procedure, light intensity acquisition for acquiring, with the optical sensor, a first light intensity when the light of the illumination is applied to the observation point and a second light intensity when the light is applied to a point other than the observation point. a procedure;
Direct component (A) due to single scattered light at the observation point, global component (B) due to single scattered light at locations other than the observation point, and multiple scattering within the sample, which are included in the first light intensity Using the global component (C) by light and the global component (B) and the global component (C) contained in the second light intensity, the direct component (A) and the global component ( B) and an operation procedure for separating the global component (C);
I do.

Specifically, the sample observation device according to the present invention includes:
a focusing mechanism that focuses the optical sensor and illumination in the Z direction onto an XY plane containing an observation point within the sample;
In a state where both the optical sensor and the illumination are focused on the observation point by the focusing mechanism, a first light intensity when the observation point is illuminated with the light of the illumination, and when the illumination is applied to a point other than the observation point. a light intensity acquisition means for acquiring the second light intensity of with the optical sensor;
Direct component (A) due to single scattered light at the observation point, global component (B) due to single scattered light at locations other than the observation point, and multiple scattering within the sample, which are included in the first light intensity Using the global component (C) by light and the global component (B) and the global component (C) contained in the second light intensity, the direct component (A) and the global component ( B) and a computing unit that separates the global component (C);
Prepare.

If this sample observation device is of a transmissive type (a type in which light passes through the sample), the direction of the light path is the Z direction. In the focusing procedure, the focusing mechanism adjusts the focal point of the optical sensor-side lens between the optical sensor and the sample in the Z direction to the XY plane including the observation point in the sample, and adjusts the focus to the sample in the Z direction. On the other hand, the illumination-side lens between the illumination on the opposite side of the optical sensor and the sample is focused on the XY plane including the observation point.
When this sample observation device is of a reflective type (a type in which light is reflected by the sample), the thickness direction of the sample is defined as the Z direction. In the focusing procedure, the focusing mechanism adjusts the focal point of the optical sensor-side lens between the optical sensor and the sample in the Z direction to the XY plane including the observation point in the sample, and adjusts the focus to the sample in the Z direction. On the other hand, the illumination-side lens between the illumination on the opposite side of the optical sensor and the sample is focused on the XY plane including the observation point. Note that the photosensor-side lens and the illumination-side lens may be one common lens.

When observing a translucent object with a microscope, global components (B) and (C) are superimposed on the direct component (A) and must be separated. A plane in which the high-frequency illumination and/or the camera (light sensor) are out of focus (defocused plane) results in blurry images and homogenous light intensity. On the other hand, on the plane (focus plane) where the illumination and the camera (optical sensor) are focused, the pattern of the bright and dark parts of the illumination becomes clear, and the observation point can be illuminated or not illuminated. The camera observes the direct component (A), the global component (B) and the global component (C) when the observation point is irradiated with light, and the global component (B) and the global component when the observation point is not irradiated with light. Observe (C). The present invention utilizes the difference (difference) between the observation result when the observation point is irradiated with light and the observation result when the observation point is not irradiated with light to extract the direct component (A) (out of focus Since the light intensity is uniform on the plane, the global components (B) and (C) when the observation point is irradiated with light and the global components (B) and (C) when the observation point is not irradiated with light have the same information. Become.).

The calculator subtracts the second light intensity from the first light intensity to calculate a direct component (A), and performs an operation to obtain an image of the observation point.

The calculator subtracts the second light intensity from the first light intensity to calculate a direct component (A), and calculates the direct component (A) and the light intensity of each wavelength of the illumination from the observation point. It is characterized by performing an operation for obtaining an absorption coefficient.

By incorporating the sample observation device into a microscope, it becomes possible to remove the global component from the transmitted light and extract the component directly. That is, a microscope according to the present invention is a microscope including the sample observation device, the illumination, the optical sensor, and the focusing mechanism.

Therefore, the present invention can provide a sample observation method, a sample observation apparatus, and a microscope that enable sharpening of an image of a sample and spatial spectrum analysis even when the optical system has an adjustable focus. .

In the sample observation apparatus according to the present invention, the XY plane including the observation point is moved in the Z direction within the sample, and the optical sensor and the It further comprises a controller that focuses the illumination on the observation point, causes the light intensity acquisition means to acquire the first light intensity and the second light intensity, and causes the calculator to perform the calculation. do. By sequentially moving the focal points of both the sensor-side lens and the illumination-side lens in the Z direction, this sample observation device creates three-dimensional images and performs spatial spectrum analysis based on continuous calculation results in the Z direction. becomes possible.

The illumination is a high-frequency illumination formed of a photomask in which a bright portion that transmits light and a dark portion that blocks light are combined at an arbitrary ratio, and the light intensity acquisition means includes the optical sensor and the focal point of the illumination. It may be a drive mechanism that arbitrarily moves the sample or arbitrarily moves the photomask while maintaining .

The illumination has a pattern in which a bright portion that causes the light source to emit light and a dark portion that extinguishes the light source are combined at an arbitrary ratio, and the light intensity acquisition means obtains the light portion and the dark portion while maintaining the ratio. It may be a light source control unit that is moved arbitrarily.

INDUSTRIAL APPLICABILITY The present invention can provide a sample observation method, a sample observation apparatus, and a microscope that enable sharpening of an image of a sample and spatial spectrum analysis even when the optical system has an adjustable focus.

It is an image for explaining the subject of the present invention. It is a figure explaining transmitted light and scattered light. It is a figure explaining the observation method concerning this invention. It is a figure explaining the observation method concerning this invention. It is a figure explaining the image acquired with the microscope which concerns on this invention. It is a figure explaining the experiment which measures an absorption coefficient performed with the microscope which concerns on this invention, and a conventional method. It is a figure explaining the experimental result which measures an absorption coefficient performed with the microscope which concerns on this invention, and a conventional method. It is a figure explaining the microscope concerning this invention.

Embodiments of the present invention will be described with reference to the accompanying drawings. The embodiments described below are examples of the present invention, and the present invention is not limited to the following embodiments. In particular, in the following embodiments, a transmission microscope will be described, but similar results can be obtained using a reflection microscope as long as the illumination and the camera are on the same side of the sample. In addition, in this specification and the drawings, constituent elements having the same reference numerals are the same as each other.

[1] Superimposition of transmitted light and scattered light As described above, one of the reasons for unclear images observed with a transmitted light microscope is that the measured light differs from the light that has passed through the desired observation point. This is because the scattered light from the is superimposed and absorbed at different points in the sample. In this section, we examine these lights as they pass through the sample in a transmission microscope and describe how they overlap to reach the camera (image sensor).

To account for all the overlapping light, consider measuring the light intensity at the observation point with a transmission microscope. The light that reaches the optical sensor consists of observation point single-scattered light (directly transmitted light) that is scattered only at the observation point, multi-point single-scattered light that is scattered only once at another point at a different depth from the observation point, and Three types of multiply scattered light are considered. FIG. 2 is an explanatory diagram of the path of each light, and the observation points are indicated by white circles. The solid black arrow in FIG. 2(b) is observation point single scattered light (directly transmitted light) that reaches the optical sensor without being scattered after being scattered only at the observation point. This light is the direct component of the observed object. The dashed arrows in FIG. 2(c) indicate the single scattered light scattered only once at a point other than the observation point located at a different depth from the observation point within the sample, and the multiple scattered light scattered multiple times within the sample. It is light, and it is a global component. The scattered light of the global component overlaps with the single scattered light from the observation point (directly transmitted light) and reaches the same optical sensor (FIG. 2(a)). The global component reduces the proportion of the direct component at the photosensor and obscures the image. Therefore, there is a need to separate these overlapping lights reaching the light sensor into direct and global components.

[2] Separation of Direct Component and Global Component FIG. 3A is a diagram for explaining the transmission microscope of this embodiment that separates the direct component and the global component. The transmission microscope of the present embodiment is a microscope that includes an illumination 13, an optical sensor 15, an optical sensor-side lens 12, and an illumination-side lens 14. By incorporating the transmission-type sample observation device 100, the inside of the sample 50 can be observed. It is characterized by being able to observe single scattered light that scatters only at the point K. FIG. 8 is a diagram for explaining the X, Y, and Z directions, an observation point K within the sample 50, and an XY plane (slice) including the observation point K, in order to explain the transmission microscope of this embodiment.

Here, the transmission type sample observation device 100
A step of focusing the optical sensor side lens 12 between the optical sensor 15 and the sample 50 to the observation point K in the sample 50 having a depth of focus thinner than the thickness of the sample 50, and a focus thinner than the thickness of the sample 50 a focusing mechanism 18 that has a depth and performs at least one of a step of focusing an illumination-side lens 14 between the illumination 13 on the opposite side of the optical sensor 15 with respect to the sample 50 and the sample 50 to the observation point K;
light intensity acquisition means 19 for acquiring, with a photosensor 15, a first light intensity when the light of the illumination 13 is applied to the observation point K and a second light intensity when the light is applied to a point other than the observation point K;
A computing unit 17 that uses the light intensity obtained by subtracting the second light intensity from the first light intensity as a component of the transmitted light of the observation point K;
Prepare.

The calculator 17 controls the focus mechanism 18 and the light intensity acquisition means 19 so as to obtain transmission images (images obtained by the optical sensor 15) in the respective states of FIGS. 3B and 3C, which will be described later.
The focus mechanism 18 performs focus adjustment based on the calculation of the calculator 17, and can be realized by using a focus adjustment mechanism using a stepping motor, for example. Alternatively, the operator may manually operate a predetermined operation unit such as a focus adjustment knob to adjust the focus. Further, the focus mechanism 18 can move the focal points of both the sensor-side lens 12 and the illumination-side lens 14 in the optical axis direction (Z direction). This function allows the slice to be moved in the Z direction, ie to focus on an observation point K at any depth within the sample 50 .

The drive mechanism of the light intensity acquisition means 19 is for moving so as to switch the states of FIGS. It may be a drive mechanism. Alternatively, the states of FIGS. 3B and 3C may be exchanged by the operator operating the operating section for moving the sample 50 or the operating section for moving the photomask.

In the case of this embodiment, the illumination 13 is a high-frequency illumination formed of a photomask in which bright portions that transmit light and dark portions that block light are combined at an arbitrary ratio. In addition, the light intensity acquisition means arbitrarily moves the sample 50 in the X direction or the Y direction while maintaining the focus of the optical sensor 15 and the illumination 13, or arbitrarily moves the photomask (the light part and the dark part are changed in the X direction or the Y direction). It is a driving mechanism for arbitrary movement in the Y direction.

The transmission microscope of this embodiment utilizes the bright and dark portions of the high-frequency illumination, and further utilizes the difference in optical information between the focused and unfocused states of the optical sensor 15 and the high-frequency illumination 13 to directly Separate the component and the global component. Here, a plane (XY plane) in the sample parallel to the plane on which an image of high-frequency illumination is formed is called a "slice" (see FIG. 8). For example, assume that the thickness of the sample 50 is 10 μm and the thickness of the in-focus slice is 3 μm (the numerical value varies depending on the sample and lens performance). FIG. 4 is a diagram illustrating in-focus and out-of-focus slices. The slices where neither the photosensor nor the RF illumination are in focus are α and γ, and the slice in focus is β. If the photosensor is in focus, the image looks like FIG. 4(a), and if the photosensor is out of focus, the image looks like FIG. 4(b). On the other hand, if the high frequency illumination is in focus, the image will look like FIG. 4(c), and if the high frequency illumination is out of focus, the image will look like FIG. 4(d). That is, if either the optical sensor or the high-frequency illumination is out of focus, an image with a blurred pattern (FIGS. 4(b) and 4(d)) is obtained. It is assumed that the light intensity is uniform in such a pattern-blurred slice. In other words, it is assumed that the direct component is included when both the optical sensor side and the high-frequency illumination side are in focus, and only the global component is included when either the optical sensor side or the high-frequency illumination side is out of focus. I reckon.

First, consider the case where there is no high frequency illumination pattern and both the camera and the illumination are in focus, and the case where either the camera or the illumination is out of focus and the light intensity is uniform. The light intensity L[p] captured at pixel p of photosensor 15 corresponding to observation point K is the sum of the direct component D[p] and the global component G[p].
(Formula 1)
L[p]=D[p]+G[p]

FIGS. 3(b) and 3(c) show the state when the sample 50 is irradiated using the high-frequency illumination 13 through a photomask having a checkerboard pattern of light-transmitting bright portions and light-blocking dark portions. It is a figure shown typically. For clarity of explanation, the illustration of the transmission type sample observation apparatus 100 is omitted in FIGS. 3(b) and 3(c).

[Description of Separation Operation]
Here, the content of calculation performed by the calculator 17 will be described.
The target slice containing the observation point K is focused by the optical sensor 15 and the high frequency illumination 13 . FIG. 3B is a schematic diagram when the photomask or the sample 50 is moved by the light intensity acquisition means 19 to irradiate the observation point X with light (position the bright portion of the photomask). When the observation point X is irradiated with light, the optical sensor 15 detects the direct light from the observation point X (direct component (A)) and the light from other parts (out-of-focus observation points other than the other points). Information including both a global component (B) that is a component of single scattered light and a global component (C) that is a component of multiple scattered light is acquired. On the other hand, FIG. 3C is a schematic diagram when the photomask or the sample 50 is moved so that the observation point X is not irradiated with light (the dark portion of the photomask is positioned). If the observation point X is not irradiated with light, there is no direct light from the observation point X (direct component (A)), and the optical sensor 15 detects only light from other parts (global components (B) and (C)). Get information about
In addition, "component (D) due to multipoint single scattered light from other than the observation point in focus" is imaged on different pixels in the optical system of the camera, so component (D) is considered in the pixel under consideration. No need.

Assuming that the illumination light is uniform in the out-of-focus slice (Fig. 4(d)) as described above, the out-of-focus The light intensity within the slice (α, γ) is half the light intensity of the bright portion of the focused slice β. That is, the light intensity of the multiple scattered light and single scattered light from portions other than the slice β is half the intensity of the light reflected or transmitted at the observation point K. FIG.

ここで、Ｓ［ｐ、ｘ］は点ｘ＝［ｘ、ｙ、ｚ］^ｔにおける単一散乱光の光強度である。Ｍ［ｐ、ｘ］は最後の散乱点がxである多重散乱光の光強度である。Using this, the luminance L + [p] at the pixel p of the photosensor corresponding to the observation point K when the observation point K is irradiated with light, and the luminance L ₊ [p] when the observation point K is not irradiated with light can be _calculated as follows.

where S[p,x] is the light intensity of the single scattered light at the point x=[x,y,z] ^t . M[p, x] is the light intensity of the multiply scattered light whose last scattering point is x.

A pixel p corresponds to a set of points x since the depth of focus d has a range. Let Ω(p,d) be the area corresponding to pixel p and depth of focus d, and let Ψ(p) denote all points x in sample 50 corresponding to pixel p.

As shown in FIG. 3(b), the first term in equation (2) is the direct transmitted light that illuminated observation point K in the target slice (β; z=d). The second term is the single scattered light at different out-of-focus slices (α, γ) and the third term is the multiple scattered light in the global component. Therefore, the light D[p] from the target slice and the light G[p] from the other slices are expressed as follows.

To obtain the direct and global components at all pixels of the photosensor 15, the high frequency illumination pattern is moved to obtain the maximum L _max and minimum L _min light intensity at each pixel. Since L ₊ =L _max and L ₋ =L _min , D[p] and G[p] of each pixel can be obtained using equation (4) to separate the transmitted light from the sample 50 and the scattered light. be able to.
As for the method of moving the high-frequency illumination pattern, for example, there is a method of moving along a moving path that is set in advance so as to include both bright and dark areas of the high-frequency illumination. As a method of moving the high-frequency pattern filter, a method of randomly moving the high-frequency pattern filter in the XY directions on the XYZ-axis electric stage may be used. For this method, it is necessary to acquire a sufficient amount of observation images of the target slice to capture all points in the bright area.
The calculation of the calculator 17 has been described above.

Note that it is necessary to pay attention to the sizes of the bright and dark portions of the photomask that forms the high-frequency illumination. If the bright area of the photomask is large, the directly transmitted light scattered at the observation point will be scattered several times within the same illumination area. In the transmission microscope of this embodiment, such light that has been scattered several times within the same illumination area is also determined to be a direct component. Since the number of scattering times within the same illumination area depends on the characteristics of the sample and the frequency of the high-frequency illumination, it is necessary to appropriately adjust the frequency of the high-frequency illumination according to the characteristics of the sample.

式（２ａ）、式（３ａ）、式（４ａ）において、ａ＝１／２、ｂ＝０の場合が式（２）、式（３）、式（４）となる。For example, in the above embodiment, the pattern of the high-frequency illumination was described as a checkered pattern in which the bright and dark portions are 1:1. Alternatively, irregular pattern illumination in which bright portions and dark portions are not regularly arranged may be used. In this case, the coefficients of the second and third terms of Equation (2) and the coefficients of each term of Equation (3) are changed according to the ratio of the bright portion and the dark portion. Specifically, when the pattern of high-frequency illumination is a checkerboard pattern in which the ratio of bright and dark parts is 1:1, each coefficient of formulas (2) and (3) is "1/2". , the coefficient is changed according to the ratio of the bright portion and the dark portion (the sum of the coefficients is 1). for example,
Ratio of light area = a,
dark area ratio = 1 - a,
b: the transmittance of the dark area when the transmittance of the bright area is 1 (0.0≦b<1)
Then, the above equations (2), (3), and (4) become equations (2a), (3a), and (4a), respectively.

In formulas (2a), (3a), and (4a), formulas (2), (3), and (4) are obtained when a=1/2 and b=0.

In the above embodiment, the high-frequency illumination is configured using a photomask, but the high-frequency illumination may be configured by arranging a plurality of minute light sources such as LEDs without using a photomask. That is, the illumination has a pattern in which a bright portion that causes the light source to emit light and a dark portion that extinguishes the light source are combined at an arbitrary ratio, and the light intensity acquisition means maintains the ratio while maintaining the ratio between the bright portion and the dark portion. It is characterized by being a light source control unit that arbitrarily moves a dark portion.
Of course, LEDs can provide irregular pattern illumination as well as high frequency illumination.

(Example 1)
FIG. 5(a) is a photograph for explaining a transmission microscope system using high-frequency illumination of this embodiment. This transmission microscope system consists of a microscope, a light source, a high-frequency pattern filter (photomask), a camera (optical sensor), and an XYZ axis motorized stage. An Olympus BX53 microscope was used, a halogen lamp was used as the light source, and a Suruga Seiki KXT04015-LC was used as the XYZ axis motorized stage. As high-frequency pattern filters, photomasks with various checkered patterns (high-frequency illumination) having a ratio of 1:1 between the illuminated area (bright area) and the non-illuminated area (dark area) were prepared as shown in FIGS. . Specifically, pattern sizes ranging from 1×1 μm to 16×16 μm were prepared. The size of the pattern is determined according to the amount of scatterers contained in the sample, such as the site in the living body. The illumination lens is placed on top of the filter and can change the focus position of the illumination.

First, with respect to the depth direction (Z direction) of the sample, the optical sensor 15 and the high frequency illumination 13 are focused on a target slice at a certain depth. Either can be focused first, but both must be focused. For example, the specimen stage of the microscope is moved to focus the optical sensor 15 on the desired target slice so that the specimen 50 can be viewed. The high frequency illumination 13 is then focused on the target slice. For the focusing, the illumination-side lens 14 is moved in the Z direction, and the high-frequency illumination pattern filter 13 is moved in the Z direction by the XYZ-axis motorized stage for adjustment.

Then, in order to capture all points in the bright area, the high-frequency pattern filter 13 is moved on the XYZ-axis motorized stage to obtain an observation image of the target slice. The method of moving the high frequency pattern filter is as described above. FIG. 5(b) is an image when a high-frequency pattern is irradiated on human skin, FIG. 5(c) is an image of the direct component D[p], and FIG. 5(d) is an image of the global component G[p]. be. Then, the observation image is processed as in the above-mentioned "Description of Separation Operation" to obtain the direct component D[p] and the global component G[p] at each optical sensor (pixel) of the camera. So far, we can separate the transmitted and scattered light for the current target slice.

Next, by moving the high-frequency pattern filter 13 in the XYZ motorized stage in the Z-axis direction, the optical sensor 15 and the high-frequency illumination 13 are focused on a target slice at a different depth. Then, in the target slice, the high-frequency pattern filter 13 is similarly moved by the XYZ-axis motorized stage to obtain an observation image. In the same manner as described above, the observed image can be processed as described in "Description of Separation Operation" to separate transmitted light and scattered light for the target slice.

(Effect of this embodiment)
In the methods described in Non-Patent Documents 1 to 8, all depth information, including overlap on the optical path, is observed by superimposition, and information on transmitted light and scattered light at a specific depth (target slice) from them Alternatively, it is difficult to obtain information on transmitted light and scattered light in the depth direction (Z direction). On the other hand, this transmission microscope system can acquire information on transmitted light and scattered light at a specific depth (target slice). Information can be obtained by separating transmitted and scattered light for target slices at multiple depths. This transmission microscope system continuously moves in the Z direction with a certain step width, and can continuously acquire information on the transmitted light and the scattered light in the target slice for each depth. 3D information or 3D image information can be observed. The three-dimensional information includes three-dimensional wavelength information of direct components and global components, various kinds of information extracted by calculations therefrom or images created therefrom, and results analyzed therefrom.

(Example 2)
Before describing this embodiment, problems in the field of life science research to which this embodiment is to be applied will be described.
In life science research, structural and functional analysis by non-contact or non-invasive imaging of living tissue is highly expected. Recently, as typified by confocal microscopes and light sheet microscopes, morphological visualization using fluorescence wavelength information as a clue has been widely used (Nat. Methods 8, pp.757-760, 2011). However, observation by fluorescent dye staining with such a confocal microscope or the like poses a problem that viable cells undergo discoloration or damage due to excitation light. For this reason, there is a strong demand for microscopic observation of living tissue without staining. However, since living tissue generally contains many scattering bodies, the information obtained by superimposing transmitted light and scattered light is observed, and the transmitted light, which is to be accurately observed, becomes unclear due to the influence of scattering. Therefore, in biomedical imaging such as cancer detection, correct measurement of composition information such as absorption spectrum is not easy due to the influence of scattered light when observing living tissue with a microscope. In the prior art, since it is difficult to separate the absorption and scattering spectra from the microscopic observation image, a method has been adopted in which the absorption and scattering are collectively used as attenuation terms. Therefore, in the prior art, instead of reducing the damage to living cells due to staining, the transmitted light and scattered light are separated from the microscopic observation image in the absence of staining wavelength information, and the absorption and scattering in each biological tissue are separated. Two-dimensional distribution information is extracted.

On the other hand, the transmission microscope system according to the present invention can separate the transmitted light and the scattered light from the microscope observation image as described above. Therefore, the absorption coefficient and the scattering coefficient of the sample can be obtained by performing calculations described later. Therefore, the transmission microscope system according to the present invention can eliminate the need for dyeing the sample, and can acquire information on the absorption and scattering of the sample in the presence of wavelength information. Moreover, since the transmission microscope system according to the present invention can also acquire information in the Z direction of the sample as described above, it is possible to three-dimensionally acquire the absorption coefficient and scattering coefficient of the sample.

The transmission microscope system of this embodiment will be described below.
In this embodiment, a conventional method and a method using the transmission microscope system of this embodiment will be described for spatial multispectral absorption analysis. FIG. 6(a) is a diagram illustrating analysis by a conventional method for obtaining comparison data. As the known sample 23 , a mixture of a red dye (1 mg/mL) and an intralipid diluent (0.4%) was sealed in an ampoule and placed at the center of the integrating sphere 21 . Then, the sample 23 was irradiated with the light 24 while changing the wavelength, the scattered light emitted from the sample to the surroundings was received by the integrating sphere 21, and the absorption coefficient of the sample 23 at each wavelength was analyzed by the spectrophotometer 22. . Note that the light 24 may be white light (multi-wavelength light), and the spectrophotometer 22 may perform collective analysis for each wavelength.
As the true value of the known sample 23, the absorbance was measured with a solution containing only a red dye (1 mg/mL) containing no intralipid.
Here, the analysis of the absorption coefficient using the integrating sphere as shown in FIG.

On the other hand, FIG. 6B is a diagram for explaining a method of analyzing the absorption coefficient of the sample 50 using the transmission microscope system of this embodiment. While changing the wavelength of the light source 13, the high-frequency pattern filter was moved on the XYZ-axis motorized stage as described above so that the regions of the sample 50 to be analyzed were arranged with bright portions and dark portions. Then, the calculator 17 obtains the direct component D[p] at each wavelength, and obtains the absorption coefficient of the region from the light intensity of the light source 13 . Note that the light from the light source 13 may be white light (multi-wavelength light), and the computing unit 17 may perform collective analysis for each wavelength.
The following is an example of calculation of the absorption coefficient α.
Assuming that the direct component is I _d , the global component is I _s , the intensity of the incident light is I _i , and the lesser of the depth of focus of the photosensor side lens and the depth of focus of the illumination side lens is X _f , the transmitted light intensity I _o is (Formula 1b)
I _o =I _i exp(−αX _f )
is.
again,
(Formula 2b)
I _o =I _d +I _s
Therefore, the absorption coefficient α at the observation point is
(Formula 3b)
α=max(0,−log(I _o /I _i )/X _f )
is.

FIG. 7 is a diagram comparing the absorption coefficient (dashed line) obtained by the conventional method in FIG. 6(a) and the absorption coefficient (solid line) obtained by the transmission microscope system of the present embodiment in FIG. 6(b) for each wavelength. is. As shown in FIG. 7, the absorption coefficient obtained by the transmission microscope system of this embodiment has a spectrum shape similar to that obtained by the conventional method in FIG. It can be seen that it can be replaced with the transmission microscope system of
Furthermore, the transmission microscope system of this embodiment can analyze the absorption coefficient with a smaller amount of sample than the conventional method using an integrating sphere.

Further, the transmission microscope system of this embodiment can obtain the scattering coefficient as well as the absorption coefficient. The following is an example of calculating the scattering coefficient β.
The direct component I _d using the sample thickness X _s (equation 4b)
I _d = I _o −I _s = I _i exp {−(α+β)*X _s }
It can be expressed as. Therefore, the scattering coefficient β at the observation point is (equation 5b)
β = max(0, (−log(I _d /I _i )/X _s −α)
When using the global component I _s instead of the direct component I _d ,
β = max(0, {−log(I _o −I _s )/I _i }/X _s −α)
is.

(Effect of this embodiment)
The transmission microscope system of this embodiment can not only avoid damage to living cells due to staining, but also greatly shorten the analysis time by reducing the staining process. As a result, we will contribute to the discovery of new functions of living cells, such as interactions within and between living tissues, three-dimensional dynamics of living tissues, and biochemical reactions, as well as to the elucidation of as yet unknown in vivo mechanisms. There is expected. In addition, providing new analytical equipment to the biochemical field is expected to lead to the creation of new analysis methods and the development of new research fields, and is expected to contribute to the advancement of medical technology.

The sample observation method, sample observation device, and microscope according to the present invention can be applied not only to the biochemistry field but also to the separation of speckles caused by absorption and scattering in the holography field. The present invention can also be applied to measurement and analysis techniques in various fields such as inspection of foods and goods.

Although a transmission type microscope has been described as an example in the above-described embodiments and examples of the present invention, the present invention is not limited to this. Even with a microscope, when a translucent sample is observed, the present invention can be similarly applied as a technique for removing the multi-point single-scattered light and multi-scattered light from the reflected light.

12: Optical sensor side lens 13: Light source, high frequency illumination 14: Illumination side lens 15: Optical sensor 17: Calculator 18: Focusing mechanism 19: Light intensity acquisition means 21: Integrating sphere 22: Spectrophotometer 23: Sample 24: Light 50: Sample 100: Transmission type sample observation device

Claims (12)

A sample observation method performed by a microscope equipped with illumination having a bright part and a dark part, an optical sensor, and a focusing mechanism,
a focusing procedure in which the focusing mechanism focuses the optical sensor and the illumination in the Z direction on an XY plane including an observation point in the sample;
After performing the focusing procedure, the first light intensity when the light of the illumination is the bright part irradiating the observation point and the second light intensity when the dark part is the light irradiating the observation point other than the observation point a light intensity acquisition procedure acquired by the optical sensor;
A computing unit connected to the microscope multiplies the difference between the first light intensity and the second light intensity by a coefficient corresponding to the transmittance ratio between the bright portion and the dark portion to obtain a single The difference between the first light intensity and the second light intensity considering the transmittance of the bright portion and the dark portion as the direct component (A) due to the scattered light, and the transmittance ratio and area of the bright portion and the dark portion A calculation procedure for calculating a global component of single scattered light (B) at an out-of-focus point other than the observation point and multiple scattered light (C) within the sample by multiplying a coefficient according to the ratio;
sample observation method. 2. The method according to claim 1, further comprising: moving the XY plane including the observation point in the sample in the Z direction to repeat the focusing procedure, the light intensity obtaining procedure, and the computing procedure. Sample observation method. 3. The illumination according to claim 1, wherein the illumination is a high-frequency illumination formed of a photomask in which the bright portions that transmit light and the dark portions that block light are combined in an arbitrary area ratio. Sample observation method. 3. A sample observation method according to claim 1, wherein said illumination is a pattern of illumination in which said bright portion for causing a light source to emit light and said dark portion for extinguishing said light source are combined in an arbitrary area ratio. 5. A sample observation method according to claim 1, wherein the direct component (A) calculated in said calculation procedure is used as the image of said observation point. 5. The sample observation method according to claim 1, wherein the absorption coefficient at the observation point is obtained from the direct component (A) calculated in the calculation procedure and the light intensity of each wavelength of the illumination. . A sample observation apparatus comprising: a microscope comprising illumination having a bright portion and a dark portion, an optical sensor, and a focusing mechanism; a calculator connected to the microscope; and a light intensity acquisition means connected to the microscope,
The focusing mechanism focuses the optical sensor and the illumination in the Z direction on an XY plane containing an observation point in the sample,
The light intensity acquisition means is configured to obtain a first light intensity when the light of the illumination irradiates the observation point in the bright part in a state in which the focus of both the optical sensor and the illumination is adjusted to the observation point by the focusing mechanism. Acquiring the light intensity and the second light intensity at the time of the dark area irradiated to a point other than the observation point with the optical sensor;
The arithmetic unit multiplies the difference between the first light intensity and the second light intensity by a coefficient corresponding to the transmittance ratio between the bright portion and the dark portion to obtain a direct component due to single scattered light at the observation point. (A), and a coefficient corresponding to the transmittance ratio and area ratio of the bright portion and the dark portion to the difference between the first light intensity and the second light intensity considering the transmittance of the bright portion and the dark portion multiplied by to obtain a global component of single scattered light (B) at an out-of-focus point other than the observation point and multiple scattered light (C) within the sample. An XY plane including the observation point is moved in the Z direction within the sample, and the focusing mechanism causes the optical sensor and the illumination to focus on the observation point for each position in the Z direction of the XY plane including the observation point. 8. The sample observation according to claim 7, further comprising a controller for causing said light intensity obtaining means to obtain said first light intensity and said second light intensity, and for causing said calculator to perform said calculation. Device. The illumination is high-frequency illumination formed of a photomask in which the bright portion that transmits light and the dark portion that blocks light are combined at an arbitrary area ratio,
9. The light intensity acquisition means is a driving mechanism for arbitrarily moving the sample or arbitrarily moving the photomask while maintaining the focus of the light sensor and the illumination. The sample observation device according to . The illumination is illumination of a pattern in which the bright portion that emits light from the light source and the dark portion that extinguishes the light source are combined in an arbitrary area ratio,
9. A specimen observation apparatus according to claim 7, wherein said light intensity acquisition means is a light source control section that arbitrarily moves said bright portion and said dark portion while maintaining said ratio. 11. The sample observing apparatus according to claim 7, wherein said calculator uses said direct component (A) as an image of said observation point. 11. The computing unit according to any one of claims 7 to 10, wherein the computing unit performs computation for acquiring the absorption coefficient of the observation point from the direct component (A) and the light intensity of each wavelength of the illumination. Specimen observation device.

Images