JP5489541B2 - Image analysis method and image analysis apparatus - Google Patents

Description

  The present invention relates to an image analysis method and an image analysis apparatus.

  Conventionally, methods as shown in Non-Patent Documents 1 and 2 have been proposed as Raster Image Correlation Spectroscopy (RICS). In this image analysis method, a fluorescence image composed of a raster scan image of one frame or more is acquired. In other words, an area of interest is determined in the sample to be image-analyzed, and the area is repeatedly scanned by the raster scanning method to obtain an image composed of a plurality of frames of fluorescence intensity. The fluorescence intensity in the frame is expressed as data in pixel units.

  Here, large molecules and small molecules are mixed in the sample subjected to image analysis. In such a case, if it is desired to perform image analysis on a small molecule in the sample, it is necessary to remove the influence of the large molecule.

`` Measuring Fast Dynamics in Solutions and Cells with a Laser Scanning Microscope '', Michelle A. Digman, Claire M. Brown, Parijat Sengupta, Paul W. Wiseman, Alan R. Horwitz, and Enrico Gratton, Biophysical Journal, Vol.89, P1317 -1327, August 2005. `` Fluctuation Correlation Spectroscopy with a Laser-Scanning Microscope: Exploiting the Hidden Time Structure '', Michelle A. Digman, Parijat Sengupta, Paul W. Wiseman, Claire M. Brown, Alan R. Horwitz, and Enrico Gratton, Biophysical Journal: Biophysical Letters , L33-36, 2005.

  As shown in Non-Patent Documents 1 and 2, there is an average method as a technique for removing the influence of large molecules, immobile molecules and the like. In the average method, an average image of the acquired frame data is calculated and subtracted from the original frame data. The average image is calculated from a continuous part or all of the acquired frames regardless of whether the frame data includes the influence of large molecules or immobile molecules. Therefore, the influence of large molecules and immobile molecules is removed by subtraction processing of the same average image regardless of the change of the frame data.

  Here, when there are common large molecules or immobile molecules in all the acquired images, the average image of the acquired frame data may be subtracted from all the images.

  However, among all the acquired images, there are cases where a large molecule or an immobile molecule that is common to other images does not exist in some images. In this case, since there are no large molecules or immobile molecules in the partial image, it is not appropriate to subtract the average image of the acquired frame data from the partial image. For this reason, according to the methods shown in Non-Patent Documents 1 and 2, the removal of the appropriate material in place (subtraction of the average image) is not performed for the influence of large molecules or immobile molecules.

  An object of the present invention is to provide an image analysis method of RICS that removes the influence of large molecules and immobile molecules in the right place.

The present invention is an image analysis method of scanning optical microscope to analyze the scanned image obtained by the detection and scanning of the measuring points for detecting the light from the measurement point in the sample having different molecular sizes are mixed, each scan An image acquisition step for acquiring a plurality of frames of scan images including a plurality of pixels from which image pixel data has been acquired in a time series, and an analysis region setting step for setting an analysis region for the scan image A segment value calculating step for calculating a segment value of the analysis region, a classification step for classifying the plurality of scanned images into two or more groups based on the magnitude of the variation of the segment value , and the group for each group the average image calculation step of calculating an average image of the analysis area, a new image of the analysis region by subtracting the average image from the scanned image of the analysis region in each group And a calculation step of calculating a correlation value and a new image calculation step of calculating, by raster image correlation spectroscopy based on the new image for each of the groups.

  According to the present invention, there is provided a RICS image analysis method that removes the influence of large molecules and immobile molecules in the right place.

1 schematically shows an image analysis device according to an embodiment of the invention. The functional block of the control part shown by FIG. 1 is shown. 3 is a flowchart of image analysis according to an embodiment of the present invention. The fluorescence image of the several flame | frame acquired in time series is shown. The observation area and the analysis area are shown. The fluorescence image of the some flame | frame classified into the some group is shown typically. It is the image which showed the calculation result of the spatial correlation value by RICS with respect to a small molecule | numerator by the brightness | luminance. The fitting result of the spatial correlation value by RICS with respect to a small molecule | numerator is shown. It is the image which showed the calculation result of the spatial correlation value by RICS with respect to a big molecule | numerator by the brightness | luminance. The fitting result of the spatial correlation value by RICS for a large molecule is shown.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 schematically shows an image analysis apparatus according to an embodiment of the present invention. This image analysis apparatus is configured based on a scanning confocal optical microscope for fluorescence observation of a sample.

  As shown in FIG. 1, the image analysis apparatus 100 is necessary for image analysis, a light irradiation unit 110 that irradiates the sample S with excitation light, a light detection unit 130 that detects light emitted from a measurement point in the sample S, and the like. And a sample stage 190 that supports the sample S.

  The sample S is accommodated in a sample container such as a microplate or a slide glass, and is placed on the sample stage 190. For example, the sample stage 190 supports the sample S so as to be movable in the lateral direction (xy direction) and the height direction (z direction) with respect to the light irradiation unit 110 and the light detection unit 130. For example, the sample stage 190 includes three stepping motors whose output axes are orthogonal to each other, and the sample S can be moved in the xyz direction by these stepping motors.

  The image analysis apparatus 100 is a multiple light irradiation / multiple light detection type. For this reason, the light irradiation unit 110 includes an n-channel light source system 111, and the light detection unit 130 correspondingly includes an n-channel detection system 131. Each of the n-channel detection systems 131 detects fluorescence generated by the excitation light emitted from the n-channel light source system 111. Here, the n channel includes channel 1, channel 2,... Channel n. The channel differs depending on the type of excitation light.

  The n-channel light source system 111 of the light irradiation unit 110 includes light sources 112a,..., 112n, collimating lenses 114a,..., 114n, and dichroic mirrors 116a,. The light sources 112a,..., 112n emit excitation light for exciting the fluorescent dye contained in the sample S to emit light (fluorescence) from the sample S. The wavelengths of the excitation light emitted from the light sources 112a,..., 112n are different from each other corresponding to the type of fluorescent dye contained in the sample S. The light sources 112a,..., 112n are constituted by, for example, laser light sources having an oscillation wavelength suitable for the fluorescent dye in the sample S. The collimating lenses 114a,..., 114n collimate the excitation light emitted from the light sources 112a,. The dichroic mirrors 116a,..., 116n reflect the excitation light that has passed through the collimating lenses 114a,. Each of the dichroic mirrors 116a,..., 116n transmits the excitation light incident from above in FIG. 1 and reflects the excitation light incident from the right side in FIG. As a result, the excitation lights having different wavelengths respectively emitted from the light sources 112a,..., 112n are combined into one beam after passing through the dichroic mirror 116a. The dichroic mirror 116n does not need to transmit the excitation light, and may be changed to a simple mirror.

  The light irradiation unit 110 further includes a dichroic mirror 122, a galvano mirror 124, an objective lens 126, and an objective lens driving mechanism 128. The dichroic mirror 122 reflects the excitation light from the light source system 111 toward the galvanometer mirror 124 and transmits the fluorescence emitted from the sample S. The galvanometer mirror 124 reflects the excitation light toward the objective lens 126 and changes the reflection direction thereof. The objective lens 126 converges the excitation light and irradiates the measurement point in the sample S, and takes in the light from the measurement point in the sample S. An objective lens 126 having a large NA (numerical aperture) is used to form a minute confocal region (measurement point). The confocal region thus obtained has a substantially cylindrical shape with a diameter of about 0.6 μm and a length of about 2 μm. The galvanometer mirror 124 constitutes xy scanning means for scanning the measurement point in the xy direction. The xy scanning unit may be configured using an acousto-optic modulation element (AOM), a polygon mirror, a hologram scanner, or the like in addition to using a galvano mirror. The objective lens driving mechanism 128 moves the objective lens 126 along the optical axis. Thereby, the measurement point is moved in the z direction. That is, the objective lens driving mechanism 128 constitutes z scanning means for scanning the measurement point in the z direction.

  The light detection unit 130 shares the objective lens 126, the galvano mirror 124, and the dichroic mirror 122 with the light irradiation unit 110. The light detection unit 130 further includes a converging lens 132, a pinhole 134, and a collimating lens 136. The converging lens 132 converges the light transmitted through the dichroic mirror 122. The pinhole 134 is disposed at the focal point of the converging lens 132. That is, the pinhole 134 is in a conjugate position with respect to the measurement point in the sample S, and selectively allows only light from the measurement point to pass through. The collimating lens 136 collimates the light that has passed through the pinhole 134. The light that has passed through the collimator lens 136 enters the n-channel detection system 131.

  The n-channel detection system 131 includes dichroic mirrors 138a, ..., 138n, fluorescent filters 140a, ..., 140n, and photodetectors 142a, ..., 142n. The dichroic mirrors 138a,..., 138n selectively reflect light having a wavelength in the vicinity of the fluorescent wavelength region generated from the sample S by the excitation light from the light sources 112a,. The dichroic mirror 138n does not need to transmit light, and may be changed to a simple mirror. The fluorescence filters 140a,..., 140n block the light having an undesired wavelength component from the light reflected by the dichroic mirrors 138a,..., 138n, respectively, and are generated by the excitation light from the light sources 112a,. Only the fluorescence is selectively transmitted. The fluorescence transmitted through the fluorescent filters 140a,..., 140n is incident on the photodetectors 142a,. The photodetectors 142a,..., 142n output signals corresponding to the intensity of incident light. That is, the photodetectors 142a,..., 142n output fluorescence intensity signals from the measurement points in the sample S.

  The control unit 160 is configured by a personal computer, for example. The control unit 160 acquires, stores, and displays the fluorescence image of the observation region of the sample S, waits for input of the number of frames (number of frames) of the acquired fluorescence image and the setting of the analysis region, image analysis (calculation of correlation value), diffusion time Etc. are estimated. The control unit 160 controls the galvanometer mirror 124 that is an xy scanning unit, the objective lens driving mechanism 128 that is a z scanning unit, the sample stage 190, and the like.

  FIG. 2 shows functional blocks of the control unit shown in FIG. As shown in FIG. 2, the control unit 160 includes a scanning control unit 162, an image forming unit 164, a storage unit 166, a display unit 168, an input unit 170, an analysis region setting unit 172, a data extraction unit 176, an analysis unit 178, and a stage. And a control unit 180. The scanning control unit 162, the image forming unit 164, the storage unit 166, the stage control unit 180, the galvano mirror 124, the objective lens driving mechanism 128, the stage 190, and the photodetector 142 constitute an image acquisition unit.

  When acquiring the fluorescence image of the sample S, the scanning control unit 162 controls the galvanometer mirror 124 so as to raster scan the irradiation position of the excitation light with respect to the sample S. The scanning control unit 162 also controls the objective lens driving mechanism 128 so that the irradiation position of the excitation light is z-scanned with respect to the sample S if necessary. The image forming unit 164 forms a fluorescent image of the sample S from the information on the irradiation position of the excitation light input from the scanning control unit 162 and the output signals of the photodetectors 142a,. Thereby, a fluorescence image can be acquired. The storage unit 166 sequentially stores the fluorescent images formed by the image forming unit 164. The display unit 168 displays the fluorescence image of the sample S and the analysis result. The input unit 170 includes, for example, a mouse and a keyboard, and constitutes a GUI in cooperation with the display unit 168. This GUI is used for setting the number of acquired frames, the observation area, and the analysis area. The stage control unit 180 controls the sample stage 190 according to input information from the input unit 170, for example, in order to set an observation region. The analysis area setting unit 172 sets an analysis area according to the input information from the input unit 170. The data extraction unit 176 extracts necessary data from the fluorescence image stored in the storage unit 166 based on the input information from the analysis region setting unit 172. The necessary data may be, for example, data of all pixels or data of some pixels of all fluorescent images stored in the storage unit 166, or some data stored in the storage unit 166. Data of all pixels or a part of pixels of the fluorescence image may be used. The analysis unit 178 calculates a correlation value described later on the data extracted by the data extraction unit 176. More specifically, the analysis unit 178 calculates a segment value of an analysis region, which will be described later, and classifies the plurality of fluorescent images into one or more groups based on the segment value. Then, the analysis unit 178 calculates an average image of the analysis region for each group and then subtracts the average image from each image of the analysis region for each group to calculate a new image of the analysis region. Thereafter, the analysis unit 178 calculates a correlation value based on the new image for each group.

  1, excitation light emitted from light sources 112a,..., 112n passes through collimating lenses 114a,..., 114n, dichroic mirrors 116a, ..., 116n, dichroic mirror 122, galvano mirror 124, and objective lens 126. The measurement point is irradiated. The measurement point irradiated with the excitation light is raster scanned in the xy direction by the galvanometer mirror 124. Further, if necessary, z scanning is performed by the objective lens driving mechanism 128. The sample S that has received the excitation light emits fluorescence from the measurement point. Light from the sample S (including undesired reflected light in addition to fluorescence) reaches the pinhole 134 through the objective lens 126, the galvanometer mirror 124, the dichroic mirror 122, and the converging lens 132. Since the pinhole 134 is at a position conjugate with the measurement point, only light from the measurement point in the sample S passes through the pinhole 134. The light that has passed through the pinhole 134, that is, the light from the measurement point in the sample S, enters the n-channel detection system 131 through the collimator lens 136. The light incident on the n-channel detection system 131 is separated (that is, spectrally separated) by the dichroic mirrors 138a,..., 138n, and undesired components are removed by the fluorescent filters 140a,. As a result, only the fluorescence generated by the excitation light from the light sources 112a,..., 112n enters the photodetectors 142a,. Each of the photodetectors 142a,..., 142n outputs incident light, that is, a fluorescence intensity signal indicating the intensity of fluorescence emitted from a measurement point in the sample S. This fluorescence intensity signal is input to the image forming unit 164. The image forming unit 164 processes the input fluorescence intensity signal in synchronization with the positional information in the xy direction (and z direction) for each raster scan (and z scan), and the focal plane in the sample S A fluorescent image of one frame (a flat surface or a curved surface where the measurement point has moved) is formed. The formed fluorescent image is stored in the storage unit 166. The series of operations described here is repeated for the set number of frames to be acquired, and a fluorescence image having the set number of frames is acquired. Each fluorescent image is composed of a plurality of pixels from which pixel data is acquired in time series.

  The fluorescence image stored in the storage unit 166 is processed as necessary and displayed on the display unit 168. For example, it is also possible to obtain a fluorescence image of a plurality of frames by changing the z position of the measurement point, combine them to form a three-dimensional image, and display it on the display unit 168.

  The image analysis procedure will be described below with reference to FIG. Each step will be described with reference to FIGS. 4 to 6 as appropriate.

<Step S1>:
The observation area of the sample S and the number of frames of the fluorescent image to be acquired are set. The fluorescent image of the set observation area is acquired for the set number of frames. The fluorescence image is acquired by the same scanning method for the same observation region. The acquired fluorescence images of a plurality of frames are images acquired in time series, and each image is composed of a plurality of pixels from which pixel data is acquired in time series.

FIG. 4 schematically shows a plurality of frames of fluorescence images acquired in time series. In FIG. 4, F _n represents the fluorescence image of the n th frame in one channel. That is, FIG. 4 shows an example in which an 80-frame fluorescence image is acquired.

<Step S2>:
As shown in FIG. 5, a region (analysis region) R2 to be analyzed is set with respect to a region (observation region) R1 of the acquired fluorescence image. The analysis region R2 is not limited to a part of the observation region R1, and may coincide with the observation region R1. The analysis area R2 is set as an observation area R1, that is, a scanning area in step S1 by default in terms of application. This step is not necessary when analyzing the entire observation region R1.

<Step S3>:
The segment value of the analysis region R2 is calculated. The division value is, for example, a statistical value of pixel data in the analysis region R2, and is any one of a maximum value, a minimum value, an average value, a relative difference, and a relative ratio of pixel data in the analysis region R2. Pixel data is, for example, fluorescence intensity obtained from a two-dimensional or three-dimensional observation region. The relative difference is, for example, a difference between a maximum value and a minimum value.

In the following description, the fluorescence image _F 1, _..., segment values of the analysis area of the _{F 20} is _{D a,} the fluorescence image _F 21, _..., segment values of the analysis area of the _{F 50} is significantly different from the _{D a D b} and the fluorescence image _F 51, _..., segment values of the analysis area of the _{F 80} is assumed to be _{D a.}

<Step S4>:
A plurality of images are classified into one or more groups based on the segment values. For example, the images are grouped before and after the segment value changes greatly. The number of groups is determined according to how the segment value changes.

FIG. 6 schematically shows fluorescence images of analysis regions of a plurality of frames classified into a plurality of groups. In FIG. 6, Fa _n represents the image of the analysis area of the n-th frame of the fluorescent image. Figure 6 is a fluorescence image _F 1 shown in FIG. 4, _..., the image _Fa 1 analysis region of _{F 80,} ..., _{Fa 80} the two groups _G a, shows an example of classifying into _{G b.} In Figure 6, the image _Fa 1, ..., _{Fa 20} and the image _Fa 51, ..., _{Fa 80} are classified into groups _{G a,} image _Fa 21, ..., _{Fa 50} are classified into groups _{G b.}

<Step S5>:
An average image of the analysis area is calculated for each group. The average image may be an average value of all images included in each group, or may be an average value of some images included in each group. For example, the average image is calculated by calculating the number of images included in each group and calculating the average image of each group based on the calculated number of images.

  More specifically, for all the images included in each group, an average value of pixel data is calculated for each pixel constituting the analysis region. Then, an image composed of an average value of data of each pixel calculated for each pixel may be used as the average image.

In the following description, the image _Fa 1 included in the group _{G a,} ..., _{Fa 20} and the image _Fa 51, ..., the average image of _{Fa 80} and _{A a,} the image _Fa 21 included in the group _{G b,} ..., the _{Fa 50} Let the average image be _Ab .

Image _Fa 1, ..., _{Fa 20} and the image _Fa 51, ..., instead of calculating the one of the average image _{A a} from _{Fa 80,} image _Fa 1, ..., calculates an average image _{A a1} from _{Fa 20,} image Fa Another average image A _a2 may be calculated from ₅₁ ,..., Fa ₈₀ .

<Step S6>:
A new image in the analysis area is calculated by subtracting the average image from each image in the analysis area for each group. Here, in order to subtract the average image from each image of the analysis region, for example, for each pixel constituting the analysis region, the pixel data of the average image is subtracted from the pixel data of each image, and the calculated value is used. What is necessary is just to obtain the image which becomes.

a new image of the analysis region of the n-th frame and Fb _n. Image _Fa 1 included in the group _{G a, ..., Fa 20,} Fa 51, ..., for the _{Fa 80, Fb 1 = Fa 1} -A a, ..., Fb 20 = Fa 20 -A a, Fa 51 = Fa ₅₁ -A _{a, ...,} by _Fa 51 ₌ Fa 20 -A _a, new image _{Fb 1, ..., Fb 20,} Fb 51, ..., and calculates the _{Fb 80,} image _{fa 21} included in the group _{G b,} ... , Fa ₅₀ , new images Fb ₂₁ ,..., Fb ₅₀ are calculated by Fb ₂₁ = Fa ₂₁ −A _b ,..., Fb ₅₀ = Fa ₅₀ −A _b .

New image _{Fb 1,} ..., instead of subtracting _{Fb 20, Fb} 51, ..., in order to calculate the _{Fb 80,} each image _{Fa 1, ..., Fa 20,} Fa 51, ..., a common _{A a} from _{Fa 80,} the , each image _Fa 1, ..., by subtracting the average image _{a a1} described above from _{Fa 20,} each image _Fa 51, ..., may be subtracted an average image _{a a2} described above from _{Fa 80.}

<Step S7>:
The correlation value is calculated for each group based on the new image. The data of each pixel used for the calculation of the correlation value may be the data of the pixel itself, or may be a statistical value of data of a plurality of pixels including the pixel. The plurality of pixels may be, for example, a pixel of interest and a pixel adjacent thereto. The statistical value may be, for example, any one of an average value, a maximum value, a minimum value, a relative difference, an absolute difference, and a relative ratio of pixel data. What statistics are used depends on what information is desired to be obtained by RICS analysis.

  For example, the minimum value of the pixel data is calculated for each group, the calculated minimum value is added to the data of all the pixels of the new image, and the correlation value is calculated for the new image obtained by adding the minimum value. To do. This process makes all the pixel data of the new image positive. In this example, the minimum value is added to make all the pixel data of the new image positive. However, the value to be added is not limited to the minimum value, and an arbitrary value greater than the minimum value can be added. Good.

  Further, the correlation value may be calculated by reconstructing images based on pixel data, and calculating the correlation value for the reconstructed image. For example, the data of adjacent pixels is added to halve the number of pixel data. Alternatively, the data of one pixel is divided into a plurality of pieces. Originally, once the image is acquired, the number of pixel data does not increase, but it cannot be acquired on the assumption that the intensity of the acquired pixel data spreads around the pixel data in a Gaussian distribution. Supplement pixel data. Although the number of pixel data is not essentially increasing, it looks better.

For the calculation of the correlation value, for example, the spatial autocorrelation value is calculated using the following equation (1).

Here, _Gsa is a RICS spatial autocorrelation value, I is pixel data, x and y are spatial coordinates of measurement points, ξ and ψ are changes in spatial coordinates from the measurement points, and M ₁₁ is a data product. the number of the sum calculation, M ₁ is the total number of data.

Or the calculation of a correlation value calculates a spatial cross-correlation value using following Formula (2), for example.

Where G _sc is the RICS spatial cross-correlation value, I ₁ is the pixel 1 pixel data, I ₂ is the channel 2 pixel data, x and y are the spatial coordinates of the measurement points, and ξ and ψ are the measurement points. , M ₁₂ is the number of data product-sum calculations, M ₁ is the total number of data in channel 1, and M ₂ is the total number of data in channel 2. Equation (2) is a formula for calculating a spatial cross-correlation value between channel 1 and channel 2, but the channel may be changed as appropriate.

  Further, the correlation values calculated based on each new image are averaged.

The correlation value calculated based on the new image Fb _n in the analysis area of the nth frame is set as G _sn . Then, an average correlation value G _s is calculated by G _s = (G _s1 + G _s2 +... + G _s79 + G _s80 ) / 80.

<Step S8>:
The calculation result of the spatial correlation value in the above step S7 is fitted by equation (3).

Here, G _s is the RICS spatial correlation value, S is the influence of scanning in the RICS analysis, G is the influence of time delay in the RICS analysis, D is the diffusion constant, and ξ and ψ are the spatial coordinates from the measurement point. Variation, δr is the pixel size, N is the number of molecules, W ₀ is the lateral radius of the excitation laser beam, W _Z is the longitudinal radius of the excitation laser beam, τ _p is the pixel time, τ _l is the line time, γ Is an arbitrary constant.

  Note that γ can be set as appropriate according to the system. For example, fitting may be performed with γ = 1.

  The pixel time means a shift in acquisition time between pixels. The line time means a difference in acquisition time between the first pixel of an arbitrary line and the first pixel of the next line. That is, the line time means the time required to scan one line.

In this way, the diffusion time is estimated by fitting using equation (3). Specifically, the correlation value G _sa or the correlation value G _sc for different delay times is obtained using Formula (1) or Formula (2). Here, the delay time means the difference between the acquisition time of one pixel and the acquisition time of another pixel. For example, the delay time of (ξ, ψ) = (1,1) and (ξ, ψ) = (4,2) is expressed as (4-1) τ _p + (2-1) τ ₁ . Is done.

Here, in the formula (3), when the delay time is zero (ξ = 0, ψ = 0 ), S is 1, the _{G s} expressed by 1 / N. Therefore, the number of molecules can be obtained. This is newly substituted into Equation (3).

Then, while varying the diffusion constant D is unknown, so that the correlation value and the G _sa or correlation value G _sc obtained as a measured value, a G _s obtained theoretically, the difference is minimized, adequate diffusion A constant D is obtained. Thus, the fitting by the equation (3) is to estimate the optimal number of molecules or diffusion constant in the two-dimensional or three-dimensional observation region while changing the diffusion constant D.

  The diffusion time can be obtained from the diffusion constant.

That is, the relationship between the diffusion constant and the diffusion time is expressed by the following equation (4).

Here, D is a diffusion constant, τ is a diffusion time, and W ₀ is a lateral radius of the excitation laser beam.

<Step S9>:
Display the analysis results and save as appropriate. Specifically, the calculation result of the spatial correlation value obtained in step S7 and the fitting result of the spatial correlation value obtained in step S8 are displayed. An example of the analysis result is shown in FIGS. FIG. 7 is an image showing the calculation result of the spatial correlation value for a small molecule in luminance, and FIG. 8 shows the fitting result. FIG. 9 is an image showing the calculation result of the spatial correlation value for a large molecule in luminance, and FIG. 10 shows the fitting result.

  As described above, in the present embodiment, in spatial correlation analysis based on image data such as RICS, images of a plurality of frames are classified into several groups according to changes in the frame data, and an average is obtained for each group. Image subtraction processing is performed. In other words, avoid calculating and subtracting one average image from the frames where the effects of different large molecules and immobile molecules coexist, classify the effects of large molecules and immobile molecules in advance, and average images for each classified group Is calculated and subtracted. As a result, for each group corresponding to the change in the frame data, the influence of large molecules and immobile molecules is removed in the right place.

  The embodiments of the present invention have been described above with reference to the drawings. However, the present invention is not limited to these embodiments, and various modifications and changes can be made without departing from the scope of the present invention. Also good.

  In the embodiment, the image acquired by the raster scan has been described. However, the image is not limited to the image acquired by the raster scan, and the pixel data is obtained from a plurality of pixels acquired in time series. And may be obtained by other scanning methods.

DESCRIPTION OF SYMBOLS 100 ... Image analysis apparatus 110 ... Light irradiation part, 112a, ..., 112n ... Light source, 114a, ..., 114n ... Collimate lens, 116a, ..., 116n ... Dichroic mirror, 122 ... Dichroic mirror, 124 ... Galvano mirror, 126 ... Objective lens, 128 ... Objective lens drive mechanism, 130 ... Photodetector, 132 ... Converging lens, 134 ... Pinhole, 136 ... Collimate lens, 138a, ..., 138n ... Dichroic mirror, 140a, ..., 140n ... Fluorescent filter, 142a ,..., 142 n... Photodetector, 160... Control section, 162... Scanning control section, 164... Image forming section, 166. Data extraction unit, 178 ... analysis unit, 180 ... stage control unit, 190 ... sample Stage, R1 ... observation area, R2 ... the analysis region.

Claims (18)

A scanning optical microscope image analysis method for analyzing a scanning image obtained by detecting light from a measurement point in a sample in which molecules of different sizes are mixed and scanning the measurement point to be detected,
An image acquisition step of acquiring a plurality of frames of scan images, each of which includes a plurality of pixels, in which pixel data of each scan image is acquired in time series;
An analysis region setting step for setting an analysis region for the scanned image;
A segment value calculating step for calculating a segment value of the analysis area;
A classification step of classifying the plurality of scanned images into two or more groups based on the magnitude of the change in the segment value;
An average image calculation step of calculating an average image of the analysis region for each group;
A new image calculation step of subtracting the average image from each scanned image of the analysis region for each group to calculate a new image of the analysis region;
An image analysis method comprising: calculating a correlation value by raster image correlation spectroscopy based on the new image for each group.   The image analysis method according to claim 1, wherein the segment value is one of a maximum value, a minimum value, an average value, a relative difference, and a relative ratio of pixel data in the analysis region.   The image analysis method according to claim 1 or 2, wherein the pixel data is fluorescence intensity obtained from a two-dimensional or three-dimensional observation region.   The calculation step calculates a minimum value of the pixel data for each group, adds the calculated minimum value to data of all pixels of the new image, and calculates a correlation value for the new image obtained by adding the minimum value. The image analysis method according to claim 1, wherein the image analysis method performs calculation.   5. The image analysis according to claim 1, wherein the average image calculation step calculates the number of images included in each group, and calculates an average image of each group based on the calculated number of images. Method.   6. The image analysis method according to claim 1, wherein the calculating step reconfigures the plurality of images based on the pixel data, and calculates a correlation value for the reconfigured images. .   The image analysis method according to claim 1, wherein the image is obtained by a scanning microscope. The calculation step performs a correlation calculation of a two-dimensional or three-dimensional observation region using the following formula (1) or the following formula (2):

Here, G _sa is the RICS spatial autocorrelation value, I is the pixel data, x and y are the spatial coordinates of the measurement point, ξ and ψ are the changes in the spatial coordinates from the measurement point, and M ₁₁ is the data product. The number of sum calculations, M ₁ is the total number of data,

Where G _sc is the RICS spatial cross-correlation value, I ₁ is the pixel 1 pixel data, I ₂ is the channel 2 pixel data, x and y are the spatial coordinates of the measurement points, and ξ and ψ are the measurement points. change of the spatial coordinates from, M ₁₂ is the number of data product sum calculation, M ₁ is the total number data of the channel 1, M ₂ is the total number data of the channel 2, any one of claims 1 to 7 The image analysis method described in 1. In the calculation step, the calculation result of the spatial correlation value is fitted using the following equation (3):

Here, G _s is the RICS spatial correlation value, S is the influence of scanning in the RICS analysis, G is the influence of time delay in the RICS analysis, D is the diffusion constant, and ξ and ψ are the spatial coordinates from the measurement point. Variation, δr is the pixel size, N is the number of molecules, W ₀ is the lateral radius of the excitation laser beam, W _Z is the longitudinal radius of the excitation laser beam, τ _p is the pixel time, τ _l is the line time, γ The image analysis method according to claim 8, wherein is an arbitrary constant. A scanning optical microscope having an image analysis device for analyzing a scanning image obtained by detecting light from a measurement point in a sample in which molecules of different sizes are mixed and scanning the measurement point to be detected,
Image acquisition means for acquiring, in a time series, a plurality of frames of a scan image composed of a plurality of pixels in which pixel data of each scan image is acquired in a time series;
Analysis region setting means for setting an analysis region for the scanned image;
Section value calculation means for calculating the section value of the analysis region;
The plurality of scan images are classified into two or more groups based on the magnitude of the change in the segment value , an average image of the analysis region is calculated for each group, and the scan image of the analysis region is calculated for each group. A scanning optical microscope having an image analysis apparatus, comprising: an arithmetic unit that subtracts the average image to calculate a new image in the analysis region, and calculates a correlation value for the new image by raster image correlation spectroscopy . The scanning optical microscope having an image analysis apparatus according to claim 10, wherein the division value is any one of a maximum value, a minimum value, an average value, a relative difference, and a relative ratio of pixel data in the analysis region. The scanning optical microscope having an image analysis apparatus according to claim 10 or 11, wherein the pixel data is fluorescence intensity obtained from a two-dimensional or three-dimensional observation region. The calculation step calculates a minimum value of the pixel data for each group, adds the calculated minimum value to data of all pixels of the new image, and calculates a correlation value for the new image obtained by adding the minimum value. A scanning optical microscope having the image analysis apparatus according to claim 10, which performs an operation. The image analysis apparatus according to claim 10, wherein the calculation unit calculates the number of images included in each group, and calculates the average image for each group based on the number of images. A scanning optical microscope . The image analysis apparatus according to claim 10, wherein the calculating step reconfigures the plurality of images based on the pixel data, and calculates a correlation value for the reconfigured images. A scanning optical microscope . The scanning optical microscope having the image analysis apparatus according to claim 10, wherein the image is obtained by a scanning microscope . The computing means estimates the number of molecules or the diffusion constant of a two-dimensional or three-dimensional observation region based on the following formula (1) and the following formula (2):

Here, G _sa is the RICS spatial autocorrelation value, I is the pixel data, x and y are the spatial coordinates of the measurement point, ξ and ψ are the changes in the spatial coordinates from the measurement point, and M ₁₁ is the data product. The number of sum calculations, M ₁ is the total number of data,

Where G _sc is the RICS spatial cross-correlation value, I ₁ is the pixel 1 pixel data, I ₂ is the channel 2 pixel data, x and y are the spatial coordinates of the measurement points, and ξ and ψ are the measurement points. change of the spatial coordinates from, M ₁₂ is the number of data product sum calculation, M ₁ is the total number data of the channel 1, M ₂ is the total number data of the channel 2, according to any one of claims 10 to 16 Scanning optical microscope having an image analysis apparatus. The calculation means fits the calculation result of the spatial correlation value using the following equation (3):

Here, G _s is the RICS spatial correlation value, S is the influence of scanning in the RICS analysis, G is the influence of time delay in the RICS analysis, D is the diffusion constant, and ξ and ψ are the spatial coordinates from the measurement point. Variation, δr is the pixel size, N is the number of molecules, W ₀ is the lateral radius of the excitation laser beam, W _Z is the longitudinal radius of the excitation laser beam, τ _p is the pixel time, τ _l is the line time, γ The scanning optical microscope having the image analysis apparatus according to claim 17, wherein is an arbitrary constant.

Images