JP5468805B2 - Image analysis method and image analysis apparatus - Google Patents

Description

  The present invention relates to an image analysis method and an image analysis apparatus.

  Conventionally, methods as shown in the following non-patent documents 1 and 2 have been proposed as Raster Image Correlation Spectroscopy (RICS). In this image analysis method, a fluorescence image composed of a raster scan image of one frame or more is acquired. In other words, an area of interest is determined in the sample to be image-analyzed, and the area is repeatedly scanned by the raster scanning method to obtain an image composed of a plurality of frames of fluorescence intensity. The fluorescence intensity in the frame is expressed as data in pixel units. These pieces of pixel data (pixel data) have different acquisition times and acquisition positions, and therefore the acquisition times and acquisition positions corresponding to the respective data are shifted.

Therefore, correlation characteristics due to molecular fluctuations can be obtained by performing spatial autocorrelation analysis using data of these pixels. Here, the diffusion constant and the number of molecules can be obtained from the correlation characteristics of the molecules. Further, the molecular diffusion time can be obtained from the diffusion constant .

In this way, by performing the spatial autocorrelation analysis, the diffusion time , the number of molecules, and the like can be evaluated, so that the interaction between molecules can be observed.

`` Measuring Fast Dynamics in Solutions and Cells with a Laser Scanning Microscope '', Michelle A. Digman, Claire M. Brown, Parijat Sengupta, Paul W. Wiseman, Alan R. Horwitz, and Enrico Gratton, Biophysical Journal, Vol.89, P1317 -1327, August 2005. `` Fluctuation Correlation Spectroscopy with a Laser-Scanning Microscope: Exploiting the Hidden Time Structure '', Michelle A. Digman, Parijat Sengupta, Paul W. Wiseman, Claire M. Brown, Alan R. Horwitz, and Enrico Gratton, Biophysical Journal: Biophysical Letters, L33-36, 2005.

The observation range in the RICS analysis is limited to the shift of the acquisition time and acquisition position corresponding to each data in one frame acquired by raster scanning. For example, in the fluorescence image data of 256 pixels × 256 pixels, the maximum value t _d of the observable molecular diffusion time is 256 × τ _p + 255 × τ ₁ . For the sake of simplicity, the shift in acquisition time between the last pixel of an arbitrary line and the first pixel of the next line is approximated to τ _p .

Here, τ _p is a shift in acquisition time (pixel time) between pixels. Also, τ _l is a shift in acquisition time (line time) between the first pixel of an arbitrary line and the first pixel of the next line. That is, the line time means the time required to scan one line. In this case, 256 pixels constitute one line.

  In order to observe a molecule having a short molecular diffusion time, it is necessary to set the pixel time and the line time small. This is because if the pixel time and the line time are set large, the difference in the acquisition time between the pixels with respect to the molecular diffusion time is large, so that a sufficient number of correlation calculations cannot be performed. As a result, since it becomes necessary to perform fitting based on a small number of calculation results, the accuracy of the obtained correlation characteristics is lowered.

  On the other hand, in order to observe a molecule having a long molecular diffusion time, it is necessary to set a large pixel time and line time. This is because if the pixel time and the line time are set to be small, the actual molecular diffusion time exceeds the maximum value of the molecular diffusion time that can be observed in one frame, and the molecular diffusion time cannot be calculated.

  Here, since the pixel time and the line time are determined by the scanning speed, once the measurement is completed, the pixel time and the line time in the obtained image data cannot be adjusted. Therefore, in order to appropriately obtain the diffusion constant of the molecule, it is necessary to estimate an appropriate pixel time and line time before measurement.

  However, since the molecular diffusion time of most samples is unknown, it is difficult to estimate the pixel time and line time settings in advance.

  An object of the present invention is to provide an image analysis method capable of extending an observable diffusion time.

An image analysis method according to the present invention includes an image acquisition step of acquiring, in a time series, an image of a plurality of frames including a plurality of pixels in which pixel data of each image is acquired in a time series, and an analysis region for the image An analysis area setting step for setting a correlation value; an image selection step for selecting a selection image of a plurality of frames to be used for analysis from the image; and calculating a correlation value based on pixel data in the analysis area of the selection image And a calculation step for estimating the estimated diffusion time . The selection image is selected based on the estimated diffusion time.

  According to the present invention, an image analysis method capable of extending the observable diffusion time is provided.

1 schematically shows an image analysis apparatus according to the present embodiment. The functional block of the control part shown by FIG. 1 is shown. The fluorescence image of K frame acquired in time series for image analysis is shown. The product sum calculation of the correlation calculation in the fluorescence image of the first frame is schematically shown. The product sum calculation of the correlation calculation between the fluorescence images of the first frame and the second frame is schematically shown. The product sum calculation of the correlation calculation between the fluorescence images of the first frame and the third frame is schematically shown. The sum of products calculation of the correlation calculation between the fluorescence images of the first frame and the K frame is schematically shown. The variables ξ, ψ, η and the frame used for the calculation in the equations (1) and (2) are shown. It is a flowchart of the image analysis by this embodiment. A fluorescence image area and an analysis area are shown. It is the image image which showed the result of the RICS analysis with respect to a small molecule | numerator by the brightness | luminance. The fitting results of RICS analysis for small molecules are shown. It is the image which showed the result of the analysis of RICS with respect to a big molecule with brightness. The RICS analysis fitting results for large molecules are shown.

  Hereinafter, the present embodiment will be described with reference to the drawings.

  FIG. 1 schematically shows an image analysis apparatus according to the present embodiment. This image analysis apparatus is configured based on a scanning confocal optical microscope for fluorescence observation of a sample.

  As shown in FIG. 1, the image analysis apparatus 100 is necessary for image analysis, a light irradiation unit 110 that irradiates the sample S with excitation light, a light detection unit 130 that detects light emitted from a measurement point in the sample S, and the like. And a sample stage 190 that supports the sample S.

  The sample S is accommodated in a sample container such as a microplate or a slide glass, and is placed on the sample stage 190. For example, the sample stage 190 supports the sample S so as to be movable in the lateral direction (xy direction) and the height direction (z direction) with respect to the light irradiation unit 110 and the light detection unit 130. For example, the sample stage 190 includes three stepping motors whose output axes are orthogonal to each other, and the sample S can be moved in the xyz direction by these stepping motors.

  The image analysis apparatus 100 is a multiple light irradiation / multiple light detection type. For this reason, the light irradiation unit 110 includes an n-channel light source system 111, and the light detection unit 130 correspondingly includes an n-channel detection system 131. Each of the n-channel detection systems 131 detects fluorescence generated by the excitation light emitted from the n-channel light source system 111. Here, the n channel includes channel 1, channel 2,... Channel n. The channel differs depending on the type of excitation light.

  The n-channel light source system 111 of the light irradiation unit 110 includes light sources 112a,..., 112n, collimating lenses 114a,..., 114n, and dichroic mirrors 116a,. The light sources 112a,..., 112n emit excitation light for exciting the fluorescent dye contained in the sample S to emit light (fluorescence) from the sample S. The wavelengths of the excitation light emitted from the light sources 112a,..., 112n are different from each other corresponding to the type of fluorescent dye contained in the sample S. The light sources 112a,..., 112n are constituted by, for example, laser light sources having an oscillation wavelength suitable for the fluorescent dye in the sample S. The collimating lenses 114a,..., 114n collimate the excitation light emitted from the light sources 112a,. The dichroic mirrors 116a,..., 116n reflect the excitation light that has passed through the collimating lenses 114a,. Each of the dichroic mirrors 116a,..., 116n transmits the excitation light incident from above in FIG. 1 and reflects the excitation light incident from the right side in FIG. As a result, the excitation lights having different wavelengths respectively emitted from the light sources 112a,..., 112n are combined into one beam after passing through the dichroic mirror 116a. The dichroic mirror 116n does not need to transmit the excitation light, and may be changed to a simple mirror.

  The light irradiation unit 110 further includes a dichroic mirror 122, a galvano mirror 124, an objective lens 126, and an objective lens driving mechanism 128. The dichroic mirror 122 reflects the excitation light from the light source system 111 toward the galvanometer mirror 124 and transmits the fluorescence emitted from the sample S. The galvanometer mirror 124 reflects the excitation light toward the objective lens 126 and changes the reflection direction thereof. The objective lens 126 converges the excitation light and irradiates the measurement point in the sample S, and takes in the light from the measurement point in the sample S. An objective lens 126 having a large NA (numerical aperture) is used to form a minute confocal region (measurement point). The confocal region thus obtained has a substantially cylindrical shape with a diameter of about 0.6 μm and a length of about 2 μm. The galvanometer mirror 124 constitutes xy scanning means for scanning the measurement point in the xy direction. The xy scanning unit may be configured using an acousto-optic modulation element (AOM), a polygon mirror, a hologram scanner, or the like in addition to using a galvano mirror. The objective lens driving mechanism 128 moves the objective lens 126 along the optical axis. Thereby, the measurement point is moved in the z direction. That is, the objective lens driving mechanism 128 constitutes z scanning means for scanning the measurement point in the z direction.

  The light detection unit 130 shares the objective lens 126, the galvano mirror 124, and the dichroic mirror 122 with the light irradiation unit 110. The light detection unit 130 further includes a converging lens 132, a pinhole 134, and a collimating lens 136. The converging lens 132 converges the light transmitted through the dichroic mirror 122. The pinhole 134 is disposed at the focal point of the converging lens 132. That is, the pinhole 134 is in a conjugate position with respect to the measurement point in the sample S, and selectively allows only light from the measurement point to pass through. The collimating lens 136 collimates the light that has passed through the pinhole 134. The light that has passed through the collimator lens 136 enters the n-channel detection system 131.

  The n-channel detection system 131 includes dichroic mirrors 138a, ..., 138n, fluorescent filters 140a, ..., 140n, and photodetectors 142a, ..., 142n.

  The dichroic mirrors 138a,..., 138n selectively reflect light having a wavelength in the vicinity of the fluorescence wavelength region generated from the sample S by the excitation light from the light sources 112a,. The dichroic mirror 138n does not need to transmit light, and may be changed to a simple mirror. The fluorescence filters 140a,..., 140n block the light having an undesired wavelength component from the light reflected by the dichroic mirrors 138a,..., 138n, respectively, and are generated by the excitation light from the light sources 112a,. Only the fluorescence is selectively transmitted. The fluorescence transmitted through the fluorescent filters 140a,..., 140n is incident on the photodetectors 142a,. The photodetectors 142a,..., 142n output signals corresponding to the intensity of incident light. That is, the photodetectors 142a,..., 142n output fluorescence intensity signals from the measurement points in the sample S.

  The control unit 160 is configured by a personal computer, for example. The control unit 160 acquires, stores, and displays the fluorescence image of the observation region of the sample S, waits for input of the number of frames (number) of the acquired fluorescence image, the analysis region and desired parameters, and performs image analysis (calculation of correlation values). And estimation of diffusion time. The control unit 160 controls the galvanometer mirror 124 that is an xy scanning unit, the objective lens driving mechanism 128 that is a z scanning unit, the sample stage 190, and the like.

  FIG. 2 shows functional blocks of the control unit shown in FIG. As shown in FIG. 2, the control unit 160 includes a scan control unit 162, an image forming unit 164, a storage unit 166, a display unit 168, an input unit 170, an analysis region setting unit 172, an image selection unit 174, and a data extraction unit 176. An analysis unit 178 and a stage control unit 180 are included. When acquiring the fluorescence image of the sample S, the scanning control unit 162 controls the galvanometer mirror 124 so as to raster scan the irradiation position of the excitation light with respect to the sample S. The scanning control unit 162 also controls the objective lens driving mechanism 128 so that the irradiation position of the excitation light is z-scanned with respect to the sample S if necessary. The image forming unit 164 forms a fluorescent image of the sample S from the information on the irradiation position of the excitation light input from the scanning control unit 162 and the output signals of the photodetectors 142a,. The storage unit 166 sequentially stores the fluorescent images formed by the image forming unit 164. The display unit 168 displays the fluorescence image of the sample S and the analysis result. The input unit 170 includes, for example, a mouse and a keyboard, and constitutes a GUI in cooperation with the display unit 168. This GUI is used for setting the number of acquired frames, observation region and analysis region, inputting desired parameters, and the like. The stage control unit 180 controls the sample stage 190 according to input information from the input unit 170, for example, in order to set an observation region. The analysis area setting unit 172 sets an analysis area according to the input information from the input unit 170. The image selection unit 174 selects a fluorescent image of one frame or more used for analysis based on the desired parameter input from the input unit 170. The data extraction unit 176 extracts necessary data from the fluorescence image stored in the storage unit 166 based on the input information from the analysis region setting unit 172 and the image selection unit 174. The required data varies depending on the situation. The necessary data may be, for example, data of all pixels or data of some pixels of all fluorescent images stored in the storage unit 166, or some data stored in the storage unit 166. Data of all pixels or a part of pixels of the fluorescent image may be used, or data of all pixels or a part of pixels of the one-frame fluorescent image may be used. The analysis unit 178 calculates a correlation value described later on the data extracted by the data extraction unit 176.

  1, excitation light emitted from light sources 112a,..., 112n passes through collimating lenses 114a,..., 114n, dichroic mirrors 116a, ..., 116n, dichroic mirror 122, galvano mirror 124, and objective lens 126. The measurement point is irradiated. The measurement point irradiated with the excitation light is raster scanned in the xy direction by the galvanometer mirror 124. Further, if necessary, z scanning is performed by the objective lens driving mechanism 128. The sample S that has received the excitation light emits fluorescence from the measurement point. Light from the sample S (including undesired reflected light in addition to fluorescence) reaches the pinhole 134 through the objective lens 126, the galvanometer mirror 124, the dichroic mirror 122, and the converging lens 132. Since the pinhole 134 is at a position conjugate with the measurement point, only light from the measurement point in the sample S passes through the pinhole 134. The light that has passed through the pinhole 134, that is, the light from the measurement point in the sample S, enters the n-channel detection system 131 through the collimator lens 136. The light incident on the n-channel detection system 131 is separated (that is, spectrally separated) by the dichroic mirrors 138a,..., 138n, and undesired components are removed by the fluorescent filters 140a,. As a result, only the fluorescence generated by the excitation light from the light sources 112a,..., 112n enters the photodetectors 142a,. Each of the photodetectors 142a,..., 142n outputs incident light, that is, a fluorescence intensity signal indicating the intensity of fluorescence emitted from a measurement point in the sample S. This fluorescence intensity signal is input to the image forming unit 164. The image forming unit 164 processes the input fluorescence intensity signal in synchronization with the positional information in the xy direction (and z direction) for each raster scan (and z scan), and the focal plane in the sample S A fluorescent image of one frame (a flat surface or a curved surface where the measurement point has moved) is formed. The formed fluorescent image is stored in the storage unit 166. The series of operations described here is repeated for the set number of frames to be acquired, and a fluorescence image having the set number of frames is acquired. Each fluorescent image is composed of a plurality of pixels from which pixel data is acquired in time series.

  The fluorescence image stored in the storage unit 166 is processed as necessary and displayed on the display unit 168. For example, it is also possible to obtain a fluorescence image of a plurality of frames by changing the z position of the measurement point, combine them to form a three-dimensional image, and display it on the display unit 168.

  The image analysis method according to this embodiment will be described below. The most important part of the image analysis according to the present embodiment is that a desired parameter is introduced into the spatial correlation calculation formula in order to extend the observable diffusion time in the correlation analysis based on image image data such as RICS. It is to have done.

The following equation (1) represents a spatial correlation calculation formula used for analysis of RICS used in the present embodiment.

Here, G _s is the correlation value of RICS, I ₁₀ is the fluorescence intensity data of the first frame of channel 1 (pixel data), I _1η is the fluorescence intensity data of the η + 1 frame of channel 1 (pixel data), η Is an integer of 0 or more desired parameter (η = 0, 1,..., K−1), x and y are spatial coordinates of measurement points, ξ and ψ are changes in spatial coordinates from the measurement points, and M _0η Is the number of product-sum calculations of the first frame of channel 1 and η + 1 frame, M ₀ is the number of data of the first frame of channel 1, and M _η is the number of data of η + 1 frame of channel 1.

The following equation (2) represents a fitting equation used for analysis of RICS used in the present embodiment.

Here, S is the influence of scanning in the RICS analysis, G is the influence of time delay in the RICS analysis, D is the diffusion constant, δ _r is the pixel size, N is the number of molecules, and W ₀ is the lateral direction of the excitation laser beam. The radius, W _Z is the longitudinal radius of the excitation laser beam, τ _p is the pixel time, τ _l is the line time, and τ _f is the frame time.

When η = 0, the expression (1) is expressed by the following expression (3). Here, τ _f is the difference in acquisition time between the first pixel of an arbitrary frame and the first pixel of the next frame. That is, the frame time means the time required to scan one frame.

Here, I ₁₀ is the fluorescence intensity data (pixel data) of the first frame of channel 1, x and y are the spatial coordinates of the measurement point, ξ and ψ are the changes in the spatial coordinates from the measurement point, and M ₀₀ the first frame number of product sum calculations of the channel 1, M ₀ is the number of data of the first frame of the channel 1.

Moreover, Formula (2) is represented by following Formula (4).

  Expressions (3) and (4) are the same as the conventional spatial correlation calculation expression and fitting expression in the RICS analysis, respectively. That is, the spatial correlation calculation formula for RICS analysis used in the present embodiment is the same as the conventional spatial correlation calculation formula for RICS analysis when η = 0.

  The desired parameter η introduced in the present embodiment is for selecting a fluorescence image to be used for correlation analysis with respect to a plurality of frames of fluorescence images. More specifically, in addition to the fluorescence image of the first frame, a fluorescence image used for correlation analysis is selected. For this reason, when η = 0, there is no fluorescence image used for correlation analysis other than the fluorescence image of the first frame, and as a result, the spatial correlation calculation formula of RICS analysis is the same as the conventional one.

  The fluorescence image used for the correlation analysis may be all of the acquired fluorescence images, for example.

Hereinafter, an example in which the acquired fluorescence image is used for correlation analysis will be described. As shown in FIG. 3, it is assumed that a K-frame fluorescence image has been acquired. Here, K is an integer of 2 or more. In this case, η = K−1.

  In the calculation using the equations (1) and (2), the values of η are sequentially changed to 0, 1, 2,..., K−1, and ξ and ψ are further changed in order for each value of η. While doing. For example, when the analysis region R2 is a region of 256 pixels × 256 pixels, ξ and ψ are sequentially changed to 0, 1, 2,.

  For example, with respect to η = 0, ψ = 0, calculation processing is performed for each value of ξ while increasing ξ by 0, 1, 2,..., 255, and then ψ is increased by 1, η = 0 , Ψ = 1, ξ is incremented by 1 with 0, 1, 2,..., 255 while performing calculation processing for each value of ξ, and this calculation processing is repeated until ψ = 255. Thereafter, η is increased by 1, and for η = 1, ψ = 0, calculation is performed for each value of ξ while ξ is increased by 1, 0, 1, 2,..., 255. Next, η = With respect to 1, ψ = 1, calculation processing is performed for each value of ξ while increasing ξ by 0, 1, 2,..., 255, and this calculation processing is repeated until ψ = 255. This calculation process is repeated until η = K−1 while increasing η by 1.

FIG. 4 schematically shows the product-sum calculation in G _s (2, 4, 0). This calculation is a correlation calculation between pixel data in the analysis region R2 of the image of the same frame (first frame). This is the same as the conventional correlation calculation of the RICS analysis.

FIG. 5 schematically shows the product-sum calculation in G _s (2, 4, 1). This calculation is a correlation calculation between the data of the pixels in the analysis region R2 of the image of different frames. Specifically, this is a correlation calculation between the pixel data in the analysis region R2 of the first frame image and the pixel data in the analysis region R2 of the second frame image.

Similarly, FIG. 6 schematically shows the product-sum calculation in G _s (2, 4, 2). That is, the correlation calculation between the pixel data in the analysis region R2 of the first frame image and the pixel data in the analysis region R2 of the third frame image is shown. FIG. 7 schematically shows the product-sum calculation in G _s (2, 4, K−1). That is, the correlation calculation between the pixel data in the analysis region R2 of the first frame image and the pixel data in the analysis region R2 of the K frame image is shown.

FIG. 8 shows the variables ξ, ψ, η in the equations (1) and (2) and the frame used for the calculation. FIG. 8 shows that the delay time is extended in the calculation using the equations (1) and (2) according to the present embodiment. Specifically, the delay time beyond the 256τ _p + 255τ _l, have been extended to _{256τ p + 255τ l + (K} -1) τ f.

Here, the delay time means the difference between the acquisition time of one pixel and the acquisition time of another pixel. For example, the delay time between (ξ, ψ, η) = (1, 1, 0) and (ξ, ψ, η) = (4, 2, 0) is (4-1) τ _p + ( 2-1) It is represented by τ _l + (0-0) τ _f .

Then, one correlation value G _s is obtained for one delay time. Here, ideally, the correlation value G _s increases as the delay time decreases, and the correlation value G _s decreases as the delay time increases.

  If all of the acquired fluorescence images are used for the correlation analysis, a correct analysis result can be obtained. However, if all the fluorescence images are used, the calculation processing increases accordingly, so that the calculation takes time and the efficiency is poor. For this reason, instead of using all of the acquired fluorescence images for correlation analysis, appropriately reduce the number of frames of the fluorescence image used for correlation analysis, that is, within a range where the value of η can be sufficiently correlated. A small setting is recommended.

  However, if the value of η is too small, the number of data is insufficient. In other words, a sufficient delay time cannot be obtained, and the analysis result is not correct. That is, if the value of η is too small, when observing a molecule having a long diffusion time, the molecular diffusion time may be longer than half the maximum delay time. In this case, the diffusion time cannot be calculated appropriately.

  Therefore, the value of η is preferably set as small as possible within a range where an appropriate analysis result is obtained. Such a value of η may be set based on the estimated diffusion time (estimated diffusion time), for example, by estimating the diffusion time.

  Such a value of η is set as follows, for example.

First, RICS analysis is performed using Equations (3) and (4) where the desired parameter η in Equations (1) and (2) is 0, and the diffusion time is estimated. Specifically, correlation values G _s for different delay times are obtained using Equation (3). Then, from the relationship between the delay time and the correlation value G _s , the diffusion constant and the number of molecules are obtained using Equation (4).

That is, when the delay time is zero (ξ = 0, ψ = 0), S is 1 and G _s can be expressed by 1 / N. Therefore, the number of molecules can be obtained. By newly substituting this into equation (4), the diffusion constant corresponding to each delay time can be obtained. The diffusion time can be obtained from the diffusion constant.

That is, the relationship between the diffusion constant and the diffusion time is expressed by the following equation (5).

Here, D is a diffusion constant, τ is a diffusion time, and W ₀ is a lateral radius of the excitation laser beam.

  If a molecule having a short diffusion time is analyzed, the obtained diffusion time indicates an appropriate diffusion time even if η is zero. This is because when the maximum delay time is larger than twice the diffusion time, the fitting can be sufficiently performed using the equation (4).

  On the other hand, if a molecule with a long diffusion time is analyzed, if η is 0, the diffusion time obtained from the equation (4) does not necessarily match the actual diffusion time. do not do. Therefore, in this case, it is only an estimation of the diffusion time.

Next, η _min is obtained using the following equation (6).

Here, τ _D is the estimated diffusion time, ξ _max , ψ _max are the maximum values of changes in spatial coordinates in one frame, and n is an integer of 2 or more.

  Further, the minimum integer satisfying the above equation (7) is set to η.

  In general, the maximum delay time is about twice as long as the diffusion time for calculating the correlation value. If the maximum delay time is shorter than twice the diffusion time, it is difficult to fit appropriately using the equation (2). Therefore, if η is calculated using equations (6) and (7) with at least n = 2, it can be considered that fitting can be performed appropriately using equation (2). However, n may be set to an integer of 3 or more according to the accuracy required for the analysis result. Accordingly, it is considered that an appropriate analysis result can be obtained if the RICS analysis uses Expression (1) and Expression (2) in which η satisfying Expression (7) is substituted.

  As can be understood from the above description, the RICS correlation analysis in which the desired parameter η is introduced according to the present embodiment is roughly divided into three parts.

  The first part is a part for estimating the diffusion time using the expressions (3) and (4) when the desired parameter η in the expressions (1) and (2) is zero. First, data of an area (analysis area) where correlation analysis is performed is extracted from a plurality of frames of fluorescence images (not necessary when data of the entire frame is used). Next, RICS analysis is performed using equations (3) and (4) with the desired parameter set to zero based on the data in the analysis region, and the diffusion time is estimated. In the RICS analysis using the equations (3) and (4) when the desired parameter is zero, the main part of the correlation calculation is the product of the data in the same frame (of the data at both ends of the arrow) as in FIG. Consists of sum calculation.

  The second part is a part for calculating the desired parameter using the equations (6) and (7) based on the estimated diffusion time (estimated diffusion time). By using the non-zero desired parameter calculated in this way, the observable diffusion time is extended.

  In the final part, the desired parameter calculated in the second part is substituted into Equations (1) and (2) to calculate a correlation value. Then, the RICS analysis including the diffusion time is performed again. In addition, as shown in FIGS. 3 to 7, the RICS analysis using the calculated desired parameters using the equations (1) and (2) is performed as follows. The calculation is expanded to the product-sum calculation of the data between the different frames (the data at both ends of the arrow).

  The main procedure of the image analysis is shown in the flowchart of FIG.

<Step S1>:
After setting the observation area of the sample S and the number of frames of the fluorescent image to be acquired, the above-described operation for acquiring the fluorescent image of one frame is repeated as many times as the set number of frames to acquire the fluorescent image of the set number of frames. To do. Of course, the fluorescence image is acquired by the same scanning method for the same observation region. The acquired fluorescence images of a plurality of frames are images acquired in time series, and each image includes a plurality of pixels in which pixel data is acquired in time series.

<Step S2>:
As shown in FIG. 10, an analysis region R2 is set for the fluorescence image region R1. The analysis region R2 is not limited to a part of the fluorescence image region R1, and may coincide with the fluorescence image region R1. The analysis region R2 is set as the fluorescence image region R1, that is, the scanning region in step S1 by default in terms of application, and this step is not necessary when the entire fluorescence image region R1 is analyzed.

<Step S3>:
Expression (3) and Expression (4) are obtained by setting the desired parameter η in Expression (1) and Expression (2) to zero.

<Step S4>:
Based on the analysis region R2 set in step S2, pixel data of the analysis region R2 of the fluorescence image of the first frame is extracted. A diffusion constant is calculated from the extracted data and equations (3) and (4) (RIICS correlation analysis) in step S3. Then, using the calculated diffusion constant, the diffusion time is estimated from Equation (5).

<Step S5>:
The desired parameter η is calculated using the estimated diffusion time and equations (6) and (7).

<Step S6>:
The calculated desired parameter η is substituted into formulas (1) and (2) to obtain a new RICS spatial correlation formula.

<Step S7>:
A fluorescent image to be used for analysis is selected based on the desired parameter η calculated in step S5. More specifically, images acquired continuously from the first frame to the η + 1 frame are selected. Further, based on the analysis region R2 set in step S2, pixel data of the analysis region R2 of the fluorescence image used for analysis is extracted. Correlation analysis is performed using the extracted data and the new RICS spatial correlation calculation formula obtained in step S6.

<Step S8>:
The result calculated in step S7 is displayed as an image with luminance values.

  11 to 14 show the results of RICS analysis. FIG. 11 is an image showing the correlation calculation results for small molecules in luminance, and FIG. 12 shows the fitting results. FIG. 13 is an image showing the correlation calculation result for a large molecule in luminance, and FIG. 14 shows the fitting result.

  According to the present embodiment, in the RICS image analysis, the observable diffusion time can be extended without adjusting the pixel time and the line time. For this reason, the diffusion time suitable for the sample can be estimated using the already acquired data without re-collecting data. This makes it possible to calculate an appropriate diffusion time for both molecules having a large diffusion time and molecules having a small diffusion time.

  The embodiments of the present invention have been described above with reference to the drawings. However, the present invention is not limited to these embodiments, and various modifications and changes can be made without departing from the scope of the present invention. Also good.

  In the embodiment described above, in selecting an image to be used for analysis, the first frame and the subsequent images are selected as necessary. However, the head of the selected image is not necessarily the first frame. For example, when a j-frame image from a k-frame image is used for analysis, the image to be selected may be an image from the i-th frame to the i + j−1-th frame. However, i + j-1 ≦ k. In the embodiment, the images to be selected are continuous in time series, but this is not always necessary. However, since a correlation value is obtained for each delay time, if raster scanning is periodic, the frame number of the selected image needs to be periodic. For example, the images used for the analysis may be selected every second frame, the first frame, the fourth frame, and the seventh frame.

  In the embodiment, as the data of each pixel used for the calculation of the correlation value, the data of the pixel is used for the correlation calculation as it is. However, even if the data of a plurality of pixels are read and the statistical values thereof are used for the correlation calculation. Good. Examples of the statistical value include an average value, a maximum value, a minimum value, a relative difference, an absolute difference, and a relative ratio. What statistics are used depends on what information is desired to be obtained by RICS analysis. For example, classification by maximum value can remove the same molecular effect from the same maximum value classification frame. The same applies to other cases, and the same molecular effect can be removed from the same classification frame. In other words, when the standard changes, the classification of the frame changes, and different molecular effects can be removed.

  In the embodiment, the image acquired by the raster scan has been described. However, the image is not limited to the image acquired by the raster scan, and the pixel data is obtained from a plurality of pixels acquired in time series. And may be obtained by other scanning methods.

  In the embodiment, the pixel data is described as fluorescence intensity data, but the pixel data may be other data.

In the embodiment, the autocorrelation analysis in which the correlation analysis is performed using the pixel data of the same channel has been described. Instead of this, cross-correlation analysis may be performed in which correlation analysis is performed using pixel data of different channels. In this case, it is represented by the following formula (8).

Here, G _s ′ is the cross correlation value of RICS, I ₁₀ is the fluorescence intensity data of the first frame of channel 1 (pixel data), and I _2η is the fluorescence intensity data of the η + 1 frame of channel 2 (pixel data). Η is an integer of 0 or more desired parameter (η = 0, 1,..., K−1), x and y are spatial coordinates of the measurement point, ξ and ψ are changes in the spatial coordinates from the measurement point, M ₁₂ is the number of product-sum calculations for the first frame of channel 1 and the η + 1 frame of channel 2, M ₁ is the number of data of the first frame of channel 1, and M ₂ is the number of data of the η + 1 frame of channel 2. .

DESCRIPTION OF SYMBOLS 100 ... Image analysis apparatus 110 ... Light irradiation part, 112 ... Light source, 116 ... Mirror, 122 ... Dichroic mirror, 124 ... Galvano mirror, 126 ... Objective lens, 128 ... Objective lens drive mechanism, 130 ... Light detection part, 132 ... Converging lens, 134 ... pinhole, 136 ... collimating lens, 138 ... fluorescent filter, 140 ... converging lens, 142 ... photodetector, 160 ... control unit, 162 ... scanning control unit, 164 ... image forming unit, 166 ... storage unit DESCRIPTION OF SYMBOLS 168 ... Display part, 170 ... Input part, 172 ... Analysis area | region setting part, 174 ... Image selection part, 176 ... Data extraction part, 178 ... Analysis part, 180 ... Stage control part, 190 ... Sample stage, R1 ... Fluorescence image Region, R2 ... Analysis region.

Claims (20)

An image acquisition step of acquiring, in time series, a plurality of frames of a plurality of pixels in which pixel data of each image is acquired in time series;
An analysis region setting step for setting an analysis region for the image;
An image selection step for selecting a selection image of a plurality of frames to be used for analysis from the image;
A calculation step of calculating a correlation value between the selected images of the plurality of frames based on pixel data in the analysis region of the selected image using raster image correlation spectroscopy (RICS) ;
An estimation step for estimating an estimated diffusion time ;
The selected image is selected based on the estimated diffusion time .   In the calculation step, the pixel data in the analysis region of the previous image in time series in the selected image and the pixel data in the analysis region of the selection image acquired later than the pixel are obtained. The image analysis method according to claim 1, wherein a correlation value is calculated. The image analysis method according to claim 1, wherein the selected images of the plurality of frames include all of the images. The image analysis method according to claim 1 , wherein the selected image of the plurality of frames includes a part of the image that is continuous in the time series of the image. The image is an image of K frames, where K is an integer greater than or equal to 2,
Selection of the selected images of the plurality of frames and calculation of correlation values are performed using the following equations (1) and (2).
Here, G _s is the correlation value of RICS, I ₁₀ is the pixel data of the first frame of channel 1, I _1η is the data of the pixel of the η + 1 frame of channel 1, and η is an integer desired parameter (η = 0, 1,..., K-1), x, y are the spatial coordinates of the measurement point, ξ, ψ are the changes in the spatial coordinates from the measurement point, M _0η is the first frame of channel 1 and η + 1 frame The number of product-sum calculations for the eye, M ₀ is the number of data in the first frame of channel 1, M _η is the number of data in η + 1 frame of channel 1,
Here, S is the influence of scanning in the RICS analysis, G is the influence of time delay in the RICS analysis, D is the diffusion constant, δ _r is the pixel size, N is the number of molecules, and W ₀ is the lateral direction of the excitation laser beam. The image analysis method according to claim 1, wherein the radius, W _Z is a longitudinal radius of the excitation laser beam, τ _p is a pixel time, τ _l is a line time, and τ _f is a frame time. Estimating the estimated diffusion time using the following equation (3) and equation (4),
Here, I ₁₀ is the pixel data of the first frame of channel 1, x and y are the spatial coordinates of the measurement point, ξ and ψ are the amount of change in the spatial coordinates from the measurement point, and M ₀₀ is 1 of channel 1. th frame in the number of product sum calculations, M ₀ is the number of data of the first frame of the channel 1,
The image analysis method according to claim 5 , wherein a value of η is set based on the estimated diffusion time. Η _min is calculated based on the following equation (6),
Here, τ _D is the estimated diffusion time, ξ _max , ψ _max is the maximum value of the change in spatial coordinates in one frame, n is an integer of 2 or more,
The image analysis method according to claim 6 , wherein a minimum value that satisfies the above expression (7) is set to η. A light irradiation step of irradiating the sample while scanning the excitation light;
The said image is a fluorescence image of the said sample, The data of the said pixel respond | correspond to the fluorescence intensity emitted from the said sample according to irradiation of the said excitation light, Any one of Claim 1 thru | or 7 The image analysis method described in 1. Wherein the data is a statistical value of the fluorescence intensity, the statistic average value of the fluorescence intensity, maximum value, minimum value, the relative difference is either an absolute difference, one of claims 1 to 8 The image analysis method as described in one. The images were obtained by scanning microscope, image analysis method according to any one of claims 1 to 9. An image acquisition unit that acquires, in time series, a plurality of frames of a plurality of pixels in which pixel data of each image is acquired in time series;
An analysis region setting unit for setting an analysis region for the image;
An image selection unit for selecting a selection image of a plurality of frames to be used for analysis from the image;
A calculation unit that calculates a correlation value between the selected images of the plurality of frames based on pixel data in the analysis region of the selected image using raster image correlation spectroscopy (RICS) ;
An estimation unit for estimating an estimated diffusion time ;
The image analysis apparatus in which selection of the selected images of the plurality of frames is performed based on the estimated diffusion time . The calculation unit includes: pixel data in the analysis region of the previous image in time series in the selection image; and pixel data in the analysis region of the selection image acquired later than the pixel. The image analysis apparatus according to claim 11 , wherein a correlation value is calculated. The image analysis apparatus according to claim 11 , wherein the selected images of the plurality of frames are all of the images. The image analysis apparatus according to claim 11 , wherein the selection image of the plurality of frames is formed of a part of the image that is continuous in the time series of the image. The image is an image of K frames, where K is an integer greater than or equal to 2,
Selection of the selected images of the plurality of frames and calculation of correlation values are performed using the following equations (1) and (2).
Here, G _s is the correlation value of RICS, I ₁₀ is the pixel data of the first frame of channel 1, I _1η is the data of the pixel of the η + 1 frame of channel 1, and η is an integer desired parameter (η = 0, 1,..., K-1), x, y are the spatial coordinates of the measurement point, ξ, ψ are the changes in the spatial coordinates from the measurement point, M _0η is the first frame of channel 1 and η + 1 frame The number of product-sum calculations for the eye, M ₀ is the number of data in the first frame of channel 1, M _η is the number of data in η + 1 frame of channel 1,
Here, S is the influence of scanning in the RICS analysis, G is the influence of time delay in the RICS analysis, D is the diffusion constant, δ _r is the pixel size, N is the number of molecules, and W ₀ is the lateral direction of the excitation laser beam. The image analysis apparatus according to claim 11 , wherein the radius, W _Z is a longitudinal radius of the excitation laser beam, τ _p is a pixel time, τ _l is a line time, and τ _f is a frame time. Estimating the diffusion time using the following equation (3) and equation (4),
Here, I ₁₀ is the pixel data of the first frame of channel 1, x and y are the spatial coordinates of the measurement point, ξ and ψ are the amount of change in the spatial coordinates from the measurement point, and M ₀₀ is 1 of channel 1. th frame in the number of product sum calculations, M ₀ is the number of data of the first frame of the channel 1,
The image analysis device according to claim 15 , wherein a value of η is set based on the estimated diffusion time. Η _min is calculated based on the following equation (6),
Here, τ _D is the estimated diffusion time, ξ _max , ψ _max is the maximum value of changes in spatial coordinates in one frame, n is a positive number of 2 or more,
The image analysis device according to claim 16 , wherein the minimum value that satisfies the above expression (7) is set to η. It further has a light irradiation part that irradiates the sample while scanning the excitation light,
The said image is a fluorescence image of the said sample, The data of the said pixel respond | correspond to the fluorescence intensity | strength emitted from the said sample according to irradiation of the said excitation light, Any one of Claim 11 thru | or 17 The image analysis apparatus described in 1. Wherein the data is a statistical value of the fluorescence intensity, the statistic average value of the fluorescence intensity, maximum value, minimum value, the relative difference is either an absolute difference, one of claims 11 to 18 The image analysis apparatus described in one. The images were obtained by scanning microscope, image analyzer according to any one of claims 11 to 19.