KR101446844B1 - Apparatus and method for estimating fluorescence lifetime - Google Patents

Abstract

An apparatus and a method for estimating a fluorescence time constant are disclosed. The disclosed fluorescence time constant estimator measures a photon excited from a fluorescence sample with time to generate a plurality of measured values, models a photon noise as a Poisson model to generate a base matrix, And then estimates the fluorescence time constant of the photon using the generated objective function. The proposed fluorescence time constant estimation apparatus has an advantage that it can accurately discriminate each fluorescence time constant even when a plurality of fluorescence time constants are present.

Description

[0001] APPARATUS AND METHOD FOR ESTIMATING FLUORESCENCE LIFETIME [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for estimating a fluorescence time constant obtained from an FLIM (Fluorescence Lifetime Imaging Microscopy) image, and more particularly, to a technique for estimating a fluorescence time constant from a change in an index of a photon acquired from an FLIM image .

Understanding the complex and dynamic structure of cells and molecules that make up living tissue in the post-genome era has been a major challenge in biology research. Knowledge of the temporal and spatial organics of cells and molecules has improved the quality of human health and quality of life, making social and economic investments in life research. Bio-imaging and bio-imaging sequences play the most important role in obtaining this biological knowledge. In particular, during the past two decades, advances have been made in microscopy imaging hardware and methodologies for imaging cells and molecules. In particular, confocal microscopy of fluorescent microscopy has been used to restore the three-dimensional structure by increasing the optical resolution by using spatial pinholes and point illumination to improve the resolution of specimens from ordinary optical microscopes.

Recent confocal microscopy images have evolved to enable real-time observation of the microstructure of living cells. These developments have made it possible to study the mechanism of genes and the mechanisms by which diseases occur. FLIM (Fluorescence Lifetime Imaging Microscopy) images of confocal microscopy images show that when the fluorescence of the fluorescent protein is excited by the laser light source in each region of the cell, the photon amount emitted is exponentially decreasing with time (Fluorescence lifetime). FLIM is characterized not only by its microstructure but also by its pH, oxygen partial pressure, modulus information, and the occurrence of FRET, an interaction between proteins.

The number of photons emitted from the fluorescent protein at the time of FLIM imaging is measured over time. The average of the numbers of photons measured is proportional to the brightness of the fluorescent material but follows the Poisson probability density function. Therefore, the Poisson noise should be suppressed from the observed values, but the research for this is still primitive. In addition, it is often difficult to separate two or more time constants from noisy data in the conventional method.

The purpose of the following embodiments is to accurately estimate the fluorescence time constant of an FLIM image.

The purpose of the following examples is to exclude the influence of gun fluorescence on the Poisson's constant.

According to an exemplary embodiment of the present invention, there is provided an image processing apparatus including a measurement unit for measuring photons excited from a fluorescent sample with time to generate a plurality of measured values, a base matrix generator for generating a base matrix by modeling the noise of the photon with a Poisson model, An objective function generator for generating an objective function using the generated base matrix and the measured values according to the time, and an estimator for estimating a fluorescence time constant of the photon using the generated objective function, Device is provided.

Here, the estimator may estimate the fluorescence time constant such that the value of the objective function is minimized.

The objective function may include a negative Poisson log-likelihood of the Poisson model and an L1 norm normalization term for the distribution of the fluorescence time constant.

Further, the objective function may be determined according to the following equation (1).

[Equation 1]

는 상기 포와송 모델의 음의 포와송 로그 확률식(negative Poisson log-likelihood)이고 하기 수학식 2에 따라서 결정된다.

는 형광 시정수의 분포도에 대한 L1 norm 정규화항으로서, 하기 수학식 4에 따라서 결정되고,

는 정규화항을 조절하는 인자이다.
here,

Is a negative Poisson log-likelihood of the Poisson model and is determined according to the following equation (2).

Is an L1 norm normalization term for the distribution of the fluorescence time constant, and is determined according to the following equation (4)

Is a factor controlling the normalization term.

&Quot; (2) "

는 광자의 측정값들 중에서 i번째 값이다. A는 상기 기저 행렬로서, 하기 수학식 3에 따라서 결정되고,

는 상수이다.
Here, 1 means a row vector whose element values are all 1s.

Is the i-th value among the measured values of the photon. A is the base matrix and is determined according to the following equation (3)

Is a constant.

&Quot; (3) "

은 현미경 응답 함수이고, *는 컨벌루션 연산을 의미한다.
here,

Is a microscope response function, and * denotes a convolution operation.

&Quot; (4) "

는 추정된 i번째 형광 시정수의 분포도이다.
here,

Is the distribution of the estimated i-th fluorescence time constant.

The base matrix may include a plurality of elements whose fluorescence time constant differs according to each row and sampling times are different according to each column.

According to another exemplary embodiment, there is provided a method comprising: measuring photons excited from a fluorescent sample over time to produce a plurality of measured values; modeling the photon noise into a Poisson model to generate a basis matrix; Generating an objective function using the base matrix and the measured values according to the time, and estimating a fluorescence time constant of the photon using the generated objective function.

Here, the estimating step may estimate the fluorescence time constant so that the value of the objective function is minimized.

The objective function may include a negative Poisson log-likelihood of the Poisson model and an L1 norm normalization term for the distribution of the fluorescence time constant.

Further, the objective function can be determined according to the following equation (5).

&Quot; (5) "

는 상기 포와송 모델의 음의 포와송 로그 확률식(negative Poisson log-likelihood)이고 하기 수학식 6에 따라서 결정된다.

는 형광 시정수의 분포도에 대한 L1 norm 정규화항으로서, 하기 수학식 8에 따라서 결정되고,

는 정규화항을 조절하는 인자이다.
here,

Is a negative poisson log-likelihood of the Poisson model and is determined according to Equation (6) below.

Is an L1 norm normalization term for the distribution of the fluorescence time constant and is determined according to the following equation (8)

Is a factor controlling the normalization term.

&Quot; (6) "

는 광자의 측정값들 중에서 i번째 값이다. A는 상기 기저 행렬로서, 하기 수학식 6에 따라서 결정되고,

는 상수이다.
Here, 1 means a row vector whose element values are all 1s.

Is the i-th value among the measured values of the photon. A is the base matrix and is determined according to the following equation (6)

Is a constant.

&Quot; (7) "

은 현미경 응답 함수이고, *는 컨벌루션 연산을 의미한다.here,

Is a microscope response function, and * denotes a convolution operation.

&Quot; (8) "

는 추정된 i번째 형광 시정수의 분포도이다.
here,

Is the distribution of the estimated i-th fluorescence time constant.

The base matrix may include a plurality of elements whose fluorescence time constant differs according to each row and sampling times are different according to each column.

According to the embodiments described below, it is possible to accurately estimate a plurality of fluorescence time constants in an FLIM image.

According to the following embodiments, it is possible to exclude the influence of the poofing noise of the fluorescence time constant.

FIG. 1 is a diagram showing photon changes with time of a fluorescent time constant. FIG.
2 is a block diagram showing the structure of a fluorescence time constant estimating apparatus according to an exemplary embodiment.
3 is a diagram showing a result of estimation of the fluorescence time constant.
4 is a flowchart illustrating steps of estimating a fluorescent time constant according to an exemplary embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

FIG. 1 is a diagram showing photon changes with time of a fluorescent time constant. FIG.

Fluorescence Lifetime Imaging Microscopy (FLIM) is an imaging technique that observes the temporal decay rate of photon counts emitted by irradiating a cell containing a fluorescent substance with laser light. The brightness of stimulated fluorescent cells decreases exponentially with time, which is called the fluorescence lifetime. In other words, the image obtained by the FLIM technique is a image in which the fluorescence time constant is represented by a brightness value or a pseudo color, which is essential information for cell analysis such as pH, rigidity and oxygen partial pressure required for cell analysis, Lt; / RTI >

The most common method for acquiring FLIM images is the TCSPC method. TCSPC instruments stimulate fluorescent samples by applying laser pulses in short time periods. Since the observation period is not long after laser stimulation, the probability of observing several photons is very low. Therefore, the first photon observed for each laser stimulus is detected to obtain a histogram on the time axis. The obtained histogram can be modeled in exponential decay form over time. The exponentially decreasing rate of change of the histogram obtained is called the fluorescence time constant, and FLIM can be expressed as an image by mapping the fluorescence time constant value to brightness or pseudo color.

What should be considered in the histogram from photon emission is the IRF (instrument response function), which is the response function of the microscope system itself. Ideal systems will have a delta function-like IRF if the pulse width applies a stimulus pulse that converges to nearly zero and the ideal operation without delay in both detection and electronics. In practice, however, the TCSPC limits the time accuracy of the pulse response of the detector by the timing uncertainty of the detector. In addition, the stimulated light source is also wider than the ideal pulse width, and the observed histogram is observed in convolution with the IRF.

FLIM can be very helpful in understanding the local structure and photochemical properties of the DNA of binding DNA or small molecules by analyzing the fluorescence time constant distribution well. Also, the distribution of fluorescent time constants can help to analyze chemical changes. For example, in the case of Ethidium bromide, it has a single fluorescence time constant when it is in water but a plurality of fluorescence time constants when atoms and ions are inserted.

FIG. 1 (a) is a diagram showing photon variation of an FLIM image having one fluorescence time constant, and FIG. 1 (b) is a diagram showing photon variation of an FLIM image having a plurality of fluorescence time constants. The abscissa of Fig. 1 represents time and the ordinate represents the number of photons measured.

As shown in FIG. 1, it is very difficult to estimate the number of fluorescence time constants or the number of fluorescence time constants only by visually observing the photon change of the FLIM image.

Therefore, it is necessary to accurately estimate the value of the fluorescence time constant and the number of fluorescence time constant in the FLIM image.

2 is a block diagram showing the structure of a fluorescence time constant estimating apparatus according to an exemplary embodiment. The fluorescence time constant estimating apparatus 200 according to the exemplary embodiment includes a measuring unit 210, a base matrix generating unit 220, an objective function generating unit 230, and an estimating unit 240.

The measuring unit 210 measures photons excited from the fluorescence specimen with time to generate a plurality of measured values. In this case, the average of the numbers of photons measured is proportional to the brightness of the fluorescent material. In addition, noise may occur in the measurement process, and the noise of the photon may follow the Poisson probability density function.

The base matrix generator 220 generates a base matrix by modeling the noise of the photon as a Poisson model. According to one aspect, the basis matrix may include a plurality of elements whose fluorescence time constant differs according to each row and the sampling time differs according to each column. According to one aspect, the base matrix can be determined according to Equation (9).

&Quot; (9) "

은 현미경 응답 함수이고, *는 컨벌루션 연산을 의미한다. 또한,

는 샘플링 시간을 나타내고,

는 형광 시정수들을 나타낸다.here,

Is a microscope response function, and * denotes a convolution operation. Also,

Represents the sampling time,

Represents fluorescence time constants.

The objective function generator 230 generates an objective function using the generated base matrix and photon measurement values according to time. According to one aspect, the objective function may include a negative Poisson log-likelihood of the Poisson model and a L1 norm normalization term for the distribution of the fluorescence time constant. The objective function can be determined as shown in Equation (10).

&Quot; (10) "

는 상기 포와송 모델의 음의 포와송 로그 확률식(negative Poisson log-likelihood)이고 하기 수학식 13에 따라서 결정된다.

는 형광 시정수의 분포도에 대한 L1 norm 정규화항이다.

는 정규화항을 조절하는 인자로서, 상수이다.here,

Is a negative poisson log-likelihood of the Poisson model and is determined according to the following equation (13).

Is the L1 norm normalization term for the distribution of the fluorescence time constant.

Is a constant that controls the normalization term.

에 대한 확률은 하기 수학식 11와 같다.
Observation vector of Poisson model

Is given by Equation (11). &Quot; (11) "

&Quot; (11) "

는 i번째 정준기(canonical basis) 벡터이다. 여기서,

는 관측 벡터

의 i번째 원소를 의미하며, i번째로 측정된 측정값을 나타낸다. 이 경우에, 포와송 모델의 음의 포와송 로그 확률식

는 하기 수학식 12에 따라 결정될 수 있다.
Here, A is the base matrix, f is the distribution of the fluorescence time constant to be estimated,

Is the i-th canonical basis vector. here,

Is an observation vector

Represents the i-th measured element of the i-th element. In this case, the Poisson's Poisson probability equation of Poisson's model

Can be determined according to the following equation (12).

&Quot; (12) "

Here, 1 denotes a column vector composed of m 1's.

는

에 대해 항상 일정하므로 목적 함수를 만들 때는 생략할 수 있다. 또한,

인 경우 로그함수가 발산할 수 있으므로, 로그 함수 내부에 상수를 더하여 음의 포와송 로그 확률식

을 하기 수학식 13과 같이 변형할 수 있다.

The

, So it can be omitted when creating the objective function. Also,

, The log function can be diverged. Therefore, by adding a constant to the log function,

Can be modified as shown in Equation (13).

&Quot; (13) "

는 상수로서,

정도의 값을 가질 수 있다.
here,

As a constant,

Can be used.

In Equation (10), pen (f) is L1 norm of f, which is the distribution of the time constant, and can be calculated as Equation (14).

&Quot; (14) "

는 추정된 i번째 형광 시정수의 분포도이다.
here,

Is the distribution of the estimated i-th fluorescence time constant.

The estimator 240 estimates the fluorescence time constant of the photon using the generated objective function. According to one aspect, the estimator 240 can estimate the fluorescence time constant so that the value of the objective function is minimized.

According to one aspect, the FLIM image may comprise a plurality of fluorescence time constants. In this case, accurate fluorescence time constants could not be estimated by conventional analysis techniques using naked eye or z-transform.

3 is a diagram showing a result of estimation of a fluorescence time constant according to an exemplary embodiment.

Even when a plurality of fluorescence time constants are included in the FLIM image, the fluorescence time constant estimation method described in FIG. 2 can be used to not only distinguish each fluorescence time constant but also accurately estimate the fluorescence time constant.

4 is a flowchart illustrating steps of estimating a fluorescent time constant according to an exemplary embodiment.

In step 410, the fluorescence time constant estimator measures photons excited from the fluorescence specimen over time to generate a plurality of measured values. The fluorescence time constant estimating apparatus can generate an observation vector containing a plurality of measurement values as elements.

In step 420, the fluorescence time constant estimator models the photon noise into a Poisson model to generate a basis matrix. According to one aspect, the basis matrix may include a plurality of elements whose fluorescence time constant differs according to each row and the sampling time differs according to each column. The basis matrix can be determined as shown in Equation (7).

In step 430, the fluorescence time constant estimator generates an objective function using the generated base matrix and time-dependent measurements. Here, the objective function may include a negative Poisson log-likelihood of the Poisson model and a L1 norm normalization term for the distribution of the fluorescence time constant. According to one aspect, the fluorescence time constant estimating apparatus can generate an objective function according to Equation (8).

In step 440, the fluorescence time constant estimator estimates the fluorescence time constant of the photon using the generated objective function. According to one aspect, the fluorescence time constant estimator can estimate the fluorescence time constant such that the value of the objective function is minimized.

According to the embodiment illustrated in FIG. 4, even when a plurality of fluorescence time constants are included in the FLIM image, not only the fluorescence time constant can be distinguished, but also the value of the fluorescence time constant can be accurately estimated.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

200: Fluorescence time constant estimation device
210:
220: Base matrix generating unit
230: Object function generating unit
340:

Claims (11)

A measuring unit for measuring photons excited by the fluorescence specimen with time by applying a laser pulse to the fluorescence specimen to generate a plurality of measured values;
A base matrix generating unit for modeling the noise of the photon as a Poisson model to generate a base matrix;
An objective function generator for generating an objective function using the generated base matrix and a plurality of measured values according to the time; And
And estimating a plurality of fluorescence time constants of the photon using the generated objective function,
Lt; / RTI >
The probability for the observed vector of the Poisson model is calculated based on the basis matrix, the distribution of the estimated plurality of fluorescence time constants and the canonical basis vector,
Wherein the objective function is generated based on a probability for the observation vector,
Wherein the objective function comprises a negative Poisson log-likelihood of the Poisson model and a normalization term for the distribution of the plurality of fluorescence time constants. The method according to claim 1,
Wherein the estimator estimates the plurality of fluorescence time constants so that the value of the objective function is minimized. 3. The method of claim 2,
Wherein the normalization term for the distribution of the plurality of fluorescence time constants is a L1 norm normalization term. The method of claim 3,
Wherein the objective function is determined according to the following equation (1).

[Equation 1]

here,

Is a Poisson's Poisson probability equation of the Poisson model and is determined according to the following equation (2).

Is an L1 norm normalization term for the distribution of the plurality of fluorescent time constants, and is determined according to the following equation (4)

Is a factor controlling the L1 norm normalization term.

&Quot; (2) "

Here, 1 means a row vector whose element values are all 1s.

Is the i-th value among the measured values of the photon. A is the base matrix and is determined according to the following equation (3)

Is a constant.

&Quot; (3) "

here,

Is a microscope response function, and * denotes a convolution operation.

&Quot; (4) "

here,

Is the distribution of the estimated i-th fluorescence time constant.
The method of claim 3,
Wherein the basis matrix includes a plurality of elements whose fluorescence time constants are different according to each row and sampling times are different according to each column. Measuring a photon excited from the fluorescent sample with time by applying a laser pulse to the fluorescent sample to generate a plurality of measured values;
Modeling the noise of the photon with a Poisson model to generate a basis matrix;
Generating an objective function using the generated base matrix and a plurality of measured values according to the time; And
Estimating a plurality of fluorescence time constants of the photon using the generated objective function
Lt; / RTI >
The probability for the observed vector of the Poisson model is calculated based on the basis matrix, the distribution of the estimated plurality of fluorescence time constants and the canonical basis vector,
Wherein the objective function is generated based on a probability for the observation vector,
Wherein the objective function comprises a negative Poisson log-likelihood of the Poisson model and a normalization term for the distribution of the plurality of fluorescence time constants. The method according to claim 6,
Wherein the estimating step estimates the plurality of fluorescence time constants to minimize the value of the objective function. 8. The method of claim 7,
Wherein the normalization term for the distribution of the plurality of fluorescence time constants is an L1 norm normalization term. 9. The method of claim 8,
Wherein the objective function is determined according to the following equation (5).

&Quot; (5) "

here,

Is a Poisson's Poisson probability equation of the Poisson model and is determined according to the following equation (6).

Is an L1 norm normalization term for the distribution of the plurality of fluorescent time constants, and is determined according to the following equation (8)

Is a factor controlling the L1 norm normalization term.

&Quot; (6) "

Here, 1 means a row vector whose element values are all 1s.

Is the i-th value among the measured values of the photon. A is the base matrix and is determined according to the following equation (6)

Is a constant.

&Quot; (7) "

here,

Is a microscope response function, and * denotes a convolution operation.

&Quot; (8) "

here,

Is the distribution of the estimated i-th fluorescence time constant.
9. The method of claim 8,
Wherein the base matrix includes a plurality of elements whose fluorescence time constants are different according to each row and sampling times are different according to each column. A computer-readable recording medium on which a program for executing the method according to any one of claims 6 to 10 is recorded.

Images