JP2003065948A - Apparatus, method and program for processing microscopic image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a microscopic-image processing apparatus which can analyze a tissue sample without depending on an observation device and by which the quantification and the conversion of a staining state and the structure of the tissue itself can be clarified. SOLUTION: The microscopic-image processing apparatus is provided with a spectral-transmittance-image estimation part 301 which estimates a spectral transmittance image on the basis of the signal value of a camera obtained by photographing the tissue sample, a pigment-amount-distribution calculation part 302 which calculates the pigment amount distribution of a stain solution for the tissue sample on the basis of the spectral transmittance image estimated by the part 301 and a staining-state analysis part 303 which analyzes the staining state and the structure of the tissue itself on the basis of the pigment-amount distribution calculated by the part 302.

Description

Detailed Description of the Invention

[0001]

The present invention relates to a microscope image processing apparatus, a microscope image processing method, and a microscope image processing program.

[0002]

2. Description of the Related Art For example, there is known a pathological diagnosis support technique for estimating a spectral transmittance image of a tissue sample from a microscope image of the tissue sample taken by a digital camera, and analyzing the stained state of the tissue sample using the estimated image. Has been.

The stained pathological tissue specimen is greatly affected by the stained state. Attempts have been made to quantify and standardize the staining state in the process of preparing a stained sample, but the process is very complicated and is currently difficult. No attempt has been made to quantify or standardize the staining state by image processing.

Among the conventional researches on image processing technology for the purpose of pathological diagnosis support, the most similar research to this image processing technology is "Cell image analysis by independent component analysis, Toshifumi Nakagiri et al., 2000. Proceedings of IEICE General Conference ”. In this research, by performing independent component analysis of the spectral image of a tissue sample, important spectral components are extracted, and the tissues such as nuclei and sinusoids are extracted based on those components.

On the other hand, a technique for estimating an image of spectral information from an image acquired by a digital camera has been actively developed. A typical example is “Natural color
reproduction of human skin for telemedicine, Yuri
OHYA, Takashi OBI, MasahiroYAMAGUCHI, Nagaaki OHYA
MA, and Yasuhiro KOMIYA, Medical Imaging 1998 Pro
c.of SPIE (1998) ”,“ Estimation of transmittance s
pectra from multibandmicrographs of fungi and its
application to segmentation of conidia andhypae, M
asaru Takeya, Norimichi Tsumura, Hideaki Haneishi,
and Yoichi Miyake, Applied Optics Vol.38 No.16 (19
99) ”and so on.

The spectrum estimation method using the statistical information of the spectrum of the subject proposed in the former study is used in the estimation of the spectral transmittance in the present image processing technique. The latter study is a rare example of spectral transmittance image estimation of an optical microscope image. In this study, fungi are identified using estimated spectral transmittance images.

[0007]

In the conventional pathological diagnosis support technique, the image of the signal values of the RGB3 bands obtained from the camera is processed as it is. Therefore, the color information used for image analysis is affected by the observation device (specifically, the difference in light source level of the microscope, the presence / absence of a filter, the spectral sensitivity characteristic of the camera, etc.).
Such an influence becomes a very large noise in analyzing the difference in the characteristics of the pathological tissue specimen, and thus the obtained analysis results depend on the observation device, which makes quantitative analysis difficult. There is a current situation.

In addition, dyes of various manufacturers are used for staining tissue specimens, and various staining processes are used. As a result, the color of the stained pathological tissue specimen is greatly affected. Such effects also make it difficult to analyze and quantify the difference in the properties of the pathological tissue specimen itself.

The present invention has been made in view of the above problems, and an object thereof is to provide a microscope image processing apparatus, a microscope image processing method and a microscope image processing for analyzing a tissue specimen independent of an observation apparatus. To provide a program.

Another object of the present invention is to quantify the dyeing state by estimating the dye concentration contained in the dyeing solution,
A microscope image processing apparatus, a microscope image processing method, and a microscope image processing program for converting and clarifying the structure of the tissue itself.

[0011]

In order to achieve the above object, a first aspect of the present invention is a microscope image processing apparatus, in which a spectral transmittance image is obtained from a signal value of a camera obtained by photographing a tissue sample. A spectral transmittance image estimating unit for estimating, a pigment amount distribution calculating unit for calculating the pigment amount distribution of the staining liquid of the tissue specimen from the spectral transmittance image estimated by the spectral transmittance image estimating unit,
A dyeing state analysis unit that analyzes the dyeing state and the structure of the tissue itself from the dye amount distribution calculated by the dye amount distribution calculation unit.

A second aspect of the present invention is the microscope image processing apparatus according to the first aspect, wherein the staining state analysis section has means for calculating a parameter indicating the staining state.

A third aspect of the present invention is the microscope image processing apparatus according to the first aspect, wherein the staining state analysis section has means for calculating a nuclear occupancy distribution image.

In a fourth aspect of the invention, in the microscope image processing apparatus according to the first aspect, the dyeing state analysis section has means for estimating the spectral transmittance image after the dyeing state conversion.

A fifth aspect of the present invention is a microscope image processing method, which comprises a spectral transmittance image estimating step of estimating a spectral transmittance image from a signal value of a camera obtained by photographing a tissue sample, and the spectral transmittance image estimating step. The dye amount distribution calculating step for calculating the dye amount distribution of the staining liquid of the tissue sample from the spectral transmittance image estimated in the rate image estimating step, and the dyeing state and the tissue from the dye amount distribution calculated in this dye amount distribution calculating step And a staining state analysis step of analyzing the structure of itself.

A sixth aspect of the present invention is a microscope image processing program, which has a spectral transmittance image estimating function for estimating a spectral transmittance image from a signal value of a camera obtained by photographing a tissue sample in a computer. A dye amount distribution calculation function for calculating the dye amount distribution of the staining liquid of the tissue sample from the spectral transmittance image estimated by this spectral transmittance image estimation function, and staining from the dye amount distribution calculated by this dye amount distribution calculation function It realizes a staining state analysis function for analyzing the state and the structure of the tissue itself.

[0017]

BEST MODE FOR CARRYING OUT THE INVENTION First, an outline of an embodiment of the present invention will be described. The present invention takes an approach of using spectral transmittance instead of a signal value of a camera as color information. The spectral transmittance, which is a physical quantity, has a feature that it does not depend on the spectral characteristics of a microscope device or a camera. Furthermore, since it is possible to apply a physical model of light,
It is possible to estimate the dye concentration by using the spectral transmittance image.

The first invention is to photograph an arbitrary tissue sample stained with H & E using a camera and a microscope, estimate a spectral transmittance image from the photographed image, and estimate the spectral transmittance image from the estimated spectral transmittance image. Dye amount distributions of H & E dyes, specifically, hematoxylin and eosin, are obtained. Then, it is an image processing method for calculating a dyeing state parameter, specifically, the concentration of a dyeing reagent, and a quantitative value indicating the goodness of dyeing selectivity by analyzing the dye amount distribution.

A second invention uses the dye amount distribution of the dyes (hematoxylin, eosin) in the first invention,
This is a method of calculating a distribution image of the occupancy ratio of the nucleus in the target stained specimen.

A third aspect of the present invention is a simulation method for converting a target stained sample into a highly selective staining state using the staining state parameter of the first aspect.

The above-mentioned second and third inventions can be used according to the purpose and application.

An embodiment of this system will be described in detail below with reference to the drawings.

(First Embodiment) FIG. 1 shows a system for carrying out the method of microscope image processing of the present invention. As shown in FIG. 1, in the present method, first, the image acquisition apparatus 102 captures an image of a tissue sample 101 that is H & E stained as a subject. Then, the arithmetic processing device 103 performs the processing of this method on the image sent from the image acquisition device 102. Then, the result obtained here is sent to the output device 104, and the output device 104
An image obtained by converting the stained state of the tissue specimen 101 in (1) or a quantitative value indicating the stained state and the nuclear occupancy is presented as the processing result 105.

The image acquisition device 102 comprises a microscope 201 and a digital camera 202 as shown in FIG. 203
Is a tissue sample. FIG. 4 shows an example of the spectral transmittance characteristic of the microscope apparatus 201, and FIG. 5 shows an example of the spectral sensitivity characteristic of the digital camera 202.

As shown in FIG. 3, the arithmetic processing unit 103 includes a spectral transmittance image estimation unit 301 and a dye amount distribution calculation unit 3
No. 02, the staining state analysis unit 303, is divided into three parts. The details of each part will be explained below.

In the spectral transmittance image estimation unit 301, the signal value image 305 of the camera input from the image input device 304 is input.
The spectral transmittance image 310 is estimated by performing a calculation with respect to the first calculation unit 309. At this time, the spectral transmittance characteristic M (λ) 306 (Fig.
(See FIG. 5), the spectral sensitivity characteristic S (λ) 307 of the camera (see FIG. 5), and the statistical information 308 of the spectral transmittance of the H & E stained tissue sample.

The statistical information 308 is specifically the principal component axes up to the top three axes, and its graph is shown in FIG. Figure 6
Indicates the principal component P (λ) of the spectral transmittance of the tissue sample that has been H & E stained.

In the following, the signal value image 30 of the camera is calculated using the equation.
A flow of processing for estimating the spectral transmittance image 310 from 5 will be described.

Hereinafter, the wavelength range λ = 400 to 760 nm is k
Spectral transmittance characteristic of microscope 306
M (λ), spectral sensitivity characteristic of camera 307 S (λ), H
Statistical information on spectral transmittance of tissue specimens stained with & E 3
08 P (λ) is expressed discretely. In addition, since a common calculation formula holds for all pixels on the image, the following formulas (1) to
In (4), the position coordinates of pixels are not shown.

The relationship between the camera signal value image 305 and the spectral transmittance image 310 can be described by the following equation.

G = (MS) ^t T (1) (Note: as shown below, M, g, T are vectors and S is a matrix.) Camera signal value image 305: g = [g ₁ , g ₂ , g ₃ ] ^t spectral transmittance image 310: T = [T ₁ , T ₂ , ..., T _k ] ^t Spectral sensitivity characteristic of camera 307: S = [s ₁ , s ₂ , s ₃ ], s _i = [S _i1 , s _i2 , ..., S _ik ] ^t Microscope spectral transmittance characteristics 306: M = [M ₁ , M ₂ , ...,
M _k ] On the other hand, T can be described by the following equation by using the statistical information 308 of the spectral transmittance of the tissue sample that is H & E stained.

T = Pα (2) (Note: as shown below, α is a vector and P is a matrix.) Principal axis of spectral transmittance of H & E stained tissue specimen: P = [p ₁ , P ₂ , ..., P _k ] ^t , p _i = [p _i1 ,
p _i2 , ..., p _ik ] ^t H & E stained tissue sample spectral transmittance principal component expansion coefficient: α = [α ₁ , α ₂ , ..., α _k ] ^t where H & E stained Since the spectral transmittance of a tissue sample can be approximated to some extent by the main components up to the top three axes,
It approximates using the following formula.

[0033]

[Equation 1]

By substituting the equation (3) into the equation (1), the following equation is obtained.

[0035] At ^{T e = P 'αe = P} ' {(MS) t P '} -1 g (4) (4) equation, g (camera signal value image: 305) from the spectral transmittance image T Get the estimate T ^e .

In the dye amount distribution calculation unit 302, the second calculation unit 311 calculates the dye amount distribution image 312 of the two dyes of hematoxylin and eosin contained in the H & E stain solution from the spectral transmittance image 310. At this time, the absorbance spectrum 313 of the dye prepared in advance is used. FIG. 7 shows the absorbance spectra E (λ) 313 of the hematoxylin dye and the eosin dye.

The flow of processing for estimating the dye amount distribution image 312 from the spectral transmittance image 310 using the formula will be described below.

In the following, the wavelength range λ = 400 to 760 nm is treated by being discretized into k pieces. Further, in the following formula, the position coordinates of the image are represented as (x, y).

The relationship between the spectral transmittance image 310, the dye absorbance spectrum 313, and the dye amount distribution image 312 can be expressed by the following equation.

-Log {T (x, y)} = EC (x, y) (5) (Note: As shown below, T and C are vectors and E is a matrix.) Spectral transmittance image 310: T (x, y) = [T ₁ (x,
y), T ₂ (x, y), ..., T _k (x, y),] Absorbance spectrum 313 of dye: E = [ε _H , ε _E ], ε
_H = [ε _H1 , ε _H2 , ..., ε _Hk ], ε _E = [ε _E1 , ε _E2 ,
..., ε _Ek ] Dye amount distribution image 312: C (x, y) = [C _H (x,
y), C _E (x, y)] (5) is modified to obtain the following formula.

C ^e (x, y) = E ⁺ L (x, y) (6) Here, L (x, y) = − log {T (x, y)} (L
(X, y) = [L ₁ (x, y), L ₂ (x, y), ..., L
_k (x, y)]), and E ⁺ is a pseudo inverse matrix of E.

In the dyeing state analysis unit 303, from the dye amount distribution image 312, the parameter 315 indicating the dyeing state, the nucleus occupancy distribution image 316, and the spectral transmittance converted into a favorable dyeing state are calculated by the third computing unit 314. The image 317 is calculated and output to the output device 320 according to the application.

The parameter 315 indicating the dyeing state is ・ Concentration parameter of each dye molecule in the tissue sample, specifically
Is the concentration of hematoxylin in the nucleus a_nuH, In the cytoplasm
Concentration a_cyE ・ Parameters showing poor selectivity of staining, specifically,
Eosin concentration in the nucleus a _nuE, Cytosolic hematoxylin
Concentration a_cyH Is.

Thereafter, this

[Equation 2] Notated as

Here, assuming that the respective concentrations in the nucleus and in the tissues other than the nucleus are constant, the dye amount distribution image 312 is obtained.
C (x, y) can be decomposed by the following formula.

C (x, y) = r (x, y) A (7) r (x, y) = [r _nu (x, y), r _cy (x, y)] (r _nu : nuclear Occupancy rate in the thickness direction, r _cy : Occupancy rate in the thickness direction of tissues other than the nucleus) (Note: r is a vector.) The calculation procedure is shown below.

1. From all the images, search for (x, y) that maximizes C _H , and at this point r (x, y) =
By _assuming [1,0], a _nuH and a _nuE are calculated.

2. From all the images, search for (x, y) that minimizes C _H , and at this point r (x, y) =
By _assuming [0, 1], a _cyE and a _cyH are calculated.

(Second Embodiment) In the second embodiment,
C (x, y) obtained in the first embodiment and

[Equation 3] From the above, the occupancy distribution image 31 in the thickness direction of the nucleus is calculated using
6

[Equation 4] To get

R ^e (x, y) = C (x, y) A ⁻¹ (8) (Third Embodiment) In the third embodiment, the first embodiment and the second embodiment are described. Got in

[Equation 5] A ′ obtained here and r ^e (x, obtained in (8)
From y), the dye amount distribution C'in a good dyeing state
Get (x, y).

C ′ (x, y) = r ^e (x, y) A ′ (9) By applying the obtained C ′ (x, y) to the equation (5), the equation (10) is obtained, By transforming equation (10), (1
Formula (1) is obtained, and the spectral transmittance image 317 T ″ (x, y) converted into a favorable dyed state is obtained.

−log {T ″ (x, y)} = EC ′ (x, y) (10) T ″ (x, y) = exp {−EC ′ (x, y)} (11) where: In the case of the spectral transmittance image 317 after the dyeing state conversion, the signal value image 319 of the output device 320 is calculated using the characteristic data 318 of the display device and the spectral transmittance characteristic 306 of the microscope.
Need to be converted to.

[0053]

According to the present invention, since the spectral transmittance is used as the color information instead of the signal value of the camera, the analysis of the pathological tissue sample does not depend on the imaging device (the characteristics of the microscope or the camera). Will be possible.

Furthermore, since the dye concentration of the dyeing solution is estimated based on the physical model of light, the dyeing state can be quantified and standardized. This will clarify the nature of the tissue itself (here, calculate the nuclear occupancy distribution), and
It is possible to analyze a pathological tissue specimen that is not affected by the staining state.

[Brief description of drawings]

FIG. 1 is a diagram showing a system for implementing a method of microscopic image processing according to the present invention.

FIG. 2 is an external perspective view of the image acquisition device.

FIG. 3 is a diagram for explaining the flow of processing in the arithmetic processing device shown in FIG.

FIG. 4 is a diagram showing a spectral transmittance characteristic M (λ) 306 of the microscope.

FIG. 5: Spectral sensitivity characteristic S (λ) 307 of digital camera
FIG.

FIG. 6 is a diagram showing the principal component P (λ) of the spectral transmittance of a tissue sample that has been H & E stained.

FIG. 7: Absorption spectrum E of dye molecules in H & E staining solution
It is a figure which shows ((lambda)).

[Explanation of symbols]

101 Tissue Specimen 102 Image Acquisition Device 103 Arithmetic Processing Device 104 Output Device 105 Processing Result 201 Microscope 202 Digital Camera 203 Tissue Specimen 301 Spectral Transmittance Image Estimating Unit 302 Dye Amount Distribution Calculating Unit 303 Staining State Analyzing Unit 304 Image Input Device 305 Camera Signal Value image 306 Spectral transmittance characteristic of microscope 307 Spectral sensitivity characteristic of camera 308 Statistical information 309 H & E stained tissue sample 309 First calculation unit 310 Spectral transmission image 311 Second calculation unit 312 Pigment amount distribution image 313 Hematoxylin, eosin Absorbance coefficient spectrum 314 of dye molecule 313 Third computing unit 315 Parameter 316 indicating staining state Nuclear occupancy distribution image 317 Spectral transmittance image 318 after staining state conversion Characteristic data 319 of display device Output device after staining state conversion Signal value image 320 Output device

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Masahiro Yamaguchi             2-31-19 Shiba, Minato-ku, Tokyo Communications and Broadcasting Organization             Within F term (reference) 2G045 AA24 AA25 BB24 FA16 FA19                       JA01 JA20                 2G059 AA01 BB12 CC20 EE01 EE13                       FF03 HH02 KK04 MM01 MM20                       NN06

Claims (6)

[Claims] 1. A spectral transmittance image estimating unit for estimating a spectral transmittance image from a signal value of a camera obtained by photographing a tissue sample, and a spectral transmittance image estimated by the spectral transmittance image estimating unit. A dye amount distribution calculation unit that calculates the dye amount distribution of the staining liquid of the tissue sample, and a dye state analysis unit that analyzes the dye state and the structure of the tissue itself from the dye amount distribution calculated by this dye amount distribution calculation unit. A microscope image processing apparatus comprising: 2. The microscope image processing apparatus according to claim 1, wherein the dyed state analysis unit has means for calculating a parameter indicating a dyed state. 3. The staining state analysis unit has means for calculating an occupancy distribution image of nuclei.
The microscope image processing apparatus described. 4. The microscope image processing apparatus according to claim 1, wherein the dyed state analysis unit has means for estimating a spectral transmittance image after dyeing state conversion. 5. A spectral transmittance image estimating step of estimating a spectral transmittance image from a signal value of a camera obtained by photographing a tissue sample, and a spectral transmittance image estimated in the spectral transmittance image estimating step. A dye amount distribution calculation step for calculating the dye amount distribution of the staining liquid of the tissue specimen; and a staining condition analysis step for analyzing the staining condition and the structure of the tissue itself from the dye amount distribution calculated in this dye amount distribution calculation step. A method for processing a microscope image, which comprises: 6. A spectral transmittance image estimating function for estimating a spectral transmittance image from a signal value of a camera obtained by photographing a tissue sample in a computer, and spectral transmittance estimated by the spectral transmittance image estimating function. Function to calculate the pigment amount distribution of the staining solution of the tissue sample from the rate image, and the staining state analysis function to analyze the staining state and the structure of the tissue itself from the pigment amount distribution calculated by this pigment amount distribution calculating function And a microscope image processing program to realize.