JP2011095225A - Apparatus and method for processing image, and microscope system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To precisely estimate a staining quantity of a staining sample dyed by a predetermined dyeing pigment. <P>SOLUTION: In one of the embodiments, a storage 16 stores a plurality of reference spectra in different conditions of a staining pigment staining the staining sample of an object to be observed as reference spectrum information 163. A spectrum acquiring part 141 obtains spectral characteristic values each pixel of staining sample images with staining samples picked up. A sample preparing condition estimating part 142 estimates a preparation condition when, for instance, the staining sample is prepared, and obtains the preparation condition of the staining sample. A reference spectrum determining part 146 selects the reference spectrum corresponding to the preparation condition of the staining sample from the reference spectrum information 163 and determines the optimal reference spectrum of the staining pigment. A pigment quantity estimating part 147 estimates the pigment quantity of staining pigment staining the staining sample using the optimal reference spectrum of the staining pigment based on the spectral characteristic values obtained by the spectrum acquiring part 141. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to an image processing apparatus, an image processing method, and a microscope system for processing a stained specimen image obtained by imaging a stained specimen stained with a predetermined staining pigment.

  For example, in pathological diagnosis, a tissue sample obtained by organectomy or needle biopsy is sliced to a thickness of several microns to prepare a sample, and in order to obtain various findings, it is widely observed using a microscope. ing. In particular, transmission observation using an optical microscope is one of the most popular observation methods because the equipment is relatively inexpensive and easy to handle, and has been performed for a long time. Here, the specimen collected from the living body hardly absorbs and scatters light and is almost colorless and transparent. For this reason, it is common to dye with a dye when preparing a specimen.

  Various dyeing methods have been proposed, and the total number thereof reaches 100 or more. Particularly, regarding pathological specimens, blue-violet hematoxylin (hereinafter referred to as “dye H”) and red eosin. Hematoxylin-eosin staining (hereinafter referred to as “H & E staining”) using two dyes (hereinafter referred to as “dye E”) as staining dyes is used as standard. In the specimen after H & E staining (stained specimen), cell nuclei, bone tissue and the like are stained blue-purple, and cytoplasm, connective tissue, erythrocytes and the like are stained red, and these can be easily visually recognized. As a result, the observer can grasp the size and positional relationship of the elements constituting the tissue such as the cell nucleus, and can judge the state of the specimen morphologically.

  By the way, the observation of the stained specimen is performed by visual observation, and is also performed by imaging the stained specimen and displaying it on the display device. Attempts have been made to support diagnosis. As an example, there is a known method for quantitatively estimating the amount of dye of a dye that stains a point (sample point) on a stained specimen based on a stained specimen image obtained by multiband imaging of a stained specimen. It is applied to various purposes. For example, Non-Patent Document 1 discloses a method of estimating a dye amount and correcting color information of a stained specimen image based on the estimated dye amount. Non-Patent Document 2 discloses a technique for quantitatively evaluating the staining state of a specimen based on the estimated amount of dye. Patent Document 1 discloses a method of classifying tissues in a specimen based on an estimated amount of pigment and dividing an image into regions for each tissue.

Here, a method for quantitatively estimating the amount of pigment from a stained specimen image (multiband image) will be described using a stained specimen stained with H & E as an example. Prior to the estimation of the amount of pigment, for example, a background without a specimen is imaged in a state in which illumination light is irradiated to obtain a multiband image of the background (illumination light). First, assuming that the multiband image of the background is I ₀ and the multiband image of the stained specimen to be observed is I, the spectral transmittance t (x, λ) at each pixel position is calculated according to the following equation (1). x is a position vector representing a pixel of the multiband image, and λ is a wavelength. I (x, λ) represents the pixel value at the pixel position (x) at the wavelength λ of the multiband image I, and I ₀ (x, λ) represents the pixel position (x) at the wavelength λ of the multiband image I _0. Represents the pixel value.

Regarding the spectral transmittance t (x, λ), the Lambert-Beer law holds. For example, when the stained specimen is dyed with two types of dyes, dye H and dye E, the following equation (2) is established at each wavelength λ according to Lambert-Beer's law.

In Equation (2), k _H (λ) and k _E (λ) are coefficients specific to the substance determined depending on the wavelength λ, k _H (λ) is a coefficient corresponding to the dye H, k _E (λ ) Is a coefficient corresponding to the dye E. For example, the values of k _H (λ) and k _E (λ) are the dye spectral characteristic values of the dye H and the dye E staining the stained specimen (hereinafter referred to as the dye spectrum of the dye dye staining the stained specimen). The characteristic value is called “reference spectrum”). D _H (x) and d _E (x) correspond to the dye amounts of the dye H and the dye E at each sample point of the stained specimen corresponding to each pixel position (x) of the multiband image. More specifically, d _H (x) is obtained as a relative value with respect to this dye amount when the dye amount of the dye H in the stained specimen stained with only the dye H is “1”. Similarly, d _E (x) is obtained as a relative value with respect to the dye amount when the dye amount of the dye E in the stained specimen stained only with the dye E is “1”. The amount of dye is also called concentration.

Here, the above equation (2) is established independently for each wavelength λ. Equation (2) is a linear form of d _H (x) and d _E (x), and a method for solving this is generally known as multiple regression analysis. For example, if Equation (2) is used simultaneously for two or more different wavelengths, these can be solved.

For example, when equations are combined for M (M ≧ 2) wavelengths λ ₁ , λ ₂ ,..., Λ _M , they can be expressed as the following equation (3). [] ^T represents a transposed matrix, and [] ^-t represents an inverse matrix.

Then, when the above equation (3) is solved using least square estimation, the following equation (4) is obtained, and an estimated value d ^ _H (x) of the dye amount of the dye H and an estimated value d ^ of the dye amount of the dye E: _E (x) is obtained respectively. Here, d ^ indicates that a symbol "^ (hat)" representing an estimated value is attached on d.

  Then, an estimated value of the amount of the dye H and the dye E at an arbitrary sample point on the stained specimen can be obtained by this equation (4).

JP 2005-331394 A

“Color Correction of Pathological Images Based on Dye Amount Quantification”, OPTICAL REVIEW Vol.12, No.4 (2005), p.293-300 “Development of support systems for pathology using spectral transmittance-The quantification method of stain conditions”, Proceedings of SPIE-Image Processing Vol.4684 (2002), p.1516-1523

By the way, the reference spectrum of the dye used for estimating the amount of dye such as k _H (λ) and k _E (λ) described above is, for example, a specimen individually stained with the dye (hereinafter referred to as “single stain”). It is obtained by preparing a sample in advance and measuring its spectral characteristic value (spectrum) using a measuring instrument or the like. However, the spectrum of the staining dye that actually stains the stained specimen may vary depending on the stained specimen to be observed. For this reason, the case where the spectrum of the staining dye that stains the stained specimen to be observed is different from the reference spectrum acquired in advance occurs, and there is a problem that the estimation accuracy of the dye amount is lowered.

  Here, it is known that the spectrum of the staining pigment varies depending on the staining state of the stained specimen (the degree of staining such as dark / light). This variation in the staining state occurs depending on the preparation conditions of the stained specimen, such as the time required for staining (staining time) and the thickness of the specimen, but is always uniform because of the individual differences such as the experience of the clinical laboratory technician who creates the specimen. Difficult to keep on. On the other hand, since a stained specimen is observed and diagnosed by a doctor, there may be a case where a stained state according to the doctor's preference is required. It is also known that the spectrum of a staining dye varies depending on the hydrogen ion index (pH) of a drug (staining solution). The state of the drug such as pH can vary depending on the facility that stains the specimen. Even in the same facility, it may change depending on the time since opening the medicine to be used.

  The present invention has been made in view of the above-described conventional problems, and is an image processing apparatus, an image processing method, and a microscope that can accurately estimate the amount of dye of a stained specimen stained with a predetermined staining dye. The purpose is to provide a system.

  In order to solve the above-described problems and achieve the object, an image processing apparatus according to an aspect of the present invention uses the stained specimen image obtained by imaging a stained specimen of an observation target stained with a predetermined staining pigment. An image processing apparatus for estimating a dye amount of a specimen, a dye spectral characteristic storage unit that stores a plurality of dye spectral characteristic values in different staining states for the dye, and pixels of pixels constituting the stained specimen image A spectral characteristic acquisition unit that acquires a spectral characteristic value at a sample point on the stained specimen corresponding to the pixel, a creation condition acquisition unit that acquires a creation condition of the stained specimen, and the dye spectral characteristic From among the plurality of dye spectral characteristic values stored in the storage means, the dye spectral characteristic value in the staining state corresponding to the preparation condition of the stained specimen is selected, and the optimum dye spectral characteristic value of the stained dye is determined. The dye of the staining dye at the sample point on the stained specimen using the optimum dye spectral characteristic value of the staining dye based on the spectral characteristic value acquired by the dye spectral characteristic determination unit and the spectral characteristic acquisition unit And a pigment amount estimation means for estimating the amount.

  According to the image processing apparatus according to this aspect, a plurality of dye spectral characteristic values in different staining states are stored for the dye that is staining the stained specimen to be observed, and the plurality of dye spectral characteristic values are stored. From the above, it is possible to select the dye spectral characteristic value in the staining state according to the preparation conditions of the stained specimen, and to determine the optimum dye spectral characteristic value of the stained dye. Then, based on the spectral characteristic value acquired for each pixel constituting the stained specimen image, the dye amount at the sample point on the stained specimen can be estimated using the determined optimum dye spectral characteristic value of the stained dye. Therefore, it is possible to accurately estimate the dye amount of the stained specimen to be observed.

  In addition, an image processing method according to another aspect of the present invention includes an observation target that is stained with the staining dye in an image processing apparatus that stores a plurality of dye spectral characteristic values in different staining states for a predetermined staining dye. An image processing method for estimating a dye amount of the stained specimen using a stained specimen image obtained by imaging the stained specimen, and based on a pixel value of a pixel constituting the stained specimen image, the pixel corresponding to the pixel Spectral characteristic acquisition step of acquiring spectral characteristic values at sample points on the stained specimen, creation condition acquisition step of acquiring the preparation conditions of the stained specimen, and preparation conditions of the stained specimen from the plurality of dye spectral characteristic values A dye spectral characteristic value in a dyeing state according to the dye, and determining the optimum dye spectral characteristic value of the dye, and based on the spectral characteristic value acquired in the spectral characteristic acquisition step Characterized in that it comprises a and a dye amount estimation step of estimating a dye amount of the staining dye in the specimen point on the stained sample using the optimum dye spectral characteristic values of the staining dye.

  In addition, the microscope system according to another aspect of the present invention includes an image acquisition unit that acquires a stained specimen image by imaging a stained specimen of an observation object that is stained with a predetermined staining dye using a microscope, and at least the staining dye Spectral characteristic storage means for storing a plurality of dye spectral characteristic values in different staining states and a sample point on the stained specimen corresponding to the pixel based on the pixel value of the pixel constituting the stained specimen image Spectral characteristic acquisition means for acquiring spectral characteristic values in the above; creation condition acquisition means for acquiring preparation conditions for the stained specimen; and the staining among the plurality of dye spectral characteristic values stored in the spectral characteristic storage means A dye spectral characteristic value in a staining state corresponding to a specimen preparation condition is selected, and a dye spectral characteristic determination unit that determines an optimum dye spectral characteristic value of the stained dye and a spectral characteristic acquisition unit A dye amount estimating means for estimating the dye amount of the staining dye at a sample point on the stained specimen using the optimum dye spectral property value of the staining dye based on the acquired spectral characteristic value; Features.

  According to the present invention, it is possible to accurately estimate the amount of dye in a stained specimen stained with a predetermined staining dye.

FIG. 1 is a block diagram illustrating a functional configuration of the image processing apparatus according to the first embodiment. FIG. 2 is a diagram showing a reference spectrum graph of a pair of dye H and dye E having the same creation conditions. FIG. 3 is an explanatory diagram for explaining a method for determining a creation condition determination parameter. FIG. 4 is a diagram showing a partial reference spectrum graph. FIG. 5 is a diagram illustrating an example of the creation condition determination parameter distribution. FIG. 6 is a flowchart illustrating a processing procedure performed by the image processing apparatus according to the first embodiment. FIG. 7 is a flowchart showing a detailed processing procedure of the specimen creation condition estimation process. FIG. 8 is a diagram illustrating an example of the analysis region selection screen. FIG. 9 is a diagram illustrating an example of the analysis region confirmation screen. FIG. 10 is a diagram illustrating an example of an absorbance graph. FIG. 11 is a diagram showing an example of an averaged graph created based on the absorbance graph shown in FIG. FIG. 12 is a diagram showing a second derivative square graph in which a second derivative mean square value is graphed. FIG. 13 is a diagram illustrating an example of a flat wavelength section selection screen. FIG. 14 is a diagram illustrating an example of the creation condition correction screen. FIG. 15 is a flowchart showing a detailed processing procedure of the image display processing in the first embodiment. FIG. 16 is a diagram illustrating an example of an observation screen for a display image in the first embodiment. FIG. 17 is a block diagram illustrating a functional configuration of the image processing apparatus according to the second embodiment. FIG. 18 is a flowchart illustrating a detailed processing procedure of the image display processing according to the second embodiment. FIG. 19 is a diagram showing an example of an observation screen in the second embodiment. FIG. 20 is a schematic diagram illustrating the overall configuration of the microscope system according to the third embodiment. FIG. 21 is a block diagram illustrating a functional configuration of the microscope system according to the third embodiment. FIG. 22 is a diagram for explaining a data configuration example of characteristic data. FIG. 23 is another diagram illustrating a data configuration example of characteristic data. FIG. 24 is a flowchart illustrating a processing procedure performed by the observation system control unit according to the third embodiment. FIG. 25 is a diagram illustrating an example of a stained specimen attribute designation screen according to the third embodiment.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. In the present embodiment, a pathological specimen (biological tissue specimen) that has been subjected to H & E staining is exemplified as a stained specimen to be observed. An image processing apparatus that performs multiband imaging using the stained specimen as a subject and estimates the amount of dye at each point (sample point) of the stained specimen based on the obtained multiband image will be described. Note that the present invention is not limited to the embodiments. Moreover, in description of drawing, the same code | symbol is attached | subjected and shown to the same part.

  Here, in the present embodiment, a stained specimen that has been subjected to H & E staining as described above is an observation target. For this reason, the dyes that stain the stained specimen are the dye H and the dye E, but in the actual stained specimen, in addition to the absorbing components of these stained dyes, for example, erythrocytes that have an absorbing component at the time of no staining Etc. can exist. That is, even if erythrocytes are not stained, they have a unique color and are observed as the color of erythrocytes after H & E staining. Therefore, in the following description, the dyes are described as three types of dye H, dye E, and red blood cell color (hereinafter referred to as “dye R”).

(Embodiment 1)
FIG. 1 is a block diagram illustrating a functional configuration of the image processing apparatus 1 according to the first embodiment. The image processing apparatus 1 according to the first embodiment controls a stained specimen image capturing unit 11 that captures a stained specimen image, an operation unit 12, a display unit 13, an image processing unit 14, a storage unit 16, and each unit of the apparatus. And a control unit 17. Here, the configuration excluding the stained specimen image capturing unit 11 can be realized by using a general-purpose computer such as a workstation or a personal computer.

  The stained specimen image capturing unit 11 includes a multiband camera that captures an observation image of a stained specimen to be imaged, and adjusts the wavelength of light that passes through the tunable filter, the two-dimensional CCD camera, and the tunable filter. Filter controller, camera controller for controlling the two-dimensional CCD camera, and the like. In the first embodiment, a stained specimen to be observed (hereinafter referred to as “observed stained specimen”) is an imaging target.

  Here, the stained specimen image capturing unit 11 is connected to an optical microscope capable of transmitting and observing the stained specimen. This optical microscope includes a light source for emitting illumination light, an objective lens, and a stained specimen on which the stained specimen is placed and moved in the optical axis direction of the objective lens and in a plane perpendicular to the optical axis direction. An illumination optical system for transmitting illumination, an objective lens, an observation optical system for forming an observation image of the stained specimen, etc., and irradiating the stained specimen with illumination light from a light source by the illumination optical system, and an objective lens In cooperation with, the observation optical system forms an observation image of the stained specimen.

  The stained specimen image imaging unit 11 projects an observation image of the stained specimen observed by the optical microscope onto an image sensor of a two-dimensional CCD camera via a tunable filter, and obtains a stained specimen image by performing multiband imaging. The tunable filter is a filter capable of electrically adjusting the wavelength of transmitted light. In the first embodiment, the tunable filter has a wavelength of an arbitrary width of 1 [nm] or more (hereinafter referred to as “selected wavelength width”). Use a band selectable one. For example, commercially available products such as a liquid crystal tunable filter “VariSpec (VariSpec)” manufactured by Cambridge Research and Instrumentation may be used as appropriate. For example, the stained specimen image capturing unit 11 obtains a stained specimen image as a multiband image by capturing an observation image of the stained specimen while sequentially selecting a wavelength band for each predetermined selection wavelength width using the tunable filter.

  The pixel value of the stained specimen image obtained by the stained specimen image imaging unit 11 corresponds to the light intensity in the wavelength band selected by the tunable filter, and the pixel value of the wavelength band selected for each specimen point of the stained specimen is obtained. It is done. Here, each sample point of the stained specimen means each position on the stained specimen corresponding to each pixel of the projected image sensor, and in the following, each specimen point on the stained specimen is each of the stained specimen images. Assume that it corresponds to the pixel position.

  In addition, although the structure using a tunable filter was illustrated as a structure of the stained specimen image imaging part 11, it is not limited to this, If the intensity | strength information of the light in each sample point of the stained specimen made into an imaging target can be acquired Good. For example, the imaging method disclosed in Japanese Patent Laid-Open No. 7-120324 is used, and a stained specimen is subjected to multiband imaging by a frame sequential method while switching a predetermined number (for example, 16) of band-pass filters by rotating them with a filter wheel. It is good also as composition to do.

  The operation unit 12 is realized by, for example, a keyboard, a mouse, a touch panel, various switches, and the like, and outputs an operation signal corresponding to the operation input to the control unit 17. The display unit 13 is realized by a flat panel display such as an LCD or an EL display, or a display device such as a CRT display, and displays various screens according to display signals input from the control unit 17.

  The image processing unit 14 is realized by hardware such as a CPU. The image processing unit 14 includes a spectrum acquisition unit 141 as a spectral characteristic acquisition unit, a sample creation condition estimation processing unit 142 as a creation condition acquisition unit, a dye amount estimation unit 147 as a dye amount estimation unit, and a display image generation A display image generation unit 148 as means.

  The spectrum acquisition unit 141 acquires a spectrum at each pixel position constituting a stained sample image (hereinafter referred to as “observed stained sample image”) obtained by the stained sample image capturing unit 11 performing multiband imaging of the observed stained sample. To do.

  The specimen creation condition estimation processing unit 142 performs a process for estimating the observation stained specimen creation conditions. The sample creation condition estimation processing unit 142 includes an analysis region setting unit 143 as an analysis region setting unit, a feature amount acquisition unit 144 as a feature amount acquisition unit, a creation condition estimation unit 145 as a creation condition estimation unit, and a dye And a reference spectrum determining unit 146 as spectral characteristic determining means.

  The analysis region setting unit 143 sets an analysis region in the observation stained specimen image according to the user operation input from the operation unit 12 in response to the selection input request from the analysis region selection input request unit 171. The feature amount acquisition unit 144 acquires the feature amount of the analysis region set by the analysis region setting unit 143. The creation condition estimation unit 145 estimates the creation condition when the observation stained specimen is created based on the feature amount of the analysis region. The reference spectrum determining unit 146 then selects each dye (dye H, dye E, and dye R) from the reference spectra stored in the reference spectrum information 163 based on the creation conditions estimated by the creation condition estimation unit 145. ) For each of the dyes, the optimum dye spectral characteristic value (hereinafter, the optimum dye spectral characteristic value is referred to as “optimal reference spectrum”) is determined.

  The pigment amount estimation unit 147 uses the optimum reference spectra of the pigment H, the pigment E, and the pigment R determined by the reference spectrum determination unit 146 based on the spectrum acquired by the spectrum acquisition unit 141 for each pixel position of the observation stained specimen. To estimate the amount of pigment in the observed stained specimen. The display image generation unit 148 generates an image (display image) for displaying the observation stained specimen.

  The storage unit 16 is realized by various IC memories such as ROM and RAM such as flash memory that can be updated and stored, information storage media such as a built-in or data communication terminal, a hard disk connected by a data communication terminal, a CD-ROM, and a reading device thereof. It is. In the storage unit 16, a program for operating the image processing apparatus 1 and realizing various functions of the image processing apparatus 1, data used during execution of the program, and the like are temporarily or permanently stored. Is remembered. For example, the storage unit 16 stores an image processing program 161 for estimating the pigment amount at each sample position on the observation stained specimen. The storage unit 16 stores reference spectrum information 163 as a dye spectral characteristic storage unit.

The reference spectrum information 163 stores reference spectrum data of each dye (dye H, dye E, and dye R) acquired in advance. The reference spectrum information 163 stores combinations of reference spectra of the dye H and the dye E in a plurality of different staining states. The reference spectra of the dye H and the dye E for each dye in a plurality of different staining states are obtained from, for example, single dyed specimens of the dye H and the dye E prepared under different preparation conditions. Examples of the preparation conditions include factors that change the staining state, such as the staining time required for staining the observation stained specimen, the thickness of the specimen, the pH of the drug used for staining, and the like. Further, the variance σ ² _base is calculated in advance for each reference spectrum and stored together.

  Here, a method for obtaining the reference spectra of the dye H and the dye E will be described. For example, a plurality of preparation conditions for acquiring a combination of reference spectra in advance (in Embodiment 1, three combinations of a staining time, a pH of a drug to be used, and a specimen thickness) are defined. Then, for each of a plurality of defined creation conditions, a single stained specimen (hereinafter referred to as “H single-stained specimen”) stained only with the dye H according to the creation conditions is created, and according to the creation conditions. A single-stained specimen stained only with dye E (hereinafter referred to as “E-stained specimen”) is prepared.

  Then, the obtained single H-stained specimen for each production condition is subjected to multiband imaging using, for example, the stained specimen image capturing unit 11. Thereafter, each H single stained specimen is sequentially processed, and its spectrum is estimated. For example, the spectral transmittance t (x, λ) at a plurality of sample points on the H single stained sample to be processed is calculated in the same procedure as the processing procedure performed by the spectrum acquisition unit 141 in step a3 in FIG. A calculation is performed based on the multiband image, and the spectral transmittance t (x, λ) is converted into a spectral absorbance a (x, λ). The sample point to be sampled can be set at an arbitrary position on the H single stained sample to be processed, but it is desirable to sample a pixel having a representative color distribution as the dye H. Then, the average of the spectral absorbances a (x, λ) at the obtained plurality of sample points is calculated and used as the reference spectrum of the dye H for the processing conditions for the processed H single stained sample. The same process is performed for the dye E to acquire a reference spectrum for each creation condition. Then, each combination of the reference spectra of the dye H and the dye E having the same creation conditions is set in the reference spectrum information 163 in association with the creation conditions.

  The reference spectrum information 163 stores, for example, the reference spectrum of the dye R acquired as follows regardless of the creation conditions. That is, an unstained specimen that is not stained is prepared and multiband imaging is performed. Then, based on the multiband image, the spectral transmittance t (x, λ) at a plurality of sample points on the unstained sample is calculated, and this is converted into the spectral absorbance a (x, λ). . At this time, an erythrocyte region is selected as a sample point to be sampled. Then, the average of the spectral absorbance a (x, λ) at the obtained plurality of sample points is calculated and used as the reference spectrum of the dye R.

  The method for obtaining the reference spectrum of each staining dye is not limited to this, and each staining dye (dye H in the staining state corresponding to each creation condition defined in advance using a measuring instrument such as a spectroscope is used. In addition, a configuration may be adopted in which the spectra of dye E and dye R) are acquired.

  The storage unit 16 also stores a creation condition determination parameter distribution (not shown) of the creation condition determination parameter Adj used in the process of estimating the creation condition when the creation condition estimation unit 145 creates the observation stained specimen. Here, the creation condition determination parameter Adj is determined for each combination of the reference spectra of the dye H and the dye E having the same creation conditions stored in the reference spectrum information 163 in advance, and the creation condition determination parameter distribution is determined by these. It is created based on the creation condition determination parameter Adj for each creation condition.

  As will be described later, in the first embodiment, the analysis site is the nucleus. Here, the dye H stains a nucleus in particular among tissues in the specimen. For this reason, the spectrum of the nuclear region in the stained specimen is mainly characterized by the reference spectrum of dye H. Therefore, in this example, combinations of reference spectra of the dye H and the dye E having the same creation conditions are sequentially determined. Then, the shape of the reference spectrum graph obtained by graphing the reference spectra of the dye H and the dye E of the combination to be determined is analyzed, and the reference spectrum feature of the dye H with respect to the dye E (hereinafter referred to as “H reference feature”). Based on the extracted H reference features, the creation condition determination parameter Adj (for the creation condition corresponding to the combination to be determined) is determined for the combination to be determined.

  FIG. 2 is a diagram showing a reference spectrum graph of a pair of dye H and dye E having the same creation conditions. This reference spectrum graph is obtained by plotting the reference spectrum value for each wavelength with the horizontal axis as the wavelength and the vertical axis as the absorbance value. In FIG. The reference spectrum graph is indicated by a two-dot chain line.

Here, a method of determining the creation condition determination parameter Adj will be described with reference to FIG. 3 while focusing on the combination of the reference spectra of the dye H and the dye E shown in FIG. First, as shown in FIG. 3, the reference spectrum graphs of the dye E, the reference spectral values to detect a peak wavelength P _E of maximum. Subsequently, than the detected peak wavelength P _E to obtain a wavelength H _S intersecting Toka reference spectrum graph of the reference spectrum graphs and dye E dye H in the long-wavelength side. The wavelength H _S is an approximate expression that approximates a reference spectral value for each wavelength of the dye H (hereinafter referred to as “H spectrum approximate expression”) and an approximate expression that approximates a reference spectral value for each wavelength of the dye E ( (Hereinafter referred to as “E spectrum approximation formula”), and the intersection of these approximation formulas in the wavelength band longer than the peak wavelength P _E is obtained.

4, than the peak wavelength P _E of the reference spectrum graph determining target dye H and the dye E is a diagram showing a partial reference spectrum graph at a predetermined wavelength band of the long wavelength side. In FIG. 4, a graph G1 of the H spectrum approximation formula and a graph G3 of the E spectrum approximation formula are also shown. As shown in FIG. 4, than the peak wavelength P _E of the reference spectrum of the dye E for calculating the intersection point P1 between the graph G1 and the graph G3 in the long wavelength side. Then, the calculated wavelength of the intersection P1 is acquired as the wavelength H _{S at} which the reference spectrum graphs of the dye H and the dye E intersect as shown in FIG.

Subsequently, as shown in FIG. 3, the peak wavelength P _{H at} which the reference spectrum value of the dye H is maximum on the longer wavelength side than the acquired wavelength H _S is detected.

Further, in consideration of an error due to variance _σ ² base_H the reference spectrum of the dye H constituting the combination of decision object, based on the standard deviation sigma _{Base_H,} long wavelength relative to the peak wavelength P _H according to the following equation (5) A dispersion wavelength width W _{base — H} having a predetermined width is set on the side.

Then, as shown in FIG. 3, the first H reference feature R _W is calculated based on the dispersion wavelength width W _{base —} H, the wavelength H _S acquired and detected as described above, and the peak wavelength P _H. . The H reference characteristic R _W a is the wavelength interval is expressed by the following Formula (6).
R _W = (P _H + W _{base —} H) −H _S (6)

Further, as shown in FIG. 3, the wavelength H and the reference spectral values a (H _S) in the _S, 2 nd H wavelengths between change (difference) between the reference spectral values a (P _E) at the peak wavelength P _E Calculated as the reference feature R _ΔP . The H reference feature R _ΔP is expressed by the following equation (7).

Thereafter, according to the following equation (8), the creation condition determination parameter Adj for the combination to be determined is calculated using the values of the H reference features R _W and R _ΔP . k is a coefficient for which an arbitrary value is set.

The coefficient k may be defined based on the H spectrum approximation formula and the E spectrum approximation formula obtained when calculating the wavelength H _S. For example, the change rate η between the wavelengths H _S and the peak wavelength P _E may be calculated based on these approximate equations, and the change rate η may be defined according to the following equation (9).
k = 1-η (9)

  If the creation condition determination parameter Adj is determined for each combination of the dye H and dye E reference spectra for each creation condition stored in the reference spectrum information 163 by the above processing procedure, the creation condition judgment parameter distribution is created. To do. FIG. 5 is a diagram illustrating an example of the creation condition determination parameter distribution. As shown in FIG. 5, the creation condition determination parameter distribution represents the distribution of the creation condition determination parameter Adj in the creation condition space with the preparation time staining time, specimen thickness, and hydrogen ion concentration index (pH) as axes. It is a thing.

  Returning to FIG. 1, the control unit 17 is realized by hardware such as a CPU. The control unit 17 configures the image processing apparatus 1 based on operation signals input from the operation unit 12, image data input from the stained specimen image capturing unit 11, programs and data stored in the storage unit 16, and the like. Instructions to each unit, data transfer, and the like are performed, and the overall operation of the image processing apparatus 1 is comprehensively controlled.

  The control unit 17 includes an analysis region selection input request unit 171, a creation condition input request unit 172, a dye selection input request unit 173 as a dye selection unit, and an image display processing unit 175 as a display processing unit. . The analysis region selection input request unit 171 performs processing for requesting selection input of analysis region candidate regions (analysis region candidates), and is input via the operation unit 12 according to a user operation such as a pathologist or a clinical laboratory technician. Select analysis area candidates. The creation condition input request unit 172 performs a process of requesting correction input of the creation condition estimated by the creation condition estimation unit 145 and receives a user operation via the operation unit 12. The pigment selection input request unit 173 performs processing for requesting selection input of a display target pigment, and receives a user operation via the operation unit 12. For example, the image display processing unit 175 performs processing for displaying a display image of the observation stained specimen on the display unit 13.

  FIG. 6 is a flowchart illustrating a processing procedure performed by the image processing apparatus 1 according to the first embodiment. Note that the processing described here is realized by the operation of each unit of the image processing apparatus 1 according to the image processing program 161 stored in the storage unit 16.

  In the first embodiment, as shown in FIG. 6, first, the control unit 17 controls the operation of the stained specimen image capturing unit 11 and sequentially images the observed stained specimen (step a1). The obtained image data of the stained specimen image of the observation stained specimen is stored in the storage unit 16.

  Subsequently, the spectrum acquisition unit 141 acquires a spectrum at each pixel position of the observation stained specimen image (step a3). For example, the spectrum acquisition unit 141 acquires the spectrum at each pixel position by estimating the spectrum at the sample point on the observation stained specimen corresponding to each pixel constituting the observation stained specimen image.

Here, a detailed spectrum estimation procedure will be described. The spectral transmittance t (x, λ) at the sample point on the stained specimen is an arbitrary pixel represented by the position vector x of the stained specimen image obtained by multiband imaging, as shown in the following formula (1) in the background art. It is obtained by dividing the pixel value I (x, λ) at the position (x) by the pixel value I ₀ (x, λ) at the corresponding pixel position (x) of the multiband image of the background (illumination light).

In practice, the wavelength λ can be observed only discretely. Therefore, when the number of sample points in the wavelength direction is M, the spectral transmittance t (x, λ) is expressed as an M-dimensional vector as shown in the following equation (10). [] ^T represents a transposed matrix.

The obtained spectral transmittance t (x, λ) can be converted into the spectral absorbance a (x, λ) according to the following equation (11). Hereinafter, the spectral absorbance is simply referred to as “absorbance”.
a (x, λ) = − log (t (x, λ)) (11)

  In the first embodiment, the spectrum acquisition unit 141 calculates the spectral transmittance t (x, λ) according to the equation (10), and calculates the spectral transmittance t (x, λ) according to the equation (11) as the absorbance a (x, λ). The process of converting to λ) is performed for all the pixels of the observation stained specimen image, and the absorbance a (x, λ) is acquired as the spectrum of each pixel position (x). The spectrum (absorbance a (x, λ)) data of each pixel position (x) of the acquired observation stained specimen image is the spectral transmittance t (x, λ) of each pixel position (x) calculated in the acquisition process. It is stored in the storage unit 16 together with the data.

  After that, as shown in FIG. 6, the spectrum acquisition unit 141 is an RGB image of the observation stained specimen (hereinafter referred to as “observation stained RGB image”) based on the spectrum of each pixel position of the acquired observation specimen image. Are synthesized (step a5). The image data of the synthesized observation stained RGB image is stored in the storage unit 16, appropriately displayed on the display unit 13, and presented to the user.

Specifically, the spectrum acquisition unit 141 converts the spectral transmittance calculated in the process of acquiring the spectrum for each pixel position of the observation stained specimen image into an RGB value, and synthesizes the observation staining RGB image. If the spectral transmittance at an arbitrary pixel position (x) on the observation stained specimen image is T (x), the RGB value G _RGB (x) is expressed by the following equation (12).
G _RGB (x) = HT (x) (12)

Here, H in Expression (12) is a matrix defined by the following Expression (13). This matrix H is also called a system matrix, F is the spectral transmittance of the tunable filter, S is the spectral sensitivity characteristic of the camera, and E is the spectral radiation characteristic of the illumination.
H = FSE (13)

  Subsequently, as shown in FIG. 6, the process proceeds to the sample creation condition estimation process (step a7). FIG. 7 is a flowchart showing a detailed processing procedure of the specimen creation condition estimation process.

  As shown in FIG. 7, in the sample creation condition estimation process, first, the analysis region selection input request unit 171 receives an analysis region candidate selection operation by the user and selects an analysis region candidate (step b1). For example, the analysis region selection input request unit 171 displays an analysis region selection screen on the display unit 13 and performs a process of notifying the user of an analysis region candidate selection input request. Then, the analysis region selection input request unit 171 notifies the analysis region setting unit 143 of selection information of the analysis region candidate input by the user in response to the notification of the selection input request.

  FIG. 8 is a diagram illustrating an example of the analysis region selection screen. As shown in FIG. 8, the analysis region selection screen includes an observation stained image display unit W11. In this observation stained image display portion W11, the observation stained RGB image synthesized in step a5 in FIG. 6 is displayed. The analysis region selection screen includes an analysis site menu M11, a selection mode menu M13, a manual setting menu M15, and a graph mode menu M17. In addition, an OK button B11 for confirming the operation is arranged on the analysis region selection screen.

  The analysis region menu M11 is for designating a region (tissue) reflected in the region to be selected as an analysis region candidate, and alternatively selects one of “nucleus”, “cytoplasm”, “fiber”, and “others”. Selectable radio buttons are arranged.

  In the selection mode menu M13, radio buttons RB131 and RB133 that can alternatively select “manual” or “tissue” as the analysis region candidate selection mode are arranged. “Manual” is a selection mode in which analysis region candidates are manually selected according to a user operation. For example, a seed region designated by the user on the observation stained image display unit W11 is selected as an analysis region candidate. “Tissue” is a selection mode in which a region of a tissue designated by a user is specified by image processing and selected as an analysis region candidate. In the example of FIG. 8, any one of “nucleus”, “cytoplasm”, and “fiber” can be designated by radio buttons RB135 to RB137.

  In the manual setting menu M15, settings relating to “manual” which is one of the selection modes are performed. For example, in the manual setting menu M15, an input box IB151 for inputting a block size and an input box IB153 for inputting the number of blocks are arranged, and desired values can be set respectively. The block size is the size of the seed region designated on the observation stained image display portion W11. For example, when “2” is input in the input box IB151 as illustrated in FIG. The size is × 2 pixels. In addition, a plurality of seed areas can be designated, and the number corresponds to the number of blocks. The user inputs a desired number of blocks into the input box IB153 (“1” in FIG. 8).

  When the selection mode is “manual”, when a seed region is designated on the observation stained image display unit W11, the spectrum acquired for the corresponding pixel position is displayed in a graph on the graph display unit W13. . In the graph mode menu M17, settings relating to graph display in the graph display section W13 are performed. For example, in the graph mode menu M17, a check box CB171 for displaying the spectrum average graph of the seed region designated on the observation stained image display unit W11 in the graph display unit W13 is arranged. When the check box CB171 is not checked, the spectrum graph of each pixel position constituting the seeds area is displayed in a graph. For example, when the input value of the input box IB151 is “2” in the manual setting menu M15, the seed region is composed of 4 pixels, and the spectrum acquired for each wavelength at the corresponding 4 pixel positions is obtained. The plotted spectrum graph is displayed. On the other hand, when the check box CB171 is checked, a spectrum average graph in which the average value for each wavelength of the spectrum is plotted is displayed together with the spectrum graph of each pixel position constituting the seeds region. For example, in the graph display unit W13 in FIG. 8, a spectrum graph at each pixel position indicated by a broken line and a spectrum average graph indicated by a solid line are displayed.

  In the graph mode menu M17, radio buttons RB171 and RB173 that can alternatively select an absorbance graph or a spectral transmittance graph as the types of spectrum graphs to be displayed on the graph display unit W13 are arranged. If the absorbance is selected by the radio button RB171, the absorbance, which is the spectrum acquired for each pixel position in the seed region, is displayed in a graph. On the other hand, when the spectral transmittance is selected by the radio button RB173, the spectral transmittance calculated in the process of acquiring the absorbance is displayed in a graph.

  For example, as an operation procedure when selecting “manual” as the selection mode and selecting an analysis region candidate, the user first clicks a desired position on the observation stained image display unit W11 with the mouse constituting the operation unit 12. Specify the seed area. At this time, a marker MK11 indicating a seeds region is displayed at a designated (clicked) position on the observation stained image display portion W11. Further, the spectrum at each pixel position constituting the seeds region is displayed in a graph on the graph display portion W13. The seed region once designated can be moved by, for example, dragging and dropping the marker MK11 on the observation stained image display portion W11. According to this, the user can designate a seed region while confirming a spectrum on the graph display unit W13. When a value of 2 or more is input as the number of blocks, a new seed region is designated by clicking again on another position on the observation stained image display portion W11. When confirming the operation, an OK button B11 is clicked.

  When the operation is confirmed as described above, the analysis region selection input request unit 171 analyzes the designated seed region (the pixel position of the marker MK11 on the observation stained image display unit W11) as the process of step b1 in FIG. Select as region candidate. At this time, the analysis region selection input request unit 171 notifies the selection information of the reference candidate region to the analysis region setting unit 143 of the image processing unit 14. Here, the selection information of the analysis region candidate includes the pixel position of the analysis region candidate.

  In the analysis area selection screen of FIG. 8, when “tissue” is selected as the selection mode in the selection mode menu M13, the user selects one of the radio buttons RB135 to RB137 as an internal process. The process of extracting the pixel position where the tissue designated by the above is shown using the teacher data is performed, and the analysis area candidate is selected. Teacher data is created in advance by measuring representative spectral characteristic patterns and color information of each tissue. Then, the region of the tissue designated by the user is extracted by performing image processing on the observation stained specimen image or the observation stained RGB image.

  When the analysis region candidate is selected, as shown in FIG. 7, the analysis region setting unit 143 subsequently sets the analysis region (step b3). For example, the analysis region setting unit 143 searches for a pixel having a pixel value similar to the analysis region candidate in the observation stained RGB image based on the selection information of the analysis region candidate notified from the analysis region selection input request unit 171. Process and set the analysis area.

  As a specific processing procedure, first, the analysis region setting unit 143 maps each pixel value of the observation stained RGB image to the RB color space. At this time, if the analysis region candidate is composed of a plurality of pixel positions, for example, the average value of the mapping points of each pixel constituting the analysis region candidate (the coordinate value in the RB color space of each pixel of the analysis region candidate) (Average value) is calculated and set as the analysis region candidate representative point.

Subsequently, the analysis region setting unit 143 sequentially sets each pixel other than the analysis region candidate as a processing target, and calculates a distance Dist between the mapping point of the analysis region candidate (or the analysis region candidate representative point) and the mapping point of the processing target pixel. calculate. Then, the obtained distance Dist is obtained as the degree of similarity for the processing target pixel. Here, the mapping point (or analysis area candidate representative point) of the analysis area candidate is S (R, B), and the pixel (x _i , y _i ) (i = 1, 2,..., N) to be processed. If the mapping point (coordinate value in the RG color space) is s (r _i , b _i ), the distance Dist between s (r _i , b _i ) and S (R, B) is expressed by the following equation (14). expressed. n represents the number of pixels other than analysis area candidates to be processed.

Then, the analysis region setting unit 143 performs threshold processing on the similarity of each pixel other than the obtained analysis region candidate, and extracts a pixel having a high similarity (for example, a pixel having a similarity greater than or equal to a predetermined threshold). Then, the analysis region setting unit 143 sets a region constituted by the extracted pixels and the analysis region candidate pixels as the analysis region. The threshold value STh used in the threshold processing is set based on the pixel value of the analysis region candidate in the observation stained RGB image. For example, when the analysis region candidate is composed of a plurality of pixel positions, the variance V (S) of the mapping points (coordinate values in the RB color space) of each pixel is obtained and set according to the following equation (15). k is a coefficient and can be set arbitrarily.

  Note that the method of calculating the similarity is not limited to the above-described method, and can be appropriately selected and used. For example, the similarity of the luminance value, the similarity of the color distribution, or the similarity of the spectrum is calculated. At this time, one similarity may be calculated, or a total similarity may be calculated by combining the plurality of similarities.

  The analysis region set as described above is confirmed and displayed on the display unit 13 and presented to the user. For example, the analysis area selection input request unit 171 performs processing for displaying an analysis area confirmation screen on the display unit 13. At this time, the analysis area selection input request unit 171 performs processing for notifying the user of an analysis area correction input request.

  FIG. 9 is a diagram illustrating an example of the analysis region confirmation screen. As shown in FIG. 9, the analysis region confirmation screen includes an observation stained image display unit W21 and an analysis region display unit W23. In the observation stained image display portion W21, an observation stained RGB image is displayed. In addition, an analysis region identification image in which the analysis region in the observation stained RGB image is identified and displayed is displayed on the analysis region display unit W23. For example, an image in which the pixel value outside the analysis region is replaced with a predetermined color (for example, white) and the pixels outside the analysis region are not displayed is displayed.

  This analysis area confirmation screen includes a correction mode menu M21, and can be corrected when it is determined that the set analysis area is insufficient or insufficient. In the correction mode menu M21, radio buttons RB211 and RB213 that can alternatively select “add” or “delete” as the correction mode of the analysis area are arranged. In addition, a confirmation button B21 for confirming the operation is arranged on the analysis region confirmation screen.

  For example, when the user determines that an analysis area setting omission has occurred in the analysis area identification image of the analysis area display unit W23, the user selects the radio button RB211 and analyzes, for example, the analysis on the observation stained image display unit W21. Click the position of the pixel you want to add as an area (additional pixel). On the other hand, when it is determined that the analysis region is excessively set by the analysis region identification image of the analysis region display unit W23, the radio button RB213 is selected and, for example, from the analysis region on the observation stained image display unit W21 Click the position of the pixel to be excluded (excluded pixel). To confirm the operation, click the confirm button B21.

  When a correction operation is input in response to the notification of the correction input request for the analysis area as described above (step b5: Yes in FIG. 7), the analysis area selection input request unit 171 sends the correction information to the analysis area. The setting unit 143 is notified. Here, the correction information includes a pixel position designated as an additional pixel, a pixel position designated as an excluded pixel, and the like. The analysis area setting unit 143 corrects the analysis area according to the correction information (step b7).

  For example, when an additional pixel is set in the correction information, the analysis region setting unit 143 is a pixel that is similar to the additional pixel and is linked to the additional pixel based on the pixel value of the observation stained RGB image. Extract pixels. For example, the luminance value is threshold-processed in order from the adjacent pixel of the additional pixel. The threshold is set based on the luminance value of the additional pixel, for example. Then, a pixel having a luminance value similar to that of the additional pixel is extracted as long as it is connected to the additional pixel. The analysis region setting unit 143 adds the pixels extracted in this way to the analysis region. On the other hand, when an excluded pixel is set in the correction information, the analysis region setting unit 143 is a pixel similar to the excluded pixel and connected to the excluded pixel based on the pixel value of the observation stained RGB image. Extract pixels. For example, the luminance value is thresholded in order from the adjacent pixel of the excluded pixel. For example, the threshold is set based on the luminance value of the excluded pixel. Then, pixels that have similar luminance values to the excluded pixels are extracted as long as they are connected to the excluded pixels. The analysis region setting unit 143 excludes the pixels extracted in this way from the analysis region.

  Finally, the analysis region setting unit 143 assigns an analysis region label corresponding to the analysis region to the pixel position of each pixel constituting the analysis region set and corrected as described above. That is, when the site specified in the analysis site menu M11 in FIG. 8 is “nucleus”, the analysis region label Obj_N is assigned to the pixel position of the analysis region. Similarly, if the designated site is “cytoplasm”, the analysis region label Obj_C is assigned, and if the designated site is “fiber”, the analysis region label Obj_F is given. If “other”, the analysis region label Obj_O is assigned.

  Here, the analysis region is preferably a representative tissue region such as the nucleus, cytoplasm, or fiber region included in the observation stained specimen. That is, it is desirable that the seed region specified by the user on the analysis region selection screen in FIG. 8 is this representative tissue region. Hereinafter, in the first embodiment, it is assumed that the user designates a region in which a nucleus appears on the observation stained image display unit W11 in FIG. 8 as a seed region, and designates the nucleus with the radio button RB111 in the analysis site menu M11. To do.

  Subsequently, the feature amount acquisition unit 144 acquires the feature amount of the set analysis region (the pixel position to which the analysis region label is assigned in the observation stained specimen image). For example, the feature amount acquisition unit 144 graphs the spectrum (absorbance) for each wavelength acquired by the spectrum acquisition unit 141 for each pixel constituting the analysis region, and analyzes the shape of the graphed absorbance graph to acquire the feature amount. To do.

First, as shown in FIG. 7, the feature amount acquisition unit 144 creates an absorbance graph based on the spectrum acquired for the pixels in the analysis region (step b9). Here, the pixel in the analysis region is i (i = 1, 2, 3,..., N), and the number of spectral wavelengths is D. If the spectrum of pixel i at λ _d is a _i (λ _d ) (d = 1, 2, 3,..., D), the absorbance vector A (λ) is expressed by the following equation (16). The

The average absorbance vector A ベ ク ト ル (λ) is expressed by the following equation (17). A￣ indicates that a symbol “￣” representing an average value is attached on A.

In step b9 in FIG. 7, the feature amount acquisition unit 144 graphs, for example, the average absorbance vector A ベ ク ト ル (λ), and creates an absorbance graph. At this time, the feature amount acquisition unit 144 calculates the variance σ ² _{Obj_N} . FIG. 10 is a diagram illustrating an example of an absorbance graph. As shown in FIG. 10, the feature quantity acquisition unit 144 plots the calculated average absorbance vector A 吸 光 (λ) for each wavelength, with the horizontal axis being the wavelength (number of wavelengths) and the vertical axis being the absorbance value. An absorbance graph is obtained.

  Subsequently, as illustrated in FIG. 7, the feature amount acquisition unit 144 performs shape analysis of the absorbance graph and acquires the feature amount (step b11). Here, as described above, in the first embodiment, the nucleus stained with the dye H is used as the analysis site, and the spectrum of the nucleus region is mainly characterized by the reference spectrum of the dye H. Specifically, the spectrum in the analysis region has a characteristic that the absorbance value decreases on the long wavelength side under the influence of the reference spectrum of the dye H (for example, illustrated in FIG. 10). Further, it can be assumed that the amount of decrease in the absorbance value at this time correlates with the stained state of the observation stained specimen.

  Therefore, in the first embodiment, first, the feature quantity acquisition unit 144 has a small change in absorbance between wavelengths in the long wavelength band based on the absorbance graph created based on the spectrum of the analysis region. As illustrated, a wavelength section (hereinafter referred to as “flat wavelength section”) Z in which the absorbance graph is flat is calculated as one of the feature amounts. Further, the feature quantity acquisition unit 144 calculates a peak change ΔP as a feature quantity different from the flat wavelength section Z. For example, the feature amount acquisition unit 144 sets the difference between the absorbance value at the peak wavelength P and the average value of the absorbance values in the flat wavelength section Z as the peak change degree ΔP.

  As a specific calculation procedure of the flat wavelength section Z, first, the wavelength at which the absorbance value is maximum is detected as the peak wavelength P from the absorbance graph. Note that a wavelength band in which the peak wavelength P appears may be learned in advance for each tissue such as a nucleus, cytoplasm, or fiber that is assumed as an analysis site. Then, the peak wavelength P may be detected with reference to the absorbance value of the wavelength band learned for the designated analysis site. In this way, it is not necessary to refer to the entire absorbance graph in order to detect the peak wavelength P.

  Subsequently, in the wavelength band (PD wavelength band) from the detected peak wavelength P to the wavelength D, which is the longest wavelength, an average graph is created by averaging the absorbance between two consecutive wavelengths. FIG. 11 is a diagram showing an example of an averaged graph created based on the absorbance graph shown in FIG. In FIG. 11, the absorbance graph in the PD wavelength band is shown by a broken line together with the averaging graph shown by a solid line.

Subsequently, in order to emphasize the wavelength change, the second derivative L (i) of the absorbance is calculated according to the following equation (18), and the average L 波長 (i) between the wavelengths of the second derivative L (i) (hereinafter, “ Is calculated according to the following equation (19). Note that L￣ indicates that the symbol “￣” is attached on L.

Furthermore, in order to exclude the influence of the change direction, the square value of the second derivative average L 平均 (i) is calculated. Here, the average absorbance vector A￣ (λ) includes an error due to the variance σ ² _{Obj_N} . Therefore, in consideration of this error, the target wavelength to be targeted when the flat wavelength section Z is calculated is limited. For example, the dispersion wavelength width W _{Obj_N} is calculated according to the following equation (20) based on the standard deviation σ _{Obj_N} of the dispersion σ ² _{Obj_N} calculated for the average absorbance vector A 次 (λ). step represents the wavelength interval of the observation stained specimen image (the selected wavelength width of the tunable filter when the stained specimen image capturing unit 11 captures the observation stained specimen image).

Then, the target wavelength is the wavelength P + W _{Obj_N} considering the dispersion wavelength width W _{Obj_N} calculated for the peak wavelength P, and the square value L￣ (i) ² (i = P + W _{Obj_N} ,) of the second-order differential average L￣ (i). P + W _{Obj_N} +2, P + W _{Obj_N} +3,..., P + W _{Obj_N} + D) is calculated.

FIG. 12 shows a second derivative square graph obtained by graphing the second derivative mean square value Lｉ (i) ² . The feature quantity acquisition unit 144 sequentially refers to the value of the second derivative mean square value L￣ (i) ² on the longer wavelength side than P + W _{Obj_N} based on this second derivative square graph, and _{firstly sets a} predetermined threshold value. The following wavelength is detected as S. In addition, the wavelength that is equal to or smaller than a predetermined threshold is detected as E. For example, the wavelength S and the wavelength E are detected using a threshold value indicated by a one-dot chain line in FIG. Based on the detected wavelength S and wavelength E, the wavelength section from the wavelength S to the wavelength E-1 is defined as a flat wavelength section Z.

  Note that the method of calculating the flat wavelength section Z is not limited to this. For example, it is good also as a structure which selects according to user operation. FIG. 13 is a diagram illustrating an example of a flat wavelength section selection screen. As shown in FIG. 13, the flat wavelength section selection screen includes an image display unit W31 on which an observation staining RGB image or an analysis region image is displayed. Which image is displayed can be selected from the image display menu M31. That is, when “HE” is selected in the image display menu M31, an observation stained RGB image is displayed on the image display unit W31. On the other hand, when “analysis region” is selected, an analysis region image in which the analysis region in the observation stained RGB image is identified and displayed is displayed on the image display unit W31. For example, an image in which each pixel outside the analysis region is not displayed by replacing a pixel value outside the analysis region with a predetermined color (for example, white) is displayed.

  The flat wavelength section selection screen includes a graph display unit W33, and displays the absorbance graph created in step b9 in FIG. The display wavelength of the absorbance graph displayed on the graph display unit W33 can be selected as appropriate from the display wavelength menu M33. For example, in the display wavelength menu M33, the user inputs a value on the short wavelength side of the wavelength range to be displayed in the input box IB331 and inputs a value on the long wavelength side in the input box IB333. In addition, it is good also as a structure which displays not only an absorbance graph but the averaged graph (refer FIG. 11) created based on the absorbance graph on the graph display part W33.

  Then, the user inputs a desired wavelength S into the input box IB351 of the section selection menu M35 and inputs a desired wavelength E-1 into the input box IB353 while viewing the absorbance graph displayed on the graph display unit W33. Thereby, the wavelength section from the wavelength S of the input box IB351 to the wavelength E-1 of the input box IB353 is selected as the flat wavelength section Z. Further, when a value is input to the input boxes IB351 and IB353 in this way, the currently selected flat wavelength section Z is indicated by a broken line as illustrated in FIG. 13 on the graph display unit W33, and is displayed on the absorbance graph. The identification is displayed.

Further, as a specific procedure for calculating the peak change degree ΔP, first, the absorbance value average a￣ _flat in the flat wavelength section Z is calculated. Here, a￣ indicates that the symbol “付 い” is attached on a. The absorbance value average a- _flat, each wavelength absorbance value of a (lambda _i) in the flat wavelength interval Z (i = S, S + 1, ···, E) and when, is expressed by the following equation (21) The “step” represents the wavelength interval of the observation stained specimen image.

Then, based on the calculated absorbance value average a に 従 っ て_flat in the flat wavelength section Z, the peak change degree ΔP is calculated according to the following equation (22). a (P) is an absorbance value at the peak wavelength P.

  Each feature amount of the flat wavelength section Z and the peak change degree ΔP calculated by the feature amount acquisition unit 144 as described above is stored in the storage unit 16.

  Then, if the flat wavelength section Z and the peak change degree ΔP are calculated as the feature quantities of the analysis region, then the creation condition estimation unit 145 performs observation staining based on the flat wavelength section Z and the peak change degree ΔP. Estimate sample preparation conditions. In the first embodiment, as preparation conditions, the staining time required for staining the observation stained specimen, the pH of the drug used for staining, and the specimen thickness are estimated. In the creation condition estimation performed here, the creation condition determination parameter distribution of the creation condition determination parameter Adj determined in advance and stored in the storage unit 16 as described above is used.

As described above, in the first embodiment, the analysis site is a nucleus, and the spectrum of the nucleus region is mainly characterized by the reference spectrum of the dye H. Therefore, flat wavelength interval Z and peak change degree [Delta] P by the feature amount acquisition unit 144 has acquired as a feature quantity, it can be assumed to correlate with H reference feature R _W and R _{[Delta] P} in accordance with the staining state of the observation stained specimen. Here, as shown in the equation (8) and described above, the creation condition determination parameter Adj determined in advance is obtained by dividing the H reference feature R _ΔP by the H reference feature R _W. Therefore, as shown in FIG. 7, the creation condition estimation unit 145 first calculates the creation condition determination parameter Adj _A for the analysis region according to the following equation (23) as the creation condition parameter calculation means (step b13).

Then, the creation condition estimation unit 145 estimates the creation condition of the observation stained specimen by selecting the creation condition determination parameter Adj corresponding to the calculated creation condition determination parameter Adj _A of the analysis region from the creation condition determination parameter distribution (step) b15). If there is a creation condition determination parameter Adj that matches the creation condition determination parameter Adj _{A of the} analysis region, the creation condition determination parameter Adj is selected, and the creation condition is estimated as the creation condition of the observation stained specimen. If there is no matching creation condition determination parameter Adj, a creation condition determination parameter Adj whose value is similar to the creation condition determination parameter Adj _{A of} the analysis region is selected, and the creation condition is estimated as the creation condition of the observation stained specimen. Specifically, the creation condition determination parameter Adj that minimizes the absolute value of the difference from the analysis region creation condition determination parameter Adj _A is selected as a similar creation condition determination parameter Adj. Here, there may be a plurality of creation condition determination parameters Adj that minimize the absolute value of the difference. In such a case, the preparation conditions of the observation stained specimen are based on the plurality of creation condition determination parameters Adj. (Estimate multiple preparation conditions of observation stained specimens).

In step b15 described above, the creation condition determination parameter Adj having the smallest absolute value of the difference from the creation condition determination parameter Adj _A of the analysis region is selected as a similar creation condition determination parameter Adj. On the other hand, the absolute value of the difference is subjected to threshold processing using a predetermined threshold value set in advance, and for example, the creation condition determination parameter Adj whose absolute value of the difference is smaller than the threshold is selected as a similar creation condition determination parameter Adj. It is good. If a plurality of creation condition determination parameters Adj are selected at this time, a plurality of preparation conditions for observation stained specimens may be estimated based on each creation condition determination parameter Adj.

Also, since the same creation condition determination parameter Adj value can be defined for different creation conditions, there may be a plurality of creation condition determination parameters Adj that match the creation condition determination parameter Adj _{A of} the analysis region. Even in such a case, the preparation condition of the observation stained specimen is estimated based on the plurality of preparation condition determination parameters Adj.

  The observation staining specimen creation conditions estimated as described above are displayed on the display unit 13 for confirmation and presented to the user. For example, the creation condition input request unit 172 performs processing for displaying the creation condition correction screen on the display unit 13. At this time, the creation condition input request unit 172 performs a process of notifying the user of a creation condition correction input request.

  FIG. 14 is a diagram illustrating an example of the creation condition correction screen. As shown in FIG. 14, the creation condition correction screen includes an observation stained image display unit W41. In this observation stained image display portion W41, an observation stained RGB image is displayed. The creation condition correction screen includes a creation condition display tab TM41 (TM41-1, 2, 3,...). When a plurality of preparation condition determination parameters Adj are selected in step b15 and a plurality of preparation conditions of observation stained specimens are estimated, each preparation condition can be selected and displayed on this preparation condition display tab TM41. Yes. In this creation condition display tab TM41, the estimated staining time, specimen thickness, and pH values are displayed in a modifiable manner. For example, the estimated staining time can be corrected by changing the numerical values of the input boxes IB411 and IB413. Similarly, the estimated sample thickness can be modified by changing the value in the input box IB415, and the estimated pH can be modified by changing the value in the input box IB417. In addition, a registration button B41 for confirming an operation on the creation condition display tab TM41 and registering correction contents and an OK button B43 for finishing correction input are arranged on the creation condition correction screen.

  Then, when a correction operation is input in response to the notification of the correction input request for the creation condition as described above (step b17: Yes in FIG. 7), the creation condition input request unit 172 estimates the correction information as the creation condition estimation. The unit 145 is notified. Here, the correction information includes the corrected staining time, specimen thickness, and pH values. The creation condition estimation unit 145 corrects the creation condition estimated according to the correction information (step b19). The observation staining specimen creation conditions estimated and corrected as described above are stored in the storage unit 16.

Here, when the registration button B41 is clicked on the creation condition correction screen, each value of the creation condition entered in each of the input boxes IB411 and IB413 of the selected creation condition display tab TM41, and the creation condition determination parameter of the analysis region _A combination with Adj _A is added to the creation condition determination parameter distribution as a new creation condition determination parameter Adj. For example, as the process performed by the reference spectrum determining unit 146, the optimum reference spectrum of each dye is determined according to the optimum reference spectrum determining procedure described later, the determined optimum reference spectrum is stored in the storage unit 16, and each value of the creation condition Are added to the creation condition determination parameter distribution stored in the storage unit 16 in association with the optimum reference spectrum that has been determined to be combined with the analysis region creation condition determination parameter Adj _A and updated.

  Subsequently, the reference spectrum determination unit 146 selects the reference spectra of the corresponding dye H, dye E, and dye R from the reference spectrum information 163 according to the preparation conditions of the observation stained specimen estimated and corrected by the preparation condition estimation unit 145 (step) b21) is determined as the optimum reference spectrum used for estimating the pigment amount of the observation stained specimen (step b23).

  That is, the reference spectrum determining unit 146, when the preparation condition of the observation stained specimen is an estimated value by the preparation condition estimation unit 145 (when it is not corrected according to the user operation), the dye H associated with the preparation condition In addition, the reference spectrum of the dye E and the reference spectrum of the dye R are read out from the reference spectrum information 163 and selected as the optimum reference spectrum.

  On the other hand, when the preparation condition of the observation stained specimen is corrected by the user operation, the reference spectrum determination unit 146 selects and corrects the reference spectrum stored in the reference spectrum information 163 according to the corrected preparation condition. Determine the optimal reference spectrum.

Here, the procedure for determining the optimum reference spectrum in this case will be described. First, the reference spectrum deciding unit 146 determines that the distance on the creation condition determination parameter distribution is t based on the dyeing time t, the specimen thickness d, and the hydrogen ion index (pH) p, which are the creation conditions after correction. The minimum creation condition determination parameter Adj is selected from the creation condition determination parameter distribution. Then, the reference spectrum determination unit 146 acquires the staining time for the selected creation condition determination parameter Adj as t ₀ , the specimen thickness as d ₀ , and the hydrogen ion index (pH) as p ₀ .

Subsequently, the reference spectrum determination unit 146 compares the staining times t ₀ and t, the specimen thicknesses d ₀ and d, and the hydrogen ion index (pH) p ₀ and p, respectively, and extracts a creation condition that minimizes the difference. . Then, the reference spectrum determination unit 146 creates a correction matrix according to the created creation conditions. For example, when the difference between the sample thicknesses d ₀ and d is minimum, a correction matrix is created according to the following equation (24).

In other words, the conversion matrix shown in the equation (25) defines the fluctuation of the reference spectrum when the staining time is changed with the specimen thickness d ₀ fixed, for example, the specimen thickness measured in advance. This can be realized by a function that approximates the amount of fluctuation of the reference spectrum of each dye (dye H, dye E, and dye R) according to the change in the dyeing time with respect to d ₀ . Specifically, a plurality of conversion matrixes for sample thicknesses are prepared, and a conversion matrix corresponding to the value of the sample thickness d ₀ is selected and used. Similarly, the conversion matrix shown in the equation (26) defines the fluctuation of the reference spectrum when the pH is changed with the specimen thickness d ₀ fixed, for example, the specimen thickness measured in advance. It can be realized by a function that approximates the variation amount of the reference spectrum of each dye (dye H, dye E, and dye R) according to the change in pH with respect to d ₀ .

  Note that a conversion matrix that defines the fluctuation of the reference spectrum when the thickness and pH of the specimen are changed with the staining time fixed, or the reference spectrum when the staining time and the thickness of the specimen are changed with the pH fixed, respectively. A conversion matrix in which a variation is defined is prepared in the same manner, and is used as appropriate according to a creation condition that minimizes the difference. Note that the conversion matrix is not limited to that illustrated. For example, it may be defined by modeling a change in the reference spectrum of each staining dye accompanying a change in staining time or a change in pH when the thickness of the specimen is fixed. Similarly, changes in the reference spectrum of each staining dye due to changes in specimen thickness or pH when the staining time is fixed, changes in staining time or specimen thickness when pH is fixed The conversion matrix may be defined by modeling the change in the reference spectrum of each staining dye.

Then, using this correction matrix and correcting the reference spectrum of each staining dye associated with the dyeing time t ₀ , the specimen thickness d ₀ and the hydrogen ion index (pH) p ₀ as the creation conditions, the reference spectrum is determined. The unit 146 sets the corrected reference spectrum as the optimum reference spectrum. Then, the process returns to step a7 in FIG. 6, and then proceeds to step a9.

  In step a9, the dye amount estimation unit 147 performs the sample creation condition estimation process in step a7 based on the spectrum (absorbance a (x, λ)) acquired for each pixel position of the observed stained sample image. The amount of dye in the observed stained specimen is estimated using the optimum reference spectrum determined for each dye.

As shown in the background art and shown in equation (2), Lambert-Beer's law holds for the spectral transmittance t (x, λ). Further, the spectral transmittance t (x, λ) can be converted into the absorbance a (x, λ) using the above equation (11). Also in the first embodiment, the amount of pigment is estimated by applying these equations. That is, according to Lambert-Beer's law, the absorbance a (x, λ) at each sample point on the observed stained sample corresponding to each pixel (x, y) of the observed stained sample image is expressed by the following equation (27). expressed. However, at this time, the optimum reference spectrum determined for the dye H is used as the reference spectrum k _H of the dye H, the optimum reference spectrum determined for the dye E is used as the reference spectrum k _E of the dye E, and the reference spectrum k of the dye R is used. _{As R} , the optimum reference spectrum determined for the dye R is used.

  For the dye amounts of the dye H, the dye E, and the dye R at each sample point on the observation stained specimen corresponding to each pixel (x, y), for example, the method described in Expression (3) in the background art is applied, It can be estimated (calculated) by performing multiple regression analysis. The estimated pigment amount data is stored in the storage unit 16.

  When the pigment amount is estimated as described above, the process proceeds to image display processing as shown in FIG. 6 (step a11). In this image display process, an image (display image) for displaying the observation stained specimen is generated based on the pigment amount estimated in step a9 and displayed on the display unit 13. FIG. 15 is a flowchart showing a detailed processing procedure of the image display processing in the first embodiment.

  In the image display processing, first, the image display processing unit 175 performs processing for displaying the RGB image (observation stained RGB image) of the observation stained specimen synthesized in step a5 of FIG. 6 on the display unit 13 as a display image (step c1). ).

  Subsequently, the pigment selection input request unit 173 performs processing for displaying a notification of a selection target pigment selection input request on the display unit 13, and a display target pigment selection operation is input in response to the notification of the selection input request. As long as there is no selection of a display target dye (step c3: No), the process proceeds to step c9.

  Then, when the selection operation of the display target pigment by the user is input (step c3: Yes), the display image generation unit 148 observes staining with the display target pigment based on the observation staining RGB image. A display image that expresses the region (pixel position including the display target dye) on the sample in an identifiable manner is generated (step c5). For example, the display image generation unit 148 includes a display target dye based on the dye amount at each sample point on the observation stained specimen estimated for each pixel of the observation stained specimen image in Step a9 in FIG. A pixel position where the dye amount of the dye is not “0” is selected and set as a display target dye staining region. For example, when the dye H is selected as the display target dye, the dye amount of the dye H is not “0”, and the pixel position including the dye H is selected to set the display target dye staining region. Then, the display image generation unit 148 generates a display image in which the pixels in the display target dye staining region are identifiable from other pixels based on the observation staining RGB image.

  Subsequently, the image display processing unit 175 performs a process for displaying the display image generated in step c5 on the display unit 13 (step c7), and then proceeds to step c9. At this time, the display image generated in step c7 may be displayed instead of the display image already processed, or may be displayed side by side.

  Then, in step c9, it is determined whether or not to end the image display, and while not ending (step c9: No), the process returns to step c3 to accept the display target dye selection operation. For example, when an image display end operation is input by the user, it is determined to end (step c9: Yes). Then, the process returns to step a11 in FIG.

  Here, an operation example when the user observes the display image will be described. FIG. 16 is a diagram illustrating an example of an observation screen for a display image in the first embodiment. The observation screen shown in FIG. 16 includes two image display parts W51 and W53. In addition, the observation screen includes a dye selection menu M51 for selecting a display target dye so that each dye can be individually selected as a display target dye. In FIG. 16, the dye H is selected as the display target dye by the check box CB51.

  Then, for example, an observation stained RGB image is displayed on the image display unit W51 on the left side in FIG. On the other hand, a display target dye identification image in which the display target dye staining region is identified and displayed is displayed on the right image display unit W53. This display target dye identification image is an example of a display image generated by the process of step c5 in FIG. 15, and is generated as an image in which the display target dye staining area is displayed and the other areas are not displayed, for example. The In FIG. 16, a display target dye staining area set for the dye H in the observation dyed RGB image is displayed, and a display target dye identification image in which the other pixels are not displayed is displayed. As internal processing in this case, for example, the display image generation unit 148 sets a display target dye region based on the display target dye selected in the dye selection menu M51, and pixels outside the set display target dye staining region. The display target dye identification image is generated by replacing the value with a predetermined color (for example, white).

  Furthermore, the observation screen includes a drawing menu M53 for designating a drawing mode of the display target pigment identification image displayed on the image display unit W53. For example, in the example of FIG. 16, radio buttons that can alternatively select any one of “none”, “outline”, “color”, and “pattern” are arranged as the drawing mode. As illustrated in FIG. 16, when “None” is selected in the drawing menu M53, the display target dye identification image is displayed as it is on the image display unit W53. When “contour” is selected, a display target dye-stained region for each display target dye is identified and displayed surrounded by a contour line in the display target dye identification image. When “color” is selected, the display target dye-stained area for each display target dye in the display target dye identification image is replaced with a predetermined drawing color for identification display. The drawing color is set in advance for each display target dye. When “pattern” is selected, the display target dye-stained area for each display target dye is identified and displayed in a predetermined fill pattern in the display target dye identification image. The fill pattern is set in advance for each display target dye. For example, when two or more display target dyes are selected in the dye selection menu M51, if “color” or “pattern” is selected in the drawing menu M53, the display target dye staining region of each display target dye can be identified. Can be displayed. In addition, a user setting button B53 is arranged in the drawing menu M53, and the color and fill pattern to be assigned to the display target pigment or the identification display item presented in the drawing menu M53 can be edited by clicking.

  The identification display method is not limited to the above. For example, it is possible to change the hue of each pixel position including the dye amount of the display target dye stepwise according to the value of the dye amount.

  As described above, according to the first embodiment, it is possible to estimate the preparation condition of the observation stained specimen. Then, a reference spectrum corresponding to the estimated creation condition can be selected from the combination of the reference spectra of each staining dye stored in the reference spectrum information 163, and the optimum reference spectrum of each staining dye can be determined. Then, based on the spectrum acquired for each pixel of the observed stained specimen image, the dye amount of the stained dye at the sample point on the observed stained specimen can be estimated using the determined optimum reference spectrum of each stained dye. Therefore, it is possible to accurately estimate the dye amount of the stained specimen to be observed using the optimum reference spectrum of each stained dye according to the preparation condition of the observation stained specimen. Moreover, the user does not need to record the preparation conditions of the observation stained specimen, and can save the user's trouble.

  In addition, the pixel position of the observation stained specimen image including the display target dye selected by the user is selected based on the estimated dye amount of each staining dye, and the region on the observation stained specimen stained with the display target dye (display) It is possible to generate a display image in which the pixel position including the target pigment) can be identified. Therefore, an image representing the inside of the observation stained specimen with high visibility can be presented to the user. According to this, the observation efficiency by the user can be improved. For the user, by selecting a desired staining pigment to be observed, it is possible to observe the region of the desired staining pigment in the observation stained specimen individually or in combination with good visibility.

  In the first embodiment described above, the creation condition estimation unit 145 estimates the creation condition of the observation stained specimen. On the other hand, it is good also as a structure which does not estimate creation conditions. For example, instead of the specimen creation condition estimation process performed in step a7 in FIG. 6, a process for obtaining the creation conditions when creating the observation stained specimen according to the user operation may be performed.

  In this case, the creation condition input request unit 172 performs display processing on the display unit 13 as a creation condition input request unit, for example, a creation condition input screen similar to the creation condition correction screen illustrated in FIG. A process for notifying the user of an input request for the creation condition at the time of creation is performed. Then, the creation condition input request unit 172 acquires the creation condition input by the user in response to the notification of the input request as the creation condition of the observation stained specimen. Subsequently, the reference spectrum determination unit 146 refers to the creation condition determination parameter distribution based on the creation condition of the observation stained specimen acquired according to the user operation by the processing by the creation condition input request unit 172, and the corresponding creation condition determination parameter. Select Adj. Then, the reference spectrum determination unit 146 reads the reference spectrum of the selected creation condition determination parameter Adj from the reference spectrum information 163 and determines the optimum reference spectrum of each dye used for dye amount estimation. At this time, if there is a creation condition that matches the acquired creation condition, the reference spectrum determination unit 146 determines the reference spectrum of each corresponding staining dye as the optimum reference spectrum. On the other hand, if there is no matching creation condition, the reference spectrum determination unit 146 selects and corrects the reference spectrum of each staining dye by the same processing procedure as steps b21 and b23 in FIG. Determine a reference spectrum.

(Embodiment 2)
FIG. 17 is a block diagram illustrating a functional configuration of the image processing apparatus 1b according to the second embodiment. In FIG. 17, the same components as those of the image processing apparatus 1 described in the first embodiment are denoted by the same reference numerals.

  As shown in FIG. 17, the image processing apparatus 1b according to the second embodiment includes a stained specimen image capturing unit 11, an operation unit 12, a display unit 13, an image processing unit 14b, a storage unit 16b, and a control unit 17b. With.

  The image processing unit 14b includes a spectrum acquisition unit 141, a specimen creation condition estimation processing unit 142, a dye amount estimation unit 147, a dye amount correction unit 149b as a dye amount correction unit, and a spectrum synthesis unit as a spectral characteristic synthesis unit. 150b and a display image generation unit 148b as display image generation means. The dye amount correction unit 149b determines the dye H, the dye E, and the dye R estimated by the dye amount estimation unit 147 in accordance with a user operation input from the operation unit 12 in response to the adjustment input request from the dye amount adjustment input request unit 177b. Correct the dye amount. The spectrum synthesizing unit 150b synthesizes the spectral transmittance t (x, λ) based on the dye amounts of the dye H, the dye E, and the dye R that have been corrected by the dye amount correcting unit 149b.

  In addition, the storage unit 16b stores an image processing program 161b for estimating and correcting the dye amount at each sample position on the observation stained specimen, and reference spectrum information 163.

  Then, the control unit 17b includes an analysis region selection input request unit 171, a creation condition input request unit 172, a dye selection input request unit 173 as a dye designation unit, a dye amount adjustment input request unit 177b, and a display processing unit. Image display processing unit 175b. The dye amount adjustment input request unit 177 b performs a process of requesting an adjustment input of the dye amount, and accepts an adjustment operation of the dye amount by the user via the operation unit 12.

  The image processing apparatus 1b according to the second embodiment performs the image display process shown in FIG. 18 in place of the image display process at step a11 in the processing procedure shown in FIG. 6 in the first embodiment. Note that the processing performed by the image processing apparatus 1b is realized by the operation of each unit of the image processing apparatus 1b according to the image processing program 161b stored in the storage unit 16b.

  In the image display processing, first, as shown in FIG. 18, the image display processing unit 175b displays the observation stained RGB image synthesized in step a5 of FIG. 6 described in the first embodiment on the display unit 13 as a display image. (Step d1).

  Subsequently, the pigment selection input request unit 173 performs processing for displaying a notification of a selection input request for a display target pigment on the display unit 13. If a display target dye selection operation is input in response to the notification of the selection input request (step d3: Yes), the dye amount correction unit 149b sets a dye other than the display target dye as a non-display dye. The amount is corrected (step d5). For example, among the pigment amounts estimated for each pixel of the observation stained specimen image in step a9 in FIG. 6 described in the first embodiment, the pigment amounts of the pigments other than the display target pigment are all replaced with “0” for correction. To do.

Subsequently, the spectrum synthesizer 150b synthesizes the spectral transmittance t (x, λ) based on the dye amounts of the dye H, the dye E, and the dye R after correction (step d7). For example, the spectrum synthesizer 150b uses the optimum reference spectrum determined for each staining dye in the sample preparation condition estimation process of FIG. 7 shown in Embodiment 1 according to the following equation (28) to perform the spectral analysis at the pixel position (x). The transmittance t (x, λ) is newly synthesized.

  Subsequently, the display image generation unit 148b converts the spectral transmittance t (x, λ) of each newly synthesized pixel position (x) into an RGB value, and generates a display image by combining the RGB images ( Step d9). The process of converting the spectral transmittance t (x, λ) into RGB values is performed using the equations (12) and (13) described in the first embodiment, as in the procedure of step a5 in FIG. The RGB image synthesized here is an image representing the dyeing state of only the display target dye (only the dye amount of the display target dye is visualized).

  Then, the image display processing unit 175b performs a process of displaying the display image generated in step d9 on the display unit 13 (step d11), and then proceeds to step d23. At this time, the display image generated in step d9 may be subjected to display processing in place of the display image already displayed, or the display image may be displayed side by side.

  In addition, while the display image is displayed as described above, an operation for adjusting the amount of the dye for the display target dye is accepted. For example, the dye amount adjustment input request unit 177b performs processing for displaying a notification of a dye amount adjustment input request on the display unit 13, and when a dye amount adjustment operation is input in response to the notification of the adjustment input request. (Step d13: Yes), the input adjustment amount is notified to the dye amount correction unit 149b.

  Then, the dye amount correction unit 149b corrects the dye amount of the display target dye according to the adjustment amount notified from the dye amount adjustment input request unit 177b (step d15). Thereafter, the spectrum synthesizer 150b newly adds the spectral transmittance t (x) according to the above-described equation (30) in the same processing procedure as step d7 based on the corrected dye amounts of the dye H, the dye E, and the dye R. , Λ) (step d17). Then, the display image generation unit 148b converts the spectral transmittance t (x, λ) of each newly synthesized pixel position into RGB values using the above-described equations (12) and (13) in the same procedure as in step d9. And a display image is generated by synthesizing the RGB image (step d19).

  Subsequently, the image display processing unit 175b performs a process of displaying the display image generated in step d19 on the display unit 13 (step d21), and then proceeds to step d23. At this time, the display image generated in step d19 may be subjected to display processing in place of the display image already displayed, or may be configured to perform display processing side by side.

  Then, in step d23, it is determined whether or not to end the image display. If the display is not completed (step d23: No), the process returns to step d3 to accept a display target dye selection operation. For example, when an end operation of image display is input by the user, it is determined to end (step d23: Yes).

  Here, an operation example when the user observes the display image will be described. FIG. 19 is a diagram illustrating an example of an observation screen for a display image in the second embodiment. The observation screen shown in FIG. 19 includes three image display units W61, W63, and W65. The observation screen also includes a dye selection menu M61 for selecting a display target dye and a drawing menu M63 for designating a drawing mode of a display target dye-stained image displayed on the image display units W63 and W65.

  Then, for example, an observation stained RGB image is displayed on the image display unit W61 on the left side in FIG. On the other hand, display target dye-stained images representing only the dye amount of the display target dye are displayed on the center and right image display portions W63 and W65, respectively. This display target dye-stained image corresponds to the display image generated by the processing of steps d9 and d19 in FIG. 18, and in FIG. 19, an image representing the dyeing state of only the dye H selected as the display target dye is shown. Is displayed.

  Further, the observation screen includes a dye amount adjustment menu M65, and a slider bar SB65 for adjusting the dye amount of the display target dye, an OK button B65 for confirming an operation on the slider bar SB65, and the like are arranged. For example, when observing and diagnosing the display target dye-stained image displayed on the image display unit W63 or the image display unit W65, when the user wants to darken or thin the staining state of the display target dye, the user In the amount adjustment menu M65, the slider bar SB65 is operated to input the amount of adjustment of the amount of pigment to be displayed. In FIG. 19, the display target dye-stained image displayed on the right-side image display unit W65 uses the slider bar SB65, and the dye H is more dye than the display target dye-stained image displayed on the central image display unit W63. It is an image displayed by adjusting the amount thinly.

  As described above, according to the second embodiment, the same effects as those of the first embodiment can be obtained, and the pigment amount of the display target pigment estimated according to the user's adjustment operation can be corrected. A display image can be generated by synthesizing the spectrum of each pixel position based on the dye amount of each dye after correction and synthesizing the RGB image. Alternatively, by correcting the dye amount of a dye other than the display target dye to zero, an image in which the dye amount of only the display target dye is visualized can be generated as a display image. Therefore, it is possible to present to the user an image representing the inside of the observation stained specimen with high visibility by adjusting the staining state with each staining dye. For the user, it is possible to adjust the dye amount by selecting the desired dye and to remove the dye unnecessary for observation / diagnosis, etc., and adjust the dyeing condition of each dye individually for observation with good visibility. . Therefore, it is possible to improve diagnosis accuracy.

(Embodiment 3)
In the third embodiment, the present invention is applied to a microscope system. FIG. 20 is a schematic diagram illustrating the overall configuration of the microscope system 2 according to the third embodiment. FIG. 21 is a block diagram showing a functional configuration of the microscope system 2. The microscope system 2 according to the third embodiment includes an observation unit 3 as an image acquisition unit, an observation system control unit 5, and a characteristic data storage unit 7 as a spectral characteristic storage unit, and data can be exchanged between these units. Connected to and configured.

  As shown in FIG. 21, the observation unit 3 includes a stained sample observation unit 31 for observing an observation stained sample, and a stained sample image imaging unit 33 for capturing an observation stained sample image. Here, the stained specimen image capturing unit 33 is configured similarly to the stained specimen image capturing unit 11 described in the first embodiment.

  The stained specimen observation unit 31 is configured by a microscope capable of transmitting and observing the observation stained specimen. The light source and the objective lens that emit illumination light and the observation stained specimen are placed on the optical axis direction of the objective lens and the optical axis direction. An electric stage that moves in a vertical plane, an illumination optical system for transmitting and illuminating the observation stained specimen on the electric stage, an objective lens, an observation optical system for forming an observation image of the observation stained specimen, and the like are provided. The stained specimen observation unit 31 irradiates the observation stained specimen with illumination light from a light source by the illumination optical system, and forms an observation image of the observation stained specimen by the observation optical system in cooperation with the objective lens.

  The observation system control unit 5 is used by a doctor or the like to observe and diagnose an observation stained sample image captured by the stained sample image capturing unit 33 of the observation unit 3, and is a general-purpose computer such as a workstation or a personal computer. Realized. The observation system control unit 5 gives an operation instruction to the stained specimen observation unit 31 and the stained specimen image imaging unit 33 constituting the observation unit 3, and processes the observation stained specimen image input from the stained specimen image imaging unit 33. Show on the display.

  Specifically, the observation system control unit 5 includes an operation unit 52, a display unit 53, an image processing unit 54, a storage unit 56, and a control unit 57. Here, the microscope system 2 according to the third embodiment is based on the configuration of the image processing apparatus 1 according to the first embodiment. The image processing unit 54 includes a spectrum acquisition unit 541 and a specimen creation condition estimation process. A sample creation condition estimation processing unit 542, an analysis region setting unit 543, a feature quantity acquisition unit 544, a creation condition estimation unit 545, and a coloring matter amount estimation unit 547 and a display image generation unit 548. A reference spectrum determination unit 546.

  The control unit 57 includes an analysis region selection input request unit 571, a creation condition input request unit 572, a dye selection input request unit 573, an image display processing unit 575, a stained specimen attribute input request unit 576, and characteristic data. A selection unit 577 and a system environment setting unit 578 are included.

  The analysis region selection input request unit 571, the creation condition input request unit 572, the dye selection input request unit 573, and the image display processing unit 575 constituting the respective units 541 to 548 of the image processing unit 54 and the control unit 57 are described in the embodiment. The same processing as that of the functional unit having the same name described in 1 is performed. Although the case where the first embodiment is applied is illustrated here, the image processing unit 54 and the control unit 57 are configured based on the modification of the first embodiment and the configuration of the second embodiment. Also good.

  In this control unit 57, the stained specimen attribute input requesting part 576 specifies an attribute value indicating the attribute of the observation stained specimen in accordance with a user operation. Here, it is assumed that the attribute (stained specimen attribute) of the observation stained specimen is composed of four attribute items of the staining type, the organ, the target tissue, and the facility, and the stained specimen attribute input request unit 576 The attribute values of the four attribute items are designated according to the user operation. In the third embodiment, the user specifies the observation magnification of the microscope (stained specimen observation unit 31) when observing the observed stained specimen together with designation of the stained specimen attribute of the observed stained specimen. It has become.

  Further, the characteristic data selection unit 577 selects one or more characteristic data from the characteristic data stored in the characteristic data storage unit 7 based on the stained sample attribute designated by the stained sample attribute input request unit 576. select.

  The system environment setting unit 578 sets system parameters for setting the operating environment (system environment) of the observation unit 3. For example, the system environment setting unit 578 sets an observation parameter for setting the operating environment of the stained specimen observation unit 31 and an imaging parameter for setting the operating environment of the stained specimen image capturing unit 33 as the system parameters.

  In the storage unit 56, an observation system control unit 57 is operated, and a program for realizing various functions provided in the observation system control unit 57, data used during execution of the program, observation staining, and the like. An image processing program 561 for estimating the dye amount at each sample position on the sample is stored.

  The characteristic data storage unit 7 stores characteristic data corresponding to the attribute value of each attribute item of the stained specimen attribute. The characteristic data storage unit 7 is realized by, for example, a database device connected to the observation system control unit 5 via a network, and is installed in a separate place away from the observation system control unit 5 to store characteristic data. Manage characteristic data. The characteristic data may be stored in the storage unit 56 of the observation system control unit 57.

  22 and 23 are diagrams for explaining an example of the data configuration of the characteristic data stored in the characteristic data storage unit 7. Here, FIG. 22 shows a list of characteristic data stored in the characteristic data storage unit 7 in association with the staining type which is one of the attribute items. FIG. 23 shows a list of characteristic data stored in the characteristic data storage unit 7 in association with a facility that is one of the attribute items. The association of the characteristic data for each attribute item shown in FIGS. 22 and 23 is managed using, for example, a known database management tool. However, the data structure of the characteristic data is not limited to this, and any structure may be used as long as the characteristic data corresponding to the attribute value can be acquired by specifying the attribute value of each attribute item.

  Specifically, as shown in FIG. 22, the characteristic data storage unit 7 stores, as characteristic data on the type of staining, for each attribute value, the staining pigment and facility as the attribute item and the observation magnification as the observation parameter. The staining conditions, the specimen thickness and pH, the measurement date, and the spectral characteristic values are stored in association with each other. Spectral characteristic values (data sets A-01 to 03, A-11 to 13, A-21, A-31, etc.) associated with this staining type are spectral characteristic values (measured in advance) with respect to the corresponding staining type. Spectral data) at the corresponding facility (medical facility where the stained specimen (single stained specimen) from which the spectral characteristic value is to be measured), the corresponding observation magnification, staining time, specimen thickness, and pH As a condition, a spectral characteristic value measured on the corresponding measurement date is stored. Here, the spectral characteristic value corresponds to the reference spectrum described in the first embodiment, and is set as, for example, spectral absorbance. However, spectral feature values such as spectral transmittance and spectral reflectance may be used.

  In addition, as shown in FIG. 23, in the characteristic data storage unit 7, as characteristic data about a facility, for each attribute value, a staining type and an organ that are attribute items, an observation magnification that is an observation parameter, and a measurement date And the system spectral characteristic values of the white image signal value, the illumination spectral characteristic value, and the camera spectral characteristic value are stored in association with each other. The system spectral characteristic value associated with this facility stores the spectral characteristic value for the system measured at the corresponding facility. Specifically, the white image signal values (data sets C-01 to C-05,...) Are image signal values obtained by imaging an area where no tissue exists, and correspond with corresponding observation magnifications. The image signal value measured on the measurement date is stored. Further, the illumination light spectral characteristic values (data sets C-11 to C-15,...) Of the stained specimen observation unit 31 measured using a spectrometer or the like at the corresponding observation magnification at the corresponding observation magnification. It is a spectral characteristic value of illumination. Further, the camera spectral characteristic values (data sets C-21 to C-25,...) Are obtained from the camera (two-dimensional CCD camera) of the stained specimen image capturing unit 33 measured at the corresponding observation magnification with the corresponding observation magnification. ) Spectral characteristic value.

  In addition, it corresponds to the case where special imaging is performed with respect to a fiber region or a phase object, for example, spectral characteristic values are measured under different observation parameters such as NA value, defocus amount, and light quantity, and the values correspond to the measurement conditions. In addition, it may be stored in the characteristic data storage unit 7. In addition, depending on the facility, the staining process and reagents used for staining may differ. Corresponding to such a case, the spectral characteristic value may be measured under the condition corresponding to each facility, and the value may be stored in the characteristic data storage unit 7 in association with the measurement condition.

  FIG. 24 is a flowchart illustrating a processing procedure performed by the observation system control unit 5 according to the third embodiment. In FIG. 24, processing steps similar to those in the first embodiment are denoted by the same reference numerals.

  As shown in FIG. 24, first, the stained specimen attribute input request unit 576 performs processing for displaying a stained specimen attribute designation screen on the display unit 53 and notifying a designation for designation of a stained specimen attribute, via the operation unit 52. An operation for designating a stained specimen attribute and an observation magnification by the user is accepted (step e11).

  FIG. 25 is a diagram illustrating an example of a stained specimen attribute designation screen according to the third embodiment. In the stained specimen attribute designation screen shown in FIG. 25, spin boxes B81 to B84 for designating attribute values of four attribute items of a staining type, an organ, a target tissue, and a facility, and a spin box for designating an observation magnification B85, an OK button B81 for confirming the operation in each of these spin boxes, a cancel button B82 for canceling the operation, and the like are arranged. Each of the spin boxes B81 to B84 presents a list of attribute values that can be specified for the corresponding attribute item as an option and accepts a specifying operation. The attribute value of each attribute item stored in the characteristic data storage unit 7 is Presented as an option. For example, in the spin box B81 for designating the staining type, in addition to the HE staining, VB staining, and DAB staining exemplified in FIG. 22 or FIG. 23, the attribute values of the staining types such as orsein staining and MT staining are applicable. Each option of “Others” to be specified when the attribute value is not included is presented. Similarly, each spin box B82 to B84 of the organ, the target tissue, and the facility is presented with the attribute value stored in the characteristic data storage unit 7 and each option of “other”. Further, the spin box B85 presents a value that can be specified as an observation magnification value as an option and prompts selection, and the value of the observation magnification stored in the characteristic data storage unit 7 is presented as an option.

  On this stained specimen attribute designation screen, the user designates the type of staining applied to the observed stained specimen as the staining type. Further, the user designates the organ from which the observed stained specimen is collected as the organ. Further, the user designates a target tissue to be noted when observing and diagnosing the observation stained specimen. In addition, the user designates, for example, a medical facility where the observed stained specimen is collected as a facility. The user also specifies the observation magnification together with these four stained specimen attributes. In addition, a memo column M81 is arranged on the stained specimen attribute designation screen so that the user can freely enter items such as the date when the observed stained specimen was created and the date when the observed stained specimen was observed and diagnosed. It has become.

  Here, on the stained specimen attribute designation screen of FIG. 25, the user selects “HE staining” as the staining type, “kidney” as the organ, “elastic fiber” as the target tissue, “A hospital” as the facility, and “20” as the observation magnification. The process after step e12 in FIG. 24 will be described by taking as an example a case where “double” is designated.

  If the attribute value of the stained specimen attribute is designated, then, as shown in FIG. 24, the characteristic data selection unit 577 refers to the characteristic data storage unit 7 and corresponds to the designated attribute value of the stained specimen attribute. One or more characteristic data are selected (step e12). Specifically, in the case of the exemplified conditions, from the characteristic data on the staining type shown in FIG. 22, the staining type is “HE staining”, the facility is “A hospital”, and the observation magnification is “20 times”. Records R71 to R74 are selected, and corresponding spectral characteristic value data sets A-01, A-02, A-11, and A-12 are acquired. Also, from the characteristic data on the facility shown in FIG. 23, a record in which the facility is “A hospital”, the staining type is “HE staining”, the organ is “kidney”, and the observation magnification is “20 times”. R75 is selected, and the corresponding system spectral characteristic value data sets C-01, C-11, and C-21 are acquired.

  Subsequently, the system environment setting unit 578 sets system parameters (observation parameters and imaging parameters) based on the characteristic data selected in step e12 (step e13). Here, the imaging parameter is a value related to the operation of the multiband camera, and the system environment setting unit 578 notifies the stained specimen image capturing unit 33 of the set imaging parameter value and instructs the stained specimen image capturing unit 33 to operate. I do. In response to the operation instruction from the system environment setting unit 578, the stained specimen image imaging unit 33, for example, according to the notified imaging parameter, sets a gain band, an exposure time, a wavelength band selected by a tunable filter (selected wavelength width) ) To drive the multiband camera.

In the third embodiment, the system environment setting unit 578 sets a selected wavelength band (selected wavelength width) of the tunable filter as one of the imaging parameters. For example, the selected wavelength range in the wavelength band around the wavelength H _S described with reference to FIG. 3 (e.g. before and after 5 [nm]), the 1 [nm] is a wavelength width of selectable minimized by tunable filter Set. Specifically, based on the acquired spectral characteristic value data sets A-01, A-02, A-11, and A-12, the spectral characteristic values of the data sets A-01 and A-11 having the same creation conditions are used. To determine the wavelength H _S. Similarly, the wavelength H _S is obtained by combining the data sets A-02 and A-12 having the same creation conditions. Then, the selected wavelength width in the wavelength band near the wavelength H _S obtained for each combination is set to 1 [nm], for example. On the other hand, the selection wavelength width in a wavelength band other than the wavelength band of 5 [nm] before and after the wavelength H _S is set to an initial value (for example, 5 [nm]). The stained specimen image capturing unit 33 sequentially selects wavelength bands to be selected by the tunable filter according to the selected wavelength width for each wavelength band set here, and captures an observation stained specimen image for each selected wavelength band.

The system environment setting unit 578 sets the exposure time as the second imaging parameter. For example, the system environment setting unit 578 uses the white image signal value data set selected in step e12 (data set C-01 in the exemplified conditions) so that the maximum value of the white image signal value has a predetermined luminance value. The exposure time is adjusted to an exposure time in a wavelength band other than the wavelength band of 5 nm before and after the wavelength H _S. On the other hand, regarding the exposure time in the wavelength band of 5 nm before and after the wavelength H _S , the system environment setting unit 578 first gives various operation instructions to the stained specimen observation unit 31 and the stained specimen image capturing unit 33 of the observation unit 3. The white image signal value is acquired at the designated observation magnification. Then, the system environment setting unit 578 calculates the exposure time for each measurement wavelength using the acquired white image signal value. According to this, it is possible to set the exposure time in the vicinity of the wavelength H _S according to the environment at the time of observation (when the observation stained specimen image is captured).

  Here, the selected wavelength band (selected wavelength width) and exposure time of the tunable filter and the exposure time are set as the imaging parameters. However, other values related to the settings are also appropriately set as necessary. Can be set as

  On the other hand, the observation parameter is a value related to the operation of the microscope, and the system environment setting unit 544 notifies the stained sample observation unit 31 of the set observation parameter value and gives an operation instruction to the stained sample observation unit 31. In response to the operation instruction from the system environment setting unit 544, the stained specimen observation unit 31 switches light source dimming control according to, for example, switching of the observation magnification of the objective lens or the switched observation magnification according to the notified observation parameter, etc. Adjustment of various parts of the microscope accompanying observation of the observation stained specimen is performed, such as switching of various optical elements and movement of the electric stage.

  In the third embodiment, the system environment setting unit 544 sets the observation magnification value designated in response to the notification in step e11 as the observation parameter. Note that the observation parameter to be set is not limited to the observation magnification, and values such as a focus position and a diaphragm can be appropriately set as the observation parameter.

  Thereafter, the system environment setting unit 578 sequentially outputs the exposure time in the corresponding wavelength band together with the selected wavelength width to the stained specimen image capturing unit 33 and outputs the value of the observation magnification set as the observation parameter to the stained specimen observation unit 31. To do. According to this, the optimal operating environment (system environment) of the observation unit 3 can be automatically set according to the observation target specimen. As a result, the observation unit 3 operates according to the system parameters set by the system environment setting unit 578, and acquires an observation stained sample image by performing multiband imaging of the observation stained sample for each selected wavelength width (step e14).

  Subsequently, the spectrum acquisition unit 541 of the image processing unit 54 acquires a spectrum at each pixel position of the observation stained specimen image (step a3). Specifically, the spectrum acquisition unit 541 estimates the spectrum at the sample point on the observation stained specimen corresponding to each pixel constituting the observation stained specimen image in the same manner as in the first embodiment, and calculates the spectrum at each pixel position. get.

  And the spectrum acquisition part 541 synthesize | combines an observation dyeing RGB image based on the spectrum of each pixel position of the acquired observation sample image (step a5). The process of step a5 can be realized in the same manner as step a5 of FIG. 6 described in the first embodiment. However, in the third embodiment, the spectrum acquisition unit 541 uses the characteristic data selected in step e12 when calculating the system matrix H constituting the above equation (12). Specifically, the data set C-21 of the camera spectral characteristic value selected as the spectral sensitivity characteristic S of the camera in the above equation (13) representing the system matrix H is used. Moreover, the data set C-11 of the illumination light spectral characteristic value selected as the spectral radiation characteristic E of illumination is used. According to this, it is possible to synthesize an observation-stained RGB image using each value of the spectral sensitivity characteristic S of the camera and the spectral radiation characteristic E of illumination appropriate for the designated facility.

  Thereafter, the sample creation condition estimation processing unit 542 creates a creation condition determination parameter distribution used in the sample creation condition estimation process based on the characteristic data selected in step e12 (step e6). Specifically, in the case of the exemplified conditions, a creation condition determination parameter distribution is created based on the creation condition determination parameter Adj determined by the spectral characteristic value data sets A-01, A-02, A-11, and A-12. To do.

  Here, the creation condition determination parameter Adj is determined for each combination of the reference spectra of the dye H and the dye E having the same creation condition as described in the first embodiment. In the illustrated conditions, as shown in FIG. 22, the creation conditions of the data set A-01 and the data set A-11 are the same. The creation conditions of the data set A-02 and the data set A-12 are the same. Therefore, by combining these, creation condition determination parameter Adj determined by each spectral characteristic value of data set A-01 and data set A-11, and creation determined by each spectral characteristic value of data set A-02 and data set A-12 A creation condition judgment parameter distribution is created based on the condition judgment parameter Adj.

  The method for determining the creation condition determination parameter Adj and the method for creating the creation condition determination parameter distribution are the same as those in the first embodiment. The creation condition determination parameter Adj may be determined in advance and stored in the characteristic data storage unit 7 or the storage unit 56. In this case, the creation condition determination parameter Adj is determined for each combination of spectral characteristic values of the same condition stored as a data set in the characteristic data storage unit 7 in advance, and in step e6, the selected characteristic data The creation condition determination parameter distribution may be created by distributing the creation condition determination parameter Adj according to the distribution in the creation condition space. The creation condition determination parameter distribution may be created in advance. That is, as described above, all the creation condition determination parameters Adj determined in advance as described above are distributed in the creation condition space as a creation condition determination parameter distribution, and the selected one of the creation condition determination parameter distributions is selected. Only the creation condition determination parameter Adj corresponding to the characteristic data may be referred to.

  When the creation condition determination parameter distribution is created as described above, the process proceeds to the sample creation condition estimation process (step a7), and the same process as that of the first embodiment (see FIG. 7) is performed to optimize each staining dye. Obtain a reference spectrum. However, in the sample creation condition estimation process of the third embodiment, the creation condition determination parameter Adj is selected from the creation condition determination parameter distribution set in step e6 of FIG. 24 as the process of step b15 of FIG.

  Then, based on the spectrum (absorbance a (x, λ)) acquired in step a3 for each pixel position of the observed stained sample image, the dye amount estimation unit 547 performs each staining dye in the sample creation condition estimation process in step a7. The amount of pigment in the observed stained specimen is estimated using the optimum reference spectrum determined for each (step a9). Thereafter, the process proceeds to image display processing (step a11), and processing similar to that in the first embodiment is performed.

  As described above, according to the third embodiment, the characteristic data determined according to the attribute value indicating the attribute of the specimen is stored for each attribute value, and the characteristic according to the attribute value of the observation stained specimen or the like is stored. Data can be selected. The optimum reference spectrum of each staining dye can be determined by selecting the creation condition determination parameter Adj using the selected characteristic data and estimating the creation condition of the observation stained specimen. Therefore, the pigment amount of the observation stained specimen can be estimated with higher accuracy.

  In each of the above-described embodiments, the case where a stained specimen that has been subjected to H & E staining is used as an observation target is described. Since the stained specimen that has been subjected to H & E staining is the target, the dyes H, E, and R The case of estimating the quantity was illustrated. On the other hand, the present invention can be similarly applied to specimens stained with other dyes, and the amount of the dye can be estimated. Further, like the dye R in each of the above-described embodiments, the unique color of the specimen itself can be handled in the same manner.

  As described above, the image processing apparatus, the image processing method, and the microscope system of the present invention are suitable for accurately estimating the dye amount of a stained specimen stained with a predetermined dye.

DESCRIPTION OF SYMBOLS 1,1b Image processing apparatus 11 Stained specimen image imaging part 12,52 Operation part 13,53 Display part 14,14b, 54 Image processing part 141,541 Spectrum acquisition part 142,542 Sample preparation condition estimation processing part 143,543 Analysis area Setting unit 144, 544 Feature amount acquisition unit 145, 545 Creation condition estimation unit 146, 546 Reference spectrum determination unit 147, 547 Dye amount estimation unit 148, 148b, 548 Display image generation unit 149b Dye amount correction unit 150b Spectrum synthesis unit 16, 16b, 56 Storage unit 161, 161b, 561 Image processing program 163 Reference spectrum information 17, 17b Control unit 171, 571 Analysis region selection input request unit 172, 572 Creation condition input request unit 173, 573 Dye selection input request unit 175, 175b 575 Image display processing 177b Dye amount adjustment input request section 576 Stained specimen attribute input request section 577 Characteristic data selection section 578 System environment setting section 2 Microscope system 3 Observation section 31 Stained specimen observation section 33 Stained specimen image imaging section 5 Observation system control section 7 Characteristic data storage Part

Claims (19)

An image processing apparatus that estimates a dye amount of the stained specimen using a stained specimen image obtained by imaging a stained specimen to be observed stained with a predetermined staining dye,
A dye spectral characteristic storage means for storing a plurality of dye spectral characteristic values in different staining states for the dye;
Spectral characteristic acquisition means for acquiring a spectral characteristic value at a sample point on the stained specimen corresponding to the pixel based on the pixel value of the pixel constituting the stained specimen image;
Creation condition acquisition means for acquiring the preparation condition of the stained specimen;
The dye spectral characteristic value in the staining state corresponding to the preparation condition of the stained specimen is selected from the plurality of dye spectral characteristic values stored in the dye spectral characteristic storage means, and the optimum dye spectral characteristic of the stained dye is selected. A dye spectral characteristic determining means for determining a value;
Based on the spectral characteristic value acquired by the spectral characteristic acquisition means, the dye amount estimation means for estimating the dye amount of the staining dye at the sample point on the stained specimen using the optimum dye spectral characteristic value of the staining dye When,
An image processing apparatus comprising:   The creation condition acquisition means includes creation condition estimation means for estimating a creation condition when the stained specimen is created based on the spectral characteristic value acquired by the spectral characteristic acquisition means, and the creation condition estimation means The image processing apparatus according to claim 1, wherein the estimated creation condition is acquired as the creation condition of the stained specimen.   The creation condition acquisition means includes creation condition input request means for requesting input of a creation condition when the stained specimen is created, and the creation condition input in response to the request by the creation condition input request means is the staining The image processing apparatus according to claim 1, wherein the image processing apparatus is acquired as a specimen creation condition. The creation condition estimation means includes
Analysis region setting means for setting a predetermined analysis region in the stained specimen image;
Based on the spectral characteristic value acquired by the spectral characteristic acquisition means for the pixels constituting the analysis area, the characteristic quantity acquisition means for acquiring the characteristic quantity of the analysis area;
Have
The image processing apparatus according to claim 2, wherein a creation condition when the stained specimen is created is estimated based on the feature amount.   The image processing apparatus according to claim 4, wherein the analysis region is a region of a nucleus in the stained sample reflected in the stained sample image. The spectral characteristic acquisition means acquires the spectral characteristic value for each predetermined wavelength or for each predetermined wavelength band,
The feature amount acquisition unit creates a spectral characteristic graph by graphing the spectral characteristic value of each pixel in the analysis region for each predetermined wavelength or each predetermined wavelength band, and analyzes a graph shape of the spectral characteristic graph The image processing apparatus according to claim 4, wherein the feature amount is calculated.   The feature amount acquisition means calculates a difference between a flat wavelength section in which a graph shape of the spectral characteristic graph is flat and / or a spectral characteristic value average in the flat wavelength section and a spectral characteristic value at a peak wavelength of the spectral characteristic graph. The image processing apparatus according to claim 6, wherein the image processing apparatus calculates the amount. The dye spectral characteristic storage means is a single stained specimen individually dyed with the staining dye as a plurality of dye spectral characteristic values in different staining states for the staining dye, and a plurality of single dye samples having different preparation conditions. The dye spectral characteristic value for each preparation condition acquired from one stained specimen is stored,
The dye spectral characteristic determining means selects a corresponding dye spectral characteristic value from the dye spectral characteristic values for each preparation condition stored in the dye spectral characteristic storage means based on the preparation condition of the stained specimen. The image processing apparatus according to claim 1.   The creation condition estimation means refers to a creation condition determination parameter distribution in which creation condition determination parameters for each dye spectral characteristic value for each creation condition are distributed in a predetermined creation condition space defined by two or more creation conditions. 9. The image processing according to claim 8, wherein a creation condition when the staining creation condition is created is estimated by selecting a creation condition judgment parameter corresponding to the creation condition judgment parameter for the analysis region. apparatus.   The dye spectral property determination means matches the preparation condition of the dyed specimen when it matches the preparation condition of any dye spectral characteristic value stored in the dye spectral characteristic storage means. The image processing apparatus according to claim 1, wherein a dye spectral characteristic value corresponding to a creation condition is determined as an optimum dye spectral characteristic value of the staining dye.   The dye spectral characteristic determining means determines that the preparation condition of the stained specimen does not match any of the dye spectral characteristic value preparation conditions stored in the dye spectral characteristic storage means. Select the dye spectral characteristic value that is most similar to the specimen preparation conditions, and correct the selected dye spectral characteristic value according to the difference between the dye spectral characteristic value preparation condition and the dye preparation preparation condition to optimize the dye dye The image processing apparatus according to claim 1, wherein a dye spectral characteristic value is determined. A dye selection means for designating a display target dye;
Based on the dye amount of the stained dye at the sample point on the stained specimen estimated by the dye amount estimating means, the pixel position in the stained specimen image including the display target dye can be distinguished from other pixel positions. Display image generating means for generating the display image represented in
Display processing means for displaying the display image on a display unit;
The image processing apparatus according to claim 1, further comprising: A dye amount correcting means for correcting the dye amount of the stained dye at the sample point on the stained specimen estimated by the dye amount estimating means;
Spectral characteristic synthesis means for synthesizing spectral characteristic values based on the dye amount of the dye dye after correction by the dye amount correction means;
Display image generating means for synthesizing RGB images based on the spectral characteristic values synthesized by the spectral characteristic synthesizing means, and generating a display image representing the dye amount of the stained dye after correcting the staining state with the dye. When,
Display processing means for displaying the display image on a display unit;
The image processing apparatus according to claim 1, further comprising: Provided with a dye selection means for specifying a display target dye,
The image processing apparatus according to claim 13, wherein the dye amount correcting unit corrects the dye amount of the display target dye. Provided with a dye selection means for specifying a display target dye,
The dye amount correction means corrects the dye amount of a staining dye other than the display target dye to zero,
The image processing apparatus according to claim 13, wherein the display image generation unit generates the display image without displaying a staining dye other than the display target dye. In the image processing apparatus storing a plurality of dye spectral characteristic values in different staining states for a predetermined staining dye, the stained specimen using a stained specimen image obtained by imaging a stained specimen of an observation target stained with the staining dye An image processing method for estimating the amount of pigment in
Based on the pixel values of the pixels constituting the stained sample image, a spectral characteristic acquisition step of acquiring a spectral characteristic value at a sample point on the stained sample corresponding to the pixel;
A preparation condition acquisition step of acquiring the preparation conditions of the stained specimen;
A dye spectral characteristic value in a dyed state corresponding to a preparation condition of the stained specimen is selected from the plurality of dye spectral characteristic values, and a dye spectral characteristic determining step for determining an optimum dye spectral characteristic value of the stained dye;
Based on the spectral characteristic value acquired in the spectral characteristic acquisition step, the dye amount estimation step of estimating the dye amount of the staining dye at the sample point on the stained specimen using the optimum dye spectral characteristic value of the staining dye When,
An image processing method comprising: An image acquisition means for acquiring a stained specimen image by imaging a stained specimen of an observation target stained with a predetermined staining pigment using a microscope;
Spectral characteristic storage means for storing a plurality of dye spectral characteristic values in different staining states for at least the staining dye;
Spectral characteristic acquisition means for acquiring a spectral characteristic value at a sample point on the stained specimen corresponding to the pixel based on the pixel value of the pixel constituting the stained specimen image;
Creation condition acquisition means for acquiring the preparation condition of the stained specimen;
Select a dye spectral characteristic value in a staining state according to the preparation condition of the stained specimen from the plurality of dye spectral characteristic values stored in the spectral characteristic storage means, and an optimum dye spectral characteristic value of the stained dye A dye spectral characteristic determining means for determining
Based on the spectral characteristic value acquired by the spectral characteristic acquisition means, the dye amount estimation means for estimating the dye amount of the staining dye at the sample point on the stained specimen using the optimum dye spectral characteristic value of the staining dye When,
A microscope system comprising: In the spectral characteristic storage means, the plurality of dye spectral characteristic values are stored for each attribute value indicating the attribute of the stained specimen,
Based on the plurality of dye spectral characteristic values for the attribute value of the stained specimen to be observed, system parameters for setting an operating environment of the image acquisition unit when imaging the stained specimen to be observed are set. The microscope system according to claim 17, further comprising a system environment setting unit. The spectral characteristic storage means further stores a system spectral characteristic value measured in advance for each attribute value indicating the attribute of the stained specimen,
A system for setting a system parameter for setting an operating environment of the image acquisition means when imaging the stained specimen to be observed based on the system spectral characteristic value of the attribute value of the stained specimen to be observed The microscope system according to claim 17, further comprising an environment setting unit.

Images