JP6059271B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing method.

  Inventions for performing pathological diagnosis by performing image processing have been proposed (see, for example, Patent Documents 1 to 3). However, since pathological diagnosis of the digestive tract such as the esophagus, stomach, small intestine, and large intestine requires specialized knowledge and experience, some tissues of the digestive tract are excised and diagnosed by a microscope outside the body.

  On the other hand, in recent years, endoscopic endoscopies that can predict pathological tissues in the body (in the digestive tract) in real time with a high magnification function of a super-magnification function (380 to 450 times) have been developed, which is an alternative to pathological diagnosis Although it is expected, specialized training is required for image decoding. (Non-Patent Document 1). For this reason, a computer diagnosis support system for endocytoscopy images has been developed by prior research (Non-Patent Document 2), and it has been found that the system is useful for differentiating non-tumors, adenomas, and cancers.

JP 2006-153742 A WO2012 / 011579 WO2012 / 157201 Publication

Kudo S, Wakamura K, Ikehara N, Mori Y, Inoue H, Hamani S. Diagnosis of collective relations with a novel endoscopic classification-a pilot study-. Endoscopy 2011; 43: 869-875 Mori Y, Kudo SE, Wakamura K, “Novel computer-aided diagnostic system for collec- tional registrations by using endocytos (with videos) in est. In Gastro. 2014 Epub

  An object of the present invention is to objectively perform a diagnosis of a lesion called sessile serrated adenoma / polyp (SSA / P) using a computer diagnosis support system. SSA / P has been classified as a non-tumor so far, but is an important lesion that has recently been found to be a potential tumor. In SSA / P, the shape of the cell nucleus is the same as that of a non-tumor, so in the existing computer diagnosis support system (Non-patent Document 2) based only on the shape of the cell nucleus, SSA / P is a non-tumor. / P could not be determined.

An object of the present invention is to make it possible to determine whether a tumor is a non-tumor, an adenoma, cancer, or SSA / P , even if it is not a person with specialized knowledge and experience. It is also intended to obtain a more accurate diagnosis for adenoma / cancer diagnosis.

  The inventors performed classification using the results of the characteristic analysis of all cell nuclei contained in the endoscopic image and the results of texture analysis obtained from the same endoscopic image as parameters, so that non-tumor, It was discovered that the diagnostic accuracy of adenomas and cancers was improved, and that SSA / P could be classified.

Specifically, the image processing apparatus according to the present invention extracts cell nuclei contained in an endoscopic image obtained by enlarging and imaging cells of the mucous membrane of the digestive tract irradiated with light through an endoscope, Using all the extracted cell nuclei, the function of measuring the characteristics of the cell nuclei, the function of performing texture analysis on the overall density of the endoscopic image, and the characteristics of the cell nuclei and the results of the texture analysis, non-tumor And a function of identifying any one of adenoma, cancer and SSA / P and outputting any one of non-tumor, adenoma, cancer and SSA / P as a diagnostic result.

Specifically, the image processing method according to the present invention is included in the endoscopic image obtained by the image processing apparatus by enlarging and imaging the cells of the mucous membrane of the digestive tract irradiated with light through the endoscope. A procedure for extracting cell nuclei, measuring the characteristics of the cell nuclei for all the extracted cell nuclei, a procedure for performing image analysis on the overall density of the endoscopic image, and an image processing device for Using the characteristics of the cell nucleus and the result of the texture analysis, it is identified whether it is non-tumor, adenoma, cancer, or SSA / P, and any of non-tumor, adenoma, cancer, or SSA / P is diagnosed And a procedure for outputting as a result.

  The endoscopic image may be an image obtained by enlarging and imaging a cell using an endoscope equipped with an optical microscope.

  The endoscopic image may be an image obtained by enlarging and imaging a cell using a confocal endoscope.

  The endoscopic image may be an image obtained by enlarging and imaging stained cells.

According to the present invention, it is possible to determine whether the tumor is non-tumor, adenoma, cancer, or SSA / P , even if the person has no specialized knowledge and experience. In addition, the diagnosis of adenoma / cancer can be performed with higher accuracy than existing diagnosis support systems.

2 shows an example of an image processing method according to the present embodiment. An example of the endoscopic image which concerns on this embodiment is shown. An example of the shape of the extracted cell nucleus is shown. An example of the pixel which performs texture analysis is shown. An example of texture analysis is shown. An example of the result of a texture analysis is shown.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In addition, this invention is not limited to embodiment shown below. These embodiments are merely examples, and the present invention can be implemented in various modifications and improvements based on the knowledge of those skilled in the art. In the present specification and drawings, the same reference numerals denote the same components.

  FIG. 1 shows an example of an image processing method according to this embodiment. In the image processing method according to the present embodiment, the image processing apparatus sequentially executes the feature amount calculation procedure S101, the identification procedure S102, and the diagnosis result output procedure S103. The image processing apparatus according to the present embodiment is a computer including a CPU (Central Processing Unit) and a memory. The image processing apparatus according to the present embodiment performs each procedure by executing a computer program stored in a memory. The computer program can be stored in any recording medium that can be read from the computer.

In the feature amount calculation procedure S101, the image processing apparatus executes the cell nucleus measurement procedure and the texture analysis procedure, and calculates the feature of the cell nucleus included in the endoscopic image. In the cell nucleus measurement procedure, cell nuclei included in the endoscopic image are extracted, and the characteristics of the cell nuclei are measured for all the extracted cell nuclei. In the texture analysis procedure, texture analysis is performed on the entire endoscope image.
In the identification procedure S102, the image processing apparatus performs classification corresponding to the pathological diagnosis using the feature value obtained in the feature value calculation procedure S101.
In the diagnostic result output procedure S103, the image processing apparatus outputs the classification result of the identification procedure S102.

  In the present invention, an endoscopic image obtained by enlarging and imaging a cell irradiated with light through an endoscope is used. The endoscopic image obtained by magnifying imaging is an image in which the shape of the cell nucleus can be specified, and for example, an image that is magnifying and imaged using an endocytoscope-type or confocal-type endoscope can be used. In the confocal system, an aperture is arranged at a position (image position) conjugate with the focal position of the objective lens. In the present embodiment, it is preferable to use endocytoscopy in which an optical microscope is mounted on an endoscope, whereby a clear color image can be obtained as compared with a confocal system.

  The endoscopic image is preferably an image obtained by enlarging and imaging the stained cells. The staining solution is preferably one that can acquire a tissue image and has high safety. For example, methylene blue (3,7-bis (dimethylamino) phenothiazinium chloride) or crystal violet ([4- {bis ( 4-dimethylaminophenyl) methylene} -2,5-cyclohexadiene-1-ylidene] dimethylammonium chloride) or combinations thereof. In particular, these double staining allows the cell nucleus to be dyed in blue and the cytoplasm into pink, so that a clear tissue image can be obtained while maintaining safety.

  FIG. 2 shows an example of an endoscopic image according to this embodiment. An endoscopic image includes a plurality of cell nuclei. Since the invention according to this embodiment uses the entire endoscopic image, it is preferable to cut out a region useful for pathological diagnosis.

  Further, the endoscopic image in the present embodiment is an image obtained by enlarging a tissue to be pathologically diagnosed. The magnification of enlargement is an arbitrary magnification capable of measuring the shape of the cell nucleus, for example, 380 times. The tissue of the endoscopic image may be an image of a part of the living body excised or an image of the surface of the living body. The surface of a living body is, for example, an arbitrary part that can be observed with an endoscope. It is.

The cell nucleus measurement procedure in the feature amount calculation procedure S101 will be described.
First, all cell nuclei contained in the endoscopic image are extracted. In the extraction of cell nuclei, the region of the cell nuclei is segmented to remove artifacts. The segmentation of the cell nucleus region uses, for example, Otsu's binarization method with an R component. In the artifact removal, for example, a pixel in which white pixels of a binarized image are continuous is defined as one region, and the area, the major axis, and the roundness are calculated for each region. The area set in the range (for example, 30 μm ² to 500 μm ² ), the major axis set in the value (for example 30 μm or less), and the roundness set in the value (for example 0.3 or more) is left as the analysis target, Remove other areas. The major axis and the roundness are calculated, for example, by approximating the region to an ellipse. When the number of extracted nuclei is equal to or less than a preset number (for example, 30), it is considered that the reliability of the image is low, and an example in which diagnosis is not possible may be used. FIG. 3 shows an example of the extracted cell nucleus.

  Next, the characteristics of all extracted cell nuclei are measured. The characteristics of the cell nucleus are, for example, the cell nucleus major axis DL, the cell nucleus minor axis DS, the cell nucleus perimeter, the cell nucleus area, the cell nucleus roundness, and the cell nucleus color. For the area and roundness, an average value and a standard deviation are preferably used.

The texture analysis procedure in the feature amount calculation procedure S101 will be described.
First, an endoscopic image is divided into predetermined pixels. For example, as shown in FIG. 4, the pixel is divided into pixels P11 to P89.
Then, texture analysis is performed for each pixel. The shade of each pixel changes depending on the arrangement of glandular cavities and nuclei. In particular, the arrangement of glandular cavities and cell nuclei can be quantified by comparing the density of adjacent pixels. Therefore, in the texture analysis, a pattern (= texture) represented by shading change is analyzed. The method of texture analysis is arbitrary, and for example, Local Binary Pattern (LBP) can be used.

  LBP, which is an example of texture analysis, will be described with reference to FIG. The LBP is a method of quantifying the magnitude of light and shade of the pixel PX located at the center in the 3 × 3 pixel region and the adjacent pixels P1 to P8 by a binary system.

  For example, when the pixel PX is the pixel P23 shown in FIG. 4, the pixels P1 to P8 are P12, P13, P14, P24, P34, P33, P32, and P22, respectively. The density of the pixel P23 is compared with the density of the pixel P12, and is set to “1” if the density of the pixel P23 is higher than the density of the pixel P12, and is set to “0” if the density of the pixel P23 is lower than the density of the pixel P12. Whether the pixels P2 to P8 are “1” or “0” is determined similarly to the pixel P1. As a result, binary data such as 00111001, for example, in which the shades of the pixels P1 to P8 are digitized can be obtained. In this case, when “00111001” is converted to decimal, a value of 57 in 256 gradations is obtained.

  FIG. 6 shows an example of the result of texture analysis. Texture analysis is performed on all pixels, and a histogram showing the distribution of pixel values is created.

  The classification in the identification procedure S102 will be described. In the present embodiment, classification corresponding to pathological diagnosis is performed using the feature amount obtained in the feature amount calculation step S101. The classification corresponding to the pathological diagnosis is, for example, non-tumor, adenoma, cancer, SSA / P. For classification, for example, SVM (Support Vector Machine) is used.

  For each data point of SVM, for example, in addition to each feature of all cell nuclei extracted from the endoscopic image, a histogram obtained by texture analysis is used. Each feature of the cell nucleus is a major axis DL, a minor axis DS, a perimeter, an area, a roundness, and a color. The histogram obtained by texture analysis is, for example, all elements calculated by texture analysis. For example, when 256 gradations are used, there are 256 elements.

  Here, at the time of classification, data serving as learning samples is given to the image processing apparatus for each of non-tumor, adenoma, cancer, and SSA / P. The image processing apparatus according to the present embodiment provides a histogram obtained by texture analysis in addition to each feature of all cell nuclei extracted from an endoscopic image as a learning sample, so that non-tumor and SSA / P Can be identified.

  The data points of the SVM are arbitrary, but are, for example, all elements of the average, standard deviation, and coefficient of variation of all feature parameters of the cell nucleus and all 256 elements calculated by texture analysis. As a result, a composite vector having a total of about 300 dimensional elements is generated as a data point from one endoscopic image, and applied to the SVM.

  In the diagnostic result output procedure S103, the image processing apparatus outputs the classification result of the identification procedure S102. For example, the image processing apparatus outputs any of non-tumor, adenoma, cancer, and SSA / P as a diagnostic result. The output is displayed on, for example, a monitor provided in the computer. The classification result is stored in the memory together with the endoscopic image.

  The above-described feature amount calculation procedure S101, identification procedure S102, and diagnosis result output procedure S103 are performed for each endoscope image. Thereby, even if it is not a person with specialized knowledge and experience, it can be judged whether each site | part which wants to perform pathological diagnosis is a non-tumor, an adenoma, cancer, or SSA / P. In particular, by taking an endoscopic image of the mucous membrane in the body such as the digestive tract using an endoscope, real-time automatic diagnosis is possible, and the quality of endoscopy can be improved.

  The invention according to the present embodiment can be applied not only to an endoscopic image but also to a microscope image obtained by imaging a tissue collected from a living body with a microscope.

  In the present embodiment, the case where the features of the cell nucleus are the major axis DL, the minor axis DS, the perimeter, the area, the roundness, and the color will be described, but the present invention is not limited to this. For example, eccentricity, chordal ratio, uneven shape, fractal dimension, line concentration, and density contrast may be included.

  In the present embodiment, an example in which classification is performed using the SVM is shown, but any algorithm other than the SVM can be used. Such algorithms include, for example, neural networks, naive Bayes classifiers, decision trees, cluster analysis, linear regression analysis and logistic regression analysis.

Claims (8)

Function to extract cell nuclei contained in endoscopic images obtained by magnifying cells of gastrointestinal mucosa irradiated with light through an endoscope, and measure the characteristics of the cell nuclei for all extracted cell nuclei When,
A function of performing texture analysis on the overall shade of the endoscopic image;
Using the characteristics of the cell nucleus and the result of the texture analysis, it is identified whether the tumor is a non-tumor, adenoma, cancer or SSA / P (sessile serrated adenoma / polyp), and the non-tumor, adenoma, cancer and SSA / P A function of outputting any of P as a diagnosis result;
An image processing apparatus comprising: The endoscopic image is an image obtained by enlarging and imaging cells using an endoscope equipped with an optical microscope.
The image processing apparatus according to claim 1. The endoscopic image is an image obtained by enlarging and imaging a cell using a confocal endoscope,
The image processing apparatus according to claim 1. The endoscopic image is an image obtained by enlarging and imaging stained cells.
The image processing apparatus according to claim 1. The image processing device extracts the cell nuclei contained in the endoscopic image obtained by enlarging the cells of the digestive tract mucosa irradiated with light through the endoscope, and for all the extracted cell nuclei, A procedure for measuring features;
The image processing apparatus performs a texture analysis for the entire shade of the endoscopic image, and
The image processing apparatus identifies whether the tumor is a non-tumor, adenoma, cancer, or SSA / P (sessellated adenoma / polyp) using the characteristics of the cell nucleus and the result of the texture analysis. A procedure for outputting any of cancer and SSA / P as a diagnostic result;
An image processing method. The endoscopic image is an image obtained by enlarging and imaging cells using an endoscope equipped with an optical microscope.
The image processing method according to claim 5. The endoscopic image is an image obtained by enlarging and imaging a cell using a confocal endoscope,
The image processing method according to claim 5. The endoscopic image is an image obtained by enlarging and imaging stained cells.
The image processing method according to claim 5.

Images