JP4925111B2 - Similar image search device - Google Patents

Description

  The present invention relates to a similar image retrieval apparatus capable of supporting a diagnosis by a doctor by retrieving past similar case images using image processing in the medical field, particularly in gastric biopsy diagnosis.

  Conventionally, the following methods have been proposed as image processing techniques for supporting pathological diagnosis. As a finding in histopathological diagnosis, it is known that the ratio of nucleus to cytoplasm (nucleocytoplasmic ratio) in cancer cells significantly increases as compared with normal cells. If such basic and important knowledge can be incorporated into the histopathological diagnosis, it is considered that the diagnostic accuracy is greatly improved.

Therefore, the conventional method shown in Patent Document 1 below uses this nucleus-cytoplasm ratio to sum the pixel number Sh of the region where the nucleus is estimated to be distributed and the pixel number Se of the region where the cytoplasm is estimated to be distributed. When Sh + Se is smaller than a predetermined threshold Ts, or when Sh + Se is greater than or equal to a predetermined threshold Ts and Sh / Se is smaller than a predetermined threshold Tr, it is estimated that the specific region does not include a cancer site, When Sh + Se is equal to or greater than a predetermined threshold Ts and Sh / Se is equal to or greater than a predetermined threshold Tr, the specific region is estimated to include a cancer site, and diagnosis support is performed.
JP 2004-286666 A

  When a pathologist makes a diagnosis visually, it is known to make a diagnosis by comprehensively determining the shape and distribution of the nucleus and cavity. In the conventional method, it is determined whether the specific local region of the diagnostic image is cancer by the threshold value determination of the nucleus-cytoplasm ratio. However, although the conventional method can diagnose whether the target local region is cancer, the similarity between the two images cannot be calculated for the entire image from the viewpoint of the image diagnosis of the doctor.

  The present invention has been made in view of the above problems, and extracts a component having a high medically subjective similarity using the nucleus-cytoplasm ratio and applies it to the target pathological tissue specimen. It is an object of the present invention to provide a similar image search apparatus capable of searching a pathological tissue sample that a doctor feels subjectively similar from a past database and assisting the diagnosis of the doctor.

In order to achieve the above-mentioned object, the present invention provides an image database for storing specimen tissue image data, an image data input means for inputting specimen tissue image data, and an entire image or an entire designated ROI region. On the other hand, a medical subjective similarity calculation unit that extracts components having a high medically subjective similarity, and a similarity search for searching similar image data from the image database based on the extracted similarity Means for displaying the image data input from the image data input means and the image data from the image database searched by the similarity search means, and the medical subjective similarity calculation unit. Image data background separation means for separating the image data input to the image data input means into a cytoplasm region and a background / cavity region; and the image data background separation A cell nucleus extracting means for extracting cell nuclei from the cytoplasmic region separated by the means, a cell nucleus locally based on the cytoplasmic region separated by the image data background separating means and the area occupied by the cell nucleus extracted by the cell nucleus extracting means -Local cell nucleus-cytoplasm ratio calculation means for calculating the cytoplasm area ratio, and the local cell nucleus-cytoplasm area ratio calculated by the local cell nucleus-cytoplasm ratio calculation means. And a wide area feature amount calculating means for calculating a characteristic amount of the entire image from the frequency distribution, and the similarity search means is a feature amount of the entire image calculated by the wide area feature amount calculating means. by calculating the similarity between the image data stored in the image database based on the point of extracting medically highly subjective similarity components There is a first feature.

A color information extracting unit configured to extract color information from the cytoplasm region separated by the image data background separating unit; and the similarity search unit searches the image database for image data similar to the color information. There is a second feature in this way.

The third feature is that the determination of the background region in the image data background separation unit and / or the extraction of the cell nucleus in the cell nucleus extraction unit is performed by a discriminant analysis method using luminance values or color information. .

  According to the present invention, there is provided a similar image search device capable of searching a pathological tissue sample having a high medically subjective similarity to a target pathological tissue sample from a past database and supporting a doctor's diagnosis. can do.

  Further, according to the present invention, the distribution of cell nuclei with respect to the cytoplasm is quantitatively determined, and the distribution is used as the similarity between the sample images. Therefore, the doctor can support diagnosis without setting a special threshold. Similar images can be provided.

  The present invention is particularly useful in the diagnosis of gastric cancer, but can also be applied to a large number of specimen tissue images such as the large intestine, liver and the like.

  Hereinafter, the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing a configuration of an embodiment of a similar image search apparatus of the present invention.

  In the figure, the image database 1 includes image data of past cases, and feature amounts of similarity of a region of interest (ROI; Region of Interest) image in the image data (cell nucleus-cytoplasmic area ratio, color information, etc.) , And doctor's findings data etc. for images are stored. The image data and the doctor's findings data are linked to each other.

  The image data input device 2 can input image data such as JPEG and BITMAP, which are general image formats, or DICOM images, which are medical image formats. Also, when entering, ROI can be specified to search for a specific area of image data.

  FIG. 2 shows an example of image data input to the image data input device 2. This image data 10 shows an example of a biopsy image of a stomach and is stained with HE staining liquid. The HE staining solution is a well-known staining solution composed of hematoxylin and eosin, and stains the cell region and the cell nucleus region of the target image in different colors. It should be noted that the light-colored regions (for example, regions A and B in FIG. 2) of the image data 10 are the background region and the other regions are the cytoplasmic regions, and the dark-colored regions (FIG. 2) , For example, C region) is a cell nucleus region. The region B is generally called a cavity region, but is regarded as a background region in the present invention.

  The medical subjective similarity calculation unit 3 extracts components having high medically subjective similarity for the entire image or the entire designated ROI region. The image data background separation device 31, cell nucleus extraction The apparatus 32 includes a local nucleus-cytoplasm ratio calculating unit 33 and a wide area feature amount calculating unit 34.

  The image data background separation device 31 separates the background region from the input image and outputs the cytoplasm region 31a. The background area has a luminance different from that of the cytoplasm area and tends to have almost uniform luminance. In addition, a part of the background area often touches the screen edge (E in FIG. 2). For this reason, using the luminance value for the entire image, a well-known discriminant analysis method (Otsu, “Automatic threshold selection method based on discriminant and least-squares criteria,” IEICE Transactions, Vol. J63-D, No. .4, pp.349-356, 1980.) Obtain background area candidates. The average and variance of the luminance values of the background region candidates that are in contact with the screen edge are obtained, and the region within a certain threshold α that correlates to the variance from this average value, that is, the region within (average ± α). Is determined to be the background area. By this method, it is possible to perform robust background region separation independent of a fixed threshold.

  The above-described processing of the image data background separation device 31 is represented by a flowchart as shown in FIG. In step S1, image data is input from the image data input device 2. In step S2, the entire input image is binarized using luminance values, and background region candidates are obtained by the discriminant analysis method. In step S3, an area in contact with the screen edge is acquired from the background area candidates. In step S4, the average value and the variance value of the luminance values in the area in contact with the screen edge E are calculated. In step S5, the area of the average value ± α is determined as the background. In step S6, a background area is removed from the image data. Accordingly, the image data background separation device 31 outputs the cytoplasmic region 31a in the image data.

  Next, in the cell nucleus extraction apparatus 32, the cytoplasm region is separated into a cell nucleus region and other regions (hereinafter referred to as cell regions). As described above, since the image data is stained with, for example, an HE staining solution, the cell region and the cell nucleus region of the target image are stained in different colors. Therefore, the image from which the background area has been removed by the image data background separation device 31 is discriminated by the R component using the difference in color information between the cell area and the cell nucleus area (R (red) component in the RGB color space). An accurate cell nucleus region 32a is extracted using an analysis method. In the image of FIG. 2, it can be generally considered that the darker the black, the larger the R component.

  In the local cell nucleus-cytoplasm ratio calculation device 33, as shown in FIG. 4, the image 10 of FIG. 2 is divided into square (for example, square) blocks having a predetermined size. Next, the nucleus-cytoplasm area ratio is calculated within the divided rectangular regions 10a, 10b, 10c,..., 10p,. That is, for each of all the square regions, for example, the occupation ratio of the area occupied by the cell nucleus to the cytoplasm area obtained by removing the background region or the cavity region from the image of the square region 10p representatively shown in FIG. Expressed as a fraction.

  In the wide area feature amount calculation means 34, the occupancy rate of the cell nuclei in all square areas is expressed by a histogram of frequency distribution. If all the target square areas are background areas, they are not added to the histogram. Next, preferably, histogram normalization and dimension compression are performed. These features are expressed as vectors.

For example, when the histogram of the frequency distribution is as shown in FIG. 6 (horizontal axis;%, vertical axis; frequency), the small frequency is not taken into consideration, and the dimension is compressed into the range P having the large frequency. The features of the dimension-compressed range P are expressed as (p1, p2,..., Pn) = (V ₁ , V ₂ ,..., V _n ) and a vector.

  In the similarity search device 6, the similarity with the image data stored in the image database 1 is calculated based on the features (vectors) obtained by the wide area feature quantity calculation means 34. For example, the similarity is expressed by a vector Euclidean distance. When the feature vectors of the two images a and b are Va and Vb, and the respective k-th vectors are Vak and Vbk, the similarity D is expressed by the following equation (1).

  The search result output device 7 displays the search result based on the similarity D. For example, the input image and the image on the database 1 are preferably used for image data having a similarity D equal to or greater than a predetermined size, or for a predetermined number of image data, preferably in the order of the similarity. , The doctor's findings data on the database 1 are attached and displayed.

  As described above, according to the present embodiment, a pathological tissue specimen similar to the target pathological tissue specimen can be searched from the database and provided to a doctor or the like, so that a doctor's diagnosis can be supported. In addition, a robust similar image that does not depend on a fixed threshold value can be provided.

  Next, a second embodiment of the present invention will be described with reference to the block diagram of FIG. In FIG. 7, reference numeral 8 denotes a color information extracting device, and other reference numerals are the same as or equivalent to those in FIG. This embodiment is different from the first embodiment in that a color information extraction device 8 is added.

  The color information extraction device 8 divides into squares as shown in FIG. 4, extracts color information of the rectangular cytoplasmic region, and sends it to the wide area feature amount calculation device 34. The wide area feature amount calculation device 34 expresses the feature of the color information by, for example, a histogram. The cytoplasm region is rarely distributed over the entire range of the color space, and the range of the color space may be compressed. Furthermore, normalization can be performed so that the image size has a flexible property, and the dimensions may be compressed to maintain robustness against slight color differences in order to improve calculation speed. These features are expressed as vectors.

For example, when the histogram of the color information is as shown in FIG. 8 (horizontal axis; color density (represented by 8 bits), vertical axis; frequency), the color space may be distributed in the range of Q in the figure, for example. Since there are many, dimension compression is carried out to the range Q. Features of dimensions compressed range Q _{is, (q1, q2, ····,} qn) = (V 1, V 2, ····, V n) is represented by a vector.

  Similar to the above, the similarity search device 6 calculates the similarity with the image data stored in the image database 1 based on the extracted feature (vector) for each square. Then, a search result is displayed based on both the similarity of the feature of the color information and the similarity of the cell nucleus-cytoplasm area ratio. Since other than the above is the same as the first embodiment, the description thereof is omitted.

  According to this embodiment, the image database can be searched with both the similarity of the cell nucleus-cytoplasm area ratio and the similarity of the feature of the color information, so that a more accurate search result can be obtained.

  As a modification of the present invention, as shown by the dotted lines in FIGS. 1 and 7, the image data, the data obtained by the nucleus-cytoplasm ratio calculation device 33, and the color information extraction device 8 are attached with the observation data. Stored in the image database 1.

  The image data input to the image data input device is not limited to gastric cancer, and can be image data for any biopsy image such as the liver and large intestine. The finding data may be at least one of a disease name, a disease site, a treatment method, a finding, a patient name, a patient sex, a patient age, a doctor name in charge, an imaging date and time, imaging means, and follow-up information. it can.

It is a block diagram which shows the structure of 1st Embodiment of this invention. It is a figure which shows the example of a biopsy image dye | stained with HE dye liquid. It is a flowchart which shows the process example of the image data background separation apparatus of FIG. It is explanatory drawing in the case of dividing a biopsy image dye | stained with HE dye | staining liquid into a square block. It is a figure which shows the biopsy image example in one block. It is explanatory drawing of the histogram of cell nucleus-cytoplasm area ratio. It is a block diagram which shows the structure of 2nd Embodiment of this invention. It is explanatory drawing of the histogram of color information of cytoplasm.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Image database, 2 ... Image data input device, 3 ... Medical subjective similarity calculation part, 6 ... Similarity search device, 7 ... Search result output device, 8 ... Color information extraction device, 10... Image data, 10a, 10b,..., 10n block, 31 ... image data background separation device, 32 ... cell nucleus extraction device, 33 ... local cell nucleus- Cytoplasm ratio calculation device, 34... Wide area feature amount calculation means, 31 a... Cytoplasm region, 32 a.

Claims (11)

An image database holding image data of the specimen tissue;
Image data input means for inputting the image data of the specimen tissue;
A medical subjective similarity calculation unit that extracts components with high medically subjective similarity for the entire image or the entire designated ROI region;
Similarity search means for searching similar image data from the image database based on the similarity extracted in the above;
Search result output means for displaying image data input from the image data input means and image data from the image database searched by the similarity search means,
The medical subjective similarity calculation unit,
Image data background separation means for separating the image data input to the image data input means into a cytoplasm region and a background / cavity region;
Cell nucleus extraction means for extracting cell nuclei from the cytoplasm region separated by the image data background separation means;
A local cell nucleus-cytoplasm ratio calculation means for locally calculating a cell nucleus-cytoplasm area ratio based on the cytoplasm region separated by the image data background separation means and the area occupied by the cell nucleus extracted by the cell nucleus extraction means;
The local cell nucleus-cytoplasmic area ratio calculated by the local cell nucleus-cytoplasm ratio calculating means is combined to determine the frequency distribution of the local cell nucleus-cytoplasmic area ratio with respect to the entire image, and the feature amount of the entire image is calculated from the frequency distribution. It consists of wide area feature quantity calculation means,
The similarity search means medically calculates a similarity with the image data stored in the image database based on the feature quantity of the whole image calculated by the wide area feature quantity calculation means. A similar image retrieval apparatus that extracts a component having a high subjective similarity . The similar image search device according to claim 1,
The image database stores doctor's findings data for the image data in addition to the image data, and the image data and doctor's findings data are displayed as search results. The similar image search device according to claim 1 ,
Further comprising color information extraction means for extracting color information from the cytoplasm region separated by the image data background separation means,
The similarity search means searches for similar image data from the image database in consideration of the color information. The similar image search device according to claim 1 ,
Similarity characterized in that the image data input to the image data input means and the nuclear nucleus-cytoplasm ratio ratio calculated by the local cell nucleus-cytoplasm ratio calculation means are attached to the finding data and registered in the image database. Image search device. The similar image search device according to claim 3 ,
The image data input to the image data input means, the cell nucleus-cytoplasm area ratio calculated by the local cell nucleus-cytoplasm ratio calculation means, and the color information extracted by the color information extraction means, with the findings data, A similar image retrieval apparatus that registers in the image database. The similar image search device according to any one of claims 1 to 5 ,
The image data background separation means obtains a background region candidate by discriminant analysis using the luminance value for the entire image data, and an average value and a variance value of the luminance values of the background region in contact with the screen edge among the background region candidates A similar image retrieval apparatus characterized in that determination of a background area of the image data is separated based on the image data. The similar image search device according to any one of claims 1 to 6 ,
The similar image retrieval apparatus, wherein the cell nucleus extraction means extracts a cell nucleus by a discriminant analysis method using a color component for the cytoplasm region. The similar image search device according to any one of claims 1 to 7 ,
The local cell nucleus-cytoplasm ratio calculation means divides an image into blocks and calculates a cell nucleus-cytoplasm area ratio in the block. The similar image search device according to claim 8,
The wide area feature amount calculation unit obtains a frequency distribution (histogram) of the entire block of the local cell nucleus-cytoplasm area ratio, and vectorizes the frequency distribution (histogram). The similar image search device according to any one of claims 1 to 9 ,
The similar image search device, wherein the search result output means displays similar images similar to the image data and finding data attached to the similar images, with rankings of similarities. The similar image search device according to any one of claims 1 to 10 ,
The image data input means, similar image retrieval apparatus characterized by inputting image data ROI specified.