WO2013102949A1

WO2013102949A1 - Image processing method, image processing device, image processing program, and memory medium

Info

Publication number: WO2013102949A1
Application number: PCT/JP2012/000016
Authority: WO
Inventors: 公太郎岡田; 和昭中根; 成昭松浦; 鈴木　貴
Original assignee: 株式会社知能情報システム; 国立大学法人大阪大学
Priority date: 2012-01-04
Filing date: 2012-01-04
Publication date: 2013-07-11
Also published as: JPWO2013102949A1; JP5762571B2

Abstract

An image processing method, wherein there is obtained an optimum threshold value for binarization processing without influence from factors such as staining conditions of biological tissue. This invention comprises: a step for plotting, on the basis of color density, a plurality of predetermined dots selected from an image of biological tissue on a two-dimensional field comprising XY reference axes having two different colors, and obtaining a scatter plot; a step for calculating the regression line of the predetermined dots on the scatter plot, setting imaginary straight lines oriented in a direction orthogonal to the regression line at equidistant positions on the regression line, and setting regions demarcated by the imaginary straight lines on the two-dimensional field; a step for obtaining a histogram showing the number of predetermined dots included in each region; a step for obtaining a first peak value, which is a distribution peak value, from the histogram; a step for differentiating the distribution line of the histogram and calculating a histogram differentiation first peak value, which is the point of inflection of the distribution line; and a step for calculating the threshold value from the first peak value and the histogram differentiation first peak value.

Description

Image processing method, image processing apparatus, image processing program, and recording medium

The present invention relates to an image processing method for obtaining a binarized image by performing binarization processing on a biological tissue image.
The present invention also relates to an image processing apparatus that executes the image processing method, an image processing program for operating the image processing method on a computer, and a recording medium on which the image processing program is recorded.

In the field of pathological diagnosis, it is widely performed to determine the presence or absence of cancer cells by analyzing biological tissue images. At the time of such pathological diagnosis, an attempt has been widely made to ensure objectivity by suppressing the occurrence of human error by performing image analysis processing on a biological tissue image. In such image analysis, the digitized image is not analyzed as it is, but the analysis is often performed after image processing is performed to organize the information necessary for the analysis, and the analysis result depends on the accuracy of the image processing. Are inevitable.

That is, the image processing that is most frequently performed in image analysis is a process of converting a biological tissue image stained by a process called binarization into a two-color image. For this reason, the accuracy of binarization affects the accuracy of image analysis, and in order to perform high-accuracy image analysis, setting an appropriate threshold value for binarization processing becomes a bottleneck. Patent Documents 1 to 3 can be cited as conventional examples of such threshold setting methods. In Patent Document 1, a threshold value for binarization processing is calculated from the maximum value and the minimum value of each density signal in the red, green, and blue wavelength regions. In Patent Document 2, the threshold value is obtained from the peak of the density histogram of the cell image. In Patent Document 3, a binarized image is obtained while adjusting a threshold value to a biological tissue image.

Japanese Patent Publication No.59-4058 Japanese Patent Publication No. 4-18347 Japanese Patent Laid-Open No. 5-180832

The biological tissue image is obtained by imaging the stained biological tissue with an imaging device through a microscope. For this reason, variations are likely to occur depending on various conditions such as staining conditions for living tissue or imaging conditions for imaging living tissue. Therefore, from the method of calculating a threshold for binarization processing from the maximum value and minimum value of the density signal in the wavelength region of each color as in Patent Document 1, or from the peak of the density histogram of the cell image as in Patent Document 2. In the method of obtaining the threshold value, it is inevitable that the threshold value will be different due to variations in the tissue image, and the objectivity is ensured by using the binarized image obtained by image processing using the threshold value. However, accurate pathological diagnosis is difficult. In the method described in Patent Document 3, it is indispensable to adjust the threshold value to a large or small value, an optimal threshold value cannot be obtained quickly, and image analysis and pathological diagnosis cannot be performed quickly.

The present invention was made for the purpose of solving the problems of the conventional image processing method as described above, and is not affected by variations in staining conditions of living tissue or imaging conditions of living tissue, It is an object of the present invention to provide an image processing method capable of obtaining an optimum threshold value for binarization processing appropriately and quickly, and thus greatly contributing to improvement and speed of objectivity of pathological diagnosis.

The image processing method according to the present invention plots a plurality of predetermined points selected from a biological tissue image on a two-dimensional field composed of XY reference axes of two different colors based on the color density of each predetermined point, A step of obtaining a scatter diagram of a predetermined point, a regression line of the predetermined point in the scatter diagram is calculated, and a large number of virtual points oriented in a direction orthogonal to the regression line at equal intervals on the regression line From the step of setting a straight line, setting a number of regions partitioned by these virtual straight lines on the two-dimensional field, obtaining a histogram indicating the number of each predetermined point included in each region, A step of obtaining a first peak value (Hp) that is a distribution peak value, and a differentiation line of the histogram are differentiated to calculate a histogram differentiation first peak value (Dp) that is an inflection point of the distribution line. Substituting the first peak value (Hp) and the histogram differential first peak value (Dp) into the following formula to calculate a threshold value used when binarizing the biological tissue image A step of obtaining a threshold straight line orthogonal to the regression line having the threshold as an intersection; and a step of binarizing the biological tissue image using the threshold straight line to obtain a binarized image; ,including.
<Formula>
(Hp−Dp) × variable + Dp = threshold variable: a value (0.5 to 0.9) determined in advance according to the biological tissue image to be a pathological diagnosis target

The first peak value (Hp) in the present invention means a peak value appearing at a position closest to the origin in the histogram. The histogram differential first peak value (Dp) is a peak value that appears at the closest position from the origin in the primary differential curve. In short, the distribution line from the origin to the first peak value (Hp) is changed. It means a music point.
Further, in the present invention, “a plurality of predetermined points selected from a biological tissue image are plotted on a two-dimensional field composed of XY reference axes of two different colors based on the color density of each predetermined point” If only the aspect of plotting the blue and red luminances of a plurality of pixel points arbitrarily selected from a biological tissue image on a two-dimensional field with the blue luminance on the X axis and the red luminance on the Y axis It is a concept including a mode in which all pixel points included in the biological tissue image are selected and the luminance of all the pixel points is plotted on a two-dimensional field.

The threshold straight line in the present invention is obtained by substituting the threshold obtained from the first peak value (Hp) and the histogram differential first peak value (Dp) as the value of the X coordinate and substituting the X coordinate into the equation of the regression line. This is a straight line that passes through a point on the two-dimensional plot (point on the regression line) with the Y coordinate value obtained as the Y coordinate value, and is orthogonal to the regression line.
Also, the binarization process using the threshold line means a process of binarizing depending on which of the two-dimensional plot regions each predetermined point is divided by the threshold line.

When the biological tissue image is a pathological diagnosis image for determining whether or not an image of a cancer tissue is included, the variable is preferably set to 0.6.

The image processing apparatus according to the present invention plots a plurality of predetermined points selected from a biological tissue image on a two-dimensional field composed of XY reference axes of two different colors based on the luminance of the color of each predetermined point. A scatter diagram creating means for creating a scatter diagram of a predetermined point; a regression line calculating means for calculating a regression line of a predetermined point in the scatter diagram; and orthogonally spaced relative to the regression line at equidistant positions on the regression line A plurality of virtual straight lines directed in the direction to be set, a region setting means for setting a plurality of regions partitioned by the virtual straight lines on the two-dimensional field, and the number of each predetermined point included in each region A histogram creating means for creating a histogram, a differentiation means for differentiating the histogram distribution line obtained by the histogram creating means to obtain a primary differential curve, and the histogram distribution A peak value that obtains a first peak value (Hp) that is a distribution peak value from, and a histogram derivative first peak value (Dp) that is an inflection point of the distribution line from a primary differential curve obtained by the differentiating means Substituting the calculation means, the first peak value (Hp) and the histogram differential first peak value (Dp) into the following formula, calculates a threshold value used when the biological tissue image is binarized. Threshold value calculation means, threshold value straight line calculation means for obtaining a threshold straight line orthogonal to the regression line with the threshold value obtained by the threshold value calculation means as an intersection, and binarization for a biological tissue image using the threshold value straight line Binarization processing means for performing processing to obtain a binarized image.
<Formula>
(Hp−Dp) × variable + Dp = threshold variable: a value (0.5 to 0.9) determined in advance according to the biological tissue image to be a pathological diagnosis target

When the biological tissue image is a pathological diagnosis image for determining whether or not an image of a cancer tissue is included, it is desirable to set the variable to 0.6.

The present invention is an image processing program for operating the image processing apparatus, and is an image processing program for causing a computer to function as each means.

Further, the present invention is a computer-readable recording medium on which the image processing program is recorded.

In the present invention, the first peak value (Hp) and the histogram differential first peak value (Dp), which are distribution peak values, are obtained from the regression line of the predetermined points in the scatter diagram and the histogram indicating the distribution status of the predetermined points. calculate. Next, a threshold for binarization processing is calculated from the first peak value (Hp) and the histogram differential first peak value (Dp), a threshold straight line is calculated using the threshold, and the threshold straight line is used. Then, binarization processing is performed on the biological tissue image to obtain a binarized image.

As described above, when a threshold line for binarization processing is obtained using a histogram in which the predetermined points are counted in a direction orthogonal to the regression line, the position related to the distance from the origin of the conventional XY reference axis is obtained. In the method of obtaining a threshold value using a histogram indicating the distribution of fixed points, influences such as variations in staining conditions of living tissue or variations in imaging conditions of living tissue can be eliminated. Therefore, according to the present invention, it is possible to obtain an optimal threshold line for binarization processing without being affected by variations in staining conditions or imaging conditions, thereby improving and speeding up the objectiveness of pathological diagnosis. You can contribute a lot.

For example, the same biological tissue is stained with the same staining time, and even when imaging is performed under the same imaging conditions, the final biological body obtained if the color condition of the staining agent is different. Tissue images will be different. That is, when a reddish stain is used, the biological tissue image is also reddish, and when a bluish stain is used, the biological tissue image is also bluish. 20 (a) and 20 (b), all pixel points included in these two types of biological tissue images are determined from the red and blue XY reference axes based on the luminance (red and blue color intensities) of each pixel point. A scatter diagram of pixel points obtained by plotting on a two-dimensional field is schematically shown. FIG. 20A shows a scatter diagram when using a reddish stain, and FIG. 20B shows a scatter diagram when using a bluish stain. That is, when a reddish stain is used, as shown in FIG. 20A, each pixel point has a high red color intensity and a low blue color intensity. On the other hand, when a bluish stain is used, each pixel point has a high blue color intensity and a low red color intensity, as shown in FIG.

FIG. 21 (a) is a histogram showing the frequency of the X-axis values created from the scatter diagrams of FIGS. 20 (a) and 20 (b). FIG. 21B is a histogram created from the scatter diagrams of FIGS. 20A and 20B and counting each pixel point in a direction orthogonal to the regression line. Note that the histogram in which each pixel point is counted in a direction orthogonal to the regression line referred to here is, specifically, a large number of pixels that are oriented at equal intervals on the regression line in a direction orthogonal to the regression line. Is created by counting the number of pixels belonging to the area defined by the virtual straight line.

21 (a) and 21 (b), hatched areas indicate areas that should be employed in the binarization process. That is, if a binarization process is performed using a threshold straight line in which the hatched area is adopted, finally, the same binarized image can be obtained regardless of the color condition of the staining agent. In other words, the “threshold value” in FIGS. 21A and 21B indicates the optimum threshold value for the binarization process.

As can be seen from the comparison between FIG. 21A and FIG. 21B, in the histogram of FIG. 21A, the shapes of the two distribution curves are completely different. That is, in the histogram of FIG. 21A, the shapes of the two distribution lines are completely different due to the influence of the staining condition called the color condition of the staining agent. For this reason, it is impossible to obtain a uniquely optimum threshold value for binarization processing from these two distribution lines. That is, in the histogram of FIG. 21A, no correlation can be found between various values such as the first peak values of these two distribution lines and the optimum threshold value.
As described above, it is impossible to calculate the optimum threshold using the histogram indicating the frequency of the X-axis value.

On the other hand, in the histogram of FIG. 21B, the position of two distribution lines (position in the X-axis direction), the peak height (height in the Y-axis direction), the distribution line width (width in the X-axis direction), and the like. Although different in point, the shapes of the two distribution lines are substantially the same. More specifically, in the example shown in FIG. 21B, the optimum threshold position (the position in the X-axis direction) in the entire width in the X-axis direction of the distribution line is the same in the two distribution lines. For this reason, it is possible to uniquely obtain the same threshold value for the optimum binarization process from these two distribution lines. That is, in the histogram of FIG. 21B, there is a correlation between these two distribution lines and the optimum threshold value, and therefore it is possible to uniquely obtain the optimum threshold value based on the distribution line. It is.

In addition, the inventors of the present invention compared the optimum threshold value with the distribution line, and as a result of intensive studies on the correlation, the first peak value (Hp) of the distribution line and the histogram that is the inflection point of the distribution line. It was found that an optimum threshold value can be calculated from the differential first peak value (Dp).

Specifically, the optimum threshold value always takes a value smaller than the first peak value (Hp), and the optimum threshold value always takes a value larger than the histogram differential first peak value (Dp). I found. In addition, the inventors have found that there is a relationship represented by the following mathematical formula among these three parties, and have completed the present invention.
<Formula>
(Hp−Dp) × variable + Dp = threshold (optimum threshold)

However, the “variable” used in the threshold value calculation method as described above differs depending on the biological tissue image to be subjected to pathological diagnosis. According to the knowledge of the present inventors, when the biological tissue image is a pathological diagnosis image for determining whether or not an image of a cancer tissue is included, the variable can be set to 0.6. preferable. This is because when the binarization process is performed using the threshold value obtained by adopting the variable (0.6) to create a binarized image, the peripheral edge of the cell nucleus is clearly displayed. Based on the rule of thumb. These variables may be selected automatically or manually when the pathological diagnosis target is determined on the program.

As described above, when the image processing method or the image processing apparatus according to the present invention is used, binarization processing can be performed appropriately and quickly without being affected by variations in staining conditions of living tissue or imaging conditions of living tissue. The optimal threshold for can be obtained. Therefore, it can greatly contribute to the improvement of the objectivity and speeding up of the pathological diagnosis. The image processing method can be provided in the form of program data (image processing program) that runs on a computer, and can also be provided in the form of a recording medium on which the image processing program is recorded.

1 is a schematic diagram illustrating a configuration of an image processing apparatus according to the present invention. 3 is a flowchart illustrating an image processing method according to the present invention. It is a figure which shows an example of a biological tissue image. This is a binarized image of the biological tissue image, which is obtained by performing binarization processing with an optimal threshold line. It is the binarized image of the biological tissue image, and is obtained by performing binarization processing with the first peak value (Hp). It is the binarized image of the biological tissue image, and is obtained by performing binarization processing with the histogram differential first peak value (Dp). FIG. 6 is a scatter diagram of predetermined points obtained by plotting a plurality of predetermined points selected from a biological tissue image on a two-dimensional field composed of different two-color reference axes based on the luminance of the color of each predetermined point. In FIG. 7, a scatter diagram is created by plotting all pixel points constituting a living tissue image on a two-dimensional field. It is the histogram which counted the number of each predetermined point in the direction orthogonal to the regression line of each predetermined point. It is a primary differential curve obtained by differentiating the distribution line of the histogram of FIG. (A) to (c) are obtained by plotting a plurality of predetermined points selected from a biological tissue image on a two-dimensional field consisting of different two-color reference axes based on the luminance of the color of each predetermined point. It is the figure which showed typically the scatter diagram of the predetermined point which was done. (A) to (c) are scatter diagrams of biological tissue images obtained by coloring the same biological tissue using colorants having different color conditions, and (a) is a diagram in (b). In comparison, a scatter diagram of a biological tissue image colored with a reddish colorant, and (c) shows a scatter diagram of a biological tissue image colored with a bluish colorant compared to (b). ing. It is the histogram which counted the number of each predetermined point in the direction orthogonal to the regression line of each predetermined point, a dashed-two dotted line respond | corresponds to Fig.10 (a), a solid line corresponds to FIG.10 (b), and a broken line corresponds to FIG.10 (c). is doing. FIG. 11 is a first-order differential curve obtained by differentiating the histogram distribution lines in FIG. 11. The two-dot chain line corresponds to FIG. 10A, the solid line corresponds to FIG. 10B, and the broken line corresponds to FIG. is doing. (A) to (c) are histograms for explaining the binarization processing using the threshold value, and the shaded area indicates the pixel region used in the binarization processing. A scatter diagram of predetermined points obtained by plotting a plurality of predetermined points selected from a biological tissue image on a two-dimensional field consisting of different two-color reference axes based on the luminance of the color of each predetermined point This is schematically shown. The figure shows a scatter diagram created from biological tissue images captured using three different types of imaging devices for biological tissues stained under the same staining conditions for the same biological tissue. Yes. It is a histogram in which the number of each predetermined point is counted in a direction orthogonal to the regression line of each predetermined point, (a) corresponds to a scatter diagram specified by a regression line indicated by a two-dot chain line in FIG. 14, (b) 14 corresponds to the scatter diagram specified by the regression line indicated by the solid line in FIG. 14, and (c) corresponds to the scatter diagram specified by the regression line indicated by the broken line in FIG. 14. Schematic representation of a scatter diagram of predetermined points obtained by plotting a plurality of predetermined points selected from a biological tissue image on a two-dimensional field composed of different two-color reference axes based on the color density of each predetermined point It is shown as an example. (A) to (c) are scatter diagrams of biological tissue images captured under the same coloring conditions and different exposure conditions (imaging conditions) with respect to the same biological tissue. When the exposure is less than that in (b), (c) shows the case where there is more exposure than in (b). It is a histogram which shows the number of each predetermined point contained in each area prescribed | regulated by a virtual straight line, a dashed-two dotted line respond | corresponds to Fig.16 (a), a continuous line corresponds to FIG.16 (b), and a broken line shows FIG. This corresponds to c). FIG. 17 is a first-order differential curve obtained by differentiating the histogram distribution line in FIG. 17, the two-dot chain line corresponds to FIG. 16A, the solid line corresponds to FIG. 16B, and the broken line in FIG. This corresponds to c). (A) to (c) are histograms for explaining the binarization processing using the threshold value, and the shaded area indicates the pixel region used in the binarization processing. (A) and (b) were obtained by plotting a plurality of predetermined points selected from a biological tissue image on a two-dimensional field consisting of different two-color reference axes based on the color density of each predetermined point. It is a figure which shows typically the scatter diagram of a predetermined point. (A) and (b) are scatter diagrams of biological tissue images obtained by coloring the same biological tissue using colorants having different color conditions, and (a) shows a reddish colorant. FIG. 5B is a scatter diagram of a biological tissue image colored using a coloring method, and FIG. 5B is a scatter diagram of a biological tissue image colored using a bluish colorant. (A) is a histogram showing the distribution of predetermined points according to the distance from the origin of the XY reference axis of each predetermined point, and (b) shows the number of each predetermined point included in each region defined by a virtual straight line. It is a histogram to show.

The present invention is an image for binarizing a biological tissue image that is performed as a pre-processing of image analysis when performing pathological diagnosis for determining whether or not cancer cells or the like are included in the biological tissue by image analysis. Intended for processing methods. Such a biological tissue image is obtained by imaging a stained biological tissue with a microscope, but the biological tissue image often varies depending on various conditions such as staining conditions of the biological tissue or imaging conditions. Is as described above.

(When there is variation in the dyeing agent)
FIGS. 10A to 10C show scatter diagrams created from biological tissue images obtained using three kinds of stains having different hues. That is, FIGS. 10A to 10C show a plurality of predetermined points selected from the biological tissue image (here, all pixel points constituting the biological tissue image) based on the color density of each predetermined point. 10A and 10C are scatter diagrams of predetermined points obtained by plotting on a two-dimensional field composed of a blue and XY reference axes. In FIGS. 10A to 10C, pixels corresponding to the predetermined points are represented by R, G, and B. Each predetermined point is plotted on a two-dimensional field based on the color components of R (red: vertical axis) and B (blue: horizontal axis).

In FIGS. 10 (a) to 10 (c), a red or blue hue is changed up and down from one biological tissue image to create a total of three biological tissue images, and these are used as dyes having different hues. It is assumed that the image is a biological tissue image obtained by staining. More specifically, FIG. 10A is a scatter diagram created based on sample image data obtained by raising the red hue from the biological tissue image as the original data according to FIG. 10B. It is assumed that the entire biological tissue is stained with a reddish stain rather than the form shown in FIG. 10 (b). FIG. 10C is a scatter diagram created based on the sample image data obtained by raising the blue hue from the biological tissue image which is the original data according to FIG. 10B, and FIG. The case where the whole biological tissue is dye | stained using the bluish dye rather than the form shown to (4) is assumed. When a reddish stain is used, as shown in FIG. 10A, each pixel point has a strong red color intensity and a low blue color intensity. On the other hand, when a bluish stain is used, as shown in FIG. 10C, each pixel point has a high blue color intensity and a low red color intensity.

Next, a histogram indicating the number of each predetermined point included in each area defined by the virtual straight line is created. Specifically, as shown in FIGS. 10 (a) to 10 (c), after calculating a regression line at a predetermined point in the scatter diagram, it is orthogonal to the regression line at equal intervals on the regression line. A large number of virtual straight lines oriented in the direction are set, and the partitioned area is set on the two-dimensional field by the virtual straight lines. Next, a histogram is created based on the affiliation status of each predetermined point with respect to the region. FIG. 11 shows a histogram. The two-dot chain line in FIG. 11 corresponds to the distribution diagram in FIG. 10A, the solid line corresponds to the distribution diagram in FIG. 10B, and the broken line in FIG. Corresponds to the distribution map. In FIG. 11, Hp indicates the first peak value of the histogram.

Next, differentiate the histogram distribution line to create a first derivative curve. FIG. 12 shows such a first derivative curve, the two-dot chain line corresponds to the distribution diagram of FIG. 10 (a), the solid line corresponds to the distribution diagram of FIG. 10 (b), and the broken line represents FIG. 10 (c). Corresponds to the distribution map. Moreover, Dp in FIG. 12 shows a histogram differential first peak value. Further, the left and right histogram shift distances (d1 ′ and d2 ′) with respect to the first-order differential curve located at the center of FIG. 12 are the left and right histogram shift distances (d1 ′ and d2) with respect to the histogram located at the center of FIG. It is noted that it is completely consistent with ′).

Next, the optimum threshold value is calculated by substituting the first peak value (Hp) and the histogram differential first peak value (Dp) into the above formula (variable: 0.6). That is, three threshold values corresponding to the respective histograms in FIGS. 10A to 10C are calculated.

Next, the threshold value obtained from the first peak value (Hp) and the histogram differential first peak value (Dp) is set as the value of the X coordinate, and the value of the Y coordinate obtained by substituting the X coordinate into the regression line equation A threshold straight line that passes through a point on the two-dimensional plot (a point on the regression line) with Y as the value of the Y coordinate and is orthogonal to the regression line is calculated.

Next, binarization processing is performed using a threshold line. Specifically, the binarization process is performed depending on which of the two-dimensional plot regions each predetermined point is divided by the threshold straight line.

The shaded areas in FIGS. 13A to 13C indicate areas that are employed in the binarization process using these threshold lines. As is clear from FIGS. 13A to 13C, it can be seen that the same area region can be extracted by binarization using a threshold line in any histogram.
That is, by using the image processing method according to the present invention, it is possible to always obtain the same binarized image without being affected by the staining condition such as the color condition of the staining agent. This means that the binarization accuracy is remarkably improved, and therefore pathological diagnosis can be executed more accurately. In addition, human error intervention can be suppressed and accurate pathological diagnosis can be performed. It is also excellent in that an optimum binarized image can be obtained more quickly.

(When there are variations in imaging equipment)
In a camera that is an imaging device, an image processing-dedicated IC called an image engine performs various image processing such as color space conversion, gamma correction, and noise reduction on data output from an imaging device. For this reason, it is widely known that the tendency of image quality differs when the manufacturer or type of camera is different. In other words, even when the biological tissue is stained under the same staining condition, if the imaging condition of the type of the imaging device is different, it is inevitable that the finally obtained biological tissue image will be different.

FIG. 14 is a diagram conceptually showing a scatter diagram of three biological tissue images obtained by imaging using three different types of imaging devices. Here, the scatter diagram specified by the regression line shown by the solid line in FIG. 14 is used as the original data, and the scatter diagram related to the original data is shifted in the direction in which red is emphasized, A specified scatter diagram is obtained, and this is a scatter diagram of a biological tissue image captured using a camera that tends to emphasize red. In addition, by shifting the scatter diagram related to the original data in the direction in which the red color is reduced, the scatter diagram specified by the regression line indicated by the two-dot chain line is obtained, and this tends to reduce the red color. It is a scatter diagram of a biological tissue image captured using a camera. In FIG. 14, these three scatter diagrams are shown on the same two-dimensional field (vertical axis: red, horizontal axis: blue) for comparison.

Next, for each of the three scatter diagrams shown in FIG. 14, a histogram indicating the distribution status of the predetermined points is created. Specifically, after calculating a regression line of a predetermined point in the scatter diagram, a large number of virtual lines directed in a direction orthogonal to the regression line are set at equal intervals on the regression line, A region partitioned by the virtual straight line is set on the two-dimensional field. Next, a histogram indicating the distribution status of the predetermined points is created based on the affiliation status of each predetermined point with respect to the region. FIG. 15 shows a histogram, and the histogram of FIG. 15A corresponds to the scatter diagram specified by the regression line shown by the two-dot chain line of FIG. 14, and the histogram of FIG. 14 corresponds to the scatter diagram specified by the regression line indicated by the solid line, and the histogram of FIG. 15C corresponds to the scatter diagram specified by the regression line indicated by the broken line in FIG. Thus, by creating a histogram, the first peak value (Hp) of each histogram can be obtained.

Next, a first derivative curve is created by differentiating the distribution lines of each histogram to obtain a histogram derivative first peak value (Dp).

Next, the optimum threshold value is calculated by substituting the first peak value (Hp) and the histogram differential first peak value (Dp) into the above formula (variable: 0.6). That is, three threshold values corresponding to the histograms of FIGS. 15A to 15C are calculated.

The hatched areas in FIGS. 15A to 15C indicate areas that are employed in the binarization process using these threshold lines. As is apparent from FIGS. 15A to 15C, it can be seen that the same area region can be extracted by binarization processing using a threshold straight line in any histogram.
That is, by using the image processing method according to the present invention, it is possible to always obtain the same binarized image without being affected by imaging conditions such as image quality directivity variations caused by using different imaging devices. . This means that the binarization accuracy is remarkably improved, and therefore pathological diagnosis can be executed more accurately. In addition, human error intervention can be suppressed and accurate pathological diagnosis can be performed. It is also excellent in that an optimum binarized image can be obtained more quickly.

(When exposure conditions vary)
Even when the biological tissue stained under the same staining condition is imaged using the same imaging device, the final biological tissue obtained when the imaging condition called the exposure condition is different. It is inevitable that the images will be different.

FIGS. 16 (a) to 16 (c) show the distribution of biological tissue images imaged under three different exposure conditions using the same imaging device for one biological tissue stained under the same staining conditions. It is a figure which shows a figure notionally. The biological tissue image that is the original data of the scatter diagram in FIG. 16A is an image that is captured with a smaller exposure than the biological tissue image that is the original data of the scatter diagram in FIG. In addition, the biological tissue image that is the original data of the scatter diagram of FIG. 16C is captured with a larger exposure than the biological tissue image that is the original data of the scatter diagram of FIG. That is, when the exposure is reduced, each pixel point constituting the biological tissue image moves in a direction approaching the origin, and when the exposure is opened, each pixel point moves in a direction away from the origin.

The two-dot chain line in Fig. 16 (a), the solid line in Fig. 16 (b), and the broken line in Fig. 16 (c) indicate the regression line of each scatter diagram. As is apparent from the regression lines in FIGS. 16A to 16C, the regression lines are the same even for biological tissue images captured under different exposure conditions.

Next, a histogram is created for each of the three scatter diagrams shown in FIG. Specifically, after calculating a regression line of a predetermined point in the scatter diagram, a large number of virtual lines directed in a direction orthogonal to the regression line are set at equal intervals on the regression line, An area partitioned by the virtual straight line is set on the two-dimensional field. Next, a histogram indicating the distribution status of the predetermined points is created based on the affiliation status of each predetermined point with respect to the region.

FIG. 17 shows the histogram created as described above. The histogram indicated by the two-dot chain line in FIG. 17 corresponds to the scatter diagram of FIG. 16A and is indicated by the solid line in FIG. Corresponds to the scatter diagram of FIG. 16 (b), and the histogram indicated by a broken line in FIG. 17 corresponds to the scatter diagram of FIG. 16 (c). Thus, the first peak value (Hp) of each histogram can be obtained.

Next, as shown in FIG. 18, the distribution line of each histogram is differentiated to create a primary differential curve, and a histogram differential first peak value (Dp) is obtained. 18 corresponds to the scatter diagram of FIG. 16 (a), and the primary differential curve shown by the solid line in FIG. 18 corresponds to the scatter diagram of FIG. 16 (b). A primary differential curve indicated by a broken line in FIG. 18 corresponds to the scatter diagram in FIG. Thereby, the histogram differential first peak value (Dp) of each histogram can be obtained.

Next, the optimum threshold value is calculated by substituting the first peak value (Hp) and the histogram differential first peak value (Dp) into the above formula (variable: 0.6). That is, three threshold values corresponding to each histogram in FIG. 17 are calculated.

In FIG. 19 (a) to 19 (c), the hatched area indicates an area that is employed in the binarization process using these threshold lines. As can be seen from FIGS. 19A to 19C, in any of the histograms, it is understood that an area region having the same ratio with respect to the whole can be extracted by the binarization process using the threshold line. That is, if the image processing method according to the present invention is used, the same binarized image can always be obtained without being affected by the imaging condition such as the exposure condition. This means that the binarization accuracy is remarkably improved, and therefore pathological diagnosis can be executed more accurately. In addition, human error intervention can be suppressed and accurate pathological diagnosis can be performed. It is also excellent in that an optimum binarized image can be obtained more quickly.

(Configuration of image processing apparatus)
As shown in FIG. 1, an image processing apparatus 1 according to the present invention includes an image acquisition unit 2, a storage unit 3, an image processing unit 4, a display control unit 5, a display unit 6, and a primary storage unit 7. The image processing apparatus 1 may be one obtained by installing a dedicated image processing program on a commercially available personal computer.

The image acquisition unit 2 acquires a biological tissue image obtained by imaging a biological tissue from an external device (for example, an imaging device), and causes the storage unit 3 to store the biological tissue image. Specific examples of the image acquisition unit 2 include a capture card that directly receives a biological tissue image captured by an imaging device, a card reader that reads information from a memory card, or a reader device that reads information from a USB memory. it can.

The method for staining biological tissue in the present invention is not particularly limited, and examples thereof include HE (Hematoxilin-Eosin) staining. In HE staining, cell nuclei and cytoplasm are stained, and an overall picture of cells and cell structures can be grasped. FIG. 3 shows a biological tissue image obtained by HE staining.

The storage unit 3 is a non-volatile storage device such as a hard disk or a flash memory. In addition to the biological tissue image acquired via the image acquisition unit 2, a scatter diagram and a regression line created by the image processing unit 4 are used. Various data such as a region partitioned by virtual straight lines, a histogram, a primary differential curve, a first peak value (Hp), and a histogram differential first peak value (Dp) are stored. The storage unit 3 stores an OS program in addition to various control programs for controlling the image acquisition unit 2, the display control unit 5, and the like. Further, an application program (image processing program) for executing the image processing method according to the present invention is installed and stored in advance. A ROM (Read Only Memory) (not shown) stores system programs and the like.

The image processing unit 4 is a central processing unit called a CPU (Central Processing Unit), and a primary storage unit 7 which is a RAM (Random Access Memory) based on an application program stored in the storage unit 3. Image processing is executed as a work area. Specifically, the image processing unit 4 is based on an application program, a scatter diagram creation unit (scatter diagram creation unit), a regression line calculation unit (regression line calculation unit), a region setting unit (region setting unit), and a histogram creation Section (histogram creation means), differentiation section (differentiation means), peak value calculation section (peak value calculation means), threshold calculation section (threshold calculation means), threshold straight line calculation section (threshold straight line calculation means), and binarization processing Part (binarization processing means). Specific operations of the image processing unit 4 will be described later.

The display control unit 5 controls the display unit 6 to display the processing result by the image processing unit 4 on the display unit 6, and is a graphic chip or a graphic card equipped with a graphic chip. The display unit 6 is a liquid crystal display device, for example.

Next, an image processing method by the image processing apparatus 1 having the above configuration will be described with reference to the flowchart of FIG. First, when a biological tissue image as shown in FIG. 3 is acquired from a state in which an application program related to image processing is started (S1), the image processing unit 4 functions as a scatter diagram creation unit as a scatter diagram. (See FIG. 7) is created (S2). However, a scatter diagram may be created from the biological tissue image stored in advance in the storage unit 3 (S2).

Specifically, the image processing unit 4 plots a plurality of predetermined points selected from the biological tissue image on a two-dimensional field composed of different two-color reference axes based on the color density of each predetermined point. A scatter diagram as shown in FIG. 7 is created. In FIG. 7, a scatter diagram is created using a two-dimensional field in which the horizontal axis (X axis) is blue and the vertical axis (Y axis) is red. Further, tens of thousands of pixels corresponding to each predetermined point included in the biological tissue image are decomposed into R, G, and B color components, and each predetermined point is determined based on the R (red) and B (blue) color components. Are plotted on a two-dimensional field. The image processing unit 4 displays the obtained scatter diagram on the display unit 6 via the display control unit 5. In addition, the image processing unit 4 attaches a file name related to the biological tissue image to the scatter diagram and stores it in the storage unit 3.

Next, the image processing unit 4 functions as a regression line calculation unit, and calculates a regression line by the least square method. That is, a regression line is calculated based on all the plots on the scatter diagram or arbitrary predetermined points extracted from the scatter diagram (S3). The image processing unit displays the obtained regression line on the display unit 6 via the display control unit 5. Further, the image processing unit 4 attaches a file name related to the biological tissue image to the regression line and stores it in the storage unit 3.

Next, the image processing unit 4 functions as a region setting unit, and sets the two-dimensional field to a large number of regions (S4). Specifically, the image processing unit 4 sets a large number of virtual straight lines oriented in a direction orthogonal to the regression line at equal intervals on the regression line, and defines areas partitioned by the virtual line. Set on a two-dimensional field. In FIG. 7, a form in which several virtual straight lines are drawn is shown, but actually, as described above, as shown in FIGS. 16 (a) to (c), A large number of virtual straight lines are drawn at equally spaced positions. The image processing unit 4 displays the obtained virtual straight line and region on the display unit 6 via the display control unit 5. Further, the image processing unit 4 stores the file name related to the biological tissue image in the storage unit 3 in comparison with the virtual straight line and the region.

Next, the image processing unit 4 functions as a histogram creation unit, and creates a histogram (see FIG. 8) indicating the number of predetermined points included in the region of each predetermined point (S5). The image processing unit 4 displays the obtained histogram on the display unit 6 via the display control unit 5. Further, the image processing unit 4 attaches a file name associated with the biological tissue image to the histogram and stores it in the storage unit 3.

Next, the image processing unit 4 functions as a peak value calculation unit and obtains the first peak value (Hp) from the previous histogram (S6). The first peak value (Hp) here indicates the first peak value appearing at the closest position from the origin in the distribution line of the histogram. However, in the example of the histogram shown in FIG. 8, only one peak appears. The image processing unit 4 displays the obtained first peak value (Hp) on the display unit 6 via the display control unit 5. In addition, the image processing unit 4 attaches a name associated with the biological tissue image to the first peak value (Hp) and stores it in the storage unit 3.

Next, the image processing unit 4 functions as a differentiation unit and performs a differentiation process on the histogram distribution line obtained in the previous S6 to obtain a primary differential curve (S7). FIG. 9 shows a first-order differential curve obtained by such differential processing. The image processing unit 4 displays the obtained primary differential curve on the display unit 6 via the display control unit 5. Further, the image processing unit 4 attaches a file name related to the biological tissue image to the first-order differential curve and stores the file name in the storage unit 3.

Next, the image processing unit 4 functions as a peak value calculation unit, and obtains a histogram differential first peak value (Dp) from the previous primary differential curve (S8). The histogram differential first peak value (Dp) here refers to the first peak value appearing at the closest position from the origin in the primary differential curve. That is, the histogram differential first peak value (Dp) is an inflection point of the distribution line of the previous histogram from the origin to the first peak value (Hp). The image processing unit 4 displays the obtained histogram differential first peak value (Dp) on the display unit 6 via the display control unit 5. Further, the image processing unit 4 attaches a name associated with the biological tissue image to the histogram differential first peak value (Dp) and stores it in the storage unit 3.

Next, the image processing unit 4 functions as a threshold value calculation unit, and calculates the first peak value (Hp) obtained in the previous S6 and the histogram differential first peak value (Dp) obtained in S8 by the mathematical expression. The threshold value is calculated by substituting ((Hp−Dp) × variable + Dp = threshold value) (S9). As described above, the variables included in the mathematical formula are different depending on the biological tissue image to be pathologically diagnosed. As in this embodiment, the biological tissue image includes an image of cancer tissue. In the case of the pathological diagnosis image for determining whether or not it is determined, the variable is set to 0.6. As described above, the selection of the variable is automatically or manually selected when the pathological diagnosis target is determined on the program of the image processing apparatus. The image processing unit 4 displays the obtained threshold value on the display unit 6 via the display control unit 5. In addition, the image processing unit 4 assigns a name associated with the biological tissue image to the threshold value and stores it in the storage unit 3.

Next, the image processing unit 4 functions as a threshold value calculation unit, and calculates a threshold line from the threshold value obtained in S9 (S10). Specifically, a point on the two-dimensional plot (on the regression line) with the previous threshold as the value of the X coordinate and the value of the Y coordinate obtained by substituting the X coordinate in the regression line equation as the value of the Y coordinate. ), And a threshold straight line orthogonal to the regression line is calculated. The image processing unit 4 displays the obtained threshold straight line on the display unit 6 via the display control unit 5. In addition, the image processing unit 4 attaches a name associated with the biological tissue image to the threshold line and stores the name in the storage unit 3.

Next, the image processing unit 4 functions as a binarization processing unit, performs binarization processing on the biological tissue image using the threshold line obtained in S10, and obtains a binarized image (S11). ). The image processing unit 4 displays the obtained binarized image on the display unit 6 via the display control unit 5. Further, the image processing unit 4 attaches a file name related to the biological tissue image to the binarized image and stores it in the storage unit 3.

FIG. 4 is a binarized image obtained in S11. 5 is a binarized image obtained by binarization processing using the first peak value (Hp), and FIG. 6 is binarization using the histogram differential first peak value (Dp). It is the binarized image obtained by processing. In FIGS. 4 to 6, black portions are pixels used in the binarization process. 4 to 6, the binarized image obtained by the image processing method according to the present embodiment is positioned between the first peak value (Hp) and the histogram differential first peak value (Dp). It can be seen that it was obtained using the threshold value. That is, since the black part in FIG. 4 is less than the black part shown in FIG. 5 and more than the black part shown in FIG. 6, the binarized image obtained by the image processing method according to the present embodiment is the first. It can be seen that it was obtained using a threshold value located between the peak value (Hp) and the histogram differential first peak value (Dp).

The present inventors have confirmed that the use of the binarized image obtained by the image processing method according to the present embodiment as described above enables pathological diagnosis more accurately than in the past. That is, in this type of pathological diagnosis, the presence or absence of cancer cells is confirmed based on the number of spaces surrounded by the outer edge of the cancer tissue, but the binarized image obtained by the image processing method according to the present embodiment is used. It has been confirmed that the presence or absence of cancer cells can be determined with high accuracy. As described above, the image processing method according to the present embodiment can greatly contribute to the improvement of objectivity of pathological diagnosis based on image analysis.

The scatter diagrams, histograms, and the like shown in the above embodiments are examples, and it goes without saying that these scatter diagrams and the like differ depending on the biological tissue image to be subjected to pathological diagnosis. In the above embodiment, the image processing unit 4 displays these on the display unit 6 every time a scatter diagram or a histogram is obtained. However, the present invention is not limited to this, and the image processing unit 4 Only the finally obtained binarized image may be displayed on the display unit 6.

1 Image processing device 4 Image processing unit

Claims

Plotting a plurality of predetermined points selected from the biological tissue image on a two-dimensional field consisting of XY reference axes of two different colors based on the color density of each predetermined point, and obtaining a scatter diagram of the predetermined points; ,
Calculating a regression line of a predetermined point in the scatter diagram;
Setting a plurality of virtual straight lines oriented in a direction orthogonal to the regression lines at equal intervals on the regression lines, and setting regions defined by the virtual lines on the two-dimensional field; ,
Obtaining a histogram indicating the number of each predetermined point included in each region;
Obtaining a first peak value (Hp) which is a distribution peak value from the histogram;
Differentiating the distribution line of the histogram to calculate a histogram differentiation first peak value (Dp) that is an inflection point of the distribution line;
Substituting the first peak value (Hp) and the histogram differential first peak value (Dp) into the following formula to calculate a threshold value used when binarizing the biological tissue image;
Obtaining a threshold line orthogonal to the regression line having the threshold as an intersection;
Calculating a threshold line using the threshold;
Performing a binarization process on a biological tissue image using a threshold line to obtain a binarized image;
An image processing method including:
<Formula>
(Hp−Dp) × variable + Dp = threshold variable: a value (0.5 to 0.9) determined in advance according to the biological tissue image to be a pathological diagnosis target
The biological tissue image is a pathological diagnosis image for determining whether an image of a cancer tissue is included,
The image processing method according to claim 1, wherein the variable is set to 0.6.
A plurality of predetermined points selected from a tissue image are plotted on a two-dimensional field consisting of XY reference axes of two different colors based on the color density of each predetermined point, and a scatter diagram for creating a scatter diagram of the predetermined points is created Drawing means;
Regression line calculating means for calculating a regression line of a predetermined point in the scatter diagram;
A region setting for setting a plurality of virtual straight lines oriented in a direction orthogonal to the regression line at equal intervals on the regression line, and setting a region partitioned by the virtual line on the two-dimensional field Means,
A histogram creating means for creating a histogram indicating the number of each predetermined point included in each region;
Differentiating means for differentiating the distribution line of the histogram obtained by the histogram creating means to obtain a primary differential curve;
A first peak value (Hp) that is a distribution peak value is obtained from the distribution line of the histogram, and a histogram differential first peak value (Dp) that is an inflection point of the distribution line is obtained from the primary differential curve obtained by the differentiating means. ) To obtain a peak value,
A threshold value calculation means for calculating a threshold value used when binarizing the biological tissue image by substituting the first peak value (Hp) and the histogram differential first peak value (Dp) into the following equation: ,
Threshold straight line calculating means for calculating a threshold straight line using the threshold obtained by the threshold calculating means;
Threshold line calculation means for obtaining a threshold line orthogonal to the regression line with the threshold obtained by the threshold calculation means as an intersection;
An image processing apparatus comprising: binarization processing means for performing binarization processing on a biological tissue image using the threshold line to obtain a binarized image.
<Formula>
(Hp−Dp) × variable + Dp = threshold
The biological tissue image is a pathological diagnosis image for determining whether an image of a cancer tissue is included,
The image processing apparatus according to claim 3, wherein the variable is set to 0.6.
An image processing program for operating the image processing apparatus according to claim 3 or 4 for causing a computer to function as each of the above means.
A computer-readable recording medium on which the image processing program according to claim 5 is recorded.