JP2019028887A - Image processing method - Google Patents

Abstract

To provide an image processing method for extracting an area to be observed by an observer from an image of an object.SOLUTION: An image processing method includes: (a) a step S1 for acquiring an image of an object; (b) a step S2 for determining feature quantities of pixels of the image; (c) a step S3 for determining posterior probabilities of the pixels using a predetermined function from the feature quantities of the pixels; (d) a step S4 for creating a likelihood map of the image of the object by converting the posterior probabilities of the pixels to luminance values; (e) a step S5 for dividing the likelihood map into a plurality of areas including pixels high in similarity of luminance values; and (f) a step S7 for executing automatic diagnosis on each of the divided areas. A step S6 can be selectively included to display the divided or automatically diagnosed likelihood map on display means.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an image processing method, and more specifically, a texture analysis technique is used to extract a region to which an observer should pay attention from an image of an object, and various types by the observer regarding the extracted region. The present invention relates to an image processing method / automatic diagnosis method for facilitating recognition, determination, etc.

  Conventionally, many image processes for image pattern recognition have been proposed. For example, in the past, Patent Document 1 proposes a method of reading a pattern (character) by selecting and classifying feature amounts extracted from an image using a predetermined mask using a multivariate analysis technique. For example, Patent Document 2 discloses a derivation of a higher-order local co-occurrence feature obtained by extending a higher-order local autocorrelation (HLAC) feature that can be used for texture analysis, and a texture recognition method using the derivation.

  On the other hand, as an application example of image recognition using a texture analysis technique, there is image diagnosis in the medical field. For example, image diagnosis using a tomographic image obtained by photographing a part of a human body by CT, MRI or the like is widely performed. Yes. In such a diagnosis using medical images, not only is the diagnosis of the doctor viewing the image more accurate, but it can also be made as quickly and easily as possible in the medical situation where diagnosis must be performed in a limited time. It has been demanded. That is, the clinical significance of the diagnosis (localization diagnosis) of the lesion site is greater than the diagnosis (presence diagnosis) of the presence or absence of the lesion.

Patent Publication No. 58-47064 JP 2008-21044 A

  The present invention relates to an image processing method / automatic diagnosis for extracting a region to which an observer should pay attention from an image of an object so that various kinds of recognition and determination by an observer using the image of the object can be performed more easily. It aims to provide a method.

  An image processing method according to an aspect of the present invention includes (a) a step of acquiring an image of an object, (b) a step of obtaining a feature amount for each pixel of the image, and (c) a predetermined amount from the feature amount for each pixel. Obtaining a posterior probability for each pixel using an identification function; (d) converting a posterior probability for each pixel into a luminance value to create a likelihood map of an image of the object; and (e) likelihood. Dividing the map into a plurality of regions each including a pixel having a high similarity in luminance value, and (f) performing an automatic diagnosis for each of the divided regions. In the automatic diagnosis step (f), diagnosis is performed using not only the likelihood in the divided region, but also, for example, secondary features such as statistics and texture features derived from the likelihood, and position information. Can be included.

  A computer program for performing image processing according to another aspect of the present invention includes: (a) a step of storing an image of an object in a memory; and (b) a feature for each pixel of an image read from the memory. A step of obtaining an amount, (c) a step of obtaining a posterior probability for each pixel from a feature amount for each pixel using a predetermined discriminant function, and (d) converting the posterior probability for each pixel into a luminance value, A likelihood map of the image of (b), (e) a step of dividing the likelihood map into a plurality of regions each including a pixel having a high similarity in luminance value, and (f) automatic for each divided region Performing a diagnosis, and (g) displaying a likelihood map after division or automatic diagnosis on a display.

  According to the present invention, a plurality of regions including pixels with high luminance value similarity are individually diagnosed and automatically reflected in a likelihood map that is converted into luminance values, reflecting texture characteristics in units of pixels of an image of an object. Diagnosis can be performed. Further, by looking at the likelihood map, various types of recognition and determination in each region can be easily performed visually.

It is a figure which shows the structure of the image processing system of one Embodiment of this invention. It is a figure which shows the structure of the image processing apparatus of one Embodiment of this invention. It is a figure which shows the flow of the image processing method of one Embodiment of this invention. It is a figure which shows the flow of the image processing method of one Embodiment of this invention. It is a figure which shows the HLAC mask pattern of one Embodiment of this invention. It is a figure which shows the HLTI mask pattern of one Embodiment of this invention. It is a figure which shows the extended mask pattern corresponding to the multi sequence of one Embodiment of this invention. It is a figure which shows the image at the time of calculating | requiring a likelihood map from the image of one Embodiment of this invention. It is a figure which shows the likelihood map after the division | segmentation of one Example of this invention.

  Embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a diagram showing a configuration of an image processing system according to an embodiment of the present invention. The image processing system 1 includes an image acquisition device 2 that acquires an image of an object, and an image processing device 4 that can communicate with the device 2 via a communication line 3. The image acquisition device 2 may basically be any device that can obtain an image of an object as a set of pixel data (luminance information). For example, an imaging device such as a camera that captures an image of the object, medical diagnosis Alternatively, it may include an X-ray imaging apparatus and an ultrasonic diagnostic apparatus used for non-destructive inspection and the like, and a CT scan, MRI, or PET mainly for obtaining a tomographic image for medical diagnosis. The image acquisition device 2 collectively stores various image data from other image capturing devices connected to be communicable, and stores one data for outputting the image data designated by the image processing device 4. It may be a storage (storage device).

  The communication line 3 includes a wired or wireless communication path. In order to send image data from the image acquisition device 2 to the image processing device 4 and various control signals or image processing results from the image processing device 4 to the image acquisition device 2. Used to send. Note that the image processing apparatus 4 can be built in or attached to the image acquisition apparatus 2 as a part of the image acquisition apparatus 2, and in this case, the communication line 3 is used in a form embedded in the image acquisition apparatus 2. It is done.

  The image processing apparatus 4 has a function for executing the image processing method of the present invention by predetermined software, and can be configured as one computer (PC) as shown in FIG. 2, for example. The image processing apparatus (PC) 4 includes an arithmetic processing unit (CPU) 40, a storage unit 41, and various I / Fs 43 connected to each other via a bus 42. The various I / Fs 43 are used as a generic term including an input I / F, an output I / F, an external storage I / F, an external communication I / F, and the like. Each I / F is connected to input / output means 44 such as a corresponding keyboard, mouse, and communication port, display means 45 such as a CRT or LCD, external storage means 46 such as a USB-connected semiconductor memory or HDD, and the like. The storage unit 41 can include a semiconductor memory such as a RAM and a ROM, an HDD, and the like.

  In the image processing apparatus (PC) 4, the image data from the image acquisition apparatus 2 is stored in the storage means 41 or the external storage means 46 via the input / output means 44 such as a communication port. The arithmetic processing unit (CPU) 40 executes predetermined software, reads the stored image data, and processes the image data according to an image processing flow described below in detail. The processing result is stored in the storage means 41 or the external storage means 46 and then displayed as an image on the display means 45.

  FIG. 3 is a diagram showing a flow of the image processing method according to the embodiment of the present invention. The processing flow of FIG. 3 is executed by various corresponding software (computer programs) in the image processing system illustrated in FIGS. 1 and 2, for example. In the following description, each processing flow will be described by taking an example of processing a tomographic image (prostate tomographic image) captured by MRI. Note that basically the same processing flow can be applied to processing other types of images.

  In step S1 of FIG. 3, an image of the object is acquired. Specifically, in this example, a tomographic image of the prostate imaged by MRI which is the image acquisition device 2 is acquired. In imaging of a tomographic image of the prostate by MRI, it is possible to acquire a plurality of different types of tomographic images by changing the pulse sequence of the magnetic field applied to the prostate to obtain a magnetic resonance image. For example, there are a T2-weighted image (T2WI), a diffusion-weighted image (DWI), a diffusion coefficient image (ADC), a dynamic contrast image (DCEI), a T1-weighted image (T1WI), a diffusion kurtosis image (DKI), and the like. In this example, as a tomographic image of the prostate, at least two types of images, for example, a T2-weighted image (T2WI) and a diffusion coefficient image (ADC) are acquired. The acquired tomographic image of the prostate is stored in the storage unit 41 or the external storage unit 46 of the image processing apparatus (PC) 4.

  In step S2, a feature amount for each pixel of the image is obtained. In this example, the feature amount is specifically obtained as follows. A tomographic image of the prostate by MRI includes the luminance value of each pixel in the image as information (data) as image data. Since the luminance value of this pixel does not include texture information (features), the boundary of the luminance value does not necessarily coincide with the boundary of the texture. Therefore, in the present invention, as a method of expressing the texture information for each pixel, the higher-order local auto-correlation feature (HLAC) is expanded and various features other than auto-correlation are added. Use features that can represent textures. This extended feature quantity is referred to herein as higher-order local textural information (HLTI). Furthermore, in the conventional HLAC, where the feature amounts are used in the image, the HLTI is calculated in units of pixels in the present invention.

  The higher-order local autocorrelation feature (HLAC) is a feature amount that represents a local shape feature, is position-invariant, is robust against noise, and has a relatively small calculation load. When the mask size is 3 × 3, the HLAC is calculated by adding the product of the luminance values of pixels that apply to the black portions of the 35 types of mask patterns shown in FIG. 5 while scanning the mask. Note that the numbers (2, 3) of reference points in the mask patterns M26 to M35 in FIG. 5 mean the number of multiplications of luminance values of the same pixel.

ただし、参照点に乗算数が付記されているマスクパターン（図６のＭ４０〜Ｍ４７）は下記（２）のように定義する。

参照点が１点のみであるマスクパターンである図６のＭ１、Ｍ３８、Ｍ３９の場合、Auto-Corrrelationはそれぞれｉ、ｉ²、ｉ³とする。
Contras：

Homogeneity：

The higher-order local texture information amount (HLTI) employed in the present invention adds Contrast and Homogeneity using a difference in pixel luminance values in addition to Auto-Correlation that multiplies pixel luminance values based on this HLAC. Assuming that the luminance values of reference points on the mask pattern are i, j, and k (i, j for 2 points, i for 1 point), Auto-Correlation, Contrast, and Homogeneity are expressed by the following equation (1). It is calculated as (7). However, in the case of a mask pattern with one reference point (M1, M38, M39 in FIG. 6), Contrast and Homogeneity are not defined.
Auto-Correlation:

However, the mask pattern (M40 to M47 in FIG. 6) in which the multiplication number is added to the reference point is defined as (2) below.

In the case of M1, M38, and M39 in FIG. 6, which are mask patterns having only one reference point, Auto-Corrrelation is set to i, i ² , and i ³ , respectively.
Contras:

Homogeneity:

  When the feature amounts are added in the region as in the conventional HLAC calculation, overlapping patterns are excluded by mask scanning. In the present invention, since the feature amount is calculated for each pixel, it is necessary to take these patterns into consideration. Therefore, the high-order local texture information amount (HLTI) mask patterns employed in the present invention are 47 types of patterns M1 to M47 shown in FIG. Reference numerals (2, 3) in the mask patterns M38 to M47 in FIG. 6 mean the number of multiplications of the luminance values of the same pixel. However, for the calculation of Contrast and Homogeneity, the patterns M1 and M38 to M47 that take the same point in the mask pattern are excluded.

  Furthermore, since the present invention uses two types of images with different pulse sequences, a T2-weighted image (T2WI) and a diffusion coefficient image (ADC), an extended mask pattern of multi-sequential HLTI (msHLTI) corresponding to these images is used. Use. FIG. 7 shows an extended mask pattern of msHLTI between T2WI and ADC. There are eleven types of extended mask patterns F1 to F11, and each type includes a pattern T2 corresponding to a T2-weighted image and a pattern ADC corresponding to a diffusion coefficient image. The reference point number (2) in the F3 pattern ADC means the number of multiplications of the luminance value of the same pixel.

  Generally, ADC has a lower resolution than T2WI (for example, T2WI: 512 × 512 px, ADC: 128 × 128 px), and there is a large difference in the occupied area per pixel. exclude. Contrast and Homogeneity exclude the F2 and F3 patterns. As a result of this extension, the msHLTI mask pattern is increased to a total of 61 types of mask patterns including the ADC HLTI 47 patterns, the ADC M1, M38, and M39 patterns, and the extended 11 patterns (FIG. 4). . The parameters of the above equations (1) to (6) are calculated while scanning the 61 types of mask patterns of msHLTI, and the T2 weighted image (T2WI) and the diffusion coefficient image (ADC) are calculated for each pixel. The feature amount (feature vector) of msHLTI is obtained. Further, the mask pattern can be further enlarged by changing the distance (correlation width) between the center pixel and the edge pixel among the reference points in the mask pattern.

Next, in step S3 of FIG. 3, a posterior probability (likelihood) for each pixel is calculated using a predetermined discriminant function from the obtained feature amount (feature vector) for each pixel. Here, the posterior probability (likelihood) of each pixel, means a probability of belonging to the pixel (corresponding feature vectors) in the feature space class (region) R _m. With this posterior probability (likelihood) for each pixel, it is possible to clarify which region each pixel including texture information belongs to and its boundary. For this purpose, it is necessary to obtain a function in advance. A posteriori probability (likelihood) for each pixel is calculated from the obtained feature quantity (feature vector) for each pixel using the previously obtained discrimination function.

  FIG. 4 shows a flow of a method for obtaining the discrimination function. In the present invention, the discriminant function is obtained by a so-called machine learning method (kernel learning method) using teacher data. As the machine learning method, for example, a support model machine (SVM), a neural network, or a mathematical model (algorithm) such as AdaBoost can be used. In step S10, teacher data of an object image is acquired. The teacher data of the image of the target object is acquired in the same manner as in step S1 of FIG. 3, and in the tomographic image of the prostate imaged by the MRI that is the image acquisition device 2 in this example, that is, the target object image. This means an image in which a lesion (lesion) is specified and labeled for each pixel. The captured tomographic image of the prostate includes two types of images with different pulse sequences, a T2-weighted image (T2WI) and a diffusion coefficient image (ADC).

  In step S11 of FIG. 4, the feature amount for each pixel of the teacher data of the acquired object image is calculated. For the calculation, multi-sequential HLTI (msHLTI) is calculated in units of pixels using scanning of a total of 61 types of mask patterns, for example, as in step S2 of FIG. In step S12, a discrimination function is acquired by a machine learner using the feature quantity (feature vector) for each pixel of the obtained teacher data. At that time, each class (region) on the feature space can be labeled based on the presence or absence of the lesion (lesion) by the teacher data including information on the lesion (lesion). Note that since the discriminant function is a function for obtaining the posterior probability, it can also be called a posterior probability conversion function. As the machine learner, the above-described support vector machine (SVM), Fisher's linear discriminant analysis, neural network, or a mathematical model (algorithm) such as AdaBoost can be used.

  Returning to FIG. 3, a likelihood map is created in step S4. The likelihood map is obtained by converting the posterior probability for each pixel obtained in step S3 into a corresponding luminance value and expressing the luminance distribution of the entire tomographic image. That is, the posterior probability for each pixel reflecting the texture feature of the object is represented as a luminance value on the corresponding image. The created likelihood map is stored in the storage unit 41 or the external storage unit 46 of the image processing apparatus (PC) 4. The likelihood map can be displayed as an image on the display means 45 of the image processing apparatus (PC) 4.

FIG. 8 shows an image of the processing flow from steps S1 to S4 in FIG. 3 described above. For each pixel 10 of the image obtained in step S1 in the lower left of FIG. 8, feature vectors (M1, M2,..., M47,..., H2,..., H37, C2,. C37, F _M 1, determined _{··, F M 11, F H} 1, F H 4, ··, F H 11, F C 1, F C 4, ··, the F _C 11). In step S3, the feature vector is subjected to an identification function (a posteriori probability conversion function) to obtain a posteriori probability (likelihood) for each pixel. In step S4, the posterior probability (likelihood) for each pixel is converted into a luminance value 10 ', and a likelihood map for the entire lower right image in FIG. 8 is created. It is also possible to calculate the posterior probability as a pixel value having a plurality of channels by calculating as a likelihood vector using a plurality of posterior probability conversion functions.

  In step S5 of FIG. 3, the obtained likelihood map is divided into a plurality of regions each including a pixel having a high similarity in luminance value. For example, a superpixel method can be used for the division. The superpixel method is an image region using a k-nearest neighbor method in a three-dimensional space (coordinates of an xy coordinate and a color space (R, G, B) in a color image) of coordinates and luminance values on an xy plane of an image. This is a division method, and the similarity of the luminance values of the accommodation pixels in the superpixel is increased, so that the selectivity of the lesion (lesion) tissue can be increased.

  In step S6, the divided likelihood map is stored in the storage means 41 or the external storage means 46 of the image processing apparatus (PC) 4 and can be displayed as an image on the display means 45. In the next step S7, the likelihood map after division is automatically diagnosed, and a region having a lesion (prostate cancer in this example) is specified (localized diagnosis). The determination can also be obtained by an algorithm that can be added to the method of the present invention. Specifically, for example, a region where the average likelihood (probability) in each divided region is 0.5 or more can be diagnosed as cancer. In this case, machine learning is used for each region using other statistics such as median, variance, kurtosis, and skewness, position information, shape, and secondary features that extract texture features from likelihood maps. You can also make a diagnosis. Furthermore, the diagnosis result can be redisplayed on the image of the likelihood map after the division displayed on the display means 45 so that the cancerous part can be recognized by color coding or the like. The above is the description of each flow of the method of the present invention.

  An embodiment of the present invention will be described with reference to FIG. FIG. 9 is a diagram showing a likelihood map after division according to an embodiment of the present invention. FIG. 9 shows likelihood maps after division of three prostate tomographic images (a), (b), and (c) obtained by the above-described method flow of the present invention. In this embodiment, SVM (RBF kernel) is used as a machine learning method for obtaining the posterior probability / discriminant function, and the superpixel method is used for dividing the likelihood map. The upper part of the figure shows the result of automatic diagnosis of prostate cancer using the average likelihood value in the region. In the upper diagrams, automatic cancer diagnosis is performed for each region, and regions diagnosed with cancer are connected to regions that are not. The lower part of the figure shows each region in the prostate after division. A plurality of areas divided by black lines in each figure in the lower stage are areas obtained by dividing the likelihood map. That is, in each image, a region having a high probability of containing prostate cancer cells and a region not having the same are separated by a black line. The doctor can determine the presence or absence of cancer from each image.

  Embodiments of the present invention have been described with reference to the drawings. However, the present invention is not limited to these embodiments. Furthermore, the present invention can be implemented in variously modified, modified, and modified forms based on the knowledge of those skilled in the art without departing from the spirit of the present invention.

1: Image processing system 2: Image acquisition device 3: Communication line 4: Image processing device 10: Pixel 40: Arithmetic processing device (CPU)
41: Storage means 42: Bus 43: Various I / F
44: Input / output means 45: Display means 46: External storage means

Claims (10)

An image processing method comprising:
Obtaining an image of the object;
Obtaining a feature value for each pixel of the image;
Obtaining a posterior probability for each pixel using a predetermined discriminant function from the feature quantity for each pixel;
Converting the posterior probability for each pixel into a luminance value, and creating a likelihood map of the image of the object;
Dividing the likelihood map into a plurality of regions each including a pixel having a high similarity in luminance value;
A step of performing automatic diagnosis for each divided area;
An image processing method including: The step of obtaining a feature amount for each pixel of the image includes:
The image processing method according to claim 1, further comprising: obtaining higher-order local texture information for each pixel by mask scanning of the image using a predetermined mask pattern. The step of obtaining the posterior probability for each pixel includes:
The other image of the object is used as teacher data, and higher-order local texture information for each pixel is input to a discrimination function obtained by a machine learner using a feature amount for each pixel of the teacher data. The image processing method according to claim 2, further comprising a step of obtaining a posterior probability for each pixel.   The image processing method according to claim 3, wherein the machine learner includes a support vector machine.   The step of dividing the likelihood map into a plurality of regions each including a pixel having a high similarity in luminance value includes a step of dividing the likelihood map into a plurality of regions using a superpixel method. The image processing method in any one of -4.   The image processing method according to claim 2, wherein the step of acquiring an image of the object includes a step of acquiring at least two types of images by photographing the object under different conditions.   The image processing method according to claim 6, wherein the predetermined mask pattern includes a mask pattern selected to correspond to the at least two types of images.   The image processing method according to claim 1, wherein the step of acquiring an image of the object includes a step of acquiring a tomographic image of a part of a human body as the image.   The image processing method according to claim 8, wherein the tomographic image of the part of the human body is imaged using CT, MRI, or PET. A computer program for performing image processing on a computer,
Storing an image of the object in memory;
Obtaining a feature value for each pixel of the image read from the memory;
Obtaining a posterior probability for each pixel using a predetermined discriminant function from the feature quantity for each pixel;
Converting the posterior probability for each pixel into a luminance value, and creating a likelihood map of the image of the object;
Dividing the likelihood map into a plurality of regions each including a pixel having a high similarity in luminance value;
A step of performing automatic diagnosis for each divided area;
Displaying the likelihood map after division or after automatic diagnosis on a display;
A computer program that runs

Images