WO2019001208A1 - Segmentation algorithm for choroidal neovascularization in oct image - Google Patents

Abstract

Disclosed in the present invention is a segmentation algorithm for choroidal neovascularization in an OCT image, comprising the following steps: S01: designing a structure priori learning method for trained images, and constructing a structure priori matrix, the structure priori matrix being used for differentiating a choroidal neovascularization area and a background area; S02: converting, on the basis of the structure priori matrix, an OCT original image into a significance enhanced image for enhancing the significance of the choroidal neovascularization area; S03: using a multi-scale analysis on the significance enhanced image to divide the significance enhanced image into m scales; S04: performing training on the basis of each scale and obtaining m well-trained convolutional neural network models; S05: processing a test image by using steps S01, S02 and S03, and performing a test using the convolutional neural network models well-trained in step S04 to output m segmentation results, and fusing the m segmentation results to obtain a final segmentation result. The present invention can significantly improve the accuracy of segmentation of choroidal neovascularization in an OCT image.

Description

A choroidal neovascularization algorithm in OCT images Technical field

The invention relates to a choroidal neovascularization algorithm in an OCT image, and belongs to the technical field of retinal image segmentation.

Background technique

The existing auto-segmentation techniques for choroidal neovascularization are mostly based on fundus fluorescein angiography images. Compared with fundus fluorescein angiography, OCT images have the advantages of non-invasive, high-speed, high-resolution, three-dimensional imaging, etc., which has more important clinical significance for the auxiliary diagnosis of common clinical ophthalmic diseases such as age-related macular degeneration.

There is currently no choroidal neovascularization algorithm based on OCT images. Choroidal neovascularization in OCT images faces many challenges: large texture changes, different grayscale quality, inconsistent shape and size, blurred boundaries, and a large number of plaque noise. These problems make it difficult to achieve a more accurate segmentation effect by traditional methods. Convolutional neural networks have powerful learning capabilities, and have achieved great success in medical image segmentation (for example, gray matter segmentation of MR brain images, segmentation of cell membranes under electron microscopy, mitosis detection of breast pathology images, etc.). Therefore, this framework is considered for use in the choroidal neovascularization task in OCT images. However, due to the complex nature of OCT images, there are two shortcomings in the use of convolutional neural networks to segment choroidal neovascularization: (1) The traditional method first needs to divide the image into several patches and then train the convolution based on patch. Neural network segmentation model. However, there are some structural correlations (such as local similarity) between patch and patch. The traditional method does not consider this effective structural information when training the convolutional neural network segmentation model, thus limiting the segmentation performance of the model. (2) The traditional model is built on a single scale patch. The size of the choroidal neovascularization is not fixed, so it is difficult for a single-scale patch to obtain effective context information, thus affecting the segmentation accuracy.

Summary of the invention

The technical problem to be solved by the present invention is to provide a choroidal neovascularization algorithm in an OCT image based on a multi-scale structured prior convolutional neural network capable of improving the segmentation accuracy of choroidal neovascularization in an OCT image.

In order to solve the above technical problems, the technical solution adopted by the present invention is:

An choroidal neovascularization algorithm in an OCT image includes the following steps:

S01: Design a structural prior learning method for the training image, and construct a structural prior matrix, which is used to distinguish the choroidal neovascular region from the background region, and includes the following steps:

a: superpixel segmentation of the training image to obtain a number of superpixels;

b: extracting features;

c: marking each of the superpixels into two categories according to the correct labeling of the training image, respectively a choroidal neovascular region and a background region;

d: using a number of the superpixel construction dictionary marked;

e: classifying all the superpixels to obtain a global structure prior graph;

f: calculating a local similarity structure prior to the prior structure of the global structure, and obtaining the prior matrix of the structure;

S02: converting the original OCT image into a significant enhancement image based on the structural prior matrix for enhancing the saliency of the choroidal neovascular region;

S03: using a multi-scale analysis on the significant enhancement image, dividing the significant enhancement image into m scales;

S04: obtaining m trained convolutional neural network models based on each scale training;

S05: Perform superpixel segmentation on the test image, extract feature, classify, and calculate a structure prior matrix by using steps a, b, e, and f in step S01, and use the structure prior matrix pair test image obtained in S05 according to step S02. Image conversion is performed, and the converted test image is divided into m scales by using step S03, and the convolutional neural network model trained in step S04 is used for testing, and m segmentation results are output, and m segments are merged. This is the final segmentation result.

Subpixel segmentation using the SLIC algorithm.

The features include an average gray value for each of the superpixels, a texture feature based on a co-occurrence matrix, and a local grayscale feature.

Using the K-means algorithm to construct a dictionary, assuming that there are N patient data in the training set, each data includes 2 categories, and K-means is used to form K class, then the data of N patients can be aggregated into 2KN class, and 2KN is obtained. Cluster centers, which form a dictionary D, as shown in equation (1):

D=[C _1,1 ,C _1,2 ... C _1,K ,B _1,1 ,B _1,2 ,..B _1,K, ...C _n,k ..B _n,k ...C _{N, 1} ,..C _N,K ,B _N,1 ,..B _N,K ] (1)

Where C _n,k represents the kth cluster center from the choroidal neovascular region of the nth patient; B _n,k represents the kth cluster center from the background region of the nth patient, n=1 , 2, ... N, k = 1, 2, ... K.

Each of the superpixels is classified using a sparse representation, and the classification process is formalized as shown in the formula (2), where x is a required sparse coefficient, and y is the superpixel;

Arg min _x ||x|| ₁ subject to Dx=y (2)

Equation (2) is solved using the SLEP toolbox to obtain the solution of x; the classification result of the superpixel is obtained using equation (3), where x _i represents the sparse coefficient of the i-th class, i=1, 2, ... 2KN ;

r _i (y)=||y-Dx _i || ₂ (3)

Calculating 2KN r _i (y) according to formula (3). When the value of r(y) is the smallest, the category at this time is the category of the superpixel; for all the superpixel classifications of each image, ie A global spatial structure prior view is available.

Based on the global spatial structure prior graph, the Gaussian probability density function is used to calculate the local similar structure prior, as shown in the following equation:

M(a,b)=exp(-(cor(a,b)-c) ² /u ² ) (4)

Where cor(a,b) is the coordinate of each point in the global spatial structure prior view, c is the center of the choroidal neovascularization in the global spatial structure prior view, u is the first global spatial structure The radius of the choroid in the graph is obtained by averaging the distance between the center c and the choroid boundary point, and the obtained matrix M is the structural prior matrix.

The image is transformed using the structure prior matrix M, and the conversion formula is as shown in equation (5):

I _s =MI ₀ (5)

Where I ₀ is the original image and I _s is the significant enhancement image.

The fusion method in step S05 employs a maximum voting criterion.

The beneficial effects achieved by the invention are that the structural prior matrix can acquire the structural correlation information between the patches, and the structural prior learning method is used to calculate the structural prior matrix in the image, which can be used to enhance the choroidal neovascular region and the background region. The correlation between patches, multi-scale analysis can obtain effective context information of different scales, and further improve the segmentation accuracy of choroidal neovascularization in OCT images.

DRAWINGS

1 is a flow chart of choroidal neovascularization based on a multi-scale structured prior convolutional neural network;

Figure 2 is a typical convolutional neural network framework diagram;

Fig. 3 is a view showing an example of choroidal neovascularization.

Detailed ways

The invention is further described below in conjunction with the drawings. The following examples are only intended to more clearly illustrate the technical solutions of the present invention, and are not intended to limit the scope of the present invention.

Example 1

A choroidal neovascularization algorithm in an OCT image, as shown in Fig. 1, the method mainly comprises two stages: a training phase and a testing phase, and the specific steps are as follows:

(1) The training phase mainly includes two parts: “structure prior learning” and “multi-scale structure prior convolutional neural network training”. The specific steps are as follows:

S01: designing a structural prior learning method for the training image, and constructing a structural prior matrix for distinguishing the choroidal neovascular region and the background region;

The structural prior learning method includes the following steps:

a: super-pixel segmentation of the training image to obtain a number of super-pixel regions, using SLIC algorithm for segmentation;

b: extracting features including average gray value of each of the superpixels, texture features based on a co-occurrence matrix, and local grayscale features;

c: marking each of the superpixels into two categories according to the correct labeling of the training image, respectively a choroidal neovascular region and a background region;

d: using the labeled super-pixel construction dictionary, using the K-means algorithm to construct the dictionary, assuming that there are N patient data in the training set, each data includes 2 categories, and K-means is used to form K, then The data of N patients can be aggregated into 2KN classes, and 2KN cluster centers are obtained. The cluster centers form a dictionary D, as shown in formula (1):

D=[C _1,1 ,C _1,2 ... C _1,K ,B _1,1 ,B _1,2 ,..B _1,K, ...C _n,k ..B _n,k ...C _{N, 1} ,..C _N,K ,B _N,1 ,..B _N,K ] (1)

Wherein C _n,k represents the kth cluster center from the choroidal neovascular region of the nth patient; B _n,k represents the kth cluster center from the background region of the nth patient;

e: classifying all the superpixels, obtaining a global structure prior, and classifying each of the superpixels using a sparse representation, and the classification process is formalized as shown in formula (2), where x is a required sparseness a coefficient, y is the superpixel;

Arg min _x ||x|| ₁ subject to Dx=y (2)

Equation (2) is solved using the SLEP toolbox to obtain the solution of x; the classification result of the superpixel is obtained using equation (3), where x _i represents the sparse coefficient of the i-th class, i=1, 2, ... 2KN ;

r _i (y)=||y-Dx _i || ₂ (3)

Calculating 2KN r _i (y) according to formula (3). When the value of r(y) is the smallest, the category at this time is the category of the superpixel; for all the superpixel classifications of each image, ie A global spatial structure prior view is available;

f: Based on the global structure prior, the Gaussian probability density function is used to calculate the local similar structure prior, and the structural prior matrix is obtained, and the local potential function is used to calculate the local similar structure prior, as shown in the following formula:

M(a,b)=exp(-(cor(a,b)-c) ² /u ² ) (4)

Where cor(a,b) is the coordinate of each point in the global spatial structure prior view, c is the center of the choroidal neovascularization in the global spatial structure prior view, u is the first global spatial structure The radius of the choroid in the map is obtained from the average of the distance between the center c and the choroid boundary point. The obtained matrix M is the structural prior matrix. Figure 1 shows the structure prior view, which can be seen from the map. The elements in the local region of the matrix have similar values. From a global perspective, the choroidal neovascularization and the elements in the background region have significantly different values. Therefore, the prior matrix should be used to improve the segmentation model. The distinction between the background area and the target area.

S02: Converting the original OCT image into a significant enhancement image based on the structural prior matrix for enhancing the saliency of the choroidal neovascular region, and converting the image using the prior matrix M of the structure, and converting the formula as a formula (5) ):

I _s =MI ₀ (5)

Where, I ₀ is the original image, and I _s is the significant enhancement image. As shown in Figure 1, after image conversion, the smear of the choroidal neovascularization is indeed enhanced, because the significant enhancement image incorporates the structure prior. information;

S03: using a multi-scale analysis on the significant enhancement image, dividing the significant enhancement image into m scales;

S04: Obtaining m trained convolutional neural network models based on each scale training, extracting patches for images for each scale, training a structure prior volume convolutional neural network model for each scale patch, and finally obtaining m trained structure prior volume convolutional neural network model, each patch extraction method: taking a pixel point p in the image as the center, extracting a square area with side length s, the pixel class mark is the patch The class tag, by classifying the patch using a convolutional neural network, completes the classification of the pixels, ie, completes the segmentation task, which uses a classical convolutional neural network, as shown in Figure 2, in a volume In the neural network, it mainly consists of an input layer, a convolution layer, a pooling layer, a fully connected layer, and an output layer. The convolution layer generally has C convolution kernels, which convolve the images separately and output different mappings. The convolutional layer can learn the local characteristics of different levels in the image. Generally, a pooling layer is added after the convolutional layer. The output of the convolutional layer is the input of the pooling layer. The pooling layer generally uses the maximum pooling method to downsample the input mapping, that is, select the neighborhood in a neighborhood. The largest point in the neighborhood represents the neighborhood. The pooling layer can reduce the size of the map, thereby reducing the computational complexity. After the subsequent layers of the convolution layer-pooling layer loop, a fully connected layer is connected, which converts all the output maps of the pooled layer into a column vector. Generally, a fully connected layer is followed by an output layer, and the output layer is connected. After a softmax function outputs the probability that each input picture belongs to each class, and selects the category with the highest probability as the input picture, the weight of the convolutional neural network is usually solved by using the random gradient descent method;

(2) Test phase

Performing super pixel segmentation, extracting features, classifying, and calculating a structure prior matrix by using steps a, b, e, and f in step S01, and performing image on the test image based on the structure prior matrix obtained in S05 by using step S02. Converting, the converted test image is divided into m scales by using step S03, and the convolutional neural network model trained in step S04 is used for testing, and m segmentation results are output, and m segmentation results are merged. For the final segmentation result, the fusion method uses the maximum voting criteria.

(3) Experimental results

The method of the invention was tested on 15 patient data with choroidal neovascularization. The feasibility and effectiveness of the method were tested by a three-fold cross-validation method.

As shown in Fig. 3, an example of eight choroidal neovascular segmentation is shown, in which the red line represents the boundary region of the choroidal neovascularization manually segmented by the expert, and the white region represents the region automatically segmented by the present invention, as can be seen from the segmentation result of Fig. 3. This method can effectively segment the choroidal neovascularization area and has less error with the manual segmentation result, using Dyce similarity coefficient (DSC), true positive rate (TPVF), false positive rate (FPVF) and p value (p-values). As an objective indicator of the evaluation method, the results are shown in Table 1 and Table 2.

Table 1 Multiscale structure prior convolutional neural networks and segmentation performance based on sparse representation

Multi-scale structure prior

Sparse expression

P-values

	Convolutional neural network
DSC	0.7806±0.067	0.4677±0.070	0.003
TPVF	0.8024±0.081	0.8751±0.091	0.2357
FPVF	0.0036±0.001	0.0265±0.007	0.0002

Table 2 Segmentation performance of multiscale structure prior convolutional neural networks and convolutional neural networks

The results show that the method of segmentation of choroidal neovascularization is significantly better than the segmentation effect based on sparse representation and convolutional neural network.

The above is only a preferred embodiment of the present invention, and it should be noted that those skilled in the art can make several improvements and modifications without departing from the technical principles of the present invention. It should also be considered as the scope of protection of the present invention.

Claims (8)

1. A choroidal neovascularization algorithm in an OCT image, characterized in that it comprises the following steps:

   S01: Design a structural prior learning method for the training image, and construct a structural prior matrix, which is used to distinguish the choroidal neovascular region from the background region, and includes the following steps:

   a: superpixel segmentation of the training image to obtain a number of superpixels;

   b: extracting features;

   c: marking each of the superpixels into two categories according to the correct labeling of the training image, respectively a choroidal neovascular region and a background region;

   d: using a number of the superpixel construction dictionary marked;

   e: classifying all the superpixels to obtain a global structure prior graph;

   f: calculating a local similarity structure prior to the prior structure of the global structure, and obtaining the prior matrix of the structure;

   S02: converting the original OCT image into a significant enhancement image based on the structural prior matrix for enhancing the saliency of the choroidal neovascular region;

   S03: using a multi-scale analysis on the significant enhancement image, dividing the significant enhancement image into m scales;

   S04: obtaining m trained convolutional neural network models based on each scale training;

   S05: performing superpixel segmentation on the test image by using steps a, b, e, and f in S01, extracting features, classifying, and calculating a structure prior matrix, and performing image on the test image by using the structure prior matrix obtained by S02 in S05. Converting, using S03, dividing the converted test image into m scales, utilizing

   The convolutional neural network model trained in S04 is tested, and m segmentation results are output, and the m segmentation results are merged to obtain the final segmentation result.
2. The choroidal neovascularization algorithm in an OCT image according to claim 1, wherein the superpixel segmentation is performed using a SLIC algorithm.
3. The choroidal neovascularization algorithm in an OCT image according to claim 2, wherein the feature comprises an average gray value of each of the superpixels, a texture feature based on a co-occurrence matrix, and a local grayscale feature.
4. The choroidal neovascularization algorithm in an OCT image according to claim 1, wherein the dictionary is constructed using the K-means algorithm, assuming that there are N patient data in the training set, each data includes 2 classifications, and K is used. The -means are grouped into K classes, and the data of N patients can be aggregated into 2KN classes, and 2KN cluster centers are obtained. The cluster centers form a dictionary D, as shown in formula (1):

   D=[C _1,1 ,C _1,2 ... C _1,K ,B _1,1 ,B _1,2 ,..B _1,K, ...C _n,k ..B _n,k ...C _{N, 1} ,..C _N,K ,B _N,1 ,..B _N,K ] (1)

   Where C _n,k represents the kth cluster center from the choroidal neovascular region of the nth patient; B _n,k represents the kth cluster center from the background region of the nth patient, n=1 , 2, ... N, k = 1, 2, ... K.
5. The choroidal neovascularization algorithm in an OCT image according to claim 1, wherein each of said superpixels is classified using a sparse representation, and the classification process is formalized as shown in formula (2), wherein x is the required sparse coefficient, and y is the super pixel;

   Arg min _x ||x|| ₁ subject to Dx=y (2)

   Equation (2) is solved using the SLEP toolbox to obtain the solution of x; the classification result of the superpixel is obtained using equation (3), where x _i represents the sparse coefficient of the i-th class, i=1, 2, ... 2KN ;

   r _i (y)=||y-Dx _i || ₂ (3)

   Calculating 2KN r _i (y) according to formula (3). When the value of r(y) is the smallest, the category at this time is the category of the superpixel; for all the superpixel classifications of each image, ie A global spatial structure prior view is available.
6. The choroidal neovascularization algorithm in an OCT image according to claim 5, wherein the local similarity structure prior is calculated by using a Gaussian probability density function based on the global spatial structure prior graph, as shown in the following formula:

   M(a,b)=exp(-(cor(a,b)-c) ² /u ² ) (4)

   Where cor(a,b) is the coordinate of each point in the global spatial structure prior view, c is the center of the choroidal neovascularization in the global spatial structure prior view, u is the first global spatial structure The radius of the choroid in the graph is obtained by averaging the distance between the center c and the choroid boundary point, and the obtained matrix M is the structural prior matrix.
7. The choroidal neovascularization algorithm in an OCT image according to claim 6, wherein the image is transformed using the structural prior matrix M, and the conversion formula is as shown in formula (5):

   I _s =MI ₀ (5)

   Where I ₀ is the original image and I _s is the significant enhancement image.
8. The choroidal neovascularization algorithm in an OCT image according to claim 1, wherein the fusion method in S05 employs a maximum voting criterion.

Images