CN114399511A - Choroid sublayer and choroid blood vessel segmentation network model and training method thereof - Google Patents
Choroid sublayer and choroid blood vessel segmentation network model and training method thereof Download PDFInfo
- Publication number
- CN114399511A CN114399511A CN202210062989.3A CN202210062989A CN114399511A CN 114399511 A CN114399511 A CN 114399511A CN 202210062989 A CN202210062989 A CN 202210062989A CN 114399511 A CN114399511 A CN 114399511A
- Authority
- CN
- China
- Prior art keywords
- choroidal
- sublayer
- choroid
- shared
- layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30101—Blood vessel; Artery; Vein; Vascular
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a choroid sublayer and choroid blood vessel segmentation network model and a training method thereof, wherein the model comprises the following steps: the shared encoder module is used for carrying out down-sampling and feature extraction on an image to be segmented to obtain a first shared feature; the shared decoder module is used for acquiring the first shared characteristic to perform upsampling to obtain a second shared characteristic; a first decoder module to extract a choroidal sublayer segmentation specificity feature in the second shared feature; a second decoder module for extracting choroidal vessel segmentation specific features in the second shared features; and the classification module is used for calculating to obtain a segmentation result of the image to be segmented according to the second shared feature, the specific feature of the choroid sublayer segmentation and the specific feature of the choroid blood vessel segmentation. By implementing the invention, the division of the choroid sublayer and choroidal vessels is achieved by the provision of the five modules described above, i.e. a multi-task division is achieved by a multi-stream network structure.
Description
Technical Field
The invention relates to the technical field of image segmentation, in particular to a choroid sublayer and choroid blood vessel segmentation network model and a training method thereof.
Background
The choroid, which is located between the retina and sclera, is one of the most vascularized structures of the human body, provides nutrients to the outer retina and plays a critical role in the human visual system. In clinical practice, Optical Coherence Tomography (OCT) can separate the choroidal structures from the interior of the retina, which is an effective method for a wide range of applications in neurology, ophthalmology, gastrointestinal and cardiac diseases. To analyze ocular diseases based on OCT, researchers have proposed quantitative choroidal biomarkers such as Choroidal Thickness (CT), Choroidal Volume (CV), Choroidal Vascular Density (CVD), Choroidal Vascular Index (CVI). Although these biomarkers quantitatively reflect the structure of the choroid, they do not work well for accurate segmentation of the choroid layer and blood vessels.
During the past decade, a number of approaches to choroidal structure segmentation based on OCT B-Scan (B-Scan) have emerged. These methods can be largely classified into graph-based methods and learning-based methods. The former generally extends the approach for retinal surface detection to the choroid, proposing new criteria to construct a graphical model and search for the choroidal surface. This method is too dependent on experimental assumptions and takes a long time, limiting its clinical application. The latter uses Deep Convolutional Neural Networks (DCNN) as a boundary feature detector or semantic classifier to perform choroid layer segmentation. Because DCNN has strong generalization ability, the methods can automatically extract the distinguishing features of the choroid layer, and the segmentation performance is improved to a great extent. However, most of the present DCNN focuses only on the division of the choroid layer. There was less research on the boundary of the choroidal sublayer and choroidal vessels.
Disclosure of Invention
In view of this, embodiments of the present invention provide a choroid sublayer and choroid blood vessel segmentation network model and a training method thereof, so as to solve the technical problem in the prior art that choroid blood vessel segmentation accuracy is poor.
The technical scheme provided by the invention is as follows:
in a first aspect, embodiments of the present invention provide a choroidal sublayer and choroidal vascular segmentation network model, comprising: the shared encoder module is used for carrying out down-sampling and feature extraction on an image to be segmented to obtain a first shared feature of choroidal sublayer segmentation and choroidal blood vessel segmentation, and the resolution of the first shared feature is lower than a threshold value; a shared decoder module, configured to obtain the first shared feature for upsampling, so as to obtain a second shared feature of the choroid sublayer segmentation and the choroid blood vessel segmentation, where a resolution of the second shared feature is higher than a threshold; a first decoder module to extract a choroidal sublayer segmentation specific feature in the second shared features; a second decoder module for extracting choroidal vessel segmentation specific features in the second shared features; and the classification module is used for calculating a segmentation result of the image to be segmented according to the second shared feature, the choroid sublayer segmentation specific feature and the choroid blood vessel segmentation specific feature.
Optionally, the shared encoder module comprises: the shared decoder comprises a preset number of sub-decoding modules, and the sub-coding modules are connected with the sub-decoding modules with the same corresponding spatial resolution.
Optionally, the sub-coding module comprises: a convolution layer, an activation function layer and a space pooling layer; the sub-decoding module includes: an upsampling layer, a convolutional layer, and an activation function layer.
Optionally, the number of decoding modules of the first decoder module and/or the second decoder module is preset, and the decoding modules include: the decoding module comprises an up-sampling layer, a convolution layer and an activation function layer, wherein the convolution layer in the decoding module is a single-layer convolution layer.
Optionally, the classification module comprises: a connection layer and a classification layer, wherein the connection layer is used for performing feature connection on the second shared features and the choroidal sublayer segmentation specific features and the choroidal vessel segmentation specific features respectively to obtain a first feature map and a second feature map; and the classification layer is used for classifying according to the first characteristic diagram and the second characteristic diagram to obtain a segmentation result of the image to be segmented.
Optionally, the loss function of the segmented network model is a multitask loss function with a regularization term.
Optionally, the multitask loss function with the regularization term is represented by the following formula:
wherein, P1Representing a choroidal vascular probability map, P2Representing the choroidal sublayer probability map, Y1Representing the actual choroidal vessel mask map, Y2A diagram of the actual choroidal sublayer mask is shown,
and expressing a cross entropy loss function, wherein phi is a regularization constraint term, and lambda is a preset parameter.
In a second aspect, the present invention provides a method for training a choroid sublayer and choroidal vessel segmentation network model, including: acquiring a training sample set; inputting the training sample set into the choroid sublayer and choroid blood vessel segmentation network model according to any one of the first aspect and the first aspect of the embodiments of the present invention, and training to obtain a pre-trained choroid sublayer and choroid blood vessel segmentation network model.
A third aspect of embodiments of the present invention provides a computer-readable storage medium having stored thereon computer instructions for causing a computer to perform a method of training a model of a choroidal sub-layer and choroidal vessel segmentation network according to the second aspect of embodiments of the present invention.
A fourth aspect of an embodiment of the present invention provides an electronic device, including: a memory and a processor, the memory and the processor being communicatively coupled to each other, the memory storing computer instructions, the processor performing the method of training a choroidal sub-layer and choroidal vessel segmentation network model according to the second aspect of the embodiments of the present invention by executing the computer instructions.
The technical scheme provided by the invention has the following effects:
according to the choroid sublayer and choroid blood vessel segmentation network model provided by the embodiment of the invention, the shared decoder module and the shared decoder module are arranged, so that the extraction of the shared characteristics of choroid sublayer segmentation and choroid blood vessel segmentation can be realized; meanwhile, the first decoder module and the second decoder module are arranged to respectively extract the specific features of the choroid sublayer segmentation and the choroid blood vessel segmentation in the shared features, and finally the classification module is arranged to realize the pixel classification of the choroid sublayer and the choroid blood vessel based on the shared features and the specific features. Therefore, the segmentation network model realizes the segmentation of the choroid sublayer and the choroid vessels by the shared decoder module, the first decoder module, the second decoder module and the classification module, namely, realizes the multi-task segmentation by a multi-stream network structure.
According to the training method of the choroid sublayer and choroid blood vessel segmentation network model provided by the embodiment of the invention, the segmentation network model adopting the shared decoder module, the first decoder module, the second decoder module and the classification module is trained, so that the obtained pre-trained model can directly classify images to be segmented, and the pixel classification result of the choroid sublayer and choroid blood vessel segmentation is obtained. The network model is provided with a shared decoder module, a first decoder module, a second decoder module and a classification module structure, and multi-task segmentation is realized through a multi-stream network structure.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a block diagram of a choroidal sub-layer and choroidal vascular segmentation network model, according to an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of a computer-readable storage medium provided according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
As described in the background, there is an increasing need for quantitative choroidal analysis by clinicians, and the diagnosis and study of choroidal or chorioretinal diseases is increasingly dependent on OCT image analysis. In most cases, the measurement can be performed only manually, which is time-consuming, labor-consuming and not suitable for large-scale application and research. However, the choroidal results are measured manually in clinic, so that the choroidal results have high subjectivity, and the standard deviation exists among different doctors, so that the quantitative comparison of the diagnosis and treatment of diseases is not facilitated. Meanwhile, the accuracy of choroidal segmentation results is more demanding on the experience of the doctor, and also brings challenges to clinical analysis of the choroid. In addition, most of the analysis software of the existing OCT manufacturers in the market has no automatic division function of choroid boundaries and blood vessels, and individual manufacturers only have automatic division of the upper and lower choroid boundaries, and the accuracy is low.
Also, by analyzing the prior art, choroidal sub-layer boundaries and choroidal vessel segmentation are a more challenging task than choroidal layer segmentation, since the vessels are densely distributed between the choroid. Although these vessels are relatively irregular in shape, it can be observed that the distribution of choroidal vessels is highly correlated with the location of the choroidal sublayer. All choroidal vessels are located between Bruch's Membrane (BM) and the choroid-scleral Interface (CSI), which means that the result of choroidal sublayer segmentation can be a prerequisite for choroidal vessel segmentation. Furthermore, large choroidal vessels are located in the lower-middle part of the choroidal sublayer, while almost all choroidal microvessels (small choroidal vessels) are located near the BM, the upper boundary of the choroidal sublayer. This property also contributes to improving the rationality of the vessel segmentation result.
Considering the correlation between the choroidal vessels and the choroidal sublayer, a single DCNN structure that can jointly segment both can be used to aid choroidal analysis. However, when they are jointly split, antagonism also exists. The purpose of the choroidal sublayer segmentation is to obscure the internal structure of the choroid, making it easily separable from the retina layer and background. While choroidal vessel segmentation tends to preserve its internal structure to better identify choroidal vessels and choroidal stroma.
Compared with a single-task learning strategy, namely two different learning modes are strictly trained aiming at different tasks, the embodiment of the invention provides a choroid sublayer and choroid blood vessel segmentation network model, the segmentation network model adopts a segmentation production line of OCT B-Scan choroid analysis, and the production line can automatically segment the choroid sublayer and the choroid blood vessels, namely, a complete CNN is trained to complete all tasks.
In the present embodiment, a choroidal sublayer and choroidal blood vessel segmentation network model is provided, which may be implemented in an electronic device, and fig. 1 is a block diagram of a choroidal sublayer and choroidal blood vessel segmentation network model according to an embodiment of the present invention, as shown in fig. 1, the segmentation network model comprising: the shared encoder module is used for carrying out down-sampling and feature extraction on an image to be segmented to obtain a first shared feature of choroidal sublayer segmentation and choroidal blood vessel segmentation, and the resolution of the first shared feature is lower than a threshold value; a shared decoder module, configured to obtain the first shared feature for upsampling, so as to obtain a second shared feature of the choroid sublayer segmentation and the choroid blood vessel segmentation, where a resolution of the second shared feature is higher than a threshold; a first decoder module to extract a choroidal sublayer segmentation specific feature in the second shared features; a second decoder module for extracting choroidal vessel segmentation specific features in the second shared features; and the classification module is used for calculating a segmentation result of the image to be segmented according to the second shared feature, the choroid sublayer segmentation specific feature and the choroid blood vessel segmentation specific feature.
According to the choroid sublayer and choroid blood vessel segmentation network model provided by the embodiment of the invention, the shared decoder module and the shared decoder module are arranged, so that the extraction of the shared characteristics of choroid sublayer segmentation and choroid blood vessel segmentation can be realized; meanwhile, the first decoder module and the second decoder module are arranged to respectively extract the specific features of the choroid sublayer segmentation and the choroid blood vessel segmentation in the shared features, and finally the classification module is arranged to realize the pixel classification of the choroid sublayer and the choroid blood vessel based on the shared features and the specific features. Therefore, the segmentation network model realizes the segmentation of the choroid sublayer and the choroid vessels by the shared decoder module, the first decoder module, the second decoder module and the classification module, namely, realizes the multi-task segmentation by a multi-stream network structure.
In order to achieve accurate positioning and improve efficiency, the shared encoder module, the shared decoder module, the first decoder module and the second decoder module can all adopt a U-shaped structure, namely, Network structures such as U-Net, AttU-Net (Attention U-shaped Network structure), R2U-Net (recursive Residual Convolutional Neural Network) and the like can be adopted. Specifically, the common shared encoder module, the shared decoder module, the first decoder module, and the second decoder module are all convolution structures. The shared encoder module is used for extracting the discriminative features of the image based on the input image, in the process, the resolution of the image is reduced to 1/4, namely the resolution of the first shared feature extracted by the shared encoder is lower, and the shared decoder module is used for restoring the spatial resolution, and mapping the discriminative features of the higher layer into the spatial pixels of each position, namely the resolution of the second shared feature extracted by the shared decoder is higher. Meanwhile, the first decoder module and the second decoder module are also used for restoring the spatial resolution and mapping the high-level discriminating characteristic to the spatial pixel of each position.
In one embodiment, the shared encoder module comprises: the shared decoder comprises a preset number of sub-decoding modules, and the sub-coding modules are connected with the sub-decoding modules with the same corresponding spatial resolution. Wherein the sub-encoding module comprises: a convolution layer, an activation function layer and a space pooling layer; the sub-decoding module includes: an upsampling layer, a convolutional layer, and an activation function layer. Specifically, when the sub-coding modules and the sub-decoding modules are connected, jump connection between symmetrical layers is adopted, namely, the output characteristic of the first sub-coding module and the input characteristic of the last sub-decoding module are connected in characteristic dimension, so that fusion of two parts of characteristics is realized through convolution layers in the two modules.
The positioning can be enhanced by a hopping connection between the shared encoder module and the shared decoder module. The method can ensure that semantic features (which are beneficial to identifying whether the pixels are blood vessels) provided by the shared decoder module can be fused with low-level detail features (which are beneficial to identifying whether the pixels are boundaries) provided by the shared encoder module, so that a segmentation result with better edge fitting degree is obtained.
The shared encoder module and the shared decoder module may include five sub-encoding modules or five sub-decoding modules, respectively. In the sub-coding module, a conventional U-Net structure may be employed, i.e. a total of two convolutional layers. And the activation function layer may employ a ReLU activation function. In the sub-decoding module, the up-sampling layer adopts a bilinear interpolation mode to perform up-sampling operation, and further characteristic refinement is performed through the convolution layer after interpolation. Meanwhile, the activation function layer in the sub-decoding module may also adopt a ReLU activation function.
In an embodiment, the first decoder module and/or the second decoder module has a preset number of decoding modules, and the decoding modules include: the decoding module comprises an up-sampling layer, a convolution layer and an activation function layer, wherein the convolution layer in the decoding module is a single-layer convolution layer. Compared with a two-layer convolutional layer structure in a conventional U-Net structure, the first decoder module and the second decoder module only adopt one convolutional layer for decoding, which is equivalent to reducing nearly half of time and space complexity. Thereby, a lightweight decoder structure is achieved.
In one embodiment, the classification module comprises: a connection layer and a classification layer, wherein the connection layer is used for performing feature connection on the second shared features and the choroidal sublayer segmentation specific features and the choroidal vessel segmentation specific features respectively to obtain a first feature map and a second feature map; and the classification layer is used for classifying according to the first characteristic diagram and the second characteristic diagram to obtain a segmentation result of the image to be segmented. In particular, the connection layer in the classification module is configured to connect the second shared feature and the choroidal sublayer segmentation specific feature to obtain a first feature map, and to connect the second shared feature and the choroidal vessel segmentation specific feature to obtain a second feature map. The classification layer adopts a full-connection layer to realize classification operation, namely the full-connection layer acquires a first feature map and a second feature map after connection to perform classification operation, and maps the C-dimensional feature map into 1 dimension, so that the output feature represents the probability that the position is a choroidal blood vessel or a choroidal sublayer.
In an embodiment, to avoid the bias of the segmented network model towards a simple task, namely choroidal sublayer segmentation, a multitask loss function with a new regularization term is employed in the segmented network model, which can adaptively balance the two tasks during training. Wherein, the multitask loss function with the regularization term is expressed by the following formula:
wherein, P1Representing a choroidal vascular probability map, P2Representing the choroidal sublayer probability map, Y1Representing the actual choroidal vessel mask map, Y2A diagram of the actual choroidal sublayer mask is shown,
and expressing a cross entropy loss function, wherein phi is a regularization constraint term, and lambda is a preset parameter.
Specifically, the cross entropy loss function and the regularization constraint term are specifically expressed by the following formulas:
wherein, yijIs the true label, p, of the jth pixel in the ith B-scanijIs a label predicted by the segmentation method for the jth pixel in the ith B-scan,
and
respectively represent
And
thereby bringing phi into
In (1), the following formula can be obtained:
therefore, it can be seen from the above formula that
Is greater than
In case (i.e. when the task of choroidal sublayer segmentation fits better), the loss function will be more inclined to the task of choroidal vessel segmentation and vice versa. Thus, the effect of dynamically balancing two tasks is achieved.
The choroid sublayer and choroidal vessel segmentation network models provided by embodiments of the present invention are effective (with accuracy greater than 0.980) in choroidal thickness boundaries and vessel-dependent choroidal analysis, and the performance of the more difficult task, i.e., choroidal vessel segmentation, can be significantly improved by a multitask learning strategy in the segmentation network model. Because the first decoder module and the second decoder module adopt a lightweight decoder structure, the characteristics specific to the task can be captured according to the shared characteristics, the method is simpler and more targeted, and the model scale is reduced by nearly 25%.
The embodiment of the invention also provides a training method of the choroid sublayer and choroid blood vessel segmentation network model, which comprises the following steps: acquiring a training sample set; inputting the training sample set into the choroid sublayer and choroid blood vessel segmentation network model described in the above embodiment for training, and obtaining a pre-trained choroid sublayer and choroid blood vessel segmentation network model.
Wherein the training sample set includes a plurality of OCT image samples in which a choroidal sublayer and choroidal blood vessels are labeled. By inputting the training sample set into the structure of the segmentation network model provided in the above embodiment for training, the obtained model can directly process the image to be segmented, and generate the pixel classification results of the choroid sublayer and the choroid blood vessels. During training, end-to-end training can be adopted, so that training time is saved.
According to the training method of the choroid sublayer and choroid blood vessel segmentation network model provided by the embodiment of the invention, the segmentation network model adopting the shared decoder module, the first decoder module, the second decoder module and the classification module is trained, so that the obtained pre-trained model can directly classify images to be segmented, and the pixel classification result of the choroid sublayer and choroid blood vessel segmentation is obtained. The network model is provided with a shared decoder module, a first decoder module, a second decoder module and a classification module structure, and multi-task segmentation is realized through a multi-stream network structure.
An embodiment of the present invention further provides a storage medium, as shown in fig. 2, on which a computer program 601 is stored, where the instructions are executed by a processor to implement the steps of the training method of the choroid sublayer and choroidal vessel segmentation network model in the above-described embodiments. The storage medium is also stored with audio and video stream data, characteristic frame data, an interactive request signaling, encrypted data, preset data size and the like. The storage medium may be a magnetic Disk, an optical Disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a Flash Memory (Flash Memory), a Hard Disk (Hard Disk Drive, abbreviated as HDD) or a Solid State Drive (SSD), etc.; the storage medium may also comprise a combination of memories of the kind described above.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic Disk, an optical Disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a Flash Memory (Flash Memory), a Hard Disk (Hard Disk Drive, abbreviated as HDD) or a Solid State Drive (SSD), etc.; the storage medium may also comprise a combination of memories of the kind described above.
An embodiment of the present invention further provides an electronic device, as shown in fig. 3, the electronic device may include a processor 51 and a memory 52, where the processor 51 and the memory 52 may be connected by a bus or in another manner, and fig. 3 takes the connection by the bus as an example.
The processor 51 may be a Central Processing Unit (CPU). The Processor 51 may also be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, or combinations thereof.
The memory 52, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as the corresponding program instructions/modules in the embodiments of the present invention. The processor 51 executes the non-transitory software programs, instructions and modules stored in the memory 52 to execute various functional applications of the processor and data processing, i.e. to implement the training method of the choroidal sublayer and choroidal vessel segmentation network models in the above-described method embodiments.
The memory 52 may include a storage program area and a storage data area, wherein the storage program area may store an operating device, an application program required for at least one function; the storage data area may store data created by the processor 51, and the like. Further, the memory 52 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory 52 may optionally include memory located remotely from the processor 51, and these remote memories may be connected to the processor 51 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The one or more modules are stored in the memory 52 and when executed by the processor 51 perform a method of training a choroidal sub-layer and choroidal vessel segmentation network model in an embodiment.
The specific details of the electronic device may be understood by referring to the corresponding related descriptions and effects in the embodiments, and are not described herein again.
Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations fall within the scope defined by the appended claims.
Claims (10)
1. A model of the choroidal sublayer and choroidal vascular segmentation network, comprising:
the shared encoder module is used for carrying out down-sampling and feature extraction on an image to be segmented to obtain a first shared feature of choroidal sublayer segmentation and choroidal blood vessel segmentation, and the resolution of the first shared feature is lower than a threshold value;
a shared decoder module, configured to obtain the first shared feature for upsampling, so as to obtain a second shared feature of the choroid sublayer segmentation and the choroid blood vessel segmentation, where a resolution of the second shared feature is higher than a threshold;
a first decoder module to extract a choroidal sublayer segmentation specific feature in the second shared features;
a second decoder module for extracting choroidal vessel segmentation specific features in the second shared features;
and the classification module is used for calculating a segmentation result of the image to be segmented according to the second shared feature, the choroid sublayer segmentation specific feature and the choroid blood vessel segmentation specific feature.
2. The choroidal sublayer and choroidal vessel segmentation network model of claim 1, wherein said shared encoder module comprises: the shared decoder comprises a preset number of sub-decoding modules, and the sub-coding modules are connected with the sub-decoding modules with the same corresponding spatial resolution.
3. The choroidal sublayer and choroidal vessel segmentation network model according to claim 2, wherein said sub-coding modules comprise: a convolution layer, an activation function layer and a space pooling layer; the sub-decoding module includes: an upsampling layer, a convolutional layer, and an activation function layer.
4. The choroidal sub-layer and choroidal vessel segmentation network model of claim 1 wherein a preset number of decoding modules are first and/or second decoder modules, said decoding modules comprising: the decoding module comprises an up-sampling layer, a convolution layer and an activation function layer, wherein the convolution layer in the decoding module is a single-layer convolution layer.
5. The choroidal sublayer and choroidal vessel segmentation network model of claim 1, wherein said classification module comprises: a connection layer and a classification layer,
the connection layer is used for performing characteristic connection on the second shared characteristic and the choroid sublayer segmentation specific characteristic and the choroid blood vessel segmentation specific characteristic respectively to obtain a first characteristic diagram and a second characteristic diagram;
and the classification layer is used for classifying according to the first characteristic diagram and the second characteristic diagram to obtain a segmentation result of the image to be segmented.
6. The choroidal sublayer and choroidal blood vessel segmentation network model of claim 1 wherein the loss function of the segmentation network model is a multitasking loss function with a regularization term.
7. The choroidal sublayer and choroidal vessel segmentation network model of claim 6 wherein said multitask loss function with regularization term is represented by the formula:
wherein, P1Representing a choroidal vascular probability map, P2Representing the choroidal sublayer probability map, Y1Representing the actual choroidal vessel mask map, Y2Representing the actual choroidal sublayer maskA block diagram of the mold,
and expressing a cross entropy loss function, wherein phi is a regularization constraint term, and lambda is a preset parameter.
8. A method of training a model of the choroidal sublayer and choroidal vessel segmentation network, comprising:
acquiring a training sample set;
inputting the training sample set into the model of the choroid sub-layer and choroidal vessel segmentation network of any of claims 1 to 7 for training, resulting in a pre-trained model of the choroid sub-layer and choroidal vessel segmentation network.
9. A computer-readable storage medium storing computer instructions for causing a computer to perform the method of training a choroidal sub-layer and choroidal vessel segmentation network model according to claim 8.
10. An electronic device, comprising: a memory and a processor communicatively coupled to each other, the memory storing computer instructions, the processor performing the method of training a choroidal sub-layer and choroidal vessel segmentation network model according to claim 8 by executing the computer instructions.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210062989.3A CN114399511A (en) | 2022-01-19 | 2022-01-19 | Choroid sublayer and choroid blood vessel segmentation network model and training method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210062989.3A CN114399511A (en) | 2022-01-19 | 2022-01-19 | Choroid sublayer and choroid blood vessel segmentation network model and training method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114399511A true CN114399511A (en) | 2022-04-26 |
Family
ID=81231699
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210062989.3A Pending CN114399511A (en) | 2022-01-19 | 2022-01-19 | Choroid sublayer and choroid blood vessel segmentation network model and training method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114399511A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115456962A (en) * | 2022-08-24 | 2022-12-09 | 中山大学中山眼科中心 | Choroidal vascular index prediction method and device based on convolutional neural network |
-
2022
- 2022-01-19 CN CN202210062989.3A patent/CN114399511A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115456962A (en) * | 2022-08-24 | 2022-12-09 | 中山大学中山眼科中心 | Choroidal vascular index prediction method and device based on convolutional neural network |
| CN115456962B (en) * | 2022-08-24 | 2023-09-29 | 中山大学中山眼科中心 | Choroidal blood vessel index prediction method and device based on convolutional neural network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7058373B2 (en) | Lesion detection and positioning methods, devices, devices, and storage media for medical images | |
| CN109886933B (en) | Medical image recognition method and device and storage medium | |
| CN110428432B (en) | Deep neural network algorithm for automatically segmenting colon gland image | |
| US10789499B2 (en) | Method for recognizing image, computer product and readable storage medium | |
| WO2020140370A1 (en) | Method and device for automatically detecting petechia in fundus, and computer-readable storage medium | |
| US9177102B2 (en) | Database and imaging processing system and methods for analyzing images acquired using an image acquisition system | |
| US20220383661A1 (en) | Method and device for retinal image recognition, electronic equipment, and storage medium | |
| CN111862009B (en) | Classifying method of fundus OCT (optical coherence tomography) images and computer readable storage medium | |
| CN114066884B (en) | Retinal blood vessel segmentation method and device, electronic device and storage medium | |
| WO2019232910A1 (en) | Fundus image analysis method, computer device and storage medium | |
| US11972571B2 (en) | Method for image segmentation, method for training image segmentation model | |
| WO2021159811A1 (en) | Auxiliary diagnostic apparatus and method for glaucoma, and storage medium | |
| CN112132801B (en) | Lung bulla focus detection method and system based on deep learning | |
| CN113012163A (en) | Retina blood vessel segmentation method, equipment and storage medium based on multi-scale attention network | |
| CN113397475A (en) | OCT (optical coherence tomography) -image-based Alzheimer's disease risk prediction method, system and medium | |
| Kumar et al. | Two-stage framework for optic disc segmentation and estimation of cup-to-disc ratio using deep learning technique | |
| WO2024074921A1 (en) | Distinguishing a disease state from a non-disease state in an image | |
| Abbasi-Sureshjani et al. | Boosted exudate segmentation in retinal images using residual nets | |
| Zhao et al. | Attention residual convolution neural network based on U-net (AttentionResU-Net) for retina vessel segmentation | |
| CN114399511A (en) | Choroid sublayer and choroid blood vessel segmentation network model and training method thereof | |
| KR102472886B1 (en) | Method for providing information on diagnosing renal failure and device using the same | |
| WO2021224162A1 (en) | Method and system for identifying abnormal images in a set of medical images | |
| CN114612373A (en) | Image identification method and server | |
| Lenka et al. | Glaucoma Detection from Retinal Fundus Images using Graph Convolution Based Multi-task Model | |
| CN117876801B (en) | Method for predicting diabetic nephropathy based on fundus blood vessel characteristics and artificial intelligence |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |