CN116563314A - Lumbar vertebrae segmentation method, device, electronic equipment and computer readable storage medium - Google Patents

Lumbar vertebrae segmentation method, device, electronic equipment and computer readable storage medium Download PDF

Info

Publication number
CN116563314A
CN116563314A CN202310275163.XA CN202310275163A CN116563314A CN 116563314 A CN116563314 A CN 116563314A CN 202310275163 A CN202310275163 A CN 202310275163A CN 116563314 A CN116563314 A CN 116563314A
Authority
CN
China
Prior art keywords
network
segmentation
task
lumbar vertebra
fadnet
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310275163.XA
Other languages
Chinese (zh)
Inventor
张逸凌
刘星宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Longwood Valley Medtech Co Ltd
Original Assignee
Longwood Valley Medtech Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Longwood Valley Medtech Co Ltd filed Critical Longwood Valley Medtech Co Ltd
Priority to CN202310275163.XA priority Critical patent/CN116563314A/en
Publication of CN116563314A publication Critical patent/CN116563314A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/12—Edge-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30008—Bone
    • G06T2207/30012—Spine; Backbone

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Medical Informatics (AREA)
  • Quality & Reliability (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Image Analysis (AREA)

Abstract

The application provides a lumbar vertebra segmentation method and device based on a multi-task FADNet network model, electronic equipment and a computer-readable storage medium. The method comprises the steps of obtaining lumbar vertebra images to be segmented; inputting the lumbar vertebra image into a preset multi-task FADNet network model, and outputting a lumbar vertebra segmentation result; the lumbar vertebra segmentation result comprises segmentation results of a plurality of parts in lumbar vertebra; the multi-task FADNet network model is obtained based on model training of a multi-task FADNet network, and the multi-task FADNet network adopts a multi-layer convolution network, a downsampling network and an upsampling network structure for extracting features; meanwhile, a feature sharing mechanism is adopted in the up-sampling process, and up-sampling features of two network branches are shared to improve the segmentation precision; and adding a network of attention mechanisms in the jump connection between each pair of adjacent decoder branches and encoder branches of each layer to enhance the segmentation of boundary features.

Description

Lumbar vertebrae segmentation method, device, electronic equipment and computer readable storage medium
Technical Field
The application belongs to the technical field of deep learning intelligent recognition, and particularly relates to a lumbar vertebra segmentation method, device, electronic equipment and computer readable storage medium based on a multi-task FADNet network model.
Background
The traditional lumbar vertebra segmentation method is based on threshold segmentation, edge detection or region growing, has poor segmentation precision, roughly segments a target region and cannot accurately segment the boundary of the target region. Moreover, the method is cumbersome and time-consuming to operate.
Therefore, how to perform lumbar vertebra segmentation quickly and accurately is a technical problem that needs to be solved by those skilled in the art.
Disclosure of Invention
The embodiment of the application provides a lumbar vertebra segmentation method, device, electronic equipment and computer readable storage medium based on a multi-task FADNet network model, which can perform lumbar vertebra segmentation quickly and accurately.
In a first aspect, an embodiment of the present application provides a lumbar vertebra segmentation method based on a multi-task fadnaet network model, including:
acquiring lumbar vertebra images to be segmented;
inputting the lumbar vertebra image into a preset multi-task FADNet network model, and outputting a lumbar vertebra segmentation result;
the lumbar vertebra segmentation result comprises segmentation results of a plurality of parts in lumbar vertebra;
the multi-task FADNet network model is obtained based on model training of a multi-task FADNet network, and the multi-task FADNet network adopts a multi-layer convolution network, a downsampling network and an upsampling network structure for extracting features; meanwhile, a feature sharing mechanism is adopted in the up-sampling process, and up-sampling features of two network branches are shared to improve the segmentation precision; and adding a network of attention mechanisms in the jump connection between each pair of adjacent decoder branches and encoder branches of each layer to enhance the segmentation of boundary features.
Optionally, the feature sharing mechanism is: and for each scale, fusing the up-sampling result of the first network branch and the up-sampling result of the second network branch under the same scale, fusing the fused result and the attention characteristic of each network branch under the same scale, and inputting the fused result into the next up-sampling convolution.
Optionally, before inputting the lumbar vertebra image into the preset multi-task fadnaet network model, the method further includes:
acquiring a lumbar vertebra image dataset;
labeling lumbar vertebra areas of lumbar vertebra images in the lumbar vertebra image data set, and determining the lumbar vertebra areas as segmentation masks; wherein, each segmentation mask corresponds to the lumbar vertebra image one by one;
converting the image format of each lumbar vertebra image and the corresponding segmentation mask into a PNG format;
all the lumbar images converted into PNG format and the corresponding segmentation masks are divided into a training set, a verification set and a test set according to a preset proportion.
Optionally, after dividing all lumbar images converted into PNG format and the corresponding segmentation masks into training set, verification set and test set according to the preset proportion, the method further includes:
and performing model training on the multi-task FADNet network by using the training set, and performing verification and test by using the verification set and the test set to obtain a multi-task FADNet network model.
Optionally, performing model training on the multi-task FADNet network by using a training set, and performing verification and testing by using a verification set and a test set to obtain a multi-task FADNet network model, including:
setting the batch_size of training to 64 in the model training process;
setting the initialized learning rate as 1e-4, adding a learning rate attenuation strategy, and carrying out 5000 times of iteration, wherein the learning rate attenuation is 0.9 of the last learning rate;
setting an optimizer as an Adam optimizer;
setting a loss function as DICE loss;
and setting 1000 times of each iteration, performing one-time verification on the training set and the verification set, judging the network training stop time through an early-stop method, and obtaining the multi-task FADNet network model.
Optionally, the attention mechanism network includes a location attention mechanism network and a channel attention mechanism network;
wherein the location attention mechanism network is used for selectively aggregating the features of each location through a weighted sum of the features at all locations;
channel attention mechanism networks for selectively emphasizing the existence of interdependent channel maps by integrating the correlation features between all channel maps.
Optionally, the location attention mechanism network is configured to selectively aggregate the features of each location by a weighted sum of the features at all locations, including:
initializing to generate a position attention matrix for modeling the relationship between any two points;
performing matrix multiplication on the position attention matrix and the feature matrix to obtain a multiplication result;
and adding the multiplication result and the feature matrix element by element to obtain a result which has certain characterization capability on the global semantics finally.
In a second aspect, an embodiment of the present application provides a lumbar vertebra segmentation apparatus based on a multi-task fadnaet network model, including:
the image acquisition module is used for acquiring lumbar vertebra images to be segmented;
the lumbar vertebra segmentation result acquisition module is used for inputting lumbar vertebra images into a preset multi-task FADNet network model and outputting lumbar vertebra segmentation results;
the lumbar vertebra segmentation result comprises segmentation results of a plurality of parts in lumbar vertebra;
the multi-task FADNet network model is obtained based on model training of a multi-task FADNet network, and the multi-task FADNet network adopts a multi-layer convolution network, a downsampling network and an upsampling network structure for extracting features; meanwhile, a feature sharing mechanism is adopted in the up-sampling process, and up-sampling features of two network branches are shared to improve the segmentation precision; and adding a network of attention mechanisms in the jump connection between each pair of adjacent decoder branches and encoder branches of each layer to enhance the segmentation of boundary features.
In a third aspect, an embodiment of the present application provides an electronic device, including: a processor and a memory storing computer program instructions;
the processor executes the computer program instructions to implement the lumbar vertebra segmentation method based on the multi-task FADNet network model as shown in the first aspect.
In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, where computer program instructions are stored, where the computer program instructions, when executed by a processor, implement a lumbar vertebra segmentation method based on a multi-task fadnat network model as shown in the first aspect.
The lumbar vertebra segmentation method based on the multi-task FADNet network model comprises the following steps: acquiring lumbar vertebra images to be segmented; inputting the lumbar vertebra image into a preset multi-task FADNet network model, and outputting a lumbar vertebra segmentation result; the lumbar vertebra segmentation result comprises segmentation results of a plurality of parts in lumbar vertebra; the multi-task FADNet network model is obtained based on model training of a multi-task FADNet network, and the multi-task FADNet network adopts a multi-layer convolution network, a downsampling network and an upsampling network structure for extracting features; meanwhile, a feature sharing mechanism is adopted in the up-sampling process, and up-sampling features of two network branches are shared to improve the segmentation precision; and adding a network of attention mechanisms in the jump connection between each pair of adjacent decoder branches and encoder branches of each layer to enhance the segmentation of boundary features.
In one aspect, the method multitasking FADNet network employs a multi-layer convolutional network, a downsampling network, and an upsampling network structure for extracting features; meanwhile, a feature sharing mechanism is adopted in the up-sampling process, and up-sampling features of two network branches are shared to improve the segmentation precision; and add the attention mechanism network in the jump connection between every pair of adjacent decoder branches and encoder branches of each layer, in order to strengthen the segmentation of the boundary characteristic, can improve the accuracy rate of lumbar vertebra segmentation; on the other hand, the method can output the segmentation results of a plurality of parts in the lumbar vertebra at the same time, and can improve the efficiency of lumbar vertebra segmentation.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described below, it will be obvious that the drawings in the description below are some embodiments of the present application, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flow chart of a lumbar vertebra segmentation method based on a multi-task fadnaet network model according to an embodiment of the present application;
fig. 2 is a schematic diagram of a multi-task fadnaet network structure provided in an embodiment of the present application;
FIG. 3 is a schematic diagram of an attention mechanism network architecture provided in one embodiment of the present application;
FIG. 4 is a schematic diagram of a deep aggregation pyramid network architecture provided in one embodiment of the present application;
FIG. 5 is a schematic diagram of three-dimensional reconstruction of DICOM data according to one embodiment of the present application;
fig. 6 is a schematic structural diagram of a lumbar vertebra segmentation apparatus based on a multi-task fadnaet network model according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Features and exemplary embodiments of various aspects of the present application are described in detail below to make the objects, technical solutions and advantages of the present application more apparent, and to further describe the present application in conjunction with the accompanying drawings and the detailed embodiments. It should be understood that the specific embodiments described herein are intended to be illustrative of the application and are not intended to be limiting. It will be apparent to one skilled in the art that the present application may be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present application by showing examples of the present application.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
The traditional lumbar vertebra segmentation method is based on threshold segmentation, edge detection or region growing, has poor segmentation precision, roughly segments a target region and cannot accurately segment the boundary of the target region. Moreover, the method is cumbersome and time-consuming to operate.
In order to solve the problems in the prior art, embodiments of the present application provide a lumbar vertebra segmentation method, device, equipment and computer readable storage medium based on a multi-task fadnaet network model. The lumbar vertebra segmentation method based on the multi-task FADNet network model provided by the embodiment of the application is first described below.
Fig. 1 shows a flow chart of a lumbar vertebra segmentation method based on a multi-task fadnaet network model according to an embodiment of the present application. As shown in fig. 1, the lumbar vertebra segmentation method based on the multi-task fadnaet network model includes:
s101, acquiring a lumbar vertebra image to be segmented;
s102, inputting the lumbar vertebra image into a preset multi-task FADNet network model, and outputting a lumbar vertebra segmentation result;
the lumbar vertebra segmentation result comprises segmentation results of a plurality of parts in lumbar vertebra;
the multi-task FADNet network model is obtained based on model training of a multi-task FADNet network, and the multi-task FADNet network adopts a multi-layer convolution network, a downsampling network and an upsampling network structure for extracting features; meanwhile, a feature sharing mechanism is adopted in the up-sampling process, and up-sampling features of two network branches are shared to improve the segmentation precision; and adding a network of attention mechanisms in the jump connection between each pair of adjacent decoder branches and encoder branches of each layer to enhance the segmentation of boundary features.
In one embodiment, the feature sharing mechanism is: and for each scale, fusing the up-sampling result of the first network branch and the up-sampling result of the second network branch under the same scale, fusing the fused result and the attention characteristic of each network branch under the same scale, and inputting the fused result into the next up-sampling convolution.
Specifically, fig. 2 is a schematic diagram of a multi-task fadnaet network structure provided in an embodiment of the present application, where the multi-task fadnaet network structure shown in fig. 2 integrates multi-scale and multi-level feature information, so that shallow local details and deep abstract features can be enhanced complementarily, and a better segmentation effect can be obtained. The multi-task FADNet network outputs a plurality of segmentation results which are respectively used for extracting the characteristics of different vertebral body parts of the lumbar vertebra. And the single task segmentation only outputs one result, namely, only one part can be segmented. The multitasking can divide a plurality of parts simultaneously, greatly shorten the division time.
The multi-tasking FADNet network employs a multi-layer convolution, downsampling, and upsampling architecture to extract features. And simultaneously, a feature sharing mechanism is adopted in the up-sampling process, up-sampling features of two branch networks are shared, and the segmentation precision is improved. And fusing the upsampling result of the first network branch with the upsampling result of the second network branch under the same scale, fusing the fused result with the attention characteristic of each branch under the same scale, and inputting the fused result into the next upsampling convolution. And in the next up-sampling process, the features of the upper branch and the lower branch are fused, the fusion result and the attention features of the branches under the same scale are fused, and the fusion result is input into the next up-sampling convolution. The final output result of the network stacks the first network branch and the second network branch results.
And there is a jump connection between each pair of adjacent decoder branches and encoder branches of each layer, where attention mechanisms are added in the jump connection, enhancing the segmentation of boundary features. The convolutional layer uses residual convolution, i.e. residual units consisting of 1x1,3x3 and 1x1 convolution kernels, in order to reduce the feature loss. Due to the presence of the jump connection and residual convolution, and more information flow paths in the U-shaped network, the segmentation accuracy can be improved finally.
In one aspect, the method multitasking FADNet network employs a multi-layer convolutional network, a downsampling network, and an upsampling network structure for extracting features; meanwhile, a feature sharing mechanism is adopted in the up-sampling process, and up-sampling features of two network branches are shared to improve the segmentation precision; and add the attention mechanism network in the jump connection between every pair of adjacent decoder branches and encoder branches of each layer, in order to strengthen the segmentation of the boundary characteristic, can improve the accuracy rate of lumbar vertebra segmentation; on the other hand, the method can output the segmentation results of a plurality of parts in the lumbar vertebra at the same time, and can improve the efficiency of lumbar vertebra segmentation.
In one embodiment, before inputting the lumbar image into the preset multi-task fadnaet network model, the method further comprises:
acquiring a lumbar vertebra image dataset;
labeling lumbar vertebra areas of lumbar vertebra images in the lumbar vertebra image data set, and determining the lumbar vertebra areas as segmentation masks; wherein, each segmentation mask corresponds to the lumbar vertebra image one by one;
converting the image format of each lumbar vertebra image and the corresponding segmentation mask into a PNG format;
all the lumbar images converted into PNG format and the corresponding segmentation masks are divided into a training set, a verification set and a test set according to a preset proportion.
Specifically, a lumbar medical image dataset is obtained, the lumbar region is marked manually, and finally only the label containing the lumbar part is extracted as our segmentation mask. And converting the DICOM data of the two-dimensional cross section into a picture in a PNG format, labeling the picture converted by the segmentation mask into the picture in the PNG format, and dividing the picture into a training set, a verification set and a test set according to the proportion of 6:2:2 after the picture is disordered.
In one embodiment, after dividing all lumbar images converted into PNG format and the corresponding segmentation masks into training set, verification set and test set according to the preset proportion, the method further comprises:
and performing model training on the multi-task FADNet network by using the training set, and performing verification and test by using the verification set and the test set to obtain a multi-task FADNet network model.
In one embodiment, the training set is used for model training of the multi-task FADNet network, and the verification set and the test set are used for verification and test, so as to obtain a multi-task FADNet network model, which comprises the following steps:
setting the batch_size of training to 64 in the model training process;
setting the initialized learning rate as 1e-4, adding a learning rate attenuation strategy, and carrying out 5000 times of iteration, wherein the learning rate attenuation is 0.9 of the last learning rate;
setting an optimizer as an Adam optimizer;
setting a loss function as DICE loss;
and setting 1000 times of each iteration, performing one-time verification on the training set and the verification set, judging the network training stop time through an early-stop method, and obtaining the multi-task FADNet network model.
In one embodiment, the Loss function Loss calculation formula used is:
CELoss＝-[y log y′+(1-y)log(1-y′)]
Loss＝α·CELoss+(1-α)·DiceLoss
wherein CELoss is a cross entropy loss function, diceLoss is a Dice loss function, y is a tag value, y' is a predicted value, and α is a loss weight coefficient.
In one embodiment, the attention mechanism network includes a location attention mechanism network and a channel attention mechanism network;
wherein the location attention mechanism network is used for selectively aggregating the features of each location through a weighted sum of the features at all locations;
channel attention mechanism networks for selectively emphasizing the existence of interdependent channel maps by integrating the correlation features between all channel maps.
In one embodiment, a location attention mechanism network for selectively aggregating features of each location by a weighted sum of features at all locations, comprises:
initializing to generate a position attention matrix for modeling the relationship between any two points;
performing matrix multiplication on the position attention matrix and the feature matrix to obtain a multiplication result;
and adding the multiplication result and the feature matrix element by element to obtain a result which has certain characterization capability on the global semantics finally.
Specifically, the attention mechanism network structure shown in fig. 3 is composed of a location attention module and a channel attention module. In the position attention module, a position attention moment array is firstly initialized and generated for modeling the relation between any two points, then the attention matrix and the feature matrix are subjected to matrix multiplication, and then the multiplication result and the original feature matrix are subjected to element-by-element addition to obtain a result which finally has certain characterization capability on global semantics. The operation of the channel attention module is similar except that the multiplication is calculated in the channel dimension. And finally, aggregating the results of the two modules to obtain a better characterization result and outputting the attention characteristic.
In one embodiment, fig. 4 is a schematic diagram of a deep aggregation pyramid network structure provided in one embodiment of the present application, where the network structure includes four branches, each of which uses a convolution kernel of 1x1 or 3x3 to extract features, and an input feature map obtains four outputs through the four branches in parallel, and then fuses the results of the four branches to be output through the convolution kernel of 1x 1.
The DICOM data is reconstructed in three dimensions, with the reconstruction effect shown in fig. 5.
Based on the FADNet network, a focus optimization mechanism and a multi-scale feature fusion module are introduced, shallow detail features are fused with deep abstract features obtained through the multi-scale feature fusion module, enhanced content features are obtained, and the precision of lumbar segmentation is improved.
The FADNet network is designed into a multi-task mechanism, so that a plurality of parts of the lumbar vertebra can be segmented at the same time, and compared with a single-task mechanism, the segmentation time can be reduced; and simultaneously, a feature sharing mechanism is adopted in the up-sampling process, up-sampling features of two branch networks are shared, and the segmentation precision is improved.
Dual attention optimization mechanism: the method is used for calculating global feature dependency relations in space and channel dimensions, a position attention module is used for learning spatial interdependencies of features, and a channel attention module is designed for simulating the channel interdependencies, so that more accurate segmentation results are facilitated.
The method provided by the application can accurately divide the lumbar vertebra, the division precision is superior to that of the existing neural network, a doctor is more accurately assisted in operation planning, and the success rate of operation is improved.
Fig. 6 is a schematic structural diagram of a lumbar vertebra segmentation apparatus based on a multi-task fadnaet network model according to an embodiment of the present application, as shown in fig. 6, where the lumbar vertebra segmentation apparatus based on the multi-task fadnaet network model includes:
an image acquisition module 601, configured to acquire a lumbar vertebra image to be segmented;
the lumbar vertebra segmentation result acquisition module 602 is configured to input a lumbar vertebra image into a preset multi-task fadnaet network model, and output a lumbar vertebra segmentation result;
the lumbar vertebra segmentation result comprises segmentation results of a plurality of parts in lumbar vertebra;
the multi-task FADNet network model is obtained based on model training of a multi-task FADNet network, and the multi-task FADNet network adopts a multi-layer convolution network, a downsampling network and an upsampling network structure for extracting features; meanwhile, a feature sharing mechanism is adopted in the up-sampling process, and up-sampling features of two network branches are shared to improve the segmentation precision; and adding a network of attention mechanisms in the jump connection between each pair of adjacent decoder branches and encoder branches of each layer to enhance the segmentation of boundary features.
Fig. 7 shows a schematic structural diagram of an electronic device according to an embodiment of the present application.
The electronic device may include a processor 701 and a memory 702 storing computer program instructions.
In particular, the processor 701 described above may include a Central Processing Unit (CPU), or an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or may be configured to implement one or more integrated circuits of embodiments of the present application.
Memory 702 may include mass storage for data or instructions. By way of example, and not limitation, memory 702 may comprise a Hard Disk Drive (HDD), floppy Disk Drive, flash memory, optical Disk, magneto-optical Disk, magnetic tape, or universal serial bus (Universal Serial Bus, USB) Drive, or a combination of two or more of the foregoing. The memory 702 may include removable or non-removable (or fixed) media, where appropriate. The memory 702 may be internal or external to the electronic device, where appropriate. In a particular embodiment, the memory 702 may be a non-volatile solid state memory.
In one embodiment, memory 702 may be Read Only Memory (ROM). In one embodiment, the ROM may be mask-programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically Erasable PROM (EEPROM), electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.
The processor 701 reads and executes the computer program instructions stored in the memory 702 to implement any one of the lumbar vertebra segmentation methods based on the multi-task fadnaet network model in the above embodiments.
In one example, the electronic device may also include a communication interface 703 and a bus 710. As shown in fig. 7, the processor 701, the memory 702, and the communication interface 703 are connected by a bus 710 and perform communication with each other.
The communication interface 703 is mainly used for implementing communication between each module, device, unit and/or apparatus in the embodiments of the present application.
Bus 710 includes hardware, software, or both that couple components of the electronic device to one another. By way of example, and not limitation, the buses may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an infiniband interconnect, a Low Pin Count (LPC) bus, a memory bus, a micro channel architecture (MCa) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a video electronics standards association local (VLB) bus, or other suitable bus, or a combination of two or more of the above. Bus 710 may include one or more buses, where appropriate. Although embodiments of the present application describe and illustrate a particular bus, the present application contemplates any suitable bus or interconnect.
In addition, in combination with the lumbar vertebra segmentation method based on the multi-task FADNet network model in the above embodiment, the embodiments of the present application may provide a computer readable storage medium for implementation. The computer readable storage medium has stored thereon computer program instructions; the computer program instructions, when executed by the processor, implement any of the lumbar segmentation methods of the above embodiments based on the multi-tasking fadnaet network model.
It should be clear that the present application is not limited to the particular arrangements and processes described above and illustrated in the drawings. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present application are not limited to the specific steps described and illustrated, and those skilled in the art can make various changes, modifications, and additions, or change the order between steps, after appreciating the spirit of the present application.
The functional blocks shown in the above-described structural block diagrams may be implemented in hardware, software, firmware, or a combination thereof. When implemented in hardware, it may be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, a plug-in, a function card, or the like. When implemented in software, the elements of the present application are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine readable medium or transmitted over transmission media or communication links by a data signal carried in a carrier wave. A "machine-readable medium" may include any medium that can store or transfer information. Examples of machine-readable media include electronic circuitry, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio Frequency (RF) links, and the like. The code segments may be downloaded via computer networks such as the internet, intranets, etc.
It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, the present application is not limited to the order of the above-described steps, that is, the steps may be performed in the order mentioned in the embodiments, may be different from the order in the embodiments, or several steps may be performed simultaneously.
Aspects of the present application are described above with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions/acts specified in the flowchart and/or block diagram block or blocks. Such a processor may be, but is not limited to being, a general purpose processor, a special purpose processor, an application specific processor, or a field programmable logic circuit. It will also be understood that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware which performs the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In the foregoing, only the specific embodiments of the present application are described, and it will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the systems, modules and units described above may refer to the corresponding processes in the foregoing method embodiments, which are not repeated herein. It should be understood that the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive various equivalent modifications or substitutions within the technical scope of the present application, which are intended to be included in the scope of the present application.

Claims (10)

1. The lumbar vertebra segmentation method based on the multi-task FADNet network model is characterized by comprising the following steps of:
acquiring lumbar vertebra images to be segmented;
inputting the lumbar vertebra image into a preset multi-task FADNet network model, and outputting a lumbar vertebra segmentation result;
the lumbar vertebra segmentation result comprises segmentation results of a plurality of parts in lumbar vertebra;
the multi-task FADNet network model is obtained based on model training of a multi-task FADNet network, and the multi-task FADNet network adopts a multi-layer convolution network, a downsampling network and an upsampling network structure for extracting features; meanwhile, a feature sharing mechanism is adopted in the up-sampling process, and up-sampling features of two network branches are shared to improve the segmentation precision; and adding a network of attention mechanisms in the jump connection between each pair of adjacent decoder branches and encoder branches of each layer to enhance the segmentation of boundary features.
2. The lumbar vertebra segmentation method based on the multi-task fadnat network model according to claim 1, wherein the feature sharing mechanism is as follows: and for each scale, fusing the up-sampling result of the first network branch and the up-sampling result of the second network branch under the same scale, fusing the fused result and the attention characteristic of each network branch under the same scale, and inputting the fused result into the next up-sampling convolution.
3. The lumbar spine segmentation method based on the multi-task fadnaet network model according to claim 1, wherein before inputting the lumbar spine image into a preset multi-task fadnaet network model, the method further comprises:
acquiring a lumbar vertebra image dataset;
labeling a lumbar vertebra region of a lumbar vertebra image in the lumbar vertebra image data set, and determining the lumbar vertebra region as a segmentation mask; wherein each segmentation mask corresponds to the lumbar vertebra image one by one;
converting the image format of each lumbar vertebra image and the corresponding segmentation mask into a PNG format;
all the lumbar images converted into PNG format and the corresponding segmentation masks are divided into a training set, a verification set and a test set according to a preset proportion.
4. The lumbar vertebrae segmentation method based on the multi-task fadnaet network model according to claim 3, wherein after dividing all the lumbar vertebrae images converted into PNG format and the corresponding segmentation masks thereof into a training set, a verification set and a test set according to a preset ratio, the method further comprises:
and performing model training on the multi-task FADNet network by using the training set, and performing verification and testing by using the verification set and the test set to obtain the multi-task FADNet network model.
5. The lumbar vertebra segmentation method based on the multi-task fadnaet network model according to claim 4, wherein the training the multi-task fadnaet network by using the training set, and verifying and testing by using the verification set and the test set, to obtain the multi-task fadnaet network model comprises:
setting the batch_size of training to 64 in the model training process;
setting the initialized learning rate as 1e-4, adding a learning rate attenuation strategy, and carrying out 5000 times of iteration, wherein the learning rate attenuation is 0.9 of the last learning rate;
setting an optimizer as an Adam optimizer;
setting a loss function as DICE loss;
and setting 1000 times of each iteration, performing one-time verification on the training set and the verification set, judging the network training stop time through an early-stop method, and obtaining the multi-task FADNet network model.
6. The lumbar spine segmentation method based on the multi-task fadnaet network model according to claim 1, wherein the attention mechanism network comprises a location attention mechanism network and a channel attention mechanism network;
wherein the location attention mechanism network is configured to selectively aggregate features of each location by a weighted sum of features at all locations;
the channel attention mechanism network is used for selectively emphasizing the channel mapping with interdependence by integrating the correlation features between all the channel mappings.
7. The method for lumbar segmentation based on a multi-tasking fadnaet network model according to claim 6, wherein the location attention mechanism network for selectively aggregating features of each location by a weighted sum of features at all locations comprises:
initializing to generate a position attention matrix for modeling the relationship between any two points;
performing matrix multiplication on the position attention matrix and the feature matrix to obtain a multiplication result;
and adding elements to the multiplication result and the feature matrix to obtain a result with certain characterization capability on global semantics.
8. The utility model provides a lumbar vertebrae segmentation device based on multitasking FADNet network model which characterized in that includes:
the image acquisition module is used for acquiring lumbar vertebra images to be segmented;
the lumbar vertebra segmentation result acquisition module is used for inputting the lumbar vertebra image into a preset multi-task FADNet network model and outputting a lumbar vertebra segmentation result;
the lumbar vertebra segmentation result comprises segmentation results of a plurality of parts in lumbar vertebra;
the multi-task FADNet network model is obtained based on model training of a multi-task FADNet network, and the multi-task FADNet network adopts a multi-layer convolution network, a downsampling network and an upsampling network structure for extracting features; meanwhile, a feature sharing mechanism is adopted in the up-sampling process, and up-sampling features of two network branches are shared to improve the segmentation precision; and adding a network of attention mechanisms in the jump connection between each pair of adjacent decoder branches and encoder branches of each layer to enhance the segmentation of boundary features.
9. An electronic device, the electronic device comprising: a processor and a memory storing computer program instructions;
the processor, when executing the computer program instructions, implements a lumbar spine segmentation method based on a multi-task fadnaet network model as claimed in any one of claims 1-7.
10. A computer readable storage medium, wherein computer program instructions are stored on the computer readable storage medium, and when executed by a processor, the computer program instructions implement the lumbar spine segmentation method based on the multi-tasking fadnat network model according to any one of claims 1 to 7.
CN202310275163.XA 2023-03-21 2023-03-21 Lumbar vertebrae segmentation method, device, electronic equipment and computer readable storage medium Pending CN116563314A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310275163.XA CN116563314A (en) 2023-03-21 2023-03-21 Lumbar vertebrae segmentation method, device, electronic equipment and computer readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310275163.XA CN116563314A (en) 2023-03-21 2023-03-21 Lumbar vertebrae segmentation method, device, electronic equipment and computer readable storage medium

Publications (1)

Publication Number Publication Date
CN116563314A true CN116563314A (en) 2023-08-08

Family

ID=87492286

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310275163.XA Pending CN116563314A (en) 2023-03-21 2023-03-21 Lumbar vertebrae segmentation method, device, electronic equipment and computer readable storage medium

Country Status (1)

Country Link
CN (1) CN116563314A (en)

Similar Documents

Publication Publication Date Title
CN111401201B (en) Aerial image multi-scale target detection method based on spatial pyramid attention drive
CN109086811B (en) Multi-label image classification method and device and electronic equipment
CN111291825B (en) Focus classification model training method, apparatus, computer device and storage medium
CN111325750B (en) Medical image segmentation method based on multi-scale fusion U-shaped chain neural network
CN115512169B (en) Weak supervision semantic segmentation method and device based on gradient and region affinity optimization
Liu et al. The measurement of Cobb angle based on spine X-ray images using multi-scale convolutional neural network
CN114861842B (en) Few-sample target detection method and device and electronic equipment
CN116543221A (en) Intelligent detection method, device and equipment for joint pathology and readable storage medium
CN114511581B (en) Multi-task multi-resolution collaborative esophageal cancer lesion segmentation method and device
CN116363150A (en) Hip joint segmentation method, device, electronic equipment and computer readable storage medium
CN113177554B (en) Thyroid nodule identification and segmentation method, system, storage medium and equipment
CN116152197A (en) Knee joint segmentation method, knee joint segmentation device, electronic equipment and computer readable storage medium
CN114066905A (en) Medical image segmentation method, system and device based on deep learning
CN116309636A (en) Knee joint segmentation method, device and equipment based on multi-task neural network model
CN116309612B (en) Semiconductor silicon wafer detection method, device and medium based on frequency decoupling supervision
CN116959307A (en) Hip arthroscope operation auxiliary teaching system based on virtual reality
CN116704549A (en) Position detection method, device, equipment and storage medium for three-dimensional space key points
CN110210314B (en) Face detection method, device, computer equipment and storage medium
CN116563314A (en) Lumbar vertebrae segmentation method, device, electronic equipment and computer readable storage medium
CN114118127B (en) Visual scene sign detection and recognition method and device
CN116188492B (en) Hip joint segmentation method, device, electronic equipment and computer readable storage medium
CN113256622A (en) Target detection method and device based on three-dimensional image and electronic equipment
CN113139617A (en) Power transmission line autonomous positioning method and device and terminal equipment
CN117350992A (en) Multi-task segmentation network metal implant identification method based on self-guiding attention mechanism
CN112464966A (en) Robustness estimation method, data processing method, and information processing apparatus

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination