CN117078692B - Medical ultrasonic image segmentation method and system based on self-adaptive feature fusion - Google Patents

Medical ultrasonic image segmentation method and system based on self-adaptive feature fusion Download PDF

Info

Publication number
CN117078692B
CN117078692B CN202311321509.1A CN202311321509A CN117078692B CN 117078692 B CN117078692 B CN 117078692B CN 202311321509 A CN202311321509 A CN 202311321509A CN 117078692 B CN117078692 B CN 117078692B
Authority
CN
China
Prior art keywords
features
ultrasonic image
adaptive
convolution
deep
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202311321509.1A
Other languages
Chinese (zh)
Other versions
CN117078692A (en
Inventor
王东骥
程海博
涂燕晖
陈一昕
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong Future Network Research Institute Industrial Internet Innovation Application Base Of Zijinshan Laboratory
Original Assignee
Shandong Future Network Research Institute Industrial Internet Innovation Application Base Of Zijinshan Laboratory
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong Future Network Research Institute Industrial Internet Innovation Application Base Of Zijinshan Laboratory filed Critical Shandong Future Network Research Institute Industrial Internet Innovation Application Base Of Zijinshan Laboratory
Priority to CN202311321509.1A priority Critical patent/CN117078692B/en
Publication of CN117078692A publication Critical patent/CN117078692A/en
Application granted granted Critical
Publication of CN117078692B publication Critical patent/CN117078692B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10132—Ultrasound image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20004—Adaptive image processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Medical Informatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Databases & Information Systems (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Ultra Sonic Daignosis Equipment (AREA)
  • Image Processing (AREA)

Abstract

The invention relates to the technical field of image processing, and discloses a medical ultrasonic image segmentation method and a system based on self-adaptive feature fusion, which acquire ultrasonic images; performing sample increment and marking transformation on the ultrasonic image to obtain an ultrasonic image data set; constructing an adaptive integrated neural network model based on a ResNet-101 network and a plurality of deep convolutions; training the self-adaptive integrated neural network model based on a Pytorch framework; inputting an ultrasonic image array into a ResNet-101 network to obtain shallow features and deep features, combining the deep features through a plurality of deep convolution processes to output multi-scale deep features, fusing the multi-scale deep features by using a residual jump structure, outputting fused features, and converting the fused features into a prediction category result by using a softmax function. The invention has the advantages of improving the segmentation efficiency, the detail recognition capability, the edge fineness and the scene adaptability, and realizing the effects of finer, quicker and more accurate medical ultrasonic image segmentation compared with the prior proposal.

Description

Medical ultrasonic image segmentation method and system based on self-adaptive feature fusion
Technical Field
The invention relates to the technical field of image processing, in particular to a medical ultrasonic image segmentation method and system based on self-adaptive feature fusion.
Background
Medical ultrasound imaging is a practical medical imaging technique that enables non-invasive visualization of soft tissue through acoustic wave propagation and reflection. Medical ultrasound image segmentation aims at automatically identifying and extracting objects of interest in ultrasound images, such as body tissues, organs or lesion areas, thereby increasing diagnostic efficiency and accuracy. The accurate segmentation can improve the reliability of quantitative analysis of the ultrasonic image, reduce the workload of doctors and provide support for subsequent diagnosis and operation.
However, the original ultrasonic image often contains a large amount of stray noise information, which prevents the identification and subsequent quantitative analysis of a key region, and the ultrasonic image needs to be processed and then used, while the traditional image segmentation method has the problems of loose structure, unsmooth information fusion among models, insufficient feature integration of a deep learning segmentation model, limited multi-scale semantic information expression, high calculation cost and the like, so that the image segmentation accuracy is low, and the segmentation efficiency is low.
Disclosure of Invention
In order to improve the image segmentation efficiency, the invention provides a medical ultrasonic image segmentation method and a system based on self-adaptive feature fusion, which are used for segmenting images more finely, more rapidly and more accurately.
In a first aspect, the present invention provides a medical ultrasound image segmentation method based on adaptive feature fusion, which adopts the following technical scheme:
a medical ultrasonic image segmentation method based on self-adaptive feature fusion comprises the following steps:
acquiring an ultrasonic image;
performing sample increment and marking transformation on the ultrasonic image to obtain an ultrasonic image data set;
constructing an adaptive integrated neural network model based on a ResNet-101 network and a plurality of deep convolutions;
training the self-adaptive integrated neural network model based on a Pytorch framework;
inputting an ultrasonic image array into a ResNet-101 network to obtain shallow features and deep features, combining the deep features through a plurality of deep convolution processes to output multi-scale deep features, fusing the multi-scale deep features by using a residual jump structure, outputting fused features, and converting the fused features into a prediction category result by using a softmax function.
Further, the performing sample increment and label transformation on the ultrasonic image to obtain an ultrasonic image data set includes:
and carrying out translation transformation, multi-scale rotation transformation, scale scaling, overturn inversion, noise augmentation and image enhancement on the ultrasonic image in sequence to obtain an ultrasonic image array.
Further, the constructing an adaptive integrated neural network model based on the ResNet-101 network and a plurality of deep convolutions includes:
the self-adaptive integrated neural network model comprises a basic network block, a multi-depth convolution block, a self-adaptive integrated block and an up-sampling block;
inputting an ultrasonic image array into a basic network block, wherein the basic network block adopts a ResNet-101 network, the ResNet-101 network comprises a plurality of residual blocks, each residual block uses 3X3 convolution layers and jump connection, the ResNet-101 network completes preliminary feature extraction of an image and outputs the results of the plurality of residual blocks, and the results of the plurality of residual blocks are shallow features and deep features of the network.
Further, the multi-depth convolution block comprises 4 convolution branches, wherein the first branch consists of 1×1 convolution layers, the 2 nd-4 th branch consists of 1×1 convolution layers and 2-4 3×3 convolution layers connected in series, and if the 3×3 convolution layers have missing features, zero padding is used for the missing features on each 3×3 convolution layer;
the multi-depth convolution block receives deep features output by ResNet-101, multi-branch small-size convolution with different depths is used for extracting multi-scale features in parallel, all outputs of four branches are combined and connected, and multi-scale deep features are output.
Further, the adaptive integrated block performs primary fusion on the multi-scale deep feature fusion based on a 3×3 fusion convolution to obtain primary fusion features;
the method comprises the steps of inputting primary fusion features into a residual jump structure, embedding a channel attention structure into the residual jump structure, mapping and converting the primary fusion features into D-dimensional weight vectors with global average pooling by using global pooling branches by the channel attention structure, wherein D represents the number of channels, multiplying the primary fusion features of each channel by corresponding elements in the weight vectors to obtain a plurality of vector values, performing matrix point addition on the vector multiplied vector values, and outputting the fusion features.
Further, the up-sampling block comprises up-sampling and a 1x1 convolution layer, the up-sampling is utilized to amplify the fusion characteristic, the 1x1 convolution layer reduces the channel number processing to one channel, the amplified fusion characteristic enters the channel processed by the 1x1 convolution layer, a characteristic diagram is output, and the obtained characteristic diagram is converted into a prediction type result through a softmax function.
Further, the training the adaptive integrated neural network model based on the Pytorch framework includes:
based on the Pytorch framework, mapping probability distribution distance between a prediction category result and a real label, training a network model by using an Adam algorithm, and supervising and training and optimizing the model by using a cross entropy loss function.
In a second aspect, a multi-scale medical ultrasound image segmentation system based on adaptive feature fusion, comprising:
a data acquisition module configured to scan the patient's body by the ultrasound probe module to acquire an ultrasound image;
the data conversion module is configured to perform sample increment and mark conversion on the ultrasonic image to form an ultrasonic image data set;
an adaptive integrated neural network model module configured to provide an adaptive integrated neural network model;
and the result output module is configured to input the real-time ultrasonic image into the trained self-adaptive integrated neural network model, and the self-adaptive integrated neural network model outputs an ultrasonic image segmentation result.
In a third aspect, a computer readable storage medium has stored therein a plurality of instructions adapted to be loaded by a processor of a terminal device and to perform the method and system for medical ultrasound image segmentation based on adaptive feature fusion.
In a fourth aspect, a terminal device includes a processor and a computer readable storage medium, the processor configured to implement instructions; the computer readable storage medium is for storing a plurality of instructions adapted to be loaded by a processor and to perform the method and system for medical ultrasound image segmentation based on adaptive feature fusion.
In summary, the invention has the following beneficial technical effects:
the invention builds a self-adaptive integrated neural network model, realizes the integration of shallow and deep features in the neural network for an ultrasonic image, and particularly provides a method for extracting multi-scale features by using a multi-depth convolution block, wherein multi-scale information can be obtained by using simple convolution of multi-branches 1×1 and 3×3 under very small calculation cost, and the invention provides a self-adaptive integrated block which can finish feature fusion in space and channel dimension by simply convoluting and global pooling after splicing deep and shallow features.
Under the condition of low calculation cost, the invention effectively improves the detail recognition capability, the edge fineness and the scene adaptability of the segmentation, realizes the medical ultrasonic image segmentation with higher speed and higher precision than the prior proposal, and provides powerful technical support for a clinician to better judge the illness state of a patient.
Drawings
FIG. 1 is a flow chart of a method and system for medical ultrasound image segmentation based on adaptive feature fusion in accordance with the present invention.
FIG. 2 is a flow chart of an adaptive integrated neural network model in a medical ultrasound image segmentation method and system based on adaptive feature fusion.
FIG. 3 is a block diagram of a multi-depth convolution block in a medical ultrasound image segmentation method and system based on adaptive feature fusion according to the present invention.
Fig. 4 is a diagram of the configuration of an adaptive integration block in a medical ultrasound image segmentation method and system based on adaptive feature fusion according to the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments.
The embodiment of the invention discloses a medical ultrasonic image segmentation method and system based on self-adaptive feature fusion.
Example 1
Referring to fig. 1, the medical ultrasound image segmentation method based on adaptive feature fusion according to the present embodiment includes:
scanning the body of a patient through a preset ultrasonic probe module to obtain an ultrasonic image;
performing sample increment and mark transformation on the ultrasonic image to form an ultrasonic image data set;
constructing an adaptive integrated neural network model, and inputting an ultrasonic image data set into the adaptive integrated neural network model for training;
inputting the real-time ultrasonic image into a trained self-adaptive integrated neural network model, and outputting an ultrasonic image segmentation result by the self-adaptive integrated neural network model.
The method specifically comprises the following steps:
firstly, acquiring ultrasonic images, scanning the bodies of a plurality of patients through ultrasonic probes preset in a hospital, and acquiring a plurality of groups of ultrasonic images for use by a subsequent training model if different patients possibly have different parameters such as age, wearing and the like.
Secondly, in order to make the model more robust, sample augmentation is needed to be carried out on the original ultrasonic image, corresponding changes are synchronously carried out on the real marked image, and according to the characteristics of the medical ultrasonic image, the sample augmentation method used in the proposal has the following steps under the premise of not changing the image property:
translation transformation, namely translating the image along the horizontal axis and the vertical axis, and expanding the number of samples.
The multi-scale rotation transformation is that the sample is randomly rotated within a certain angle range, and the missing part after rotation is filled by mirror image, so that the sample with more angles can be obtained compared with the existing method which only uses a plurality of fixed angle rotations.
Scaling, namely scaling the sample in a certain range to obtain samples with multiple resolutions.
Inversion, namely, performing inversion in the horizontal and vertical directions on the sample, and reversing the distribution of the sample.
Noise enhancement, namely adding different types and levels of noise into the image, and improving the robustness of the model. The true image of the mark remains unchanged.
Image enhancement such as brightness and contrast is adjusted. The marker image is unchanged.
By adopting the synchronous sample augmentation and the mark transformation, the scale of the ultrasonic image training set can be obviously expanded, the adaptability of the model to various conditions is improved, and the generalization performance of the model is enhanced. And meanwhile, the semantic consistency of the amplified sample and the mark is strictly ensured.
After a series of sample augmentation operations, a richer ultrasound image dataset is output.
And thirdly, constructing an adaptive integrated neural network model, wherein the adaptive integrated neural network model consists of a basic network block, a multi-depth convolution block, an adaptive integrated block and an up-sampling block. The basic network block is a ResNet-101 network, the multi-depth convolution block receives the features output by ResNet-101, different receptive field features are extracted through convolution operations of different depths, extraction of multi-scale features is completed, and finally up-sampling of the feature images is completed by using three self-adaptive integrated blocks, so that the network outputs the feature images with the same resolution as the marked images, and the self-adaptive integrated blocks perform weighted fusion on the features of different branches and introduce a attention mechanism in the channel dimension. The overall structure of the model is shown in fig. 2, and the specific structure of each module is as follows:
(1) The ResNet-101 network contains multiple residual blocks, each using 3x3 convolutional layers and a skip connection. The jump connection can directly transfer input to output, residual learning is realized, gradient disappearance or gradient explosion is avoided, meanwhile, the used ResNet-101 network calls an ImageNet data set, the number of samples is increased by utilizing the ImageNet data set, a large amount of pre-training is performed, rich general image characteristic representations are obtained from a large-scale image data set, and a good characteristic extraction basis is provided for an ultrasonic image processing task. The ResNet-101 network completes the primary feature extraction of the image through jump connection, and outputs the results of a plurality of residual blocks, namely the shallow features and the deep features of the network.
The convolution layer formula is as follows:
；(1)
wherein,Krepresenting the output channel index (i.e. the output channel index),crepresenting the index of the input channelm，n) Representing the coordinates of convolution kernels of the respective sizeshAndw(reference to 1x1 and 3x3 convolutions in this disclosure refers to the convolution kernel size),biasfor the corresponding offset of the channel(s),representing all positions of the convolution kernelm，n) And all input channelsc And (5) summing.
It should be noted that all convolution formulas in the whole adaptive integrated neural network model are the formulas.
(2) The multi-depth convolution block receives deep features output by ResNet-101, takes the deep features as input, and uses multi-branch small-size convolution with different depths to extract multi-scale deep features in parallel, namely, different network depths correspond to different receptive fields, so that different scales are mapped.
Referring to fig. 3, the present module contains 4 parallel branches. The first branch consists of only 1x1 convolution layers, which is used to reduce the number of channels, reduce the complexity of the model and preserve the original scale features. The 2-4 branches consist of 1x1 convolutional layers and 2-4 3x3 convolutional layers in series. The 3x3 convolution captures local features and the multi-layer stack extracts semantic information of different scales. In addition, zero padding operations are used on each 3x3 convolutional layer, i.e., zero padding around the input signature to ensure that the convolved output signature is consistent in size with the input. The multi-scale is extracted by a multi-branch simple convolution mode, so that the calculation efficiency is greatly improved, and information brought by pooling or cavity convolution is avoided. Batch normalization is used to reduce internal covariate offset after deconvolution, and then all the outputs of the four branches are combined and connected to output multi-scale deep features. The multi-depth convolution block provided by the invention only uses the simple 1 multiplied by 1 and 3 multiplied by 3 convolutions of multiple branches, so that the parameter quantity and the calculation cost are reduced to the greatest extent compared with the existing multi-scale feature extraction method, in addition, more detail features are reserved and the influence of grid effect is avoided because the multi-scale is acquired without pooling or cavity convolution.
And finally, using a ReLU function as an activation function to enable the multi-depth convolution block to learn more nonlinear characteristics, and obtaining better model performance.
The formula for the ReLU function as an activation function is as follows:
； (2)
wherein the method comprises the steps ofxFor input, ifThen->I.e. back to itself. If->ThenI.e. truncated to 0.ReLU is simply calculated and implementedNonlinear processing is performed, and the problem of gradient disappearance is avoided.
(3) The main body of the self-adaptive integrated block is formed by nesting a residual jump structure and a channel attention structure. Specifically, the module receives shallow features output from the ResNet-101 network and deep features output from the multi-depth convolution block, performs space preliminary fusion on the two features through 3×3 fusion convolution, and simultaneously realizes feature dimension reduction and reduces calculation amount. Further, a residual jump structure is added to the fused features, so that the features are further fused in a channel, feature fusion in two dimensions of space and channel is realized, compared with the existing method, the method is characterized in that feature dimension reduction is firstly carried out, and then the space fusion and the channel fusion are integrated into one module, so that complex multi-module fusion is avoided, and redundant calculation is reduced.
Then, considering that features from different channels generally have different degrees of importance to the semantic labeling task, embedding a channel attention structure in the residual structure, the structure uses global pooling branches to transform feature mapping into a D-dimensional (D represents the number of channels) weight vector with global averaging pooling, the D-dimensional represents the weight of each channel, i.e. the features of a certain channel are more important, then all features of each channel are multiplied by corresponding elements in the weight vector, and multiplication can enhance the output features of the channel with larger weight and more importance. The method for obtaining the channel information through global pooling also greatly reduces the parameter quantity and the calculated quantity of the channel attention.
And (5) performing matrix point addition after scalar multiplication, and outputting fusion characteristics. The construction of the adaptive integrated blocks is described with reference to fig. 4.
Wherein the matrix point addition formula is:
(3)
，i=1,2,…m，j=1,2,…n (4)
wherein matrix C is the result of the addition of the points of matrices a and B.iAndjrepresenting the matrix coordinates. I.e. the first of the result matrix CiLine 1jThe column elements are the direct addition of the corresponding position elements of matrix a and matrix B.
(4) The resulting feature map of the fused features is then converted to probability values for the predicted class by upsampling and a 1x1 convolution layer to reduce the number of channels to the number of classes, and then by a softmax function. Wherein, the softmax function formula is as follows:
,i=1, 2, … K (5)
wherein,represents +.>To the power. />Represents the sum of K, i.e. the sum of the exponents of all K. The output element belongs to the (0, 1) interval and the sum is 1, indicating the classification probability.
Fourth, based on Pytorch frame, mapping the probability distribution distance between the prediction category result and the real label, training a network model by using Adam algorithm, monitoring and training by cross entropy loss function, and optimizing the model.
The training method is implemented using a Pytorch framework, in particular using cross entropy as a loss function, mapping the probability distribution distance between the predicted outcome and the true annotation. During model training, all weight parameters are initialized with He, which takes into account the number of input elements, sampling from a uniform distribution.
The Adam optimization algorithm is basically a combination of Momentum and RMSprop.
1. Setting an initial value, generally initializing V dw =0；S dw =0；V db =0；S db =0；
2. In the t-th iteration, dw and db are calculated by a mini-batch gradient descent method;
3. calculating a Momentum exponential weighted average;
4. updating with RMSprop;
5. calculating correction deviation of Momentum and RMSprop;
and 6, updating the weight.
In the scheme, the initial learning rate is set to be 1e-3, the initial learning rate is attenuated in stages in the training process, the final attenuation is 1e-5, and the momentum value is set to be 0.9, so that the model is ensured to be sufficiently converged, wherein Adam uses an adaptive learning rate and momentum mechanism, and the convergence speed is higher. And simultaneously setting an early-stopping mechanism, and stopping training when the indexes of the model on the verification set are continuously 10 epochs (the iteration times) and are not improved any more, so as to prevent over-fitting. Further, the batch size is set to be 16, the weight parameter is penalized by L2 regularization, the strength is 1e-4, so that the model tends to select smaller weight, and the complexity of the model is reduced.
The self-adaptive integrated neural network model after training is used for model application, and a hospital can use the self-adaptive integrated neural network model to carry out intelligent segmentation prediction on a newly acquired ultrasonic image and output an ultrasonic image segmentation result by using the self-adaptive integrated neural network model. Accurately extracting lesion areas, organ boundaries and the like. Based on the segmentation result, quantitative analysis of the information of the size, the shape and the like of the lesions can be realized, and references are provided for doctors to judge the illness state and plan the treatment scheme.
Example 2
A multi-scale medical ultrasound image segmentation system based on adaptive feature fusion, comprising:
a data acquisition module configured to scan the patient's body by the ultrasound probe module to acquire an ultrasound image;
the data conversion module is configured to perform sample increment and mark conversion on the ultrasonic image to form an ultrasonic image data set;
an adaptive integrated neural network model module configured to provide an adaptive integrated neural network model;
and the result output module is configured to input the real-time ultrasonic image into the trained self-adaptive integrated neural network model, and the self-adaptive integrated neural network model outputs an ultrasonic image segmentation result.
A computer readable storage medium having stored therein a plurality of instructions adapted to be loaded and executed by a processor of a terminal device for the medical ultrasound image segmentation method and system based on adaptive feature fusion.
A terminal device comprising a processor and a computer readable storage medium, the processor configured to implement instructions; the computer readable storage medium is for storing a plurality of instructions adapted to be loaded by a processor and to perform the method and system for medical ultrasound image segmentation based on adaptive feature fusion.
The above embodiments are not intended to limit the scope of the present invention, so: all equivalent changes in structure, shape and principle of the invention should be covered in the scope of protection of the invention.

Claims (6)

1. A medical ultrasonic image segmentation method based on self-adaptive feature fusion is characterized in that: comprising the following steps:
acquiring an ultrasonic image;
performing sample increment and marking transformation on the ultrasonic image to obtain an ultrasonic image data set;
constructing an adaptive integrated neural network model based on a ResNet-101 network and a plurality of deep convolutions;
training the self-adaptive integrated neural network model based on a Pytorch framework;
inputting an ultrasonic image array into a ResNet-101 network to obtain shallow features and deep features, combining the deep features through a plurality of deep convolution processes to output multi-scale deep features, fusing the multi-scale deep features by using a residual jump structure, outputting fused features, and converting the fused features into a prediction category result by using a softmax function;
the construction of the adaptive integrated neural network model based on the ResNet-101 network and a plurality of deep convolutions comprises the following steps:
the self-adaptive integrated neural network model comprises a basic network block, a multi-depth convolution block, a self-adaptive integrated block and an up-sampling block;
inputting an ultrasonic image array into a basic network block, wherein the basic network block adopts a ResNet-101 network, the ResNet-101 network comprises a plurality of residual blocks, each residual block uses 3X3 convolution layers and jump connection, the ResNet-101 network completes the primary feature extraction of an image and outputs the results of the plurality of residual blocks, and the results of the plurality of residual blocks are the shallow features and the deep features of the network;
the multi-depth convolution block comprises 4 convolution branches, wherein the first branch consists of 1X1 convolution layers, the 2 nd-4 th branch consists of 1X1 convolution layers and 2-4 3X3 convolution layers which are connected in series, and if the 3X3 convolution layers have missing features, zero filling is used for the missing features on each 3X3 convolution layer;
the multi-depth convolution block receives deep features output by ResNet-101, multi-branch small-size convolution with different depths is used for extracting multi-scale features in parallel, all outputs of four branches are combined and connected, and multi-scale deep features are output;
the self-adaptive integrated block performs preliminary fusion on multi-scale deep feature fusion based on a 3×3 fusion convolution to obtain preliminary fusion features;
inputting the primary fusion features into a residual jump structure, embedding a channel attention structure into the residual jump structure, mapping and converting the primary fusion features into D-dimensional weight vectors with global average pooling by using global pooling branches by the channel attention structure, wherein D represents the number of channels, multiplying the primary fusion features of each channel by corresponding elements in the weight vectors to obtain a plurality of vector values, performing matrix point addition on the vector multiplied vector values, and outputting the fusion features;
the up-sampling block comprises up-sampling and a 1x1 convolution layer, the up-sampling is utilized to amplify the fusion characteristic, the 1x1 convolution layer reduces the channel number processing to one channel, the amplified fusion characteristic enters the channel processed by the 1x1 convolution layer, a characteristic diagram is output, and the obtained characteristic diagram is converted into a prediction type result through a softmax function.
2. The medical ultrasound image segmentation method based on adaptive feature fusion according to claim 1, wherein: the ultrasonic image is subjected to sample increment and marking transformation to obtain an ultrasonic image data set, and the method comprises the following steps:
and carrying out translation transformation, multi-scale rotation transformation, scale scaling, overturn inversion, noise augmentation and image enhancement on the ultrasonic image in sequence to obtain an ultrasonic image array.
3. The medical ultrasound image segmentation method based on adaptive feature fusion according to claim 1, wherein: training the adaptive integrated neural network model based on the Pytorch framework comprises the following steps:
based on the Pytorch framework, mapping probability distribution distance between a prediction category result and a real label, training a network model by using an Adam algorithm, and supervising and training and optimizing the model by using a cross entropy loss function.
4. A multi-scale medical ultrasound image segmentation system based on adaptive feature fusion, based on the medical ultrasound image segmentation method based on adaptive feature fusion of claim 1, comprising:
a data acquisition module configured to scan the patient's body by the ultrasound probe module to acquire an ultrasound image;
the data conversion module is configured to perform sample increment and mark conversion on the ultrasonic image to form an ultrasonic image data set;
an adaptive integrated neural network model module configured to provide an adaptive integrated neural network model;
and the result output module is configured to input the real-time ultrasonic image into the trained self-adaptive integrated neural network model, and the self-adaptive integrated neural network model outputs an ultrasonic image segmentation result.
5. A computer readable storage medium having stored therein a plurality of instructions adapted to be loaded by a processor of a terminal device and to perform a medical ultrasound image segmentation method based on adaptive feature fusion as claimed in claim 1.
6. A terminal device comprising a processor and a computer readable storage medium, the processor configured to implement instructions; a computer readable storage medium is for storing a plurality of instructions adapted to be loaded by a processor and to perform a medical ultrasound image segmentation method based on adaptive feature fusion as defined in claim 1.
CN202311321509.1A 2023-10-13 2023-10-13 Medical ultrasonic image segmentation method and system based on self-adaptive feature fusion Active CN117078692B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311321509.1A CN117078692B (en) 2023-10-13 2023-10-13 Medical ultrasonic image segmentation method and system based on self-adaptive feature fusion

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311321509.1A CN117078692B (en) 2023-10-13 2023-10-13 Medical ultrasonic image segmentation method and system based on self-adaptive feature fusion

Publications (2)

Publication Number Publication Date
CN117078692A CN117078692A (en) 2023-11-17
CN117078692B true CN117078692B (en) 2024-02-06

Family

ID=88702779

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311321509.1A Active CN117078692B (en) 2023-10-13 2023-10-13 Medical ultrasonic image segmentation method and system based on self-adaptive feature fusion

Country Status (1)

Country Link
CN (1) CN117078692B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117272245B (en) * 2023-11-21 2024-03-12 陕西金元新能源有限公司 Fan gear box temperature prediction method, device, equipment and medium
CN117708706B (en) * 2024-02-06 2024-05-28 山东未来网络研究院(紫金山实验室工业互联网创新应用基地) Method and system for classifying breast tumors by enhancing and selecting end-to-end characteristics
CN117982106B (en) * 2024-04-02 2024-06-25 天津市肿瘤医院（天津医科大学肿瘤医院） MRI image-based breast cancer chemotherapy curative effect prediction system and method

Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111507990A (en) * 2020-04-20 2020-08-07 南京航空航天大学 Tunnel surface defect segmentation method based on deep learning
CN112381097A (en) * 2020-11-16 2021-02-19 西南石油大学 Scene semantic segmentation method based on deep learning
CN112446890A (en) * 2020-10-14 2021-03-05 浙江工业大学 Melanoma segmentation method based on void convolution and multi-scale fusion
CN112861795A (en) * 2021-03-12 2021-05-28 云知声智能科技股份有限公司 Method and device for detecting salient target of remote sensing image based on multi-scale feature fusion
CN113077471A (en) * 2021-03-26 2021-07-06 南京邮电大学 Medical image segmentation method based on U-shaped network
CN113393469A (en) * 2021-07-09 2021-09-14 浙江工业大学 Medical image segmentation method and device based on cyclic residual convolutional neural network
CN113850825A (en) * 2021-09-27 2021-12-28 太原理工大学 Remote sensing image road segmentation method based on context information and multi-scale feature fusion
CN114419015A (en) * 2022-01-24 2022-04-29 东北大学 Brain function fusion analysis method based on multi-modal registration
WO2022241995A1 (en) * 2021-05-18 2022-11-24 广东奥普特科技股份有限公司 Visual image enhancement generation method and system, device, and storage medium
CN115393584A (en) * 2022-08-02 2022-11-25 哈尔滨理工大学 Establishment method based on multi-task ultrasonic thyroid nodule segmentation and classification model, segmentation and classification method and computer equipment
CN115424103A (en) * 2022-08-18 2022-12-02 重庆理工大学 Improved U-Net brain tumor segmentation method based on attention mechanism and multi-scale feature fusion
WO2023063874A1 (en) * 2021-10-14 2023-04-20 Exo Imaging, Inc. Method and system for image processing based on convolutional neural network
CN116385832A (en) * 2023-04-10 2023-07-04 思腾合力(天津)科技有限公司 Bimodal biological feature recognition network model training method
CN116563204A (en) * 2023-03-08 2023-08-08 江苏科技大学 Medical image segmentation method integrating multi-scale residual attention

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111739075B (en) * 2020-06-15 2024-02-06 大连理工大学 Deep network lung texture recognition method combining multi-scale attention
US11580646B2 (en) * 2021-03-26 2023-02-14 Nanjing University Of Posts And Telecommunications Medical image segmentation method based on U-Net

Patent Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111507990A (en) * 2020-04-20 2020-08-07 南京航空航天大学 Tunnel surface defect segmentation method based on deep learning
CN112446890A (en) * 2020-10-14 2021-03-05 浙江工业大学 Melanoma segmentation method based on void convolution and multi-scale fusion
CN112381097A (en) * 2020-11-16 2021-02-19 西南石油大学 Scene semantic segmentation method based on deep learning
CN112861795A (en) * 2021-03-12 2021-05-28 云知声智能科技股份有限公司 Method and device for detecting salient target of remote sensing image based on multi-scale feature fusion
CN113077471A (en) * 2021-03-26 2021-07-06 南京邮电大学 Medical image segmentation method based on U-shaped network
WO2022241995A1 (en) * 2021-05-18 2022-11-24 广东奥普特科技股份有限公司 Visual image enhancement generation method and system, device, and storage medium
CN113393469A (en) * 2021-07-09 2021-09-14 浙江工业大学 Medical image segmentation method and device based on cyclic residual convolutional neural network
CN113850825A (en) * 2021-09-27 2021-12-28 太原理工大学 Remote sensing image road segmentation method based on context information and multi-scale feature fusion
WO2023063874A1 (en) * 2021-10-14 2023-04-20 Exo Imaging, Inc. Method and system for image processing based on convolutional neural network
CN114419015A (en) * 2022-01-24 2022-04-29 东北大学 Brain function fusion analysis method based on multi-modal registration
CN115393584A (en) * 2022-08-02 2022-11-25 哈尔滨理工大学 Establishment method based on multi-task ultrasonic thyroid nodule segmentation and classification model, segmentation and classification method and computer equipment
CN115424103A (en) * 2022-08-18 2022-12-02 重庆理工大学 Improved U-Net brain tumor segmentation method based on attention mechanism and multi-scale feature fusion
CN116563204A (en) * 2023-03-08 2023-08-08 江苏科技大学 Medical image segmentation method integrating multi-scale residual attention
CN116385832A (en) * 2023-04-10 2023-07-04 思腾合力(天津)科技有限公司 Bimodal biological feature recognition network model training method

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
基于多尺度特征融合模型的遥感图像建筑物分割;徐胜军;欧阳朴衍;郭学源;Khan Taha Muthar;;计算机测量与控制(第07期);正文 *
深层聚合残差密集网络的超声图像左心室分割;吴宣言;缑新科;朱子重;魏域林;王凯;;中国图象图形学报(第09期);正文 *

Also Published As

Publication number Publication date
CN117078692A (en) 2023-11-17

Similar Documents

Publication Publication Date Title
CN110211140B (en) Abdominal blood vessel segmentation method based on 3D residual U-Net and weighting loss function
Cai et al. A review of the application of deep learning in medical image classification and segmentation
CN117078692B (en) Medical ultrasonic image segmentation method and system based on self-adaptive feature fusion
CN112529894B (en) Thyroid nodule diagnosis method based on deep learning network
CN110689543A (en) Improved convolutional neural network brain tumor image segmentation method based on attention mechanism
CN112258514B (en) Segmentation method of pulmonary blood vessels of CT (computed tomography) image
CN113450328B (en) Medical image key point detection method and system based on improved neural network
CN113516659A (en) Medical image automatic segmentation method based on deep learning
CN115496771A (en) Brain tumor segmentation method based on brain three-dimensional MRI image design
CN113570627B (en) Training method of deep learning segmentation network and medical image segmentation method
CN113223005B (en) Thyroid nodule automatic segmentation and grading intelligent system
CN112819831B (en) Segmentation model generation method and device based on convolution Lstm and multi-model fusion
CN111091575B (en) Medical image segmentation method based on reinforcement learning method
CN113034507A (en) CCTA image-based coronary artery three-dimensional segmentation method
CN114549538A (en) Brain tumor medical image segmentation method based on spatial information and characteristic channel
Sun et al. ISSMF: Integrated semantic and spatial information of multi-level features for automatic segmentation in prenatal ultrasound images
Aslam et al. Liver-tumor detection using CNN ResUNet
Li et al. AttentionNet: Learning where to focus via attention mechanism for anatomical segmentation of whole breast ultrasound images
Liu et al. 3-D prostate MR and TRUS images detection and segmentation for puncture biopsy
CN113538363A (en) Lung medical image segmentation method and device based on improved U-Net
WO2022227193A1 (en) Liver region segmentation method and apparatus, and electronic device and storage medium
CN113379691A (en) Breast lesion deep learning segmentation method based on prior guidance
Peng et al. 3D Segment and Pickup Framework for Pancreas Segmentation
CN117952995B (en) Cardiac image segmentation system capable of focusing, prompting and optimizing
Hwang et al. RBCA-Net: Reverse boundary channel attention network for kidney tumor segmentation in CT images

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant