WO2021243783A1 - 一种提取b超图像显著纹理特征的方法及其应用 - Google Patents

一种提取b超图像显著纹理特征的方法及其应用 Download PDF

Info

Publication number
WO2021243783A1
WO2021243783A1 PCT/CN2020/099590 CN2020099590W WO2021243783A1 WO 2021243783 A1 WO2021243783 A1 WO 2021243783A1 CN 2020099590 W CN2020099590 W CN 2020099590W WO 2021243783 A1 WO2021243783 A1 WO 2021243783A1
Authority
WO
WIPO (PCT)
Prior art keywords
liver fibrosis
ultrasound
model
ultrasound image
block
Prior art date
Application number
PCT/CN2020/099590
Other languages
English (en)
French (fr)
Inventor
郑敏
阮东升
郑能干
施毓
金林峰
Original Assignee
浙江大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 浙江大学 filed Critical 浙江大学
Priority to US17/398,142 priority Critical patent/US20210374956A1/en
Publication of WO2021243783A1 publication Critical patent/WO2021243783A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images

Definitions

  • the invention belongs to the technical field of B-ultrasound diagnosis, and specifically relates to a method for extracting significant texture features of a B-ultrasound image and its application, in particular to a B-ultrasound non-invasive diagnosis method for liver fibrosis based on a context-activated residual network.
  • Liver fibrosis is a key step in the development of various chronic liver diseases to cirrhosis and an important link that affects the prognosis of chronic liver diseases.
  • liver biopsy is regarded as the gold standard for diagnosing liver fibrosis.
  • this method is very traumatic and makes it difficult for many patients to accept.
  • domestic and foreign researchers have devoted themselves to the research of non-invasive diagnosis methods for liver fibrosis.
  • Ultrasound imaging diagnosis is widely used in the diagnosis of liver fibrosis due to its safety, low cost, non-invasive, real-time and other advantages.
  • the diagnostic accuracy of ultrasound liver fiber is very dependent on the doctor's clinical diagnosis experience. In order to solve this problem, it is particularly important to develop an objective, accurate and intelligent computer-aided diagnosis method.
  • B-ultrasonic liver fibrosis diagnosis methods based on computer-aided diagnosis are mainly divided into two categories: based on traditional machine learning methods and based on deep learning methods.
  • the former usually uses one or more texture extraction algorithms, such as gray-level co-occurrence matrix and wavelet transform, to model the texture information of the liver parenchymal region in the B-ultrasound image, and then use support vector machines and other classifiers to complete the classification of liver fibrosis.
  • Such methods often cannot be applied to large-scale data sets and cannot accurately classify liver fibrosis.
  • the latter uses the convolutional neural network in deep learning to achieve end-to-end liver fibrosis classification. Compared with the former, the latter usually obtains better classification performance and is easy to train and promote.
  • the present invention provides a method for extracting significant texture features of B-ultrasound images based on context-activated residual network for non-invasive diagnosis of B-ultrasound liver fibrosis.
  • This method designs a lightweight channel attention mechanism, which can effectively capture the texture characteristics of different liver fibrosis in B-ultrasound images and accurately grade liver fibrosis.
  • a method for extracting significant texture features of B-ultrasound images based on context activation residual network including the following steps:
  • each B-mode image Three image blocks of different sizes and resolutions are intercepted in each B-mode image (preferably: the resolution size is 60 ⁇ 60, 100 ⁇ 100, and 120 ⁇ 120, respectively, and the number of interceptions for each resolution is 2 Up to 4 photos.), data enhancement.
  • the above-mentioned image block data set is divided into a training set and a test set according to a certain ratio (first training set: test set is 7:3), in which the B-ultrasound image block data is used as the input data of the model, and the liver biopsy result is used as the model's input data. Label data.
  • liver fibrosis is divided into three categories: normal or mild liver fibrosis (S0-S1), moderate liver fibrosis (S2-S3) and liver cirrhosis (S4).
  • the deep residual network based on the context activation residual network is formed by stacking multiple context activation residual blocks.
  • the context activation residual block consists of two parts: the residual block and the context activation block.
  • the residual block is used to extract texture features in the B-mode image, and each channel of the residual block is responsible for extracting texture information of different features.
  • the context activation block aims to strengthen the important texture features in the residual block, while suppressing the useless texture features, so that the residual block can extract more prominent texture features in the B-mode image.
  • the function expression of the context activation residual block is as follows:
  • x and y are the input and output of the residual block respectively.
  • BN( ⁇ ) is a batch normalization operation
  • ReLU( ⁇ ) is a modified linear unit
  • F( ⁇ ) is a context activation block.
  • W 1 and W 3 are 1 ⁇ 1 convolutions
  • W 2 is 3 ⁇ 3 convolutions.
  • the context activation block F( ⁇ ) mainly includes three operations: global context aggregation, group normalization, and context activation.
  • W and H are the width and length of the feature map
  • C represents the number of channels of the feature map
  • Grouping normalization aims to eliminate the inconsistency of texture feature distribution caused by different samples and enhance the robustness of the model.
  • Context activation aims to learn the importance weight of each channel through global context information.
  • the importance weight describes the importance of the texture feature learned by each channel. The larger the value, the more important the texture feature.
  • the specific process is: perform a simple linear transformation on each channel and use the sigmoid function ⁇ to normalize it to 0 to 1, as shown below:
  • the output of the context activation block can be expressed as
  • the context activation block is embedded into the residual block, and the output of the residual block can be reformulated as:
  • the specific method for pre-training the non-invasive diagnosis model of liver fibrosis using ImageNet in the step (3) is as follows:
  • the training set in the ImageNet dataset is used to train the non-invasive diagnosis model of liver fibrosis, the input is natural images, and the label is the category of each image.
  • the cross entropy between the output value of the liver fibrosis non-invasive diagnosis model and the label is used as the objective function, and the weights in the model are continuously calculated and updated through the back propagation algorithm and the gradient descent method until the value of the objective function is less than the set value When the threshold or the total number of training is reached, the pre-training of the non-invasive diagnosis model for liver fibrosis is completed.
  • the training set is obtained, the B-ultrasound image blocks of different resolutions are uniformly adjusted to a resolution of 120 ⁇ 120, and they are used as the input of the pre-trained B-ultrasound non-invasive diagnosis model for liver fibrosis.
  • the final output layer size of the B-ultrasound non-invasive diagnosis model of liver fibrosis was changed from 1000 to 3, and the pathological results of liver biopsy were used as labels.
  • the cross-entropy between the output value of the liver fibrosis non-invasive diagnosis model and the label is still used as the objective function, and the weights in the model are fine-tuned through the backpropagation algorithm and gradient descent method until the value of the objective function is less than the set threshold or reaches the total value.
  • the specific process of obtaining the liver fibrosis grading result of the test set in the step (5) is as follows:
  • step (2) the test set is obtained, the B-mode image blocks of different resolutions are uniformly adjusted to a resolution of 140 ⁇ 140, and the central area of the image block is intercepted, and the resolution size is 120 ⁇ 120. It is input into the fine-tuned B-ultrasound non-invasive diagnosis model of liver fibrosis, and the category corresponding to the largest value in the output vector is used as the final liver fibrosis grading result of the B-ultrasound image.
  • the present invention also provides a B-ultrasound liver fibrosis non-invasive diagnosis method based on the context-activated residual network, medical equipment using the method for extracting significant texture features of the B-ultrasound image, and its application in the non-invasive diagnosis of liver fibrosis.
  • the present invention proposes a new attention mechanism network.
  • the network proposes a channel attention module, context activation residual block, which can use global context feature information to learn the texture feature of each channel in the network feature Importance degree, strengthen important texture features, suppress useless texture features and noise, thereby effectively improving the network's ability to model B-mode image texture.
  • the context activation residual network is formed by stacking multiple context activation residual blocks.
  • the context-activated residual network is designed to effectively model B-mode liver fibrosis texture information.
  • the network uses global context information to enhance important texture features and suppress useless texture features, making the deep residual network capture more prominent B-mode images
  • the texture information can be divided into two stages: training and testing.
  • training phase the B-ultrasound image block is used as input, and the pathological result of liver biopsy is used as the label training context to activate the residual network.
  • the testing phase input the B-ultrasound image block into the trained non-invasive diagnosis model of liver fibrosis to get the liver fibrosis grading of each ultrasound image.
  • the invention realizes the rapid and accurate estimation of the liver fibrosis classification of the B-ultrasound image from the perspective of data driving.
  • Figure 1 shows the context activation block diagram.
  • Figure 2 shows the residual block diagram
  • Figure 3 is the context activation residual block diagram.
  • Figure 4 is the overall framework of the B-ultrasonic liver fibrosis significant texture extraction method
  • the non-invasive diagnosis method of B-ultrasound liver fibrosis based on the context activated residual network of the present invention specifically includes the following steps:
  • liver fibrosis Construct a multi-center B-ultrasound liver fibrosis data set, including B-ultrasound images and pathological results of liver biopsy; according to the METAVIR scoring system and combined with clinical treatment experience, liver fibrosis is divided into three categories: normal or mild Liver fibrosis (S0-S1), moderate liver fibrosis (S2-S3) and liver cirrhosis (S4).
  • S0-S1 normal or mild Liver fibrosis
  • S2-S3 moderate liver fibrosis
  • S4 liver cirrhosis

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Biomedical Technology (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Multimedia (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Medical Informatics (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Public Health (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Pathology (AREA)
  • Databases & Information Systems (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • Ultra Sonic Daignosis Equipment (AREA)

Abstract

一种提取B超图像显著纹理特征的方法及其应用，公开了一种新的通道注意力机制网络，上下文激活残差网络，有效地建模B超肝纤维化纹理信息，该网络利用全局上下文信息加强重要的纹理特征并抑制无用的纹理特征，使得深度残差网络捕捉B超图像中更显著的纹理信息。其过程分为两个阶段：训练和测试。在训练阶段，将Ｂ超图像块作为输入，肝穿活检的病理结果作为标签训练上下文激活残差网络。在测试阶段，将Ｂ超图像块输入到训练好的肝纤维化无创诊断模型中即可得到每张超声图像的肝纤维化分级。该方法从数据驱动的角度实现了快速且准确地估计Ｂ超图像的肝纤维化分级。

Description

一种提取B超图像显著纹理特征的方法及其应用 技术领域
本发明属于B超诊断技术领域，具体涉及一种提取B超图像显著纹理特征的方法及其应用，尤其是基于上下文激活残差网络的B超肝纤维化无创诊断方法。
背景技术
肝纤维化是各种慢性肝病向肝硬化发展过程中的关键步骤和影响慢性肝病预后的重要环节。在临床诊断中，肝脏组织活检被视为诊断肝纤维化的金标准，然而，该方法创伤性很大导致很多患者不易接受。近年来，国内外学者致力于无创性肝纤维化诊断方法的研究。超声影像诊断因其安全，廉价，无创，实时等优点被广泛地应用在肝纤维化诊断中。然而，受限于超声成像质量，超声肝纤维的诊断准确率非常依赖于医生的临床诊断经验。为了解决这一问题，发展一种客观，准确和智能的计算机辅助诊断方法显得尤为重要。
目前，基于计算机辅助诊断的B超肝纤维化诊断方法主要分为两类：基于传统机器学习方法和基于深度学习方法。前者通常采用一种或多种纹理提取算法，如灰度共生矩阵和小波变换，建模B超图像中肝实质区域的纹理信息，然后采用支持向量机等分类器完成对肝纤维化的分级。此类方法往往无法应用到大规模的数据集中且无法准确地分级肝纤维化。后者采用深度学习中的卷积神经网络实现端到端的肝纤维化分级，相比于前者，后者通常能获得更好的分类性能，且易于训练和推广。然而，现有的此类方法由于缺乏额外的监督，网络无法精准地提取B超图像中肝纤维化的纹理特征，导致诊断率非常低，很难应用到临床诊断中。因此，如何简单有效地捕捉B超图像中最关键的纹理特征是精确诊断肝纤维化的一个关键性问题。
发明内容
针对现有技术所存在的上述技术问题，本发明提供了一种用于B超肝纤维化无创诊断的基于上下文激活残差网络的提取B超图像显著纹理特征的方法。该方法设计了一种轻量级的通道注意力机制，能够有效地捕捉B超图像中不同肝纤维化的纹理特征，精准分级肝纤维化。
一种基于上下文激活残差网络提取B超图像显著纹理特征的方法，包括如下步骤：
(1)建立多中心的超声影像数据集，包括不同病人的B超图像和相对应的肝活检结果。
(2)在每一张B超图像中截取三种不同大小分辨率的图像块(优选：分辨率大小分别为60×60，100×100和120×120，每种分辨率的截取数量在2到4张。)，进行数据增强。将上述的图像块数据集按照一定的比例(优先训练集：测试集为7：3)，划分为训练集和测试集，其中B超图像块数据作为模型的输入数据，肝活检结果作为模型的标签数据。
(3)利用ImageNet 2012数据集对基于上下文激活残差网络的深度残差网络进行迁移学习，获得预训练后的肝纤维化分级模型。
(4)将步骤(2)中获得的B超影像训练集微调预训练后的肝纤维化分级模型，最终得到基于上下文激活残差网络的B超肝纤维化分级模型。
(5)将步骤(2)中获得的B超影像测试集输入到B超肝纤维化分级模型中，从而得到测试集中B超图像的肝纤维化分级结果。
根据METAVIR评分系统并结合临床治疗经验，将肝纤维化分为三个类别：正常或轻度肝纤维化(S0-S1)，中度肝纤维化(S2-S3)和肝硬化(S4)。
所述的基于上下文激活残差网络的深度残差网络由多个上下文激活残差块堆叠而成。上下文激活残差块由两部分组成：残差块和上下文激活块。残差块用于提取B超图像中的纹理特征，残差块的每一个通道负责提取不同特征的纹理信息。上下文激活块旨在加强残差块中重要的纹理特征，同时抑制无用的纹理特征，使得残差块能够提取B超图像中更为显著的纹理特征。上下文激活残差块的函数表达式如下：
y＝f(x)+x＝ReLU(F(BN(W 3g(W 2g(W 1x)))))+x
g(·)＝ReLU(BN(·))
其中：x和y分别为残差块的输入和输出。BN(·)为批归一化操作，ReLU(·)为修正线性单元，F(·)为上下文激活块。W 1和W 3为1×1卷积，W 2为3×3卷积。
上下文激活块F(·)主要包含3个操作：全局上下文聚合，分组归一化，上下文激活。为了简化，令o＝F(·)，全局上下文聚合旨在获得全局的纹理信息，具体为通过全局平均池化操作获得通道表征矢量z＝[z 1，z 2，...，z k，...，z c]：
Figure PCTCN2020099590-appb-000001
其中，W和H为特征图的宽度和长度，C表示特征图的通道数量，k∈{1，2，...，C}。分组归一化旨在消除由不同样本所引起的纹理特征分布不一致性，增强模型的鲁棒性，具体为将通道表征矢量z按通道维度进行分组，然后对每一组内的特征矢量进行归一化操作，得到归一化后的通道表征矢量v＝[v 1，...，v i，...，v G]，v i可以表示为：
Figure PCTCN2020099590-appb-000002
Figure PCTCN2020099590-appb-000003
其中：p i＝[z mi+1,...,z m(i+1)],
Figure PCTCN2020099590-appb-000004
G表示分组的数量，S i表示第i组的通道索引集合，∈为一个小的常数，为了数值计算上的稳定。上下文激活旨在通过全局上下文信息，学习每个通道的重要性权重，重要性权重刻画了每个通道学习到的纹理特征的重要性程度，数值越大表示该纹理特征越重要。具体过程为：对每一个通道执行一个简单的线性变换并使用sigmoid函数δ将其归一化到0到1，如下所示：
a＝δ(β·v+γ)，
其中β和γ为可学习的权重和偏置，·表示对应通道相乘。利用学习到的纹理重要性权重重新调整输入，上下文激活块的输出可以表述为
Figure PCTCN2020099590-appb-000005
最后，将上下 文激活块嵌入到残差块中，残差块的输出可以被重新表述为：
Figure PCTCN2020099590-appb-000006
所述的步骤(3)中使用ImageNet对肝纤维化无创诊断模型预训练的具体方法如下：
将ImageNet数据集中的训练集用于肝纤维化无创诊断模型的训练，输入为自然图像，标签为每个图像的类别。以肝纤维化无创诊断模型的输出值与标签之间的交叉熵作为目标函数，通过反向传播算法和梯度下降法不断地计算和更新该模型中的权重，直到目标函数的值小于设定的阈值或到达总的训练次数，肝纤维化无创诊断模型预训练完成。
所述的步骤(4)中B超肝纤维化无创诊断模型的微调具体过程如下：
根据步骤(2)获得训练集，将不同分辨率的B超图像块统一调整到120×120的分辨率，将其作为预训练后B超肝纤维化无创诊断模型的输入。将B超肝纤维化无创诊断模型的最后的输出层大小从1000改为3，肝穿活检的病理结果作为标签。依然以肝纤维化无创诊断模型的输出值与标签之间的交叉熵作为目标函数，通过反向传播算法和梯度下降法微调模型中的权重，直到目标函数的值小于设定的阈值或到达总的训练次数，最终得到基于上下文激活残差网络的B超肝纤维化无创诊断模型。
所述的步骤(5)中获得测试集的肝纤维化分级结果的具体过程如下：
根据步骤(2)获得测试集，将不同分辨率的B超图像块统一调整到140×140的分辨率，截取图像块的中心区域，分辨率大小为120×120。将其输入到微调后的B超肝纤维化无创诊断模型中，将输出矢量中最大的值所对应的类别作为该B超图像的最终肝纤维化分级结果。
另外本发明还提供了一种基于上下文激活残差网络的B超肝纤维化无创诊断方法和利用提取B超图像显著纹理特征的方法的医疗设备及其在肝纤维化无创诊断中的应用。
本发明提出了一种新的注意力机制网络，该网络提出了一种通道注意力模块，上下文激活残差块，其能够利用全局的上下文特征信息，学习网络特征中每一个通道的纹理特征的重要性程度，加强重要的纹理特征，抑制无用的纹理特征和噪声，从而有效地提高网络对B超图像纹理建模能力。
上下文激活残差网络由多个上下文激活残差块堆叠而成。上下文激活残差网络，旨在有效地建模B超肝纤维化纹理信息，该网络利用全局上下文信息加强重要的纹理特征并抑制无用的纹理特征，使得深度残差网络捕捉B超图像中更显著的纹理信息。其过程主要可以分为两个阶段：训练和测试。在训练阶段，将B超图像块作为输入，肝穿活检的病理结果作为标签训练上下文激活残差网络。在测试阶段，将B超图像块输入到训练好的肝纤维化无创诊断模型中即可得到每张超声图像的肝纤维化分级。本发明从数据驱动的角度实现了快速且准确地估计B超图像的肝纤维化分级。
附图说明
图1为上下文激活块图。
图2为残差块图。
图3为上下文激活残差块图。
图4为B超肝纤维化显著纹理提取方法的整体框架
具体实施方式
为了更为具体地描述本发明，下面结合附图及具体实施方式对本发明的技术方案进行详细说明。
本发明基于上下文激活残差网络的B超肝纤维化无创诊断方法，具体包括如下步骤：
S1.构建多中心的B超肝纤维化数据集，包括B超图像和肝穿活检的病理结果；根据METAVIR评分系统并结合临床治疗经验，将肝纤维化分为三个类别：正常或轻度肝纤维化(S0-S1)，中度肝纤维化(S2-S3)和肝硬化(S4)。将肝穿活检的病理结果作为标签，记为l＝[l 1，l 2，l 3]。
S2.从每张B超图像中截取3种不同大小分辨率的图像块，分辨率大小分别为60×60，100×100和120×120。每种分辨率的截取数量在2到4张。将获得的B超图像块按照一定比例(训练集：测试集为7:3)划分为训练集和测试集。
S3.将上下文激活块(图1)嵌入到残差块(图2)中，构成上下文激活残差块，其结构如图3所示，设定网络深度为d，简单地堆叠这个残差块，建立B超肝纤维化无创诊断模型。
S4.利用ImageNet数据集对B超肝纤维化无创诊断模型进行预训练，通过 反向传播算法和梯度下降算法不断更新模型中的权重参数。
S5.将B超图像训练集中的图像块统一调整到120×120，作为B超肝纤维化无创诊断模型的输入，相对应的肝穿活检病理结果作为标签，通过反向传播算法和梯度下降算法进一步更新模型中的权重参数。
S6.将B超图像测试集中图像调整到140×140，截取中心区域120×120，输入到训练好B超肝纤维化无创诊断模型中，得到输出矢量
Figure PCTCN2020099590-appb-000007
将矢量
Figure PCTCN2020099590-appb-000008
中最大的值的所对应的类别即为该B超图像的肝纤维化分级。

Claims (10)

  1. 一种提取B超图像显著纹理特征的方法，包括如下步骤：
    (1)建立多中心的超声影像数据集，包括不同病人的B超图像和相对应的肝活检结果；
    (2)在每一张B超图像中截取三种不同大小分辨率的图像块进行数据增强；将上述的图像块数据集按照一定的比例，划分为B超影像训练集和B超影像测试集，其中B超图像块数据作为模型的输入数据，肝活检结果作为模型的标签数据；
    (3)利用ImageNet 2012数据集对深度上下文激活残差网络进行迁移学习，获得预训练后的肝纤维化分级模型；
    (4)将步骤(2)中获得的B超影像训练集微调预训练后的肝纤维化分级模型，最终得到基于上下文激活残差网络的B超肝纤维化分级模型；
    (5)将步骤(2)中获得的B超影像测试集输入到步骤(3)肝纤维化分级模型中，从而得到B超影像测试集中B超图像的肝纤维化分级结果；
    步骤(3)所述深度上下文激活残差网络由多个上下文激活残差块堆叠而成，上下文激活残差块由两部分组成：残差块和上下文激活块；残差块用于提取B超图像中的纹理特征，残差块的每一个通道负责提取不同特征的纹理信息；上下文激活块用于加强残差块中重要的纹理特征，同时抑制无用的纹理特征，使得残差块能够提取B超图像中更为显著的纹理特征，嵌入上下文激活块的残差块的函数表达式如下：
    y＝f(x)+x＝ReLU(F(BN(W 3g(W 2g(W 1x)))))+x
    g(·)＝ReLU(BN(·))
    其中：x和y分别为残差块的输入和输出；BN(·)为批归一化操作，ReLU(·)为修正线性单元，F(·)为上下文激活块；W 1和W 3为1×1卷积，W 2为3×3卷积。
  2. 根据权利要求1所述的方法，其特征在于：上下文激活块F(·)主要包含3个操作：用于获得全局的纹理信息的全局上下文聚合；用于消除由不同样本所引起的纹理特征分布不一致性，增强模型的鲁棒性的分组归一化；及通过全局上下文信息，学习每个通道的重要性权重，重要性权重刻画了每个通道学习 到的纹理特征的重要性程度，数值越大表示该纹理特征越重要的上下文激活。
  3. 根据权利要求2所述的方法，其特征在于：
    为了简化，令o＝F(·)，
    全局上下文聚合具体为通过全局平均池化操作获得通道表征矢量z＝[z 1，z 2，…，z k，…，z c]：
    Figure PCTCN2020099590-appb-100001
    其中，W和H为特征图的宽度和长度，C表示特征图的通道数量，k∈{1，2，…，C}，i和j表示特征图中坐标为(i，j)的空间位置点
    分组归一化为将通道表征矢量z按通道维度进行分组，然后对每一组内的特征矢量进行归一化操作，得到归一化后的通道表征矢量v＝[v 1，…，v i，…，v G]，v i可以表示为：
    Figure PCTCN2020099590-appb-100002
    Figure PCTCN2020099590-appb-100003
    其中：
    Figure PCTCN2020099590-appb-100004
    G表示分组的数量，S i表示第i组的通道索引集合，ε为一个小的常数，为了数值计算上的稳定，n表示通道索引，i表示组索引，μ i和σ i分别表示第i组特征的均值和方差，v i表示第i组归一化后的特征向量；
    上下文激活具体过程为：对每一个通道执行一个简单的线性变换并使用sigmoid函数δ将其归一化到0到1，如下所示：
    a＝δ(β·v+γ)，
    其中β和γ为可学习的权重和偏置，·表示对应通道相乘；利用学习到的 纹理重要性权重重新调整输入，上下文激活块的输出可以表述为
    Figure PCTCN2020099590-appb-100005
    最后，将上下文激活块嵌入到残差块中，残差块的输出可以被重新表述为：
    Figure PCTCN2020099590-appb-100006
  4. 根据权利要求1所述的方法，其特征在于：所述的步骤(3)中使用ImageNet对肝纤维化无创诊断模型预训练的具体方法如下：
    将ImageNet数据集中的B超影像训练集用于肝纤维化无创诊断模型的训练，输入为自然图像，标签为每个图像的类别；以肝纤维化无创诊断模型的输出值与标签之间的交叉熵作为目标函数，通过反向传播算法和梯度下降法不断地计算和更新该模型中的权重，直到目标函数的值小于设定的阈值或到达总的训练次数，肝纤维化无创诊断模型预训练完成。
  5. 根据权利要求1所述方法，其特征在于：所述的步骤(4)中B超肝纤维化无创诊断模型的微调具体过程如下：
    根据步骤(2)获得B超影像训练集，将不同分辨率的B超图像块统一调整到120×120的分辨率，将其作为预训练后B超肝纤维化无创诊断模型的输入；将B超肝纤维化无创诊断模型的最后的输出层大小从1000改为3，肝穿活检的病理结果作为标签；依然以肝纤维化无创诊断模型的输出值
    Figure PCTCN2020099590-appb-100007
    与标签l＝[l 1，l 2，l 3]之间的交叉熵作为目标函数，通过反向传播算法和梯度下降法微调模型中的权重，直到目标函数的值小于设定的阈值或到达总的训练次数，最终得到基于上下文激活残差网络的B超肝纤维化无创诊断模型。
  6. 根据权利要求5所述的方法，其特征在于：所述的肝纤维化无创诊断模型的交叉熵目标函数表达式如下所示：
    Figure PCTCN2020099590-appb-100008
    其中m为总的训练样本数，
    Figure PCTCN2020099590-appb-100009
    表示B超肝纤维化无创诊断模型的输出结果，l i为0或者1，当且仅当第i类处为1。θ表示模型中的训练参数。
  7. 根据权利要求1所述的方法，其特征在于：所述的步骤(5)中获得B超影像测试集的肝纤维化分级结果的具体过程如下：
    根据步骤(2)获得B超影像测试集，将不同分辨率的B超图像块统一 调整到140×140的分辨率，截取图像块的中心区域，分辨率大小为120×120。将其输入到微调后的B超肝纤维化无创诊断模型中，得到输出矢量
    Figure PCTCN2020099590-appb-100010
    将输出矢量中最大的值所对应的类别作为该B超图像的最终肝纤维化分级结果。
  8. 权利要求1所述方法在肝纤维化诊断中的应用。
  9. 利用权利要求1所述方法的医疗设备。
  10. 权利要求9所述医疗设备在肝纤维化无创诊断中的应用。
PCT/CN2020/099590 2020-05-31 2020-06-30 一种提取b超图像显著纹理特征的方法及其应用 WO2021243783A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/398,142 US20210374956A1 (en) 2020-05-31 2021-08-10 Method for Extracting Significant Texture Features of B-ultrasonic Image and Application Thereof

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010487055.5A CN111783792B (zh) 2020-05-31 2020-05-31 一种提取b超图像显著纹理特征的方法及其应用
CN202010487055.5 2020-05-31

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/398,142 Continuation US20210374956A1 (en) 2020-05-31 2021-08-10 Method for Extracting Significant Texture Features of B-ultrasonic Image and Application Thereof

Publications (1)

Publication Number Publication Date
WO2021243783A1 true WO2021243783A1 (zh) 2021-12-09

Family

ID=72753680

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/099590 WO2021243783A1 (zh) 2020-05-31 2020-06-30 一种提取b超图像显著纹理特征的方法及其应用

Country Status (2)

Country Link
CN (1) CN111783792B (zh)
WO (1) WO2021243783A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114897779A (zh) * 2022-04-12 2022-08-12 华南理工大学 基于融合注意力的宫颈细胞学图像异常区域定位方法及装置
CN117636064A (zh) * 2023-12-21 2024-03-01 浙江大学 一种基于儿童病理切片的神经母细胞瘤智能分类系统

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112862797B (zh) * 2021-02-23 2024-03-19 复旦大学附属华山医院 一种肝纤维化无损预测方法及系统
CN113033667B (zh) * 2021-03-26 2023-04-18 浙江机电职业技术学院 一种超声图像两阶段深度学习的乳腺肿瘤分类方法和装置

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
UA79297C2 (en) * 2005-02-25 2007-06-11 Inst Pediatrics Obstetrics Method for the diagnosis of the chronic diffuse lesions of the liver
JP4744833B2 (ja) * 2004-09-24 2011-08-10 株式会社東芝 超音波診断装置
CN102334122A (zh) * 2009-02-26 2012-01-25 昂热大学 肝纤维化或肝硬化的改进诊断
CN103169506A (zh) * 2013-03-19 2013-06-26 安徽皖仪科技股份有限公司 一种自动识别肝癌的超声诊断装置和方法
WO2014011925A2 (en) * 2012-07-11 2014-01-16 University Of Mississippi Medical Center Method for the detection and staging of liver fibrosis from image acquired data
CN103919576A (zh) * 2014-04-18 2014-07-16 上海市杨浦区中心医院 肝实质超声无向性纹理定量测量仪及肝实质纹理测量方法
CN105956620A (zh) * 2016-04-29 2016-09-21 华南理工大学 一种基于稀疏表示的肝脏超声图像识别方法
CN107014996A (zh) * 2009-03-19 2017-08-04 昂热大学 用于评估肝纤维化进展的非侵入方法
CN108241865A (zh) * 2016-12-26 2018-07-03 哈尔滨工业大学 基于超声图像的多尺度多子图肝纤维化多级量化分期方法
CN109063712A (zh) * 2018-06-22 2018-12-21 哈尔滨工业大学 一种基于超声图像的多模型肝脏弥漫性疾病智能诊断方法及系统

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140105125A1 (en) * 2012-10-16 2014-04-17 Qualcomm Incorporated Criteria for ue-initiated bearer deactivation when maximum number of active bearers has been reached
CN108073876B (zh) * 2016-11-14 2023-09-19 北京三星通信技术研究有限公司 面部解析设备和面部解析方法
CN110729045A (zh) * 2019-10-12 2020-01-24 闽江学院 一种基于上下文感知残差网络的舌图像分割方法
CN110992270A (zh) * 2019-12-19 2020-04-10 西南石油大学 基于注意力的多尺度残差注意网络图像超分辨率重建方法

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4744833B2 (ja) * 2004-09-24 2011-08-10 株式会社東芝 超音波診断装置
UA79297C2 (en) * 2005-02-25 2007-06-11 Inst Pediatrics Obstetrics Method for the diagnosis of the chronic diffuse lesions of the liver
CN102334122A (zh) * 2009-02-26 2012-01-25 昂热大学 肝纤维化或肝硬化的改进诊断
CN107014996A (zh) * 2009-03-19 2017-08-04 昂热大学 用于评估肝纤维化进展的非侵入方法
WO2014011925A2 (en) * 2012-07-11 2014-01-16 University Of Mississippi Medical Center Method for the detection and staging of liver fibrosis from image acquired data
CN103169506A (zh) * 2013-03-19 2013-06-26 安徽皖仪科技股份有限公司 一种自动识别肝癌的超声诊断装置和方法
CN103919576A (zh) * 2014-04-18 2014-07-16 上海市杨浦区中心医院 肝实质超声无向性纹理定量测量仪及肝实质纹理测量方法
CN105956620A (zh) * 2016-04-29 2016-09-21 华南理工大学 一种基于稀疏表示的肝脏超声图像识别方法
CN108241865A (zh) * 2016-12-26 2018-07-03 哈尔滨工业大学 基于超声图像的多尺度多子图肝纤维化多级量化分期方法
CN109063712A (zh) * 2018-06-22 2018-12-21 哈尔滨工业大学 一种基于超声图像的多模型肝脏弥漫性疾病智能诊断方法及系统

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114897779A (zh) * 2022-04-12 2022-08-12 华南理工大学 基于融合注意力的宫颈细胞学图像异常区域定位方法及装置
CN114897779B (zh) * 2022-04-12 2024-04-23 华南理工大学 基于融合注意力的宫颈细胞学图像异常区域定位方法及装置
CN117636064A (zh) * 2023-12-21 2024-03-01 浙江大学 一种基于儿童病理切片的神经母细胞瘤智能分类系统
CN117636064B (zh) * 2023-12-21 2024-05-28 浙江大学 一种基于儿童病理切片的神经母细胞瘤智能分类系统

Also Published As

Publication number Publication date
CN111783792A (zh) 2020-10-16
CN111783792B (zh) 2023-11-28

Similar Documents

Publication Publication Date Title
WO2021243783A1 (zh) 一种提取b超图像显著纹理特征的方法及其应用
WO2022041307A1 (zh) 一种构建半监督图像分割框架的方法及系统
CN108806792B (zh) 深度学习面诊系统
CN103714536B (zh) 基于稀疏表示的多模态磁共振图像的分割方法及装置
CN108257135A (zh) 基于深度学习方法解读医学图像特征的辅助诊断系统
CN107506761A (zh) 基于显著性学习卷积神经网络的脑部图像分割方法及系统
CN106204532B (zh) 基于特征数据挖掘及神经网络的肿瘤的分类方法
CN112270666A (zh) 基于深度卷积神经网络的非小细胞肺癌病理切片识别方法
CN113344864A (zh) 一种基于深度学习的超声甲状腺结节良恶性预测方法
CN111767952B (zh) 一种可解释的肺结节良恶性分类方法
CN104484886B (zh) 一种mr图像的分割方法及装置
CN112784879A (zh) 一种基于小样本域自适应的医学影像分割或分类方法
CN106780453A (zh) 一种基于深度信任网络实现对脑肿瘤分割的方法
CN101551854B (zh) 不平衡医学影像处理系统及其处理方法
CN111430025B (zh) 一种基于医疗影像数据扩增的疾病诊断模型训练方法
CN110910377B (zh) 一种基于神经网络的脑梗死mri图像识别方法
CN113782184A (zh) 一种基于面部关键点与特征预学习的脑卒中辅助评估系统
CN108877923A (zh) 一种基于深度学习的舌苔生成中药处方的方法
CN114332572B (zh) 基于显著图引导分层密集特征融合网络用于提取乳腺病变超声图像多尺度融合特征参数方法
Masood et al. Development of automated diagnostic system for skin cancer: Performance analysis of neural network learning algorithms for classification
CN116386853A (zh) 面向智慧医疗的深度可分离卷积双重聚合联邦学习方法
CN116525075A (zh) 基于少样本学习的甲状腺结节计算机辅助诊断方法及系统
CN115471701A (zh) 基于深度学习和迁移学习的肺腺癌组织学亚型分类方法
CN115170857A (zh) 一种基于联邦迁移学习的胰腺癌图像识别方法
CN108090507A (zh) 一种基于集成方法的医疗影像纹理特征处理方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20938665

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20938665

Country of ref document: EP

Kind code of ref document: A1