WO2022127227A1 - 一种基于多视图半监督的淋巴结的分类方法、系统及设备 - Google Patents

一种基于多视图半监督的淋巴结的分类方法、系统及设备 Download PDF

Info

Publication number
WO2022127227A1
WO2022127227A1 PCT/CN2021/118605 CN2021118605W WO2022127227A1 WO 2022127227 A1 WO2022127227 A1 WO 2022127227A1 CN 2021118605 W CN2021118605 W CN 2021118605W WO 2022127227 A1 WO2022127227 A1 WO 2022127227A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
grained
features
fine
feature
Prior art date
Application number
PCT/CN2021/118605
Other languages
English (en)
French (fr)
Inventor
辛景民
罗怡文
刘思杰
Original Assignee
西安交通大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 西安交通大学 filed Critical 西安交通大学
Publication of WO2022127227A1 publication Critical patent/WO2022127227A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10132—Ultrasound image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20092—Interactive image processing based on input by user
    • G06T2207/20104—Interactive definition of region of interest [ROI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing

Definitions

  • the invention belongs to the field of medical image processing, and in particular relates to a method, system and device for classifying lymph nodes based on multi-view semi-supervised.
  • FNA fine-needle aspiration
  • the metastatic lymph nodes of thyroid cancer have certain ultrasound characteristics, which can be used to analyze the cervical lymph node metastasis in patients with differentiated thyroid cancer before surgery, and provide a reference for lymph node dissection.
  • the cervical lymph nodes have the characteristics of division.
  • the second, third, and fourth regions cover the hypopharynx and the front of the neck, and are the key targets for removal of the neck.
  • lymph node classification methods are mainly divided into two categories, including traditional manual design features and data-driven methods.
  • most of the early methods are artificially designed features, mainly including grayscale features of images, target size, aspect ratio, and medical description features.
  • the features obtained by this type of method are mostly traditional gray-scale-based image features or medical features that require fine calibration.
  • the descriptive information obtained by traditional methods is relatively single, which is not enough to effectively extract features from gray-scale ultrasound images.
  • Fine medical feature calibration requires the calibration of professional physicians, which is usually not easy to obtain; these limitations limit the extraction of effective information from images.
  • the development of deep learning in the field of image analysis has made data-driven methods successful in the field of medical imaging.
  • lymph node ultrasound images usually only focus on the internal features of lymph node ultrasound images, such as whether there is calcification, liquefaction, and echo in the nodule.
  • Homogeneous et al ignoring the environmental information of the lymph nodes.
  • lymph nodes have large morphological differences in different subregions, and their background information is also quite different. Therefore, nodule environmental information also plays a very important role in the classification process. Only extracting features from the internal information of the nodule and ignoring the environmental information will affect the effectiveness of the final feature extraction, which in turn will affect the accuracy of the classification.
  • the purpose of the present invention is to provide a method, system and device for classifying lymph nodes based on multi-view semi-supervised, so as to overcome the deficiencies of the prior art.
  • the present invention adopts the following technical solutions:
  • a multi-view semi-supervised classification method for lymph nodes including the following steps:
  • image preprocessing is performed on the original gray-scale ultrasound image of the lymph node, and image reconstruction is performed on the preprocessed image using an image reconstruction neural network;
  • the ROI of the nodule area in the original gray-scale ultrasound image of the lymph node is trimmed to obtain a fine-grained image, and the fine-grained image is weighted by the vgg16 network with spatial and channel attention mechanisms added to different levels, and then global averaged Pooling, splicing feature outputs of different levels to obtain fine-grained fusion features;
  • down-sampling is performed first, and then up-sampling is performed to obtain the feature expressions of the original image at different scales.
  • the residual module is used to operate, and cross-layer connections are added during the up-sampling process. Plus realize the fusion of features to complete the image reconstruction.
  • the output at different levels of the decoder in the image reconstruction neural network realizes multi-scale feature extraction, and the features of different scales are stitched through two convolution operations and an expansion layer and then a stitching layer is used to obtain multi-scale fusion. coarse-grained image features.
  • the multi-scale feature extraction of the original image is performed by the decoder part of the image reconstruction neural network, and each sampling in the decoder is regarded as a granularity of the image, and the image features under each granularity are extracted.
  • the features of each granularity are subjected to two convolution operations, and the matrix is expanded into a one-dimensional vector, and a stitching layer is used to stitch the features of different granularities to obtain the fused multi-scale fusion coarse-grained image features. .
  • the obtained multi-scale fusion features are subjected to semi-supervised multi-task learning through two linear fully connected layers; the multi-scale fusion features use the fuzzy clustering method to model the image background information, and the background partitions are obtained as semi-supervised multi-task learning.
  • the pseudo-label information in supervised learning, the obtained background pseudo-label and the original target label are subjected to semi-supervised multi-task learning through two linear fully connected layers; the multi-scale fusion features use the fuzzy clustering method to model the image background information, and the background partitions are obtained as semi-supervised multi-task learning.
  • the pseudo-label information in supervised learning, the obtained background pseudo-label and the original target label are obtained multi-scale fusion features.
  • fuzzy clustering method is used to model the partition labels that are not in the source labels, and the fuzzy clustering algorithm is performed by fused multi-scale features:
  • u i,j represents the membership degree of the ith sample belonging to the jth class
  • vj represents the prototype of the cluster center
  • the parameter m is the weighted index to determine the ambiguity of the classification result.
  • the data volume of fine-grained images is increased by means of image enhancement, and the fine-grained images are input into the fine-grained fusion feature network. modeling.
  • image enhancement is first performed on the ROI image, and the original image is randomly resized by fixing the original image ratio.
  • the adjustment range of the image scaling is set to 64 to 256, and the enhanced image I' is used as the input of the fine-grained network.
  • the fine-grained network uses vgg16 as the initial model of the model, and uses SC-block to increase the attention mechanism module at different stages of the model; SC-block contains attention mechanisms in two dimensions of space and channel; for the spatial attention mechanism part, use volume
  • the convolution with the product kernel of 1x1 compresses the channel domain of the image features, and normalizes the compressed features by using the sigmoid function to obtain the probability map of the weighting matrix of the feature map in the spatial dimension.
  • a multi-view semi-supervised lymph node classification system comprising an image preprocessing and reconstruction module, a coarse-grained image feature acquisition module, a fine-grained image feature acquisition module and a classification module;
  • the image preprocessing and reconstruction module is used to perform image preprocessing on the original gray-scale ultrasound images of lymph nodes, and perform image reconstruction on the preprocessed images; the reconstructed images are transmitted to the coarse-grained image feature acquisition module and the fine-grained image feature acquisition.
  • the coarse-grained image feature collection module performs multi-scale feature extraction on the reconstructed image to obtain multi-scale fusion coarse-grained image features;
  • image, the fine-grained image is globally pooled, and the features of different levels of the global pooling are spliced to obtain fine-grained fusion features;
  • the classification module fuses coarse-grained image features and fine-grained fusion features, using two fully connected layers
  • the classification result is obtained by sigmoid activation joint output and output.
  • a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, when the processor executes the computer program, the above-mentioned multi-view semi-supervised lymph node-based algorithm is implemented.
  • the steps of the classification method comprising a processor, and a computer program stored in the memory and executable on the processor, when the processor executes the computer program, the above-mentioned multi-view semi-supervised lymph node-based algorithm is implemented.
  • the present invention has the following beneficial technical effects:
  • the present invention is a method for classifying lymph nodes based on multi-view semi-supervision.
  • image preprocessing is performed on the original gray-scale ultrasound image of the lymph node, and the U-shaped neural network is used for image reconstruction on the preprocessed image.
  • the results are superimposed in parallel to achieve weighted fusion, and the multi-granularity self-expression information of the original image is obtained, and then the multi-granularity information is fused to obtain the multi-scale fusion coarse-grained image features.
  • Pseudo-labels are generated for the missing environmental information in the original labels of the granular view, and coarse-grained feature representation learning is performed in a semi-supervised manner;
  • the vgg16 network with spatial and channel attention mechanisms added to the layers is weighted and then subjected to global average pooling, and the feature outputs of different levels are spliced to obtain fine-grained fusion features. Attempt to fuse the features, so that the fused features can have both the environmental information and the detailed information of the nodule, and obtain a richer and more accurate description, which can then be accurately classified and improve the classification accuracy.
  • the fuzzy clustering method is used to generate pseudo-labels for the missing environmental information in the original labels of the coarse-grained view.
  • the fuzzy clustering method uses the membership matrix instead of one-hot encoding, which reduces the number of clustering compared with the traditional hard clustering method.
  • the impact of class errors on the model; the coarse-grained image features are trained in a multi-task manner, and the pseudo-labels and real labels are jointly supervised for network learning.
  • the pseudo-labels are continuously updated iteratively to ensure that Validity of pseudo-labels.
  • the adopted SC-block attention mechanism module is used at different levels of vgg16, so that the network model can focus on more discriminative places in both spatial and channel dimensions, and then perform multi-scale features. Fusion results in better fine-grained image descriptions.
  • the multi-view information is realized by the fusion of the multi-scale features of the two views.
  • the multi-view information not only includes the partition and environment information in the coarse-grained image, but also contains the fine-grained part of the internal details of the nodule. , and by fusing the two views information, a more comprehensive and effective nodule information is obtained, thereby improving the detection accuracy.
  • a multi-view semi-supervised lymph node classification system using image preprocessing and reconstruction module, coarse-grained image feature acquisition module, fine-grained image feature acquisition module and classification module; through the nodule internal information and combined with environmental information to extract the final feature , which effectively improves the accuracy of classification, has a simple structure, and can quickly achieve classification.
  • FIG. 1 is a general diagram of a multi-view semi-supervised neural network in an embodiment of the present invention.
  • FIG. 2 is a schematic structural diagram of a branch of a coarse-grained view neural network in an embodiment of the present invention.
  • FIG. 3 is a schematic structural diagram of a fine-grained view neural network branch in an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of clustering of a sample space with or without pseudo-label auxiliary training in an embodiment of the present invention.
  • a multi-view based semi-supervised lymph node classification method includes the following steps:
  • S1 Perform image preprocessing on the original gray-scale ultrasound image (coarse-grained image) of the lymph node, and perform image reconstruction on the preprocessed image using an image reconstruction neural network (Hourglass network);
  • the image reconstruction neural network includes an encoder and a decoder.
  • the structure of the encoder and decoder includes a down-sampling layer and an up-sampling layer.
  • down-sampling is performed first to obtain the feature expressions of the original image at different scales, and then up-sampling is performed. Before each sampling, the residual module is used to operate, and cross-layer connections are added during the upsampling process to achieve feature fusion and complete image reconstruction.
  • S2 Output the features at different levels of the decoder in the image reconstruction neural network to achieve multi-scale feature extraction.
  • the extracted multi-scale features are processed by two convolution operations and an expansion layer, and then a splicing layer is used for feature splicing to obtain multi-scale fusion coarse-grained image features, that is, multi-scale fusion features; each convolution operation includes sequential 3x3 convolution and batch normalization performed.
  • the specific method of extracting and merging the multi-granularity information of the lymph node by using the image reconstruction method in the S2 is: performing multi-scale feature extraction of the original image through the decoder part of the image reconstruction neural network, and extracting the multi-scale features of the original image by each sampling in the decoder.
  • the image features under each granularity are extracted.
  • the features of each granularity are subjected to two convolution operations, and the matrix is expanded into a one-dimensional vector, using A concatenation layer concatenates features of different granularities to obtain fused multi-scale fusion features.
  • the multi-scale fusion feature uses the fuzzy clustering method to model the background information of the image, and the partition of the background is obtained as the pseudo-label information in the semi-supervised learning.
  • the obtained background pseudo-label and the original target label (while supervising the network study).
  • the features of the fusion layer have both classification and partition information, and the fusion loss function is used for optimization during the training process.
  • the fine-grained image is obtained by trimming the ROI containing the nodule region in the original gray-scale ultrasound image of the lymph node.
  • image enhancement is performed on the original image to expand the data set.
  • the granular image is input into the fine-grained fusion feature network.
  • the architecture of the fine-grained fusion feature network is based on the vgg16 network.
  • the attention mechanism module SC block and a global pooling module are respectively added.
  • the pooled features of different levels are spliced to obtain fine-grained fusion features, which are then classified by ReLu activation and a fully connected layer activated by sigmoid.
  • fuzzy clustering uses fused multi-scale features for the fuzzy clustering algorithm:
  • u i,j represents the membership degree of the ith sample belonging to the jth class
  • vj represents the prototype of the cluster center
  • the parameter m is the weighted index to determine the ambiguity of the classification result.
  • the solution of this optimization problem usually adopts an interactive strategy, that is, first V is given to minimize U, and then U is given to minimize V, and the original function is transformed into two simple quadratic optimization problems. Therefore, the FCM algorithm can be easily deployed into a neural network and perform fuzzy clustering to generate pseudo-labels.
  • the update formula in the clustering iteration process is shown in formula (2).
  • the fusion loss function at this stage includes the crossover loss for classification and the mean square error loss for clustering, which is defined as Equation (3),
  • the classification loss is the cross entropy loss
  • the clustering loss is the mean square error
  • y i is the real label
  • u' i is the clustering pseudo-label in the current training process
  • ⁇ and ⁇ are the hyperparameters of the fusion loss, respectively
  • the purpose is to Balance the two types of losses.
  • image enhancement is performed on the ROI image, and the original image is randomly resized by fixing the original image ratio. effects of elongation.
  • the adjustment range of image scaling is set from 64 to 256, and the enhanced image I′ is used as the input dataset for the fine-grained network.
  • the fine-grained network uses vgg16 as the initial model of the model.
  • the designed SC-block attention mechanism module is used in different stages of the model.
  • the SC-block attention mechanism module designed here contains attention mechanisms in both spatial and channel dimensions.
  • the convolution with a convolution kernel of 1x1 is used to compress the channel domain of the image features
  • the sigmoid function is used to normalize the compressed features to obtain the probability of the weighting matrix of the feature map in the spatial dimension.
  • the purpose is to focus on important features in the spatial domain.
  • the global pooling layer is used to compress the spatial domain of image features, and then the weight matrix of the channel domain is obtained by using a fully connected layer activated by sigmoid. pay attention. Finally, the feature maps are dot-producted with the weighted matrices of the spatial domain and the channel domain, respectively, and then summed together with the original image to obtain the attention-weighted feature matrix X atten .
  • a global pooling layer and a matrix expansion layer are used to reduce the dimensionality of the features to obtain one dimension, and finally the features of different granularities are fused through a splicing layer.
  • X fine where L is the number of layers selected.
  • the specific method of multi-view fusion is to use a splicing layer to splicing the multi-scale fusion features of the coarse-grained image and the fine-grained image, and use the multi-view features to perform final classification.
  • cross-entropy loss is used for training.
  • the present invention reconstructs the image by using the encoder-decoder model to obtain multi-granularity image self-expression information, and then fuses the information to obtain the multi-scale features of the coarse-grained image, in order to It can model the missing lymph node partition information in the original label for auxiliary classification, and use the fuzzy clustering method to generate pseudo-labels for the missing environmental information in the original label of the coarse-grained view.
  • the fuzzy clustering uses the membership matrix to replace the original one- Hot coding reduces the impact of clustering errors on the model compared to traditional hard clustering methods; coarse-grained image features are trained in a multi-task manner, and pseudo-labels and real labels are jointly supervised for network learning. Among them, During the training process, the pseudo-labels are continuously updated iteratively in the form of interaction to ensure the validity of the pseudo-labels.
  • Fine-grained images were applied for feature extraction of the internal information of lymph node nodules.
  • the SC-block attention mechanism module is used in different layers of vgg16, enabling the network model to focus on more discriminative places. Through the fusion of multi-scale features in the network, richer image descriptions are obtained.
  • the multi-view information is realized by the fusion of the multi-scale features of the two views.
  • the multi-view information not only includes the partition and environment information in the coarse-grained image, but also contains the internal details of the nodule in the fine-grained part.
  • the information of the two views is fused to obtain more comprehensive and effective nodule information, thereby improving the detection accuracy.
  • the distribution of multi-scale features is visualized with and without pseudo-label supervision, respectively.
  • the features of the labels are visualized. Since the extracted sample features are high-dimensional features, and the projection from high-dimensional to low-dimensional will not change the distribution of the samples, the high-dimensional features are reduced by the principal component analysis method (2-dimensional features) to facilitate the visualization of the results. .
  • Figure 4(A) represents the image embedding supervised only by the classification label
  • Figure 4(B) represents the image embedding supervised by both the classification label and the pseudo-label. From the distribution of data points in the image, it can be seen that after adding clustering information for semi-supervised multi-task learning, the boundaries between hidden layer features of different categories (red, yellow, green, blue) are clearer, and in low-dimensional space The sample points of different classes overlap less, and the intra-class distance is smaller. It can be proved that the clustering information can help the network to better learn the spatial information and effectively assist in the classification of samples.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Software Systems (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Quality & Reliability (AREA)
  • Multimedia (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Medical Informatics (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

本发明公开了一种基于多视图半监督的淋巴结的分类方法、系统及设备，对淋巴结的原始灰阶超声图像进行图像预处理，通过对预处理后的图像使用U型神经网络进行图像重建，加权融合得到多尺度融合的粗粒度图像特征，利用半监督的方式进行粗粒度特征表示学习；裁剪原始灰阶超声图像包含结节的ROI区域得到细粒度图像，将细粒度图像通过不同层级中加入了空间和通道注意力机制的vgg16网络进行加权处理后再进行全局平均池化，将不同层级的特征输出进行拼接得到细粒度融合特征，多视图信息通过对粗粒度视图特征和细粒度试图特征进行融合得到，使得融合特征能同时具备结节的环境信息和细节信息，获取到更加丰富准确地描述，能够进行准确分类，提高了分类精确度。

Description

一种基于多视图半监督的淋巴结的分类方法、系统及设备 技术领域
本发明属于医学图像处理领域，具体涉及一种基于多视图半监督的淋巴结的分类方法、系统及设备。
背景技术
临床上，通常以细针吸取活检(FNA)作为淋巴结鉴别的金标准，虽然能提供准确的结果，但侵入性检查甚至手术都可能导致颈部淋巴结病变，并在一定程度上影响患者的身体状况。超声作为一种非侵入性的方法，以其方便、经济的特点，已成为颈部淋巴结术前信息采集最常用的方法。甲状腺癌转移淋巴结具有一定的超声特征，可用于分析分化型甲状腺癌患者术前颈部淋巴结转移的情况，为淋巴结清扫提供参考。颈部淋巴结具有分区特性，其中二区、三区、四区，覆盖了下咽后颈前等部位，是颈廓重点的清除对象。
目前淋巴结分类方法主要分为两类，包括传统人工设计特征和数据驱动的方法。对于淋巴结分类任务，早期的方法多为人工设计特征，主要包括图像的灰度特征，目标的大小、纵横比以及医学描述特征等。该类方法所得到的特征多为传统基于灰度的图像特征或需要精细标定的医学特征，通常传统方法获得的描述性信息较为单一，不足以对灰阶超声图像进行足够有效的特征提取；而精细的医学特征标定需要专业医师的标定，则通常不易获得；这些局限性限制了图像有效信息的提取。深度学习在图像分析领域的发展使得基于数据驱动的方法在医疗影像领域获得成功，然而目前现有的方法通常只关注于淋巴结超声图像的内部特征，如结节内部是否有钙化、液化、回声是否均匀等，忽略了淋巴结的环境信息。然 而，淋巴结在不同分区的情况下具有很大的形态学差异，其背景信息也存在较大的不同。因此，结节环境信息在分类过程中也占有十分重要作用，只通过结节内部信息进行特征提取而忽略环境信息会对最终特征提取的有效性造成影响，进而对分类的准确性造成影响。
发明内容
本发明的目的在于提供一种基于多视图半监督的淋巴结的分类方法、系统及设备，以克服现有技术的不足。
为达到上述目的，本发明采用如下技术方案：
一种基于多视图半监督的淋巴结的分类方法，包括以下步骤：
S1，对淋巴结的原始灰阶超声图像进行图像预处理，对预处理后的图像使用图像重建神经网络进行图像重建；
S2，对重建后的图像进行多尺度特征提取，通过构建环境信息伪标签的方式进行半监督学习，得到多尺度融合的粗粒度图像特征；
S3，将淋巴结的原始灰阶超声图像中结节区域的ROI进行剪裁得到细粒度图像，将细粒度图像通过不同层级中加入了空间和通道注意力机制的vgg16网络进行加权处理后再进行全局平均池化，将不同层级的特征输出进行拼接得到细粒度融合特征；
S4，将粗粒度图像特征和细粒度融合特征进行融合，使用两层全连接层通过sigmoid激活联合输出得到分类结果。
进一步的，进行图像重建时先进行下采样，再进行上采样，得到原始图像不同尺度下的特征表达，每次采样之前均使用残差模块进行操作，在上采样过程中添加跨层连接进行点加实现特征的融合，完成图像的重建。
进一步的，将图像重建神经网络中解码器的不同层级处进行输出实现多尺度特征提取，不同尺度的特征通过两个卷积操作和一个展开层后使用一个拼接层进行特征拼接，得到多尺度融合的粗粒度图像特征。
进一步的，通过图像重建神经网络的解码器部分进行原始图像的多尺度特征提取，将解码器中每次采样看作是图像的一个粒度，对每一个粒度下的图像特征进行提取处理，其中在进行最终的特征融合之前每个粒度的特征均进行两次卷积操作，并进行矩阵展开为一维向量，使用一个拼接层对不同粒度的特征进行拼接得到融合的多尺度融合的粗粒度图像特征。
进一步的，将得到的多尺度融合特征分别通过两个线性全连接层进行半监督的多任务学习；多尺度融合特征使用模糊聚类的方法对图像背景信息进行建模，得到背景的分区作为半监督学习中的伪标签信息，得到的背景伪标签与原始目标标签。
进一步的，使用模糊聚类的方法对源标签中没有的分区标签进行建模，使用融合多尺度特征进行模糊聚类算法：
如(1)式所示：
Figure PCTCN2021118605-appb-000001
Figure PCTCN2021118605-appb-000002
其中u i,j代表第i个样本属于第j类的隶属度，v j代表了聚类中心的原型，参数m是确定分类结果模糊度的加权指数。
进一步的，通过图像增强的方式增加细粒度图像的数据量，将细粒度图像输入细粒度融合特征网络，在细粒度融合特征网络中通过使用空间和通道注意力机 制的方法对淋巴结的内部信息进行建模。
进一步的，首先对ROI图像进行图像增强，采用固定原始图像比例的方式，对原始图像进行随机尺寸调整，图像缩放的调整范围设置为64到256,增强后的图像I′作为细粒度网络的输入数据集：
I′＝scale(I,(64,256))
细粒度网络以vgg16作为模型的初始模型，在模型的不同阶段使用SC-block增加注意力机制模块；SC-block包含空间和通道两个维度的注意力机制；对于空间注意力机制部分，使用卷积核为1x1的卷积对图像特征的通道域进行压缩，通过使用sigmoid函数作用于压缩的特征进行归一化，得到特征图在空间维度的加权矩阵的概率图。
一种基于多视图半监督的淋巴结的分类系统，包括图像预处理重建模块、粗粒度图像特征采集模块、细粒度图像特征采集模块和分类模块；
图像预处理重建模块用于对淋巴结的原始灰阶超声图像进行图像预处理，并对预处理后的图像进行图像重建；将重建后的图像传输至粗粒度图像特征采集模块和细粒度图像特征采集模块，粗粒度图像特征采集模块对重建后的图像进行多尺度特征提取得到多尺度融合的粗粒度图像特征；细粒度图像特征采集模块对重建后的图像中结节区域的ROI进行剪裁得到细粒度图像，将细粒度图像进行全局池化，并将全局池化的不同层级的特征进行拼接得到细粒度融合特征；分类模块将粗粒度图像特征和细粒度融合特征进行融合，使用两层全连接层通过sigmoid激活联合输出得到分类结果并输出。
一种计算机设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现上述基于多 视图半监督的淋巴结的分类方法的步骤。
与现有技术相比，本发明具有以下有益的技术效果：
本发明一种基于多视图半监督的淋巴结分类方法，首先对淋巴结的原始灰阶超声图像进行图像预处理，通过对预处理后的图像使用U型神经网络进行图像重建，通过将每次上采样的结果并行叠加实现加权融合，获取原始图像的多粒度自表达信息，进而对多粒度信息进行融合得到多尺度融合的粗粒度图像特征，为了能够建模原始标签中缺失的淋巴结分区信息，对粗粒度视图原始标签中缺失的环境信息进行伪标签生成，利用半监督的方式进行粗粒度特征表示学习；通过裁剪原始灰阶超声图像包含结节的ROI区域得到细粒度图像，将细粒度图像通过不同层级中加入了空间和通道注意力机制的vgg16网络进行加权处理后再进行全局平均池化，将不同层级的特征输出进行拼接得到细粒度融合特征，多视图信息通过对粗粒度视图特征和细粒度试图特征进行融合得到，使得融合特征能同时具备结节的环境信息和细节信息，获取到更加丰富准确地描述，进而能够进行准确分类，提高了分类精确度。
进一步的，利用模糊聚类的方法对粗粒度视图原始标签中缺失的环境信息进行伪标签生成，模糊聚类的方法使用隶属度矩阵代替one-hot编码，相比传统硬聚类方法减少了聚类错误对模型的影响；粗粒度图像特征在进行训练时，以多任务的方式进行，伪标签和真实标签共同进行网络学习的监督，其中，在训练过程中伪标签不断进行迭代更新，以保证伪标签的有效性。
进一步的，采用的SC-block注意力机制模块被用于vgg16的不同层级，使得网络模型能够同时在空间和通道两个维度将注意力集中在更具有判别性的地方，再通过进行多尺度特征融合得到更好的细粒度图像描述。
进一步的，多视图的信息通过两个视图的多尺度特征的融合进行实现，多视图信息不仅包含了粗粒度图像中所具有的分区和环境信息，同时也包含细粒度部分的结节内部细节信息，通过对两个视图信息进行融合，获得更加全面有效的结节信息，进而提高了检测的准确性。
一种基于多视图半监督的淋巴结的分类系统，采用图像预处理重建模块、粗粒度图像特征采集模块、细粒度图像特征采集模块和分类模块；通过结节内部信息并结合环境信息对最终特征提取，有效提高了分类的准确性，结构简单，能够快速实现分类。
附图说明
为了更清楚的说明本发明实施例的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，应当理解，以下附图仅示出了本发明的某些实施例，因此不应被看作是对范围的限定，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他相关的附图。
图1为本发明实施例中的多视图半监督神经网络总图。
图2为本发明实施例中粗粒度视图神经网络分支的结构示意图。
图3为本发明实施例中细粒度视图神经网络分支的结构示意图。
图4为本发明实施例中样本空间在有无伪标签辅助训练情况下的聚类示意图。
具体实施方式
下面结合附图对本发明做进一步详细描述：
下面结合附图对本发明做进一步详细描述：
如图1所示，一种基于多视图半监督的淋巴结分类方法，包括以下步骤：
S1：对淋巴结的原始灰阶超声图像(粗粒度图像)进行图像预处理，对预处 理后的图像使用图像重建神经网络(Hourglass网络)进行图像重建；
图像重建神经网络包括编码器和解码器，编码器和解码器结构包括下采样层和上采样层，进行图像重建时先进行下采样，得到原始图像不同尺度下的特征表达，再进行上采样，每次采样之前均使用残差模块进行操作，在上采样过程中添加跨层连接进行点加实现特征的融合，完成图像的重建。
S2：将图像重建神经网络中解码器的不同层级处的特征进行输出，实现多尺度特征提取。提取到的多尺度特征通过两个卷积操作和一个展开层处理后使用一个拼接层进行特征拼接，得到多尺度融合的粗粒度图像特征，即多尺度融合特征；每次卷积操作均包括依次进行的3x3卷积和批次归一化。
所述S2中通过使用图像重建的方法对淋巴结的多粒度信息进行提取融合的具体方法为：通过图像重建神经网络的解码器部分进行原始图像的多尺度特征提取，将解码器中每次采样看作是图像的一个粒度，对每一个粒度下的图像特征进行提取处理，其中在进行最终的特征融合之前每个粒度的特征均进行两次卷积操作，并进行矩阵展开为一维向量，使用一个拼接层对不同粒度的特征进行拼接得到融合的多尺度融合特征。
S3:得到的多尺度融合的粗粒度图像特征分别通过两个线性全连接层进行半监督的多任务学习(包括聚类和分类)；
具体的，多尺度融合特征使用模糊聚类的方法对图像背景信息进行建模，得到背景的分区作为半监督学习中的伪标签信息，得到的背景伪标签与原始目标标签(同时对网络进行监督学习)。
通过这种半监督学习训练的方式，使得融合层的特征(粗粒度特征)同时具有分类和分区的信息，训练过程中使用融合损失函数进行优化。
S4:如图3所示，细粒度图像由淋巴结的原始灰阶超声图像中包含结节区域的ROI进行剪裁得到，为了增加图像的多样性，对原始图像进行图像增强来扩大数据集，将细粒度图像输入细粒度融合特征网络，细粒度融合特征网络的架构基于vgg16网络，其中，在细粒度融合特征网络的不同尺度部分分别加入注意力机制模块SC block和一个全局池化模块，将通过全局池化的不同层级的特征进行拼接得到细粒度融合特征，再通过ReLu激活和一层由sigmoid激活的全连接层进行分类。
S5:将上述所得粗粒度图像特征和细粒度融合特征进行融合，最终使用两层全连接层,通过sigmoid激活联合输出得到最终的分类结果。
如图2所示，所述S3中通过使用模糊聚类的方法对淋巴结的环境信息进行建模的具体方法为：
使用模糊聚类的方法对源标签中没有的分区标签进行建模，我们使用融合多尺度特征进行模糊聚类算法：
如(1)式所示：
Figure PCTCN2021118605-appb-000003
Figure PCTCN2021118605-appb-000004
其中u i,j代表第i个样本属于第j类的隶属度，v j代表了聚类中心的原型，参数m是确定分类结果模糊度的加权指数。
该优化问题的求解通常采用交互策略，即先给定V使U最小化，再给定U使V最小化，将原函数转化为两个简单的二次优化问题。因此，FCM算法可以很容易地部署到神经网络中，并进行模糊聚类来生成伪标签。聚类迭代过程中的 更新公式如公式(2)所示。
Figure PCTCN2021118605-appb-000005
Figure PCTCN2021118605-appb-000006
一旦确定了一组群集中心{V＝v1，v2，...vc}，将通过公式(2B)重新计算隶属度U，直到满足条件||U(k+1)-U(K)||≤δ。我们采用半监督的多任务学习方法，使用伪标签和真实标签对网络进行监督，使得网络能够同时获得分类和聚类信息。其中，由于在此处使用模糊聚类的方法对伪标签进行建模，即我们使用联合概率对图像背景所属类别进行描述，其相比于传统硬聚类算法的优势在于，使用概率分布代替one-hot编码,能够减少由于伪标签的不确定性在训练过程中对网络带来的影响，进而保证网络的训练不会由于为标签错误而出现偏差。该阶段的融合损失函数包括分类的交叉损失和聚类的均方误差损失，其定义为公式(3)，
Figure PCTCN2021118605-appb-000007
其中，分类损失为交叉熵损失，聚类损失为均方误差，y i为真实标签，u′ i为当前训练过程中的聚类伪标签，α和β分别为融合损失的超参数，目的在于平衡两类损失。
所述S4中通过使用空间和通道注意力机制的方法对淋巴结的内部信息进行建模的具体方法为：
首先对ROI图像进行图像增强，采用固定原始图像比例的方式，对原始图像 进行随机尺寸调整，其目的在于模拟现实中对视图的放大和缩小，同时保留了结节的形态学信息，避免由于拉伸形变带来的影响。图像缩放的调整范围设置为64到256,增强后的图像I′作为细粒度网络的输入数据集。
I′＝scale(I,(64,256))
细粒度网络以vgg16作为模型的初始模型，为了能够对图像的不同尺度进行特征提取，在模型的不同阶段使用设计的SC-block注意力机制模块。此处所设计的SC-block注意力机制模块包含空间和通道两个维度的注意力机制。对于空间注意力机制部分，使用卷积核为1x1的卷积对图像特征的通道域进行压缩，通过使用sigmoid函数作用于压缩的特征进行归一化，得到特征图在空间维度的加权矩阵的概率图，其目的在于对空间域上重要的特征进行关注。
对于通道注意力机制模块，使用全局池化层将图像特征的空间域进行压缩，再通过使用一个通过sigmoid进行激活的全连接层得到通道域的权重矩阵，其目的在于对通道域上重要的特征进行关注。最后，分别使用空间域和通道域的加权矩阵对特征图进行点积后与原始图像一同进行求和，得到注意力加权特征矩阵X atten。
X atten＝x ch+x sp+x ori
来自网络不同层级的特征通过SC-block进行加权处理后，分别使用一个全局池化层和一个矩阵展开层来对特征进行降维处理得到一维，最后不同粒度的特征通过一个拼接层进行融合得到X fine，其中L为所选取的层级数。
Figure PCTCN2021118605-appb-000008
得到融合的多尺度细粒度特征X fine后，应用两个全连接层进行分类训练，该 过程中的训练损失为交叉熵损失。
所述S5中，多视图融合的具体方法为，对粗粒度图像和细粒度图像的多尺度融合特征使用一个拼接层进行拼接，使用多视图的特征进行最终的分类，这里使用交叉熵损失进行训练：
Figure PCTCN2021118605-appb-000009
其中，yi表示结节的真实类别标签，y^i表示预测的概率。
本发明通过结合淋巴结粗粒度视图和细粒度视图，通过利用编码器解码器模型对图像进行重建，得到多粒度的图像自表达信息，进而对这些信息进行融合获得粗粒度图像的多尺度特征，为了能够建模原始标签中缺失的淋巴结分区信息来进行辅助分类，利用模糊聚类的方法对粗粒度视图原始标签中缺失的环境信息进行伪标签生成，模糊聚类使用隶属度矩阵代替原始的one-hot编码，相比传统硬聚类方法减少了聚类错误对模型的影响；粗粒度图像特征在进行训练时，以多任务的方式进行，伪标签和真实标签共同进行网络学习的监督，其中，在训练过程中伪标签以交互的形式不断进行迭代更新，确保伪标签的有效性。
细粒度图像(包含结节的ROI)被应用于淋巴结结节内部信息的特征提取。SC-block注意力机制模块被用于vgg16的不同层，使得网络模型能够将注意力集中在更具有判别性的地方。通过对网络中的多尺度特征融合，得到更丰富的图像描述。
多视图的信息通过两个视图的多尺度特征的融合进行实现，多视图信息不仅包含了粗粒度图像中所具有的分区和环境信息，同时也包含细粒度部分的结节内部细节信息，通过对两个视图信息进行融合，获得更加全面有效的结节信息，进而提高了检测的准确性。
为了评估该网络的性能，我们选择了三个评价指标：准确率(P＝TP+TN/(TP+FP+TN+FN))、召回率(R＝TP/(TP+FN))和F1得分(F1＝2.P·R/(P+R))，其中P和R分别代表准确率和召回率，TP、FP、TN和FN分别代表真阳性、假阳性、真阴性和假阴性的数量。由于我们提出的方法是基于结节的两种视图，因此我们分别对这两种视图在其他网络上的性能进行了比较。我们比较了不同网络的分类性如Exception，ResNet50，Mobilnet V2，并计算了上述指标。所有比较的模型都是由ImageNet预先训练的。实验结果如表I所示，从表I可以看出，我们的模型在准确率和F1得分方面取得了最好的结果，但在召回率方面略逊于其中一个模型。然而，对于二分类问题，若模型结果呈现高召回率和低准确率的结果，则表明该模型倾向于将所有样本归为一类，不能做出有效的决策。
表I
Figure PCTCN2021118605-appb-000010
为了充分分析该方法各部分的有效性，我们将对比实验分为以下五个部分进行消融实验：1)原始ROI图像分类，2)baseline全图像分类，3)加入SC-block的细粒度网络分类，4)加入半监督聚类分析的粗粒度网络分类，5)多视图半监督学习框架的结果。如表II所示，我们提出的模型在准确率(0.803)和F1得分(0.820)方面达到了最高水平，同时召回率也可以保持在较高的水平(0.854)。通过消融实验的对比，我们可以证明FCM聚簇部分和SC-块在分类任务上的积极影响，最终的分类结果表明，半监督多视图框架能够进行有效的分类，在精度上达到最高值。
表II
Figure PCTCN2021118605-appb-000011
为了进一步探讨软聚类半监督任务对粗粒度网络嵌入空间的影响的可解释性，分别在有无伪标签监督的情况下对多尺度特征的分布进行可视化。具体步骤在于，粗粒度网络的拼接层获取的特征进行FCM聚类，得到数据集中每个样本的隶属度矩阵，使用u＝max(u′ i)得到每个样本所属的类别，对已得到伪标签的特征进行可视化。由于提取到的样本特征为高维特征，且高维到低维的投影不会改变样本的分布，因此这里通过主成分分析法对高维特征进行降维(2维特征)，以便于可视化结果。如图4(A)表示仅由分类标签监督的图像嵌入，图4(B)表示分类标签和伪标签共同监督的图像嵌入。由图像中数据点的分布可以看出，加入聚类信息进行半监督多任务学习后，不同类别(红、黄、绿、蓝)的隐含层特征之间的边界更加清晰，低维空间中的不同类别的样本点重叠更少，且类内距离更小。可以证明，聚类信息可以帮助网络更好地学习空间信息，有效对样本进行辅助分类。
以上内容仅为说明本发明的技术思想，不能以此限定本发明的保护范围，凡是按照本发明提出的技术思想，在技术方案基础上所做的任何改动，均落入本发明权利要求书的保护范围之内。

Claims (10)

  1. 一种基于多视图半监督的淋巴结的分类方法，其特征在于，包括以下步骤：
    S1，对淋巴结的原始灰阶超声图像进行图像预处理，对预处理后的图像使用图像重建神经网络进行图像重建；
    S2，对重建后的图像进行多尺度特征提取，通过构建环境信息伪标签的方式进行半监督学习，得到多尺度融合的粗粒度图像特征；
    S3，将淋巴结的原始灰阶超声图像中结节区域的ROI进行剪裁得到细粒度图像，将细粒度图像通过不同层级中加入了空间和通道注意力机制的vgg16网络进行加权处理后再进行全局平均池化，将不同层级的特征输出进行拼接得到细粒度融合特征；
    S4，将粗粒度图像特征和细粒度融合特征进行融合，使用两层全连接层通过sigmoid激活联合输出得到分类结果
  2. 根据权利要求1所述的一种基于多视图半监督的淋巴结的分类方法，其特征在于，进行图像重建时先进行下采样，再进行上采样，得到原始图像不同尺度下的特征表达，每次采样之前均使用残差模块进行操作，在上采样过程中添加跨层连接进行点加实现特征的融合，完成图像的重建。
  3. 根据权利要求1所述的一种基于多视图半监督的淋巴结的分类方法，其特征在于，将图像重建神经网络中解码器的不同层级处进行输出实现多尺度特征提取，不同尺度的特征通过两个卷积操作和一个展开层后使用一个拼接层进行特征拼接，得到多尺度融合的粗粒度图像特征。
  4. 根据权利要求3所述的一种基于多视图半监督的淋巴结的分类方法，其特征在于，通过图像重建神经网络的解码器部分进行原始图像的多尺度特征提取， 将解码器中每次采样看作是图像的一个粒度，对每一个粒度下的图像特征进行提取处理，其中在进行最终的特征融合之前每个粒度的特征均进行两次卷积操作，并进行矩阵展开为一维向量，使用一个拼接层对不同粒度的特征进行拼接得到融合的多尺度融合的粗粒度图像特征。
  5. 根据权利要求3所述的一种基于多视图半监督的淋巴结的分类方法，其特征在于，将得到的多尺度融合特征分别通过两个线性全连接层进行半监督的多任务学习；多尺度融合特征使用模糊聚类的方法对图像背景信息进行建模，得到背景的分区作为半监督学习中的伪标签信息，得到的背景伪标签与原始目标标签。
  6. 根据权利要求5所述的一种基于多视图半监督的淋巴结的分类方法，其特征在于，具体的，使用模糊聚类的方法对源标签中没有的分区标签进行建模，使用融合多尺度特征进行模糊聚类算法：
    如(1)式所示：
    Figure PCTCN2021118605-appb-100001
    Figure PCTCN2021118605-appb-100002
    其中u i,j代表第i个样本属于第j类的隶属度，v j代表了聚类中心的原型，参数m是确定分类结果模糊度的加权指数。
  7. 根据权利要求1所述的一种基于多视图半监督的淋巴结的分类方法，其特征在于，具体的，通过图像增强的方式增加细粒度图像的数据量，将细粒度图像输入细粒度融合特征网络，在细粒度融合特征网络中通过使用空间和通道注意力机制的方法对淋巴结的内部信息进行建模。
  8. 根据权利要求7所述的一种基于多视图半监督的淋巴结的分类方法，其 特征在于，首先对ROI图像进行图像增强，采用固定原始图像比例的方式，对原始图像进行随机尺寸调整，图像缩放的调整范围设置为64到256,增强后的图像I′作为细粒度网络的输入数据集：
    I′＝scale(I,(64,256))
    细粒度网络以vgg16作为模型的初始模型，在模型的不同阶段使用SC-block增加注意力机制模块；SC-block包含空间和通道两个维度的注意力机制；对于空间注意力机制部分，使用卷积核为1x1的卷积对图像特征的通道域进行压缩，通过使用sigmoid函数作用于压缩的特征进行归一化，得到特征图在空间维度的加权矩阵的概率图。
  9. 一种基于权利要求1所述的一种基于多视图半监督的淋巴结的分类方法的多视图半监督的淋巴结的分类系统，其特征在于，包括图像预处理重建模块、粗粒度图像特征采集模块、细粒度图像特征采集模块和分类模块；
    图像预处理重建模块用于对淋巴结的原始灰阶超声图像进行图像预处理，并对预处理后的图像进行图像重建；将重建后的图像传输至粗粒度图像特征采集模块和细粒度图像特征采集模块，粗粒度图像特征采集模块对重建后的图像进行多尺度特征提取得到多尺度融合的粗粒度图像特征；细粒度图像特征采集模块对重建后的图像中结节区域的ROI进行剪裁得到细粒度图像，将细粒度图像进行全局池化，并将全局池化的不同层级的特征进行拼接得到细粒度融合特征；分类模块将粗粒度图像特征和细粒度融合特征进行融合，使用两层全连接层通过sigmoid激活联合输出得到分类结果并输出。
  10. 一种计算机设备，其特征在于，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序 时实现上述基于多视图半监督的淋巴结的分类方法的步骤。
PCT/CN2021/118605 2020-12-15 2021-09-15 一种基于多视图半监督的淋巴结的分类方法、系统及设备 WO2022127227A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011482944.9 2020-12-15
CN202011482944.9A CN112529878B (zh) 2020-12-15 2020-12-15 一种基于多视图半监督的淋巴结的分类方法、系统及设备

Publications (1)

Publication Number Publication Date
WO2022127227A1 true WO2022127227A1 (zh) 2022-06-23

Family

ID=75000427

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/118605 WO2022127227A1 (zh) 2020-12-15 2021-09-15 一种基于多视图半监督的淋巴结的分类方法、系统及设备

Country Status (2)

Country Link
CN (1) CN112529878B (zh)
WO (1) WO2022127227A1 (zh)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115100476A (zh) * 2022-07-04 2022-09-23 福州大学 一种基于结构建模局部提议网络的细粒度分类方法
CN115272776A (zh) * 2022-09-26 2022-11-01 山东锋士信息技术有限公司 基于双路卷积与双注意的高光谱图像分类方法及存储介质
CN115393316A (zh) * 2022-08-24 2022-11-25 维都利阀门有限公司 具有冲蚀状态监测系统的闪蒸阀及其监测方法
CN116129129A (zh) * 2022-10-09 2023-05-16 南京恩博科技有限公司 一种人物交互检测模型及检测方法
CN116452896A (zh) * 2023-06-16 2023-07-18 中国科学技术大学 用于提升细粒度图像分类性能的方法、系统、设备及介质
CN116452865A (zh) * 2023-04-03 2023-07-18 南通大学 基于模糊逻辑的跳跃式注意力肺部病理图像分类方法
CN116486183A (zh) * 2023-06-21 2023-07-25 耕宇牧星(北京)空间科技有限公司 基于多种注意力权重融合特征的sar图像建筑区分类方法
CN116543216A (zh) * 2023-05-10 2023-08-04 北京建筑大学 细粒度图像分类优化方法及系统
CN116563615A (zh) * 2023-04-21 2023-08-08 南京讯思雅信息科技有限公司 基于改进多尺度注意力机制的不良图片分类方法
CN116958659A (zh) * 2023-07-04 2023-10-27 阿里巴巴达摩院(杭州)科技有限公司 图像分类方法、训练图像分类模型的方法及装置
CN117021435A (zh) * 2023-05-12 2023-11-10 浙江闽立电动工具有限公司 修边机的修边控制系统及其方法
CN117115573A (zh) * 2023-10-25 2023-11-24 华侨大学 一种有毒生物图像分类识别方法、装置、设备及存储介质

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112529878B (zh) * 2020-12-15 2024-04-02 西安交通大学 一种基于多视图半监督的淋巴结的分类方法、系统及设备
CN113642611B (zh) * 2021-07-16 2024-04-12 重庆邮电大学 一种基于多粒度的胎儿心脏超声图像识别方法
CN113537390B (zh) * 2021-08-06 2023-10-10 国网新源水电有限公司富春江水力发电厂 一种基于图卷积网络的动态点云位置预测方法
CN114118199A (zh) * 2021-09-01 2022-03-01 济宁安泰矿山设备制造有限公司 一种用于智能泵腔体内窥镜故障诊断的图像分类方法及系统
CN114067107B (zh) * 2022-01-13 2022-04-29 中国海洋大学 基于多粒度注意力的多尺度细粒度图像识别方法及系统
CN114550307B (zh) * 2022-04-22 2022-09-09 中国科学技术大学 动作定位模型的训练方法、装置及动作定位方法
CN114936615B (zh) * 2022-07-25 2022-10-14 南京大数据集团有限公司 一种基于表征一致性校对的小样本日志信息异常检测方法
CN116564534A (zh) * 2023-04-03 2023-08-08 北京林业大学 中医临床数据的多视图聚类方法、装置及电子设备
CN116778233B (zh) * 2023-06-07 2024-02-06 中国人民解放军国防科技大学 一种基于图神经网络的不完全深度多视图半监督分类方法
CN116485791B (zh) * 2023-06-16 2023-09-29 华侨大学 基于吸收度的双视图乳腺肿瘤病变区自动检测方法及系统

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180228424A1 (en) * 2015-04-02 2018-08-16 Eulji University Industry Academy Cooperation Foundation Triple-fusion imaging device for sentinel lymphadenectomy during laparoscopic surgery
CN109685115A (zh) * 2018-11-30 2019-04-26 西北大学 一种双线性特征融合的细粒度概念模型及学习方法
CN110533024A (zh) * 2019-07-10 2019-12-03 杭州电子科技大学 基于多尺度roi特征的双二次池化细粒度图像分类方法
CN110598029A (zh) * 2019-09-06 2019-12-20 西安电子科技大学 基于注意力转移机制的细粒度图像分类方法
CN110598654A (zh) * 2019-09-18 2019-12-20 合肥工业大学 多粒度交叉模态特征融合行人再识别方法和再识别系统
CN112529878A (zh) * 2020-12-15 2021-03-19 西安交通大学 一种基于多视图半监督的淋巴结的分类方法、系统及设备

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108399406B (zh) * 2018-01-15 2022-02-01 中山大学 基于深度学习的弱监督显著性物体检测的方法及系统
CN110569870A (zh) * 2019-07-25 2019-12-13 中国人民解放军陆军工程大学 基于多粒度标签融合的深度声学场景分类方法及系统
CN111738355B (zh) * 2020-07-22 2020-12-01 中国人民解放军国防科技大学 注意力融合互信息的图像分类方法、装置及存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180228424A1 (en) * 2015-04-02 2018-08-16 Eulji University Industry Academy Cooperation Foundation Triple-fusion imaging device for sentinel lymphadenectomy during laparoscopic surgery
CN109685115A (zh) * 2018-11-30 2019-04-26 西北大学 一种双线性特征融合的细粒度概念模型及学习方法
CN110533024A (zh) * 2019-07-10 2019-12-03 杭州电子科技大学 基于多尺度roi特征的双二次池化细粒度图像分类方法
CN110598029A (zh) * 2019-09-06 2019-12-20 西安电子科技大学 基于注意力转移机制的细粒度图像分类方法
CN110598654A (zh) * 2019-09-18 2019-12-20 合肥工业大学 多粒度交叉模态特征融合行人再识别方法和再识别系统
CN112529878A (zh) * 2020-12-15 2021-03-19 西安交通大学 一种基于多视图半监督的淋巴结的分类方法、系统及设备

Cited By (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115100476A (zh) * 2022-07-04 2022-09-23 福州大学 一种基于结构建模局部提议网络的细粒度分类方法
CN115393316A (zh) * 2022-08-24 2022-11-25 维都利阀门有限公司 具有冲蚀状态监测系统的闪蒸阀及其监测方法
CN115393316B (zh) * 2022-08-24 2023-06-09 维都利阀门有限公司 具有冲蚀状态监测系统的闪蒸阀及其监测方法
CN115272776A (zh) * 2022-09-26 2022-11-01 山东锋士信息技术有限公司 基于双路卷积与双注意的高光谱图像分类方法及存储介质
CN115272776B (zh) * 2022-09-26 2023-01-20 山东锋士信息技术有限公司 基于双路卷积与双注意的高光谱图像分类方法及存储介质
CN116129129B (zh) * 2022-10-09 2023-11-03 南京恩博科技有限公司 一种人物交互检测模型及检测方法
CN116129129A (zh) * 2022-10-09 2023-05-16 南京恩博科技有限公司 一种人物交互检测模型及检测方法
CN116452865B (zh) * 2023-04-03 2023-11-07 南通大学 基于模糊逻辑的跳跃式注意力肺部病理图像分类方法
CN116452865A (zh) * 2023-04-03 2023-07-18 南通大学 基于模糊逻辑的跳跃式注意力肺部病理图像分类方法
CN116563615A (zh) * 2023-04-21 2023-08-08 南京讯思雅信息科技有限公司 基于改进多尺度注意力机制的不良图片分类方法
CN116563615B (zh) * 2023-04-21 2023-11-07 南京讯思雅信息科技有限公司 基于改进多尺度注意力机制的不良图片分类方法
CN116543216A (zh) * 2023-05-10 2023-08-04 北京建筑大学 细粒度图像分类优化方法及系统
CN117021435A (zh) * 2023-05-12 2023-11-10 浙江闽立电动工具有限公司 修边机的修边控制系统及其方法
CN117021435B (zh) * 2023-05-12 2024-03-26 浙江闽立电动工具有限公司 修边机的修边控制系统及其方法
CN116452896B (zh) * 2023-06-16 2023-10-20 中国科学技术大学 用于提升细粒度图像分类性能的方法、系统、设备及介质
CN116452896A (zh) * 2023-06-16 2023-07-18 中国科学技术大学 用于提升细粒度图像分类性能的方法、系统、设备及介质
CN116486183B (zh) * 2023-06-21 2023-09-26 耕宇牧星(北京)空间科技有限公司 基于多种注意力权重融合特征的sar图像建筑区分类方法
CN116486183A (zh) * 2023-06-21 2023-07-25 耕宇牧星(北京)空间科技有限公司 基于多种注意力权重融合特征的sar图像建筑区分类方法
CN116958659A (zh) * 2023-07-04 2023-10-27 阿里巴巴达摩院(杭州)科技有限公司 图像分类方法、训练图像分类模型的方法及装置
CN116958659B (zh) * 2023-07-04 2024-04-02 阿里巴巴达摩院(杭州)科技有限公司 图像分类方法、训练图像分类模型的方法及装置
CN117115573A (zh) * 2023-10-25 2023-11-24 华侨大学 一种有毒生物图像分类识别方法、装置、设备及存储介质
CN117115573B (zh) * 2023-10-25 2024-01-05 华侨大学 一种有毒生物图像分类识别方法、装置、设备及存储介质

Also Published As

Publication number Publication date
CN112529878A (zh) 2021-03-19
CN112529878B (zh) 2024-04-02

Similar Documents

Publication Publication Date Title
WO2022127227A1 (zh) 一种基于多视图半监督的淋巴结的分类方法、系统及设备
Murtaza et al. Deep learning-based breast cancer classification through medical imaging modalities: state of the art and research challenges
Hu et al. Unsupervised learning for cell-level visual representation in histopathology images with generative adversarial networks
Li et al. Deep learning attention mechanism in medical image analysis: Basics and beyonds
CN109064455B (zh) 一种基于bi-rads的乳腺超声图像多尺度融合的分类方法
CN110189308A (zh) 一种基于bm3d和稠密卷积网络融合的肿瘤检测方法和装置
CN109614991A (zh) 一种基于Attention的多尺度扩张性心肌的分割分类方法
Wang et al. Ensemble of 3D densely connected convolutional network for diagnosis of mild cognitive impairment and Alzheimer’s disease
Yao et al. Pneumonia detection using an improved algorithm based on faster r-cnn
Fan et al. Evolutionary neural architecture search for retinal vessel segmentation
Li et al. Automatic recognition and classification system of thyroid nodules in CT images based on CNN
Hu et al. A state-of-the-art survey of artificial neural networks for whole-slide image analysis: from popular convolutional neural networks to potential visual transformers
Tsivgoulis et al. An improved SqueezeNet model for the diagnosis of lung cancer in CT scans
CN112381846A (zh) 一种基于非对称网络的超声甲状腺结节分割方法
Ren et al. An improved U-net based retinal vessel image segmentation method
Lu Computer-aided diagnosis research of a lung tumor based on a deep convolutional neural network and global features
Roffilli Advanced machine learning techniques for digital mammography.
Zou et al. Deep learning and its application in diabetic retinopathy screening
JP2024508852A (ja) 医療画像における病変分析方法
Ye et al. Sparse representation optimization of image Gaussian mixture features based on a convolutional neural network
Wu et al. Mscan: Multi-scale channel attention for fundus retinal vessel segmentation
Azhagiri et al. EAN: enhanced AlexNet deep learning model to detect brain tumor using magnetic resonance images
Xiao et al. Interpretable fine-grained BI-RADS classification of breast tumors
Kao A Deep Learning Architecture For Histology Image Classification
Wang et al. Adaptive binary coding for scene classification based on convolutional networks

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21905159

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21905159

Country of ref document: EP

Kind code of ref document: A1