CN114972859A - Pixel classification method, model training method, device, equipment and medium - Google Patents

Pixel classification method, model training method, device, equipment and medium Download PDF

Info

Publication number
CN114972859A
CN114972859A CN202210545928.2A CN202210545928A CN114972859A CN 114972859 A CN114972859 A CN 114972859A CN 202210545928 A CN202210545928 A CN 202210545928A CN 114972859 A CN114972859 A CN 114972859A
Authority
CN
China
Prior art keywords
pulmonary artery
pixel
pixels
image sequence
skeleton
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210545928.2A
Other languages
Chinese (zh)
Other versions
CN114972859B (en
Inventor
郝增号
刘恩佑
张欢
陈宽
王少康
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Infervision Medical Technology Co Ltd
Original Assignee
Infervision Medical Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Infervision Medical Technology Co Ltd filed Critical Infervision Medical Technology Co Ltd
Priority to CN202210545928.2A priority Critical patent/CN114972859B/en
Publication of CN114972859A publication Critical patent/CN114972859A/en
Application granted granted Critical
Publication of CN114972859B publication Critical patent/CN114972859B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Molecular Biology (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)

Abstract

The embodiment of the specification provides a pixel classification method, a model training method, a device, equipment and a medium. The method comprises the following steps: acquiring a pixel set representing a pulmonary artery skeleton in a pulmonary artery image sequence and an adjacency matrix representing an adjacency relation between pixels in the pixel set of the pulmonary artery skeleton; extracting primary classification features of the pixels in the pixel set; wherein the primary classification feature is used to characterize the pels in the set of pels; and determining the pulmonary artery category to which the picture elements belong in the picture element set of the pulmonary artery skeleton by using the primary classification characteristic and the adjacency matrix. The primary classification characteristic of the pulmonary artery skeleton pixel is extracted, and the characteristic of the pixel in the pixel set is extracted by fusing an image attention network which is used for constructing an adjacent matrix of the pixel in the pixel set expressing the pulmonary artery skeleton and is used as a pixel classification model, so that the classification accuracy of the pixel expressing the pulmonary artery in a pulmonary artery image sequence is improved.

Description

像元的分类方法、模型训练方法、装置、设备和介质Classification method of pixel, model training method, device, equipment and medium

技术领域technical field

本说明书实施方式涉及图像处理领域,具体像元的分类方法、模型训练方法、装置、设备和介质。The embodiments of this specification relate to the field of image processing, a specific pixel classification method, a model training method, an apparatus, a device, and a medium.

背景技术Background technique

随着医学影像检查对于辅助医学诊疗的作用逐渐提升,肺部医学影像检查有助于医生了解患者的肺部状况。对肺部医学影像中表示肺动脉的像元进行分类有助于医生在为患者进行手术时了解患者肺动脉的解剖结构。现有的肺动脉分类方法主要是将基于肺部医学影像提取的肺动脉图像直接输入神经网络进行分类,容易出现肺动脉分类错误的情况。With the increasing role of medical imaging examinations in assisting medical diagnosis and treatment, pulmonary medical imaging examinations can help doctors understand the condition of patients' lungs. Classifying the pixels representing the pulmonary arteries in medical images of the lungs can help doctors understand the anatomy of a patient's pulmonary arteries when operating on a patient. The existing pulmonary artery classification methods mainly input the pulmonary artery images extracted based on pulmonary medical images directly into the neural network for classification, which is prone to misclassification of the pulmonary artery.

发明内容SUMMARY OF THE INVENTION

有鉴于此,本说明书多个实施方式致力于提供像元的分类方法、模型训练方法、装置、设备和介质,以提供一种可以在一定程度上提升肺动脉图像序列中表示肺动脉的像元的分类的准确性的方法。In view of this, various embodiments of the present specification aim to provide a method for classifying pixels, a method for model training, an apparatus, a device, and a medium, so as to provide a classification method that can improve the classification of pixels representing pulmonary arteries in a sequence of pulmonary artery images to a certain extent method of accuracy.

本说明书一个实施方式提出了一种像元的分类方法,包括:获取肺动脉图像序列中表示肺动脉骨架的像元集合,和表示所述肺动脉骨架的像元集合中像元之间的邻接关系的邻接矩阵;提取所述肺动脉骨架的像元集合中像元的初级分类特征;其中,所述初级分类特征用于表征所述肺动脉骨架的像元集合中的像元;使用所述初级分类特征和所述邻接矩阵确定所述肺动脉骨架的像元集合中像元属于的肺动脉类别。An embodiment of the present specification proposes a method for classifying pixels, including: acquiring a pixel set representing a pulmonary artery skeleton in a pulmonary artery image sequence, and an adjacency representing an adjacency relationship between pixels in the pixel set of the pulmonary artery skeleton matrix; extract the primary classification features of the pixels in the pixel set of the pulmonary artery skeleton; wherein, the primary classification features are used to characterize the pixels in the pixel set of the pulmonary artery skeleton; use the primary classification features and all The adjacency matrix determines the pulmonary artery category to which a pixel in the pixel set of the pulmonary artery skeleton belongs.

本说明书一个实施方式提出了一种像元分类模型的训练方法,包括:构建训练样本;其中,所述训练样本包括肺动脉图像序列和肺动脉图像序列中表示肺动脉的像元对应的类别标签;获取肺动脉图像序列中表示肺动脉骨架的像元集合,和表示所述肺动脉骨架的像元集合中像元之间的邻接关系的邻接矩阵;提取所述肺动脉骨架的像元集合中像元的初级分类特征;其中,所述初级分类特征用于表征所述肺动脉骨架的像元集合中的像元;基于所述初级分类特征和所述邻接矩阵确定的所述肺动脉骨架的像元集合中像元的预测肺动脉类别;根据所述预测肺动脉类别和所述表示肺动脉的像元对应的类别标签生成的损失更新所述像元分类模型。An embodiment of the present specification proposes a training method for a pixel classification model, including: constructing a training sample; wherein the training sample includes a pulmonary artery image sequence and a category label corresponding to a pixel representing a pulmonary artery in the pulmonary artery image sequence; obtaining a pulmonary artery a pixel set representing the pulmonary artery skeleton in the image sequence, and an adjacency matrix representing the adjacency relationship between the pixels in the pixel set of the pulmonary artery skeleton; extracting primary classification features of the pixels in the pixel set of the pulmonary artery skeleton; Wherein, the primary classification feature is used to characterize the pixel in the pixel set of the pulmonary artery skeleton; the predicted pulmonary artery of the pixel in the pixel set of the pulmonary artery skeleton determined based on the primary classification feature and the adjacency matrix class; updating the pixel classification model according to the predicted pulmonary artery class and the loss generated by the class label corresponding to the pixel representing the pulmonary artery.

本说明书一个实施方式提出了一种像元的分类装置,包括:肺动脉骨架及邻接矩阵获取模块,用于获取肺动脉图像序列中表示肺动脉骨架的像元集合,和表示所述肺动脉骨架的像元集合中像元之间的邻接关系的邻接矩阵;初级分类特征提取模块,用于提取所述肺动脉骨架的像元集合中像元的初级分类特征;其中,所述初级分类特征用于表征所述肺动脉骨架的像元集合中的像元;肺动脉骨架分类模块,用于使用所述初级分类特征和所述邻接矩阵确定所述肺动脉骨架的像元集合中像元属于的肺动脉类别。An embodiment of the present specification proposes an apparatus for classifying pixels, including: a pulmonary artery skeleton and an adjacency matrix acquisition module, configured to acquire a pixel set representing the pulmonary artery skeleton in a pulmonary artery image sequence, and a pixel set representing the pulmonary artery skeleton The adjacency matrix of the adjacency relationship between the pixels; the primary classification feature extraction module is used to extract the primary classification features of the pixels in the pixel set of the pulmonary artery skeleton; wherein, the primary classification features are used to characterize the pulmonary artery A pixel in the pixel set of the skeleton; a pulmonary artery skeleton classification module, configured to use the primary classification feature and the adjacency matrix to determine the pulmonary artery category to which a pixel in the pixel set of the pulmonary artery skeleton belongs.

本说明是一个实施方式提出了一种像元分类模型的训练装置,包括:训练样本构建模块,用于构建训练样本;其中,所述训练样本包括肺动脉图像序列和肺动脉图像序列中表示肺动脉的像元对应的类别标签;肺动脉骨架及邻接矩阵获取模块,用于获取肺动脉图像序列中表示肺动脉骨架的像元集合,和表示所述肺动脉骨架的像元集合中像元之间的邻接关系的邻接矩阵;初级分类特征提取模块,用于提取所述肺动脉骨架的像元集合中像元的初级分类特征;其中,所述初级分类特征用于表征所述肺动脉骨架的像元集合中的像元;肺动脉类别预测模块,用于基于所述初级分类特征和所述邻接矩阵确定的所述肺动脉骨架的像元集合中像元的预测肺动脉类别;像元分类模型生成模块,用于根据所述预测肺动脉类别和所述表示肺动脉的像元对应的类别标签生成的损失更新所述像元分类模型。This description is an embodiment of a training device for a pixel classification model, including: a training sample building module for constructing a training sample; wherein the training sample includes a pulmonary artery image sequence and an image representing a pulmonary artery in the pulmonary artery image sequence The category label corresponding to the element; the pulmonary artery skeleton and the adjacency matrix acquisition module is used to obtain the pixel set representing the pulmonary artery skeleton in the pulmonary artery image sequence, and the adjacency matrix representing the adjacency relationship between the pixels in the pixel set of the pulmonary artery skeleton ; The primary classification feature extraction module is used to extract the primary classification features of the pixels in the pixel set of the pulmonary artery skeleton; wherein, the primary classification features are used to represent the pixels in the pixel set of the pulmonary artery skeleton; Pulmonary artery a category prediction module, used for predicting the pulmonary artery category of pixels in the pixel set of the pulmonary artery skeleton determined based on the primary classification feature and the adjacency matrix; a pixel classification model generation module, used for predicting the pulmonary artery category according to the The pixel classification model is updated with a loss generated from the class label corresponding to the pixel representing the pulmonary artery.

本说明书一个实施方式提出了一种电子设备,包括:处理器;用于存储所述处理器可执行指令的存储器;所述处理器,用于执行上述实施方式中所述的方法。An embodiment of the present specification provides an electronic device, including: a processor; a memory for storing instructions executable by the processor; and the processor for executing the method described in the foregoing embodiment.

本说明书一个实施方式提出了一种计算机可读存储介质,包括:所述存储介质存储有计算机程序,所述计算机程序用于执行上述实施方式所述的方法。An embodiment of the present specification provides a computer-readable storage medium, including: the storage medium stores a computer program, and the computer program is used to execute the method described in the foregoing embodiment.

本说明书多个实施方式通过提取肺动脉图像序列中表示肺动脉骨架的像元集合,并根据像元集合中像元之间的邻接关系构建邻接矩阵,然后基于肺动脉图像的初始肺动脉分类模型提取的表示肺动脉骨架的像元的初级像元特征和邻接矩阵确定像元集合中像元的肺动脉类别,实现了在一定程度上提升表示肺动脉骨架的像元的分类的准确性,进一步的提升了表示肺动脉的像元的分段染色的准确性,便于医生查看患者的肺动脉的解剖结构。Various embodiments of the present specification extract the pixel set representing the pulmonary artery skeleton in the pulmonary artery image sequence, and construct an adjacency matrix according to the adjacency relationship between the pixels in the pixel set, and then extract the image representing the pulmonary artery based on the initial pulmonary artery classification model of the pulmonary artery image. The primary pixel features and adjacency matrix of the pixels of the skeleton determine the pulmonary artery category of the pixels in the pixel set, which improves the classification accuracy of the pixels representing the pulmonary artery skeleton to a certain extent, and further improves the image representing the pulmonary artery. The accuracy of meta-segmented staining allows physicians to view the anatomy of a patient's pulmonary arteries.

附图说明Description of drawings

图1所示为一实施方式提供的一个场景示例中肺动脉分类系统交互的示意图。FIG. 1 is a schematic diagram illustrating the interaction of a pulmonary artery classification system in an example scenario provided by an embodiment.

图2所示为一实施方式提供的一个场景示例中肺动脉分段染色结果的示意图。FIG. 2 is a schematic diagram of a pulmonary artery segmented staining result in a scene example provided by an embodiment.

图3所示为一实施方式提供的一个场景示例中肺动脉分类系统交互的示意图。FIG. 3 is a schematic diagram illustrating the interaction of the pulmonary artery classification system in an example scenario provided by an embodiment.

图4所示为一实施方式提供的像元的分类方法的流程示意图。FIG. 4 is a schematic flowchart of a method for classifying pixels according to an embodiment.

图5所示为一实施方式提供的肺动脉骨架的分类结果的示意图。FIG. 5 is a schematic diagram showing the classification result of the pulmonary artery skeleton according to an embodiment.

图6所示为一实施方式提供的肺动脉骨架二分类结果的示意图。FIG. 6 is a schematic diagram of a binary classification result of the pulmonary artery skeleton provided by an embodiment.

图7所示为一实施方式提供的像元分类模型的训练方法的流程示意图。FIG. 7 is a schematic flowchart of a training method for a pixel classification model provided by an embodiment.

图8所示为一实施方式提供的像元的分类装置示意图。FIG. 8 is a schematic diagram of an apparatus for classifying pixels according to an embodiment.

图9所示为一实施方式提供的像元分类模型的训练装置示意图。FIG. 9 is a schematic diagram of a training device for a pixel classification model provided by an embodiment.

图10所示为一实施方式提供的电子设备的示意图。FIG. 10 is a schematic diagram of an electronic device provided by an embodiment.

具体实施方式Detailed ways

为了使本技术领域的人员更好的理解本说明书方案,下面将结合本说明书实施方式中的附图,对本说明书实施方式中的技术方案进行清楚、完整地描述,显然,所描述的实施方式仅是本说明书一部分实施方式,而不是全部的实施方式。基于本说明书中的实施方式,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施方式,都属于本说明书保护的范围。In order to enable those skilled in the art to better understand the solutions of the present specification, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present specification. Obviously, the described embodiments are only It is a part of embodiment of this specification, and it is not all embodiment. Based on the implementations in this specification, all other implementations obtained by persons of ordinary skill in the art without creative efforts shall fall within the protection scope of this specification.

请参阅图1和图2,本说明书实施方式提供一种肺动脉分类系统的场景示例。患者A在就诊时,医生可以给患者A开具了肺部CT影像检查单,患者A在进行肺部CT影像检查后可以获得肺部CT影像序列,然后将肺部CT影像序列发送给客户端。客户端可以将肺部CT影像序列发送给服务器。服务器首先将肺部CT影像序列切分成表示左肺的第一肺部影像序列和表示右肺的第二肺部影像序列。然后服务器可以分别将第一肺部影像序列和第二肺部影像序列通过上采样和/或下采样处理成数据量为192*128*96的第一目标肺部影像序列和第二目标肺部影像序列。服务器分别对第一肺部影像序列和第二肺部影像序列进行预处理,得到表示左肺的肺血管图像序列和表示右肺的肺血管图像序列,以及表示左肺的肺气管图像序列和表示右肺的肺气管图像序列。对表示左肺的肺血管图像序列和表示右肺的肺血管图像序列进行拼接处理得到肺血管图像序列。对表示左肺的肺气管图像序列和表示右肺的肺气管图像序列进行拼接处理得到肺气管图像序列。然后,服务器可以对肺血管图像序列进行预分割处理得到表示肺动脉的肺动脉图像序列。Referring to FIG. 1 and FIG. 2 , the embodiment of the present specification provides a scenario example of a pulmonary artery classification system. When patient A visits a doctor, the doctor can issue a lung CT image examination sheet to patient A. After the lung CT image examination, patient A can obtain the lung CT image sequence, and then send the lung CT image sequence to the client. The client can send the lung CT image sequence to the server. The server first divides the lung CT image sequence into a first lung image sequence representing the left lung and a second lung image sequence representing the right lung. Then, the server may process the first lung image sequence and the second lung image sequence into a first target lung image sequence and a second target lung image sequence with a data volume of 192*128*96 through up-sampling and/or down-sampling, respectively. image sequence. The server preprocesses the first lung image sequence and the second lung image sequence, respectively, to obtain a pulmonary blood vessel image sequence representing the left lung, a pulmonary blood vessel image sequence representing the right lung, and a pulmonary trachea image sequence and representation representing the left lung. Lung trachea image sequence of the right lung. The pulmonary blood vessel image sequence is obtained by stitching the pulmonary blood vessel image sequence representing the left lung and the pulmonary blood vessel image sequence representing the right lung. The pulmonary trachea image sequence is obtained by stitching the pulmonary trachea image sequence representing the left lung and the pulmonary trachea image sequence representing the right lung. Then, the server may perform pre-segmentation processing on the pulmonary blood vessel image sequence to obtain a pulmonary artery image sequence representing the pulmonary artery.

在获取到肺动脉图像序列之后,服务器可以将肺动脉图像序列和肺气管图像序列输入到ResU-net模型提取肺动脉图像序列中表示肺动脉的像元的初始像元特征。服务器还可以使用skeletonize算法提取肺动脉图像序列中表示肺动脉骨架的第一像元,并构造表示第一像元之间邻接关系的第一邻接矩阵。然后,服务器可以将第一邻接矩阵作为ResU-net模型的注意力机制,并基于初始像元特征提取第一像元对应的扩展特征,根据扩展特征生成第一像元对应的肺动脉类别。After acquiring the pulmonary artery image sequence, the server may input the pulmonary artery image sequence and the pulmonary trachea image sequence into the ResU-net model to extract initial pixel features of the pixels representing the pulmonary artery in the pulmonary artery image sequence. The server may also use the skeletonize algorithm to extract the first pixel representing the pulmonary artery skeleton in the pulmonary artery image sequence, and construct a first adjacency matrix representing the adjacency relationship between the first pixels. Then, the server can use the first adjacency matrix as the attention mechanism of the ResU-net model, extract the extended feature corresponding to the first pixel based on the initial pixel feature, and generate the pulmonary artery category corresponding to the first pixel according to the extended feature.

在生成了第一像元对应的肺动脉类别后,服务器可以将基于第一像元的分类结果生成表示肺动脉骨架分支的第二像元,并构造表示第二像元之间邻接关系的第二邻接矩阵。然后,服务器可以将第二邻接矩阵作为ResU-net模型的注意力机制,基于扩展特征提取第二像元对象的分支扩展特征,根据分支扩展特征生成第二像元对应的肺动脉类别。在第二像元对应的肺动脉类别与第一像元对应的肺动脉不一致时,将第二像元对应的肺动脉类别替换第一像元对应的肺动脉类别。After generating the pulmonary artery category corresponding to the first pixel, the server may generate a second pixel representing the skeleton branch of the pulmonary artery based on the classification result of the first pixel, and construct a second adjacency representing the adjacency relationship between the second pixels matrix. Then, the server can use the second adjacency matrix as the attention mechanism of the ResU-net model, extract branch extension features of the second pixel object based on the extension features, and generate a pulmonary artery category corresponding to the second pixel according to the branch extension features. When the pulmonary artery category corresponding to the second pixel is inconsistent with the pulmonary artery category corresponding to the first pixel, the pulmonary artery category corresponding to the second pixel is replaced with the pulmonary artery category corresponding to the first pixel.

最后,以第一像元作为种子点,采用预设的区域生长算法确定表示肺动脉的像元对应的肺动脉类别,并根据预设的颜色为表示肺动脉的像元赋予不同的颜色。例如,将肺动脉主干赋予红色,右肺上叶尖段赋予蓝色,右肺上叶后端赋予黄色,左肺下叶背段赋予橙色等。最后,服务器可以将分段染色后的肺动脉图像序列发送给客户端,供医生查看肺动脉详细的解剖结构,从而了解患者的肺部病变程度。Finally, using the first pixel as a seed point, a preset region growth algorithm is used to determine the pulmonary artery category corresponding to the pixel representing the pulmonary artery, and different colors are assigned to the pixel representing the pulmonary artery according to the preset color. For example, the main pulmonary artery is given red, the apical segment of the right upper lobe is given blue, the rear end of the right upper lobe is given yellow, and the dorsal segment of the left lower lobe is given orange. Finally, the server can send the segmented and stained pulmonary artery image sequence to the client, so that the doctor can view the detailed anatomical structure of the pulmonary artery, so as to understand the degree of lung lesions of the patient.

以上所述仅为本说明书提供的一个场景示例而已,并不用以限制本发明,凡在本发明的精神和原则之内,所作的任何修改、等同替换等,均应包含在本发明的保护范围之内。The above is only an example of a scenario provided in this specification, and is not intended to limit the present invention. Any modification, equivalent replacement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention. within.

请参阅图3,本说明书实施方式提供一种肺动脉分类系统。且本说明书提供的肺动脉图像的分类方法和/或肺动脉图像分类模型的训练方法可以应用于该肺动脉分类系统。该肺动脉分类系统可以包括由医学成像设备110、客户端120和服务器130形成的硬件环境。医学成像设备110与客户端120连接,服务器130通过网络与客户端120通过通信网络相连。其中,通信网络可以是有线网络或无线网络。医学成像设备110对肺部进行检查并成像,得到肺部图像序列。通过通信医学成像设备110将肺部图像序列传输至客户端120。客户端120向服务器130发送肺部图像序列,服务器130接收肺部图像序列。其中,医学成像设备110可以但不限于是超声波医学设备、CT医学检查设备、MRI医学检查设备中的至少一个。客户端120可以是具有网络访问能力的电子设备。具体的,例如,客户端可以是台式电脑、平板电脑、笔记本电脑、智能手机、数字助理、智能可穿戴设备、导购终端、电视机、智能音箱、麦克风等。其中,智能可穿戴设备包括但不限于智能手环、智能手表、智能眼镜、智能头盔、智能项链等。或者,客户端也可以为能够运行于所述电子设备中的软件。本领域技术人员可以知晓,上述客户端120的数量可以一个或多个,其类型可以相同或者不同。比如上述客户端120可以为一个,或者上述客户端120为几十个或几百个,或者更多数量。本申请实施方式中对客户端120的数量和设备类型不加以限定。服务器130可以是具有一定运算处理能力的电子设备。其可以具有网络通信模块、处理器和存储器等。当然,所述服务器也可以是指运行于所述电子设备中的软体。所述服务器还可以为分布式服务器,可以是具有多个处理器、存储器、网络通信模块等协同运作的系统。或者,服务器还可以为若干服务器形成的服务器集群。或者,随着科学技术的发展,服务器还可以是能够实现说明书实施方式相应功能的新的技术手段。例如,可以是基于量子计算实现的新形态的“服务器”。Referring to FIG. 3 , an embodiment of the present specification provides a pulmonary artery classification system. And the classification method of pulmonary artery image and/or the training method of pulmonary artery image classification model provided in this specification can be applied to the pulmonary artery classification system. The pulmonary artery classification system may include a hardware environment formed by a medical imaging device 110 , a client 120 and a server 130 . The medical imaging apparatus 110 is connected with the client 120, and the server 130 is connected with the client 120 through a communication network through a network. The communication network may be a wired network or a wireless network. The medical imaging device 110 examines and images the lungs, resulting in a sequence of lung images. The sequence of lung images is transmitted to the client 120 via the communicating medical imaging device 110 . The client 120 sends the lung image sequence to the server 130, and the server 130 receives the lung image sequence. The medical imaging device 110 may be, but is not limited to, at least one of an ultrasonic medical device, a CT medical inspection device, and an MRI medical inspection device. Client 120 may be an electronic device with network access capabilities. Specifically, for example, the client may be a desktop computer, a tablet computer, a notebook computer, a smart phone, a digital assistant, a smart wearable device, a shopping guide terminal, a TV, a smart speaker, a microphone, and the like. Among them, smart wearable devices include but are not limited to smart bracelets, smart watches, smart glasses, smart helmets, smart necklaces, and the like. Alternatively, the client can also be software that can run in the electronic device. Those skilled in the art may know that the number of the above-mentioned clients 120 may be one or more, and their types may be the same or different. For example, the number of the above-mentioned clients 120 may be one, or the number of the above-mentioned clients 120 may be dozens or hundreds, or more. The number of clients 120 and device types are not limited in the embodiments of the present application. The server 130 may be an electronic device with a certain computing processing capability. It may have a network communication module, processor and memory, among others. Of course, the server may also refer to software running in the electronic device. The server may also be a distributed server, which may be a system with multiple processors, memories, network communication modules, and the like that operate cooperatively. Alternatively, the server may also be a server cluster formed by several servers. Or, with the development of science and technology, the server may also be a new technical means capable of realizing the corresponding functions of the implementation manners of the description. For example, it can be a new form of "server" based on quantum computing.

请参阅图4和图5,本说明书实施方式提供像元的分类方法。所述像元的分类方法可以应用于电子设备。所述像元的分类方法可以包括以下步骤。Referring to FIG. 4 and FIG. 5 , the embodiment of the present specification provides a method for classifying pixels. The classification method of the picture element can be applied to electronic equipment. The classification method of the picture element may include the following steps.

步骤S210:获取肺动脉图像序列中表示肺动脉骨架的像元集合,和表示所述肺动脉骨架的像元集合中像元之间的邻接关系的邻接矩阵。Step S210: Obtain a pixel set representing the pulmonary artery skeleton in the pulmonary artery image sequence, and an adjacency matrix representing the adjacency relationship between the pixels in the pixel set of the pulmonary artery skeleton.

在一些情况下,处于同一肺动脉分支上表示肺动脉的像元具有相同的图像特征,如果直接对表示肺动脉的像元类别进行分类,可能会导致分类错误。因此,可以先提取表示肺动脉骨架的像元和表示肺动脉骨架的像元之间的邻接关系。In some cases, pixels representing the pulmonary artery on the same branch of the pulmonary artery have the same image features, which may lead to misclassification if the category of the pixel representing the pulmonary artery is directly classified. Therefore, the adjacency relationship between the pixel representing the pulmonary artery skeleton and the pixel representing the pulmonary artery skeleton can be extracted first.

所述肺动脉图像序列是基于肺部医学影像提取的表示肺动脉的图像序列。具体的,患者在进行一次肺部CT(Computed Tomography,电子计算机断层扫描)之后,可以生成肺部CT医学影像序列。通过对肺部CT医学影像序列进行分割处理,将肺部CT医学影像序列中表示肺动脉的像元赋值为1,表示非肺动脉的像元赋值为0,从而生成了一个三维的肺动脉图像序列。The pulmonary artery image sequence is an image sequence representing the pulmonary artery extracted based on a pulmonary medical image. Specifically, after a patient performs a lung CT (Computed Tomography, electronic computed tomography) scan, a lung CT medical image sequence can be generated. By segmenting the lung CT medical image sequence, the pixel representing the pulmonary artery in the lung CT medical image sequence is assigned as 1, and the pixel representing the non-pulmonary artery is assigned as 0, thereby generating a three-dimensional pulmonary artery image sequence.

所述肺动脉骨架是基于肺动脉图像序列中表示肺动脉的像元的连通区域细化成一个像素的宽度,用于表示肺动脉的像元集合中像元的特征提取和拓扑表示。具体的,例如,将二值化的肺动脉图像序列作为骨架提取模型的输入,使用morphology模块中的Skeletonize()函数,得到了一个细化了包括一个宽度的肺动脉骨架图像序列。其中,肺动脉骨架图像序列也是一个二值化的图像序列。The pulmonary artery skeleton is refined into a width of one pixel based on the connected area of the pixels representing the pulmonary artery in the pulmonary artery image sequence, and is used for feature extraction and topological representation of the pixels in the pixel set representing the pulmonary artery. Specifically, for example, the binarized pulmonary artery image sequence is used as the input of the skeleton extraction model, and the Skeletonize() function in the morphology module is used to obtain a refined pulmonary artery skeleton image sequence including a width. Among them, the pulmonary artery skeleton image sequence is also a binarized image sequence.

所述邻接矩阵用于表示所述像元集合中像元之间的邻接关系。具体的,例如,所述像元集合中包括3000个像元,那么可以构造一个3000*3000的二维矩阵,两个相同的像元之间没有邻接关系,因此在二维矩阵的对角线上的像元用“0”表示;对于两个不相同的像元,如果这两个像元在空间范围上的26邻域内相邻,那么在二维矩阵中用“1”表示,如果在26邻域内均不相邻,那么这两个像元的相关关系在二维矩阵中用“0”表示。The adjacency matrix is used to represent the adjacency relationship between the picture elements in the picture element set. Specifically, for example, if the pixel set includes 3000 pixels, then a 3000*3000 two-dimensional matrix can be constructed. There is no adjacency relationship between two identical pixels, so the diagonal of the two-dimensional matrix Pixels on the above are represented by "0"; for two different pixels, if the two pixels are adjacent in the 26-neighborhood on the spatial extent, they are represented by "1" in the two-dimensional matrix, if in the 26 are not adjacent in the neighborhood, then the correlation between the two pixels is represented by "0" in the two-dimensional matrix.

步骤S220:提取所述肺动脉骨架的像元集合中像元的初级分类特征;其中,所述初级分类特征用于表征所述肺动脉骨架的像元集合中的像元。Step S220 : extract the primary classification features of the pixels in the pixel set of the pulmonary artery skeleton; wherein, the primary classification features are used to characterize the pixels in the pixel set of the pulmonary artery skeleton.

在一些情况下,在对图像中的像元进行分类的过程中,首先要提取肺动脉图像中像元的特征,然后基于图像中像元的特征进行分类。因此,可以先利用常规的图像分类方法提取图像中像元的特征。In some cases, in the process of classifying the pixels in the image, the features of the pixels in the pulmonary artery image are first extracted, and then the classification is performed based on the features of the pixels in the image. Therefore, the features of the pixels in the image can be extracted first by using the conventional image classification method.

所述初级分类特征是基于初级像元分类模型的编码器提取得到的。具体的,例如,使用ResU-net模型的编码器提取肺动脉图像序列中表示肺动脉像元的初始像元特征。可以先将肺动脉图像序列沿三个方向分别进行切片处理,得到3个视图方向(俯视、正视、侧视)的二维数据,然后将3个视图方向的2维数据分别输入到ResU-net网络模型,将3个方向上提取的特征进行融合处理,得到肺动脉图像序列中表示肺动脉像元的初级分类特征。然后将表示肺动脉像元的初级分类特征中表示肺动脉骨架的像元的特征提取出来作为像元集合中像元的初级分类特征。The primary classification feature is extracted by an encoder based on the primary pixel classification model. Specifically, for example, the encoder of the ResU-net model is used to extract the initial pixel features representing the pulmonary artery pixels in the pulmonary artery image sequence. The pulmonary artery image sequence can be sliced along three directions to obtain 2D data in 3 view directions (top view, front view, side view), and then input the 2D data in the 3 view directions into the ResU-net network. Model, the features extracted in three directions are fused to obtain the primary classification features representing pulmonary artery pixels in the pulmonary artery image sequence. Then, the feature of the pixel representing the pulmonary artery skeleton in the primary classification feature representing the pulmonary artery pixel is extracted as the primary classification feature of the pixel in the pixel set.

步骤S230:使用所述初级分类特征和所述邻接矩阵确定所述肺动脉骨架的像元集合中像元属于的肺动脉类别。Step S230: Use the primary classification feature and the adjacency matrix to determine the pulmonary artery category to which a pixel in the pixel set of the pulmonary artery skeleton belongs.

在一些情况下,如果仅仅基于初级分类特征对肺动脉骨架的像元进行分类,可能会出现分类错误的情况。因此,可以将邻接矩阵作为像元分类模型的图注意力网络,然后基于图注意力网络和初级分类特征提取像元集合中像元的扩展特征,然后使用扩展特征确定像元集合中像元的肺动脉类别。In some cases, if the pixels of the pulmonary artery skeleton are classified based only on primary classification features, misclassification may occur. Therefore, the adjacency matrix can be used as the graph attention network of the pixel classification model, and then based on the graph attention network and the primary classification features, the extended features of the cells in the pixel set can be extracted, and then the extended features can be used to determine the pixels in the pixel set. Pulmonary artery category.

所述使用所述初级分类特征和所述邻接矩阵确定所述像元集合中像元的肺动脉类别的方法可以是将初级像元特征作为像元集合中像元的特征,然后使用图注意力网络(Graph Attention Networks,GATs)进一步描述相邻像元对于该像元分类的重要性,将邻接矩阵作为图注意力网络的掩膜,从而进一步丰富肺动脉图像序列中表示骨架的像元的特征。但是,以上只是本说明书示例性的图像中像元的特征提取方法,本说明书实施方式对图像的特征提取方法并不进行限制。The method for determining the pulmonary artery category of the pixel in the pixel set using the primary classification feature and the adjacency matrix may be to use the primary pixel feature as the feature of the pixel in the pixel set, and then use a graph attention network (Graph Attention Networks, GATs) further describe the importance of adjacent pixels for the classification of this pixel, and use the adjacency matrix as a mask for the graph attention network, thereby further enriching the features of the pixels representing the skeleton in the pulmonary artery image sequence. However, the above is only an exemplary method for extracting features of pixels in an image in this specification, and the embodiments of this specification do not limit the method for extracting features in images.

通过提取表示肺动脉骨架像元的初级分类特征和构建表示肺动脉骨架的像元集合中像元的邻接矩阵作为像元分类模型的图注意力网络进行融合提取像元集合中像元的特征,提升了肺动脉图像序列中表示肺动脉的像元的分类的准确性。By extracting the primary classification features of the pixels representing the pulmonary artery skeleton and constructing the adjacency matrix of the pixels in the pixel set representing the pulmonary artery skeleton as the pixel classification model, the graph attention network performs fusion to extract the features of the pixels in the pixel set. Accuracy of classification of pixels representing pulmonary arteries in a sequence of pulmonary artery images.

在一些实施方式中,提取所述肺动脉骨架的像元集合中像元的初级分类特征的步骤,可以包括:获取所述肺动脉图像序列对应的肺气管图像序列;其中,所述肺气管图像序列与所述肺动脉图像序列基于相同的肺部图像序列得到;构建肺动脉图像序列中表示肺动脉的像元集合;其中,所述肺动脉骨架的像元集合是所述肺动脉的像元集合的子集;基于所述肺动脉图像序列和所述肺气管图像序列,提取所述肺动脉的像元集合中的像元的初级分类特征;其中,所述初级分类特征用于表征所述肺动脉骨架的像元集合中的像元。In some embodiments, the step of extracting the primary classification features of the pixels in the pixel set of the pulmonary artery skeleton may include: acquiring a pulmonary trachea image sequence corresponding to the pulmonary artery image sequence; wherein the pulmonary trachea image sequence is the same as the The pulmonary artery image sequence is obtained based on the same lung image sequence; a pixel set representing the pulmonary artery in the pulmonary artery image sequence is constructed; wherein, the pixel set of the pulmonary artery skeleton is a subset of the pixel set of the pulmonary artery; The pulmonary artery image sequence and the pulmonary trachea image sequence are extracted, and the primary classification feature of the pixels in the pixel set of the pulmonary artery is extracted; wherein, the primary classification feature is used to characterize the image in the pixel set of the pulmonary artery skeleton. Yuan.

在一些情况下,肺动脉血管和支气管之间具有伴行关系。因此,在进行肺动脉血管的分类的过程中,可以将肺气管的分布信息作为参考,从而更加准确和快速的确定肺动脉血管的分类信息,从而提高了肺动脉血管分类的准确性。In some cases, there is a concomitant relationship between the pulmonary artery vessels and the bronchi. Therefore, in the process of classifying the pulmonary arteries, the distribution information of the pulmonary trachea can be used as a reference, so that the classification information of the pulmonary arteries can be determined more accurately and quickly, thereby improving the accuracy of the classification of the pulmonary arteries.

在一些实施方式中,像元的分类方法还可以包括:将获取的肺部图像序列调整至指定分辨率,得到目标肺部图像序列;基于所述目标肺部图像序列,确定所述肺部图像序列中表示肺血管的像元集合;其中,所述肺血管包括肺动脉血管和肺静脉血管;对所述肺部图像序列中表示肺血管的像元集合中的像元进行分类,得到表示肺动脉血管的动脉像元集合;基于所述动脉像元集合生成肺动脉图像序列。In some embodiments, the method for classifying pixels may further include: adjusting the acquired lung image sequence to a specified resolution to obtain a target lung image sequence; and determining the lung image based on the target lung image sequence A set of pixels representing pulmonary blood vessels in the sequence; wherein, the pulmonary blood vessels include pulmonary artery blood vessels and pulmonary vein blood vessels; classifying the pixels in the set of pixels representing pulmonary blood vessels in the lung image sequence to obtain a set of pixels representing pulmonary arterial blood vessels; A set of arterial pixels; a sequence of pulmonary artery images is generated based on the set of arterial pixels.

在一些情况下,在对肺动脉图像序列中表示肺动脉的像元进行分类的过程之前,还需要获取到肺部图像序列中表示肺动脉的像元,根据表示肺动脉的像元形成肺动脉图像序列。在将肺部图像序列输入肺动脉分割模型之前,为了便于模型的运算,还需要将肺部图像序列调整至指定的分辨率。具体的,例如,根据患者肺部CT影像中表示肺动脉的像元在肺部CT影像中的具体范围的各项同性之后的平均比例为1.84:1.28:1,然后将模型的输入改成【192,128,96】。当肺部图像序列的尺寸大于【192,128,96】时,将肺部图像序列通过卷积核下采样至【192,128,96】,得到目标肺部图像序列;当肺部图像序列的尺寸小于【192,128,96】时,将肺部图像序列通过双线性插值法上采样至【192,128,96】,得到目标肺部图像序列,从而保证了图像在模型中的比例是一致的。然后,将目标肺部图像序列输入至U-net图像分割模型,得到肺血管图像序列。最后,进一步的提取肺血管图像序列中表示肺动脉的像元,从而得到肺动脉图像序列。In some cases, before the process of classifying the pixels representing the pulmonary artery in the pulmonary artery image sequence, the pixels representing the pulmonary artery in the lung image sequence also need to be acquired, and the pulmonary artery image sequence is formed according to the pixels representing the pulmonary artery. Before the lung image sequence is input into the pulmonary artery segmentation model, in order to facilitate the operation of the model, the lung image sequence needs to be adjusted to the specified resolution. Specifically, for example, according to the average ratio of the pixels representing the pulmonary artery in the CT image of the patient's lung after the specific range of the isotropy in the CT image of the lung, the average ratio is 1.84:1.28:1, and then the input of the model is changed to [192] , 128, 96]. When the size of the lung image sequence is larger than [192, 128, 96], the lung image sequence is downsampled to [192, 128, 96] through the convolution kernel to obtain the target lung image sequence; When the size is smaller than [192, 128, 96], the lung image sequence is upsampled to [192, 128, 96] by bilinear interpolation to obtain the target lung image sequence, thus ensuring that the proportion of the image in the model is consistent. Then, the target lung image sequence is input into the U-net image segmentation model to obtain the pulmonary blood vessel image sequence. Finally, the pixels representing the pulmonary artery in the pulmonary blood vessel image sequence are further extracted to obtain the pulmonary artery image sequence.

在一些实施方式中,像元的分类方法还可以包括:将初始肺部图像序列切分成第一肺部图像序列和第二肺部图像序列;其中,所述第一肺部图像序列表示左侧肺部区域;所述第二肺部图像序列表示右侧肺部区域;相应的,在将获取的肺部图像序列调整至指定分辨率,得到目标肺部图像序列的步骤中,可以包括:分别将所述第一肺部图像序列和所述第二肺部图像序列调整至指定分辨率,得到第一目标肺部图像序列和第二目标肺部图像序列;其中所述第一目标肺部图像序列和第二目标肺部图像序列属于目标肺部图像序列。In some embodiments, the method for classifying pixels may further include: dividing the initial lung image sequence into a first lung image sequence and a second lung image sequence; wherein the first lung image sequence represents the left side the lung area; the second lung image sequence represents the right lung area; correspondingly, in the step of adjusting the acquired lung image sequence to a specified resolution to obtain the target lung image sequence, the steps may include: Adjusting the first lung image sequence and the second lung image sequence to a specified resolution to obtain a first target lung image sequence and a second target lung image sequence; wherein the first target lung image The sequence and the second target lung image sequence belong to the target lung image sequence.

在一些情况下,直接对整个肺部图像序列进行处理的时候,容易导致左右肺中肺血管的分类错误,因此可以将肺部图像序列切分成表示左肺的第一肺部图像序列和表示右肺的第二肺部图像序列。一方面,提升了肺部图像序列输入至图像分割模型的分辨率;另一方面,也可以减少模型在运行过程中的显存。但是,需要说明的是,本说明书实施方式中所述的第一肺部图像序列和第二肺部图像序列只是用于表示两者的不同,并不限制于将第一肺部图像序列表示左肺,第二肺部图像序列表示右肺。In some cases, when directly processing the entire lung image sequence, it is easy to cause misclassification of the pulmonary blood vessels in the left and right lungs. Therefore, the lung image sequence can be divided into a first lung image sequence representing the left lung and a lung image sequence representing the right lung. Second lung image sequence of the lungs. On the one hand, it improves the resolution of the lung image sequence input to the image segmentation model; on the other hand, it can also reduce the memory of the model during operation. However, it should be noted that the first lung image sequence and the second lung image sequence described in the embodiments of this specification are only used to represent the difference between the two, and are not limited to the first lung image sequence representing the left Lungs, the second lung image sequence represents the right lung.

在一些实施方式中,使用所述初级分类特征和所述邻接矩阵确定所述肺动脉骨架的像元集合中像元属于的肺动脉类别的步骤,可以包括:使用所述初级分类特征和所述邻接矩阵提取所述肺动脉骨架的像元集合中的像元扩展分类特征;基于所述扩展分类特征生成所述肺动脉骨架的像元集合中像元的肺动脉类别。In some embodiments, the step of using the primary classification feature and the adjacency matrix to determine the pulmonary artery category to which a pixel in the pixel set of the pulmonary artery skeleton belongs may include: using the primary classification feature and the adjacency matrix Extracting the pixel extended classification feature in the pixel set of the pulmonary artery skeleton; generating the pulmonary artery category of the pixel in the pixel set of the pulmonary artery skeleton based on the extended classification feature.

在一些情况下,为了确定肺动脉像元集合中像元的肺动脉类别,首先需要获取到表示肺动脉的像元对应的像元特征。因此,可以先基于常用的图像分类模型提取肺动脉像元的像元特征,然后再结合肺动脉骨架的像元集合中像元之间的邻接关系丰富像元特征的表达。In some cases, in order to determine the pulmonary artery category of the pixel in the pulmonary artery pixel set, it is first necessary to obtain the pixel feature corresponding to the pixel representing the pulmonary artery. Therefore, the pixel features of the pulmonary artery pixels can be extracted based on the commonly used image classification model, and then the adjacency relationship between the pixels in the pixel set of the pulmonary artery skeleton can be combined to enrich the expression of the pixel features.

所述扩展分类特征是基于初级分类特征和邻接矩阵中像元之间的邻接关系得到的。具体的,例如,首先利用ResU-net网络获取到初级分类特征,然后将初级分类特征和邻接矩阵输入到图注意力网络模型提取的特征作为扩展分类特征。然后,基于扩展分类特征确定肺动脉图像序列中表示肺动脉的像元集合中像元的肺动脉类别。The extended classification feature is obtained based on the primary classification feature and the adjacency relationship between the pixels in the adjacency matrix. Specifically, for example, first use the ResU-net network to obtain primary classification features, and then input the primary classification features and adjacency matrix into the features extracted by the graph attention network model as extended classification features. Then, the pulmonary artery class of the pixels in the set of pixels representing the pulmonary artery in the pulmonary artery image sequence is determined based on the extended classification features.

请参阅图6,在一些实施方式中,像元的分类方法还可以包括:基于所述肺动脉骨架的像元集合中像元的肺动脉类别确定所述肺动脉骨架的像元集合中表示肺动脉骨架分支的多个分支像元集合;其中,所述肺动脉骨架分支与所述肺动脉骨架主干相连;所述分支像元集合是所述肺动脉骨架的像元集合的子集;获取表示所述分支像元集合中像元的邻接关系的分支邻接矩阵;使用所述扩展分类特征和所述分支邻接矩阵确定所述分支像元集合中像元的肺动脉类别。Referring to FIG. 6 , in some embodiments, the method for classifying pixels may further include: determining, based on the pulmonary artery category of the pixels in the pixel set of the pulmonary artery skeleton, a branch of the pulmonary artery skeleton in the pixel set of the pulmonary artery skeleton representing the branch of the pulmonary artery skeleton. a plurality of branch pixel sets; wherein, the branches of the pulmonary artery skeleton are connected to the trunk of the pulmonary artery skeleton; the branch pixel set is a subset of the pixel set of the pulmonary artery skeleton; the acquisition represents the branch pixel set in the branch pixel set A branch adjacency matrix of adjacency relationships of pixels; using the extended classification feature and the branch adjacency matrix to determine the pulmonary artery category of the pixels in the branch pixel set.

在一些情况下,处在同一个肺动脉分支上的肺动脉像元具有相同的肺动脉特征。因此,可以将一个分支上的像元作为一个整体,从而确定整个肺动脉分支的肺动脉类别。In some cases, pulmonary artery pixels on the same pulmonary artery branch have the same pulmonary artery characteristics. Therefore, the pixels on one branch can be taken as a whole to determine the pulmonary artery class of the entire pulmonary artery branch.

在一些实施方式中,使用所述扩展分类特征和所述分支邻接矩阵确定所述分支像元集合中像元的肺动脉类别的步骤,可以包括:在所述分支像元集合中选取指定个数的像元作为参考像元集合;获取所述参考像元集合中像元的扩展分类特征;基于所述扩展分类特征和所述分支邻接矩阵确定所述分支像元集合中的像元的肺动脉类别。In some embodiments, the step of using the extended classification feature and the branch adjacency matrix to determine the pulmonary artery category of the pixel in the branch pixel set may include: selecting a specified number of The pixel is used as a reference pixel set; the extended classification feature of the pixel in the reference pixel set is obtained; the pulmonary artery category of the pixel in the branch pixel set is determined based on the extended classification feature and the branch adjacency matrix.

在一些情况下,处在同一个分支上的肺动脉像元具有相同的肺动脉特征。因此,可以先在肺动脉分支上随机的选取指定个数的像元数。然后,基于选取的像元的扩展特征和分支邻接矩阵确定分支像元集合中像元的肺动脉类别。In some cases, pulmonary artery pixels on the same branch have the same pulmonary artery characteristics. Therefore, a specified number of pixels can be randomly selected on the branch of the pulmonary artery. Then, the pulmonary artery category of the cells in the set of branch cells is determined based on the extended features of the selected cells and the branch adjacency matrix.

所述使用所述扩展分类特征和所述分支邻接矩阵确定所述分支像元集合中像元的肺动脉类别的方法可以是在骨架分支上随机的选取指定个数的像元,然后对指定个数的像元进行线性变换,得到骨架分支上的特征,基于骨架分支的特征确定肺动脉骨架分支的肺动脉类别。The method for determining the pulmonary artery category of the pixel in the branch pixel set by using the extended classification feature and the branch adjacency matrix may be to randomly select a specified number of pixels on the skeleton branch, and then select the specified number of pixels. Linear transformation is performed on the pixels of the skeleton to obtain the features on the skeleton branches, and the pulmonary artery category of the pulmonary artery skeleton branches is determined based on the features of the skeleton branches.

在一些实施方式中,像元的分类方法还可以包括:将所述像元集合中的像元作为种子像元;基于所述种子像元,采用预设的区域生长算法确定所述肺动脉图像序列中表示肺动脉的像元的肺动脉类别。In some embodiments, the method for classifying pixels may further include: using the pixels in the pixel set as seed pixels; and determining the pulmonary artery image sequence by using a preset region growing algorithm based on the seed pixels. The pulmonary artery category of the cells representing the pulmonary artery in .

在一些情况下,还需要对肺动脉上的其它像元进行分类。因此,可以基于肺动脉骨架分支上的像元的分类结果进行区域生长,然后将肺动脉骨架分支上的像元的分类结果赋予给基于肺动脉骨架分支进行区域生长的像元,从而得到肺动脉图像序列中表示肺动脉的像元的分类结果。In some cases, other pixels on the pulmonary artery also need to be classified. Therefore, region growth can be performed based on the classification results of the pixels on the pulmonary artery skeleton branches, and then the classification results of the pixels on the pulmonary artery skeleton branches can be assigned to the pixels whose regions are grown based on the pulmonary artery skeleton branches, so as to obtain the representation in the pulmonary artery image sequence. Classification results for the pixels of the pulmonary artery.

在一些实施方式中,像元的分类方法还可以包括:对所述肺动脉图像序列中表示肺动脉的像元的肺动脉类别进行染色处理;其中,不同的肺动脉类别对应有不同的颜色。In some embodiments, the method for classifying pixels may further include: coloring the pulmonary artery categories of the pixels representing the pulmonary artery in the pulmonary artery image sequence; wherein, different pulmonary artery categories correspond to different colors.

在一些情况下,为了让医生更直观的看到肺动脉的解剖结构,还可以对不同类别的肺动脉进行染色处理,从而得到肺动脉的分段染色结果。具体的,例如,肺动脉类别包括肺动脉主干、右肺上叶尖端、右肺上叶后段、右肺上叶前段、右肺中野外侧段、右肺下叶上段等类别,因此可以将肺动脉主干赋予红色、右肺上叶尖端赋予蓝色、右肺上叶后段赋予绿色、右肺上叶前段赋予橙色、右肺中野外侧段赋予紫色、右肺下叶上段赋予黄色等不同的颜色。其中,不同的类别对应有不同的颜色,且在空间上位置接近的肺动脉类别的颜色差异应当较为明显。In some cases, in order to allow doctors to see the anatomical structure of the pulmonary artery more intuitively, different types of pulmonary arteries can also be stained to obtain segmented staining results of the pulmonary artery. Specifically, for example, the pulmonary artery category includes the main pulmonary artery, the tip of the right upper lobe, the posterior segment of the right upper lobe, the anterior segment of the right upper lobe, the mid-field lateral segment of the right lung, and the upper segment of the right lower lobe. Therefore, the main pulmonary artery can be assigned to Red, blue for the tip of the right upper lobe, green for the posterior segment of the right upper lobe, orange for the anterior segment of the right upper lobe, purple for the mid-field lateral segment of the right lung, and yellow for the upper segment of the right lower lobe. Among them, different categories correspond to different colors, and the color difference of the pulmonary artery categories that are close in space should be more obvious.

请参阅图7,本说明书实施方式提供一种像元分类模型的训练方法。所述像元分类模型的训练方法可以应用于电子设备。所述像元分类模型的训练方法可以包括以下步骤。Referring to FIG. 7 , an embodiment of the present specification provides a training method for a pixel classification model. The training method of the pixel classification model can be applied to electronic equipment. The training method of the pixel classification model may include the following steps.

步骤S310:构建训练样本;其中,所述训练样本包括肺动脉图像序列和肺动脉图像序列中表示肺动脉的像元对应的类别标签。Step S310 : constructing a training sample; wherein, the training sample includes a pulmonary artery image sequence and a category label corresponding to a pixel representing a pulmonary artery in the pulmonary artery image sequence.

在一些情况下,训练样本构造的好坏直接影响模型的精度。因此,在对肺动脉图像序列进行标注需要选择专业水平较高、专业经验较为丰富的医生进行标注。但是,需要说明的是,本申请实施方式中并不限定训练样本的具体形式,可以是原始的医学影像,也可以是经过预处理后的医学影像,还可以是原始的医学影像的一部分。In some cases, the quality of the training sample construction directly affects the accuracy of the model. Therefore, when labeling the pulmonary artery image sequence, it is necessary to select doctors with higher professional level and rich professional experience for labeling. However, it should be noted that the specific form of the training sample is not limited in the embodiments of the present application, and it may be an original medical image, a pre-processed medical image, or a part of the original medical image.

步骤S320:获取肺动脉图像序列中表示肺动脉骨架的像元集合,和表示所述肺动脉骨架的像元集合中像元之间的邻接关系的邻接矩阵。Step S320: Acquire a pixel set representing the pulmonary artery skeleton in the pulmonary artery image sequence, and an adjacency matrix representing the adjacency relationship between the pixels in the pixel set of the pulmonary artery skeleton.

步骤S330:提取所述肺动脉骨架的像元集合中像元的初级分类特征;其中,所述初级分类特征用于表征所述肺动脉骨架的像元集合中的像元。Step S330: Extract the primary classification features of the pixels in the pixel set of the pulmonary artery skeleton; wherein, the primary classification features are used to characterize the pixels in the pixel set of the pulmonary artery skeleton.

步骤S340:基于所述初级分类特征和所述邻接矩阵确定的所述肺动脉骨架的像元集合中像元的预测肺动脉类别。Step S340: The predicted pulmonary artery category of the pixels in the pixel set of the pulmonary artery skeleton determined based on the primary classification feature and the adjacency matrix.

步骤S350:根据所述预测肺动脉类别和所述表示肺动脉的像元对应的类别标签生成的损失更新所述像元分类模型。Step S350: Update the pixel classification model according to the predicted pulmonary artery category and the loss generated by the category label corresponding to the pixel representing the pulmonary artery.

所述像元分类模型用于生成训练样本中表示肺动脉骨架的像元集合中像元的预测肺动脉类别。然后计算预测肺动脉类别和表示肺动脉的像元对应的类别标签的损失函数。根据损失函数更新像元分类模型,在损失函数收敛的情况下,将更新后模型的参数作为像元分类模型的参数。The pixel classification model is used to generate a predicted pulmonary artery class for the pixel in the pixel set representing the pulmonary artery skeleton in the training sample. The loss function is then calculated to predict the pulmonary artery class and the class label corresponding to the pixel representing the pulmonary artery. The pixel classification model is updated according to the loss function. When the loss function converges, the parameters of the updated model are used as the parameters of the pixel classification model.

所述初级分类特征可以是初始分类模型的编码器得到的。其中,初级分类模型可以是卷积神经网络(Convolutional Neural Network,CNN)、深度神经网络(Deep NeuralNetwork,DNN)或循环神经网络(Recurrent Neural Network,RNN)等,本实施方式中对初级分类模型的具体类型不作限定。本实施方式中初级分类模型可以包括输入层、卷积层、池化层、连接层等神经网络层,本实施方式中对此不作具体限定。另外,本实施方式中对每一种神经网络层的个数也不作限定。The primary classification features may be obtained by the encoder of the initial classification model. The primary classification model may be a convolutional neural network (Convolutional Neural Network, CNN), a deep neural network (Deep Neural Network, DNN), or a recurrent neural network (Recurrent Neural Network, RNN), etc. In this embodiment, the primary classification model is The specific type is not limited. The primary classification model in this embodiment may include neural network layers such as an input layer, a convolution layer, a pooling layer, and a connection layer, which are not specifically limited in this embodiment. In addition, in this embodiment, the number of each type of neural network layer is also not limited.

在一些实施方式中,所述肺动脉图像序列还对应有肺动脉骨架主干标签和肺动脉骨架分支标签;其中,所述肺动脉骨架分支标签对应有分支邻接矩阵;所述方法还可以包括:在所述肺动脉骨架分支标签中选取指定个数的像元作为参考像元集合;获取所述参考像元集合中像元的扩展分类特征;其中,所述展分类特征用于表征所述参考像元集合中的像元;相应的,在基于所述初级分类特征和所述邻接矩阵确定的所述像元集合中像元的预测肺动脉类别的步骤中,还可以包括:基于所述扩展特征和所述分支邻接矩阵确定所述像元集合中像元的预测肺动脉类别。In some embodiments, the pulmonary artery image sequence further corresponds to a pulmonary artery skeleton trunk label and a pulmonary artery skeleton branch label; wherein, the pulmonary artery skeleton branch label corresponds to a branch adjacency matrix; the method may further include: in the pulmonary artery skeleton Select a specified number of pixels in the branch label as the reference pixel set; obtain the extended classification feature of the pixels in the reference pixel set; wherein, the extended classification feature is used to characterize the image in the reference pixel set. Correspondingly, in the step of predicting the pulmonary artery category of the pixel in the pixel set determined based on the primary classification feature and the adjacency matrix, it may further include: based on the extended feature and the branch adjacency matrix A predicted pulmonary artery class for the cells in the set of cells is determined.

在一些情况下,在某一个分支上的像元具有相同的图像特征。因此,可以在肺动脉分支上随机的选取指定个数的像元进行特征提取,然后通过线性变换的方法得到整个分支上像元的肺动脉类别。本实施方式中所述的测肺动脉类别的确定方法与上述实施方式中的方法相同,具体细节在此不再赘述,请参见上述实施方式。但是,需要说明的是,本实施方式中已对肺动脉的主干和分支进行了标注,因此,不需要基于肺动脉骨架二分类的结果确定肺动脉分支。In some cases, the pixels on a certain branch have the same image features. Therefore, a specified number of pixels can be randomly selected on the pulmonary artery branch for feature extraction, and then the pulmonary artery category of the pixels on the entire branch can be obtained by a linear transformation method. The method for determining the pulmonary artery type described in this embodiment is the same as the method in the above embodiment, and the specific details are not repeated here, please refer to the above embodiment. However, it should be noted that in this embodiment, the trunk and branches of the pulmonary artery have been marked, so it is not necessary to determine the branch of the pulmonary artery based on the result of the binary classification of the pulmonary artery skeleton.

请参阅图8,本说明书实施方式提供一种像元的分类装置,可以包括:肺动脉骨架及邻接矩阵获取模块、初级分类特征提取模块和肺动脉骨架分类模块。Referring to FIG. 8 , an embodiment of the present specification provides an apparatus for classifying pixels, which may include: a pulmonary artery skeleton and an adjacency matrix acquisition module, a primary classification feature extraction module, and a pulmonary artery skeleton classification module.

肺动脉骨架及邻接矩阵获取模块,用于获取肺动脉图像序列中表示肺动脉骨架的像元集合,和表示所述肺动脉骨架的像元集合中像元之间的邻接关系的邻接矩阵。The pulmonary artery skeleton and adjacency matrix acquisition module is used to acquire a pixel set representing the pulmonary artery skeleton in the pulmonary artery image sequence, and an adjacency matrix representing the adjacency relationship between the pixels in the pixel set of the pulmonary artery skeleton.

初级分类特征提取模块,用于提取所述肺动脉骨架的像元集合中像元的初级分类特征;其中,所述初级分类特征用于表征所述肺动脉骨架的像元集合中的像元。The primary classification feature extraction module is configured to extract the primary classification features of the pixels in the pixel set of the pulmonary artery skeleton; wherein, the primary classification features are used to represent the pixels in the pixel set of the pulmonary artery skeleton.

肺动脉骨架分类模块,用于使用所述初级分类特征和所述邻接矩阵确定所述肺动脉骨架的像元集合中像元属于的肺动脉类别。A pulmonary artery skeleton classification module, configured to use the primary classification feature and the adjacency matrix to determine a pulmonary artery category to which a pixel in the pixel set of the pulmonary artery skeleton belongs.

请参阅图9,本说明书实施方式提供一种像元分类模型的训练装置,可以包括:训练样本构建模块、肺动脉骨架及邻接矩阵获取模块、初级分类特征提取模块、肺动脉类别预测模块和像元分类模型生成模块。Referring to FIG. 9, an embodiment of the present specification provides a training device for a pixel classification model, which may include: a training sample building module, a pulmonary artery skeleton and an adjacency matrix acquisition module, a primary classification feature extraction module, a pulmonary artery category prediction module, and a pixel classification module Model generation module.

训练样本构建模块,用于构建训练样本;其中,所述训练样本包括肺动脉图像序列和肺动脉图像序列中表示肺动脉的像元对应的类别标签。A training sample building module is used to construct a training sample; wherein, the training sample includes a pulmonary artery image sequence and a category label corresponding to a pixel representing a pulmonary artery in the pulmonary artery image sequence.

肺动脉骨架及邻接矩阵获取模块,用于获取肺动脉图像序列中表示肺动脉骨架的像元集合,和表示所述肺动脉骨架的像元集合中像元之间的邻接关系的邻接矩阵。The pulmonary artery skeleton and adjacency matrix acquisition module is used to acquire a pixel set representing the pulmonary artery skeleton in the pulmonary artery image sequence, and an adjacency matrix representing the adjacency relationship between the pixels in the pixel set of the pulmonary artery skeleton.

初级分类特征提取模块,用于提取所述肺动脉骨架的像元集合中像元的初级分类特征;其中,所述初级分类特征用于表征所述肺动脉骨架的像元集合中的像元。The primary classification feature extraction module is configured to extract the primary classification features of the pixels in the pixel set of the pulmonary artery skeleton; wherein, the primary classification features are used to represent the pixels in the pixel set of the pulmonary artery skeleton.

肺动脉类别预测模块,用于基于所述初级分类特征和所述邻接矩阵确定的所述肺动脉骨架的像元集合中像元的预测肺动脉类别。A pulmonary artery category prediction module, configured to predict a pulmonary artery category of a pixel in the pixel set of the pulmonary artery skeleton determined based on the primary classification feature and the adjacency matrix.

像元分类模型生成模块,用于根据所述预测肺动脉类别和所述表示肺动脉的像元对应的类别标签生成的损失更新所述像元分类模型。A pixel classification model generation module, configured to update the pixel classification model according to the predicted pulmonary artery category and the loss generated by the class label corresponding to the pixel representing the pulmonary artery.

关于像元的分类装置和/或像元分类模型的训练装置实现的具体功能和效果,可以参照本说明书其他实施方式对照解释,在此不再赘述。所述像元的分类装置和/或像元分类模型的训练装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。所述各模块可以以硬件形式内嵌于或独立于计算机设备中的处理器中,也可以以软件形式存储于计算机设备中的存储器中,以便于处理器调用执行以上各个模块对应的操作。The specific functions and effects implemented by the device for classifying pixels and/or the device for training pixel classification models can be explained with reference to other embodiments of the present specification, and details are not repeated here. Each module in the image element classification device and/or the image element classification model training device may be implemented in whole or in part by software, hardware, and combinations thereof. The modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

请参阅图10,在一些实施方式中可以提供一种电子设备,所述电子设备包括:处理器;用于存储所述处理器可执行指令的存储器;所述处理器,用于执行上述实施方式中所述的方法。Referring to FIG. 10 , in some embodiments, an electronic device may be provided, the electronic device includes: a processor; a memory for storing instructions executable by the processor; the processor for executing the foregoing embodiments method described in.

在一些实施方式中可以提供了一种计算机可读存储介质,其上存储有计算机程序,计算机程序被处理器执行时实现所述实施方式中的方法步骤。In some embodiments, a computer-readable storage medium may be provided on which a computer program is stored, and when the computer program is executed by a processor, implements the method steps in the described embodiments.

本领域普通技术人员可以理解实现所述实施方式方法中的全部或部分流程,是可以通过计算机程序来指令相关的硬件来完成,所述的计算机程序可存储于一非易失性计算机可读取存储介质中,该计算机程序在执行时,可包括如所述各方法的实施方式的流程。其中,本说明书所提供的各实施方式中所使用的对存储器、存储、数据库或其它介质的任何引用,均可包括非易失性和易失性存储器中的至少一种。非易失性存储器可包括只读存储器(Read-Only Memory,ROM)、磁带、软盘、闪存或光存储器等。易失性存储器可包括随机存取存储器(Random Access Memory,RAM)或外部高速缓冲存储器。作为说明而非局限,RAM可以是多种形式,比如静态随机存取存储器(Static Random Access Memory,SRAM)或动态随机存取存储器(Dynamic Random Access Memory,DRAM)等。Those of ordinary skill in the art can understand that all or part of the processes in the implementation method can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer readable In the storage medium, when the computer program is executed, it may include the flow of the implementation manner of each method. Wherein, any reference to memory, storage, database or other medium used in the various embodiments provided in this specification may include at least one of non-volatile and volatile memory. The non-volatile memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, and the like. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, the RAM may be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM).

应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器,使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

本说明书多个实施方式之间,采用递进的方式进行描述。不同的实施方式着重于描述相较于其它实施方式不相同的部分。所属领域技术人员在阅读本说明书之后,可以获知本说明书中的多个实施方式,以及实施方式揭示的多个技术特征,可以进行更多种的组合,为使描述简洁,未对所述实施方式中的各个技术特征所有可能的组合都进行描述。然而,只要这些技术特征的组合不存在矛盾,都应当认为是本说明书记载的范围。Among the various embodiments in this specification, the description is made in a progressive manner. Different embodiments focus on describing parts that are different from other embodiments. After reading this specification, those skilled in the art can know multiple embodiments in this specification, as well as multiple technical features disclosed in the embodiments, and can make more combinations. All possible combinations of the individual technical features in are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the description in this specification.

还需要说明的是,术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含,从而使得包括一系列要素的过程、方法、商品或者设备不仅包括那些要素,而且还包括没有明确列出的其他要素,或者是还包括为这种过程、方法、商品或者设备所固有的要素。在没有更多限制的情况下,由语句“包括一个……”限定的要素,并不排除在包括所述要素的过程、方法、商品或者设备中还存在另外的相同要素。It should also be noted that the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device comprising a series of elements includes not only those elements, but also Other elements not expressly listed, or which are inherent to such a process, method, article of manufacture, or apparatus are also included. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article of manufacture, or device that includes the element.

本说明书中的多个实施方式本身均着重于强调与其他实施方式不同的部分,各实施方式之间可以相互对照解释。所属领域技术人员基于一般的技术常识对本说明书中的多个实施方式的任意组合均涵盖于本说明书的揭示范围内。The various embodiments in this specification themselves emphasize the parts that are different from other embodiments, and each embodiment can be explained in comparison with each other. Any combination of the various embodiments in this specification based on general technical knowledge by those skilled in the art shall be included in the disclosure scope of this specification.

以上所述仅为本案的实施方式而已,并不用以限制本案的权利要求保护范围。对于本领域技术人员来说,本案可以有各种更改和变化。凡在本案的精神和原理之内所作的任何修改、等同替换、改进等,均应包含在本案的权利要求范围之内。The above descriptions are merely embodiments of the present case, and are not intended to limit the protection scope of the claims of the present case. For those skilled in the art, various modifications and variations can be made in this case. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this case shall be included within the scope of the claims of this case.

Claims (15)

1.一种像元的分类方法,其特征在于,所述方法包括:1. a classification method of picture element, is characterized in that, described method comprises: 获取肺动脉图像序列中表示肺动脉骨架的像元集合,和表示所述肺动脉骨架的像元集合中像元之间的邻接关系的邻接矩阵;acquiring a pixel set representing the pulmonary artery skeleton in the pulmonary artery image sequence, and an adjacency matrix representing the adjacency relationship between pixels in the pixel set of the pulmonary artery skeleton; 提取所述肺动脉骨架的像元集合中像元的初级分类特征;其中,所述初级分类特征用于表征所述肺动脉骨架的像元集合中的像元;extracting primary classification features of pixels in the pixel set of the pulmonary artery skeleton; wherein, the primary classification features are used to characterize the pixels in the pixel set of the pulmonary artery skeleton; 使用所述初级分类特征和所述邻接矩阵确定所述肺动脉骨架的像元集合中像元属于的肺动脉类别。The primary classification feature and the adjacency matrix are used to determine the pulmonary artery class to which a pixel in the pixel set of the pulmonary artery skeleton belongs. 2.根据权利要求1所述的方法,其特征在于,提取所述肺动脉骨架的像元集合中像元的初级分类特征的步骤,包括:2. The method according to claim 1, wherein the step of extracting primary classification features of pixels in the pixel set of the pulmonary artery skeleton comprises: 获取所述肺动脉图像序列对应的肺气管图像序列;其中,所述肺气管图像序列与所述肺动脉图像序列基于相同的肺部图像序列得到;acquiring a pulmonary trachea image sequence corresponding to the pulmonary artery image sequence; wherein, the pulmonary trachea image sequence and the pulmonary artery image sequence are obtained based on the same lung image sequence; 构建肺动脉图像序列中表示肺动脉的像元集合;其中,所述肺动脉骨架的像元集合是所述肺动脉的像元集合的子集;constructing a pixel set representing the pulmonary artery in the pulmonary artery image sequence; wherein the pixel set of the pulmonary artery skeleton is a subset of the pixel set of the pulmonary artery; 基于所述肺动脉图像序列和所述肺气管图像序列,提取所述肺动脉的像元集合中的像元的初级分类特征;其中,所述初级分类特征用于表征所述肺动脉骨架的像元集合中的像元。Based on the pulmonary artery image sequence and the pulmonary trachea image sequence, a primary classification feature of the pixels in the pixel set of the pulmonary artery is extracted; wherein, the primary classification feature is used to characterize the pixel set of the pulmonary artery skeleton. of pixels. 3.根据权利要求1所述的方法,其特征在于,所述方法还包括:3. The method according to claim 1, wherein the method further comprises: 将获取的肺部图像序列调整至指定分辨率,得到目标肺部图像序列;Adjust the acquired lung image sequence to the specified resolution to obtain the target lung image sequence; 基于所述目标肺部图像序列,确定所述肺部图像序列中表示肺血管的像元集合;其中,所述肺血管包括肺动脉血管和肺静脉血管;determining, based on the target lung image sequence, a set of pixels representing pulmonary blood vessels in the lung image sequence; wherein the pulmonary blood vessels include pulmonary artery blood vessels and pulmonary venous blood vessels; 对所述肺部图像序列中表示肺血管的像元集合中的像元进行分类,得到表示肺动脉血管的动脉像元集合;classifying the pixels in the pixel set representing the pulmonary blood vessels in the lung image sequence to obtain the arterial pixel set representing the pulmonary artery blood vessel; 基于所述动脉像元集合生成肺动脉图像序列。A sequence of pulmonary artery images is generated based on the set of arterial pixels. 4.根据权利要求3所述的方法,其特征在于,所述方法还包括:4. The method according to claim 3, wherein the method further comprises: 将初始肺部图像序列切分成第一肺部图像序列和第二肺部图像序列;其中,所述第一肺部图像序列表示左侧肺部区域;所述第二肺部图像序列表示右侧肺部区域;Divide the initial lung image sequence into a first lung image sequence and a second lung image sequence; wherein the first lung image sequence represents the left lung region; the second lung image sequence represents the right lung area; 相应的,在将获取的肺部图像序列调整至指定分辨率,得到目标肺部图像序列的步骤中,包括:Correspondingly, in the step of adjusting the acquired lung image sequence to a specified resolution to obtain the target lung image sequence, the steps include: 分别将所述第一肺部图像序列和所述第二肺部图像序列调整至指定分辨率,得到第一目标肺部图像序列和第二目标肺部图像序列;其中所述第一目标肺部图像序列和第二目标肺部图像序列属于目标肺部图像序列。respectively adjusting the first lung image sequence and the second lung image sequence to a specified resolution to obtain a first target lung image sequence and a second target lung image sequence; wherein the first target lung image sequence The image sequence and the second target lung image sequence belong to the target lung image sequence. 5.根据权利要求1所述的方法,其特征在于,使用所述初级分类特征和所述邻接矩阵确定所述肺动脉骨架的像元集合中像元属于的肺动脉类别的步骤,包括:5. The method according to claim 1, wherein the step of using the primary classification feature and the adjacency matrix to determine the pulmonary artery category to which a pixel in the pixel set of the pulmonary artery skeleton belongs, comprising: 使用所述初级分类特征和所述邻接矩阵提取所述肺动脉骨架的像元集合中的像元扩展分类特征;using the primary classification feature and the adjacency matrix to extract the pixel extension classification feature in the pixel set of the pulmonary artery skeleton; 基于所述扩展分类特征生成所述肺动脉骨架的像元集合中像元的肺动脉类别。A pulmonary artery category of a pixel in the pixel set of the pulmonary artery skeleton is generated based on the extended classification feature. 6.根据权利要求5所述的方法,其特征在于,所述方法还包括:6. The method according to claim 5, wherein the method further comprises: 基于所述肺动脉骨架的像元集合中像元的肺动脉类别确定所述肺动脉骨架的像元集合中表示肺动脉骨架分支的多个分支像元集合;其中,所述肺动脉骨架分支与所述肺动脉骨架主干相连;所述分支像元集合是所述肺动脉骨架的像元集合的子集;A plurality of branch pixel sets representing the branches of the pulmonary artery skeleton in the pixel set of the pulmonary artery skeleton are determined based on the pulmonary artery category of the pixels in the pixel set of the pulmonary artery skeleton; wherein, the pulmonary artery skeleton branch and the pulmonary artery skeleton trunk connected; the branch pixel set is a subset of the pixel set of the pulmonary artery skeleton; 获取表示所述分支像元集合中像元的邻接关系的分支邻接矩阵;obtaining a branch adjacency matrix representing the adjacency relationship of the pixels in the branch pixel set; 使用所述扩展分类特征和所述分支邻接矩阵确定所述分支像元集合中像元的肺动脉类别。A pulmonary artery class of a pixel in the set of branch pixels is determined using the extended classification feature and the branch adjacency matrix. 7.根据权利要求6所述的方法,其特征在于,使用所述扩展分类特征和所述分支邻接矩阵确定所述分支像元集合中像元的肺动脉类别的步骤,包括:7. The method according to claim 6, wherein the step of using the extended classification feature and the branch adjacency matrix to determine the pulmonary artery category of the pixel in the branch pixel set comprises: 在所述分支像元集合中选取指定个数的像元作为参考像元集合;Selecting a specified number of pixels in the branch pixel set as a reference pixel set; 获取所述参考像元集合中像元的扩展分类特征;obtaining the extended classification features of the pixels in the reference pixel set; 基于所述扩展分类特征和所述分支邻接矩阵确定所述分支像元集合中的像元的肺动脉类别。Pulmonary artery classes of pixels in the set of branch pixels are determined based on the extended classification feature and the branch adjacency matrix. 8.根据权利要求1所述的方法,其特征在于,所述方法还包括:8. The method of claim 1, wherein the method further comprises: 将所述像元集合中的像元作为种子像元;using the pixels in the pixel set as seed pixels; 基于所述种子像元,采用预设的区域生长算法确定所述肺动脉图像序列中表示肺动脉的像元的肺动脉类别。Based on the seed pixels, a preset region growing algorithm is used to determine the pulmonary artery category of the pixels representing the pulmonary artery in the pulmonary artery image sequence. 9.根据权利要求8所述的方法,其特征在于,所述方法还包括:9. The method according to claim 8, wherein the method further comprises: 对所述肺动脉图像序列中表示肺动脉的像元的肺动脉类别进行染色处理;其中,不同的肺动脉类别对应有不同的颜色。The pulmonary artery category of the pixel representing the pulmonary artery in the pulmonary artery image sequence is dyed; wherein, different pulmonary artery categories correspond to different colors. 10.一种像元分类模型的训练方法,其特征在于,包括:10. A training method for a pixel classification model, comprising: 构建训练样本;其中,所述训练样本包括肺动脉图像序列和肺动脉图像序列中表示肺动脉的像元对应的类别标签;constructing a training sample; wherein, the training sample includes a pulmonary artery image sequence and a category label corresponding to a pixel representing a pulmonary artery in the pulmonary artery image sequence; 获取肺动脉图像序列中表示肺动脉骨架的像元集合,和表示所述肺动脉骨架的像元集合中像元之间的邻接关系的邻接矩阵;acquiring a pixel set representing the pulmonary artery skeleton in the pulmonary artery image sequence, and an adjacency matrix representing the adjacency relationship between pixels in the pixel set of the pulmonary artery skeleton; 提取所述肺动脉骨架的像元集合中像元的初级分类特征;其中,所述初级分类特征用于表征所述肺动脉骨架的像元集合中的像元;extracting primary classification features of pixels in the pixel set of the pulmonary artery skeleton; wherein, the primary classification features are used to characterize the pixels in the pixel set of the pulmonary artery skeleton; 基于所述初级分类特征和所述邻接矩阵确定的所述肺动脉骨架的像元集合中像元的预测肺动脉类别;The predicted pulmonary artery category of the pixels in the pixel set of the pulmonary artery skeleton determined based on the primary classification feature and the adjacency matrix; 根据所述预测肺动脉类别和所述表示肺动脉的像元对应的类别标签生成的损失更新所述像元分类模型。The pixel classification model is updated based on the loss generated by the predicted pulmonary artery class and the class label corresponding to the pixel representing the pulmonary artery. 11.根据权利要求10所述的方法,其特征在于,所述肺动脉图像序列还对应有肺动脉骨架主干标签和肺动脉骨架分支标签;其中,所述肺动脉骨架分支标签对应有分支邻接矩阵;所述方法还包括:The method according to claim 10, wherein the pulmonary artery image sequence further corresponds to a pulmonary artery skeleton trunk label and a pulmonary artery skeleton branch label; wherein, the pulmonary artery skeleton branch label corresponds to a branch adjacency matrix; the method Also includes: 在所述肺动脉骨架分支标签中选取指定个数的像元作为参考像元集合;Selecting a specified number of pixels in the pulmonary artery skeleton branch label as a reference pixel set; 获取所述参考像元集合中像元的扩展分类特征;其中,所述展分类特征用于表征所述参考像元集合中的像元;Obtaining extended classification features of pixels in the reference pixel set; wherein, the extended classification features are used to characterize pixels in the reference pixel set; 相应的,在基于所述初级分类特征和所述邻接矩阵确定的所述像元集合中像元的预测肺动脉类别的步骤中,还包括:Correspondingly, in the step of predicting the pulmonary artery type of the pixel in the pixel set determined based on the primary classification feature and the adjacency matrix, further comprising: 基于所述扩展特征和所述分支邻接矩阵确定所述像元集合中像元的预测肺动脉类别。A predicted pulmonary artery class for pixels in the set of pixels is determined based on the extended features and the branch adjacency matrix. 12.一种像元的分类装置,其特征在于,包括:12. A device for classifying pixels, comprising: 肺动脉骨架及邻接矩阵获取模块,用于获取肺动脉图像序列中表示肺动脉骨架的像元集合,和表示所述肺动脉骨架的像元集合中像元之间的邻接关系的邻接矩阵;a pulmonary artery skeleton and an adjacency matrix acquisition module, used for acquiring a pixel set representing the pulmonary artery skeleton in the pulmonary artery image sequence, and an adjacency matrix representing the adjacency relationship between the pixels in the pixel set of the pulmonary artery skeleton; 初级分类特征提取模块,用于提取所述肺动脉骨架的像元集合中像元的初级分类特征;其中,所述初级分类特征用于表征所述肺动脉骨架的像元集合中的像元;a primary classification feature extraction module, configured to extract primary classification features of pixels in the pixel set of the pulmonary artery skeleton; wherein the primary classification features are used to characterize the pixels in the pixel set of the pulmonary artery skeleton; 肺动脉骨架分类模块,用于使用所述初级分类特征和所述邻接矩阵确定所述肺动脉骨架的像元集合中像元属于的肺动脉类别。A pulmonary artery skeleton classification module, configured to use the primary classification feature and the adjacency matrix to determine a pulmonary artery category to which a pixel in the pixel set of the pulmonary artery skeleton belongs. 13.一种像元分类模型的训练装置,其特征在于,包括:13. A training device for a pixel classification model, comprising: 训练样本构建模块,用于构建训练样本;其中,所述训练样本包括肺动脉图像序列和肺动脉图像序列中表示肺动脉的像元对应的类别标签;A training sample building module for constructing a training sample; wherein, the training sample includes a pulmonary artery image sequence and a category label corresponding to a pixel representing a pulmonary artery in the pulmonary artery image sequence; 肺动脉骨架及邻接矩阵获取模块,用于获取肺动脉图像序列中表示肺动脉骨架的像元集合,和表示所述肺动脉骨架的像元集合中像元之间的邻接关系的邻接矩阵;a pulmonary artery skeleton and an adjacency matrix acquisition module, used for acquiring a pixel set representing the pulmonary artery skeleton in the pulmonary artery image sequence, and an adjacency matrix representing the adjacency relationship between the pixels in the pixel set of the pulmonary artery skeleton; 初级分类特征提取模块,用于提取所述肺动脉骨架的像元集合中像元的初级分类特征;其中,所述初级分类特征用于表征所述肺动脉骨架的像元集合中的像元;a primary classification feature extraction module, configured to extract primary classification features of pixels in the pixel set of the pulmonary artery skeleton; wherein the primary classification features are used to characterize the pixels in the pixel set of the pulmonary artery skeleton; 肺动脉类别预测模块,用于基于所述初级分类特征和所述邻接矩阵确定的所述肺动脉骨架的像元集合中像元的预测肺动脉类别;a pulmonary artery category prediction module, configured to predict the pulmonary artery category of the pixels in the pixel set of the pulmonary artery skeleton determined based on the primary classification feature and the adjacency matrix; 像元分类模型生成模块,用于根据所述预测肺动脉类别和所述表示肺动脉的像元对应的类别标签生成的损失更新所述像元分类模型。A pixel classification model generation module, configured to update the pixel classification model according to the predicted pulmonary artery category and the loss generated by the class label corresponding to the pixel representing the pulmonary artery. 14.一种电子设备,其特征在于,所述电子设备包括:14. An electronic device, characterized in that the electronic device comprises: 处理器;processor; 用于存储所述处理器可执行指令的存储器;a memory for storing the processor-executable instructions; 所述处理器,用于执行上述权利要求1至11中任一项所述的方法。The processor is configured to perform the method of any one of the above claims 1 to 11. 15.一种计算机可读存储介质,所述存储介质存储有计算机程序,所述计算机程序用于执行上述权利要求1至11中任一项所述的方法。15. A computer-readable storage medium storing a computer program for performing the method of any one of the preceding claims 1 to 11.
CN202210545928.2A 2022-05-19 2022-05-19 Pixel classification method, model training method, device, equipment and medium Active CN114972859B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210545928.2A CN114972859B (en) 2022-05-19 2022-05-19 Pixel classification method, model training method, device, equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210545928.2A CN114972859B (en) 2022-05-19 2022-05-19 Pixel classification method, model training method, device, equipment and medium

Publications (2)

Publication Number Publication Date
CN114972859A true CN114972859A (en) 2022-08-30
CN114972859B CN114972859B (en) 2024-10-29

Family

ID=82985862

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210545928.2A Active CN114972859B (en) 2022-05-19 2022-05-19 Pixel classification method, model training method, device, equipment and medium

Country Status (1)

Country Link
CN (1) CN114972859B (en)

Citations (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103235929A (en) * 2013-03-11 2013-08-07 北京航空航天大学 Identification method and identification device on basis of hand vein images
CN108961273A (en) * 2018-07-03 2018-12-07 东北大学 A kind of method and system for dividing pulmonary artery and pulmonary vein from CT images
CN110338830A (en) * 2019-07-30 2019-10-18 赛诺威盛科技(北京)有限公司 A method for automatically extracting the central path of head and neck vessels in CTA images
CN110349175A (en) * 2019-06-25 2019-10-18 深圳先进技术研究院 A kind of arteriovenous malformation dividing method, system and electronic equipment
CN111028248A (en) * 2019-12-19 2020-04-17 杭州健培科技有限公司 Method and device for separating static and dynamic pulses based on CT (computed tomography) image
CN111612743A (en) * 2020-04-24 2020-09-01 杭州电子科技大学 A method of coronary centerline extraction based on CT images
CN111932554A (en) * 2020-07-31 2020-11-13 青岛海信医疗设备股份有限公司 Pulmonary blood vessel segmentation method, device and storage medium
CN111950408A (en) * 2020-07-28 2020-11-17 深圳职业技术学院 Finger vein image recognition method, device and storage medium based on rule map
CN112017167A (en) * 2020-08-24 2020-12-01 杭州深睿博联科技有限公司 Coronary artery central line generation method and device based on bidirectional coronary artery blood vessel tracking
US20200394789A1 (en) * 2019-06-12 2020-12-17 Carl Zeiss Meditec Inc Oct-based retinal artery/vein classification
US20210142470A1 (en) * 2019-11-12 2021-05-13 International Intelligent Informatics Solution Laboratory LLC System and method for identification of pulmonary arteries and veins depicted on chest ct scans
CN112861961A (en) * 2021-02-03 2021-05-28 推想医疗科技股份有限公司 Pulmonary blood vessel classification method and device, storage medium and electronic equipment
CN113011509A (en) * 2021-03-25 2021-06-22 推想医疗科技股份有限公司 Lung bronchus classification method and device, electronic equipment and storage medium
CN113256670A (en) * 2021-05-24 2021-08-13 推想医疗科技股份有限公司 Image processing method and device, and network model training method and device
CN113409328A (en) * 2021-06-02 2021-09-17 东北大学 Pulmonary artery and vein segmentation method, device, medium and equipment of CT image
CN113538415A (en) * 2021-08-16 2021-10-22 深圳市旭东数字医学影像技术有限公司 Segmentation method and device for pulmonary blood vessels in medical image and electronic equipment
CN114298999A (en) * 2021-12-24 2022-04-08 上海联影智能医疗科技有限公司 Method for detecting vascular structure variation, readable storage medium, and program product
CN114332013A (en) * 2021-12-29 2022-04-12 福州大学 Identification of target lung segments in CT images based on pulmonary artery tree grading

Patent Citations (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103235929A (en) * 2013-03-11 2013-08-07 北京航空航天大学 Identification method and identification device on basis of hand vein images
CN108961273A (en) * 2018-07-03 2018-12-07 东北大学 A kind of method and system for dividing pulmonary artery and pulmonary vein from CT images
US20200394789A1 (en) * 2019-06-12 2020-12-17 Carl Zeiss Meditec Inc Oct-based retinal artery/vein classification
CN110349175A (en) * 2019-06-25 2019-10-18 深圳先进技术研究院 A kind of arteriovenous malformation dividing method, system and electronic equipment
CN110338830A (en) * 2019-07-30 2019-10-18 赛诺威盛科技(北京)有限公司 A method for automatically extracting the central path of head and neck vessels in CTA images
US20210142470A1 (en) * 2019-11-12 2021-05-13 International Intelligent Informatics Solution Laboratory LLC System and method for identification of pulmonary arteries and veins depicted on chest ct scans
CN111028248A (en) * 2019-12-19 2020-04-17 杭州健培科技有限公司 Method and device for separating static and dynamic pulses based on CT (computed tomography) image
CN111612743A (en) * 2020-04-24 2020-09-01 杭州电子科技大学 A method of coronary centerline extraction based on CT images
CN111950408A (en) * 2020-07-28 2020-11-17 深圳职业技术学院 Finger vein image recognition method, device and storage medium based on rule map
CN111932554A (en) * 2020-07-31 2020-11-13 青岛海信医疗设备股份有限公司 Pulmonary blood vessel segmentation method, device and storage medium
CN112017167A (en) * 2020-08-24 2020-12-01 杭州深睿博联科技有限公司 Coronary artery central line generation method and device based on bidirectional coronary artery blood vessel tracking
CN112861961A (en) * 2021-02-03 2021-05-28 推想医疗科技股份有限公司 Pulmonary blood vessel classification method and device, storage medium and electronic equipment
CN113011509A (en) * 2021-03-25 2021-06-22 推想医疗科技股份有限公司 Lung bronchus classification method and device, electronic equipment and storage medium
CN113256670A (en) * 2021-05-24 2021-08-13 推想医疗科技股份有限公司 Image processing method and device, and network model training method and device
CN113409328A (en) * 2021-06-02 2021-09-17 东北大学 Pulmonary artery and vein segmentation method, device, medium and equipment of CT image
CN113538415A (en) * 2021-08-16 2021-10-22 深圳市旭东数字医学影像技术有限公司 Segmentation method and device for pulmonary blood vessels in medical image and electronic equipment
CN114298999A (en) * 2021-12-24 2022-04-08 上海联影智能医疗科技有限公司 Method for detecting vascular structure variation, readable storage medium, and program product
CN114332013A (en) * 2021-12-29 2022-04-12 福州大学 Identification of target lung segments in CT images based on pulmonary artery tree grading

Also Published As

Publication number Publication date
CN114972859B (en) 2024-10-29

Similar Documents

Publication Publication Date Title
US11880972B2 (en) Tissue nodule detection and tissue nodule detection model training method, apparatus, device, and system
CN111899245B (en) Image segmentation method, image segmentation device, model training method, model training device, electronic equipment and storage medium
CN108198184B (en) Method and system for segmentation of blood vessels in angiography images
US11972571B2 (en) Method for image segmentation, method for training image segmentation model
CN110070540B (en) Image generation method and device, computer equipment and storage medium
US20220028085A1 (en) Method and system for providing an at least 3-dimensional medical image segmentation of a structure of an internal organ
Singh et al. Deep-learning based system for effective and automatic blood vessel segmentation from Retinal fundus images
US9824492B2 (en) Hollow object model visualization in medical images
CN111368849B (en) Image processing method, image processing device, electronic equipment and storage medium
WO2024187937A1 (en) Blood vessel reconstruction method and apparatus, computer device, and readable storage medium
CN111899244B (en) Image segmentation method, network model training method, device and electronic equipment
CN107563434A (en) A kind of brain MRI image sorting technique based on Three dimensional convolution neutral net, device
CN109785399B (en) Synthetic lesion image generation method, device, equipment and readable storage medium
Marhamati et al. LAIU-Net: A learning-to-augment incorporated robust U-Net for depressed humans’ tongue segmentation
CN113744215B (en) Extraction method and device for central line of tree-shaped lumen structure in three-dimensional tomographic image
CN110008992A (en) A kind of deep learning method for prostate cancer auxiliary diagnosis
CN111524109A (en) Head medical image scoring method and device, electronic equipment and storage medium
CN111583385A (en) A personalized deformation method and system for a deformable digital human anatomy model
CN114972211B (en) Training method, segmentation method, device, equipment and medium for image segmentation model
CN113256670A (en) Image processing method and device, and network model training method and device
CN111127487B (en) A Real-time Multi-Tissue Medical Image Segmentation Method
CN113361543A (en) CT image feature extraction method and device, electronic equipment and storage medium
CN114972859B (en) Pixel classification method, model training method, device, equipment and medium
Luo et al. Research on several key problems of medical image segmentation and virtual surgery
CN115272165B (en) Image feature extraction method, image segmentation model training method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant