CN114972859A - Pixel classification method, model training method, device, equipment and medium - Google Patents
Pixel classification method, model training method, device, equipment and medium Download PDFInfo
- Publication number
- CN114972859A CN114972859A CN202210545928.2A CN202210545928A CN114972859A CN 114972859 A CN114972859 A CN 114972859A CN 202210545928 A CN202210545928 A CN 202210545928A CN 114972859 A CN114972859 A CN 114972859A
- Authority
- CN
- China
- Prior art keywords
- pulmonary artery
- pixels
- pixel
- skeleton
- image sequence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 73
- 238000012549 training Methods 0.000 title claims abstract description 36
- 210000001147 pulmonary artery Anatomy 0.000 claims abstract description 373
- 239000011159 matrix material Substances 0.000 claims abstract description 72
- 238000013145 classification model Methods 0.000 claims abstract description 36
- 210000004072 lung Anatomy 0.000 claims description 105
- 230000002685 pulmonary effect Effects 0.000 claims description 30
- 238000000605 extraction Methods 0.000 claims description 13
- 210000003437 trachea Anatomy 0.000 claims description 12
- 239000003086 colorant Substances 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 6
- 210000001367 artery Anatomy 0.000 claims description 5
- 238000010276 construction Methods 0.000 claims description 4
- 238000004043 dyeing Methods 0.000 claims description 2
- 210000003492 pulmonary vein Anatomy 0.000 claims description 2
- 238000002591 computed tomography Methods 0.000 description 14
- 238000010586 diagram Methods 0.000 description 13
- 230000008569 process Effects 0.000 description 11
- 238000002059 diagnostic imaging Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000013528 artificial neural network Methods 0.000 description 5
- 210000004204 blood vessel Anatomy 0.000 description 5
- 210000003484 anatomy Anatomy 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 239000000284 extract Substances 0.000 description 4
- 238000010186 staining Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000003709 image segmentation Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 210000000621 bronchi Anatomy 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000001356 surgical procedure Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000011426 transformation method Methods 0.000 description 1
- 238000002604 ultrasonography Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Molecular Biology (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the specification provides a pixel classification method, a model training method, a device, equipment and a medium. The method comprises the following steps: acquiring a pixel set representing a pulmonary artery skeleton in a pulmonary artery image sequence and an adjacency matrix representing an adjacency relation between pixels in the pixel set of the pulmonary artery skeleton; extracting primary classification features of the pixels in the pixel set; wherein the primary classification feature is used to characterize the pels in the set of pels; and determining the pulmonary artery category to which the picture elements belong in the picture element set of the pulmonary artery skeleton by using the primary classification characteristic and the adjacency matrix. The primary classification characteristic of the pulmonary artery skeleton pixel is extracted, and the characteristic of the pixel in the pixel set is extracted by fusing an image attention network which is used for constructing an adjacent matrix of the pixel in the pixel set expressing the pulmonary artery skeleton and is used as a pixel classification model, so that the classification accuracy of the pixel expressing the pulmonary artery in a pulmonary artery image sequence is improved.
Description
Technical Field
The embodiment of the specification relates to the field of image processing, in particular to a classification method of pixels, a model training method, a device, equipment and a medium.
Background
As the role of medical imaging examination in assisting medical diagnosis and treatment gradually increases, medical imaging examination of the lung helps doctors to understand the condition of the lung of a patient. Classifying the pixels representing the pulmonary artery in the pulmonary medical image helps a physician to understand the anatomy of the patient's pulmonary artery when performing surgery on the patient. The existing pulmonary artery classification method mainly inputs the pulmonary artery image extracted based on the lung medical image into a neural network for classification, so that the pulmonary artery classification is easy to be wrong.
Disclosure of Invention
In view of the above, embodiments of the present disclosure are directed to providing a method, an apparatus, a device and a medium for classifying picture elements, so as to provide a method that can improve the accuracy of classification of picture elements representing pulmonary arteries in a pulmonary artery image sequence to some extent.
One embodiment of the present specification provides a method for classifying pixels, including: acquiring a pixel set representing a pulmonary artery skeleton in a pulmonary artery image sequence and an adjacency matrix representing an adjacency relation between pixels in the pixel set of the pulmonary artery skeleton; extracting primary classification characteristics of pixels in a pixel set of the pulmonary artery skeleton; wherein the primary classification feature is used to characterize pixels in a set of pixels of the pulmonary artery skeleton; and determining the pulmonary artery category to which the picture elements belong in the picture element set of the pulmonary artery skeleton by using the primary classification characteristic and the adjacency matrix.
One embodiment of the present specification provides a method for training a pixel classification model, including: constructing a training sample; the training sample comprises a pulmonary artery image sequence and a category label corresponding to a pixel representing a pulmonary artery in the pulmonary artery image sequence; acquiring a pixel set representing a pulmonary artery skeleton in a pulmonary artery image sequence and an adjacency matrix representing an adjacency relation between pixels in the pixel set of the pulmonary artery skeleton; extracting primary classification characteristics of pixels in a pixel set of the pulmonary artery skeleton; wherein the primary classification feature is used to characterize pixels in a set of pixels of the pulmonary artery skeleton; a predicted pulmonary artery category of picture elements in a set of picture elements of the pulmonary artery skeleton determined based on the primary classification feature and the adjacency matrix; and updating the pixel classification model according to the loss generated by the category label corresponding to the predicted pulmonary artery category and the pixel representing the pulmonary artery.
One embodiment of the present specification provides a classification device of a pixel, including: the pulmonary artery skeleton and adjacency matrix acquisition module is used for acquiring a pixel set representing the pulmonary artery skeleton in a pulmonary artery image sequence and an adjacency matrix representing the adjacency relation between pixels in the pixel set of the pulmonary artery skeleton; the primary classification feature extraction module is used for extracting primary classification features of pixels in a pixel set of the pulmonary artery skeleton; wherein the primary classification feature is used to characterize pixels in a set of pixels of the pulmonary artery skeleton; and the pulmonary artery skeleton classification module is used for determining the pulmonary artery category to which the image elements belong in the image element set of the pulmonary artery skeleton by using the primary classification characteristic and the adjacency matrix.
This description is that an embodiment proposes a training apparatus for a pixel classification model, including: the training sample construction module is used for constructing a training sample; the training sample comprises a pulmonary artery image sequence and a category label corresponding to a pixel representing a pulmonary artery in the pulmonary artery image sequence; the pulmonary artery skeleton and adjacency matrix acquisition module is used for acquiring a pixel set representing the pulmonary artery skeleton in a pulmonary artery image sequence and an adjacency matrix representing the adjacency relation between pixels in the pixel set of the pulmonary artery skeleton; the primary classification feature extraction module is used for extracting primary classification features of pixels in a pixel set of the pulmonary artery skeleton; wherein the primary classification feature is used to characterize pixels in a set of pixels of the pulmonary artery skeleton; the pulmonary artery category prediction module is used for predicting a pulmonary artery category of pixels in a pixel set of the pulmonary artery skeleton based on the primary classification characteristic and the adjacency matrix; and the pixel classification model generation module is used for updating the pixel classification model according to the predicted pulmonary artery category and the loss generated by the category label corresponding to the pixel representing the pulmonary artery.
One embodiment of the present specification provides an electronic device including: a processor; a memory for storing the processor-executable instructions; the processor is configured to execute the method described in the foregoing embodiment.
One embodiment of the present specification proposes a computer-readable storage medium comprising: the storage medium stores a computer program for executing the method according to the above embodiment.
According to the multiple embodiments of the description, the pixel set representing the pulmonary artery skeleton in the pulmonary artery image sequence is extracted, the adjacency matrix is constructed according to the adjacency relation among the pixels in the pixel set, and then the pulmonary artery category of the pixels in the pixel set is determined based on the primary pixel characteristics of the pixels representing the pulmonary artery skeleton extracted by the initial pulmonary artery classification model of the pulmonary artery image and the adjacency matrix, so that the classification accuracy of the pixels representing the pulmonary artery skeleton is improved to a certain extent, the accuracy of the sectional staining of the pixels representing the pulmonary artery is further improved, and a doctor can conveniently check the anatomical structure of the pulmonary artery of a patient.
Drawings
Fig. 1 is a schematic diagram illustrating interaction of a pulmonary artery classification system in one example scenario provided by an embodiment.
Fig. 2 is a schematic diagram illustrating a result of segmenting and staining a pulmonary artery according to an exemplary scenario provided by an embodiment.
Fig. 3 is a schematic diagram illustrating interaction of a pulmonary artery classification system in one example scenario provided by an embodiment.
Fig. 4 is a schematic flow chart of a method for classifying pixels according to an embodiment.
Fig. 5 is a schematic diagram illustrating a classification result of a pulmonary artery skeleton according to an embodiment.
FIG. 6 is a diagram illustrating the classification result of the pulmonary artery skeleton according to an embodiment.
FIG. 7 is a flowchart illustrating a method for training a pixel classification model according to an embodiment.
Fig. 8 is a schematic diagram of a sorting apparatus for pixels according to an embodiment.
FIG. 9 is a schematic diagram of a training apparatus for a pixel classification model according to an embodiment.
Fig. 10 is a schematic diagram of an electronic device according to an embodiment.
Detailed Description
In order to make the technical solutions in the present specification better understood, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is obvious that the described embodiments are only a part of the embodiments of the present specification, but not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art without any inventive work based on the embodiments in the present specification belong to the protection scope of the present specification.
Referring to fig. 1 and 2, an exemplary scenario of a pulmonary artery classification system is provided in the embodiments of the present disclosure. When a patient A is in a visit, a doctor can make a lung CT image examination order for the patient A, the patient A can obtain a lung CT image sequence after the lung CT image examination, and then the lung CT image sequence is sent to a client. The client may send the lung CT image sequence to the server. The server first slices the lung CT image sequence into a first lung image sequence representing the left lung and a second lung image sequence representing the right lung. The server may then process the first and second lung image sequences by upsampling and/or downsampling into a first and second target lung image sequence with a data volume of 192 × 128 × 96, respectively. The server respectively preprocesses the first lung image sequence and the second lung image sequence to obtain a lung blood vessel image sequence representing the left lung, a lung blood vessel image sequence representing the right lung, a lung trachea image sequence representing the left lung and a lung trachea image sequence representing the right lung. And splicing the pulmonary vessel image sequence representing the left lung and the pulmonary vessel image sequence representing the right lung to obtain a pulmonary vessel image sequence. And splicing the lung trachea image sequence representing the left lung and the lung trachea image sequence representing the right lung to obtain the lung trachea image sequence. Then, the server may perform a pre-segmentation process on the pulmonary blood vessel image sequence to obtain a pulmonary artery image sequence representing a pulmonary artery.
After the pulmonary artery image sequence is acquired, the server can input the pulmonary artery image sequence and the pulmonary trachea image sequence into a ResU-net model to extract initial pixel features of pixels representing pulmonary arteries in the pulmonary artery image sequence. The server can also extract first image elements representing the pulmonary artery skeleton in the pulmonary artery image sequence by using the skeletonize algorithm and construct a first adjacency matrix representing adjacency relations between the first image elements. Then, the server can take the first adjacency matrix as an attention mechanism of the ResU-net model, extract an expansion feature corresponding to the first pixel based on the initial pixel feature, and generate the pulmonary artery category corresponding to the first pixel according to the expansion feature.
After the pulmonary artery category corresponding to the first pixel element is generated, the server may generate a second pixel element representing a pulmonary artery skeleton branch based on the classification result of the first pixel element, and construct a second adjacency matrix representing an adjacency relationship between the second pixel elements. Then, the server can take the second adjacency matrix as an attention mechanism of the ResU-net model, extract branch expansion characteristics of the second pixel object based on the expansion characteristics, and generate a pulmonary artery category corresponding to the second pixel according to the branch expansion characteristics. And when the pulmonary artery category corresponding to the second image element is inconsistent with the pulmonary artery corresponding to the first image element, replacing the pulmonary artery category corresponding to the first image element with the pulmonary artery category corresponding to the second image element.
And finally, determining the pulmonary artery category corresponding to the pixel representing the pulmonary artery by using the first pixel as a seed point and adopting a preset region growing algorithm, and endowing different colors to the pixel representing the pulmonary artery according to preset colors. For example, the pulmonary artery trunk is colored red, the right superior pulmonary lobe tip section is colored blue, the posterior end of the right superior pulmonary lobe is colored yellow, and the posterior left pulmonary lobe section is colored orange. Finally, the server can send the pulmonary artery image sequence after the segmented staining to the client side, so that a doctor can check the detailed anatomical structure of the pulmonary artery, and the lung lesion degree of the patient can be known.
The above description is only exemplary of the present disclosure and should not be construed as limiting the present disclosure, and any modifications, equivalents and the like that are within the spirit and principle of the present disclosure are intended to be included within the scope of the present disclosure.
Referring to fig. 3, a pulmonary artery classification system is provided in accordance with an embodiment of the present disclosure. And the classification method of the pulmonary artery image and/or the training method of the classification model of the pulmonary artery image provided by the present specification can be applied to the pulmonary artery classification system. The pulmonary artery classification system may include a hardware environment formed by the medical imaging device 110, the client 120, and the server 130. The medical imaging device 110 is connected to the client 120, and the server 130 is connected to the client 120 via a communication network. The communication network may be a wired network or a wireless network. The medical imaging device 110 examines and images the lungs resulting in a sequence of images of the lungs. The sequence of lung images is transmitted to the client 120 by the communicating medical imaging device 110. The client 120 sends the sequence of lung images to the server 130 and the server 130 receives the sequence of lung images. The medical imaging device 110 may be, but is not limited to, at least one of an ultrasound medical device, a CT medical examination device, and an MRI medical examination device, among others. Client 120 may be an electronic device with network access capabilities. Specifically, for example, the client may be a desktop computer, a tablet computer, a notebook computer, a smart phone, a digital assistant, a smart wearable device, a shopping guide terminal, a television, a smart speaker, a microphone, and the like. Wherein, wearable equipment of intelligence includes but not limited to intelligent bracelet, intelligent wrist-watch, intelligent glasses, intelligent helmet, intelligent necklace etc.. Alternatively, the client may be software capable of running in the electronic device. Those skilled in the art will appreciate that the number of clients 120 may be one or more, and the types may be the same or different. For example, the number of the clients 120 may be one, or the number of the clients 120 may be several tens or hundreds, or more. The number and the device type of the clients 120 are not limited in the embodiment of the present application. The server 130 may be an electronic device having a certain arithmetic processing capability. Which may have a network communication module, a processor, memory, etc. Of course, the server may also refer to software running in the electronic device. The server may also be a distributed server, which may be a system with multiple processors, memory, network communication modules, etc. operating in coordination. Alternatively, the server may also be a server cluster formed by several servers. Or, with the development of scientific technology, the server can also be a new technical means capable of realizing the corresponding functions of the specification implementation mode. For example, it may be a new form of "server" based on quantum computing implementations.
Referring to fig. 4 and 5, the present specification provides a method for classifying pixels. The method for classifying picture elements can be applied to electronic equipment. The method of classifying picture elements may comprise the following steps.
Step S210: acquiring a pixel set representing a pulmonary artery skeleton in a pulmonary artery image sequence and an adjacency matrix representing an adjacency relation between pixels in the pixel set of the pulmonary artery skeleton.
In some cases, the image elements representing the pulmonary artery on the same pulmonary artery branch have the same image characteristics, which may lead to classification errors if the classification of the image element class representing the pulmonary artery is directly performed. Therefore, the adjacency relationship between the picture element representing the pulmonary artery skeleton and the picture element representing the pulmonary artery skeleton may be extracted first.
The pulmonary artery image sequence is an image sequence representing a pulmonary artery extracted based on a pulmonary medical image. Specifically, after a patient performs a lung CT (Computed Tomography), a lung CT medical image sequence can be generated. By segmenting the pulmonary CT medical image sequence, the pixel representing the pulmonary artery in the pulmonary CT medical image sequence is assigned to 1, and the pixel representing the non-pulmonary artery is assigned to 0, so that a three-dimensional pulmonary artery image sequence is generated.
The pulmonary artery skeleton is refined into the width of one pixel based on a connected region of pixels representing pulmonary arteries in a pulmonary artery image sequence, and is used for feature extraction and topological representation of the pixels in a pixel set representing the pulmonary arteries. Specifically, for example, a binarized pulmonary artery image sequence is used as an input of the skeleton extraction model, and a pulmonary artery skeleton image sequence including a width refined by using a skeletonitze () function in a morphology module is obtained. The pulmonary artery skeleton image sequence is also a binary image sequence.
The adjacency matrix is used for representing adjacency relation among the image elements in the image element set. Specifically, for example, 3000 pixels are included in the pixel set, a 3000 × 3000 two-dimensional matrix may be constructed, and two identical pixels have no adjacency, so that the pixels on the diagonal of the two-dimensional matrix are denoted by "0"; for two non-identical picture elements, the correlation of the two picture elements is represented by a "1" in the two-dimensional matrix if they are adjacent in a 26-neighborhood over the spatial range, and by a "0" in the two-dimensional matrix if they are not adjacent in the 26-neighborhood.
Step S220: extracting primary classification characteristics of pixels in a pixel set of the pulmonary artery skeleton; wherein the primary classification feature is used to characterize a picture element in the set of picture elements of the pulmonary artery skeleton.
In some cases, in the process of classifying the image elements in the image, the features of the image elements in the pulmonary artery image are firstly extracted, and then classification is performed based on the features of the image elements in the image. Therefore, the features of the image elements in the image can be extracted by using a conventional image classification method.
The primary classification features are extracted by an encoder based on a primary pixel classification model. Specifically, for example, an encoder using the ResU-net model extracts initial pixel features representing pulmonary artery pixels in a sequence of pulmonary artery images. The pulmonary artery image sequence can be firstly sliced in three directions to obtain two-dimensional data in 3 view directions (top view, front view and side view), then 2-dimensional data in the 3 view directions are respectively input into a ResU-net network model, and the extracted features in the 3 directions are fused to obtain primary classification features representing pulmonary artery pixels in the pulmonary artery image sequence. And then extracting the characteristics of the pixels representing the pulmonary artery skeleton in the primary classification characteristics of the pulmonary artery pixels as the primary classification characteristics of the pixels in the pixel set.
Step S230: and determining the pulmonary artery category to which the picture elements belong in the picture element set of the pulmonary artery skeleton by using the primary classification characteristic and the adjacency matrix.
In some cases, a classification error may occur if the picture elements of the pulmonary artery skeleton are classified based on primary classification features only. Therefore, the adjacency matrix can be used as a graph attention network of the image element classification model, then the expansion features of the image elements in the image element set are extracted based on the graph attention network and the primary classification features, and then the pulmonary artery classes of the image elements in the image element set are determined by using the expansion features.
The method for determining the pulmonary artery category of the pixels in the pixel set by using the primary classification feature and the adjacency matrix can be to use the primary pixel feature as the feature of the pixels in the pixel set, then further describe the importance of the adjacent pixels to the classification of the pixels by using a Graph Attention network (GATs), and use the adjacency matrix as a mask of the Graph Attention network, so as to further enrich the features of the pixels representing the skeleton in the pulmonary artery image sequence. However, the above is only an exemplary feature extraction method for a pixel in an image in the present specification, and the feature extraction method for an image is not limited in the embodiments of the present specification.
The primary classification characteristic of the pulmonary artery skeleton pixel is extracted, and the characteristic of the pixel in the pixel set is extracted by fusing an image attention network which is used for constructing an adjacent matrix of the pixel in the pixel set expressing the pulmonary artery skeleton and is used as a pixel classification model, so that the classification accuracy of the pixel expressing the pulmonary artery in a pulmonary artery image sequence is improved.
In some embodiments, the step of extracting the primary classification features of the image elements in the set of image elements of the pulmonary artery skeleton may include: acquiring a pulmonary trachea image sequence corresponding to the pulmonary artery image sequence; wherein the pulmonary trachea image sequence and the pulmonary artery image sequence are obtained based on the same pulmonary image sequence; constructing a pixel set representing the pulmonary artery in a pulmonary artery image sequence; wherein the set of picture elements of the pulmonary artery skeleton is a subset of the set of picture elements of the pulmonary artery; extracting primary classification features of pixels in a pixel set of the pulmonary artery based on the pulmonary artery image sequence and the pulmonary trachea image sequence; wherein the primary classification feature is used to characterize a picture element in the set of picture elements of the pulmonary artery skeleton.
In some cases, there is a concomitant relationship between pulmonary arterial blood vessels and the bronchi. Therefore, in the process of classifying the pulmonary artery blood vessels, the distribution information of the pulmonary artery blood vessels can be used as reference, so that the classification information of the pulmonary artery blood vessels can be determined more accurately and rapidly, and the accuracy of classifying the pulmonary artery blood vessels is improved.
In some embodiments, the method for classifying picture elements may further include: adjusting the acquired lung image sequence to a specified resolution to obtain a target lung image sequence; determining a set of pixels representing a pulmonary vessel in the sequence of lung images based on the sequence of target lung images; wherein the pulmonary vessels include pulmonary artery vessels and pulmonary vein vessels; classifying pixels in a pixel set representing pulmonary vessels in the pulmonary image sequence to obtain an artery pixel set representing pulmonary artery vessels; and generating a pulmonary artery image sequence based on the artery pixel element set.
In some cases, before the process of classifying the pixels representing the pulmonary artery in the pulmonary artery image sequence, the pixels representing the pulmonary artery in the pulmonary artery image sequence need to be acquired, and the pulmonary artery image sequence is formed according to the pixels representing the pulmonary artery. Before the lung image sequence is input into the pulmonary artery segmentation model, the lung image sequence is required to be adjusted to a specified resolution in order to facilitate the operation of the model. Specifically, for example, according to the average ratio of the isotropy of the pixels representing the pulmonary artery in the pulmonary CT image of the patient after the isotropy of the specific range in the pulmonary CT image is 1.84:1.28:1, then the input of the model is changed to [ 192, 128, 96 ]. When the size of the lung image sequence is larger than [ 192, 128, 96 ], down-sampling the lung image sequence to [ 192, 128, 96 ] through a convolution kernel to obtain a target lung image sequence; when the size of the lung image sequence is smaller than [ 192, 128, 96 ], the lung image sequence is up-sampled to [ 192, 128, 96 ] through a bilinear interpolation method to obtain a target lung image sequence, so that the proportion of the images in the model is consistent. And then, inputting the target lung image sequence into a U-net image segmentation model to obtain a lung blood vessel image sequence. And finally, further extracting the pixel representing the pulmonary artery in the pulmonary vessel image sequence, thereby obtaining the pulmonary artery image sequence.
In some embodiments, the method for classifying picture elements may further include: segmenting the initial sequence of lung images into a first sequence of lung images and a second sequence of lung images; wherein the first sequence of lung images represents a left lung region; the second sequence of lung images represents a right lung region; correspondingly, the step of adjusting the acquired lung image sequence to a specified resolution to obtain the target lung image sequence may include: respectively adjusting the first lung image sequence and the second lung image sequence to specified resolutions to obtain a first target lung image sequence and a second target lung image sequence; wherein the first and second target lung image sequences belong to a target lung image sequence.
In some cases, when the entire lung image sequence is processed directly, it is easy to cause a classification error of the lung vessels in the left and right lungs, and therefore the lung image sequence can be sliced into a first lung image sequence representing the left lung and a second lung image sequence representing the right lung. On one hand, the resolution of the lung image sequence input into the image segmentation model is improved; on the other hand, the video memory of the model in the running process can be reduced. However, it should be noted that the first lung image sequence and the second lung image sequence described in the embodiments of the present disclosure are only used to represent the difference therebetween, and are not limited to the first lung image sequence representing the left lung and the second lung image sequence representing the right lung.
In some embodiments, the step of determining a pulmonary artery class to which a picture element belongs in a set of picture elements of the pulmonary artery skeleton using the primary classification feature and the adjacency matrix may include: extracting a pel-expanded classification feature in a set of pels of the pulmonary artery skeleton using the primary classification feature and the adjacency matrix; and generating the pulmonary artery category of the pixels in the pixel set of the pulmonary artery skeleton based on the expanded classification features.
In some cases, in order to determine the pulmonary artery category of a pixel in the pulmonary artery pixel set, a pixel feature corresponding to a pixel representing a pulmonary artery needs to be acquired first. Therefore, the pixel characteristics of the pulmonary artery pixels can be extracted based on a common image classification model, and then the expression of the pixel characteristics is enriched by combining the adjacency relation among the pixels in the pixel set of the pulmonary artery skeleton.
The extended classification features are derived based on the adjacency relationship between the primary classification features and the pixels in the adjacency matrix. Specifically, for example, the primary classification feature is first acquired by using the ResU-net network, and then the primary classification feature and the adjacency matrix are input to the feature extracted by the graph attention network model as the extended classification feature. Then, a pulmonary artery class of picture elements in a set of picture elements representing a pulmonary artery in the sequence of pulmonary artery image is determined based on the extended classification features.
Referring to fig. 6, in some embodiments, the method for classifying pixels may further include: determining a plurality of branch pixel sets representing pulmonary artery skeleton branches in the pixel set of the pulmonary artery skeleton based on the pulmonary artery category of the pixels in the pixel set of the pulmonary artery skeleton; wherein the pulmonary artery skeleton branch is connected with the pulmonary artery skeleton main trunk; the set of branch picture elements is a subset of the set of picture elements of the pulmonary artery skeleton; acquiring a branch adjacency matrix representing adjacency relation of pixels in the branch pixel set; determining a pulmonary artery class of pixels in the set of branch pixels using the expanded classification features and the branch adjacency matrix.
In some cases, the pulmonary artery picture elements that are on the same pulmonary artery branch have the same pulmonary artery characteristics. Thus, the picture elements on one branch can be taken as a whole to determine the pulmonary artery class of the entire pulmonary artery branch.
In some embodiments, the step of determining a pulmonary artery class of a pel in the set of branch pels using the expanded classification feature and the branch adjacency matrix may include: selecting pixels with a specified number from the branch pixel set as a reference pixel set; acquiring the expansion classification characteristics of the pixels in the reference pixel set; determining a pulmonary artery class of pixels in the set of branch pixels based on the expanded classification features and the branch adjacency matrix.
In some cases, the pulmonary artery picture elements on the same branch have the same pulmonary artery characteristics. Therefore, a specified number of pixels can be randomly selected from the pulmonary artery branches. Then, the pulmonary artery category of the image elements in the branch image element set is determined based on the expansion characteristics of the selected image elements and the branch adjacency matrix.
The method for determining the pulmonary artery category of the pixels in the branch pixel set by using the expanded classification characteristics and the branch adjacency matrix can be that a specified number of pixels are randomly selected on a skeleton branch, then the specified number of pixels are subjected to linear transformation to obtain the characteristics on the skeleton branch, and the pulmonary artery category of the pulmonary artery skeleton branch is determined based on the characteristics of the skeleton branch.
In some embodiments, the method for classifying picture elements may further include: taking the pixels in the pixel set as seed pixels; and determining the pulmonary artery category of the pixel representing the pulmonary artery in the pulmonary artery image sequence by adopting a preset region growing algorithm based on the seed pixel.
In some cases, it is also necessary to classify other image elements on the pulmonary artery. Therefore, the region growing can be performed based on the classification result of the pixel on the pulmonary artery skeleton branch, and then the classification result of the pixel on the pulmonary artery skeleton branch is given to the pixel performing the region growing based on the pulmonary artery skeleton branch, so that the classification result of the pixel representing the pulmonary artery in the pulmonary artery image sequence is obtained.
In some embodiments, the method for classifying picture elements may further include: carrying out dyeing processing on the pulmonary artery category of a pixel representing the pulmonary artery in the pulmonary artery image sequence; wherein different pulmonary artery categories correspond to different colors.
In some cases, in order to make the anatomical structure of the pulmonary artery more visible to the doctor, the pulmonary artery of different categories can be stained, so that the segmented staining result of the pulmonary artery can be obtained. Specifically, for example, the pulmonary artery category includes categories such as the pulmonary artery trunk, the tip of the right upper pulmonary lobe, the posterior segment of the right upper pulmonary lobe, the anterior segment of the right upper pulmonary lobe, the middle-field lateral segment of the right lung, and the upper segment of the right lower pulmonary lobe, and thus the pulmonary artery trunk may be given different colors such as red, the tip of the right upper pulmonary lobe blue, the posterior segment of the right upper pulmonary lobe green, the anterior segment of the right upper pulmonary lobe orange, the middle-field lateral segment of the right lung purple, and the upper segment of the right lower pulmonary lobe yellow. Wherein, different categories correspond to different colors, and the color difference of the pulmonary artery categories which are positioned close in space is obvious.
Referring to fig. 7, an embodiment of the present disclosure provides a method for training a pixel classification model. The training method of the pixel classification model can be applied to electronic equipment. The training method of the pixel classification model can comprise the following steps.
Step S310: constructing a training sample; the training sample comprises a pulmonary artery image sequence and a category label corresponding to a pixel representing a pulmonary artery in the pulmonary artery image sequence.
In some cases, how well the training sample constructs directly affect the accuracy of the model. Therefore, when labeling the pulmonary artery image sequence, a doctor with higher professional level and more professional experience needs to be selected for labeling. However, the embodiment of the present invention is not limited to the specific form of the training sample, and may be an original medical image, a preprocessed medical image, or a part of an original medical image.
Step S320: acquiring a pixel set representing a pulmonary artery skeleton in a pulmonary artery image sequence and an adjacency matrix representing an adjacency relation between pixels in the pixel set of the pulmonary artery skeleton.
Step S330: extracting primary classification characteristics of pixels in a pixel set of the pulmonary artery skeleton; wherein the primary classification feature is used to characterize a picture element in the set of picture elements of the pulmonary artery skeleton.
Step S340: a predicted pulmonary artery class of picture elements in the set of picture elements of the pulmonary artery skeleton determined based on the primary classification feature and the adjacency matrix.
Step S350: and updating the pixel classification model according to the loss generated by the category label corresponding to the predicted pulmonary artery category and the pixel representing the pulmonary artery.
The pixel classification model is used for generating a predicted pulmonary artery category of pixels in a pixel set representing a pulmonary artery skeleton in a training sample. And then calculating a loss function for predicting the pulmonary artery category and the category label corresponding to the pixel representing the pulmonary artery. And updating the pixel classification model according to the loss function, and taking the parameters of the updated model as the parameters of the pixel classification model under the condition of loss function convergence.
The primary classification features may be derived by an encoder of the initial classification model. The primary classification model may be a Convolutional Neural Network (CNN), a Deep Neural Network (DNN), a Recurrent Neural Network (RNN), or the like, and the specific type of the primary classification model is not limited in this embodiment. The preliminary classification model in this embodiment may include a neural network layer such as an input layer, a convolutional layer, a pooling layer, and a connection layer, which is not particularly limited in this embodiment. In the present embodiment, the number of each neural network layer is not limited.
In some embodiments, the sequence of pulmonary artery images further corresponds to a pulmonary artery skeleton trunk tag and a pulmonary artery skeleton branch tag; the pulmonary artery skeleton branch label corresponds to a branch adjacency matrix; the method may further comprise: selecting pixels with a specified number from the pulmonary artery skeleton branch label as a reference pixel set; acquiring the expansion classification characteristics of the pixels in the reference pixel set; wherein the expansion classification feature is used for characterizing the image elements in the reference image element set; correspondingly, in the step of predicting the pulmonary artery category of the image elements in the image element set determined based on the primary classification feature and the adjacency matrix, the method may further include: determining a predicted pulmonary artery class for a pixel in the set of pixels based on the expanded feature and the branch adjacency matrix.
In some cases, the picture elements on a certain branch have the same image characteristics. Therefore, a specified number of pixels can be randomly selected on the pulmonary artery branches for feature extraction, and then the pulmonary artery category of the pixels on the whole branch is obtained by a linear transformation method. The method for determining the type of the pulmonary artery in this embodiment is the same as that in the above embodiment, and details are not repeated herein, please refer to the above embodiment. However, in the present embodiment, the main arteries and branches of the pulmonary arteries are labeled, and therefore, it is not necessary to identify pulmonary artery branches based on the result of the pulmonary artery skeleton binary classification.
Referring to fig. 8, an embodiment of the present disclosure provides a pixel classification apparatus, which may include: the device comprises a pulmonary artery skeleton and adjacency matrix acquisition module, a primary classification feature extraction module and a pulmonary artery skeleton classification module.
The pulmonary artery skeleton and adjacency matrix acquisition module is used for acquiring a pixel set representing the pulmonary artery skeleton in a pulmonary artery image sequence and an adjacency matrix representing the adjacency relation between pixels in the pixel set of the pulmonary artery skeleton.
The primary classification feature extraction module is used for extracting primary classification features of pixels in a pixel set of the pulmonary artery skeleton; wherein the primary classification feature is used to characterize a picture element in the set of picture elements of the pulmonary artery skeleton.
And the pulmonary artery skeleton classification module is used for determining the pulmonary artery category to which the image elements belong in the image element set of the pulmonary artery skeleton by using the primary classification characteristic and the adjacency matrix.
Referring to fig. 9, an embodiment of the present disclosure provides a training apparatus for a pixel classification model, which may include: the system comprises a training sample construction module, a pulmonary artery skeleton and adjacency matrix acquisition module, a primary classification characteristic extraction module, a pulmonary artery category prediction module and a pixel classification model generation module.
The training sample construction module is used for constructing a training sample; the training samples comprise a pulmonary artery image sequence and a category label corresponding to a pixel representing a pulmonary artery in the pulmonary artery image sequence.
The pulmonary artery skeleton and adjacency matrix acquisition module is used for acquiring a pixel set representing the pulmonary artery skeleton in a pulmonary artery image sequence and an adjacency matrix representing the adjacency relation between pixels in the pixel set of the pulmonary artery skeleton.
The primary classification feature extraction module is used for extracting primary classification features of pixels in a pixel set of the pulmonary artery skeleton; wherein the primary classification feature is used to characterize a picture element in the set of picture elements of the pulmonary artery skeleton.
And the pulmonary artery category prediction module is used for predicting the pulmonary artery category of the image elements in the image element set of the pulmonary artery skeleton determined based on the primary classification characteristic and the adjacency matrix.
And the pixel classification model generation module is used for updating the pixel classification model according to the predicted pulmonary artery category and the loss generated by the category label corresponding to the pixel representing the pulmonary artery.
The specific functions and effects achieved by the classification device for pixels and/or the training device for the pixel classification model can be explained by referring to other embodiments in this specification, and are not described herein again. The modules in the classification device of the image elements and/or the training device of the image element classification model can be wholly or partially realized by software, hardware and the combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
Referring to fig. 10, in some embodiments, an electronic device may be provided, the electronic device including: a processor; a memory for storing the processor-executable instructions; the processor is configured to execute the method described in the foregoing embodiment.
In some embodiments, a computer-readable storage medium may be provided, on which a computer program is stored, which when executed by a processor implements the method steps in the embodiments.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments may be implemented by hardware instructions of a computer program, which may be stored in a non-volatile computer-readable storage medium, and when executed, may include processes of the embodiments of the methods. Any reference to memory, storage, database, or other medium used in the various embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.
It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The description is made in a progressive manner among the embodiments of the present specification. The different embodiments focus on the different parts described compared to the other embodiments. After reading this specification, one skilled in the art can appreciate that many embodiments and many features disclosed in the embodiments can be combined in many different ways, and for the sake of brevity, all possible combinations of features in the embodiments are not described. However, as long as there is no contradiction between combinations of these technical features, the scope of the present specification should be considered as being described.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In the present specification, the embodiments themselves are emphasized differently from the other embodiments, and the embodiments can be explained in contrast to each other. Any combination of the embodiments in this specification based on general technical common knowledge by those skilled in the art is encompassed in the disclosure of the specification.
The above description is only an embodiment of the present disclosure, and is not intended to limit the scope of the claims of the present disclosure. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present application shall be included in the scope of the claims of the present application.
Claims (15)
1. A method of classifying picture elements, the method comprising:
acquiring a pixel set representing a pulmonary artery skeleton in a pulmonary artery image sequence and an adjacency matrix representing an adjacency relation between pixels in the pixel set of the pulmonary artery skeleton;
extracting primary classification characteristics of pixels in a pixel set of the pulmonary artery skeleton; wherein the primary classification feature is used to characterize pixels in a set of pixels of the pulmonary artery skeleton;
and determining the pulmonary artery category to which the picture elements belong in the picture element set of the pulmonary artery skeleton by using the primary classification characteristic and the adjacency matrix.
2. The method of claim 1, wherein the step of extracting the primary classification features of the image elements in the set of image elements of the pulmonary artery skeleton comprises:
acquiring a pulmonary trachea image sequence corresponding to the pulmonary artery image sequence; wherein the pulmonary trachea image sequence and the pulmonary artery image sequence are obtained based on the same pulmonary image sequence;
constructing a pixel set representing a pulmonary artery in a pulmonary artery image sequence; wherein the set of picture elements of the pulmonary artery skeleton is a subset of the set of picture elements of the pulmonary artery;
extracting primary classification features of pixels in a pixel set of the pulmonary artery based on the pulmonary artery image sequence and the pulmonary trachea image sequence; wherein the primary classification feature is used to characterize a picture element in the set of picture elements of the pulmonary artery skeleton.
3. The method of claim 1, further comprising:
adjusting the acquired lung image sequence to a specified resolution to obtain a target lung image sequence;
determining a set of pixels representing a pulmonary vessel in the sequence of lung images based on the sequence of target lung images; wherein the pulmonary vessels include pulmonary artery vessels and pulmonary vein vessels;
classifying pixels in a pixel set representing pulmonary vessels in the pulmonary image sequence to obtain an artery pixel set representing pulmonary artery vessels;
and generating a pulmonary artery image sequence based on the artery pixel element set.
4. The method of claim 3, further comprising:
segmenting the initial sequence of lung images into a first sequence of lung images and a second sequence of lung images; wherein the first sequence of lung images represents a left lung region; the second sequence of lung images represents a right lung region;
correspondingly, in the step of adjusting the acquired lung image sequence to a specified resolution to obtain the target lung image sequence, the method includes:
respectively adjusting the first lung image sequence and the second lung image sequence to specified resolutions to obtain a first target lung image sequence and a second target lung image sequence; wherein the first and second target lung image sequences belong to a target lung image sequence.
5. The method of claim 1, wherein the step of determining a pulmonary artery class to which picture elements belong in the set of picture elements of the pulmonary artery skeleton using the primary classification feature and the adjacency matrix comprises:
extracting a pel-expanded classification feature in a set of pels of the pulmonary artery skeleton using the primary classification feature and the adjacency matrix;
and generating the pulmonary artery category of the pixels in the pixel set of the pulmonary artery skeleton based on the expanded classification features.
6. The method of claim 5, further comprising:
determining a plurality of branch pixel sets representing pulmonary artery skeleton branches in the pixel set of the pulmonary artery skeleton based on the pulmonary artery category of the pixels in the pixel set of the pulmonary artery skeleton; wherein the pulmonary artery skeleton branch is connected with the pulmonary artery skeleton main trunk; the set of branch pixels is a subset of the set of pixels of the pulmonary artery skeleton;
acquiring a branch adjacency matrix representing adjacency relation of pixels in the branch pixel set;
determining a pulmonary artery class of pixels in the set of branch pixels using the expanded classification features and the branch adjacency matrix.
7. The method of claim 6, wherein the step of using the expanded classification features and the branch adjacency matrix to determine the pulmonary artery class of picture elements in the set of branch picture elements comprises:
selecting pixels with a specified number from the branch pixel set as a reference pixel set;
acquiring the expansion classification characteristics of the pixels in the reference pixel set;
determining a pulmonary artery class of pixels in the set of branch pixels based on the expanded classification features and the branch adjacency matrix.
8. The method of claim 1, further comprising:
taking the pixels in the pixel set as seed pixels;
and determining the pulmonary artery category of the pixel representing the pulmonary artery in the pulmonary artery image sequence by adopting a preset region growing algorithm based on the seed pixel.
9. The method of claim 8, further comprising:
carrying out dyeing processing on the pulmonary artery category of a pixel representing the pulmonary artery in the pulmonary artery image sequence; wherein different pulmonary artery categories correspond to different colors.
10. A training method of a pixel classification model is characterized by comprising the following steps:
constructing a training sample; the training sample comprises a pulmonary artery image sequence and a category label corresponding to a pixel representing a pulmonary artery in the pulmonary artery image sequence;
acquiring a pixel set representing a pulmonary artery skeleton in a pulmonary artery image sequence and an adjacency matrix representing an adjacency relation between pixels in the pixel set of the pulmonary artery skeleton;
extracting primary classification characteristics of pixels in a pixel set of the pulmonary artery skeleton; wherein the primary classification feature is used to characterize pixels in a set of pixels of the pulmonary artery skeleton;
a predicted pulmonary artery class of picture elements in the set of picture elements of the pulmonary artery skeleton determined based on the primary classification feature and the adjacency matrix;
and updating the pixel classification model according to the loss generated by the category label corresponding to the predicted pulmonary artery category and the pixel representing the pulmonary artery.
11. The method of claim 10, wherein the pulmonary artery image sequence further corresponds to a pulmonary artery skeleton trunk tag and a pulmonary artery skeleton branch tag; wherein the pulmonary artery skeleton branch label corresponds to a branch adjacency matrix; the method further comprises the following steps:
selecting pixels with a specified number from the pulmonary artery skeleton branch label as a reference pixel set;
acquiring the expansion classification characteristics of the pixels in the reference pixel set; wherein the expansion classification feature is used for characterizing the image elements in the reference image element set;
correspondingly, in the step of determining a predicted pulmonary artery class of picture elements in the set of picture elements based on the primary classification feature and the adjacency matrix, the method further comprises:
determining a predicted pulmonary artery class for a pixel in the set of pixels based on the expanded feature and the branch adjacency matrix.
12. A classification apparatus of a picture element, comprising:
the pulmonary artery skeleton and adjacency matrix acquisition module is used for acquiring a pixel set representing the pulmonary artery skeleton in a pulmonary artery image sequence and an adjacency matrix representing the adjacency relation between pixels in the pixel set of the pulmonary artery skeleton;
the primary classification feature extraction module is used for extracting primary classification features of pixels in the pixel set of the pulmonary artery skeleton; wherein the primary classification feature is used to characterize pixels in a set of pixels of the pulmonary artery skeleton;
and the pulmonary artery skeleton classification module is used for determining the pulmonary artery category to which the image elements belong in the image element set of the pulmonary artery skeleton by using the primary classification characteristic and the adjacency matrix.
13. A training device for a pixel classification model is characterized by comprising:
the training sample construction module is used for constructing a training sample; the training sample comprises a pulmonary artery image sequence and a category label corresponding to a pixel representing a pulmonary artery in the pulmonary artery image sequence;
the pulmonary artery skeleton and adjacency matrix acquisition module is used for acquiring a pixel set representing the pulmonary artery skeleton in a pulmonary artery image sequence and an adjacency matrix representing the adjacency relation between pixels in the pixel set of the pulmonary artery skeleton;
the primary classification feature extraction module is used for extracting primary classification features of pixels in a pixel set of the pulmonary artery skeleton; wherein the primary classification feature is used to characterize pixels in a set of pixels of the pulmonary artery skeleton;
a pulmonary artery category prediction module, configured to determine a predicted pulmonary artery category of pixels in a set of pixels of the pulmonary artery skeleton based on the primary classification feature and the adjacency matrix;
and the pixel classification model generation module is used for updating the pixel classification model according to the predicted pulmonary artery category and the loss generated by the category label corresponding to the pixel representing the pulmonary artery.
14. An electronic device, characterized in that the electronic device comprises:
a processor;
a memory for storing the processor-executable instructions;
the processor configured to perform the method of any of the preceding claims 1 to 11.
15. A computer-readable storage medium, the storage medium storing a computer program for executing the method of any of the preceding claims 1 to 11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210545928.2A CN114972859B (en) | 2022-05-19 | Pixel classification method, model training method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210545928.2A CN114972859B (en) | 2022-05-19 | Pixel classification method, model training method, device, equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114972859A true CN114972859A (en) | 2022-08-30 |
| CN114972859B CN114972859B (en) | 2024-10-29 |
Family
ID=
Citations (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103235929A (en) * | 2013-03-11 | 2013-08-07 | 北京航空航天大学 | Identification method and identification device on basis of hand vein images |
| CN108961273A (en) * | 2018-07-03 | 2018-12-07 | 东北大学 | A kind of method and system for dividing pulmonary artery and pulmonary vein from CT images |
| CN110349175A (en) * | 2019-06-25 | 2019-10-18 | 深圳先进技术研究院 | A kind of arteriovenous malformation dividing method, system and electronic equipment |
| CN110338830A (en) * | 2019-07-30 | 2019-10-18 | 赛诺威盛科技（北京）有限公司 | The method for automatically extracting neck blood vessel center path in CTA image |
| CN111028248A (en) * | 2019-12-19 | 2020-04-17 | 杭州健培科技有限公司 | Method and device for separating static and dynamic pulses based on CT (computed tomography) image |
| CN111612743A (en) * | 2020-04-24 | 2020-09-01 | 杭州电子科技大学 | Coronary artery central line extraction method based on CT image |
| CN111932554A (en) * | 2020-07-31 | 2020-11-13 | 青岛海信医疗设备股份有限公司 | Pulmonary blood vessel segmentation method, device and storage medium |
| CN111950408A (en) * | 2020-07-28 | 2020-11-17 | 深圳职业技术学院 | Finger vein image recognition method and device based on rule graph and storage medium |
| CN112017167A (en) * | 2020-08-24 | 2020-12-01 | 杭州深睿博联科技有限公司 | Coronary artery central line generation method and device based on bidirectional coronary artery blood vessel tracking |
| US20200394789A1 (en) * | 2019-06-12 | 2020-12-17 | Carl Zeiss Meditec Inc | Oct-based retinal artery/vein classification |
| US20210142470A1 (en) * | 2019-11-12 | 2021-05-13 | International Intelligent Informatics Solution Laboratory LLC | System and method for identification of pulmonary arteries and veins depicted on chest ct scans |
| CN112861961A (en) * | 2021-02-03 | 2021-05-28 | 推想医疗科技股份有限公司 | Pulmonary blood vessel classification method and device, storage medium and electronic equipment |
| CN113011509A (en) * | 2021-03-25 | 2021-06-22 | 推想医疗科技股份有限公司 | Lung bronchus classification method and device, electronic equipment and storage medium |
| CN113256670A (en) * | 2021-05-24 | 2021-08-13 | 推想医疗科技股份有限公司 | Image processing method and device, and network model training method and device |
| CN113409328A (en) * | 2021-06-02 | 2021-09-17 | 东北大学 | Pulmonary artery and vein segmentation method, device, medium and equipment of CT image |
| CN113538415A (en) * | 2021-08-16 | 2021-10-22 | 深圳市旭东数字医学影像技术有限公司 | Segmentation method and device for pulmonary blood vessels in medical image and electronic equipment |
| CN114298999A (en) * | 2021-12-24 | 2022-04-08 | 上海联影智能医疗科技有限公司 | Method for detecting vascular structure variation, readable storage medium, and program product |
| CN114332013A (en) * | 2021-12-29 | 2022-04-12 | 福州大学 | CT image target lung segment identification method based on pulmonary artery tree classification |
Patent Citations (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103235929A (en) * | 2013-03-11 | 2013-08-07 | 北京航空航天大学 | Identification method and identification device on basis of hand vein images |
| CN108961273A (en) * | 2018-07-03 | 2018-12-07 | 东北大学 | A kind of method and system for dividing pulmonary artery and pulmonary vein from CT images |
| US20200394789A1 (en) * | 2019-06-12 | 2020-12-17 | Carl Zeiss Meditec Inc | Oct-based retinal artery/vein classification |
| CN110349175A (en) * | 2019-06-25 | 2019-10-18 | 深圳先进技术研究院 | A kind of arteriovenous malformation dividing method, system and electronic equipment |
| CN110338830A (en) * | 2019-07-30 | 2019-10-18 | 赛诺威盛科技（北京）有限公司 | The method for automatically extracting neck blood vessel center path in CTA image |
| US20210142470A1 (en) * | 2019-11-12 | 2021-05-13 | International Intelligent Informatics Solution Laboratory LLC | System and method for identification of pulmonary arteries and veins depicted on chest ct scans |
| CN111028248A (en) * | 2019-12-19 | 2020-04-17 | 杭州健培科技有限公司 | Method and device for separating static and dynamic pulses based on CT (computed tomography) image |
| CN111612743A (en) * | 2020-04-24 | 2020-09-01 | 杭州电子科技大学 | Coronary artery central line extraction method based on CT image |
| CN111950408A (en) * | 2020-07-28 | 2020-11-17 | 深圳职业技术学院 | Finger vein image recognition method and device based on rule graph and storage medium |
| CN111932554A (en) * | 2020-07-31 | 2020-11-13 | 青岛海信医疗设备股份有限公司 | Pulmonary blood vessel segmentation method, device and storage medium |
| CN112017167A (en) * | 2020-08-24 | 2020-12-01 | 杭州深睿博联科技有限公司 | Coronary artery central line generation method and device based on bidirectional coronary artery blood vessel tracking |
| CN112861961A (en) * | 2021-02-03 | 2021-05-28 | 推想医疗科技股份有限公司 | Pulmonary blood vessel classification method and device, storage medium and electronic equipment |
| CN113011509A (en) * | 2021-03-25 | 2021-06-22 | 推想医疗科技股份有限公司 | Lung bronchus classification method and device, electronic equipment and storage medium |
| CN113256670A (en) * | 2021-05-24 | 2021-08-13 | 推想医疗科技股份有限公司 | Image processing method and device, and network model training method and device |
| CN113409328A (en) * | 2021-06-02 | 2021-09-17 | 东北大学 | Pulmonary artery and vein segmentation method, device, medium and equipment of CT image |
| CN113538415A (en) * | 2021-08-16 | 2021-10-22 | 深圳市旭东数字医学影像技术有限公司 | Segmentation method and device for pulmonary blood vessels in medical image and electronic equipment |
| CN114298999A (en) * | 2021-12-24 | 2022-04-08 | 上海联影智能医疗科技有限公司 | Method for detecting vascular structure variation, readable storage medium, and program product |
| CN114332013A (en) * | 2021-12-29 | 2022-04-12 | 福州大学 | CT image target lung segment identification method based on pulmonary artery tree classification |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109166130B (en) | Image processing method and image processing device | |
| Zuo et al. | R2AU‐Net: attention recurrent residual convolutional neural network for multimodal medical image segmentation | |
| US11593943B2 (en) | RECIST assessment of tumour progression | |
| CN110070540B (en) | Image generation method and device, computer equipment and storage medium | |
| CN111899245A (en) | Image segmentation method, image segmentation device, model training method, model training device, electronic equipment and storage medium | |
| US20220198230A1 (en) | Auxiliary detection method and image recognition method for rib fractures based on deep learning | |
| WO2020109630A1 (en) | Method and system for providing an at least 3-dimensional medical image segmentation of a structure of an internal organ | |
| CN111368849B (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN110298844B (en) | X-ray radiography image blood vessel segmentation and identification method and device | |
| CN111899244B (en) | Image segmentation method, network model training method, device and electronic equipment | |
| JP2020501273A (en) | Learning to annotate objects in images | |
| Singh et al. | Deep-learning based system for effective and automatic blood vessel segmentation from Retinal fundus images | |
| CN111667459B (en) | Medical sign detection method, system, terminal and storage medium based on 3D variable convolution and time sequence feature fusion | |
| CN111462047A (en) | Blood vessel parameter measuring method, blood vessel parameter measuring device, computer equipment and storage medium | |
| US11494908B2 (en) | Medical image analysis using navigation processing | |
| CN110570394A (en) | medical image segmentation method, device, equipment and storage medium | |
| CN114972211B (en) | Training method, segmentation method, device, equipment and medium for image segmentation model | |
| CN112418299B (en) | Coronary artery segmentation model training method, coronary artery segmentation method and device | |
| CN115115772A (en) | Key structure reconstruction method and device based on three-dimensional image and computer equipment | |
| CN111524109A (en) | Head medical image scoring method and device, electronic equipment and storage medium | |
| CN114340496A (en) | Analysis method and related device of heart coronary artery based on VRDS AI medical image | |
| CN116958537A (en) | Lung nodule segmentation method based on U-Net model | |
| CN114972859B (en) | Pixel classification method, model training method, device, equipment and medium | |
| CN111209946A (en) | Three-dimensional image processing method, image processing model training method, and medium | |
| CN114972859A (en) | Pixel classification method, model training method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |