CN114972859A - Pixel classification method, model training method, device, equipment and medium - Google Patents

Abstract

The embodiment of the specification provides a pixel classification method, a model training method, a device, equipment and a medium. The method comprises the following steps: acquiring a pixel set representing a pulmonary artery skeleton in a pulmonary artery image sequence and an adjacency matrix representing an adjacency relation between pixels in the pixel set of the pulmonary artery skeleton; extracting primary classification features of the pixels in the pixel set; wherein the primary classification feature is used to characterize the pels in the set of pels; and determining the pulmonary artery category to which the picture elements belong in the picture element set of the pulmonary artery skeleton by using the primary classification characteristic and the adjacency matrix. The primary classification characteristic of the pulmonary artery skeleton pixel is extracted, and the characteristic of the pixel in the pixel set is extracted by fusing an image attention network which is used for constructing an adjacent matrix of the pixel in the pixel set expressing the pulmonary artery skeleton and is used as a pixel classification model, so that the classification accuracy of the pixel expressing the pulmonary artery in a pulmonary artery image sequence is improved.

Description

Classification method of pixel, model training method, device, equipment and medium

technical field

The embodiments of this specification relate to the field of image processing, a specific pixel classification method, a model training method, an apparatus, a device, and a medium.

Background technique

With the increasing role of medical imaging examinations in assisting medical diagnosis and treatment, pulmonary medical imaging examinations can help doctors understand the condition of patients' lungs. Classifying the pixels representing the pulmonary arteries in medical images of the lungs can help doctors understand the anatomy of a patient's pulmonary arteries when operating on a patient. The existing pulmonary artery classification methods mainly input the pulmonary artery images extracted based on pulmonary medical images directly into the neural network for classification, which is prone to misclassification of the pulmonary artery.

SUMMARY OF THE INVENTION

In view of this, various embodiments of the present specification aim to provide a method for classifying pixels, a method for model training, an apparatus, a device, and a medium, so as to provide a classification method that can improve the classification of pixels representing pulmonary arteries in a sequence of pulmonary artery images to a certain extent method of accuracy.

An embodiment of the present specification proposes a method for classifying pixels, including: acquiring a pixel set representing a pulmonary artery skeleton in a pulmonary artery image sequence, and an adjacency representing an adjacency relationship between pixels in the pixel set of the pulmonary artery skeleton matrix; extract the primary classification features of the pixels in the pixel set of the pulmonary artery skeleton; wherein, the primary classification features are used to characterize the pixels in the pixel set of the pulmonary artery skeleton; use the primary classification features and all The adjacency matrix determines the pulmonary artery category to which a pixel in the pixel set of the pulmonary artery skeleton belongs.

An embodiment of the present specification proposes a training method for a pixel classification model, including: constructing a training sample; wherein the training sample includes a pulmonary artery image sequence and a category label corresponding to a pixel representing a pulmonary artery in the pulmonary artery image sequence; obtaining a pulmonary artery a pixel set representing the pulmonary artery skeleton in the image sequence, and an adjacency matrix representing the adjacency relationship between the pixels in the pixel set of the pulmonary artery skeleton; extracting primary classification features of the pixels in the pixel set of the pulmonary artery skeleton; Wherein, the primary classification feature is used to characterize the pixel in the pixel set of the pulmonary artery skeleton; the predicted pulmonary artery of the pixel in the pixel set of the pulmonary artery skeleton determined based on the primary classification feature and the adjacency matrix class; updating the pixel classification model according to the predicted pulmonary artery class and the loss generated by the class label corresponding to the pixel representing the pulmonary artery.

An embodiment of the present specification proposes an apparatus for classifying pixels, including: a pulmonary artery skeleton and an adjacency matrix acquisition module, configured to acquire a pixel set representing the pulmonary artery skeleton in a pulmonary artery image sequence, and a pixel set representing the pulmonary artery skeleton The adjacency matrix of the adjacency relationship between the pixels; the primary classification feature extraction module is used to extract the primary classification features of the pixels in the pixel set of the pulmonary artery skeleton; wherein, the primary classification features are used to characterize the pulmonary artery A pixel in the pixel set of the skeleton; a pulmonary artery skeleton classification module, configured to use the primary classification feature and the adjacency matrix to determine the pulmonary artery category to which a pixel in the pixel set of the pulmonary artery skeleton belongs.

This description is an embodiment of a training device for a pixel classification model, including: a training sample building module for constructing a training sample; wherein the training sample includes a pulmonary artery image sequence and an image representing a pulmonary artery in the pulmonary artery image sequence The category label corresponding to the element; the pulmonary artery skeleton and the adjacency matrix acquisition module is used to obtain the pixel set representing the pulmonary artery skeleton in the pulmonary artery image sequence, and the adjacency matrix representing the adjacency relationship between the pixels in the pixel set of the pulmonary artery skeleton ; The primary classification feature extraction module is used to extract the primary classification features of the pixels in the pixel set of the pulmonary artery skeleton; wherein, the primary classification features are used to represent the pixels in the pixel set of the pulmonary artery skeleton; Pulmonary artery a category prediction module, used for predicting the pulmonary artery category of pixels in the pixel set of the pulmonary artery skeleton determined based on the primary classification feature and the adjacency matrix; a pixel classification model generation module, used for predicting the pulmonary artery category according to the The pixel classification model is updated with a loss generated from the class label corresponding to the pixel representing the pulmonary artery.

An embodiment of the present specification provides an electronic device, including: a processor; a memory for storing instructions executable by the processor; and the processor for executing the method described in the foregoing embodiment.

An embodiment of the present specification provides a computer-readable storage medium, including: the storage medium stores a computer program, and the computer program is used to execute the method described in the foregoing embodiment.

Various embodiments of the present specification extract the pixel set representing the pulmonary artery skeleton in the pulmonary artery image sequence, and construct an adjacency matrix according to the adjacency relationship between the pixels in the pixel set, and then extract the image representing the pulmonary artery based on the initial pulmonary artery classification model of the pulmonary artery image. The primary pixel features and adjacency matrix of the pixels of the skeleton determine the pulmonary artery category of the pixels in the pixel set, which improves the classification accuracy of the pixels representing the pulmonary artery skeleton to a certain extent, and further improves the image representing the pulmonary artery. The accuracy of meta-segmented staining allows physicians to view the anatomy of a patient's pulmonary arteries.

Description of drawings

FIG. 1 is a schematic diagram illustrating the interaction of a pulmonary artery classification system in an example scenario provided by an embodiment.

FIG. 2 is a schematic diagram of a pulmonary artery segmented staining result in a scene example provided by an embodiment.

FIG. 3 is a schematic diagram illustrating the interaction of the pulmonary artery classification system in an example scenario provided by an embodiment.

FIG. 4 is a schematic flowchart of a method for classifying pixels according to an embodiment.

FIG. 5 is a schematic diagram showing the classification result of the pulmonary artery skeleton according to an embodiment.

FIG. 6 is a schematic diagram of a binary classification result of the pulmonary artery skeleton provided by an embodiment.

FIG. 7 is a schematic flowchart of a training method for a pixel classification model provided by an embodiment.

FIG. 8 is a schematic diagram of an apparatus for classifying pixels according to an embodiment.

FIG. 9 is a schematic diagram of a training device for a pixel classification model provided by an embodiment.

FIG. 10 is a schematic diagram of an electronic device provided by an embodiment.

Detailed ways

In order to enable those skilled in the art to better understand the solutions of the present specification, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present specification. Obviously, the described embodiments are only It is a part of embodiment of this specification, and it is not all embodiment. Based on the implementations in this specification, all other implementations obtained by persons of ordinary skill in the art without creative efforts shall fall within the protection scope of this specification.

Referring to FIG. 1 and FIG. 2 , the embodiment of the present specification provides a scenario example of a pulmonary artery classification system. When patient A visits a doctor, the doctor can issue a lung CT image examination sheet to patient A. After the lung CT image examination, patient A can obtain the lung CT image sequence, and then send the lung CT image sequence to the client. The client can send the lung CT image sequence to the server. The server first divides the lung CT image sequence into a first lung image sequence representing the left lung and a second lung image sequence representing the right lung. Then, the server may process the first lung image sequence and the second lung image sequence into a first target lung image sequence and a second target lung image sequence with a data volume of 192*128*96 through up-sampling and/or down-sampling, respectively. image sequence. The server preprocesses the first lung image sequence and the second lung image sequence, respectively, to obtain a pulmonary blood vessel image sequence representing the left lung, a pulmonary blood vessel image sequence representing the right lung, and a pulmonary trachea image sequence and representation representing the left lung. Lung trachea image sequence of the right lung. The pulmonary blood vessel image sequence is obtained by stitching the pulmonary blood vessel image sequence representing the left lung and the pulmonary blood vessel image sequence representing the right lung. The pulmonary trachea image sequence is obtained by stitching the pulmonary trachea image sequence representing the left lung and the pulmonary trachea image sequence representing the right lung. Then, the server may perform pre-segmentation processing on the pulmonary blood vessel image sequence to obtain a pulmonary artery image sequence representing the pulmonary artery.

After acquiring the pulmonary artery image sequence, the server may input the pulmonary artery image sequence and the pulmonary trachea image sequence into the ResU-net model to extract initial pixel features of the pixels representing the pulmonary artery in the pulmonary artery image sequence. The server may also use the skeletonize algorithm to extract the first pixel representing the pulmonary artery skeleton in the pulmonary artery image sequence, and construct a first adjacency matrix representing the adjacency relationship between the first pixels. Then, the server can use the first adjacency matrix as the attention mechanism of the ResU-net model, extract the extended feature corresponding to the first pixel based on the initial pixel feature, and generate the pulmonary artery category corresponding to the first pixel according to the extended feature.

After generating the pulmonary artery category corresponding to the first pixel, the server may generate a second pixel representing the skeleton branch of the pulmonary artery based on the classification result of the first pixel, and construct a second adjacency representing the adjacency relationship between the second pixels matrix. Then, the server can use the second adjacency matrix as the attention mechanism of the ResU-net model, extract branch extension features of the second pixel object based on the extension features, and generate a pulmonary artery category corresponding to the second pixel according to the branch extension features. When the pulmonary artery category corresponding to the second pixel is inconsistent with the pulmonary artery category corresponding to the first pixel, the pulmonary artery category corresponding to the second pixel is replaced with the pulmonary artery category corresponding to the first pixel.

Finally, using the first pixel as a seed point, a preset region growth algorithm is used to determine the pulmonary artery category corresponding to the pixel representing the pulmonary artery, and different colors are assigned to the pixel representing the pulmonary artery according to the preset color. For example, the main pulmonary artery is given red, the apical segment of the right upper lobe is given blue, the rear end of the right upper lobe is given yellow, and the dorsal segment of the left lower lobe is given orange. Finally, the server can send the segmented and stained pulmonary artery image sequence to the client, so that the doctor can view the detailed anatomical structure of the pulmonary artery, so as to understand the degree of lung lesions of the patient.

The above is only an example of a scenario provided in this specification, and is not intended to limit the present invention. Any modification, equivalent replacement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention. within.

Referring to FIG. 3 , an embodiment of the present specification provides a pulmonary artery classification system. And the classification method of pulmonary artery image and/or the training method of pulmonary artery image classification model provided in this specification can be applied to the pulmonary artery classification system. The pulmonary artery classification system may include a hardware environment formed by a medical imaging device 110 , a client 120 and a server 130 . The medical imaging apparatus 110 is connected with the client 120, and the server 130 is connected with the client 120 through a communication network through a network. The communication network may be a wired network or a wireless network. The medical imaging device 110 examines and images the lungs, resulting in a sequence of lung images. The sequence of lung images is transmitted to the client 120 via the communicating medical imaging device 110 . The client 120 sends the lung image sequence to the server 130, and the server 130 receives the lung image sequence. The medical imaging device 110 may be, but is not limited to, at least one of an ultrasonic medical device, a CT medical inspection device, and an MRI medical inspection device. Client 120 may be an electronic device with network access capabilities. Specifically, for example, the client may be a desktop computer, a tablet computer, a notebook computer, a smart phone, a digital assistant, a smart wearable device, a shopping guide terminal, a TV, a smart speaker, a microphone, and the like. Among them, smart wearable devices include but are not limited to smart bracelets, smart watches, smart glasses, smart helmets, smart necklaces, and the like. Alternatively, the client can also be software that can run in the electronic device. Those skilled in the art may know that the number of the above-mentioned clients 120 may be one or more, and their types may be the same or different. For example, the number of the above-mentioned clients 120 may be one, or the number of the above-mentioned clients 120 may be dozens or hundreds, or more. The number of clients 120 and device types are not limited in the embodiments of the present application. The server 130 may be an electronic device with a certain computing processing capability. It may have a network communication module, processor and memory, among others. Of course, the server may also refer to software running in the electronic device. The server may also be a distributed server, which may be a system with multiple processors, memories, network communication modules, and the like that operate cooperatively. Alternatively, the server may also be a server cluster formed by several servers. Or, with the development of science and technology, the server may also be a new technical means capable of realizing the corresponding functions of the implementation manners of the description. For example, it can be a new form of "server" based on quantum computing.

Referring to FIG. 4 and FIG. 5 , the embodiment of the present specification provides a method for classifying pixels. The classification method of the picture element can be applied to electronic equipment. The classification method of the picture element may include the following steps.

Step S210: Obtain a pixel set representing the pulmonary artery skeleton in the pulmonary artery image sequence, and an adjacency matrix representing the adjacency relationship between the pixels in the pixel set of the pulmonary artery skeleton.

In some cases, pixels representing the pulmonary artery on the same branch of the pulmonary artery have the same image features, which may lead to misclassification if the category of the pixel representing the pulmonary artery is directly classified. Therefore, the adjacency relationship between the pixel representing the pulmonary artery skeleton and the pixel representing the pulmonary artery skeleton can be extracted first.

The pulmonary artery image sequence is an image sequence representing the pulmonary artery extracted based on a pulmonary medical image. Specifically, after a patient performs a lung CT (Computed Tomography, electronic computed tomography) scan, a lung CT medical image sequence can be generated. By segmenting the lung CT medical image sequence, the pixel representing the pulmonary artery in the lung CT medical image sequence is assigned as 1, and the pixel representing the non-pulmonary artery is assigned as 0, thereby generating a three-dimensional pulmonary artery image sequence.

The pulmonary artery skeleton is refined into a width of one pixel based on the connected area of the pixels representing the pulmonary artery in the pulmonary artery image sequence, and is used for feature extraction and topological representation of the pixels in the pixel set representing the pulmonary artery. Specifically, for example, the binarized pulmonary artery image sequence is used as the input of the skeleton extraction model, and the Skeletonize() function in the morphology module is used to obtain a refined pulmonary artery skeleton image sequence including a width. Among them, the pulmonary artery skeleton image sequence is also a binarized image sequence.

The adjacency matrix is used to represent the adjacency relationship between the picture elements in the picture element set. Specifically, for example, if the pixel set includes 3000 pixels, then a 3000*3000 two-dimensional matrix can be constructed. There is no adjacency relationship between two identical pixels, so the diagonal of the two-dimensional matrix Pixels on the above are represented by "0"; for two different pixels, if the two pixels are adjacent in the 26-neighborhood on the spatial extent, they are represented by "1" in the two-dimensional matrix, if in the 26 are not adjacent in the neighborhood, then the correlation between the two pixels is represented by "0" in the two-dimensional matrix.

Step S220 : extract the primary classification features of the pixels in the pixel set of the pulmonary artery skeleton; wherein, the primary classification features are used to characterize the pixels in the pixel set of the pulmonary artery skeleton.

In some cases, in the process of classifying the pixels in the image, the features of the pixels in the pulmonary artery image are first extracted, and then the classification is performed based on the features of the pixels in the image. Therefore, the features of the pixels in the image can be extracted first by using the conventional image classification method.

The primary classification feature is extracted by an encoder based on the primary pixel classification model. Specifically, for example, the encoder of the ResU-net model is used to extract the initial pixel features representing the pulmonary artery pixels in the pulmonary artery image sequence. The pulmonary artery image sequence can be sliced along three directions to obtain 2D data in 3 view directions (top view, front view, side view), and then input the 2D data in the 3 view directions into the ResU-net network. Model, the features extracted in three directions are fused to obtain the primary classification features representing pulmonary artery pixels in the pulmonary artery image sequence. Then, the feature of the pixel representing the pulmonary artery skeleton in the primary classification feature representing the pulmonary artery pixel is extracted as the primary classification feature of the pixel in the pixel set.

Step S230: Use the primary classification feature and the adjacency matrix to determine the pulmonary artery category to which a pixel in the pixel set of the pulmonary artery skeleton belongs.

In some cases, if the pixels of the pulmonary artery skeleton are classified based only on primary classification features, misclassification may occur. Therefore, the adjacency matrix can be used as the graph attention network of the pixel classification model, and then based on the graph attention network and the primary classification features, the extended features of the cells in the pixel set can be extracted, and then the extended features can be used to determine the pixels in the pixel set. Pulmonary artery category.

The method for determining the pulmonary artery category of the pixel in the pixel set using the primary classification feature and the adjacency matrix may be to use the primary pixel feature as the feature of the pixel in the pixel set, and then use a graph attention network (Graph Attention Networks, GATs) further describe the importance of adjacent pixels for the classification of this pixel, and use the adjacency matrix as a mask for the graph attention network, thereby further enriching the features of the pixels representing the skeleton in the pulmonary artery image sequence. However, the above is only an exemplary method for extracting features of pixels in an image in this specification, and the embodiments of this specification do not limit the method for extracting features in images.

By extracting the primary classification features of the pixels representing the pulmonary artery skeleton and constructing the adjacency matrix of the pixels in the pixel set representing the pulmonary artery skeleton as the pixel classification model, the graph attention network performs fusion to extract the features of the pixels in the pixel set. Accuracy of classification of pixels representing pulmonary arteries in a sequence of pulmonary artery images.

In some embodiments, the step of extracting the primary classification features of the pixels in the pixel set of the pulmonary artery skeleton may include: acquiring a pulmonary trachea image sequence corresponding to the pulmonary artery image sequence; wherein the pulmonary trachea image sequence is the same as the The pulmonary artery image sequence is obtained based on the same lung image sequence; a pixel set representing the pulmonary artery in the pulmonary artery image sequence is constructed; wherein, the pixel set of the pulmonary artery skeleton is a subset of the pixel set of the pulmonary artery; The pulmonary artery image sequence and the pulmonary trachea image sequence are extracted, and the primary classification feature of the pixels in the pixel set of the pulmonary artery is extracted; wherein, the primary classification feature is used to characterize the image in the pixel set of the pulmonary artery skeleton. Yuan.

In some cases, there is a concomitant relationship between the pulmonary artery vessels and the bronchi. Therefore, in the process of classifying the pulmonary arteries, the distribution information of the pulmonary trachea can be used as a reference, so that the classification information of the pulmonary arteries can be determined more accurately and quickly, thereby improving the accuracy of the classification of the pulmonary arteries.

In some embodiments, the method for classifying pixels may further include: adjusting the acquired lung image sequence to a specified resolution to obtain a target lung image sequence; and determining the lung image based on the target lung image sequence A set of pixels representing pulmonary blood vessels in the sequence; wherein, the pulmonary blood vessels include pulmonary artery blood vessels and pulmonary vein blood vessels; classifying the pixels in the set of pixels representing pulmonary blood vessels in the lung image sequence to obtain a set of pixels representing pulmonary arterial blood vessels; A set of arterial pixels; a sequence of pulmonary artery images is generated based on the set of arterial pixels.

In some cases, before the process of classifying the pixels representing the pulmonary artery in the pulmonary artery image sequence, the pixels representing the pulmonary artery in the lung image sequence also need to be acquired, and the pulmonary artery image sequence is formed according to the pixels representing the pulmonary artery. Before the lung image sequence is input into the pulmonary artery segmentation model, in order to facilitate the operation of the model, the lung image sequence needs to be adjusted to the specified resolution. Specifically, for example, according to the average ratio of the pixels representing the pulmonary artery in the CT image of the patient's lung after the specific range of the isotropy in the CT image of the lung, the average ratio is 1.84:1.28:1, and then the input of the model is changed to [192] , 128, 96]. When the size of the lung image sequence is larger than [192, 128, 96], the lung image sequence is downsampled to [192, 128, 96] through the convolution kernel to obtain the target lung image sequence; When the size is smaller than [192, 128, 96], the lung image sequence is upsampled to [192, 128, 96] by bilinear interpolation to obtain the target lung image sequence, thus ensuring that the proportion of the image in the model is consistent. Then, the target lung image sequence is input into the U-net image segmentation model to obtain the pulmonary blood vessel image sequence. Finally, the pixels representing the pulmonary artery in the pulmonary blood vessel image sequence are further extracted to obtain the pulmonary artery image sequence.

In some embodiments, the method for classifying pixels may further include: dividing the initial lung image sequence into a first lung image sequence and a second lung image sequence; wherein the first lung image sequence represents the left side the lung area; the second lung image sequence represents the right lung area; correspondingly, in the step of adjusting the acquired lung image sequence to a specified resolution to obtain the target lung image sequence, the steps may include: Adjusting the first lung image sequence and the second lung image sequence to a specified resolution to obtain a first target lung image sequence and a second target lung image sequence; wherein the first target lung image The sequence and the second target lung image sequence belong to the target lung image sequence.

In some cases, when directly processing the entire lung image sequence, it is easy to cause misclassification of the pulmonary blood vessels in the left and right lungs. Therefore, the lung image sequence can be divided into a first lung image sequence representing the left lung and a lung image sequence representing the right lung. Second lung image sequence of the lungs. On the one hand, it improves the resolution of the lung image sequence input to the image segmentation model; on the other hand, it can also reduce the memory of the model during operation. However, it should be noted that the first lung image sequence and the second lung image sequence described in the embodiments of this specification are only used to represent the difference between the two, and are not limited to the first lung image sequence representing the left Lungs, the second lung image sequence represents the right lung.

In some embodiments, the step of using the primary classification feature and the adjacency matrix to determine the pulmonary artery category to which a pixel in the pixel set of the pulmonary artery skeleton belongs may include: using the primary classification feature and the adjacency matrix Extracting the pixel extended classification feature in the pixel set of the pulmonary artery skeleton; generating the pulmonary artery category of the pixel in the pixel set of the pulmonary artery skeleton based on the extended classification feature.

In some cases, in order to determine the pulmonary artery category of the pixel in the pulmonary artery pixel set, it is first necessary to obtain the pixel feature corresponding to the pixel representing the pulmonary artery. Therefore, the pixel features of the pulmonary artery pixels can be extracted based on the commonly used image classification model, and then the adjacency relationship between the pixels in the pixel set of the pulmonary artery skeleton can be combined to enrich the expression of the pixel features.

The extended classification feature is obtained based on the primary classification feature and the adjacency relationship between the pixels in the adjacency matrix. Specifically, for example, first use the ResU-net network to obtain primary classification features, and then input the primary classification features and adjacency matrix into the features extracted by the graph attention network model as extended classification features. Then, the pulmonary artery class of the pixels in the set of pixels representing the pulmonary artery in the pulmonary artery image sequence is determined based on the extended classification features.

Referring to FIG. 6 , in some embodiments, the method for classifying pixels may further include: determining, based on the pulmonary artery category of the pixels in the pixel set of the pulmonary artery skeleton, a branch of the pulmonary artery skeleton in the pixel set of the pulmonary artery skeleton representing the branch of the pulmonary artery skeleton. a plurality of branch pixel sets; wherein, the branches of the pulmonary artery skeleton are connected to the trunk of the pulmonary artery skeleton; the branch pixel set is a subset of the pixel set of the pulmonary artery skeleton; the acquisition represents the branch pixel set in the branch pixel set A branch adjacency matrix of adjacency relationships of pixels; using the extended classification feature and the branch adjacency matrix to determine the pulmonary artery category of the pixels in the branch pixel set.

In some cases, pulmonary artery pixels on the same pulmonary artery branch have the same pulmonary artery characteristics. Therefore, the pixels on one branch can be taken as a whole to determine the pulmonary artery class of the entire pulmonary artery branch.

In some embodiments, the step of using the extended classification feature and the branch adjacency matrix to determine the pulmonary artery category of the pixel in the branch pixel set may include: selecting a specified number of The pixel is used as a reference pixel set; the extended classification feature of the pixel in the reference pixel set is obtained; the pulmonary artery category of the pixel in the branch pixel set is determined based on the extended classification feature and the branch adjacency matrix.

In some cases, pulmonary artery pixels on the same branch have the same pulmonary artery characteristics. Therefore, a specified number of pixels can be randomly selected on the branch of the pulmonary artery. Then, the pulmonary artery category of the cells in the set of branch cells is determined based on the extended features of the selected cells and the branch adjacency matrix.

The method for determining the pulmonary artery category of the pixel in the branch pixel set by using the extended classification feature and the branch adjacency matrix may be to randomly select a specified number of pixels on the skeleton branch, and then select the specified number of pixels. Linear transformation is performed on the pixels of the skeleton to obtain the features on the skeleton branches, and the pulmonary artery category of the pulmonary artery skeleton branches is determined based on the features of the skeleton branches.

In some embodiments, the method for classifying pixels may further include: using the pixels in the pixel set as seed pixels; and determining the pulmonary artery image sequence by using a preset region growing algorithm based on the seed pixels. The pulmonary artery category of the cells representing the pulmonary artery in .

In some cases, other pixels on the pulmonary artery also need to be classified. Therefore, region growth can be performed based on the classification results of the pixels on the pulmonary artery skeleton branches, and then the classification results of the pixels on the pulmonary artery skeleton branches can be assigned to the pixels whose regions are grown based on the pulmonary artery skeleton branches, so as to obtain the representation in the pulmonary artery image sequence. Classification results for the pixels of the pulmonary artery.

In some embodiments, the method for classifying pixels may further include: coloring the pulmonary artery categories of the pixels representing the pulmonary artery in the pulmonary artery image sequence; wherein, different pulmonary artery categories correspond to different colors.

In some cases, in order to allow doctors to see the anatomical structure of the pulmonary artery more intuitively, different types of pulmonary arteries can also be stained to obtain segmented staining results of the pulmonary artery. Specifically, for example, the pulmonary artery category includes the main pulmonary artery, the tip of the right upper lobe, the posterior segment of the right upper lobe, the anterior segment of the right upper lobe, the mid-field lateral segment of the right lung, and the upper segment of the right lower lobe. Therefore, the main pulmonary artery can be assigned to Red, blue for the tip of the right upper lobe, green for the posterior segment of the right upper lobe, orange for the anterior segment of the right upper lobe, purple for the mid-field lateral segment of the right lung, and yellow for the upper segment of the right lower lobe. Among them, different categories correspond to different colors, and the color difference of the pulmonary artery categories that are close in space should be more obvious.

Referring to FIG. 7 , an embodiment of the present specification provides a training method for a pixel classification model. The training method of the pixel classification model can be applied to electronic equipment. The training method of the pixel classification model may include the following steps.

Step S310 : constructing a training sample; wherein, the training sample includes a pulmonary artery image sequence and a category label corresponding to a pixel representing a pulmonary artery in the pulmonary artery image sequence.

In some cases, the quality of the training sample construction directly affects the accuracy of the model. Therefore, when labeling the pulmonary artery image sequence, it is necessary to select doctors with higher professional level and rich professional experience for labeling. However, it should be noted that the specific form of the training sample is not limited in the embodiments of the present application, and it may be an original medical image, a pre-processed medical image, or a part of the original medical image.

Step S320: Acquire a pixel set representing the pulmonary artery skeleton in the pulmonary artery image sequence, and an adjacency matrix representing the adjacency relationship between the pixels in the pixel set of the pulmonary artery skeleton.

Step S330: Extract the primary classification features of the pixels in the pixel set of the pulmonary artery skeleton; wherein, the primary classification features are used to characterize the pixels in the pixel set of the pulmonary artery skeleton.

Step S340: The predicted pulmonary artery category of the pixels in the pixel set of the pulmonary artery skeleton determined based on the primary classification feature and the adjacency matrix.

Step S350: Update the pixel classification model according to the predicted pulmonary artery category and the loss generated by the category label corresponding to the pixel representing the pulmonary artery.

The pixel classification model is used to generate a predicted pulmonary artery class for the pixel in the pixel set representing the pulmonary artery skeleton in the training sample. The loss function is then calculated to predict the pulmonary artery class and the class label corresponding to the pixel representing the pulmonary artery. The pixel classification model is updated according to the loss function. When the loss function converges, the parameters of the updated model are used as the parameters of the pixel classification model.

The primary classification features may be obtained by the encoder of the initial classification model. The primary classification model may be a convolutional neural network (Convolutional Neural Network, CNN), a deep neural network (Deep Neural Network, DNN), or a recurrent neural network (Recurrent Neural Network, RNN), etc. In this embodiment, the primary classification model is The specific type is not limited. The primary classification model in this embodiment may include neural network layers such as an input layer, a convolution layer, a pooling layer, and a connection layer, which are not specifically limited in this embodiment. In addition, in this embodiment, the number of each type of neural network layer is also not limited.

In some embodiments, the pulmonary artery image sequence further corresponds to a pulmonary artery skeleton trunk label and a pulmonary artery skeleton branch label; wherein, the pulmonary artery skeleton branch label corresponds to a branch adjacency matrix; the method may further include: in the pulmonary artery skeleton Select a specified number of pixels in the branch label as the reference pixel set; obtain the extended classification feature of the pixels in the reference pixel set; wherein, the extended classification feature is used to characterize the image in the reference pixel set. Correspondingly, in the step of predicting the pulmonary artery category of the pixel in the pixel set determined based on the primary classification feature and the adjacency matrix, it may further include: based on the extended feature and the branch adjacency matrix A predicted pulmonary artery class for the cells in the set of cells is determined.

In some cases, the pixels on a certain branch have the same image features. Therefore, a specified number of pixels can be randomly selected on the pulmonary artery branch for feature extraction, and then the pulmonary artery category of the pixels on the entire branch can be obtained by a linear transformation method. The method for determining the pulmonary artery type described in this embodiment is the same as the method in the above embodiment, and the specific details are not repeated here, please refer to the above embodiment. However, it should be noted that in this embodiment, the trunk and branches of the pulmonary artery have been marked, so it is not necessary to determine the branch of the pulmonary artery based on the result of the binary classification of the pulmonary artery skeleton.

Referring to FIG. 8 , an embodiment of the present specification provides an apparatus for classifying pixels, which may include: a pulmonary artery skeleton and an adjacency matrix acquisition module, a primary classification feature extraction module, and a pulmonary artery skeleton classification module.

The pulmonary artery skeleton and adjacency matrix acquisition module is used to acquire a pixel set representing the pulmonary artery skeleton in the pulmonary artery image sequence, and an adjacency matrix representing the adjacency relationship between the pixels in the pixel set of the pulmonary artery skeleton.

The primary classification feature extraction module is configured to extract the primary classification features of the pixels in the pixel set of the pulmonary artery skeleton; wherein, the primary classification features are used to represent the pixels in the pixel set of the pulmonary artery skeleton.

A pulmonary artery skeleton classification module, configured to use the primary classification feature and the adjacency matrix to determine a pulmonary artery category to which a pixel in the pixel set of the pulmonary artery skeleton belongs.

Referring to FIG. 9, an embodiment of the present specification provides a training device for a pixel classification model, which may include: a training sample building module, a pulmonary artery skeleton and an adjacency matrix acquisition module, a primary classification feature extraction module, a pulmonary artery category prediction module, and a pixel classification module Model generation module.

A training sample building module is used to construct a training sample; wherein, the training sample includes a pulmonary artery image sequence and a category label corresponding to a pixel representing a pulmonary artery in the pulmonary artery image sequence.

The pulmonary artery skeleton and adjacency matrix acquisition module is used to acquire a pixel set representing the pulmonary artery skeleton in the pulmonary artery image sequence, and an adjacency matrix representing the adjacency relationship between the pixels in the pixel set of the pulmonary artery skeleton.

The primary classification feature extraction module is configured to extract the primary classification features of the pixels in the pixel set of the pulmonary artery skeleton; wherein, the primary classification features are used to represent the pixels in the pixel set of the pulmonary artery skeleton.

A pulmonary artery category prediction module, configured to predict a pulmonary artery category of a pixel in the pixel set of the pulmonary artery skeleton determined based on the primary classification feature and the adjacency matrix.

A pixel classification model generation module, configured to update the pixel classification model according to the predicted pulmonary artery category and the loss generated by the class label corresponding to the pixel representing the pulmonary artery.

The specific functions and effects implemented by the device for classifying pixels and/or the device for training pixel classification models can be explained with reference to other embodiments of the present specification, and details are not repeated here. Each module in the image element classification device and/or the image element classification model training device may be implemented in whole or in part by software, hardware, and combinations thereof. The modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

Referring to FIG. 10 , in some embodiments, an electronic device may be provided, the electronic device includes: a processor; a memory for storing instructions executable by the processor; the processor for executing the foregoing embodiments method described in.

In some embodiments, a computer-readable storage medium may be provided on which a computer program is stored, and when the computer program is executed by a processor, implements the method steps in the described embodiments.

Those of ordinary skill in the art can understand that all or part of the processes in the implementation method can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer readable In the storage medium, when the computer program is executed, it may include the flow of the implementation manner of each method. Wherein, any reference to memory, storage, database or other medium used in the various embodiments provided in this specification may include at least one of non-volatile and volatile memory. The non-volatile memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, and the like. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, the RAM may be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM).

It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

Among the various embodiments in this specification, the description is made in a progressive manner. Different embodiments focus on describing parts that are different from other embodiments. After reading this specification, those skilled in the art can know multiple embodiments in this specification, as well as multiple technical features disclosed in the embodiments, and can make more combinations. All possible combinations of the individual technical features in are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the description in this specification.

It should also be noted that the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device comprising a series of elements includes not only those elements, but also Other elements not expressly listed, or which are inherent to such a process, method, article of manufacture, or apparatus are also included. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article of manufacture, or device that includes the element.

The various embodiments in this specification themselves emphasize the parts that are different from other embodiments, and each embodiment can be explained in comparison with each other. Any combination of the various embodiments in this specification based on general technical knowledge by those skilled in the art shall be included in the disclosure scope of this specification.

The above descriptions are merely embodiments of the present case, and are not intended to limit the protection scope of the claims of the present case. For those skilled in the art, various modifications and variations can be made in this case. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this case shall be included within the scope of the claims of this case.

Claims (15)

1. a classification method of picture element, is characterized in that, described method comprises: acquiring a pixel set representing the pulmonary artery skeleton in the pulmonary artery image sequence, and an adjacency matrix representing the adjacency relationship between pixels in the pixel set of the pulmonary artery skeleton; extracting primary classification features of pixels in the pixel set of the pulmonary artery skeleton; wherein, the primary classification features are used to characterize the pixels in the pixel set of the pulmonary artery skeleton; The primary classification feature and the adjacency matrix are used to determine the pulmonary artery class to which a pixel in the pixel set of the pulmonary artery skeleton belongs. 2. The method according to claim 1, wherein the step of extracting primary classification features of pixels in the pixel set of the pulmonary artery skeleton comprises: acquiring a pulmonary trachea image sequence corresponding to the pulmonary artery image sequence; wherein, the pulmonary trachea image sequence and the pulmonary artery image sequence are obtained based on the same lung image sequence; constructing a pixel set representing the pulmonary artery in the pulmonary artery image sequence; wherein the pixel set of the pulmonary artery skeleton is a subset of the pixel set of the pulmonary artery; Based on the pulmonary artery image sequence and the pulmonary trachea image sequence, a primary classification feature of the pixels in the pixel set of the pulmonary artery is extracted; wherein, the primary classification feature is used to characterize the pixel set of the pulmonary artery skeleton. of pixels. 3. The method according to claim 1, wherein the method further comprises: Adjust the acquired lung image sequence to the specified resolution to obtain the target lung image sequence; determining, based on the target lung image sequence, a set of pixels representing pulmonary blood vessels in the lung image sequence; wherein the pulmonary blood vessels include pulmonary artery blood vessels and pulmonary venous blood vessels; classifying the pixels in the pixel set representing the pulmonary blood vessels in the lung image sequence to obtain the arterial pixel set representing the pulmonary artery blood vessel; A sequence of pulmonary artery images is generated based on the set of arterial pixels. 4. The method according to claim 3, wherein the method further comprises: Divide the initial lung image sequence into a first lung image sequence and a second lung image sequence; wherein the first lung image sequence represents the left lung region; the second lung image sequence represents the right lung area; Correspondingly, in the step of adjusting the acquired lung image sequence to a specified resolution to obtain the target lung image sequence, the steps include: respectively adjusting the first lung image sequence and the second lung image sequence to a specified resolution to obtain a first target lung image sequence and a second target lung image sequence; wherein the first target lung image sequence The image sequence and the second target lung image sequence belong to the target lung image sequence. 5. The method according to claim 1, wherein the step of using the primary classification feature and the adjacency matrix to determine the pulmonary artery category to which a pixel in the pixel set of the pulmonary artery skeleton belongs, comprising: using the primary classification feature and the adjacency matrix to extract the pixel extension classification feature in the pixel set of the pulmonary artery skeleton; A pulmonary artery category of a pixel in the pixel set of the pulmonary artery skeleton is generated based on the extended classification feature. 6. The method according to claim 5, wherein the method further comprises: A plurality of branch pixel sets representing the branches of the pulmonary artery skeleton in the pixel set of the pulmonary artery skeleton are determined based on the pulmonary artery category of the pixels in the pixel set of the pulmonary artery skeleton; wherein, the pulmonary artery skeleton branch and the pulmonary artery skeleton trunk connected; the branch pixel set is a subset of the pixel set of the pulmonary artery skeleton; obtaining a branch adjacency matrix representing the adjacency relationship of the pixels in the branch pixel set; A pulmonary artery class of a pixel in the set of branch pixels is determined using the extended classification feature and the branch adjacency matrix. 7. The method according to claim 6, wherein the step of using the extended classification feature and the branch adjacency matrix to determine the pulmonary artery category of the pixel in the branch pixel set comprises: Selecting a specified number of pixels in the branch pixel set as a reference pixel set; obtaining the extended classification features of the pixels in the reference pixel set; Pulmonary artery classes of pixels in the set of branch pixels are determined based on the extended classification feature and the branch adjacency matrix. 8. The method of claim 1, wherein the method further comprises: using the pixels in the pixel set as seed pixels; Based on the seed pixels, a preset region growing algorithm is used to determine the pulmonary artery category of the pixels representing the pulmonary artery in the pulmonary artery image sequence. 9. The method according to claim 8, wherein the method further comprises: The pulmonary artery category of the pixel representing the pulmonary artery in the pulmonary artery image sequence is dyed; wherein, different pulmonary artery categories correspond to different colors. 10. A training method for a pixel classification model, comprising: constructing a training sample; wherein, the training sample includes a pulmonary artery image sequence and a category label corresponding to a pixel representing a pulmonary artery in the pulmonary artery image sequence; acquiring a pixel set representing the pulmonary artery skeleton in the pulmonary artery image sequence, and an adjacency matrix representing the adjacency relationship between pixels in the pixel set of the pulmonary artery skeleton; extracting primary classification features of pixels in the pixel set of the pulmonary artery skeleton; wherein, the primary classification features are used to characterize the pixels in the pixel set of the pulmonary artery skeleton; The predicted pulmonary artery category of the pixels in the pixel set of the pulmonary artery skeleton determined based on the primary classification feature and the adjacency matrix; The pixel classification model is updated based on the loss generated by the predicted pulmonary artery class and the class label corresponding to the pixel representing the pulmonary artery. The method according to claim 10, wherein the pulmonary artery image sequence further corresponds to a pulmonary artery skeleton trunk label and a pulmonary artery skeleton branch label; wherein, the pulmonary artery skeleton branch label corresponds to a branch adjacency matrix; the method Also includes: Selecting a specified number of pixels in the pulmonary artery skeleton branch label as a reference pixel set; Obtaining extended classification features of pixels in the reference pixel set; wherein, the extended classification features are used to characterize pixels in the reference pixel set; Correspondingly, in the step of predicting the pulmonary artery type of the pixel in the pixel set determined based on the primary classification feature and the adjacency matrix, further comprising: A predicted pulmonary artery class for pixels in the set of pixels is determined based on the extended features and the branch adjacency matrix. 12. A device for classifying pixels, comprising: a pulmonary artery skeleton and an adjacency matrix acquisition module, used for acquiring a pixel set representing the pulmonary artery skeleton in the pulmonary artery image sequence, and an adjacency matrix representing the adjacency relationship between the pixels in the pixel set of the pulmonary artery skeleton; a primary classification feature extraction module, configured to extract primary classification features of pixels in the pixel set of the pulmonary artery skeleton; wherein the primary classification features are used to characterize the pixels in the pixel set of the pulmonary artery skeleton; A pulmonary artery skeleton classification module, configured to use the primary classification feature and the adjacency matrix to determine a pulmonary artery category to which a pixel in the pixel set of the pulmonary artery skeleton belongs. 13. A training device for a pixel classification model, comprising: A training sample building module for constructing a training sample; wherein, the training sample includes a pulmonary artery image sequence and a category label corresponding to a pixel representing a pulmonary artery in the pulmonary artery image sequence; a pulmonary artery skeleton and an adjacency matrix acquisition module, used for acquiring a pixel set representing the pulmonary artery skeleton in the pulmonary artery image sequence, and an adjacency matrix representing the adjacency relationship between the pixels in the pixel set of the pulmonary artery skeleton; a primary classification feature extraction module, configured to extract primary classification features of pixels in the pixel set of the pulmonary artery skeleton; wherein the primary classification features are used to characterize the pixels in the pixel set of the pulmonary artery skeleton; a pulmonary artery category prediction module, configured to predict the pulmonary artery category of the pixels in the pixel set of the pulmonary artery skeleton determined based on the primary classification feature and the adjacency matrix; A pixel classification model generation module, configured to update the pixel classification model according to the predicted pulmonary artery category and the loss generated by the class label corresponding to the pixel representing the pulmonary artery. 14. An electronic device, characterized in that the electronic device comprises: processor; a memory for storing the processor-executable instructions; The processor is configured to perform the method of any one of the above claims 1 to 11. 15. A computer-readable storage medium storing a computer program for performing the method of any one of the preceding claims 1 to 11.

Images