WO2022021955A1 - 图像分割方法及装置、图像分割模型的训练方法及装置 - Google Patents

图像分割方法及装置、图像分割模型的训练方法及装置 Download PDF

Info

Publication number
WO2022021955A1
WO2022021955A1 PCT/CN2021/088407 CN2021088407W WO2022021955A1 WO 2022021955 A1 WO2022021955 A1 WO 2022021955A1 CN 2021088407 W CN2021088407 W CN 2021088407W WO 2022021955 A1 WO2022021955 A1 WO 2022021955A1
Authority
WO
WIPO (PCT)
Prior art keywords
segmentation
image
segmentation result
neural network
background
Prior art date
Application number
PCT/CN2021/088407
Other languages
English (en)
French (fr)
Inventor
刘恩佑
王少康
陈宽
Original Assignee
推想医疗科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 推想医疗科技股份有限公司 filed Critical 推想医疗科技股份有限公司
Priority to JP2021560144A priority Critical patent/JP7250166B2/ja
Priority to EP21782431.7A priority patent/EP3975117A4/en
Priority to US17/497,954 priority patent/US11972571B2/en
Publication of WO2022021955A1 publication Critical patent/WO2022021955A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30061—Lung
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30101—Blood vessel; Artery; Vein; Vascular

Definitions

  • the present application relates to the technical field of image processing, and in particular, to an image segmentation method and device, and an image segmentation model training method and device.
  • Image segmentation is very useful in imaging diagnosis. For example, dividing pulmonary blood vessels into arteries/veins may help doctors accurately diagnose lung diseases that may affect the arterial or venous tree in specific ways.
  • the embodiments of the present application aim to provide an image segmentation method and device, and an image segmentation model training method and device, which can improve the accuracy and efficiency of segmentation of arteries and veins.
  • an image segmentation method comprising: acquiring the mediastinum of the mediastinal region of the to-be-segmented image according to the to-be-segmented image including the background, the mediastinum, the artery and the vein , the first segmentation result of the artery, the vein and the background; according to the to-be-segmented image, obtain the second segmentation result of the blood vessel and the background in the extension area of the to-be-segmented image; A segmentation result and the second segmentation result are obtained, and segmentation results of the mediastinum, the artery, the vein and the background of the image to be segmented are obtained.
  • a method for training an image segmentation model including: determining a sample image, the sample image including the mediastinum, the background, the first labels of the arteries and veins, and the extension of the mediastinal region The background of the region and the second label of the blood vessels; the neural network is trained based on the sample image to generate a network model for obtaining the first segmentation results of the mediastinum, background, arteries and veins of the mediastinal region, wherein the The neural network is a 3D neural network; a cascaded neural network is trained based on the sample image to generate a segmentation model for obtaining the second segmentation result of the background and blood vessels of the extension region, wherein the cascaded neural network
  • the network includes a first neural network for feature extraction and a second neural network for generating the second segmentation result.
  • an image segmentation apparatus including: a first segmentation module configured to acquire a longitudinal image of the to-be-segmented image according to the to-be-segmented image including the background, the mediastinum, the artery and the vein the first segmentation result of the mediastinum, the artery, the vein and the background in the diaphragm region; the second segmentation module is configured to acquire, according to the to-be-segmented image, the blood vessels in the extension area of the to-be-segmented image and the second segmentation result of the background; an acquisition module configured to acquire the mediastinum, the artery, the vein and the image to be segmented according to the first segmentation result and the second segmentation result Segmentation result of the background.
  • an apparatus for training an image segmentation model comprising: a determination module configured to determine a sample image, the sample image including the mediastinal, background, arterial and venous images of the mediastinal region a first label and a second label for the background and blood vessels of the extension region; a first training module configured to train a neural network based on the sample images to generate mediastinal, background, arteries and veins for obtaining the mediastinal region The network model of the first segmentation result, wherein the neural network is a 3D neural network; the second training module is configured to train a cascaded neural network based on the sample image to generate a background for obtaining the extension region and a segmentation model of the second segmentation result of the blood vessel, wherein the cascaded neural network includes a first neural network for feature extraction and a second neural network for generating the second segmentation result.
  • an electronic device including: a processor; a memory for storing instructions executable by the processor; the processor for executing the image described in any of the foregoing embodiments A segmentation method, and/or a training method for implementing the image segmentation model described in any of the above embodiments.
  • a computer-readable storage medium where the storage medium stores a computer program, and the computer program is used to execute the image segmentation method described in any of the foregoing embodiments, and/ Or a training method for executing the image segmentation model described in any of the above embodiments.
  • An image segmentation method provided by the embodiments of the present application separates the blood vessel segmentation task of the mediastinal region from the blood vessel segmentation task of the extension region, so as to obtain the mediastinal, arterial, vein and background images of the mediastinum region respectively.
  • the first segmentation result and the second segmentation result of the blood vessels and the background in the extension area, and then the segmentation results of the mediastinum, arteries, veins and background of the image to be segmented are obtained according to the first segmentation result and the second segmentation result, which can avoid direct segmentation.
  • the size of the blood vessels in the extension region and the blood vessels in the mediastinal region are inconsistent, which affects the segmentation of blood vessels of different sizes, thereby improving the accuracy and efficiency of segmentation of arteries and veins.
  • FIG. 1 is a schematic diagram of an implementation environment provided by an embodiment of the present application.
  • FIG. 2 is a schematic flowchart of an image segmentation method provided by an embodiment of the present application.
  • FIG. 3 is a schematic flowchart of an image segmentation method provided by another embodiment of the present application.
  • FIG. 4 is a schematic diagram illustrating an implementation process of area growth provided by an embodiment of the present application.
  • FIG. 5 is a schematic flowchart of a training method for an image segmentation model provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram illustrating a training process of a cascaded neural network provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a labeled sample image provided by an embodiment of the present application.
  • FIG. 8 is a block diagram of an image segmentation apparatus provided by an embodiment of the present application.
  • FIG. 9 shows a block diagram of an image segmentation apparatus provided by another embodiment of the present application.
  • FIG. 10 shows a block diagram of an image segmentation apparatus provided by yet another embodiment of the present application.
  • FIG. 11 shows a block diagram of an apparatus for training an image segmentation model provided by an embodiment of the present application.
  • FIG. 12 is a block diagram of an apparatus for training an image segmentation model provided by another embodiment of the present application.
  • FIG. 13 shows a block diagram of an apparatus for training an image segmentation model provided by yet another embodiment of the present application.
  • FIG. 14 shows a structural block diagram of an electronic device according to an embodiment of the present application.
  • Deep learning realizes artificial intelligence in computing systems by building artificial neural networks with hierarchical structures. Since the hierarchically structured artificial neural network can extract and filter the input information layer by layer, deep learning has the capability of representation learning and can realize end-to-end supervised learning and unsupervised learning.
  • the hierarchical artificial neural network used in deep learning has various forms, and the complexity of its hierarchy is commonly referred to as "depth". According to the type of construction, the form of deep learning includes multilayer perceptrons, convolutional neural networks, and recurrent neural networks. , Deep Belief Networks, and other hybrid constructs. Deep learning uses data to update the parameters in its construction to achieve training goals. This process is generally called "learning”. Deep learning proposes a method for computers to automatically learn pattern features and integrate feature learning into building models. In the process, thus reducing the incompleteness caused by human design features.
  • Neural network is an operation model, which is composed of a large number of nodes (or neurons) connected to each other, each node corresponds to a strategy function, and the connection between each two nodes represents a weighted value for the signal passing through the connection, Call it weight.
  • the neural network generally includes multiple neural network layers, the upper and lower network layers are cascaded with each other, the output of the i-th neural network layer is connected to the input of the i+1-th neural network layer, and the output of the i+1-th neural network layer is connected. Connect to the input of the i+2th neural network layer, and so on.
  • each neural network layer After the training samples are input to the cascaded neural network layers, each neural network layer outputs an output result, and the output result is used as the input of the next neural network layer. Thus, the output is obtained through multiple neural network layer calculations, and the output layer is compared. The output prediction result and the real target value, and then adjust the weight matrix and strategy function of each layer according to the difference between the prediction result and the target value.
  • the neural network uses the training samples to continuously go through the above adjustment process, making the neural network The weights and other parameters of the neural network are adjusted until the predicted results output by the neural network are consistent with the real target results. This process is called the training process of the neural network. After the neural network is trained, a neural network model can be obtained.
  • Existing vessel segmentation schemes are mainly divided into deep learning-based vessel segmentation and traditional learning-based vessel segmentation.
  • most of the existing blood vessel segmentation schemes are calculated by using the degree of discrimination of blood vessels on the HU value.
  • the effect of this scheme is acceptable on CT images without lesions, but , once in the CT images of pneumonia and nodules or tumors, the lesions with similar HU value of blood vessels will be cut out. Therefore, the robustness of this scheme is difficult to match the requirements of existing product usage scenarios.
  • FIG. 1 is a schematic diagram of an implementation environment provided by an embodiment of the present application.
  • the implementation environment includes CT scanner 130 , server 120 and computer equipment 110 .
  • the computer device 110 can acquire the lung medical images from the CT scanner 130, and at the same time, the computer device 110 can also be connected with the server 120 through a communication network.
  • the communication network is a wired network or a wireless network.
  • the CT scanner 130 is used to perform X-ray scanning on human tissue to obtain CT images of the human tissue.
  • lung medical images may be obtained by scanning the lungs with the CT scanner 130 .
  • the computer device 110 may be a general-purpose computer or a computer device composed of a dedicated integrated circuit, etc., which is not limited in this embodiment of the present application.
  • the computer device 110 may be a mobile terminal device such as a tablet computer, or may also be a personal computer (Personal Computer, PC) such as a laptop portable computer, a desktop computer, and the like.
  • PC Personal Computer
  • Those skilled in the art may know that the number of the above-mentioned computer devices 110 may be one or more, and their types may be the same or different.
  • the number of the above computer device 110 may be one, or the number of the above computer device 110 may be dozens or hundreds, or more.
  • the embodiments of the present application do not limit the number and device types of the computer devices 110 .
  • a network model and a segmentation model may be deployed in the computer device 110, where the network model is used to segment the mediastinum, arteries, veins and background of the mediastinal region of the lung medical image to obtain a first segmentation result, and the segmentation model is used for segmenting. Segment the blood vessels and the background of the extension area of the lung medical image to obtain a second segmentation result.
  • the computer device 110 can use the network model and the segmentation model deployed thereon to perform image segmentation on the lung medical image acquired from the CT scanner 130, thereby obtaining the first segmentation result and the second segmentation result, and further obtaining the pulmonary medical image. Segmentation results of mediastinum, arteries, veins, and background.
  • the size of the blood vessels in the extension region and the blood vessels in the mediastinal region are inconsistent when the arteries, veins and background are directly segmented, resulting in different effects.
  • the segmentation of size blood vessels can improve the accuracy and efficiency of segmentation of arteries and veins.
  • the server 120 is a server, or consists of several servers, or a virtualization platform, or a cloud computing service center.
  • the server 120 receives the training images collected by the computer device 110, and trains the neural network through the training images to obtain the mediastinum, arteries, A network model for segmentation of veins and backgrounds and a segmentation model for segmentation of blood vessels and backgrounds in extended regions of lung medical images.
  • the computer device 110 can send the lung medical image acquired from the CT scanner 130 to the server, and the server 120 uses the network model and the segmentation model trained on it to perform the mediastinal, arterial, Segmentation of veins and backgrounds, and segmentation of blood vessels and backgrounds in the extended region, to obtain the segmentation results of the mediastinum, arteries, veins and backgrounds of the lung medical image, and send the segmentation results to the computer device 110 for medical personnel to view .
  • the server 120 uses the network model and the segmentation model trained on it to perform the mediastinal, arterial, Segmentation of veins and backgrounds, and segmentation of blood vessels and backgrounds in the extended region, to obtain the segmentation results of the mediastinum, arteries, veins and backgrounds of the lung medical image, and send the segmentation results to the computer device 110 for medical personnel to view .
  • the server 120 uses the network model and the segmentation model trained on it to perform the mediastinal, arterial, Segmentation of veins and backgrounds, and segmentation of blood vessels and backgrounds in the extended region, to obtain
  • FIG. 2 is a schematic flowchart of an image segmentation method provided by an embodiment of the present application.
  • the method described in FIG. 2 is executed by a computing device (for example, a server), but the embodiment of the present application is not limited thereto.
  • the server may be one server, or be composed of several servers, or be a virtualization platform, or be a cloud computing service center, which is not limited in this embodiment of the present application.
  • the method includes the following.
  • S210 According to the image to be segmented including the background, the mediastinum, the artery and the vein, obtain a first segmentation result of the mediastinum, the artery, the vein and the background in the mediastinal region of the image to be segmented.
  • the image to be segmented may be a medical image such as Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Computed Radiography (CR), or Digital Radiography (DR). This embodiment of the present application does not specifically limit this.
  • CT Computed Tomography
  • MRI Magnetic Resonance Imaging
  • CR Computed Radiography
  • DR Digital Radiography
  • the to-be-segmented image may be a lung medical image, but this is not specifically limited in the embodiments of the present application, and the to-be-segmented image may also be a medical image of other organs, as long as the medical image can combine large blood vessels and small blood vessels. It is sufficient to distinguish by means of area division, for example, the mediastinal area and the epitaxial area in the embodiments of the present application.
  • the embodiments of the present application also do not limit the specific form of the image to be segmented, which may be an original medical image, a preprocessed medical image, or a part of the original medical image.
  • the mediastinal region refers to the region near the left and right mediastinal pleura, in which there are the heart and the great blood vessels that enter and exit the heart, esophagus, trachea, thymus, nerves and lymphoid tissues; the extension region refers to outside the mediastinal region. the area containing blood vessels.
  • the mediastinal region refers to the region near the pleura of the left and right mediastinum
  • the extension region refers to the intrapulmonary region outside the mediastinal region. Vessels in the mediastinal region are larger in size than in the epitaxial region.
  • the first segmentation may be performed on the image to be segmented including the background, arteries and veins to obtain the first segmentation result of the mediastinum, arteries, veins and background in the mediastinal region, but it should be noted that this The application examples do not limit the specific implementation means of the first division.
  • S220 According to the image to be segmented, obtain a second segmentation result of the blood vessel and the background in the extension region of the image to be segmented.
  • the second segmentation may be performed on the to-be-segmented image to obtain the second segmentation result of the blood vessels and the background in the extension region, but it should be noted that the embodiment of the present application does not limit the specific implementation means of the second segmentation.
  • the second segmentation of the to-be-segmented image can separate the blood vessels from the background in the extension area of the to-be-segmented image, but does not classify the blood vessels, that is, does not distinguish whether the blood vessels are arteries or veins, as long as the blood vessels and the background can be separated. Can.
  • the embodiments of the present application also do not limit whether the specific implementation means of the first split and the second split are the same, and the two may be the same or different; and the embodiments of the present application do not limit the order in which the first split and the second split are performed.
  • the first segmentation may be performed first, the second segmentation may be performed first, or the first segmentation and the second segmentation may be performed simultaneously, as long as the respective segmentation results can be obtained.
  • S230 According to the first segmentation result and the second segmentation result, obtain segmentation results of the mediastinum, artery, vein, and background of the image to be segmented.
  • the first segmentation result and the second segmentation result may be processed to obtain the segmentation results of the mediastinum, arteries, veins, and background, but the embodiment of the present application does not limit how the first segmentation result and the second segmentation result are processed.
  • the two segmentation results are processed, as long as the final segmentation results of the background, arteries and veins can be obtained.
  • the first segmentation result refers to the segmentation result of the mediastinal region of the image to be segmented
  • the second segmentation result refers to the segmentation result of the extension region of the image to be segmented.
  • the first segmentation result and the second segmentation result can be simply superimposed to obtain the segmentation results of the mediastinum, arteries, veins and background of the image to be segmented;
  • the intermediate result is then processed to obtain segmentation results for the mediastinum, arteries, veins, and background.
  • the first segmentation results of the mediastinum, arteries, veins and background in the mediastinal region and the blood vessels in the extension region and the background are obtained respectively.
  • the second segmentation result of the background and then according to the first segmentation result and the second segmentation result, the segmentation results of the mediastinum, arteries, veins and background of the image to be segmented are obtained, which can avoid the extension when directly segmenting the arteries, veins and background.
  • the size of the blood vessels in the region and the blood vessels in the mediastinal region are inconsistent, which affects the segmentation of blood vessels of different sizes, so that the accuracy and efficiency of the segmentation of arteries and veins can be improved.
  • the image to be segmented may be input into a network model for obtaining first segmentation results of the mediastinum, arteries, veins and background of the mediastinal region for segmentation.
  • the embodiment of the present application does not limit the specific type of the network model, and the network model may be composed of any type of neural network.
  • the network model may be a convolutional neural network (Convolutional Neural Network, CNN), a deep neural network (Deep Neural Network, DNN), or a recurrent neural network (Recurrent Neural Network, RNN) or the like.
  • the network model may include neural network layers such as an input layer, a convolution layer, a pooling layer, and a connection layer, which are not specifically limited in this embodiment of the present application.
  • the embodiments of the present application do not limit the number of each neural network layer.
  • S320 According to the image to be segmented, through the network model, obtain a first segmentation result of the mediastinum, artery, vein, and background in the mediastinal region of the image to be segmented.
  • the image to be segmented is directly input into the network model to obtain the first segmentation result of the mediastinum, arteries, veins and background of the mediastinal region of the image to be segmented.
  • the first segmentation result of the mediastinum, arteries, veins and background of the mediastinal region of the to-be-segmented image can be made more accurate.
  • inputting the image to be segmented into the network model includes: performing a segmenting operation on the mediastinal region of the image to be segmented to obtain multiple segmented images, wherein each segmented image in the multiple segmented images All include the mediastinum; a plurality of segmented images are input into the network model, wherein, according to the images to be segmented, the network model is used to obtain the first segmentation results of the mediastinum, arteries, veins and background of the mediastinum region of the image to be segmented , including: obtaining multiple sub-segmentation results of the mediastinum, arteries, veins and background of the mediastinal region corresponding to the multiple segmented images through the network model; A combined operation is performed to obtain a combined segmentation result; a first segmentation result is obtained by post-processing the combined segmentation result through a connected domain algorithm.
  • the network model may be a 3D network model or a 2D network model, which is not specifically limited in this embodiment of the present application, and those skilled in the art can design specific types of network models according to actual application requirements.
  • 3D network models are widely used in the field of medical image segmentation due to their high accuracy and excellent 3D performance.
  • the 3D segmentation network model consumes a lot of computing resources. If the complete image to be segmented (ie, the original medical image) is directly input into the 3D segmentation network model, the video memory required for training the 3D segmentation network model will be very large.
  • the mediastinal region of the image to be segmented can be segmented, that is, the segmented image corresponding to the mediastinal region can be segmented to obtain a segment corresponding to the mediastinal region.
  • the multiple sliced images can be overlapping sliced images, which are then input into the network model for segmentation.
  • the embodiments of the present application do not limit the number of segmented images to be segmented, nor do they limit the size of the overlap between two adjacent segmented images.
  • the sliced image contains the mediastinum, which is helpful for the classification of blood vessels in the mediastinal region, that is, the mediastinum can be used as a reference for the 3D network model to learn the classification of arteries and veins, so that the 3D network model can better understand the arteries. and veins for class judgment.
  • the multiple segmented images are input into the network model for image segmentation.
  • a sub-segmentation result can be output, that is, one segmented image corresponds to one sub-segmentation result, then multiple segmented images correspond to multiple sub-segmentation results, and multiple sub-segmentation results.
  • multiple sub-segmentation results are combined into a mask with a size corresponding to the size of the image to be segmented, as the first segmentation result. Since multiple segmented images are overlapping segmented images, in order to ensure the smoothness of the boundary of the first segmentation result obtained after combination, only the segmentation result at the center position can be retained, and the segmentation result at the center position can be combined. . That is, in order to obtain the first segmentation result of the mediastinum, arteries, veins, and background of the mediastinal region of the image to be segmented with smooth boundaries, Gaussian smoothing can be used to combine multiple sub-segmentation results to obtain a smooth boundary of the mediastinum. Results of the first segmentation of the mediastinum, arteries, veins, and background in the diaphragmatic region.
  • the simplest combining operation can be to stitch together multiple tiled images directly, which is simple and fast, but can bring about a picket fence effect (i.e., since the network model does not perform well near the boundaries of tiled images, the consistency Poor, when the corresponding cutting results of two adjacent sliced images are combined, there will be obvious combination traces). Since the segmentation results of the network model at the center of the sliced image are more credible and perform better, we can only keep the segmentation results near the center of the sliced image. For example, the size of the sliced image is 192*192*64, but You can only keep the segmentation results of the 160*160*48 area near the center position.
  • the step size of the segmentation can be modified on the basis of the combined operation of direct splicing, and the overlapping area of the segmentation results of the two 160*160*48 areas can be averaged.
  • the disadvantage of this combination operation is that it does not use the previous a priori assumption, therefore, the combination operation of Gaussian smoothing can be used to overcome this disadvantage, so that the accuracy of the segmentation results is higher.
  • the Gaussian function can be used, and the center position of the sliced image is taken as the mean value of the Gaussian kernel, and the Gaussian weighting method is used for smoothing. In this way, it just fits the prior knowledge of the high confidence of the network model at the center position of the sliced image. Better smoothing of the first segmentation results obtained by combining operations.
  • the embodiment of the present application does not limit the function used in the Gaussian smoothing process, and a Gaussian function or other bell-shaped functions may be used.
  • the combined segmentation results can be post-processed through the connected domain algorithm to obtain the first segment.
  • a split result the embodiments of the present application do not limit the specific implementation means of the post-processing, as long as the false positive and the point where the junction is inconsistent can be removed.
  • performing post-processing on the combined segmentation results through a connected domain algorithm to obtain a first segmentation result includes: obtaining, through a connected domain algorithm, the maximum connected region of the vein and the maximum connected region of the artery in the combined segmentation result.
  • Connected domain According to the maximum connected domain of the vein and the maximum connected domain of the artery, remove the noise in the combined segmentation result to obtain the first segmentation result, where the noise includes points that are both arteries and veins and false positive points.
  • the combined segmentation results are processed in a connected domain to obtain the maximum connected domain, which includes the largest connected domain of arteries and the largest connected domain of veins.
  • the arterial maximum connected area and the vein maximum connected area in the combined segmentation result are removed to obtain the noise in the combined segmentation result.
  • the noise is removed from the combined segmentation result, and the first segmentation result can be obtained. In this way, the obtained first segmentation result does not include points that are both arteries and veins or false positive points that do not meet the requirements.
  • the noise points may include points that are both arteries and veins and false positive points, which are not specifically limited in this embodiment of the present application, and the noise points may also be other points that do not meet the requirements.
  • the to-be-segmented image may be input into a segmentation model for obtaining a second segmentation result of the blood vessels and the background of the extended region for segmentation.
  • the segmentation model may be composed of any type of neural network.
  • the segmentation model may be a convolutional neural network (Convolutional Neural Network, CNN), a deep neural network (Deep Neural Network, DNN), or a recurrent neural network (Recurrent Neural Network, RNN) or the like.
  • the segmentation model may include neural network layers such as an input layer, a convolution layer, a pooling layer, and a connection layer, which are not specifically limited in this embodiment of the present application.
  • the embodiments of the present application do not limit the number of each neural network layer.
  • the to-be-segmented image is directly input into the segmentation model to obtain a second segmentation result of the blood vessels and the background in the extended region of the to-be-segmented image.
  • the second segmentation result of the blood vessels and the background in the extension region of the to-be-segmented image can be made more accurate.
  • the segmentation model can separate the blood vessels from the background in the extension area of the image to be segmented, but does not classify the blood vessels, that is, does not distinguish whether the blood vessels are arteries or veins, as long as the blood vessels and the background can be separated.
  • the extension area refers to the area in the lung. Since blood vessels are easier to identify in the area of the lung, the segmentation model can adopt some lightweight model structures.
  • the segmentation model may be a 2D segmentation model, but this embodiment of the present application does not specifically limit this, and those skilled in the art can design the specific type of the network model according to actual application requirements. Meanwhile, the embodiments of the present application do not specifically limit the specific model structure of the segmentation model. Those skilled in the art can design the specific model structure of the segmentation model according to actual application requirements.
  • the segmentation model can be composed of ResNet18 and a feature pyramid network.
  • the size of the sliced image containing the mediastinum correlates with the correctness of the classification of arteries and veins in the extension region.
  • the CT physical resolution of the sliced image ie, the Pixelspacing resolution, the higher the Pixelspacing resolution, the higher the restoration of the real physical world, the lower the Pixelspacing resolution, the lower the restoration of the real physical space
  • the segmentation of blood vessels with a small size of the region that is, the higher the CT physical resolution of the slice image, is more helpful for the segmentation of blood vessels with a small size of the extension region.
  • the physical volume and CT physical resolution are inversely proportional.
  • the segmentation of blood vessels and the accuracy of classification of arteries and veins require increasing the size of the sliced image.
  • the CT physical resolution of the sliced image is scaled by 0.5
  • each side of the sliced image will be enlarged by 2 times. In this way, the size of the sliced image and the The size of the 3D network model will be enlarged to 8 times the original size.
  • the size of the segmented image needs to be increased by more than 8 times, which obviously increases the memory required for training the 3D segmentation network model, thereby reducing the The segmentation efficiency of the 3D network model to be segmented.
  • the 2D segmentation model is responsible for the segmentation accuracy of vessels with smaller sizes in the epitaxial region
  • the 3D network model is responsible for the size of the mediastinal region. Correctness of classification and segmentation of larger vessels.
  • the size of the sliced image can be reduced, that is, the segmentation task of the 3D network model can be simplified; at the same time, the accuracy of segmentation and classification of the mediastinum and blood vessels close to the mediastinum can also be guaranteed, that is, only the sliced
  • the physical volume of the block image is large enough to obtain a better segmentation and classification effect of the mediastinum and blood vessels close to the mediastinum.
  • the video memory is reduced by 2 times, and the video memory is reduced by 2 times, so as to obtain the same segmentation effect and classification effect of the mediastinum and blood vessels close to the mediastinum.
  • the segmentation efficiency of the 3D network model to be segmented is the same.
  • Figure 4 shows the implementation process of region growth.
  • Region Growth can combine the arteries and veins in the first segmentation result with the blood vessels in the second segmentation result to obtain segmentation results of the mediastinum, arteries, veins and background of the image to be segmented.
  • cuda implementation can be adopted.
  • the region growing algorithm is to group the pixels with similarity to form the final region. First, find a seed pixel as the starting point of growth for each region to be segmented, and then merge the pixels with the same or similar properties as the seed in the neighborhood around the seed pixel (determined according to the pre-determined growth or similar criteria) into the seed in the area where the pixel is located. And new pixels continue to grow around as seeds until no more pixels that meet the conditions can be included, and a final region is grown.
  • the segmentation results of the mediastinum, arteries, veins and background of the image to be segmented are obtained through a region growing algorithm, It includes: starting from the arteries and veins in the first segmentation result, along the arteries and veins in the first segmentation result along the blood vessels in the second segmentation result, and performing region growth with a preset blood vessel growth length to obtain the image to be segmented segmentation results of the mediastinum, arteries, veins and background.
  • the second segmentation result does not classify the arteries and veins in the extended area.
  • the blood vessels in the diaphragm area will overlap, but the blood vessels in the lung area will not. Therefore, the arteries and veins in the first segmentation result can be used as the starting point of regional growth, and the blood vessels in the second segmentation result can be used as the trajectory of regional growth.
  • the arteries and veins in the first segmentation result are further extended along the blood vessels in the second segmentation result to the extension region, that is, the arteries and veins in the first segmentation result are made along the second segmentation result. vascular growth.
  • the determined arteries and veins are taken as the starting point.
  • the arteries and veins in the extended region after the region has grown can be classified; at the same time, since the arteries and veins are the Two complete connected domains, and the false positives in the second segmentation result will not be combined with these two complete connected domains, so by region growth, the false positives in the second segmentation result can also be removed.
  • the preset blood vessel growth length can be set when the arteries and veins in the first segmentation result are regionally increased along the blood vessels in the second segmentation result (For example, for a lung medical image, the preset blood vessel growth length is a preset intrapulmonary blood vessel growth length).
  • the granularity of blood vessels can be dynamically displayed by adjusting the preset blood vessel growth length in each iteration process. This can avoid the situation that some lesions are occluded by blood vessels when medical staff view the VR of the segmented image, thereby increasing the user experience.
  • the embodiment of the present application does not limit the specific value of the preset blood vessel growth length in each iteration process, and can be selected according to different application requirements.
  • the segmentation task of arteries and veins can be disassembled into three sub-tasks, and two models are used to obtain
  • the first segmentation result and the second segmentation result can reduce the complexity of the task, and then some simple model structures can be used to reduce the video memory and speed up the prediction speed, so as to meet the real-time and resource scheduling requirements of online products.
  • FIG. 5 is a schematic flowchart of a training method for an image segmentation model provided by an embodiment of the present application.
  • the method described in FIG. 5 is executed by a computing device (for example, a server), but the embodiment of the present application is not limited thereto.
  • the server may be one server, or be composed of several servers, or be a virtualization platform, or be a cloud computing service center, which is not limited in this embodiment of the present application.
  • the method includes the following contents.
  • S510 Determine a sample image, where the sample image includes the mediastinum, the background, the first labels of the arteries and veins in the mediastinal region, and the background of the extension region and the second label of the blood vessels.
  • the first label refers to the label obtained by labeling the mediastinum, background, arteries and veins in the mediastinal region of the sample image;
  • the second label refers to the label obtained by labeling the background and blood vessels in the extension region of the sample image
  • the resulting labels, the vessels in this second label do not specifically demarcate arteries and veins.
  • the sample image mentioned in this embodiment and the image to be divided in the above-mentioned embodiment belong to the same type of image.
  • the sample image is manually marked to obtain the first label and the second label.
  • the embodiment of the present application does not limit the specific form of the sample image, which may be an original medical image, a pre-processed medical image, or a part of the original medical image.
  • the mediastinal region and the extension region mentioned in this embodiment are the same as the mediastinal region and the extension region in the embodiment of the image segmentation method above, and the specific details are not repeated here. Please refer to the embodiment of the image segmentation method above. .
  • S520 Train a neural network based on the sample images to generate a network model for obtaining the first segmentation result of the mediastinum, background, arteries and veins of the mediastinal region, where the neural network is a 3D neural network.
  • the sample images are input into a neural network, and the neural network is trained to generate the network model.
  • the neural network being trained can be any type of neural network.
  • the trained neural network may be a convolutional neural network (Convolutional Neural Network, CNN), a deep neural network (Deep Neural Network, DNN), or a recurrent neural network (Recurrent Neural Network, RNN), etc.
  • the specific type of neural network to be trained is not limited.
  • the neural network to be trained may include neural network layers such as an input layer, a convolution layer, a pooling layer, and a connection layer, which are not specifically limited in this embodiment of the present application.
  • the embodiments of the present application do not limit the number of each neural network layer.
  • the trained neural network is a 3D neural network.
  • S530 Train a cascaded neural network based on the sample image to generate a segmentation model for obtaining a second segmentation result of the background and blood vessels of the extension region, wherein the cascaded neural network includes a first neural network for feature extraction and a second neural network for generating a second segmentation result.
  • the sample images are input into a cascaded neural network, and the cascaded neural network is trained to generate the segmentation model.
  • the first neural network and the second neural network can be any type of neural network.
  • the first neural network and the second neural network may be a convolutional neural network (Convolutional Neural Network, CNN), a deep neural network (Deep Neural Network, DNN) or a recurrent neural network (Recurrent Neural Network, RNN), etc.
  • the embodiments of the present application do not limit the specific types of the first neural network and the second neural network.
  • the first neural network and the second neural network may include neural network layers such as an input layer, a convolution layer, a pooling layer, and a connection layer, which are not specifically limited in this embodiment of the present application.
  • the embodiments of the present application do not limit the number of each neural network layer.
  • the cascaded neural network includes a first neural network for feature extraction and a second neural network behind the first neural network for generating a second segmentation result.
  • the embodiments of the present application do not specifically limit the specific structure of the cascaded neural network, and the cascaded neural network may also include other neural networks.
  • the embodiments of the present application do not limit the sequence of training the network model and training the segmentation model.
  • the network model may be trained first, the segmentation model may also be trained first, or the network model and the segmentation model may be trained at the same time, as long as the trained network model can be obtained. and the segmentation model.
  • the method further includes: performing a segmenting operation on the mediastinal region of the sample image to obtain multiple segmented images, wherein each segmented image in the multiple segmented images includes mediastinum, wherein training a neural network based on the sample images to generate a network model for obtaining a first segmentation result of the mediastinum, background, arteries and veins of the mediastinal region, comprising: training a neural network based on a plurality of sliced images network to generate the network model.
  • the dicing operation mentioned in this embodiment is the same as the dicing operation in the embodiment of the above-mentioned image segmentation method, and the specific details are not repeated here. Please refer to the embodiment in the above-mentioned image segmentation method.
  • a plurality of sliced images are respectively input into a 3D neural network, and the 3D neural network is trained to generate the network model.
  • the embodiments of the present application do not limit the training process of the 3D neural network, as long as a network model for obtaining the first segmentation result of the mediastinum, arteries, veins and background of the mediastinal region can be formed.
  • the first loss function value of the 3D neural network can be obtained.
  • the smaller the value of the first loss function the closer the predicted first segmentation result is to the target result, and the higher the accuracy of the correct prediction.
  • the larger the value of the first loss function the lower the accuracy of the correct prediction.
  • the gradient back-propagation of the first loss function value is performed to update the parameters of the 3D neural network, such as weight, bias, etc., which is not limited in this application.
  • training a cascaded neural network based on a sample image to generate a segmentation model for obtaining a second segmentation result of the background and blood vessels in the extension region includes: using the first neural network to perform a Perform downsampling operations to obtain multiple first feature maps; perform upsampling and fusion operations on multiple first feature maps through the second neural network to obtain second feature maps; use a classifier to perform Activate to obtain the second segmentation result of the background and blood vessels of the extension area; obtain the loss function value of the cascaded neural network according to the second segmentation result and the second label; update the parameters of the cascaded neural network according to the loss function value , where the first neural network is a deep residual network, and the second neural network is a feature pyramid network.
  • FIG. 6 shows an example of the training process of the cascaded neural network, as follows.
  • Input the sample image into the deep residual network perform downsampling operation on it, and generate a plurality of first feature maps, that is, the first feature map 1, the first feature map 2, the first feature map 3 and the first feature map 4.
  • first feature maps that is, the first feature map 1, the first feature map 2, the first feature map 3 and the first feature map 4.
  • the embodiment of the present application does not specifically limit the number of the first feature maps, and the embodiment of the present application does not limit the multiple of downsampling.
  • the plurality of first feature maps are respectively input into the feature pyramid network, and up-sampling and fusion operations are performed on the plurality of first feature maps to generate a second feature map.
  • the first feature map 4 is input into the feature pyramid network, and the dimension of the first feature map 3 is reduced and then input into the feature pyramid network, so that it performs a fusion operation with the first feature map 4 to obtain the fusion operation.
  • the training process shown in FIG. 6 is just an example of the training process of the cascaded neural network, and is not used to limit the present application.
  • the loss function of the cascaded neural network can be obtained by calculating the similarity loss between the second segmentation result and the second label (ie, the target result of the background and the blood vessel) using the loss function.
  • the smaller the loss function value the closer the predicted second segmentation result is to the target result, and the higher the accuracy of the correct prediction. Conversely, the larger the loss function value, the lower the accuracy of the correct prediction.
  • the loss function value of the cascaded neural network is backpropagated by gradient to update the parameters of the cascaded neural network, such as weight, bias, etc., which is not limited in this application.
  • the method further includes: performing a maximum pooling operation on the region where the blood vessel is located in the second label, to obtain a target region of the sample image after the region where the blood vessel is located in the second label is expanded, wherein, Obtaining the loss function value of the cascaded neural network according to the second segmentation result and the second label includes: obtaining the loss function value of the cascaded neural network according to the second segmentation result and the second label corresponding to the target area.
  • Class imbalance will cause the model to be more inclined to learn a large number of samples, that is, negative samples (background).
  • the marked area where the positive sample is located can be "inflated” through the max pooling operation, as shown in Figure 7.
  • the left picture is the original marking result of the second label, in which the band The area with the white label is the blood vessel, and the rest of the black area is the background; the picture on the right is the labeling result after "expansion", in which the entire white area is the target area obtained by expanding the area with the white label, including the picture on the left A white-labeled area in and a black area adjacent to this area (ie, the background).
  • the imbalance problem of positive and negative samples can be effectively reduced, the blood vessel segmentation can be made more detailed, and at the same time, the convergence of the segmentation model can be accelerated.
  • the maximum pooling operation is performed on the region where the blood vessel in the second label is located (that is, the region with the white label), so as to obtain the target region (That is, the "inflated" target area).
  • the target area includes vessel labels and background labels adjacent to the vessel labels.
  • the similarity loss between the second segmentation result corresponding to the target area and the second label of the sample image is calculated, and the loss function value of the cascaded neural network can be obtained.
  • the loss function value of the cascaded neural network can be obtained.
  • the apparatus embodiments of the present application may be used to execute the method embodiments of the present application.
  • details not disclosed in the device embodiments of the present application please refer to the method embodiments of the present application.
  • FIG. 8 is a block diagram of an image segmentation apparatus provided by an embodiment of the present application. As shown in FIG. 8, the apparatus 800 includes:
  • the first segmentation module 810 is configured to obtain a first segmentation result of the mediastinum, arteries, veins and background in the mediastinal region of the image to be segmented according to the image to be segmented including the background, the mediastinum, the artery and the vein;
  • the second segmentation module 820 is configured to obtain a second segmentation result of the blood vessels and the background in the extension region of the image to be segmented according to the image to be segmented;
  • the acquiring module 830 is configured to acquire segmentation results of the mediastinum, artery, vein and background of the image to be segmented according to the first segmentation result and the second segmentation result.
  • the apparatus 800 further includes: a first input module 840 configured to input the image to be segmented into the network model.
  • the first segmentation module 810 is further configured to: obtain the first segmentation result of the mediastinum, arteries, veins and background of the mediastinal region of the image to be segmented through a network model according to the image to be segmented.
  • the first input module 840 is further configured to: perform a segmenting operation on the mediastinal region of the image to be segmented to obtain multiple segmented images, wherein each segmented image in the multiple segmented images is Mediastinum included; multiple sliced images were fed into the network model.
  • the first segmentation module 810 when the first segmentation module 810 obtains the first segmentation result of the mediastinum, arteries, veins and background in the mediastinal region of the image to be segmented through the network model according to the image to be segmented, the first segmentation module 810 is further configured to: For multiple segmented images, through the network model, multiple sub-segmentation results of the mediastinum, arteries, veins and background in the mediastinal region corresponding to the multiple segmented images are obtained; through Gaussian smoothing, the multiple sub-segmentation results are combined. In order to obtain the combined segmentation result; through the connected domain algorithm, the combined segmentation result is post-processed to obtain the first segmentation result.
  • the first segmentation module 810 when the first segmentation module 810 performs post-processing on the combined segmentation results through a connected domain algorithm to obtain the first segmentation result, the first segmentation module 810 is further configured to: obtain the combined segmentation results through the connected domain algorithm.
  • the maximum connected domain of veins and the maximum connected domain of arteries are obtained; according to the maximum connected domain of veins and the maximum connected domain of arteries, the noise points in the combined segmentation result are removed to obtain the first segmentation result, wherein the noise points include points that are both arteries and veins and false positives.
  • the apparatus 800 further includes: a second input module 850 configured to input the image to be segmented into the segmentation model.
  • the second segmentation module 820 is further configured to: obtain the second segmentation result of the blood vessel and the background in the extension region through the segmentation model according to the image to be segmented.
  • the obtaining module 830 is further configured to: obtain the mediastinum, arteries, veins and background of the image to be segmented through a region growing algorithm according to the arteries and veins in the first segmentation result and the blood vessels in the second segmentation result segmentation result.
  • the acquisition module 830 when obtaining the segmentation results of the mediastinum, arteries, veins and background of the image to be segmented through the region growing algorithm, is further configured to: take the arteries and veins in the first segmentation result as the starting point, The arteries and veins of the first segmentation result are extended along the blood vessels in the second segmentation result with a preset blood vessel growth length to obtain segmentation results of the mediastinum, arteries, veins and background of the image to be segmented.
  • FIG. 11 shows a block diagram of an apparatus for training an image segmentation model provided by an embodiment of the present application.
  • the device 1100 includes:
  • the determining module 1110 is configured to determine a sample image, the sample image includes the mediastinum, the background, the first labels of the arteries and veins in the mediastinal region, and the second label of the background and blood vessels in the extension region;
  • the first training module 1120 is configured to train a neural network based on the sample images to generate a network model for obtaining the first segmentation results of the mediastinum, background, arteries and veins of the mediastinal region, wherein the neural network is a 3D neural network.
  • the internet
  • the second training module 1130 is configured to train a cascaded neural network based on the sample image, so as to generate a segmentation model for obtaining a second segmentation result of the background and blood vessels of the extension region, wherein the cascaded neural network includes a A first neural network for feature extraction and a second neural network for generating a second segmentation result.
  • the apparatus 1100 further includes: a dicing module 1140, configured to perform a dicing operation on the mediastinal region of the sample image to obtain a plurality of diced images, wherein the plurality of diced Each slice image in the slice image includes the mediastinum.
  • a dicing module 1140 configured to perform a dicing operation on the mediastinal region of the sample image to obtain a plurality of diced images, wherein the plurality of diced Each slice image in the slice image includes the mediastinum.
  • the first training module 1120 is further configured to train a neural network based on the plurality of sliced images to generate a network model.
  • the second training module 1130 is further configured to: perform a downsampling operation on the sample image through the first neural network to obtain multiple first feature maps; through the second neural network, perform a down sampling operation on the multiple first feature maps Perform up-sampling and fusion operations on the image to obtain the second feature map; use the classifier to activate the second feature map to obtain the background of the extension region and the second segmentation result of the blood vessels; according to the second segmentation result and the second label, Obtain the loss function value of the cascaded neural network; update the parameters of the cascaded neural network according to the loss function value, wherein the first neural network is a deep residual network, and the second neural network is a feature pyramid network.
  • the apparatus 1100 further includes: a maximum pooling module 1150, configured to perform a maximum pooling operation on the region where the blood vessel in the second label is located to obtain the location where the blood vessel in the second label is located The target region of the sample image after region dilation.
  • a maximum pooling module 1150 configured to perform a maximum pooling operation on the region where the blood vessel in the second label is located to obtain the location where the blood vessel in the second label is located The target region of the sample image after region dilation.
  • the second training module 1130 when obtaining the loss function value of the cascaded neural network according to the second segmentation result and the second label, is further configured to: according to the second segmentation result corresponding to the target area and the second label, get the loss function value of the cascaded neural network.
  • FIG. 14 illustrates a block diagram of an electronic device according to an embodiment of the present application.
  • electronic device 1400 includes one or more processors 1410 and memory 1420 .
  • Processor 1410 may be a central processing unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in electronic device 1400 to perform desired functions.
  • CPU central processing unit
  • Processor 1410 may be a central processing unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in electronic device 1400 to perform desired functions.
  • Memory 1420 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory.
  • the volatile memory may include, for example, random access memory (RAM) and/or cache memory, or the like.
  • the non-volatile memory may include, for example, read only memory (ROM), hard disk, flash memory, and the like.
  • One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1410 may execute the program instructions to implement the image segmentation method and image segmentation model of the various embodiments of the present application described above training methods and/or other desired features.
  • Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.
  • the electronic device 1400 may also include an input device 1430 and an output device 1440 interconnected by a bus system and/or other form of connection mechanism (not shown).
  • the input device 1430 may be the above-mentioned microphone or microphone array for capturing the input signal of the sound source.
  • the input device 1430 may be a communication network connector.
  • the input device 1430 may also include, for example, a keyboard, a mouse, and the like.
  • the output device 1440 can output various information to the outside, including the determined symptom category information and the like.
  • the output devices 1440 may include, for example, displays, speakers, printers, and communication networks and their connected remote output devices, among others.
  • the electronic device 1400 may also include any other suitable components according to the specific application.
  • embodiments of the present application may also be computer program products comprising computer program instructions that, when executed by a processor, cause the processor to perform the "exemplary method" described above in this specification
  • the computer program product can write program codes for performing the operations of the embodiments of the present application in any combination of one or more programming languages, including object-oriented programming languages, such as Java, C++, etc. , also includes conventional procedural programming languages, such as "C" language or similar programming languages.
  • the program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server execute on.
  • embodiments of the present application may also be computer-readable storage media having computer program instructions stored thereon, the computer program instructions, when executed by a processor, cause the processor to perform the above-mentioned "Exemplary Method" section of this specification
  • the computer-readable storage medium may employ any combination of one or more readable media.
  • the readable medium may be a readable signal medium or a readable storage medium.
  • the readable storage medium may include, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses or devices, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Medical Informatics (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
  • Image Analysis (AREA)

Abstract

本申请公开了一种图像分割方法及装置、图像分割模型的训练方法及装置。该图像分割方法包括：根据包括背景、纵膈、动脉和静脉的待分割图像，获取待分割图像的纵膈区域的纵膈、动脉、静脉和背景的第一分割结果；根据待分割图像，获取待分割图像的外延区域的血管与背景的第二分割结果；根据第一分割结果和第二分割结果，获取待分割图像的纵膈、动脉、静脉和背景的分割结果，能够提高动脉与静脉的分割的准确性和分割的效率。

Description

图像分割方法及装置、图像分割模型的训练方法及装置
本申请要求2020年7月30日提交的申请号为202010752149.0的中国申请的优先权，通过引用将其全部内容并入本文。
技术领域
本申请涉及图像处理技术领域，具体涉及一种图像分割方法及装置、图像分割模型的训练方法及装置。
背景技术
图像分割在影像学诊断中大有用处。例如，将肺血管分为动脉/静脉可能有助于医生准确诊断可能以特定方式影响动脉或静脉树的肺部疾病。
发明内容
有鉴于此，本申请的实施例致力于提供一种图像分割方法及装置、图像分割模型的训练方法及装置，能够提高动脉与静脉的分割的准确性和分割的效率。
根据本申请实施例的第一方面，提供了一种图像分割方法，包括：根据包括背景、纵膈、动脉和静脉的待分割图像，获取所述待分割图像的纵膈区域的所述纵膈、所述动脉、所述静脉和所述背景的第一分割结果；根据所述待分割图像，获取所述待分割图像的外延区域的血管与所述背景的第二分割结果；根据所述第一分割结果和所述第二分割结果，获取所述待分割图像的所述纵膈、所述动脉、所述静脉和所述背景的分割结果。
根据本申请实施例的第二方面，提供了一种图像分割模型的训练方法，包括：确定样本图像，所述样本图像包括纵膈区域的纵膈、背景、动脉和静脉的第一标签以及外延区域的背景和血管的第二标签；基于所述样本图像训练神经网络，以生成用于获得所述纵膈区域的纵膈、背景、动脉和静脉的第一分割结果的网络模型，其中，所述神经网络为3D神经网络；基于所述样本图像训练级联的神经网络，以生成用于获得所述外延区域的背景和血管的第二分割结果的分割模型，其中，所述级联的神经网络包括用于特征提取的第一神经网络以及用于生成所述第二分割结果的第二神经网络。
根据本申请实施例的第三方面，提供了一种图像分割装置，包括：第一 分割模块，配置为根据包括背景、纵膈、动脉和静脉的待分割图像，获取所述待分割图像的纵膈区域的所述纵膈、所述动脉、所述静脉和所述背景的第一分割结果；第二分割模块，配置为根据所述待分割图像，获取所述待分割图像的外延区域的血管与所述背景的第二分割结果；获取模块，配置为根据所述第一分割结果和所述第二分割结果，获取所述待分割图像的所述纵膈、所述动脉、所述静脉和所述背景的分割结果。
根据本申请实施例的第四方面，提供了一种图像分割模型的训练装置，包括：确定模块，配置为确定样本图像，所述样本图像包括纵膈区域的纵膈、背景、动脉和静脉的第一标签以及外延区域的背景和血管的第二标签；第一训练模块，配置为基于所述样本图像训练神经网络，以生成用于获得所述纵膈区域的纵膈、背景、动脉和静脉的第一分割结果的网络模型，其中，所述神经网络为3D神经网络；第二训练模块，配置为基于所述样本图像训练级联的神经网络，以生成用于获得所述外延区域的背景和血管的第二分割结果的分割模型，其中，所述级联的神经网络包括用于特征提取的第一神经网络以及用于生成所述第二分割结果的第二神经网络。
根据本申请实施例的第五方面，提供了一种电子设备，包括：处理器；用于存储所述处理器可执行指令的存储器；所述处理器用于执行上述任一实施例所述的图像分割方法，和/或用于执行上述任一实施例所述的图像分割模型的训练方法。
根据本申请实施例的第六方面，提供了一种计算机可读存储介质，所述存储介质存储有计算机程序，所述计算机程序用于执行上述任一实施例所述的图像分割方法，和/或用于执行上述任一实施例所述的图像分割模型的训练方法。
本申请的实施例所提供的一种图像分割方法，通过将纵膈区域的血管分割任务与外延区域的血管分割任务分离开来，以分别获得纵膈区域的纵膈、动脉、静脉和背景的第一分割结果和外延区域的血管与背景的第二分割结果，再根据第一分割结果与第二分割结果，来获得待分割图像的纵膈、动脉、静脉和背景的分割结果，能够避免直接对动脉、静脉和背景进行分割时外延区域的血管与纵膈区域的血管的尺寸不一致而影响不同尺寸血管的分割，从而能够提高动脉与静脉的分割的准确性和分割的效率。
附图说明
通过结合附图对本申请实施例进行更详细的描述，本申请的上述以及其 他目的、特征和优势将变得更加明显。附图用来提供对本申请实施例的进一步理解，并且构成说明书的一部分，与本申请实施例一起用于解释本申请，并不构成对本申请的限制。在附图中，相同的参考标号通常代表相同部件或步骤。
图1所示为本申请实施例所提供的一种实施环境的示意图。
图2所示为本申请一个实施例提供的图像分割方法的流程示意图。
图3所示为本申请另一个实施例提供的图像分割方法的流程示意图。
图4所示为本申请一个实施例提供的区域增长的实现过程的示意图。
图5所示为本申请一个实施例提供的图像分割模型的训练方法的流程示意图。
图6所示为本申请一个实施例提供的级联的神经网络的训练过程的示意图。
图7所示为本申请一个实施例提供的标记的样本图像的示意图。
图8所示为本申请一个实施例提供的图像分割装置的框图。
图9所示为本申请另一个实施例提供的图像分割装置的框图。
图10所示为本申请又一个实施例提供的图像分割装置的框图。
图11所示为本申请一个实施例提供的图像分割模型的训练装置的框图。
图12所示为本申请另一个实施例提供的图像分割模型的训练装置的框图。
图13所示为本申请又一个实施例提供的图像分割模型的训练装置的框图。
图14所示为本申请一个实施例提供的电子设备的结构框图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
申请概述
深度学习通过建立具有阶层结构的人工神经网络，在计算系统中实现人工智能。由于阶层结构的人工神经网络能够对输入信息进行逐层提取和筛选，因此深度学习具有表征学习能力，可以实现端到端的监督学习和非监督学习。深度学习所使用的阶层结构的人工神经网络具有多种形态，其阶层的复杂度 被通称为“深度”，按构筑类型，深度学习的形式包括多层感知器、卷积神经网络、循环神经网络、深度置信网络和其它混合构筑。深度学习使用数据对其构筑中的参数进行更新以达成训练目标，该过程被通称为“学习”，深度学习提出了一种让计算机自动学习出模式特征的方法，并将特征学习融入到了建立模型的过程中，从而减少了人为设计特征造成的不完备性。
神经网络是一种运算模型，由大量的节点(或称神经元)之间相互连接构成，每个节点对应一个策略函数，每两个节点间的连接代表一个对于通过该连接信号的加权值，称之为权重。神经网络一般包括多个神经网络层，上下网络层之间相互级联，第i个神经网络层的输出与第i+1个神经网络层的输入相连，第i+1个神经网络层的输出与第i+2个神经网络层的输入相连，以此类推。训练样本输入级联的神经网络层后，通过每个神经网络层输出一个输出结果，该输出结果作为下一个神经网络层的输入，由此，通过多个神经网络层计算获得输出，比较输出层的输出的预测结果与真正的目标值，再根据预测结果与目标值之间的差异情况来调整每一层的权重矩阵和策略函数，神经网络利用训练样本不断地经过上述调整过程，使得神经网络的权重等参数得到调整，直到神经网络输出的预测结果与真正的目标结果相符，该过程就被称为神经网络的训练过程。神经网络经过训练后，可得到神经网络模型。
最近的研究表明，动脉/静脉的分类可以更好地评估肺栓塞，而动脉树的变化与慢性血栓栓塞性肺动脉高压的发展相关。此外，肺实质内动脉的改变与右心室功能障碍有关。为了检测两棵血管树的变化，医生手动分析患者的胸部CT图像以寻找异常。这个过程是耗时的，难以标准化，因此不适合大型临床研究或在现实世界的临床决策。因此，在CT图像中实现动静脉的自动分离成为人们关注的热点，它可以帮助医生准确诊断病变。
现有的血管分割方案主要分为基于深度学习的血管分割和基于传统学习的血管分割。特别地，对于肺血管分割任务来说，现有的血管分割方案大多都是利用血管在HU值上的区分度来进行计算的，这种方案在无病灶的CT图像上效果尚可接受，但是，一旦在肺炎和有结节或者肿瘤的CT图像上，就会将与血管HU值相近的病变切割出来。因此，这种方案的鲁棒性很难匹配现有产品使用场景的要求。
而对于基于深度学习的血管分割方案来说，目前主流的方向都是利用基于图结构的一些改进，这种改进包括使用图卷积和图割。但是在预测速度和显存内存等资源占用上，这种改进的算法很难满足线上产品的实时性和资源调度的需求。
在介绍了本申请的基本原理之后，下面将参考附图来具体介绍本申请的各种非限制性实施例。
示例性系统
图1是本申请实施例所提供的一种实施环境的示意图。该实施环境包括CT扫描仪130、服务器120和计算机设备110。计算机设备110可以从CT扫描仪130处获取肺部医学图像，同时，计算机设备110还可以与服务器120之间通过通信网络相连。可选的，通信网络是有线网络或无线网络。
CT扫描仪130用于对人体组织进行X线扫描，得到人体组织的CT图像。在一实施例中，通过CT扫描仪130对肺部进行扫描，可以得到肺部医学图像。
计算机设备110可以是通用型计算机或者由专用的集成电路组成的计算机装置等，本申请实施例对此不做限定。例如，计算机设备110可以是平板电脑等移动终端设备，或者也可以是个人计算机(Personal Computer，PC)，比如膝上型便携计算机和台式计算机等等。本领域技术人员可以知晓，上述计算机设备110的数量可以一个或多个，其类型可以相同或者不同。比如上述计算机设备110可以为一个，或者上述计算机设备110为几十个或几百个，或者更多数量。本申请实施例对计算机设备110的数量和设备类型不加以限定。计算机设备110中可以部署有网络模型和分割模型，网络模型用于对肺部医学图像的纵膈区域的纵膈、动脉、静脉和背景的进行分割，以获得第一分割结果，分割模型用于对肺部医学图像的外延区域的血管和背景的进行分割，以获得第二分割结果。计算机设备110可以利用其上部署的网络模型和分割模型将其从CT扫描仪130获取的肺部医学图像进行图像分割，从而获得第一分割结果以及第二分割结果，进而获得肺部医学图像的纵膈、动脉、静脉和背景的分割结果。这样，通过将纵膈区域的血管分割任务与外延区域的血管分割任务分离开来，能够避免直接对动脉、静脉和背景进行分割时外延区域的血管与纵膈区域的血管的尺寸不一致而影响不同尺寸血管的分割，从而能够提高动脉与静脉的分割的准确性和分割的效率。
服务器120是一台服务器，或者由若干台服务器组成，或者是一个虚拟化平台，或者是一个云计算服务中心。在一些可选的实施例中，服务器120接收计算机设备110采集到的训练图像，并通过训练图像对神经网络进行训练，以得到用于对肺部医学图像的纵膈区域的纵膈、动脉、静脉和背景的进行分割的网络模型和用于对肺部医学图像的外延区域的血管和背景的进行分割的分割模型。计算机设备110可以将其从CT扫描仪130获取到的肺部医学图像发送给服务器，服务器120利用其上训练得到的网络模型和分割模 型进行肺部医学图像的纵膈区域的纵膈、动脉、静脉和背景的分割和外延区域的血管和背景的分割，进而获得肺部医学图像的纵膈、动脉、静脉和背景的分割结果，并将该分割结果发送给计算机设备110，以供医护人员查看。这样，通过将纵膈区域的血管分割任务与外延区域的血管分割任务分离开来，能够避免直接对动脉、静脉和背景进行分割时外延区域的血管与纵膈区域的血管的尺寸不一致而影响不同尺寸血管的分割，从而能够提高动脉与静脉的分割的准确性和分割的效率。
示例性方法
图2是本申请一个实施例提供的图像分割方法的流程示意图。图2所述的方法由计算设备(例如，服务器)来执行，但本申请实施例不以此为限。服务器可以是一台服务器，或者由若干台服务器组成，或者是一个虚拟化平台，或者是一个云计算服务中心，本申请实施例对此不作限定。如图2所示，该方法包括以下内容。
S210：根据包括背景、纵膈、动脉和静脉的待分割图像，获取待分割图像的纵膈区域的纵膈、动脉、静脉和背景的第一分割结果。
待分割图像可以为电子计算机断层成像(Computed Tomography，CT)、核磁共振成像(Magnetic Resonance Imaging，MRI)、计算机放射成像(Computed Radiography，CR)或数字放射成像(Digital radiography，DR)等医学影像，本申请实施例对此不作具体限定。
待分割图像可以为肺部医学图像，但是本申请实施例对此不作具体限定，待分割图像还可以为其他器官的医学图像，该医学图像只要可以将尺寸较大的血管和尺寸较小的血管通过区域划分的方式区分开即可，例如，本申请实施例中的纵膈区域和外延区域。本申请实施例也并不限定待分割图像的具体形式，可以是原始医学图像，也可以是经过预处理后的医学图像，还可以是原始医学图像的一部分。
在一实施例中，纵膈区域是指靠近左右纵膈胸膜附近的区域，其间有心脏及出入心脏的大血管、食管、气管、胸腺、神经及淋巴组织等；外延区域是指纵膈区域以外的包含血管的区域。例如，对于肺部医学图像而言，纵膈区域是指靠近左右纵膈胸膜附近的区域，外延区域是指纵膈区域以外的肺内区域。纵膈区域内的血管尺寸比外延区域内的血管尺寸大。
在一实施例中，可以对包括背景、动脉和静脉的待分割图像进行第一分割，以获取纵膈区域的纵膈、动脉、静脉和背景的第一分割结果，但是需要说明的是，本申请实施例并不限定第一分割的具体实施手段。
S220：根据待分割图像，获取待分割图像的外延区域的血管与背景的第 二分割结果。
在一实施例中，可以对待分割图像进行第二分割，以获取外延区域的血管与背景的第二分割结果，但是需要说明的是，本申请实施例并不限定第二分割的具体实施手段。
对待分割图像进行第二分割可以将待分割图像的外延区域的血管与背景分割开来，但是并不对血管进行分类，即，不区分血管是动脉还是静脉，只要可以将血管和背景分割开来即可。
本申请实施例也并不限定第一分割和第二分割的具体实施手段是否相同，二者可以相同，也可以不同；且本申请实施例也并不限定执行第一分割和第二分割的先后顺序，可以先执行第一分割，也可以先执行第二分割，还可以同时执行第一分割和第二分割，只要可以得到各自的分割结果即可。
S230：根据第一分割结果和第二分割结果，获取待分割图像的纵膈、动脉、静脉和背景的分割结果。
在一实施例中，可以对第一分割结果和第二分割结果进行处理，来获取纵膈、动脉、静脉和背景的分割结果，但是本申请实施例并不限定如何对第一分割结果和第二分割结果进行处理，只要可以获得最后的背景、动脉和静脉的分割结果即可。
第一分割结果是指待分割图像的纵膈区域的分割结果，第二分割结果是指待分割图像的外延区域的分割结果。例如，可以直接将第一分割结果和第二分割结果进行简单的叠加，以得到待分割图像的纵膈、动脉、静脉和背景的分割结果；也可以将第一分割结果和第二分割结果仅作为一个中间结果，然后对该中间结果进行处理，来获得纵膈、动脉、静脉和背景的分割结果。
由此可见，通过将纵膈区域的血管分割任务与外延区域的血管分割任务分离开来，以分别获得纵膈区域的纵膈、动脉、静脉和背景的第一分割结果和外延区域的血管与背景的第二分割结果，再根据第一分割结果与第二分割结果，来获得待分割图像的纵膈、动脉、静脉和背景的分割结果，能够避免直接对动脉、静脉和背景进行分割时外延区域的血管与纵膈区域的血管的尺寸不一致而影响不同尺寸血管的分割，从而能够提高动脉与静脉的分割的准确性和分割的效率。
在本申请另一个实施例中，如图3所示的方法是图2所示的方法的示例，如图3所示的方法还包括以下内容。
S310：将待分割图像输入网络模型。
为了对待分割图像进行分割，可以将待分割图像输入到用于获得纵膈区域的纵膈、动脉、静脉和背景的第一分割结果的网络模型中进行分割。
本申请实施例对网络模型的具体类型不作限定，该网络模型可以由任意类型的神经网络构成。可选地，该网络模型可以为卷积神经网络(Convolutional Neural Network，CNN)、深度神经网络(Deep Neural Network，DNN)或循环神经网络(Recurrent Neural Network，RNN)等。该网络模型可以包括输入层、卷积层、池化层、连接层等神经网络层，本申请实施例对此不作具体限定。另外，本申请实施例对每一种神经网络层的个数也不作限定。
S320：根据待分割图像，通过网络模型，获取待分割图像的纵膈区域的纵膈、动脉、静脉和背景的第一分割结果。
将待分割图像直接输入到网络模型中，以获得待分割图像的纵膈区域的纵膈、动脉、静脉和背景的第一分割结果。通过将待分割图像输入到该网络模型中进行分割，可以使得待分割图像的纵膈区域的纵膈、动脉、静脉和背景的第一分割结果更加准确。
在一个实施例中，将待分割图像输入网络模型，包括：将待分割图像的纵膈区域进行切块操作，获得多个切块图像，其中，多个切块图像中的每个切块图像均包括所述纵膈；将多个切块图像输入网络模型，其中，根据待分割图像，通过网络模型，获取待分割图像的纵膈区域的纵膈、动脉、静脉和背景的第一分割结果，包括：根据多个切块图像，通过网络模型，获取多个切块图像对应的纵膈区域的纵膈、动脉、静脉和背景的多个子分割结果；通过高斯平滑处理，对多个子分割结果进行组合操作，以获得组合后的分割结果；通过连通域算法，对组合后的分割结果进行后处理，获得第一分割结果。
该网络模型可以为3D网络模型，也可以为2D网络模型，本申请实施例对此并不作具体限定，本领域技术人员可以根据实际应用需求来设计网络模型的具体类型。
例如，对于3D网络模型，由于其精度高，3D表现优异的原因，被广泛使用于医疗图像分割领域。但是3D分割网络模型会对计算资源消耗极大，如果直接将完整的待分割图像(即，原始医学图像)输入3D分割网络模型中，训练3D分割网络模型所需要的显存会非常大。
为了兼顾计算资源的消耗与最终分割精度，可以将将待分割图像的纵膈区域进行切块操作，即，对与纵膈区域对应的待分割图像进行切块操作，以获得与纵膈区域对应的多个切块图像，多个切块图像可以为相互重叠的切块图像，然后将其输入到网络模型中进行分割。但是需要说明的是，本申请实施例并不限定将待分割图像分割为多少个切块图像，也并不限定相邻两个切块图像之间相互重叠的尺寸为多少。
切块图像包含纵膈，有助于纵膈区域内的血管的类别判断，即，纵膈可 以作为3D网络模型进行动脉和静脉的分类学习的参考物，使3D网络模型可以更好地对动脉和静脉进行类别判断。
在一实施例中，当得到了多个切块图像后，再将多个切块图像输入到网络模型中进行图像分割。此时，每将一个切块图像输入到网络模型，就可以输出一个子分割结果，即，一个切块图像对应一个子分割结果，那么多个切块图像对应多个子分割结果，多个子分割结果对应待分割图像的纵膈区域的纵膈、动脉、静脉和背景的分割结果。
在一实施例中，将多个子分割结果组合成为与待分割图像的尺寸大小相当的mask，作为第一分割结果。由于多个切块图像为相互重叠的切块图像，因此，为了保证组合后所得到的第一分割结果的边界平滑性，可以只保留中心位置的分割结果，将中心位置的分割结果进行组合操作。即，为了获得边界平滑的待分割图像的纵膈区域的纵膈、动脉、静脉和背景的第一分割结果，可以通过高斯平滑处理，对多个子分割结果进行组合操作，以获得边界平滑的纵膈区域的纵膈、动脉、静脉和背景的第一分割结果。
应当理解，最简单的组合操作可以是将多个切块图像直接拼接在一起，其简单快捷，但是会带来栅栏效应(即，由于网络模型在切块图像的边界附近表现不好，一致性较差，当两个相邻切块图像对应的切割结果组合在一起时，会出现明显的组合痕迹)。由于网络模型在切块图像的中心位置的分割结果更可信，表现更好，所以可以只保留切块图像靠近中心位置的分割结果，例如，切块图像的尺寸为192*192*64，但是可以只保留中心位置附近的尺寸为160*160*48区域的分割结果。为了进一步缓解栅栏效应，使得网络模型的分割结果更加平滑，可以在直接拼接的组合操作的基础上，修改切块的步长，将两个160*160*48区域的分割结果的重叠区域求均值，但该组合操作的缺点在于没有使用前面的先验假设，因此，可以采用高斯平滑处理的组合操作来克服这一缺点，从而使得分割结果的精确度更高。
可以采用高斯函数，以切块图像的中心位置作为高斯核的均值，利用高斯加权的方式来进行平滑，这样，正好契合网络模型在切块图像的中心位置的置信度高的先验知识，可以更好的平滑经过组合操作所得到的第一分割结果。
但是需要说明的是，本申请实施例并不限定高斯平滑处理所采用的函数，可以采用高斯函数，也可以采用其他钟形函数。
在一实施例中，为了去掉一些体外假阳和使组合在一起的各切块图像之间的交界处表现的一致，可以通过连通域算法，对组合后的分割结果进行后处理，以获得第一分割结果。但是本申请实施例并不限定后处理的具体实施 手段，只要能够将假阳和交界处表现不一致的点去除即可。
在一个实施例中，通过连通域算法，对组合后的分割结果进行后处理，以获得第一分割结果，包括：通过连通域算法，获取组合后的分割结果中的静脉最大连通域和动脉最大连通域；根据静脉最大连通域和动脉最大连通域，去除组合后的分割结果中的噪点，以获得第一分割结果，其中，噪点包括同时为动脉和静脉的点以及假阳点。
首先，对组合后的分割结果进行连通域处理，以获得最大连通域，该最大连通域包括动脉最大连通域和静脉最大连通域。其次，去除组合后的分割结果中的动脉最大连通域和静脉最大连通域，以获得组合后的分割结果中的噪点。最后，从组合后的分割结果中去除该噪点，可以获得第一分割结果。这样所得到的第一分割结果中就不包含同时为动脉和静脉的点或者假阳点等不符合要求的点。
该噪点可以包括同时为动脉和静脉的点以及假阳点，本申请实施例对此并不作具体限定，噪点还可以为其他不符合要求的点。
S330：将待分割图像输入分割模型。
为了对待分割图像进行分割，可以将待分割图像输入到用于获得外延区域的血管和背景的第二分割结果的分割模型中进行分割。
本申请实施例对分割模型的具体类型不作限定，该分割模型可以由任意类型的神经网络构成。可选地，该分割模型可以为卷积神经网络(Convolutional Neural Network，CNN)、深度神经网络(Deep Neural Network，DNN)或循环神经网络(Recurrent Neural Network，RNN)等。该分割模型可以包括输入层、卷积层、池化层、连接层等神经网络层，本申请实施例对此不作具体限定。另外，本申请实施例对每一种神经网络层的个数也不作限定。
S340：根据待分割图像，通过分割模型，获取外延区域的血管与背景的第二分割结果。
将待分割图像直接输入到分割模型中，以获得待分割图像的外延区域的血管和背景的第二分割结果。通过将待分割图像输入到该分割模型中进行分割，可以使得待分割图像的外延区域的血管和背景的第二分割结果更加准确。
分割模型可以将待分割图像的外延区域的血管与背景分割开来，但是并不对血管进行分类，即，不区分血管是动脉还是静脉，只要可以将血管和背景分割开来即可。
例如，针对肺部医学图像，外延区域是指肺内区域，由于血管在肺内区域比较容易识别，因此，分割模型可以采用一些轻量化的模型结构。在一实施例中，该分割模型可以为2D分割模型，但是本申请实施例对此并不作具 体限定，本领域技术人员可以根据实际应用需求来设计网络模型的具体类型。同时，本申请实施例也并不具体限定分割模型的具体模型结构，本领域技术人员可以根据实际应用需求来设计分割模型的具体模型结构，例如，分割模型可以由ResNet18和特征金字塔网络构成。
综上，通过利用网络模型对纵膈区域的尺寸较大的血管进行分割，再利用分割模型对外延区域的尺寸较小的血管进行分割，可以避免仅使用单个模型对尺寸较大的血管和尺寸较小的血管进行分割时的分割效果和分割性能不平衡的缺点。
例如，如果使用单个3D网络模型对纵膈区域的血管和外延区域的血管进行分割，包含纵膈的切块图像的尺寸大小与外延区域的动脉和静脉的分类正确性具有关联性。具体地，切块图像的CT物理分辨率(即，Pixelspacing分辨率，Pixelspacing分辨率越高对现实物理世界还原度越高，Pixelspacing分辨率越低对现实物理空间还原度越低)会影响到外延区域的尺寸较小的血管的分割，即，切块图像的CT物理分辨率越高，越有助于外延区域的尺寸较小的血管的分割。在理论上，切块图像的物理体积越大，其CT物理分辨率越大，3D网络模型对尺寸较小的血管的分割以及对动脉和静脉的分类的表现就会越好。然而，在切块图像的尺寸大小不变的情况下，物理体积和CT物理分辨率成反比的关系，如果想同时提高物理体积和CT物理分辨率，即，提高3D网络模型对尺寸较小的血管的分割以及对动脉和静脉的分类的准确性，就需要增大切块图像的尺寸大小。例如，在切块图像的物理体积不变的情况下，假设将切块图像的CT物理分辨率放缩0.5，切块图像的每个边会放大2倍，这样，切块图像的尺寸大小和3D网络模型的大小都会放大到原来的8倍。也就是说，此时如果提高切块图像的物理体积的话，切块图像的尺寸大小的增长就需要超过原来的8倍，这显然会增加训练3D分割网络模型时所需要的显存，从而降低了3D网络模型对待分割图像的分割效率。
因此，为了避免上述提到的矛盾，采用两个模型来分别进行不同的分割，即，2D分割模型负责对外延区域的尺寸较小的血管的分割精度，而3D网络模型负责纵膈区域的尺寸较大的血管的分类和分割的正确性。这样，可以降低切块图像的尺寸大小，即，可以简化3D网络模型的分割任务；同时，也可以保证纵膈和靠近纵膈区域的血管的分割和分类的准确性，即，只需要保证切块图像的物理体积足够大，就可以得到一个较好的纵膈和靠近纵膈区域的血管的分割效果和分类效果。例如，将CT物理空间分辨率增大为原来的1.3倍，在获得相同的纵膈和靠近纵膈区域的血管的分割效果和分类效果的情况下，显存缩减为原来的2倍，从而提高了3D网络模型对待分割图像的 分割效率。
S350：根据第一分割结果中的动脉和静脉和第二分割结果中的血管，通过区域增长算法，得到待分割图像的纵膈、动脉、静脉和背景的分割结果。
图4示出了区域增长的实现过程，如图4所示，在获得了第一分割结果和第二分割结果后，对第一分割结果中的动脉和静脉和第二分割结果中的血管进行区域增长(RegionGrowth)，可以将第一分割结果中的动脉和静脉和第二分割结果中的血管结合在一起，以得到待分割图像的纵膈、动脉、静脉和背景的分割结果。为了使得RegionGrowth的速度达到要求，可以采取cuda实现。
应当理解，区域生长算法是将具有相似性的像素集合起来构成最终的区域。首先对每个需要分割的区域找出一个种子像素作为生长的起点，然后将种子像素周围邻域中与种子具有相同或相似性质的像素(根据事先确定的生长或相似准则来确定)合并到种子像素所在的区域中。而新的像素继续作为种子向四周生长，直到再没有满足条件的像素可以包括进来，一个最终的区域生长完成。
在本申请另一实施例中，根据第一分割结果中的动脉和静脉和第二分割结果中的血管，通过区域增长算法，得到待分割图像的纵膈、动脉、静脉和背景的分割结果，包括：以第一分割结果中的动脉和静脉为起点，将第一分割结果中的动脉和静脉沿第二分割结果中的血管，以预设血管生长长度，进行区域增长，以获得待分割图像的纵膈、动脉、静脉和背景的分割结果。
由于第一分割结果对纵膈区域的动脉和静脉进行了分类，而第二分割结果对外延区域的血管并未进行动脉和静脉的分类，同时，由于第一分割结果和第二分割结果在纵膈区域的血管会有重合，而在肺内区域的血管不重合，因此，可以将第一分割结果中的动脉和静脉作为区域增长的起点，而第二分割结果中的血管作为区域增长的轨道，通过区域增长算法，使得第一分割结果中的动脉和静脉沿着第二分割结果中的血管向外延区域进一步延伸，即，使第一分割结果中的动脉和静脉沿着第二分割结果中的血管增长。
由于动脉与静脉是两个完整的连通域，所以以确定的动脉和静脉作为起点，通过连通域的分析，可以将区域增长后的外延区域的动脉和静脉进行分类；同时，由于动脉与静脉是两个完整的连通域，而第二分割结果中的假阳不会与这两个完整的连通域相结合，所以通过区域增长，还可以去除第二分割结果中的假阳。
由于区域增长算法是一种迭代式的算法，因此，可以在每一次迭代过程中，设置第一分割结果中的动脉和静脉沿第二分割结果中的血管进行区域增 长时的预设血管生长长度(例如，对于肺部医学图像而言，预设血管生长长度为预设肺内血管生长长度)。在分割后图像的显示上，可以通过调节每次迭代过程中的预设血管生长长度，来动态显示血管的粒度。这样可以避免医护人员在查看分割后图像的VR时一些病灶被血管遮挡的情况，从而增加用户的使用体验。
但是本申请实施例并不限定每一次迭代过程中的预设血管生长长度的具体取值，可以根据不同的应用需求进行选择。
综上，通过将第一分割结果中的动脉和静脉按照第二分割结果中的血管向外延区域进一步延伸，可以使动脉和静脉的分割任务拆解为三个子任务，采用两个模型来分别得到第一分割结果和第二分割结果，可以降低任务的复杂度，进而可以采用一些简单模型结构来降低显存和加快预测速度，以满足线上产品的实时性和资源调度的需求。
图5是本申请一个实施例提供的图像分割模型的训练方法的流程示意图。图5所述的方法由计算设备(例如，服务器)来执行，但本申请实施例不以此为限。服务器可以是一台服务器，或者由若干台服务器组成，或者是一个虚拟化平台，或者是一个云计算服务中心，本申请实施例对此不作限定。如图5所示，该方法包括如下内容。
S510：确定样本图像，样本图像包括纵膈区域的纵膈、背景、动脉和静脉的第一标签以及外延区域的背景和血管的第二标签。
该第一标签是指对样本图像的纵膈区域中的纵膈、背景、动脉和静脉进行标注所得到的标签；该第二标签是指对样本图像的外延区域中的背景和血管进行标注所得到的标签，该第二标签中的血管不具体划分动脉和静脉。
本实施例中提到的样本图像与上述实施例中的待分割图像属于同一种类型的图像。该样本图像经过了人工的标记，从而得到了第一标签和第二标签。
但是，需要说明的是，本申请实施例并不限定样本图像的具体形式，可以是一个原始医学图像，也可以是经过预处理后的医学图像，还可以是原始医学图像的一部分。
本实施例所提到的纵膈区域和外延区域与上述图像分割方法中的实施例中的纵膈区域和外延区域相同，具体细节在此不再赘述，请参见上述图像分割方法中的实施例。
S520：基于样本图像训练神经网络，以生成用于获得纵膈区域的纵膈、背景、动脉和静脉的第一分割结果的网络模型，其中，神经网络为3D神经网络。
在一实施例中，将样本图像输入到神经网络中，对神经网络进行训练，以生成该网络模型。
被训练的神经网络可以为任意类型的神经网络。可选地，被训练的神经网络可以为卷积神经网络(Convolutional Neural Network，CNN)、深度神经网络(Deep Neural Network，DNN)或循环神经网络(Recurrent Neural Network，RNN)等，本申请实施例对被训练的神经网络的具体类型不作限定。被训练的神经网络可以包括输入层、卷积层、池化层、连接层等神经网络层，本申请实施例对此不作具体限定。另外，本申请实施例对每一种神经网络层的个数也不作限定。优选地，被训练的神经网络为3D神经网络。
S530：基于样本图像训练级联的神经网络，以生成用于获得外延区域的背景和血管的第二分割结果的分割模型，其中，级联的神经网络包括用于特征提取的第一神经网络以及用于生成第二分割结果的第二神经网络。
在一实施例中，将样本图像输入到级联的神经网络中，对级联的神经网络进行训练，以生成该分割模型。
第一神经网络和第二神经网络可以为任意类型的神经网络。可选地，第一神经网络和第二神经网络可以为卷积神经网络(Convolutional Neural Network，CNN)、深度神经网络(Deep Neural Network，DNN)或循环神经网络(Recurrent Neural Network，RNN)等，本申请实施例对第一神经网络和第二神经网络的具体类型不作限定。第一神经网络和第二神经网络可以包括输入层、卷积层、池化层、连接层等神经网络层，本申请实施例对此不作具体限定。另外，本申请实施例对每一种神经网络层的个数也不作限定。
该级联的神经网络包括用于特征提取的第一神经网络以及位于第一神经网络后的用于生成第二分割结果的第二神经网络。但是本申请实施例并不具体限定级联的神经网络的具体结构，该级联的神经网络还可以包括其他的神经网络。
本申请实施例并不限定训练网络模型和训练分割模型的先后顺序，可以先训练网络模型，也可以先训练分割模型，还可以同时训练网络模型和训练分割模型，只要可以得到训练完成的网络模型和分割模型即可。
在本申请另一实施例中，所述方法还包括：将样本图像的纵膈区域进行切块操作，获得多个切块图像，其中，多个切块图像中的每个切块图像均包括纵膈，其中，基于所述样本图像训练神经网络，以生成用于获得纵膈区域的纵膈、背景、动脉和静脉的第一分割结果的网络模型，包括：基于多个切块图像训练神经网络，以生成网络模型。
本实施例所提到的切块操作与上述图像分割方法中的实施例中的切块 操作相同，具体细节在此不再赘述，请参见上述图像分割方法中的实施例。
在一实施例中，将多个切块图像分别输入到3D神经网络中，对3D神经网络进行训练，以生成该网络模型。但是本申请实施例并不限定3D神经网络的训练过程，只要能够形成用于获得纵膈区域的纵膈、动脉、静脉和背景的第一分割结果的网络模型即可。
例如，利用第一损失函数，计算3D神经网络输出的每个切块图像对应的第一分割结果和每个切块图像对应的第一标签(即，纵膈、背景、动脉和静脉的目标结果)之间的相似度损失，可以得到3D神经网络的第一损失函数值。第一损失函数值越小，代表预测出的第一分割结果越接近目标结果，预测正确的准确率越高。相反，第一损失函数值越大，代表预测正确的准确率越低。将该第一损失函数值进行梯度反传，以更新该3D神经网络的参数，例如权重，偏值等，本申请对此不做限定。
在本申请另一实施例中，基于样本图像训练级联的神经网络，以生成用于获得外延区域的背景和血管的第二分割结果的分割模型，包括：通过第一神经网络，对样本图像进行下采样操作，以获得多个第一特征图；通过第二神经网络，对多个第一特征图进行上采样和融合操作，以获得第二特征图；利用分类器对第二特征图进行激活，以获得外延区域的背景和血管的第二分割结果；根据第二分割结果和第二标签，获得级联的神经网络的损失函数值；根据损失函数值，更新级联的神经网络的参数，其中，第一神经网络为深度残差网络，第二神经网络为特征金字塔网络。
图6示出了该级联的神经网络的训练过程的一个示例，具体如下。
将样本图像输入到深度残差网络中，对其进行下采样操作，生成多个第一特征图，即，第一特征图1、第一特征图2、第一特征图3以及第一特征图4，但是本申请实施例对第一特征图的个数不作具体限定，本申请实施例也并不限定下采样的倍数。
再将多个第一特征图分别输入到特征金字塔网络中，对多个第一特征图进行上采样和融合操作，生成第二特征图。具体地，将第一特征图4输入到特征金字塔网络中，并对第一特征图3进行降维后输入到特征金字塔网络中，使其与第一特征图4进行融合操作，以获得融合后的特征图1；再将第一特征图2进行降维后输入特征金字塔网络中，使其与融合后的特征图1进行融合操作，以获得融合后的特征图2；再对融合后的特征图2进行上采样操作，以获得与样本图像尺寸相同的第二特征图；最后利用分类器对第二特征图进行激活，以获得外延区域的背景和血管的第二分割结果。
但是本申请实施例也并不限定上采样的倍数。图6所示的训练过程只是 级联的神经网络的训练过程的一个示例，不用于限制本申请。
在一实施例中，利用损失函数，计算第二分割结果与第二标签(即，背景与血管的目标结果)之间的相似度损失，可以得到级联的神经网络的损失函数值。损失函数值越小，代表预测出的第二分割结果越接近目标结果，预测正确的准确率越高。相反，损失函数值越大，代表预测正确的准确率越低。
在一实施例中，将该级联的神经网络的损失函数值进行梯度反传，以更新该级联的神经网络的参数，例如权重，偏值等，本申请对此不做限定。
在本申请另一实施例中，所述方法还包括：将第二标签中的血管所在区域进行最大池化操作，获得第二标签中的血管所在区域膨胀后的样本图像的目标区域，其中，根据第二分割结果和第二标签，获得级联的神经网络的损失函数值，包括：根据目标区域对应的第二分割结果和第二标签，获取级联的神经网络的损失函数值。
在现有的分割网络中，正样本(动脉与静脉)的像素点数量远远小于负样本(背景)的像素点数量，在深度学习中，这种状况被称之为类不平衡。类不平衡会导致模型更加倾向于学习数量大的样本，也就是负样本(背景)。
为了缓解这一问题，可以通过最大值池化(max pooling)操作，将正样本所在的标记区域进行“膨胀”，如图7所示，左图为第二标签的原始标记结果，其中，带有白色标签的区域为血管，其余黑色区域为背景；右图为经过“膨胀”后的标记结果，其中，整个白色区域为带有白色标签的区域经膨胀后得到的目标区域，其包括左图中的带有白色标签的区域和与该区域临近的黑色区域(即，背景)。这样，可以有效的降低正负样本不平衡问题，使得血管分割的更加细致，同时，还可以加速分割模型的收敛。
本实施例中，将第二标签中的血管所在区域(即，带有白色标签的区域)进行最大池化操作，可以获得第二标签中的血管所在区域经膨胀后的样本图像的目标区域(即，经过“膨胀”后的目标区域)。该目标区域包括血管标签以及与血管标签临近的背景标签。
利用损失函数，计算目标区域对应的第二分割结果与样本图像的第二标签之间的相似度损失，可以得到级联的神经网络的损失函数值。这样，可以只计算目标区域中的正负样本的损失，即，只计算目标区域内的像素值的损失函数，这样可以有效的克服类不平衡问题。
示例性装置
本申请装置实施例，可以用于执行本申请方法实施例。对于本申请装置实施例中未披露的细节，请参照本申请方法实施例。
图8所示为本申请一个实施例提供的图像分割装置的框图。如图8所示， 该装置800包括：
第一分割模块810，配置为根据包括背景、纵膈、动脉和静脉的待分割图像，获取待分割图像的纵膈区域的纵膈、动脉、静脉和背景的第一分割结果；
第二分割模块820，配置为根据待分割图像，获取待分割图像的外延区域的血管与背景的第二分割结果；
获取模块830，配置为根据第一分割结果和第二分割结果，获取待分割图像的纵膈、动脉、静脉和背景的分割结果。
在一个实施例中，如图9所示，所述装置800还包括：第一输入模块840，配置为将待分割图像输入网络模型。
在一个实施例中，第一分割模块810进一步配置为：根据待分割图像，通过网络模型，获取待分割图像的纵膈区域的纵膈、动脉、静脉和背景的第一分割结果。
在一个实施例中，第一输入模块840进一步配置为：将待分割图像的纵膈区域进行切块操作，获得多个切块图像，其中，多个切块图像中的每个切块图像均包括纵膈；将多个切块图像输入网络模型。
在一个实施例中，第一分割模块810在根据待分割图像，通过网络模型，获取待分割图像的纵膈区域的纵膈、动脉、静脉和背景的第一分割结果时，进一步配置为：根据多个切块图像，通过网络模型，获取多个切块图像对应的纵膈区域的纵膈、动脉、静脉和背景的多个子分割结果；通过高斯平滑处理，对多个子分割结果进行组合操作，以获得组合后的分割结果；通过连通域算法，对组合后的分割结果进行后处理，获得第一分割结果。
在一个实施例中，第一分割模块810在通过连通域算法，对组合后的分割结果进行后处理，获得第一分割结果时，进一步配置为：通过连通域算法，获取组合后的分割结果中的静脉最大连通域和动脉最大连通域；根据静脉最大连通域和动脉最大连通域，去除组合后的分割结果中的噪点，以获得第一分割结果，其中，噪点包括同时为动脉和静脉的点以及假阳点。
在一个实施例中，如图10所示，所述装置800还包括：第二输入模块850，配置为将待分割图像输入分割模型。
在一个实施例中，第二分割模块820进一步配置为：根据待分割图像，通过分割模型，获取外延区域的血管与背景的第二分割结果。
在一个实施例中，获取模块830进一步配置为：根据第一分割结果中的动脉和静脉和第二分割结果中的血管，通过区域增长算法，得到待分割图像的纵膈、动脉、静脉和背景的分割结果。
在一个实施例中，获取模块830在通过区域增长算法，得到待分割图像的纵膈、动脉、静脉和背景的分割结果时，进一步配置为：以第一分割结果中的动脉和静脉为起点，将第一分割结果的动脉和静脉沿第二分割结果中的血管，以预设血管生长长度，进行区域增长，以获得待分割图像的纵膈、动脉、静脉和背景的分割结果。
图11所示为本申请一个实施例提供的图像分割模型的训练装置的框图。如图11所示，该装置1100包括：
确定模块1110，配置为确定样本图像，样本图像包括纵膈区域的纵膈、背景、动脉和静脉的第一标签以及外延区域的背景和血管的第二标签；
第一训练模块1120，配置为基于样本图像训练神经网络，以生成用于获得纵膈区域的纵膈、背景、动脉和静脉的第一分割结果的网络模型，其中，所述神经网络为3D神经网络；
第二训练模块1130，配置为基于所述样本图像训练级联的神经网络，以生成用于获得外延区域的背景和血管的第二分割结果的分割模型，其中，级联的神经网络包括用于特征提取的第一神经网络以及用于生成第二分割结果的第二神经网络。
在一个实施例中，如图12所示，所述装置1100还包括：切块模块1140，配置为将样本图像的纵膈区域进行切块操作，获得多个切块图像，其中，多个切块图像中的每个切块图像均包括纵膈。
在一个实施例中，第一训练模块1120进一步配置为：基于多个切块图像训练神经网络，以生成网络模型。
在一个实施例中，第二训练模块1130进一步配置为：通过第一神经网络，对样本图像进行下采样操作，以获得多个第一特征图；通过第二神经网络，对多个第一特征图进行上采样和融合操作，以获得第二特征图；利用分类器对第二特征图进行激活，以获得外延区域的背景和血管的第二分割结果；根据第二分割结果和第二标签，获得级联的神经网络的损失函数值；根据损失函数值，更新级联的神经网络的参数，其中，第一神经网络为深度残差网络，第二神经网络为特征金字塔网络。
在一个实施例中，如图13所示，所述装置1100还包括：最大池化模块1150，配置为将第二标签中的血管所在区域进行最大池化操作，获得第二标签中的血管所在区域膨胀后的样本图像的目标区域。
在一个实施例中，第二训练模块1130在根据第二分割结果和第二标签，获得级联的神经网络的损失函数值时，进一步配置为：根据目标区域对应的第二分割结果和第二标签，获取级联的神经网络的损失函数值。
示例性电子设备
下面，参考图14来描述根据本申请实施例的电子设备。图14图示了根据本申请实施例的电子设备的框图。
如图14所示，电子设备1400包括一个或多个处理器1410和存储器1420。
处理器1410可以是中央处理单元(CPU)或者具有数据处理能力和/或指令执行能力的其他形式的处理单元，并且可以控制电子设备1400中的其他组件以执行期望的功能。
存储器1420可以包括一个或多个计算机程序产品，所述计算机程序产品可以包括各种形式的计算机可读存储介质，例如易失性存储器和/或非易失性存储器。所述易失性存储器例如可以包括随机存取存储器(RAM)和/或高速缓冲存储器(cache)等。所述非易失性存储器例如可以包括只读存储器(ROM)、硬盘、闪存等。在所述计算机可读存储介质上可以存储一个或多个计算机程序指令，处理器1410可以运行所述程序指令，以实现上文所述的本申请的各个实施例的图像分割方法、图像分割模型的训练方法以及/或者其他期望的功能。在所述计算机可读存储介质中还可以存储诸如输入信号、信号分量、噪声分量等各种内容。
在一个示例中，电子设备1400还可以包括：输入装置1430和输出装置1440，这些组件通过总线系统和/或其他形式的连接机构(未示出)互连。
例如，该输入装置1430可以是上述的麦克风或麦克风阵列，用于捕捉声源的输入信号。在该电子设备是单机设备时，该输入装置1430可以是通信网络连接器。
此外，该输入设备1430还可以包括例如键盘、鼠标等等。
该输出装置1440可以向外部输出各种信息，包括确定出的征象类别信息等。该输出设备1440可以包括例如显示器、扬声器、打印机、以及通信网络及其所连接的远程输出设备等等。
当然，为了简化，图14中仅示出了该电子设备1400中与本申请有关的组件中的一些，省略了诸如总线、输入/输出接口等等的组件。除此之外，根据具体应用情况，电子设备1400还可以包括任何其他适当的组件。
示例性计算机程序产品和计算机可读存储介质
除了上述方法和设备以外，本申请的实施例还可以是计算机程序产品，其包括计算机程序指令，所述计算机程序指令在被处理器运行时使得所述处理器执行本说明书上述“示例性方法”部分中描述的根据本申请各种实施例的图像分割方法、图像分割模型的训练方法中的步骤。
所述计算机程序产品可以以一种或多种程序设计语言的任意组合来编 写用于执行本申请实施例操作的程序代码，所述程序设计语言包括面向对象的程序设计语言，诸如Java、C++等，还包括常规的过程式程序设计语言，诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算设备上执行、部分地在用户设备上执行、作为一个独立的软件包执行、部分在用户计算设备上部分在远程计算设备上执行、或者完全在远程计算设备或服务器上执行。
此外，本申请的实施例还可以是计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令在被处理器运行时使得所述处理器执行本说明书上述“示例性方法”部分中描述的根据本申请各种实施例的图像分割方法、图像分割模型的训练方法中的步骤。
所述计算机可读存储介质可以采用一个或多个可读介质的任意组合。可读介质可以是可读信号介质或者可读存储介质。可读存储介质例如可以包括但不限于电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。可读存储介质的更具体的例子(非穷举的列表)包括：具有一个或多个导线的电连接、便携式盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。
为了例示和描述的目的已经给出了以上描述。此外，此描述不意图将本申请的实施例限制到在此公开的形式。尽管以上已经讨论了多个示例方面和实施例，但是本领域技术人员将认识到其某些变型、修改、改变、添加和子组合。

Claims (15)

  1. 一种图像分割方法，其特征在于，包括：
    根据包括背景、纵膈、动脉和静脉的待分割图像，获取所述待分割图像的纵膈区域的所述纵膈、所述动脉、所述静脉和所述背景的第一分割结果；
    根据所述待分割图像，获取所述待分割图像的外延区域的血管与所述背景的第二分割结果；
    根据所述第一分割结果和所述第二分割结果，获取所述待分割图像的所述纵膈、所述动脉、所述静脉和所述背景的分割结果。
  2. 根据权利要求1所述的方法，其特征在于，还包括：
    将所述待分割图像输入网络模型，
    其中，所述根据包括背景、纵膈、动脉和静脉的待分割图像，获取所述待分割图像的纵膈区域的所述纵膈、所述动脉、所述静脉和所述背景的第一分割结果，包括：
    根据所述待分割图像，通过所述网络模型，获取所述待分割图像的纵膈区域的所述纵膈、所述动脉、所述静脉和所述背景的第一分割结果。
  3. 根据权利要求2所述的方法，其特征在于，所述将所述待分割图像输入网络模型，包括：
    将所述待分割图像的纵膈区域进行切块操作，获得多个切块图像，其中，所述多个切块图像中的每个切块图像均包括所述纵膈；
    将所述多个切块图像输入所述网络模型，
    其中，所述根据所述待分割图像，通过所述网络模型，获取所述待分割图像的纵膈区域的所述纵膈、所述动脉、所述静脉和所述背景的第一分割结果，包括：
    根据所述多个切块图像，通过所述网络模型，获取所述多个切块图像对应的所述纵膈区域的所述纵膈、所述动脉、所述静脉和所述背景的多个子分割结果；
    通过高斯平滑处理，对所述多个子分割结果进行组合操作，以获得组合后的分割结果；
    通过连通域算法，对所述组合后的分割结果进行后处理，获得所述第一分割结果。
  4. 根据权利要求3所述的方法，其特征在于，所述通过连通域算法，对所述组合后的分割结果进行后处理，获得所述第一分割结果，包括：
    通过所述连通域算法，获取所述组合后的分割结果中的静脉最大连通域 和动脉最大连通域；
    根据所述静脉最大连通域和所述动脉最大连通域，去除所述组合后的分割结果中的噪点，以获得所述第一分割结果，其中，所述噪点包括同时为所述动脉和所述静脉的点以及假阳点。
  5. 根据权利要求1至4中任一项所述的方法，其特征在于，还包括：
    将所述待分割图像输入分割模型，
    其中，所述根据所述待分割图像，获取所述待分割图像的外延区域的血管与所述背景的第二分割结果，包括：
    根据所述待分割图像，通过所述分割模型，获取所述外延区域的血管与所述背景的第二分割结果。
  6. 根据权利要求1至5中任一项所述的方法，其特征在于，所述根据所述第一分割结果和所述第二分割结果，得到所述待分割图像的所述纵膈、所述动脉、所述静脉和所述背景的分割结果，包括：
    根据所述第一分割结果中的所述动脉和所述静脉和所述第二分割结果中的所述血管，通过区域增长算法，得到所述待分割图像的所述纵膈、所述动脉、所述静脉和所述背景的分割结果。
  7. 根据权利要求6所述的方法，其特征在于，所述根据所述第一分割结果中的所述动脉和所述静脉和所述第二分割结果中的所述血管，通过区域增长算法，得到所述待分割图像的所述纵膈、所述动脉、所述静脉和所述背景的分割结果，包括：
    以所述第一分割结果中的所述动脉和所述静脉为起点，将所述第一分割结果的所述动脉和所述静脉沿所述第二分割结果中的所述血管，以预设血管生长长度，进行区域增长，以获得所述待分割图像的所述纵膈、所述动脉、所述静脉和所述背景的分割结果。
  8. 一种图像分割模型的训练方法，其特征在于，包括：
    确定样本图像，所述样本图像包括纵膈区域的纵膈、背景、动脉和静脉的第一标签以及外延区域的背景和血管的第二标签；
    基于所述样本图像训练神经网络，以生成用于获得所述纵膈区域的所述纵膈、所述背景、所述动脉和所述静脉的第一分割结果的网络模型，其中，所述神经网络为3D神经网络；
    基于所述样本图像训练级联的神经网络，以生成用于获得所述外延区域的所述背景和所述血管的第二分割结果的分割模型，其中，所述级联的神经网络包括用于特征提取的第一神经网络以及用于生成所述第二分割结果的第二神经网络。
  9. 根据权利要求8所述的训练方法，其特征在于，还包括：
    将所述样本图像的纵膈区域进行切块操作，获得多个切块图像，其中，所述多个切块图像中的每个切块图像均包括所述纵膈，
    其中，所述基于所述样本图像训练神经网络，以生成用于获得所述纵膈区域的纵膈、背景、动脉和静脉的第一分割结果的网络模型，包括：
    基于所述多个切块图像训练所述神经网络，以生成所述网络模型。
  10. 根据权利要求8或9所述的训练方法，其特征在于，所述基于所述样本图像训练级联的神经网络，以生成用于获得所述外延区域的背景和血管的第二分割结果的分割模型，包括：
    通过所述第一神经网络，对所述样本图像进行下采样操作，以获得多个第一特征图；
    通过所述第二神经网络，对所述多个第一特征图进行上采样和融合操作，以获得第二特征图；
    利用分类器对所述第二特征图进行激活，以获得所述外延区域的所述背景和所述血管的第二分割结果；
    根据所述第二分割结果和所述第二标签，获得所述级联的神经网络的损失函数值；
    根据所述损失函数值，更新所述级联的神经网络的参数，
    其中，所述第一神经网络为深度残差网络，所述第二神经网络为特征金字塔网络。
  11. 根据权利要求10所述的训练方法，其特征在于，还包括：
    将所述第二标签中的血管所在区域进行最大池化操作，获得所述第二标签中的血管所在区域膨胀后的所述样本图像的目标区域，
    其中，所述根据所述第二分割结果和所述第二标签，获得所述级联的神经网络的损失函数值，包括：
    根据所述目标区域对应的第二分割结果和所述第二标签，获取所述级联的神经网络的损失函数值。
  12. 一种图像分割装置，其特征在于，包括：
    第一分割模块，配置为根据包括背景、纵膈、动脉和静脉的待分割图像，获取所述待分割图像的纵膈区域的所述纵膈、所述动脉、所述静脉和所述背景的第一分割结果；
    第二分割模块，配置为根据所述待分割图像，获取所述待分割图像的外延区域的血管与所述背景的第二分割结果；
    获取模块，配置为根据所述第一分割结果和所述第二分割结果，获取所 述待分割图像的所述纵膈、所述动脉、所述静脉和所述背景的分割结果。
  13. 一种图像分割模型的训练装置，其特征在于，包括：
    确定模块，配置为确定样本图像，所述样本图像包括纵膈区域的纵膈、背景、动脉和静脉的第一标签以及外延区域的背景和血管的第二标签；
    第一训练模块，配置为基于所述样本图像训练神经网络，以生成用于获得所述纵膈区域的所述纵膈、所述背景、所述动脉和所述静脉的第一分割结果的网络模型，其中，所述神经网络为3D神经网络；
    第二训练模块，配置为基于所述样本图像训练级联的神经网络，以生成用于获得所述外延区域的所述背景和所述血管的第二分割结果的分割模型，其中，所述级联的神经网络包括用于特征提取的第一神经网络以及用于生成所述第二分割结果的第二神经网络。
  14. 一种电子设备，包括：
    处理器；
    用于存储所述处理器可执行指令的存储器；
    所述处理器，用于执行上述权利要求1至11中任一项所述的方法。
  15. 一种计算机可读存储介质，所述存储介质存储有计算机程序，所述计算机程序用于执行上述权利要求1至11中任一项所述的方法。
PCT/CN2021/088407 2020-07-30 2021-04-20 图像分割方法及装置、图像分割模型的训练方法及装置 WO2022021955A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2021560144A JP7250166B2 (ja) 2020-07-30 2021-04-20 画像分割方法及び装置、画像分割モデルのトレーニング方法及び装置
EP21782431.7A EP3975117A4 (en) 2020-07-30 2021-04-20 IMAGE SEGMENTATION METHOD AND APPARATUS AND IMAGE SEGMENTATION MODEL TRAINING METHOD AND APPARATUS
US17/497,954 US11972571B2 (en) 2020-07-30 2021-10-10 Method for image segmentation, method for training image segmentation model

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010752149.0A CN111899245B (zh) 2020-07-30 2020-07-30 图像分割、模型的训练方法及装置，电子设备及存储介质
CN202010752149.0 2020-07-30

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/497,954 Continuation US11972571B2 (en) 2020-07-30 2021-10-10 Method for image segmentation, method for training image segmentation model

Publications (1)

Publication Number Publication Date
WO2022021955A1 true WO2022021955A1 (zh) 2022-02-03

Family

ID=73182662

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/088407 WO2022021955A1 (zh) 2020-07-30 2021-04-20 图像分割方法及装置、图像分割模型的训练方法及装置

Country Status (2)

Country Link
CN (1) CN111899245B (zh)
WO (1) WO2022021955A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115272165A (zh) * 2022-05-10 2022-11-01 推想医疗科技股份有限公司 图像的特征提取方法、图像分割模型的训练方法和装置
CN116128954A (zh) * 2022-12-30 2023-05-16 上海强仝智能科技有限公司 一种基于生成网络的商品布局识别方法、装置及存储介质
CN116485820A (zh) * 2023-06-21 2023-07-25 杭州堃博生物科技有限公司 动静脉图像提取方法、装置及非易失性存储介质

Families Citing this family (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018176000A1 (en) 2017-03-23 2018-09-27 DeepScale, Inc. Data synthesis for autonomous control systems
US10671349B2 (en) 2017-07-24 2020-06-02 Tesla, Inc. Accelerated mathematical engine
US11157441B2 (en) 2017-07-24 2021-10-26 Tesla, Inc. Computational array microprocessor system using non-consecutive data formatting
US11893393B2 (en) 2017-07-24 2024-02-06 Tesla, Inc. Computational array microprocessor system with hardware arbiter managing memory requests
US11409692B2 (en) 2017-07-24 2022-08-09 Tesla, Inc. Vector computational unit
US11561791B2 (en) 2018-02-01 2023-01-24 Tesla, Inc. Vector computational unit receiving data elements in parallel from a last row of a computational array
US11215999B2 (en) 2018-06-20 2022-01-04 Tesla, Inc. Data pipeline and deep learning system for autonomous driving
US11361457B2 (en) 2018-07-20 2022-06-14 Tesla, Inc. Annotation cross-labeling for autonomous control systems
US11636333B2 (en) 2018-07-26 2023-04-25 Tesla, Inc. Optimizing neural network structures for embedded systems
US11562231B2 (en) 2018-09-03 2023-01-24 Tesla, Inc. Neural networks for embedded devices
SG11202103493QA (en) 2018-10-11 2021-05-28 Tesla Inc Systems and methods for training machine models with augmented data
US11196678B2 (en) 2018-10-25 2021-12-07 Tesla, Inc. QOS manager for system on a chip communications
US11816585B2 (en) 2018-12-03 2023-11-14 Tesla, Inc. Machine learning models operating at different frequencies for autonomous vehicles
US11537811B2 (en) 2018-12-04 2022-12-27 Tesla, Inc. Enhanced object detection for autonomous vehicles based on field view
US11610117B2 (en) 2018-12-27 2023-03-21 Tesla, Inc. System and method for adapting a neural network model on a hardware platform
US10997461B2 (en) 2019-02-01 2021-05-04 Tesla, Inc. Generating ground truth for machine learning from time series elements
US11150664B2 (en) 2019-02-01 2021-10-19 Tesla, Inc. Predicting three-dimensional features for autonomous driving
US11567514B2 (en) 2019-02-11 2023-01-31 Tesla, Inc. Autonomous and user controlled vehicle summon to a target
US10956755B2 (en) 2019-02-19 2021-03-23 Tesla, Inc. Estimating object properties using visual image data
EP3975117A4 (en) 2020-07-30 2022-11-16 Infervision Medical Technology Co., Ltd. IMAGE SEGMENTATION METHOD AND APPARATUS AND IMAGE SEGMENTATION MODEL TRAINING METHOD AND APPARATUS
CN111899245B (zh) * 2020-07-30 2021-03-09 推想医疗科技股份有限公司 图像分割、模型的训练方法及装置，电子设备及存储介质
CN112396565A (zh) * 2020-11-19 2021-02-23 同济大学 静脉穿刺机器人的图像及视频血管增强与分割方法和系统
CN112950734B (zh) * 2021-03-01 2023-09-22 推想医疗科技股份有限公司 冠脉重建方法、装置、电子设备和存储介质
CN113066090B (zh) * 2021-03-19 2022-02-08 推想医疗科技股份有限公司 血管分割模型的训练方法及装置、应用方法及装置
CN113516677B (zh) * 2021-04-13 2022-02-22 推想医疗科技股份有限公司 一种结构化分级管状结构血管的方法、装置及电子设备
CN114022733B (zh) * 2021-11-09 2023-06-16 中国科学院光电技术研究所 一种用于云背景下红外目标智能训练和检测方法
CN115359046B (zh) * 2022-10-19 2023-03-24 海杰亚（北京）医疗器械有限公司 一种脏器血管分割方法、装置、存储介质及电子设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110546685A (zh) * 2017-03-09 2019-12-06 皇家飞利浦有限公司 图像分割和分割预测
CN110648350A (zh) * 2019-09-26 2020-01-03 联想(北京)有限公司 一种肝脏血管分割方法、装置及电子设备
CN111462086A (zh) * 2020-03-31 2020-07-28 北京推想科技有限公司 图像分割方法及装置、神经网络模型的训练方法及装置
CN111899245A (zh) * 2020-07-30 2020-11-06 北京推想科技有限公司 图像分割、模型的训练方法及装置，电子设备及存储介质

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109785337B (zh) * 2018-12-25 2021-07-06 哈尔滨工程大学 一种基于实例分割算法的栏内哺乳动物清点方法
CN109919012A (zh) * 2019-01-28 2019-06-21 北控水务(中国)投资有限公司 一种基于卷积神经网络的污水处理指示性微生物图像识别方法
CN109934816B (zh) * 2019-03-21 2021-05-11 数坤（北京）网络科技有限公司 一种互补模型的方法、设备及计算机可读存储介质
CN109949322B (zh) * 2019-03-27 2023-06-23 中山大学 一种基于磁共振t1增强图像的脑血管图像分割方法
CN110443280B (zh) * 2019-07-05 2022-06-03 北京达佳互联信息技术有限公司 图像检测模型的训练方法、装置及存储介质
CN110516535A (zh) * 2019-07-12 2019-11-29 杭州电子科技大学 一种基于深度学习的老鼠活跃度检测方法和系统、及卫生评估方法
CN110428004B (zh) * 2019-07-31 2021-02-05 中南大学 数据失衡下基于深度学习的机械零部件故障诊断方法
CN110807764A (zh) * 2019-09-20 2020-02-18 成都智能迭迦科技合伙企业(有限合伙) 一种基于神经网络的肺癌筛查方法
CN111000563B (zh) * 2019-11-22 2020-12-11 北京理工大学 一种视网膜动脉静脉直径比自动测量方法及装置
CN111325759B (zh) * 2020-03-13 2024-04-16 上海联影智能医疗科技有限公司 血管分割方法、装置、计算机设备及可读存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110546685A (zh) * 2017-03-09 2019-12-06 皇家飞利浦有限公司 图像分割和分割预测
CN110648350A (zh) * 2019-09-26 2020-01-03 联想(北京)有限公司 一种肝脏血管分割方法、装置及电子设备
CN111462086A (zh) * 2020-03-31 2020-07-28 北京推想科技有限公司 图像分割方法及装置、神经网络模型的训练方法及装置
CN111899245A (zh) * 2020-07-30 2020-11-06 北京推想科技有限公司 图像分割、模型的训练方法及装置，电子设备及存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3975117A4 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115272165A (zh) * 2022-05-10 2022-11-01 推想医疗科技股份有限公司 图像的特征提取方法、图像分割模型的训练方法和装置
CN115272165B (zh) * 2022-05-10 2023-09-26 推想医疗科技股份有限公司 图像的特征提取方法、图像分割模型的训练方法和装置
CN116128954A (zh) * 2022-12-30 2023-05-16 上海强仝智能科技有限公司 一种基于生成网络的商品布局识别方法、装置及存储介质
CN116128954B (zh) * 2022-12-30 2023-12-05 上海强仝智能科技有限公司 一种基于生成网络的商品布局识别方法、装置及存储介质
CN116485820A (zh) * 2023-06-21 2023-07-25 杭州堃博生物科技有限公司 动静脉图像提取方法、装置及非易失性存储介质
CN116485820B (zh) * 2023-06-21 2023-09-22 杭州堃博生物科技有限公司 动静脉图像提取方法、装置及非易失性存储介质

Also Published As

Publication number Publication date
CN111899245B (zh) 2021-03-09
CN111899245A (zh) 2020-11-06

Similar Documents

Publication Publication Date Title
WO2022021955A1 (zh) 图像分割方法及装置、图像分割模型的训练方法及装置
US11972571B2 (en) Method for image segmentation, method for training image segmentation model
WO2020215984A1 (zh) 基于深度学习的医学图像检测方法及相关设备
CN110599448B (zh) 基于MaskScoring R-CNN网络的迁移学习肺部病变组织检测系统
Zhao et al. Segmentation of blood vessels using rule-based and machine-learning-based methods: a review
JP6947759B2 (ja) 解剖学的対象物を自動的に検出、位置特定、及びセマンティックセグメンテーションするシステム及び方法
US10206646B2 (en) Method and system for extracting centerline representation of vascular structures in medical images via optimal paths in computational flow fields
US9471989B2 (en) Vascular anatomy modeling derived from 3-dimensional medical image processing
JP6657132B2 (ja) 画像分類装置、方法およびプログラム
CN107169974A (zh) 一种基于多监督全卷积神经网络的图像分割方法
CN113066090B (zh) 血管分割模型的训练方法及装置、应用方法及装置
CN111899244B (zh) 图像分割、网络模型的训练方法及装置，及电子设备
Chang et al. Automatic cardiac MRI segmentation and permutation-invariant pathology classification using deep neural networks and point clouds
CN111563550B (zh) 基于图像技术的精子形态检测方法和装置
WO2022105735A1 (zh) 冠状动脉分割方法及其装置、电子设备及计算机可读存储介质
Yang et al. RADCU-Net: Residual attention and dual-supervision cascaded U-Net for retinal blood vessel segmentation
Liu et al. Self-supervised attention mechanism for pediatric bone age assessment with efficient weak annotation
Guo CSGNet: Cascade semantic guided net for retinal vessel segmentation
Shaziya et al. Pulmonary CT images segmentation using CNN and UNet models of deep learning
CN112288718B (zh) 图像处理方法及装置，电子设备及计算机可读存储介质
US20220092786A1 (en) Method and arrangement for automatically localizing organ segments in a three-dimensional image
Bhuvaneswari et al. Contrast enhancement of retinal images using green plan masking and whale optimization algorithm
CN114359208B (zh) 头颈血管分割方法及装置、可读存储介质及电子设备
CN111445456A (zh) 分类模型、网络模型的训练方法及装置、识别方法及装置
CN113744215B (zh) 三维断层扫描图像中树状管腔结构中心线的提取方法和装置

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2021560144

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2021782431

Country of ref document: EP

Effective date: 20211011

NENP Non-entry into the national phase

Ref country code: DE