CN114723683B

CN114723683B - Head and neck artery blood vessel segmentation method and device, electronic device and storage medium

Info

Publication number: CN114723683B
Application number: CN202210284383.4A
Authority: CN
Inventors: 孙岩峰; 张欢; 王少康; 陈宽
Original assignee: Infervision Medical Technology Co Ltd
Current assignee: Infervision Medical Technology Co Ltd
Priority date: 2022-03-22
Filing date: 2022-03-22
Publication date: 2023-02-17
Anticipated expiration: 2042-03-22
Also published as: CN114723683A

Abstract

The application provides a head and neck artery blood vessel segmentation method, which comprises the following steps: determining rough blood vessel region segmentation data corresponding to the head and neck medical image to be segmented based on the head and neck medical image to be segmented; based on the rough blood vessel region segmentation data, performing blood vessel segmentation on the head and neck medical image to be segmented by utilizing a first multi-scale multi-window block cutting sampling operation to obtain a blood vessel segmentation probability map and rough blood vessel segmentation data corresponding to the head and neck medical image to be segmented; determining blood vessel structure data corresponding to the head and neck medical image to be segmented based on the blood vessel segmentation probability map and the head and neck medical image to be segmented; and performing blood vessel segmentation on the head and neck medical image to be segmented by utilizing a second multi-scale multi-window block cutting sampling operation based on the blood vessel structure data to obtain blood vessel segmentation data corresponding to the head and neck medical image to be segmented. The extraction capability of the blood vessel structure is improved through multi-scale and multi-window cutting sampling operation; and the image characteristics and the blood vessel structure characteristics are combined during segmentation, and the accuracy and the integrity of segmentation are improved.

Description

Head and neck artery blood vessel segmentation method and device, electronic device and storage medium

Technical Field

The application relates to the technical field of medical image processing, in particular to a head and neck artery blood vessel segmentation method and device, electronic equipment and a computer readable storage medium.

Background

The artery blood vessels on the head and neck medical images are accurately and completely segmented, so that accurate basis is provided for subsequent analysis, and the method has important significance for discovering and treating head and neck vascular diseases.

However, in the prior art, when the head and neck artery blood vessel segmentation is performed, the extraction and understanding capability of the image features is limited, and due to limited receptive field and lack of sufficient global information, the extraction capability of the blood vessel structural features is poor, the accuracy and integrity of the artery blood vessel segmentation are affected, and a large number of cases such as fine crushing or breaking exist in the obtained blood vessel segmentation result.

Disclosure of Invention

In view of this, embodiments of the present application provide a method and an apparatus for segmenting a head and neck artery, an electronic device, and a computer-readable storage medium, so as to solve the technical problem in the prior art that the accuracy and integrity of artery vessel segmentation are poor, which causes a large amount of fine crushing or fracture in the obtained vessel segmentation result.

According to a first aspect of embodiments of the present application, there is provided a head and neck artery blood vessel segmentation method, including: determining rough blood vessel region segmentation data corresponding to the head and neck medical image to be segmented based on the head and neck medical image to be segmented; based on the rough blood vessel region segmentation data, performing blood vessel segmentation on the head and neck medical image to be segmented by utilizing a first multi-scale multi-window block sampling operation to obtain a blood vessel segmentation probability map and rough blood vessel segmentation data corresponding to the head and neck medical image to be segmented; determining blood vessel structure data corresponding to the head and neck medical image to be segmented based on the blood vessel segmentation probability map and the head and neck medical image to be segmented; based on the blood vessel structure data, performing blood vessel segmentation on the head and neck medical image to be segmented by utilizing a second multi-scale multi-window block cutting sampling operation to obtain blood vessel segmentation data corresponding to the head and neck medical image to be segmented; the first multi-scale multi-window cutting block sampling operation and the second multi-scale multi-window cutting block sampling operation are performed in the sampling and cutting mode under the scope of a plurality of sampling space resolutions and a plurality of windows by taking a preset image pixel volume as a sampling target.

In one embodiment, based on the rough blood vessel region segmentation data, performing blood vessel segmentation on the head and neck medical image to be segmented by using a first multi-scale multi-window block cutting sampling operation to obtain a blood vessel segmentation probability map and rough blood vessel segmentation data corresponding to the head and neck medical image to be segmented, including: determining a plurality of first sampling central points in the head and neck medical image to be segmented based on the rough blood vessel region segmentation data; aiming at each first sampling central point in a plurality of first sampling central points, using the first sampling central point as a center, and performing block cutting sampling on the head and neck medical image to be segmented by using N sampling spatial resolutions and M windows to obtain N first sampling block data groups corresponding to the first sampling central point, wherein each first sampling block data group in the N first sampling block data comprises M first sampling block data, N is more than or equal to 3, and M is more than or equal to 3; inputting N first sampling block data groups corresponding to the first sampling central point into a first multi-layer network vessel segmentation model to obtain vessel segmentation probability block data corresponding to the first sampling central point, wherein the first multi-layer network vessel segmentation model comprises N first network layers, and each first network layer in the N first network layers comprises a segmentation module; and obtaining a blood vessel segmentation probability map and rough blood vessel segmentation data based on the blood vessel segmentation probability block data respectively corresponding to the first sampling central points.

In one embodiment, the method for performing block cutting and sampling on a head and neck medical image to be segmented by using a first sampling central point as a center and using N sampling spatial resolutions and M windows to obtain N first sampling block data sets corresponding to the first sampling central point includes: determining the N sampling spatial resolutions and the window widths and window levels corresponding to the M windows respectively; taking the first sampling central point as a center, and respectively sampling the head and neck medical image to be segmented by utilizing N sampling spatial resolutions to obtain N pieces of first initial sampling block data corresponding to the first sampling central point; for each first initial sampling block data in the N first initial sampling block data, performing windowing sampling operation on the first initial sampling block data based on the window width and window level respectively corresponding to the M windows to obtain M first sampling block data corresponding to the first initial sampling block data; and determining N first sampling block data groups corresponding to the first sampling center point based on M first sampling block data respectively corresponding to the N first initial sampling block data.

In one embodiment, determining N sample spatial resolutions includes: determining a reference spatial resolution based on an initial spatial resolution corresponding to the sampling source domain, an image pixel volume corresponding to the sampling source domain and an image pixel volume corresponding to the sampling target domain; and calculating the reference spatial resolution in proportion to obtain N sampling spatial resolutions.

In one embodiment, the N sampled spatial resolutions are 3 sampled spatial resolutions, the 3 sampled spatial resolutions being a reference spatial resolution, 2 times the reference spatial resolution, and 4 times the reference spatial resolution, respectively; the M windows are 3 windows, the 3 windows are respectively an original graph window, a first fixed window and a second fixed window, wherein the window width of the original graph window and the window position of the window positioned in the head and neck medical image to be segmented are the same, the window width of the first fixed window is 300Hu, the window position of the first fixed window is 50Hu, the window width of the first fixed window is 300Hu, and the window position of the first fixed window is 500Hu.

In one embodiment, the N first network layers in the first multi-layer network vessel segmentation model are arranged from large to small according to the respective corresponding hierarchical resolutions; the method for obtaining the data of the blood vessel segmentation probability block corresponding to the first sampling center point by inputting the data group of N first sampling blocks corresponding to the first sampling center point into a first multilayer network blood vessel segmentation model comprises the following steps: respectively inputting the N first sampling block data groups into the segmentation modules in the respectively matched first network layers based on the sampling spatial resolutions respectively corresponding to the N first sampling block data groups, wherein the first network layer matched with a certain first sampling block data group is a first network layer corresponding to the hierarchical resolution matched with the sampling spatial resolution corresponding to the certain first sampling block data group in the N first network layers; for every two adjacent first network layers in the N first network layers, fusing the feature map corresponding to the first network layer corresponding to the large hierarchical resolution into the corresponding position in the feature map corresponding to the first network layer corresponding to the small hierarchical resolution so as to perform vessel segmentation; and performing function conversion operation on the feature graph output by the first network layer which is arranged at the last layer of the N first network layers to obtain the blood vessel segmentation probability block data corresponding to the first sampling central point.

In one embodiment, before inputting the N first sample block data sets into the partitioning modules in the matching first network layer, the method further includes: and respectively inputting the N first sampling block data groups into the self-adaptive adjusting window modules in the first network layers which are matched with the N first sampling block data groups, so as to carry out self-adaptive windowing on the sampling block data which correspond to the original image window in the M first sampling block data in each of the N first sampling block data groups.

In one embodiment, based on the blood vessel structure data, performing blood vessel segmentation on the head and neck medical image to be segmented by using a second multi-scale multi-window block sampling operation to obtain blood vessel segmentation data corresponding to the head and neck medical image to be segmented, including: determining a plurality of second sampling central points in the head and neck medical image to be segmented based on the vascular structure data; aiming at each second sampling central point in the plurality of second sampling central points, taking the second sampling central point as a center, and performing block cutting sampling on the head and neck medical image to be segmented by using N sampling spatial resolutions and M windows to obtain N second sampling block data groups corresponding to the second sampling central point, wherein each second sampling block data group in the N second sampling block data groups comprises M second sampling block data; based on the coordinates corresponding to the second sampling central point, performing block cutting and sampling operation on the vascular structure data by using N sampling spatial resolutions to obtain N vascular structure block data corresponding to the second sampling central point; inputting N second sampling block data groups and N blood vessel structure block data corresponding to a second sampling central point into a second multi-layer network blood vessel segmentation model to obtain blood vessel segmentation probability block data corresponding to the second sampling central point, wherein the second multi-layer network blood vessel segmentation model comprises N second network layers, and each second network layer in the N second network layers comprises a segmentation module; and determining blood vessel segmentation data corresponding to the head and neck medical image to be segmented based on the blood vessel segmentation probability block data corresponding to the second sampling central points.

In one embodiment, inputting N second sampling block data groups and N blood vessel structure block data corresponding to the second sampling central point into the second multi-layer network blood vessel segmentation model to obtain blood vessel segmentation probability block data corresponding to the second sampling central point, includes: inputting the N second sampling block data groups and the N blood vessel structure block data into segmentation modules in second network layers matched with the N second sampling block data groups respectively based on sampling spatial resolutions corresponding to the N second sampling block data groups and sampling spatial resolutions corresponding to the N blood vessel structure block data groups respectively, wherein the second network layer matched with a certain second sampling block data group is the second network layer corresponding to a hierarchical resolution matched with the sampling spatial resolution corresponding to a certain second sampling block data group in the N second network layers; the second network layer matched with the certain blood vessel structure block data is a second network layer corresponding to a hierarchical resolution matched with a sampling space resolution corresponding to the certain blood vessel structure block data in the N second network layers; for every two adjacent second network layers in the N second network layers, fusing the feature map corresponding to the second network layer corresponding to the large hierarchical resolution into the corresponding position in the feature map corresponding to the second network layer corresponding to the small hierarchical resolution so as to perform vessel segmentation; and performing function conversion operation on the feature map output by the second network layer which is arranged at the last layer of the N second network layers to obtain the blood vessel segmentation probability block data corresponding to the second sampling central point.

In one embodiment, the determining of the vascular structure data corresponding to the head and neck medical image to be segmented based on the vascular segmentation probability map and the head and neck medical image to be segmented comprises: performing binarization operation on the blood vessel segmentation probability map to obtain binarization blood vessel segmentation data corresponding to the head and neck medical image to be segmented; performing center line extraction operation on the binary vessel segmentation data to obtain an initial vessel center line corresponding to the head and neck medical image to be segmented; and performing blood vessel structure growth operation on the head and neck medical image to be segmented based on the initial blood vessel central line by utilizing a simulated hemodynamics mode to obtain blood vessel structure data.

In one embodiment, the training method of the first multi-layer network vessel segmentation model comprises the following steps: determining a first model to be trained, wherein the first model to be trained comprises N first network layers to be trained, each of the N first network layers to be trained comprises a segmentation module to be trained, and N hierarchical resolutions corresponding to the N first network layers to be trained are respectively matched with N sampling spatial resolutions; determining at least one first cephaloneck blood vessel sample image and first blood vessel segmentation marking data corresponding to the at least one first cephaloneck blood vessel sample image; determining N first sampling block data group samples corresponding to a plurality of first sampling central point samples in a first head and neck blood vessel sample image by utilizing a first multi-scale multi-window block cutting sampling operation based on first blood vessel segmentation marking data; for each first sampling central point sample in the multiple first sampling central point samples, obtaining N first vessel segmentation prediction layer data by utilizing N first sampling block data group samples corresponding to the first sampling central point sample based on a first model to be trained, and determining N first layer loss function values based on the N first vessel segmentation prediction layer data and first vessel segmentation marking data; weighting the N first-layer loss function values based on preset weight values corresponding to the N first-layer loss function values respectively to obtain first weighted loss function values; and adjusting parameters of the first model to be trained based on the first weighted loss function value until the first weighted loss function value meets a preset condition, thereby obtaining a first multilayer network vessel segmentation model.

In one embodiment, the obtaining N first vessel segmentation prediction layer data by using N first sample block data group samples corresponding to the first sample central point sample based on the first model to be trained, and determining N first layer loss function values based on the N first vessel segmentation prediction layer data and the first vessel segmentation flag data includes: inputting the N first sampling block data group samples into the first network layer to be trained matched with the N first sampling block data group samples for segmentation to obtain N first blood vessel segmentation prediction layer data based on the sampling spatial resolution corresponding to the N first sampling block data group samples and the hierarchical resolution corresponding to the N first network layers to be trained; for each first blood vessel segmentation prediction layer data in the N first blood vessel segmentation prediction layer data, determining skeleton region data in the first blood vessel segmentation prediction layer data, blood vessel foreground region data in the first blood vessel segmentation prediction layer data, and foreground neighborhood data in the first blood vessel segmentation prediction layer data; and determining a first layer loss function value corresponding to the first blood vessel segmentation prediction layer data based on the first blood vessel segmentation prediction layer data, the first blood vessel segmentation marking data and a preset layer loss function, wherein in the first preset layer loss function, the weight of the skeleton region data is greater than that of the foreground neighborhood region data, and the weight of the foreground neighborhood region data is greater than that of the blood vessel foreground region data.

In one embodiment, the training method of the second multi-layer network vessel segmentation model comprises the following steps: determining a second model to be trained, wherein the second model to be trained comprises N second network layers to be trained, each of the N second network layers to be trained comprises a segmentation module to be trained, and N hierarchical resolutions corresponding to the N second network layers to be trained are respectively matched with N sampling spatial resolutions; determining at least one second head and neck blood vessel sample image, blood vessel structure marking data corresponding to the at least one second head and neck blood vessel sample image, and second blood vessel segmentation marking data corresponding to the at least one second head and neck blood vessel sample image; determining N second sampling block data group samples corresponding to a plurality of second sampling center point samples in a second head and neck blood vessel sample image by utilizing second multi-scale multi-window block cutting sampling operation based on second blood vessel segmentation marking data; based on the coordinates corresponding to the second sampling central point samples, carrying out block cutting sampling operation on the blood vessel structure mark data to obtain N blood vessel structure mark block data corresponding to the second sampling central point samples; for each second sampling central point sample in the plurality of second sampling central point samples, based on a second model to be trained, obtaining N second vessel segmentation prediction layer data by using N second sampling block data group samples and N vessel structure mark block data corresponding to the second sampling central point sample, and determining N second layer loss function values based on the N second vessel segmentation prediction layer data and the second vessel segmentation mark data; weighting the N second-layer loss function values based on preset weight values corresponding to the N second-layer loss function values respectively to obtain second weighted loss function values; and adjusting parameters of the second model to be trained based on the second weighted loss function value until the second weighted loss function value meets the preset condition, so as to obtain a second multilayer network vessel segmentation model.

According to a second aspect of embodiments of the present application, there is provided a head and neck artery blood vessel segmentation device, comprising: the rough blood vessel region segmentation data determination module is configured to determine rough blood vessel region segmentation data corresponding to the head and neck medical image to be segmented based on the head and neck medical image to be segmented; the first multi-scale multi-window blood vessel segmentation module is configured to perform blood vessel segmentation on the head and neck medical image to be segmented by using first multi-scale multi-window block sampling operation based on the rough blood vessel region segmentation data to obtain a blood vessel segmentation probability map and rough blood vessel segmentation data corresponding to the head and neck medical image to be segmented; the blood vessel structure data determining module is configured to determine blood vessel structure data corresponding to the head and neck medical image to be segmented based on the blood vessel segmentation probability map and the head and neck medical image to be segmented; the second multi-scale multi-window blood vessel segmentation is configured to perform blood vessel segmentation on the head and neck medical image to be segmented by utilizing second multi-scale multi-window block sampling operation based on the blood vessel structure data to obtain blood vessel segmentation data corresponding to the head and neck medical image to be segmented; the first multi-scale multi-window block cutting sampling operation and the second multi-scale multi-window block cutting sampling operation are performed on the sampling blocks under the conditions of multiple sampling space resolutions and multiple windows by taking a preset image pixel volume as a sampling target.

According to a third aspect of embodiments herein, there is provided an electronic device comprising: a processor; and a memory having stored therein computer program instructions which, when executed by the processor, cause the processor to perform the method of head and neck artery vessel segmentation as described above in relation to the first aspect.

According to a fourth aspect of embodiments herein, there is provided a computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, cause the processor to perform the method for head and neck artery vessel segmentation as described in the first aspect above.

According to the head and neck artery blood vessel segmentation method provided by the embodiment of the application, through multi-scale and multi-window block cutting sampling operation, not only can image features be fully extracted, but also the global information of blood vessels can be fully referred to, and the blood vessel structure extraction capability is improved; and the features of the blood vessel image layer are extracted firstly, then the features of the blood vessel structure layer are extracted, and then the fusion is carried out for segmentation, so that the image features and the blood vessel structure features are fused and taken into consideration in the segmentation process, the segmentation accuracy and the segmentation integrity are greatly improved, and the probability of breakage or fracture in the blood vessel segmentation result is reduced.

Drawings

Fig. 1 is a schematic flow chart of a head and neck artery blood vessel segmentation method according to an embodiment of the present application.

Fig. 2 is a schematic flow chart of a head and neck artery blood vessel segmentation method according to an embodiment of the present application.

Fig. 3 is a schematic flow chart of a head and neck artery blood vessel segmentation method according to an embodiment of the present application.

Fig. 4 is a schematic flow chart of a head and neck artery blood vessel segmentation method according to an embodiment of the present application.

Fig. 5 is a schematic flowchart illustrating a method for segmenting a head and neck artery vessel according to an embodiment of the present application.

Fig. 6 is a flowchart illustrating a training method of a first multi-layer network vessel segmentation model according to an embodiment of the present application.

Fig. 7 is a flowchart illustrating a training method of a first multi-layer network vessel segmentation model according to an embodiment of the present application.

Fig. 8 is a flowchart illustrating a training method of a second multi-layer network vessel segmentation model according to an embodiment of the present application.

Fig. 9 is a schematic structural diagram of a head and neck artery vascular segmentation apparatus provided in an embodiment of the present application.

Fig. 10 is a schematic structural diagram of a first multi-scale multi-window vessel segmentation module according to an embodiment of the present application.

Fig. 11 is a schematic structural diagram of a first sample block data group unit according to an embodiment of the present disclosure.

Fig. 12 is a schematic structural diagram of a first vessel segmentation probability block data determination unit according to an embodiment of the present application.

Fig. 13 is a schematic structural diagram of a second multi-scale multi-window vessel segmentation module according to an embodiment of the present application.

Fig. 14 is a schematic structural diagram of a second vessel segmentation probability block data determination unit according to an embodiment of the present application.

Fig. 15 is a schematic structural diagram of a blood vessel structure data determination module according to an embodiment of the present application.

Fig. 16 is a schematic structural diagram of a first training module according to an embodiment of the present application.

Fig. 17 is a schematic structural diagram of a second training module according to an embodiment of the present application.

Fig. 18 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

As described above, for the head and neck medical image, for example: the method has the advantages that the head and neck enhanced CT (Computed Tomography) image accurately and completely segments the arterial blood vessels on the head and neck medical image, provides accurate basis for subsequent analysis of blood vessel classification, blood vessel centerline extraction, key blood vessel stenosis rate calculation, blood vessel focus detection and the like, and has important significance for discovering and treating head and neck blood vessel diseases.

However, in the prior art, when the head and neck artery blood vessel segmentation is performed, the capability of extracting and understanding image features is limited, the receptive field is limited, enough global information is lacked, the capability of extracting the blood vessel structural features is poor, the accuracy and the integrity of the artery blood vessel segmentation are affected, and the obtained blood vessel segmentation result has a large number of cases of fine crushing or fracture and the like

In order to solve the above problems, embodiments of the present application provide a method for segmenting a head and neck artery blood vessel, wherein excessive scale multi-window block-cutting sampling operations can fully extract image features, and can fully refer to blood vessel global information to improve the ability of extracting a blood vessel structure; and the features of the blood vessel image layer are extracted firstly, then the features of the blood vessel structure layer are extracted, and then the fusion is carried out for segmentation, so that the image features and the blood vessel structure features are fused and taken into consideration in the segmentation process, the segmentation accuracy and the segmentation integrity are greatly improved, and the probability of breakage or fracture in the blood vessel segmentation result is reduced.

The method for segmenting a blood vessel of a head and neck artery, the device for segmenting a blood vessel of a head and neck artery, the electronic device and the computer-readable storage medium according to the embodiments of the present application will be described in detail below with reference to fig. 1 to 18.

Exemplary head and neck artery vessel segmentation method

Fig. 1 is a schematic flow chart of a head and neck artery blood vessel segmentation method according to an embodiment of the present application. As shown in fig. 1, the method for segmenting the blood vessel of the head and neck artery comprises the following steps.

S101: and determining rough blood vessel region segmentation data corresponding to the head and neck medical image to be segmented based on the head and neck medical image to be segmented.

In particular, the rough vessel region segmentation data roughly distinguishes the artery vessels of different classes from the background region using different labels, i.e., can roughly characterize the regions to which the artery vessels of different classes each correspond. The head and neck arterial vessels include arterial vessel categories such as aorta, neck artery, and intracranial artery.

In one embodiment, a head and neck medical image to be segmented is input into a pre-trained 3d semantic segmentation model for artery vessel segmentation and vessel category prediction, one branch of the 3d semantic segmentation model is responsible for segmenting vessels, the other branches are responsible for predicting vessel categories, and the 3d semantic segmentation model outputs three artery vessel categories of aorta, neck artery and intracranial artery and respective approximate position areas of the three artery vessel categories.

It should be noted that the head and neck medical image to be segmented includes, but is not limited to, a head and neck enhanced CT image. The 3d semantic segmentation model includes, but is not limited to, the ResUnet network model structure.

It should be further noted that, because the main blood vessel types of the rough blood vessel region segmentation data are the types of artery blood vessels such as aorta, neck artery, and intracranial artery, the subsequent artery blood vessel segmentation step is performed on the basis of the rough blood vessel region segmentation data, and the subsequent blood vessel segmentation is performed on this basis essentially, therefore, the "blood vessel segmentation", "blood vessel structure", and "blood vessel region" mentioned in the description of the present application are "artery blood vessel segmentation", "artery blood vessel structure", and "artery blood vessel region" essentially, and will not be described again in the following.

S102: based on the rough blood vessel region segmentation data, performing blood vessel segmentation on the head and neck medical image to be segmented by utilizing a first multi-scale multi-window block cutting sampling operation to obtain a blood vessel segmentation probability map and rough blood vessel segmentation data corresponding to the head and neck medical image to be segmented.

In particular, the first multi-scale multi-window tile sampling operation is an isotropic tile sampling operation, the sampling process is performed at a plurality of sampling spatial resolutions and multi-window categories, and the sampling target is a preset image pixel volume (i.e., the sampled tiles are the same in pixel size).

The blood vessel segmentation probability map is used for representing the probability that each pixel point on the head and neck medical image to be segmented belongs to the artery blood vessel. The rough vessel segmentation data is a rough label obtained based on the vessel segmentation probability map that can distinguish the arterial vessel from the background.

After obtaining the regions corresponding to the different categories of arterial blood vessels, based on the regions corresponding to the different categories of arterial blood vessels, performing sufficient feature extraction on the head and neck medical image to be segmented by using a first multi-scale multi-window block sampling operation to perform blood vessel segmentation, and obtaining a blood vessel segmentation probability map and rough blood vessel segmentation data corresponding to the head and neck medical image to be segmented.

S103: and determining the blood vessel structure data corresponding to the head and neck medical image to be segmented based on the blood vessel segmentation probability map and the head and neck medical image to be segmented.

Exemplarily, performing binarization operation on the blood vessel segmentation probability map (determining pixel points with probability values larger than or equal to a preset threshold value as belonging to blood vessels, and determining pixel points with probability values smaller than the preset threshold value as not belonging to blood vessels), so as to obtain binarization blood vessel segmentation data corresponding to the head and neck medical image to be segmented; performing center line extraction operation on the binary vessel segmentation data to obtain an initial vessel center line corresponding to the head and neck medical image to be segmented; and performing vascular structure growth operation on the head and neck medical image to be segmented by utilizing a hemodynamic simulation mode based on the initial vascular center line to obtain vascular structure data.

It should be noted that, in order to avoid false positive as much as possible and provide an accurate basis for the subsequent growth of a more accurate vascular structure by following a hemodynamic simulation, the preset threshold value of the binarization operation is relatively large, for example: 0.8, 0.9, or 0.95.

S104: and based on the blood vessel structure data, performing blood vessel segmentation on the head and neck medical image to be segmented by utilizing second multi-scale multi-window block cutting sampling operation to obtain blood vessel segmentation data corresponding to the head and neck medical image to be segmented.

Specifically, the second multi-scale multi-window tile sampling operation is also an isotropic tile sampling operation, the sampling process is also performed in multiple sampling spatial resolutions and multi-window categories, and the sampling target is a preset image pixel volume (i.e., the blocks obtained by sampling are the same in pixel size).

After the blood vessel structure data are obtained, under the guidance of the blood vessel structure data, image features and blood vessel structure features are fully fused by utilizing second multi-scale multi-window block cutting sampling operation, and blood vessel segmentation is carried out on the head and neck medical image to be segmented to obtain blood vessel segmentation data corresponding to the head and neck medical image to be segmented. The blood vessel is cut apart under the guide of vascular structure, has closed structure disappearance problems such as blood vessel fracture, has avoided the problem of the free false positive blood vessel of a large amount of cracked simultaneously, improves the accuracy and the integrality of cutting apart by a wide margin.

It should be noted that, compared with the first multi-scale multi-window block cutting and sampling operation, the second multi-scale multi-window block cutting and sampling operation only has different operation objects, the operation object of the first multi-scale multi-window block cutting and sampling is the head and neck medical image to be segmented, and the operation object of the second multi-scale multi-window block cutting and sampling is the head and neck medical image to be segmented and the blood vessel structure data.

In the embodiment of the application, on the overall segmentation process, rough blood vessel region segmentation data corresponding to a head and neck medical image to be segmented is determined; based on the rough blood vessel region segmentation data, performing a first multi-scale multi-window block cutting sampling operation to perform sufficient feature extraction, and performing blood vessel segmentation on the head and neck medical image to be segmented to obtain a blood vessel segmentation probability map and rough blood vessel segmentation data; then, taking the segmentation probability map as a basis, and carrying out blood vessel structure growth on the head and neck medical image to be segmented to obtain blood vessel structure data; and finally, using the blood vessel structure data as guidance, and utilizing a second multi-scale multi-window cutting block sampling operation so as to fully fuse the image characteristics and the blood vessel structure characteristics, and performing blood vessel segmentation on the head and neck medical image to be segmented to obtain blood vessel segmentation data corresponding to the head and neck medical image to be segmented.

In the segmentation process, through multi-scale and multi-window block cutting sampling operation, not only can the image characteristics be fully extracted, but also the global information of the blood vessel can be fully referred to, and the extraction capability of the blood vessel structure is improved; and the features of the blood vessel image layer are extracted firstly, then the features of the blood vessel structure layer are extracted, and then the fusion is carried out for segmentation, so that the image features and the blood vessel structure features are fused and taken into consideration in the segmentation process, the segmentation accuracy and the segmentation integrity are greatly improved, and the probability of breakage or fracture in the blood vessel segmentation result is reduced.

Fig. 2 is a schematic flow chart of a head and neck artery blood vessel segmentation method according to an embodiment of the present application. As shown in fig. 2, based on the rough blood vessel region segmentation data, performing blood vessel segmentation on the head and neck medical image to be segmented by using a first multi-scale multi-window block sampling operation, so as to obtain a blood vessel segmentation probability map and rough blood vessel segmentation data corresponding to the head and neck medical image to be segmented, including the following steps.

S201: based on the rough blood vessel region segmentation data, a plurality of first sampling central points are determined in the head and neck medical image to be segmented.

Specifically, a plurality of first sampling central points are determined in the head and neck medical image to be segmented by using the blood vessel regions corresponding to different categories of artery blood vessels determined by the rough blood vessel region segmentation data, and sampling centers are provided for subsequent sampling.

S202: aiming at each first sampling central point in the plurality of first sampling central points, the first sampling central point is used as a center, and the N sampling spatial resolutions and the M windows are utilized to perform block cutting sampling on the head and neck medical image to be segmented, so that N first sampling block data groups corresponding to the first sampling central points are obtained.

Illustratively, each first sample block data group of the N first sample block data includes M first sample block data, N is greater than or equal to 3, and M is greater than or equal to 3.

Specifically, in the image sampling process, the sampling source domain target area sequentially samples images into the sampling target area according to the sampling matrix. The sampling matrix is obtained by calculating the sampling spatial resolution, and therefore, the sampling spatial resolution needs to be determined. The sampling spatial resolution can distinguish the size or dimension of the smallest unit in the sampling process for an index that characterizes the detailed information of the target (i.e., arterial vessel).

The window is used to purposefully observe specific parts and tissues through setting the window level and the window width, and highlight the contrast and the level of the interested structures (namely, artery vessels) in the image. The window width is the range of CT values set for optimal display of the structure of interest, the midpoint of the window width range being the so-called window level.

Although the pixel sizes (image pixel volumes) of the N first sample block data groups are the same, the N first sample block data groups adopt N sampling spatial resolutions, and because the sampling spatial resolutions are different and the covered physical spaces are different in size, the blood vessel information carried by each of the N first sample block data groups is also different.

By utilizing N sampling spatial resolutions and M windows, the head and neck medical image to be segmented is subjected to block sampling, so that not only can a foundation be provided for extracting image features from different scales, but also global information can be provided for segmentation. In addition, M first sample block data included in each first sample block data group are obtained by M window windowing samples, and different structures can present different development characteristics through the M windows, so that the retention of each dimension information of the image is ensured, and more information is provided for segmentation.

Compared with the prior art in which one sampling central point is directly sampled to obtain one sampling block, in the embodiment of the application, the N first sampling block data groups corresponding to the first sampling central point are beneficial to fully extracting image features from different scales, and are beneficial to improving the accuracy of arterial blood vessel segmentation.

S203: and inputting N first sampling block data groups corresponding to the first sampling central point into a first multilayer network vessel segmentation model to obtain vessel segmentation probability block data corresponding to the first sampling central point.

Illustratively, the first multi-layer network vessel segmentation model includes N first network layers, each of the N first network layers including one segmentation module.

Specifically, N first sampling block data groups corresponding to the first sampling central point are respectively input into the corresponding first network layers to perform artery blood vessel segmentation, and blood vessel segmentation probability block data corresponding to the first sampling central point are obtained.

S204: and obtaining a blood vessel segmentation probability map and rough blood vessel segmentation data based on the blood vessel segmentation probability block data respectively corresponding to the plurality of first sampling central points.

Specifically, the above steps (S202 and S203) are performed for each of the plurality of first sampling center points, blood vessel segmentation probability block data corresponding to each of the plurality of first sampling center points is obtained, the plurality of blood vessel segmentation probability block data are spliced to obtain a blood vessel segmentation probability map corresponding to the head and neck medical image to be segmented, and rough blood vessel segmentation data (i.e., rough blood vessel segmentation markers) are obtained according to the blood vessel segmentation probability map.

In the embodiment of the application, N sampling spatial resolutions and M windows are utilized to perform block sampling on the head and neck medical image to be segmented, so that the image features are fully extracted, the global information of blood vessels is fully referred, and the purpose of improving the extraction capability of the blood vessel structure is achieved.

Fig. 3 is a schematic flowchart of a head and neck artery blood vessel segmentation method according to an embodiment of the present application. As shown in fig. 3, the method comprises the following steps of performing block cutting and sampling on a head and neck medical image to be segmented by using N sampling spatial resolutions and M windows to obtain N first sampling block data groups corresponding to a first sampling central point.

S301: and determining the window width and the window level corresponding to the N sampling spatial resolutions and the M windows respectively.

In one embodiment, determining N sample spatial resolutions comprises: determining a reference spatial resolution based on an initial spatial resolution corresponding to a sampling source domain, an image pixel volume corresponding to the sampling source domain and an image pixel volume corresponding to a sampling target domain; and calculating the reference spatial resolution in proportion to obtain N sampling spatial resolutions.

Specifically, the initial spatial resolution corresponding to the sampling source domain is the physical spatial resolution corresponding to the head and neck medical image to be segmented. Because the image pixel volume corresponding to the sampling source domain is the same as the image pixel volume corresponding to the sampling target domain (i.e. the physical dimensions before and after sampling are the same), the reference spatial resolution is the physical spatial resolution corresponding to the segmented head and neck medical image. The N sampling spatial resolutions are obtained by calculating the reference spatial resolution in proportion.

S302: and taking the first sampling central point as a center, and sampling the head and neck medical image to be segmented by respectively utilizing N sampling spatial resolutions to obtain N pieces of first initial sampling block data corresponding to the first sampling central point.

Specifically, based on an image pixel volume corresponding to a sampling source domain, an image pixel volume corresponding to a sampling target domain and N sampling spatial resolutions, N sampling matrices are determined, and with a first sampling central point as a center, the N sampling matrices are respectively used for sampling a head and neck medical image to be segmented to obtain N pieces of first initial sampling block data corresponding to the first sampling central point.

S303: and performing windowing sampling operation on the first initial sampling block data based on the window width and window level respectively corresponding to the M windows aiming at each first initial sampling block data in the N first initial sampling block data to obtain M first sampling block data corresponding to the first initial sampling block data.

S304: and determining N first sampling block data groups corresponding to the first sampling central point based on M first sampling block data respectively corresponding to the N first initial sampling block data.

Specifically, 1 first initial sample block data corresponds to M first sample block data forming 1 first sample block data group, that is, 1 first initial sample block data corresponds to 1 first sample block data group.

And 1 first initial sampling block data corresponds to 1 first sampling block data group, 1 first sampling central point corresponds to N first initial sampling block data, and then 1 first sampling central point corresponds to N first sampling block data groups.

In the embodiment of the application, through the steps, the purpose of obtaining the data group of the N first sampling blocks corresponding to the first sampling central point is achieved, so that the purposes of fully extracting image features, fully referring to the global information of the blood vessel and improving the extraction capability of the blood vessel structure are achieved.

In an alternative embodiment, the N sampling spatial resolutions are 3 sampling spatial resolutions, and the 3 sampling spatial resolutions are a reference spatial resolution, 2 times the reference spatial resolution, and 4 times the reference spatial resolution, respectively; the M windows are 3 windows, the 3 windows are respectively an original graph window, a first fixed window and a second fixed window, wherein the window width of the original graph window and the window position of the window positioned in the head and neck medical image to be segmented are the same, the window width of the first fixed window is 300Hu, the window position of the first fixed window is 50Hu, the window width of the first fixed window is 300Hu, and the window position of the first fixed window is 500Hu.

Specifically, for example, for each first sampling central point of a plurality of first sampling central points in a head and neck medical image to be segmented, sampling the head and neck medical image to be segmented by using a reference spatial resolution, a 2-fold reference spatial resolution, and a 4-fold reference spatial resolution, respectively, to obtain a first initial sample block S, a first initial sample block 2S, and a first initial sample block 4S corresponding to the first sampling central point.

And performing windowing sampling operation on the first initial sample block S by using an original image window, a first fixed window and a second fixed window respectively to obtain a first sampling block S of an original window, a first sampling block S of a window 1 and a first sampling block S of a window 2, wherein the first sampling block S of the original window, the first sampling block S of the window 1 and the first sampling block S of the window 2 form a first sampling block data group S. By analogy, for the first initial sample block 2S and the first initial sample block 4S, windowing sampling operations are performed respectively using an original image window, a first fixed window, and a second fixed window to obtain a first sample block data group 2S (the first sample block data group 2S includes the original window first sample block 2S, the window 1 first sample block 2S, and the window 2 first sample block 2S) and a first sample block data group 4S (the first sample block data group 4S includes the original window first sample block 4S, the window 1 first sample block 4S, and the window 2 first sample block 4S). Thus, the first sample block data group S, the first sample block data group 2S, and the first sample block data group 4S corresponding to the first sample center point are obtained.

Fig. 4 is a schematic flowchart of a head and neck artery blood vessel segmentation method according to an embodiment of the present application. In this embodiment, the first multi-layer network vessel segmentation model includes N first network layers, and the N first network layers are arranged from large to small according to respective corresponding hierarchical resolutions. As shown in fig. 4, the step of inputting N first sampling block data groups corresponding to the first sampling central point into the first multi-layer network vessel segmentation model to obtain vessel segmentation probability block data corresponding to the first sampling central point includes the following steps.

S401: and respectively inputting the N first sampling block data groups into the respectively matched segmentation modules in the first network layer based on the sampling spatial resolution corresponding to the N first sampling block data groups.

Illustratively, the first network layer matched to a certain first sample block data group is, of the N first network layers, a first network layer corresponding to a hierarchical resolution matched to a sampling spatial resolution corresponding to the certain first sample block data group.

Specifically, taking the above example, the sampling spatial resolutions corresponding to the first sample block data group S, the first sample block data group 2S, and the first sample block data group 4S are the reference spatial resolution, 2 times the reference spatial resolution, and 4 times the reference spatial resolution, respectively. The first multi-layered network vessel segmentation model includes 3 first network layers, a first network layer 4S (corresponding to 4 times the reference spatial resolution), a first network layer 2S (corresponding to 2 times the reference spatial resolution), and a first network layer S (corresponding to the reference spatial resolution). And inputting the first sampling center point corresponding to the first sampling block data group S, the first sampling block data group 2S and the first sampling block data group 4S into segmentation modules in the first network layer S, the first network layer 2S and the first network layer 4S respectively to perform artery vessel segmentation.

It is to be noted that the partitioning module in each first network layer may be, but is not limited to, a ResUnet network model structure.

S402: and for every two adjacent first network layers in the N first network layers, fusing the feature maps corresponding to the first network layers corresponding to the large hierarchical resolution into the corresponding positions in the feature maps corresponding to the first network layers corresponding to the small hierarchical resolution to perform vessel segmentation.

For example, the above example is carried out, the center of the feature map corresponding to the first network layer 4S is taken as the center, a half-sized region of the feature map is resampled to be the same as the feature map, the resampled feature map is fused to a corresponding position in the feature map corresponding to the first network layer 4S, and the subsequent segmentation is performed.

S403: and performing function conversion operation on the feature graph output by the first network layer which is arranged at the last layer of the N first network layers to obtain the blood vessel segmentation probability block data corresponding to the first sampling central point.

In the embodiment of the application, considering that the hierarchical resolutions of N first network layers of the first multilayer network are different and the reception fields are different, the feature graph corresponding to the first network layer corresponding to the large hierarchical resolution is fused into the corresponding position of the feature graph corresponding to the first network layer corresponding to the small hierarchical resolution, so that the features of the upper-level network with the larger scale and the wider reception fields are fused, the features are more global, the global information loss caused by the fine dicing process is made up, and the segmentation efficiency is improved.

In an alternative embodiment, as shown in fig. 4, before the step of inputting the N first sample block data sets into the partitioning modules in the matching first network layer, the following steps are further included.

S400: and respectively inputting the N first sampling block data groups into the self-adaptive window adjusting modules in the first network layers which are matched with each other, so as to carry out self-adaptive windowing on the sampling block data corresponding to the original image window in the M first sampling block data in each of the N first sampling block data groups.

Specifically, the purpose of windowing is to highlight a target object, since one of the M first sample block data in each first sample block data group only uses an original image window, in order to better extract features during vessel segmentation, adaptive windowing is performed on the first sample block data only using the original image window. Therefore, the N first sample block data groups are input into the adaptive window adjusting module for adaptive windowing.

The self-adaptive window adjusting module is provided with learnable parameters theta and gamma which respectively correspond to window width and window level variance distribution, and the parameters are dynamically learned and adjusted in the model training process to realize the learning of a self-adaptive window, thereby realizing the self-adaptive window adding operation.

In the embodiment of the application, the important information of the image is highlighted through self-adaptive windowing operation, so that the purpose of extracting the features better is achieved.

Fig. 5 is a schematic flow chart of a head and neck artery blood vessel segmentation method according to an embodiment of the present application. As shown in fig. 5, the step of performing vessel segmentation on the head and neck medical image to be segmented by using a second multi-scale multi-window block sampling operation based on the vessel structure data to obtain vessel segmentation data corresponding to the head and neck medical image to be segmented includes the following steps.

S501: based on the vascular structure data, a plurality of second sampling center points are determined in the head and neck medical image to be segmented.

Specifically, after the problem of structural deficiency such as vascular fracture is solved, after vascular structure data are obtained, the vascular structure data are used as reference, a plurality of second sampling central points are determined in the head and neck medical image to be segmented, and sampling centers are provided for subsequent sampling.

S502: and aiming at each second sampling central point in the plurality of second sampling central points, taking the second sampling central point as a center, and performing block cutting sampling on the head and neck medical image to be segmented by utilizing N sampling spatial resolutions and M windows to obtain N second sampling block data groups corresponding to the second sampling central points.

Specifically, the specific steps of obtaining N second sample block data groups are similar to the specific steps of obtaining N second sample block data groups, which have been described in detail in step 202 and the steps in fig. 3, and are not described again here.

S503: based on the coordinates corresponding to the second sampling central point, the vascular structure data are subjected to block cutting and sampling operation by utilizing N sampling spatial resolutions, and N vascular structure block data corresponding to the second sampling central point are obtained.

Specifically, it is considered that the vascular structure data is subjected to a block sampling operation because the vascular structure data is required to be guided and the image feature and the vascular structure feature are combined to perform segmentation. And selecting coordinates corresponding to the second sampling central point from the blood vessel structure data as a sampling center, and performing block cutting and sampling operation on the blood vessel structure data by using N sampling spatial resolutions to obtain N blood vessel structure block data corresponding to the second sampling central point.

S504: and inputting the N second sampling block data groups and the N blood vessel structure block data corresponding to the second sampling central point into a second multilayer network blood vessel segmentation model to obtain blood vessel segmentation probability block data corresponding to the second sampling central point.

Illustratively, the second multi-layer network vessel segmentation model comprises N second network layers, each of the N second network layers comprising one segmentation module.

Specifically, the N second sampling block data groups corresponding to the second sampling center points respectively correspond to N sampling spatial resolutions, and the N vascular structure block data groups corresponding to the second sampling center points also respectively correspond to N sampling spatial resolutions. The second multilayer network vessel segmentation model comprises N interlayer resolutions corresponding to N second network layers respectively, and the N interlayer resolutions are matched with the N sampling spatial resolutions. Therefore, the N second sample block data groups and the N blood vessel structure block data are respectively input into the segmentation modules in the respectively matched second network layers, that is, each second network layer receives 1 second sample block data group and 1 blood vessel structure block data corresponding to the second sampling center point.

And for every two adjacent second network layers in the N second network layers, fusing the feature maps corresponding to the second network layers corresponding to the large hierarchical resolution into the corresponding positions in the feature maps corresponding to the second network layers corresponding to the small hierarchical resolution to perform vessel segmentation.

The specific details of step S504 are the same as those of the step shown in fig. 4, except that each second network layer receives 1 second sampling block data group and 1 blood vessel structure block data corresponding to the second sampling central point, and each first network layer only receives 1 first sampling block data group corresponding to the first sampling central point, and other specific details are similar, which are not repeated herein.

S505: and determining blood vessel segmentation data corresponding to the head and neck medical image to be segmented based on the blood vessel segmentation probability block data respectively corresponding to the second sampling central points.

Specifically, the above-described steps (S502, S503, and S504) are performed for each of the plurality of second sampling center points, vessel segmentation probability block data corresponding to each of the plurality of second sampling center points is obtained, and final vessel segmentation data is obtained by using stitching and binarization operations.

In the embodiment of the application, through the steps, the purposes of fully fusing image characteristics and blood vessel structure characteristics and fully referring to blood vessel global information by guiding of blood vessel structure data and through multi-scale and multi-window block cutting sampling operation, greatly improving the accuracy and the integrity of segmentation and reducing the probability of breakage or fracture in a blood vessel segmentation result are achieved.

Training method of exemplary first multilayer network vessel segmentation model

Fig. 6 is a flowchart illustrating a training method of a first multi-layer network vessel segmentation model according to an embodiment of the present application. The training method of the first multilayer network vessel segmentation model provided by the embodiment of the application is used for training a first model to be trained, the first model to be trained comprises N first network layers to be trained, each first network layer to be trained comprises a segmentation module to be trained, and N hierarchical resolutions corresponding to the N first network layers to be trained are respectively matched with N sampling space resolutions corresponding to a first multi-scale multi-window block cutting sampling operation.

As shown in fig. 6, the training method of the first multi-layer network vessel segmentation model includes the following steps.

S601: and determining first blood vessel segmentation mark data corresponding to the at least one first blood vessel neck blood vessel sample image and the at least one first blood vessel neck blood vessel sample image.

Specifically, the first blood vessel segmentation marker data corresponding to the first cephalad and cervical blood vessel sample image includes, but is not limited to, being obtained by using artificial labeling.

S602: and determining N first sampling block data group samples corresponding to a plurality of first sampling central point samples in the first head and neck blood vessel sample image by utilizing a first multi-scale multi-window block cutting sampling operation based on the first blood vessel segmentation marking data.

Specifically, since the first blood vessel segmentation marker data has segmented the head and neck artery blood vessel from the background, a plurality of first sampling center point samples are determined in the first head and neck blood vessel sample image using the first blood vessel segmentation marker data. For each first sampling central point sample in the plurality of first sampling central point samples, taking the first sampling central point sample as a center, and performing block cutting sampling on the first cephalic and cervical blood vessel sample image by using N sampling spatial resolutions and M windows to obtain N first sampling block data group samples corresponding to the first sampling central point sample, so as to obtain N first sampling block data group samples corresponding to the plurality of first sampling central point samples.

Specifically, the specific steps of obtaining N first sample block data group samples are similar to the specific steps of obtaining N first sample block data groups, which have been described in detail in step 202 and the steps in fig. 3, and are not described again here.

S603: for each first sampling central point sample in the multiple first sampling central point samples, based on a first model to be trained, N first sampling block data group samples corresponding to the first sampling central point sample are utilized to obtain N first blood vessel segmentation prediction layer data, and N first layer loss function values are determined based on the N first blood vessel segmentation prediction layer data and the first blood vessel segmentation marking data.

Specifically, since the first model to be trained includes N first network layers to be trained, each first network layer to be trained includes one segmentation module to be trained, and each first network layer to be trained needs to be supervised, first vessel segmentation prediction layer data of each first network layer to be trained needs to be obtained, each first vessel segmentation prediction layer data is a prediction value, the first vessel segmentation flag data is a target value, and N first layer loss function values are determined based on the N first vessel segmentation prediction layer data and the first vessel segmentation flag data.

S604: and weighting the N first-layer loss function values based on the preset weight values corresponding to the N first-layer loss function values respectively to obtain first weighted loss function values.

Specifically, considering that the hierarchical resolutions of the first network layers to be trained are different, the closest clinical application of the first network layer to be trained with the highest hierarchical resolution is the main hierarchical, and therefore the preset weight value of the first loss function value of the first layer corresponding to the first network layer to be trained with the highest hierarchical resolution is the maximum.

For example, N is equal to 3, the first model to be trained includes 3 first network layers to be trained, the hierarchical resolutions of the 3 first network layers to be trained are arranged from large to small, the preset weight value of the first loss function value corresponding to the first network layer to be trained with the largest hierarchical resolution is 10:1.

s605: and adjusting parameters of the first model to be trained based on the first weighted loss function value until the first weighted loss function value meets a preset condition, so as to obtain a first multilayer network vessel segmentation model.

In the embodiment of the application, a first multi-scale multi-window block cutting sampling operation is utilized to perform block cutting sampling on a first head and neck blood vessel sample image, different sampling spatial resolutions are fully utilized to cause blocks with the same image pixel volume to cover features with different physical space sizes, and features of each dimension information of the image are reserved by different windows, so that a first model to be trained learns data features at multiple angles, the robustness of the model is improved, sufficient global information is provided by utilizing different receptive fields, the model has better learning capacity, the extraction capacity of blood vessel structure information is improved, and the probability of breakage or fracture occurs in a blood vessel segmentation result. In addition, considering that the importance of different first network layers to be trained is different, N first layer loss function values are weighted, and the first weighted loss function values are utilized to adjust the parameters of the first model to be trained, so that the model is changed towards the direction with more excellent segmentation performance.

Fig. 7 is a flowchart illustrating a training method of a first multi-layer network vessel segmentation model according to an embodiment of the present application. As shown in fig. 7, the step of obtaining N first vessel segmentation prediction layer data by using N first sample block data group samples corresponding to the first sample center point sample, and determining N first layer loss function values based on the N first vessel segmentation prediction layer data and the first vessel segmentation flag data includes the following steps.

S701: and inputting the N first sampling block data group samples into the first network layer to be trained matched with each other for segmentation to obtain N first blood vessel segmentation prediction layer data based on the sampling spatial resolution corresponding to the N first sampling block data group samples and the hierarchical resolution corresponding to the N first network layers to be trained.

Specifically, the specific details of obtaining the N first blood vessel segmentation prediction layer data are similar to those of the step shown in fig. 4, and are not repeated again.

S702: for each first blood vessel segmentation prediction layer data in the N first blood vessel segmentation prediction layer data, determining skeleton region data in the first blood vessel segmentation prediction layer data, blood vessel foreground region data in the first blood vessel segmentation prediction layer data, and foreground neighborhood data in the first blood vessel segmentation prediction layer data.

S703: and determining a first layer loss function value corresponding to the first blood vessel segmentation prediction layer data based on the first blood vessel segmentation prediction layer data, the first blood vessel segmentation marking data and a preset layer loss function, wherein in the first preset layer loss function, the weight of the skeleton region data is greater than that of the foreground neighborhood region data, and the weight of the foreground neighborhood region data is greater than that of the blood vessel foreground region data.

Illustratively, the skeleton region data weight: foreground neighborhood data weight: the blood vessel foreground region data weight is 30:10:1.

in the embodiment of the application, different weights are set for different areas of the first blood vessel segmentation prediction layer data, so that a first layer loss function value is obtained, and the first model to be trained is guided to move forward in a direction with a better segmentation effect.

Training method of exemplary second multilayer network vessel segmentation model

Fig. 8 is a flowchart illustrating a training method of a second multi-layer network vessel segmentation model according to an embodiment of the present application. The training method of the second multilayer network vessel segmentation model provided by the embodiment of the application is used for training a second model to be trained, the second model to be trained comprises N second network layers to be trained, each second network layer to be trained comprises a segmentation module to be trained, and N hierarchical resolutions corresponding to the N second network layers to be trained are respectively matched with N sampling space resolutions corresponding to second multi-scale multi-window segmentation sampling operation.

As shown in fig. 8, the training method of the second multi-layer network vessel segmentation model includes the following steps.

S801: and determining at least one second head and neck blood vessel sample image, blood vessel structure mark data corresponding to the at least one second head and neck blood vessel sample image respectively, and second blood vessel segmentation mark data corresponding to the at least one second head and neck blood vessel sample image respectively.

Specifically, when the second model to be trained is trained, vascular structure labeling data needs to be introduced, and the vascular structure labeling data corresponding to the at least one second head and neck blood vessel sample image is obtained, including but not limited to manual labeling.

S802: and determining N second sampling block data group samples corresponding to a plurality of second sampling central point samples in the second head and neck blood vessel sample image by utilizing second multi-scale multi-window block cutting sampling operation based on the second blood vessel segmentation marking data.

The specific steps of obtaining N samples of the second sample block data group are similar to the specific steps of obtaining N samples of the first sample block data group, obtaining N samples of the second sample block data group, and obtaining N samples of the first sample block data group, which have been described in detail in step 202 and in the step of fig. 3, and are not repeated here.

S803: and based on the coordinates corresponding to the second sampling central point samples, carrying out block cutting and sampling operation on the blood vessel structure mark data to obtain N blood vessel structure mark block data corresponding to the second sampling central point samples.

Specifically, the obtaining of the N blood vessel structure marker block data corresponding to each of the plurality of second sampling center point samples is similar to the obtaining of the N blood vessel structure block data corresponding to the second sampling center point, which has been described in detail in step 503 and the step of fig. 3, and is not described here again.

S804: and obtaining N second vessel segmentation prediction layer data by utilizing N second sampling block data group samples and N vessel structure mark block data corresponding to the second sampling central point sample based on a second model to be trained for each second sampling central point sample in the plurality of second sampling central point samples, and determining N second layer loss function values based on the N second vessel segmentation prediction layer data and the second vessel segmentation mark data.

Specifically, first, the N second sample block data group samples and the N blood vessel structure marker block data are respectively input into the segmentation modules in the second network layers to be trained, which are respectively matched, that is, each second network layer to be trained receives 1 second sample block data group sample and 1 blood vessel structure marker block data corresponding to the second sample center point sample.

And for every two adjacent second network layers to be trained in the N second network layers to be trained, fusing the feature maps corresponding to the second network layers to be trained corresponding to the large hierarchical resolution into corresponding positions in the feature maps corresponding to the second network layers to be trained corresponding to the small hierarchical resolution to perform blood vessel segmentation prediction.

Step S804 is the same as step S504, except that each network layer to be trained receives 1 second sampling block data group sample and 1 blood vessel structure marker block data corresponding to the second sampling center point sample, and each first network layer only receives 1 first sampling block data group corresponding to the first sampling center point, and other details are similar and will not be repeated herein.

Then, based on the N second vessel segmentation prediction layer data and the second vessel segmentation flag data, specific details of the N second layer loss function values are determined, similar to steps S702 and S703, and are not described herein again.

S805: and weighting the N second-layer loss function values based on the preset weight values corresponding to the N second-layer loss function values respectively to obtain second weighted loss function values.

S806: and adjusting parameters of the second model to be trained based on the second weighted loss function value until the second weighted loss function value meets the preset condition, so as to obtain a second multilayer network vessel segmentation model.

The specific details of steps S805 and S806 are similar to steps S604 and S605, and are not described herein.

In the embodiment of the application, the vascular structure data is introduced to train the second model to be trained, so that sufficient global information of blood vessels is provided for the model, the accuracy and the integrity of segmentation are greatly improved, the probability of breakage or fracture in the blood vessel segmentation result is reduced, and the obtained second multilayer network blood vessel segmentation model has stronger robustness and generalization.

Exemplary head and neck artery vessel segmentation device

Fig. 9 is a schematic structural diagram of a head and neck artery vascular segmentation apparatus provided in an embodiment of the present application. As shown in fig. 9, the apparatus 100 for head and neck artery vessel segmentation includes a rough vessel region segmentation data determination module 101, a first multi-scale multi-window vessel segmentation module 102, a vessel structure data determination module 103, and a second multi-scale multi-window vessel segmentation module 104.

The rough blood vessel region segmentation data determination module 101 is configured to determine, based on the head and neck medical image to be segmented, rough blood vessel region segmentation data corresponding to the head and neck medical image to be segmented. The first multi-scale multi-window blood vessel segmentation module 102 is configured to perform blood vessel segmentation on the head and neck medical image to be segmented by using a first multi-scale multi-window block sampling operation based on the rough blood vessel region segmentation data to obtain a blood vessel segmentation probability map and rough blood vessel segmentation data corresponding to the head and neck medical image to be segmented. The blood vessel structure data determining module 103 is configured to determine blood vessel structure data corresponding to the head and neck medical image to be segmented based on the blood vessel segmentation probability map and the head and neck medical image to be segmented. The second multi-scale multi-window blood vessel segmentation module 104 is configured to perform blood vessel segmentation on the head and neck medical image to be segmented by using a second multi-scale multi-window block sampling operation based on the blood vessel structure data to obtain blood vessel segmentation data corresponding to the head and neck medical image to be segmented.

Illustratively, the first multi-scale multi-window tile sampling operation and the second multi-scale multi-window tile sampling operation are performed with a plurality of sampling spatial resolutions and a plurality of windows, each targeting a preset image pixel volume as a sample.

In the embodiment of the application, through multi-scale and multi-window block cutting sampling operation, not only can image features be fully extracted, but also the global information of blood vessels can be fully referred to, and the extraction energy of the blood vessel structure is improved; and the features of the blood vessel image layer are extracted firstly, then the features of the blood vessel structure layer are extracted, and then the fusion is carried out for segmentation, so that the image features and the blood vessel structure features are fused and taken into consideration in the segmentation process, the segmentation accuracy and the segmentation integrity are greatly improved, and the probability of breakage or fracture in the blood vessel segmentation result is reduced.

Fig. 10 is a schematic structural diagram of a first multi-scale multi-window vessel segmentation module according to an embodiment of the present application. As shown in fig. 10, the first multi-scale multi-window vessel segmentation module 102 further includes: a first sampling center point determining unit 1021, a first sampling block data group determining unit 1022, a first vessel segmentation probability block data determining unit 1023, and a coarse vessel segmentation data determining unit 1024.

The first sampling center point determination unit 1021 is configured to determine a plurality of first sampling center points in the head and neck medical image to be segmented based on the coarse vessel region segmentation data. The first sampling block data group determining unit 1022 is configured to, for each first sampling center point of the plurality of first sampling center points, perform block cutting and sampling on the head and neck medical image to be segmented by using the first sampling center point as a center and using N sampling spatial resolutions and M windows, to obtain N first sampling block data groups corresponding to the first sampling center point, where each first sampling block data group of the N first sampling block data groups includes M first sampling block data, N is greater than or equal to 3, and M is greater than or equal to 3. The first vessel segmentation probability block data determination unit 1023 is configured to input N first sampling block data groups corresponding to the first sampling central point into a first multi-layer network vessel segmentation model to obtain vessel segmentation probability block data corresponding to the first sampling central point, where the first multi-layer network vessel segmentation model includes N first network layers, and each of the N first network layers includes one segmentation module. The rough blood vessel segmentation data determination unit 1024 is configured to obtain a blood vessel segmentation probability map and rough blood vessel segmentation data based on blood vessel segmentation probability block data corresponding to each of the plurality of first sampling center points.

Fig. 11 is a schematic structural diagram of a first sample block data group unit according to an embodiment of the present application. As shown in fig. 11, the first sample block data group unit 1022 further includes: a first determining subunit 10221, a second determining subunit 10222, a third determining subunit 10223, and a fourth determining subunit 10224.

The first determining subunit 10221 is configured to determine window widths and window levels corresponding to the N sampling spatial resolutions and the M windows, respectively. The second determining subunit 10222 is configured to, with the first sampling central point as a center, respectively sample the head and neck medical image to be segmented by using N sampling spatial resolutions, so as to obtain N first initial sampling block data corresponding to the first sampling central point. The third determining subunit 10223 is configured to perform a windowed sampling operation on the first initial sample block data based on the window width and the window level corresponding to each of the M windows for each of the N first initial sample block data, so as to obtain M first sample block data corresponding to the first initial sample block data. The fourth determining subunit 10224 is configured to determine N first sample block data groups corresponding to the first sampling center point, based on M first sample block data respectively corresponding to the N first initial sample block data.

In an alternative embodiment, the N sampling spatial resolutions are 3 sampling spatial resolutions, and the 3 sampling spatial resolutions are a reference spatial resolution, 2 times the reference spatial resolution, and 4 times the reference spatial resolution, respectively; the M windows are 3 windows, the 3 windows are an original graph window, a first fixed window and a second fixed window respectively, wherein the window width and the window position of the original graph window at the head and neck medical image to be segmented are the same, the window width of the first fixed window is 300Hu, the window position of the first fixed window is 50Hu, the window width of the second fixed window is 300Hu, and the window position of the second fixed window is 500Hu.

Fig. 12 is a schematic structural diagram of a first vessel segmentation probability block data determination unit according to an embodiment of the present application. In this embodiment, the first multi-layer network vessel segmentation model includes N first network layers, and the N first network layers are arranged from large to small according to respective corresponding hierarchical resolutions. As shown in fig. 12, the first-vessel segmentation probability block data determination unit 1023 further includes: the first input subunit 10231, the first feature fusion subunit 10232 and the first vessel segmentation probability block data determination subunit 10233.

The first input subunit 10231 is configured to input the N first sample block data groups into respective partitioning modules in respective matched first network layers based on respective sampling spatial resolutions corresponding to the N first sample block data groups, where a first network layer matched with a certain first sample block data group is, of the N first network layers, a first network layer corresponding to a hierarchical resolution matched with a sampling spatial resolution corresponding to a certain first sample block data group; the first feature fusion subunit 10232 is configured to, for each two adjacent first network layers of the N first network layers, fuse the feature map corresponding to the first network layer corresponding to the large hierarchical resolution into the corresponding position in the feature map corresponding to the first network layer corresponding to the small hierarchical resolution for vessel segmentation; the first vessel segmentation probability block data determination subunit 10233 is configured to perform function conversion operation on the feature map output by the first network layer located in the last layer of the N first network layers to obtain the vessel segmentation probability block data corresponding to the first sampling central point.

In one embodiment, as shown in fig. 12, the first blood vessel segmentation probability block data determination unit 1023 further includes a first adaptive windowing subunit 10230, and the first adaptive windowing subunit 10230 is configured to, before inputting the N first sample block data groups into the segmentation modules in the matched first network layers, input the N first sample block data groups into the adaptive adjustment window modules in the matched first network layers, respectively, to perform an adaptive windowing operation on sample block data corresponding to an original image window among M first sample block data in each of the N first sample block data groups.

Fig. 13 is a schematic structural diagram of a second multi-scale and multi-window vessel segmentation module according to an embodiment of the present application. As shown in fig. 13, the second multi-scale multi-window vessel segmentation module 104 further includes: a second sampling center point determining unit 1041, a second sampling block data group determining unit 1042, a blood vessel structure block data determining unit 1043, a second blood vessel segmentation probability block data determining unit 1044, and a rough blood vessel segmentation data determining unit 1045. The second sampling center point determining unit 1041 is configured to determine a plurality of second sampling center points in the head and neck medical image to be segmented based on the vascular structure data. The second sampling block data group determining unit 1042 is configured to, for each of the plurality of second sampling center points, perform block slicing sampling on the head and neck medical image to be segmented by using the second sampling center point as a center and using N sampling spatial resolutions and M windows, to obtain N second sampling block data groups corresponding to the second sampling center point, where each of the N second sampling block data groups includes M second sampling block data. The blood vessel structure block data determining unit 1043 is configured to perform a block cutting sampling operation on the blood vessel structure data by using N sampling spatial resolutions based on the coordinates corresponding to the second sampling central point, so as to obtain N blood vessel structure block data corresponding to the second sampling central point. The second vessel segmentation probability block data determination unit 1043 is configured to input N second sampling block data groups and N blood vessel structure block data corresponding to the second sampling central point into a second multi-layer network vessel segmentation model to obtain vessel segmentation probability block data corresponding to the second sampling central point, where the second multi-layer network vessel segmentation model includes N second network layers, and each of the N second network layers includes a segmentation module. The vessel segmentation data determination unit 1045 is configured to determine, based on the vessel segmentation probability block data corresponding to each of the plurality of second sampling center points, vessel segmentation data corresponding to the head and neck medical image to be segmented.

Fig. 14 is a schematic structural diagram of a second vessel segmentation probability block data determination unit according to an embodiment of the present application. As shown in fig. 14, the second vessel segmentation probability block data determination unit 1043 further includes: a second input subunit 10431, a second feature fusion subunit 10432, and a second vessel segmentation probability block data determination subunit 10433.

The second input subunit 10431 is configured to input, based on sampling spatial resolutions of the N second sampling block data groups and sampling spatial resolutions of the N vascular structure block data groups, the N second sampling block data groups and the N vascular structure block data into the segmentation modules in the second network layers that are respectively matched with each other, where a second network layer that is matched with a certain second sampling block data group is, of the N second network layers, a second network layer that is corresponding to a hierarchical resolution that is matched with a sampling spatial resolution of a certain second sampling block data group; the second network layer matched with the certain blood vessel structure block data is a second network layer corresponding to a hierarchical resolution matched with a sampling spatial resolution corresponding to the certain blood vessel structure block data among the N second network layers. The second feature fusion subunit 10432 is configured to, for each adjacent two second network layers of the N second network layers, fuse the feature map corresponding to the second network layer corresponding to the large hierarchical resolution into the corresponding position in the feature map corresponding to the second network layer corresponding to the small hierarchical resolution, so as to perform the blood vessel segmentation. The second vessel segmentation probability block data determination subunit 10433 is configured to perform a function conversion operation on the feature map output by the second network layer that is located in the last layer of the N second network layers, so as to obtain vessel segmentation probability block data corresponding to the second sampling central point.

In one embodiment, as shown in fig. 14, the second vessel-segmentation probability block data determination unit 1043 further includes a second adaptive windowing subunit 10430, and the second adaptive windowing subunit 10430 is configured to, before inputting the N second sample block data groups and the N vessel-structure block data corresponding to the second sampling center point into the second multi-layer network vessel-segmentation model, respectively input the N second sample block data groups into adaptive adjustment window modules in the matched second network layer to perform an adaptive windowing operation on sample block data corresponding to an original image window in M second sample block data in each of the N second sample block data groups.

Fig. 15 is a schematic structural diagram of a blood vessel structure data determination module according to an embodiment of the present application. As shown in fig. 15, the blood vessel structure data determination module 103 further includes: a first deriving unit 1031, an initial vessel center line deriving unit 1032, and a vessel structure data deriving unit 1033.

The first obtaining unit 1031 is configured to perform binarization operation on the blood vessel segmentation probability map to obtain binarization blood vessel segmentation data corresponding to the head and neck medical image to be segmented. The initial blood vessel center line obtaining unit 1032 is configured to perform center line extraction on the binarized blood vessel segmentation data to obtain an initial blood vessel center line corresponding to the head and neck medical image to be segmented. The blood vessel structure data obtaining unit 1033 is configured to perform a blood vessel structure growth operation on the head and neck medical image to be segmented based on the initial blood vessel center line by using a simulated hemodynamic mode, so as to obtain blood vessel structure data.

In one embodiment, the apparatus 100 for segmenting a head and neck artery blood vessel further comprises: a first training module 105. The first training module 105 is configured to train the first model to be trained, resulting in a first multi-layer network vessel segmentation model.

Fig. 16 is a schematic structural diagram of a first training module according to an embodiment of the present application. As shown in fig. 16, the first training module 105 further includes: a first determining unit 1051, a second determining unit 1052, a first layer loss function value determining unit 1053, a first weighting determining unit 1054, and a first model obtaining unit 1055.

The first determining unit 1051 is configured to determine at least one first cephaloneck blood vessel sample image and first blood vessel segmentation marker data corresponding to the at least one first cephaloneck blood vessel sample image. The second determining unit 1052 is configured to determine, based on the first vessel segmentation marker data, N first sampling block data group samples corresponding to each of the plurality of first sampling center point samples in the first cephalic-cervical blood vessel sample image by using the first multi-scale multi-window dicing sampling operation. The first-layer loss function value determining unit 1053 is configured to, for each first sampling center point sample of the plurality of first sampling center point samples, obtain N first vessel segmentation prediction layer data based on the first model to be trained by using N first sampling block data group samples corresponding to the first sampling center point sample, and determine N first-layer loss function values based on the N first vessel segmentation prediction layer data and the first vessel segmentation flag data. The first weighting determining unit 1054 is configured to weight the N first-layer loss function values based on preset weight values corresponding to the N first-layer loss function values, so as to obtain a first weighted loss function value. The first model obtaining unit 1055 is configured to adjust a parameter of the first model to be trained based on the first weighted loss function value until the first weighted loss function value satisfies a preset condition, so as to obtain a first multi-layer network vessel segmentation model.

In one embodiment, the first layer loss function value determining unit 1053 is further configured to input the N first sample block data group samples into the respectively matched first network layer to be trained for segmentation based on the sampling spatial resolution corresponding to the N first sample block data group samples and the hierarchical resolution corresponding to the N first network layer to be trained, so as to obtain N first vessel segmentation prediction layer data; for each first blood vessel segmentation prediction layer data in the N first blood vessel segmentation prediction layer data, determining skeleton region data in the first blood vessel segmentation prediction layer data, blood vessel foreground region data in the first blood vessel segmentation prediction layer data, and foreground neighborhood data in the first blood vessel segmentation prediction layer data; and determining a first layer loss function value corresponding to the first vessel segmentation prediction layer data according to the first vessel segmentation prediction layer data, the first vessel segmentation marking data and a preset layer loss function, wherein in the first preset layer loss function, the weight of the skeleton region data is greater than that of the foreground neighborhood region data, and the weight of the foreground neighborhood region data is greater than that of the vessel foreground region data.

In one embodiment, the head and neck artery segmentation apparatus 100 further comprises: a second training module 106. The second training module 106 is configured to train the first model to be trained, so as to obtain a first multi-layer network vessel segmentation model.

Fig. 17 is a schematic structural diagram of a second training module according to an embodiment of the present application. As shown in fig. 17, the second training module 106 further includes: a third determining unit 1061, a fourth determining unit 1062, a blood vessel structure marker block data obtaining unit 1063, a second layer loss function value determining unit 1064, a second weight determining unit 1065, and a second model obtaining unit 1066.

The third determining unit 1061 is configured to determine at least one second head and neck blood vessel sample image, blood vessel structure label data corresponding to the at least one second head and neck blood vessel sample image, and second blood vessel segmentation label data corresponding to the at least one second head and neck blood vessel sample image. The fourth determining unit 1062 is configured to determine, based on the second blood vessel segmentation flag data, N second sampling block data group samples corresponding to the plurality of second sampling center point samples in the second head and neck blood vessel sample image by using a second multi-scale multi-window dicing sampling operation. The blood vessel structure marker block data obtaining unit 1063 is configured to perform a cutting sampling operation on the blood vessel structure marker data based on the coordinates corresponding to the plurality of second sampling center point samples, so as to obtain N blood vessel structure marker block data corresponding to the plurality of second sampling center point samples. The second-layer loss function value determining unit 1064 is configured to, for each second sample centerpoint sample of the plurality of second sample centerpoint samples, obtain, based on the second model to be trained, N second vessel segmentation prediction layer data by using N second sample block data group samples and N vessel structure marker block data corresponding to the second sample centerpoint sample, and determine N second-layer loss function values based on the N second vessel segmentation prediction layer data and the second vessel segmentation marker data. The second weighting determining unit 1065 is configured to weight the N second-layer loss function values based on preset weight values corresponding to the N second-layer loss function values, so as to obtain a second weighted loss function value. The second model obtaining unit 1066 is configured to adjust parameters of the second model to be trained based on the second weighted loss function value until the second weighted loss function value satisfies a preset condition, so as to obtain a second multi-layer network vessel segmentation model.

In one embodiment, the second-layer loss function value determining unit 1064 is further configured to, based on the sampling spatial resolution corresponding to each of the N second sampling block data set samples, the sampling spatial resolution corresponding to each of the N blood-vessel structure labeling block data, and the hierarchical resolution corresponding to each of the N second network layers to be trained, input the N second sampling block data set samples and the N blood-vessel structure labeling block data into the respectively matched second network layers to be trained for segmentation, so as to obtain N second blood-vessel segmentation prediction layer data; for each second blood vessel segmentation prediction layer data in the N second blood vessel segmentation prediction layer data, determining skeleton region data in the second blood vessel segmentation prediction layer data, blood vessel foreground region data in the second blood vessel segmentation prediction layer data, and foreground neighborhood data in the second blood vessel segmentation prediction layer data; and determining a second layer loss function value corresponding to the second vessel segmentation prediction layer data according to the second vessel segmentation prediction layer data, the second vessel segmentation marking data and a preset layer loss function, wherein in the second preset layer loss function, the weight of the skeleton region data is greater than that of the foreground neighborhood region data, and the weight of the foreground neighborhood region data is greater than that of the vessel foreground region data.

The detailed functions and operations of the other respective modules in the above-mentioned head and neck artery blood vessel segmentation apparatus have been described in detail in the head and neck artery blood vessel segmentation method described in fig. 1 to 5, the training method of the first multi-layer network blood vessel segmentation model described in fig. 6 and 7, and the training method of the second multi-layer network blood vessel segmentation model described in fig. 8, and therefore, a repeated description thereof will be omitted here.

Exemplary electronic device

Fig. 18 is a schematic structural diagram of an electronic device according to an embodiment of the application. As shown in fig. 18, the electronic device 300 includes one or more processors 310 and memory 320.

The processor 310 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 300 to perform desired functions.

Memory 320 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, read Only Memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer readable storage medium, and the processor 310 may execute the program instructions to implement the method for head and neck artery vessel segmentation of the various embodiments of the present application described above, or the method for training the first multi-layer network vessel segmentation model of the various embodiments, or the method for training the second multi-layer network vessel segmentation model of the various embodiments, and/or other desired functions.

In one example, the electronic device 300 may further include: an input device 330 and an output device 340, which are interconnected by a bus system and/or other form of connection mechanism (not shown).

Of course, for the sake of simplicity, only some of the components related to the present application in the electronic device 300 are shown in fig. 18, and components such as a bus, an input/output interface, and the like are omitted. In addition, electronic device 300 may include any other suitable components depending on the particular application.

Exemplary computer program product and computer-readable storage Medium

In addition to the above-described methods and devices, embodiments of the present application may also be a computer program product comprising computer program instructions which, when executed by a processor, cause the processor to perform the steps in a method for head and neck artery vessel segmentation provided according to various embodiments of the present application as described in the above-mentioned "exemplary method for head and neck artery vessel segmentation" section of the present specification, or to perform the steps in a method for training a first multi-layer network vessel segmentation model provided according to various embodiments of the present application as described in the above-mentioned "method for training a first multi-layer network vessel segmentation model" section of the present specification, or to perform the steps in a method for training a second multi-layer network vessel segmentation model provided according to various embodiments of the present application as described in the above-mentioned "method for training a second multi-layer network vessel segmentation model" section of the present specification.

The computer program product may write program code for carrying out operations for embodiments of the present application in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.

Furthermore, embodiments of the present application may also be a computer readable storage medium having stored thereon computer program instructions, which, when executed by a processor, cause the processor to perform the steps in the method for head and neck artery vessel segmentation provided according to the various embodiments of the present application described in the section "exemplary method for head and neck artery vessel segmentation" mentioned above in this specification, or perform the steps in the method for training the first multi-layer network vessel segmentation model provided according to the various embodiments of the present application described in the section "method for training the exemplary first multi-layer network vessel segmentation model" mentioned above in this specification, or perform the steps in the method for training the second multi-layer network vessel segmentation model provided according to the various embodiments of the present application described in the section "method for training the exemplary second multi-layer network vessel segmentation model" mentioned above in this specification.

The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

It should be noted that the above listed embodiments are only specific examples of the present application, and obviously, the present application is not limited to the above embodiments, and many similar variations exist. All modifications which can be derived or suggested by the person skilled in the art from the disclosure of the present application shall fall within the scope of protection of the present application.

It should be understood that the terms first, second, etc. used in the embodiments of the present application are only used for clearly describing the technical solutions of the embodiments of the present application, and are not used to limit the protection scope of the present application.

The above description is only for the preferred embodiment of the present application, and is not intended to limit the scope of the present application. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for segmenting head and neck artery blood vessels is characterized by comprising the following steps:

determining rough blood vessel region segmentation data corresponding to the head and neck medical image to be segmented based on the head and neck medical image to be segmented;

based on the rough blood vessel region segmentation data, performing blood vessel segmentation on the head and neck medical image to be segmented by utilizing a first multi-scale multi-window block cutting sampling operation to obtain a blood vessel segmentation probability map and rough blood vessel segmentation data corresponding to the head and neck medical image to be segmented; the rough blood vessel region segmentation data is utilized to determine a plurality of first sampling central points in the head and neck medical image to be segmented, a sampling center is provided for subsequent sampling, the blood vessel segmentation is carried out after the first multi-scale multi-window block cutting sampling operation is carried out on the head and neck medical image to be segmented by utilizing the plurality of first sampling central points, blood vessel segmentation probability block data corresponding to the plurality of first sampling central points can be obtained, the blood vessel segmentation probability block data corresponding to the plurality of first sampling central points are spliced to obtain the blood vessel segmentation probability map, and the rough blood vessel segmentation data can be obtained according to the blood vessel segmentation probability map;

determining blood vessel structure data corresponding to the head and neck medical image to be segmented based on the blood vessel segmentation probability map and the head and neck medical image to be segmented; based on the centerline extraction operation of the blood vessel segmentation probability map, an initial blood vessel centerline corresponding to the head and neck medical image to be segmented can be obtained, and based on the initial blood vessel centerline, a blood vessel structure growth operation is performed on the head and neck medical image to be segmented by utilizing a haemodynamics simulation mode, so that blood vessel structure data can be obtained;

based on the blood vessel structure data, performing blood vessel segmentation on the head and neck medical image to be segmented by utilizing a second multi-scale multi-window block cutting sampling operation to obtain blood vessel segmentation data corresponding to the head and neck medical image to be segmented; the vessel structure data is used as a reference, a plurality of second sampling central points can be determined in the head and neck medical image to be segmented, a sampling center is provided for subsequent sampling, the vessel segmentation is carried out after second multi-scale multi-window block cutting sampling operation is carried out on the head and neck medical image to be segmented by utilizing the plurality of second sampling central points, vessel segmentation probability block data corresponding to the plurality of second sampling central points can be obtained, and the vessel segmentation probability block data corresponding to the plurality of second sampling central points are spliced and binarized to obtain the vessel segmentation data;

the first multi-scale multi-window dicing sampling operation and the second multi-scale multi-window dicing sampling operation are performed by sampling and dicing in the scope of a plurality of sampling spatial resolutions and a plurality of windows by taking a preset image pixel volume as a sampling target.

2. The method for head and neck artery vessel segmentation according to claim 1, wherein the performing vessel segmentation on the head and neck medical image to be segmented based on the rough vessel region segmentation data by using a first multi-scale multi-window block sampling operation to obtain a vessel segmentation probability map and rough vessel segmentation data corresponding to the head and neck medical image to be segmented comprises:

determining the plurality of first sampling central points in the head and neck medical image to be segmented based on the rough blood vessel region segmentation data;

for each first sampling central point in the plurality of first sampling central points, taking the first sampling central point as a center, and performing block cutting sampling on the head and neck medical image to be segmented by using N sampling spatial resolutions and M windows to obtain N first sampling block data groups corresponding to the first sampling central point, wherein each first sampling block data group in the N first sampling block data groups comprises M first sampling block data, N is greater than or equal to 3, and M is greater than or equal to 3;

inputting N first sampling block data groups corresponding to the first sampling central point into a first multi-layer network vessel segmentation model to obtain vessel segmentation probability block data corresponding to the first sampling central point, wherein the first multi-layer network vessel segmentation model comprises N first network layers, and each first network layer in the N first network layers comprises a segmentation module;

and obtaining the blood vessel segmentation probability map and the rough blood vessel segmentation data based on the blood vessel segmentation probability block data corresponding to the plurality of first sampling central points respectively.

3. The method for head and neck artery vessel segmentation according to claim 2, wherein the step of performing block cutting sampling on the head and neck medical image to be segmented by using N sampling spatial resolutions and M windows with the first sampling central point as a center to obtain N first sampling block data sets corresponding to the first sampling central point comprises:

determining the respective window widths and window levels corresponding to the N sampling spatial resolutions and the M windows;

taking the first sampling central point as a center, and respectively sampling the head and neck medical image to be segmented by using the N sampling spatial resolutions to obtain N pieces of first initial sampling block data corresponding to the first sampling central point;

for each first initial sampling block data in the N first initial sampling block data, performing windowing sampling operation on the first initial sampling block data based on the window width and the window level respectively corresponding to the M windows to obtain the M first sampling block data corresponding to the first initial sampling block data;

and determining N first sampling block data groups corresponding to the first sampling center points on the basis of M first sampling block data respectively corresponding to the N first initial sampling block data.

4. The method for head and neck artery vessel segmentation according to claim 3, wherein the determining the N sampling spatial resolutions comprises:

determining a reference spatial resolution based on an initial spatial resolution corresponding to a sampling source domain, an image pixel volume corresponding to the sampling source domain, and an image pixel volume corresponding to a sampling target domain;

and calculating the reference spatial resolution in proportion to obtain the N sampling spatial resolutions.

5. The method for head and neck artery vessel segmentation according to claim 4,

the N sampling spatial resolutions are 3 sampling spatial resolutions, and the 3 sampling spatial resolutions are the reference spatial resolution, 2 times the reference spatial resolution, and 4 times the reference spatial resolution, respectively;

the M windows are 3 windows, the 3 windows are respectively an original map window, a first fixed window and a second fixed window, wherein the window width and the window position of the original map window are the same as those of the head and neck medical image to be segmented, the window width of the first fixed window is 300Hu, the window position of the first fixed window is 50Hu, the window width of the second fixed window is 300Hu, and the window position of the second fixed window is 500Hu.

6. The head and neck artery vessel segmentation method according to any one of claims 2 to 5, wherein the N first network layers in the first multilayer network vessel segmentation model are arranged from large to small according to the respective corresponding hierarchical resolutions;

the method for obtaining the data of the vessel segmentation probability block corresponding to the first sampling center point by inputting the data group of the N first sampling blocks corresponding to the first sampling center point into a first multilayer network vessel segmentation model comprises the following steps:

respectively inputting the N first sampling block data groups into segmentation modules in respectively matched first network layers based on sampling spatial resolutions respectively corresponding to the N first sampling block data groups, wherein the first network layer matched with a certain first sampling block data group is a first network layer corresponding to a hierarchical resolution matched with the sampling spatial resolution corresponding to the certain first sampling block data group in the N first network layers;

for every two adjacent first network layers in the N first network layers, fusing the feature map corresponding to the first network layer corresponding to the large hierarchical resolution into the corresponding position in the feature map corresponding to the first network layer corresponding to the small hierarchical resolution so as to perform vessel segmentation;

and performing function conversion operation on the feature graph output by the first network layer which is arranged at the last layer of the N first network layers to obtain the blood vessel segmentation probability block data corresponding to the first sampling central point.

7. The method for head and neck artery vessel segmentation according to claim 6, wherein before the inputting the N first sample block data groups into the segmentation modules in the matching first network layer, further comprising:

and respectively inputting the N first sampling block data groups into the adaptive adjustment window modules in the first network layers which are respectively matched with the N first sampling block data groups so as to carry out adaptive windowing on the sampling block data corresponding to the original image window in the M first sampling block data groups in each of the N first sampling block data groups.

8. The method for head and neck artery vessel segmentation according to any one of claims 1 to 5, wherein the performing vessel segmentation on the head and neck medical image to be segmented based on the vessel structure data by using a second multi-scale multi-window block sampling operation to obtain vessel segmentation data corresponding to the head and neck medical image to be segmented comprises:

determining the plurality of second sampling central points in the head and neck medical image to be segmented based on the vascular structure data;

aiming at each second sampling central point in the plurality of second sampling central points, taking the second sampling central point as a center, and utilizing N sampling spatial resolutions and M windows to perform block cutting sampling on the head and neck medical image to be segmented to obtain N second sampling block data groups corresponding to the second sampling central points, wherein each second sampling block data group in the N second sampling block data groups comprises M second sampling block data;

based on the coordinates corresponding to the second sampling central point, performing block cutting sampling operation on the vascular structure data by using the N sampling spatial resolutions to obtain N vascular structure block data corresponding to the second sampling central point;

inputting N second sampling block data groups and N blood vessel structure block data corresponding to the second sampling central point into a second multilayer network blood vessel segmentation model to obtain blood vessel segmentation probability block data corresponding to the second sampling central point, wherein the second multilayer network blood vessel segmentation model comprises N second network layers, and each of the N second network layers comprises a segmentation module;

and determining blood vessel segmentation data corresponding to the head and neck medical image to be segmented based on the blood vessel segmentation probability block data corresponding to the second sampling central points.

9. The method for head and neck artery vessel segmentation according to claim 8, wherein the step of inputting N second sampling block data groups and N vascular structure block data corresponding to the second sampling central point into a second multi-layer network vessel segmentation model to obtain vessel segmentation probability block data corresponding to the second sampling central point comprises:

inputting the N second sample block data groups and the N blood vessel structure block data into segmentation modules in respectively matched second network layers based on sampling spatial resolutions corresponding to the N second sample block data groups and sampling spatial resolutions corresponding to the N blood vessel structure block data groups, wherein a second network layer matched with a certain second sample block data group is a second network layer corresponding to a hierarchical resolution matched with a sampling spatial resolution corresponding to the certain second sample block data group in the N second network layers; the second network layer matched with certain blood vessel structure block data is a second network layer corresponding to a hierarchical resolution matched with the sampling space resolution corresponding to the certain blood vessel structure block data in the N second network layers;

for every two adjacent second network layers in the N second network layers, fusing the feature map corresponding to the second network layer corresponding to the large hierarchical resolution into the corresponding position in the feature map corresponding to the second network layer corresponding to the small hierarchical resolution so as to perform vessel segmentation;

and performing function conversion operation on the feature map output by the second network layer which is arranged at the last layer of the N second network layers to obtain the blood vessel segmentation probability block data corresponding to the second sampling central point.

10. The method for head and neck artery vessel segmentation according to any one of claims 1 to 5, wherein the determining the vessel structure data corresponding to the head and neck medical image to be segmented based on the vessel segmentation probability map and the head and neck medical image to be segmented comprises:

performing binarization operation on the blood vessel segmentation probability map to obtain binarization blood vessel segmentation data corresponding to the head and neck medical image to be segmented;

performing the center line extraction operation on the binary blood vessel segmentation data to obtain an initial blood vessel center line corresponding to the head and neck medical image to be segmented;

and based on the initial vessel central line, performing the vessel structure growth operation on the head and neck medical image to be segmented by utilizing a simulated hemodynamic mode to obtain the vessel structure data.

11. The method for head and neck artery vessel segmentation according to any one of claims 2 to 5, wherein the training method of the first multilayer network vessel segmentation model comprises:

determining a first model to be trained, wherein the first model to be trained comprises N first network layers to be trained, each of the N first network layers to be trained comprises a segmentation module to be trained, and N hierarchical resolutions corresponding to the N first network layers to be trained are respectively matched with the N sampling spatial resolutions;

determining at least one first cephaloneck blood vessel sample image and first blood vessel segmentation marking data corresponding to the at least one first cephaloneck blood vessel sample image;

determining N first sampling block data group samples corresponding to a plurality of first sampling central point samples in the first head and neck blood vessel sample image by utilizing the first multi-scale multi-window block cutting sampling operation based on the first blood vessel segmentation marking data;

for each first sampling central point sample in the plurality of first sampling central point samples, based on the first model to be trained, obtaining N first vessel segmentation prediction layer data by using N first sampling block data group samples corresponding to the first sampling central point sample, and determining N first layer loss function values based on the N first vessel segmentation prediction layer data and the first vessel segmentation marker data;

weighting the N first-layer loss function values based on preset weight values corresponding to the N first-layer loss function values respectively to obtain first weighted loss function values;

and adjusting parameters of the first model to be trained based on the first weighted loss function value until the first weighted loss function value meets a preset condition, so as to obtain the first multilayer network vessel segmentation model.

12. The method according to claim 11, wherein the obtaining N first vessel segmentation prediction layer data by using N first sample block data group samples corresponding to the first sample center point sample based on the first model to be trained, and determining N first layer loss function values based on the N first vessel segmentation prediction layer data and the first vessel segmentation flag data comprises:

inputting the N first sampling block data group samples into the first network layers to be trained which are matched with each other for segmentation based on the sampling spatial resolution which corresponds to the N first sampling block data group samples and the hierarchical resolution which corresponds to the N first network layers to be trained, so as to obtain N first vessel segmentation prediction layer data;

for each first vessel segmentation prediction layer data in the N first vessel segmentation prediction layer data, determining skeleton region data in the first vessel segmentation prediction layer data, vessel foreground region data in the first vessel segmentation prediction layer data, and foreground neighborhood data in the first vessel segmentation prediction layer data;

and determining a first layer loss function value corresponding to the first vessel segmentation prediction layer data based on the first vessel segmentation prediction layer data, the first vessel segmentation marking data and a first preset layer loss function, wherein in the first preset layer loss function, the weight of skeleton region data is greater than that of foreground neighborhood data, and the weight of foreground neighborhood data is greater than that of vessel foreground region data.

13. The method for head and neck artery vessel segmentation according to claim 8, wherein the method for training the second multi-layer network vessel segmentation model comprises:

determining a second model to be trained, wherein the second model to be trained comprises N second network layers to be trained, each of the N second network layers to be trained comprises a segmentation module to be trained, and N hierarchical resolutions corresponding to the N second network layers to be trained are respectively matched with the N sampling spatial resolutions;

determining at least one second head and neck blood vessel sample image, blood vessel structure mark data corresponding to the at least one second head and neck blood vessel sample image respectively, and second blood vessel segmentation mark data corresponding to the at least one second head and neck blood vessel sample image respectively;

determining, based on the second blood vessel segmentation marker data, N second sampling block data group samples corresponding to each of a plurality of second sampling center point samples in the second head and neck blood vessel sample image by using the second multi-scale multi-window dicing sampling operation;

based on the coordinates corresponding to the second sampling central point samples, performing block cutting sampling operation on the blood vessel structure mark data to obtain N blood vessel structure mark block data corresponding to the second sampling central point samples;

for each second sampling central point sample in the plurality of second sampling central point samples, based on the model to be trained, obtaining N second vessel segmentation prediction layer data by using N second sampling block data group samples and N vessel structure marker block data corresponding to the second sampling central point sample, and determining N second layer loss function values based on the N second vessel segmentation prediction layer data and the second vessel segmentation marker data;

weighting the N second-layer loss function values based on preset weight values corresponding to the N second-layer loss function values respectively to obtain second weighted loss function values;

and adjusting parameters of the second model to be trained based on the second weighted loss function value until the second weighted loss function value meets a preset condition, so as to obtain the second multilayer network vessel segmentation model.

14. A head and neck artery vessel segmentation device, comprising:

the rough blood vessel region segmentation data determination module is configured to determine rough blood vessel region segmentation data corresponding to the head and neck medical image to be segmented based on the head and neck medical image to be segmented;

the first multi-scale multi-window blood vessel segmentation module is configured to perform blood vessel segmentation on the head and neck medical image to be segmented by using first multi-scale multi-window block sampling operation based on the rough blood vessel region segmentation data to obtain a blood vessel segmentation probability map and rough blood vessel segmentation data corresponding to the head and neck medical image to be segmented; the rough blood vessel region segmentation data is utilized to determine a plurality of first sampling central points in the head and neck medical image to be segmented, a sampling center is provided for subsequent sampling, the blood vessel segmentation is carried out after the first multi-scale multi-window block cutting sampling operation is carried out on the head and neck medical image to be segmented by utilizing the plurality of first sampling central points, blood vessel segmentation probability block data corresponding to the plurality of first sampling central points can be obtained, the blood vessel segmentation probability block data corresponding to the plurality of first sampling central points are spliced to obtain the blood vessel segmentation probability map, and the rough blood vessel segmentation data can be obtained according to the blood vessel segmentation probability map;

the blood vessel structure data determining module is configured to determine blood vessel structure data corresponding to the head and neck medical image to be segmented based on the blood vessel segmentation probability map and the head and neck medical image to be segmented; based on the centerline extraction operation of the blood vessel segmentation probability map, an initial blood vessel centerline corresponding to the head and neck medical image to be segmented can be obtained, and based on the initial blood vessel centerline, a blood vessel structure growth operation is performed on the head and neck medical image to be segmented by utilizing a haemodynamics simulation mode, so that blood vessel structure data can be obtained;

performing second multi-scale multi-window blood vessel segmentation, configured to perform blood vessel segmentation on the head and neck medical image to be segmented by using second multi-scale multi-window block sampling operation based on the blood vessel structure data, so as to obtain blood vessel segmentation data corresponding to the head and neck medical image to be segmented; the vessel structure data is used as a reference, a plurality of second sampling central points can be determined in the head and neck medical image to be segmented, a sampling center is provided for subsequent sampling, the vessel segmentation is carried out after second multi-scale multi-window block cutting sampling operation is carried out on the head and neck medical image to be segmented by utilizing the plurality of second sampling central points, vessel segmentation probability block data corresponding to the plurality of second sampling central points can be obtained, and the vessel segmentation probability block data corresponding to the plurality of second sampling central points are spliced and binarized to obtain the vessel segmentation data;

and the first multi-scale multi-window cutting block sampling operation and the second multi-scale multi-window cutting block sampling operation are performed in the sampling and cutting modes under the scope of a plurality of sampling space resolutions and a plurality of windows by taking a preset image pixel volume as a sampling target.

15. An electronic device, comprising:

a processor; and

a memory having stored therein computer program instructions which, when executed by the processor, cause the processor to perform a method of head and neck artery vessel segmentation according to any one of claims 1 to 13.

16. A computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform a method of head and neck artery vessel segmentation according to any one of claims 1 to 13.