CN117115187B

CN117115187B - Carotid artery wall segmentation method, carotid artery wall segmentation device, carotid artery wall segmentation computer device, and carotid artery wall segmentation storage medium

Info

Publication number: CN117115187B
Application number: CN202311384682.6A
Authority: CN
Inventors: 钱真; 张梦泽; 乔治
Original assignee: Beijing Lianying Intelligent Imaging Technology Research Institute
Current assignee: Beijing Lianying Intelligent Imaging Technology Research Institute
Priority date: 2023-10-24
Filing date: 2023-10-24
Publication date: 2024-02-09
Anticipated expiration: 2043-10-24
Also published as: CN117115187A

Abstract

The application relates to a carotid artery wall segmentation method, a carotid artery wall segmentation device, a carotid artery wall segmentation computer device and a carotid artery wall segmentation storage medium. The method comprises the following steps: acquiring an original medical image; the original medical image comprises carotid arteries; dividing carotid artery walls of carotid arteries in the original medical image according to a preset dividing network, and determining carotid artery wall dividing masks corresponding to the original medical image; wherein the carotid wall segmentation mask comprises segmented carotid walls; the segmentation network is obtained by training a plurality of sample images and labels corresponding to the sample images, wherein the labels comprise a reference carotid wall segmentation mask corresponding to the sample images and a reference signed distance graph, and the reference signed distance graph is used for representing structural characteristics of carotid walls in the sample images. By adopting the method, the accuracy of the obtained carotid artery wall segmentation result can be improved.

Description

Carotid artery wall segmentation method, carotid artery wall segmentation device, carotid artery wall segmentation computer device, and carotid artery wall segmentation storage medium

Technical Field

The present disclosure relates to the field of image processing technologies, and in particular, to a carotid artery wall segmentation method, apparatus, computer device, and storage medium.

Background

With the continuous development of deep learning technology and imaging technology, there are many technologies that apply deep learning technology to images to realize different functions such as segmentation, detection, and classification of images. By applying deep learning techniques to images, doctors can be assisted in better analysis and processing of images.

In the related art, when the image is segmented by deep learning, taking the segmentation of carotid artery wall as an example, a doctor usually draws a carotid artery wall mask on a two-dimensional sample image, combines the carotid artery wall mask images to obtain a three-dimensional carotid artery wall mask image, trains a neural network model by using the three-dimensional carotid artery wall mask image to obtain a trained neural network model, and can be used for achieving the segmentation of carotid artery wall.

However, the above technique has a problem that the obtained segmentation result of the carotid artery wall is not sufficiently accurate.

Disclosure of Invention

In view of the foregoing, it is desirable to provide a carotid artery wall segmentation method, device, computer apparatus, and storage medium that can improve the accuracy of the segmentation result of carotid artery walls.

In a first aspect, the present application provides a carotid wall segmentation method, comprising:

Acquiring an original medical image; the original medical image comprises carotid arteries;

dividing the carotid artery wall of the carotid artery in the original medical image according to a preset dividing network, and determining a carotid artery wall dividing mask corresponding to the original medical image;

wherein the carotid wall segmentation mask comprises segmented carotid walls; the segmentation network is obtained by training a plurality of sample images and labels corresponding to each sample image, wherein the labels comprise a reference carotid wall segmentation mask corresponding to the sample images and a reference signed distance graph, and the reference signed distance graph is used for representing structural characteristics of carotid walls in the sample images.

In one embodiment, the structure of the codec of the split network is an asymmetric structure, and the number of filters in the encoder of the asymmetric structure is greater than the number of filters in the decoder.

In one embodiment, the training method of the split network includes:

dividing carotid artery walls of carotid arteries in each sample image according to an initial dividing network, and determining a predicted carotid artery wall dividing mask and a predicted signed distance graph corresponding to each sample image;

And training the initial segmentation network according to each predicted carotid wall segmentation mask and the corresponding reference carotid wall segmentation mask and according to each predicted signed distance graph and the corresponding reference signed distance graph to determine the segmentation network.

In one embodiment, the training the initial segmentation network according to each predicted carotid wall segmentation mask and the corresponding reference carotid wall segmentation mask and each predicted signed distance graph and the corresponding reference signed distance graph to determine the segmentation network includes:

calculating a first loss between each predicted carotid wall segmentation mask and a corresponding reference carotid wall segmentation mask;

calculating a second loss between each predicted signed distance map and the corresponding reference signed distance map;

the initial segmentation network is trained according to the first loss and the second loss, and the segmentation network is determined.

In one embodiment, the calculating the first loss between each predicted carotid wall segmentation mask and the corresponding reference carotid wall segmentation mask includes:

calculating similarity loss between each predicted carotid wall segmentation mask and a corresponding reference carotid wall segmentation mask;

Calculating cross entropy loss between each predicted carotid wall segmentation mask and a corresponding reference carotid wall segmentation mask;

the similarity loss and the cross entropy loss are taken as the first loss.

In one embodiment, the method further includes, before performing segmentation processing on a carotid wall of a carotid artery in the original medical image according to a preset segmentation network and determining a carotid wall segmentation mask corresponding to the original medical image:

positioning carotid arteries in an original medical image, and determining an interested region image corresponding to the original medical image;

the region of interest image comprises carotid arteries, and the size of the region of interest image is smaller than that of the original medical image.

In one embodiment, the positioning the carotid artery in the original medical image to determine the region of interest image corresponding to the original medical image includes:

inputting an original medical image into a preset positioning network, positioning the carotid artery in the original medical image, and determining intra-cavity segmentation results of the carotid artery;

and cutting the original medical image according to the intra-cavity segmentation result to obtain the region-of-interest image.

In one embodiment, the positioning network comprises a bifurcation detecting sub-network and an intra-cavity sub-network, the original medical image comprises a plurality of two-dimensional slices, inputting the original medical image into a preset positioning network, performing positioning processing on the carotid artery in the original medical image, and determining intra-cavity segmentation results of the carotid artery, wherein the method comprises the following steps:

Inputting a plurality of two-dimensional slices of an original medical image into a bifurcation detection sub-network to perform bifurcation position detection processing, and determining a target two-dimensional slice; the target two-dimensional slice is a slice at the carotid bifurcation occurrence position;

inputting a target two-dimensional slice into the intra-cavity the segmentation process is performed in the cut sub-network, intra-luminal segmentation of carotid arteries on a target two-dimensional slice is determined.

In one embodiment, the method further comprises:

and inputting the region-of-interest image and the carotid artery wall segmentation mask into a preset recognition network for recognition processing, and determining the composition corresponding to each region on the carotid artery wall.

In one embodiment, the identifying network includes a region segmentation sub-network and a classification sub-network, the inputting the region of interest image and the carotid wall segmentation mask into a preset identifying network for identifying, and determining the composition corresponding to each region on the carotid wall, including:

inputting the region-of-interest image and the carotid wall segmentation mask into a region segmentation sub-network to perform plaque identification and region segmentation processing, and determining a plurality of region images corresponding to carotid walls; each region image comprises a corresponding plaque;

And inputting the images of each region into a classification sub-network to classify the plaque in the images of each region, and determining the composition of the plaque corresponding to the images of each region.

In a second aspect, the present application also provides a carotid wall segmentation device, comprising:

the acquisition module is used for acquiring the original medical image; the original medical image comprises carotid arteries;

the segmentation module is used for carrying out segmentation processing on carotid artery walls of carotid arteries in the original medical image according to a preset segmentation network, and determining carotid artery wall segmentation masks corresponding to the original medical image; wherein the carotid wall segmentation mask comprises segmented carotid walls; the segmentation network is obtained by training a plurality of sample images and labels corresponding to each sample image, wherein the labels comprise a reference carotid wall segmentation mask corresponding to the sample images and a reference signed distance graph, and the reference signed distance graph is used for representing structural characteristics of carotid walls in the sample images.

In a third aspect, the present application also provides a computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:

In a fourth aspect, the present application also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of:

In a fifth aspect, the present application also provides a computer program product comprising a computer program which, when executed by a processor, performs the steps of:

The carotid wall segmentation method, the carotid wall segmentation device, the computer equipment and the storage medium are used for determining a carotid wall segmentation mask comprising segmented carotid walls through an original medical image comprising carotid arteries and according to a preset segmentation network, carrying out segmentation processing on the carotid walls in the original medical image; the segmentation network is obtained by training a plurality of sample images and corresponding labels, and the labels comprise a reference carotid wall segmentation mask corresponding to the sample images and a reference signed distance graph, wherein the reference signed distance graph is used for representing structural characteristics of carotid walls in the sample images. In the method, structural features of the carotid wall and a reference carotid wall segmentation mask can be added for training when the segmentation network is trained, so that the structural features of the carotid wall can be comprehensively considered when the segmentation network is trained to segment the carotid wall, the problems that the segmented carotid wall is inconsistent with an actual structure, such as fracture exists, can be avoided, and the accuracy of the segmented carotid wall is improved.

Drawings

FIG. 1 is an internal block diagram of a computer device in one embodiment;

FIG. 2 is a flow chart of a carotid wall segmentation method in one embodiment;

FIG. 3 is a diagram showing an example of the structure of a split network provided in another embodiment;

FIG. 4 is a flow chart of a carotid wall segmentation method according to another embodiment;

FIG. 5 is a flow chart of a carotid wall segmentation method according to another embodiment;

FIG. 6 is a flow chart of a carotid wall segmentation method according to another embodiment;

FIG. 7 is a flow chart of a carotid wall segmentation method according to another embodiment;

FIG. 8 is a diagram of an overall framework for carotid wall segmentation and plaque recognition in another embodiment;

FIG. 9 is a segmented cross-sectional illustration of a carotid wall in another embodiment;

FIG. 10 is a diagram of an example segmentation of the carotid artery in another embodiment;

FIG. 11 is a diagram showing an example of the composition of the carotid artery regions in another embodiment;

fig. 12 is a block diagram of a carotid wall segmentation device, in one embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.

Currently, segmentation and localization are made difficult by the small fraction of the carotid artery and its interior plaque components in the head and neck MRI images when segmenting the carotid artery wall. Second, because the real labels of the carotid wall are marked by contours on 2D slices, rather than directly delineating the 3D surface, it is also difficult to maintain a consistent and continuous shape of the carotid wall in three dimensions. Finally, by segmenting the carotid wall in multiple MRI sequences, partial misalignment of tissue structures may occur even with MRI image registration of different MRI sequences, such that there is a slight deviation in shape and position of the carotid wall in the MRI images of different sequences. Therefore, the existing technology has the problem that the segmentation of the carotid wall cannot be accurately realized.

The carotid artery wall segmentation method provided by the embodiment of the application can be applied to computer equipment, wherein the computer equipment can be a terminal or a server, and an internal structure diagram of the computer equipment can be shown in fig. 1 by taking the terminal as an example. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a carotid artery wall segmentation method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be keys, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.

It will be appreciated by those skilled in the art that the structure shown in fig. 1 is merely a block diagram of some of the structures associated with the present application and is not limiting of the computer device to which the present application may be applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, as shown in fig. 2, a carotid artery wall segmentation method is provided, and the method is applied to the computer device in fig. 1 for illustration, and the method may include the following steps:

s202, acquiring an original medical image; the original medical image includes carotid arteries.

In this step, the original medical image may be a two-dimensional image, a three-dimensional image, or the like. The original medical image may be any single-modality image, such as MR (Magnetic Resonance ) image, CT (Computed Tomography, electronic computed tomography) image, PET (Positron Emission Computed Tomography, positron emission tomography) image, etc., but may also be a multi-modality image, without limitation.

Of course, the original medical image may also include a plurality of sequence images of the same modality, such as a T1 image, a T2 image, a T1-contrast image, a TOF image, and the like, corresponding to the MR image.

Specifically, a scanning device is adopted to scan the neck of a subject, and image reconstruction is carried out on scanned data to obtain an original medical image; or the original medical image stored in advance can be obtained from a storage device such as a database or a cloud end; or may be obtained by other means, not specifically limited herein.

In addition, the above-obtained original medical image may include a carotid artery, which refers to an arterial vessel of the neck, and the carotid artery is an undivided carotid artery, including a carotid artery wall (which may also be referred to as a vessel wall) and a carotid artery lumen; of course other tissue surrounding the carotid artery may also be included in the original medical image.

S204, dividing the carotid artery wall of the carotid artery in the original medical image according to a preset dividing network, and determining a carotid artery wall dividing mask corresponding to the original medical image; the carotid wall segmentation mask includes a segmented carotid wall.

In this step, the split network may be a neural network such as a U-net network or a V-net network. The structure of the split network may be specifically set according to the actual situation, for example, may be a symmetrical structure or an asymmetrical structure. The segmentation network is mainly used for segmenting carotid artery walls of carotid arteries to obtain carotid artery wall segmentation masks, the carotid artery wall segmentation masks comprise segmented carotid artery walls, the carotid artery walls can be binary segmentation masks, the segmented carotid artery walls and the background can be respectively set to different values for display, for example, the value of each pixel/each voxel in the carotid artery walls is represented by 1, and the value of each pixel/each voxel in the background is represented by 0.

In this step, a segmentation network for segmenting the carotid wall of the carotid artery may be trained in advance, where the segmentation network may specifically be obtained by training using a plurality of sample images and a label corresponding to each sample image, where the label includes a reference carotid wall segmentation mask corresponding to the sample image and a reference signed distance graph, where the reference signed distance graph is used to characterize structural features of the carotid wall in the sample image. The structural characteristics of the carotid wall may include the shape, segmentation smoothness, topological relationship, volume, position, and other structural characteristics of the carotid wall.

The method comprises the steps that each sample image comprises a carotid artery, specifically comprises a carotid artery wall and a carotid artery inner cavity, each sample image is provided with a corresponding label, the label is data for labeling the sample images in advance, the label of each sample image can comprise a corresponding carotid artery wall segmentation mask, the carotid artery wall segmentation mask is marked as a reference carotid artery wall segmentation mask, and a reference signed distance graph for representing structural characteristics of the carotid artery wall can be further included. The signed distance map may be denoted SDM (Signed Distance Map) and the reference signed distance map may include a plurality of distance values, the absolute value of each distance value may represent the distance from a point x in carotid space in the sample image to a point on the nearest carotid wall surface S, and the sign of each distance value may represent whether the point x is inside or outside the carotid wall surface S, wherein a zero distance or zero order set represents that the point x is on the carotid wall surface.

For the reference signed distance graph of each sample image, danielsson algorithm can be used, the carotid artery wall is segmented in advance by means of manual marking points to obtain a manual segmentation graph, then the manual segmentation graph is obtained based on manual segmentation graph calculation, pixels farther from the surface of the carotid artery wall of the manual marking can be given higher weight in the reference signed distance graph, and the weight of pixels closer to the reference signed distance graph is reduced, so that the segmentation network can be ensured to prioritize more accurate labeled pixels in the training process, less dependence on less determined pixels is caused, and the problems that the segmented carotid artery wall is inconsistent with an actual structure, such as the situation of fracture, and the like, can be avoided, and the segmentation performance of the segmentation network is improved.

In particular, when the segmentation network is trained, each sample image and the corresponding label thereof can be input into the initial segmentation network to be subjected to segmentation processing, so as to obtain a corresponding predicted segmentation mask. And then combining the corresponding reference carotid artery wall segmentation mask and the reference signed distance graph in the label to train the initial segmentation network, and finally obtaining a trained segmentation network when certain conditions are met.

After the segmentation network is trained, the original medical image can be input into the segmentation network to carry out the segmentation processing of the carotid wall, and a corresponding carotid wall segmentation mask is obtained. The method can also be used for preprocessing the original medical image, inputting the preprocessed image into the segmentation network for carotid wall segmentation processing, and obtaining a corresponding carotid wall segmentation mask, wherein the preprocessing can be used for cutting the original medical image into a plurality of image blocks, or cutting an image of a carotid region from the original medical image, or other preprocessing operations, such as normalization processing, registration processing among a plurality of sequences, and the like. In addition, when the carotid artery wall in the original medical image is segmented in the segmentation network, the carotid artery in the original medical image may be segmented first, and then the segmentation is continued based on the segmented carotid artery to obtain the segmented carotid artery wall.

Further, the segmentation network may here comprise one or more input channels, corresponding to the number of sequences of the original medical images described above, i.e. when the original medical images are single sequence images, the segmentation network may be a single channel network; when the original medical image is a plurality of sequential images, the segmentation network may be a multi-channel network, and the segmentation result corresponding to each sequential image is obtained through the segmentation network.

In the carotid wall segmentation method, a carotid wall segmentation mask comprising segmented carotid walls is determined by segmenting carotid walls in an original medical image comprising carotid and according to a preset segmentation network; the segmentation network is obtained by training a plurality of sample images and corresponding labels, and the labels comprise a reference carotid wall segmentation mask corresponding to the sample images and a reference signed distance graph, wherein the reference signed distance graph is used for representing structural characteristics of carotid walls in the sample images. In the method, structural features of the carotid wall and a reference carotid wall segmentation mask can be added for training when the segmentation network is trained, so that the structural features of the carotid wall can be comprehensively considered when the segmentation network is trained to segment the carotid wall, the problems that the segmented carotid wall is inconsistent with an actual structure, such as fracture exists, can be avoided, and the accuracy of the segmented carotid wall is improved.

The following embodiments describe the specific structure of the above-described split network.

In another embodiment, on the basis of the above embodiment, the structure of the codec of the split network is an asymmetric structure, and the number of filters in the encoder of the asymmetric structure is greater than the number of filters in the decoder.

In this embodiment, the split network may be a three-dimensional split network, and the main structure thereof may be, for example, a 3D U-net structure, and in particular, a nnU-net frame may be used, but the split network in this embodiment is not limited to this structure, and reference is made to an exemplary diagram of a network structure illustrated in fig. 3, in which numbers at the top of each module respectively represent the number of channels, and numbers at the bottom represent dimensions of feature vectors.

In particular, the split network is mainly composed of an encoder-decoder structure, wherein a layer-jump connection can connect both the encoder and the decoder paths. Wherein the encoder comprises 5 levels of convolutional layers, step convolutions with the same downsampling rate. The decoder follows the same design, including transposed convolutional upsampling after the upsampling features from the lower level and the features at the same level as the encoder branches are layer-hopped. After each convolution operation, a activation process and a batch normalization process may be performed using a activation function with a slope of 0.01.

Further, to cover a wider range of data variations and enhance the overall modeling capability, the encoder and decoder structures of the split network may be arranged in an asymmetric structure, i.e. an asymmetric structure. Wherein a symmetrical structure means that the number of filters in the encoder is equal to the number of filters in the decoder, and an asymmetrical structure means that the number of filters in the encoder is greater than the number of filters in the decoder. For the number of filters in the encoder and the number of filters in the decoder here, it may be that the number of filters in the decoder remains the same as the original number of 3D U-net structures, the number of filters in the encoder may be doubled; of course, other conditions are also possible and are not particularly limited herein.

In this embodiment, the structure of the codec of the segmentation network is an asymmetric structure, and the number of filters in the encoder of the asymmetric structure is greater than the number of filters in the decoder, so that the segmentation network can capture complex details, utilize richer information from the input original medical image, and effectively process various anatomical variations and complexities existing in the carotid wall image, thereby improving the segmentation accuracy and the overall performance of the segmentation network.

The foregoing embodiment simply describes the training process of the split network, and the following embodiment describes the training process of the split network specifically.

In another embodiment, another carotid artery wall segmentation method is provided, and based on the above embodiment, as shown in fig. 4, the training method of the segmentation network may include the following steps:

s302, dividing carotid artery walls of carotid arteries in each sample image according to an initial dividing network, and determining a predicted carotid artery wall dividing mask and a predicted signed distance graph corresponding to each sample image.

In this step, before training the segmentation network, each sample image including the carotid artery and the corresponding reference signed distance graph may be obtained first, then each sample image and each reference signed distance graph are input into the initial segmentation network, and the carotid artery wall in each sample image is subjected to segmentation processing, so as to obtain a predicted carotid artery wall segmentation mask corresponding to each sample image, where the predicted carotid artery wall segmentation mask includes the carotid artery wall segmented in the training process; meanwhile, in the training process, the initial segmentation network can also output a predicted signed distance graph corresponding to each sample image, and the predicted signed distance graph can represent structural characteristics of carotid artery walls in the corresponding sample images.

In addition, each sample image may be a sample image including a plurality of sequences, for example, sample images including four sequences, respectively: the T1 sample image, the T2 sample image, the T1-contrast sample image, the TOF sample image, etc., may be subjected to data enhancement operations on multiple sequences of sample images in order to avoid possible inter-sequence registration errors. Specifically, in the training process, the T1 sample image and the corresponding label thereof are fixed, and small random translation and rotation operations are applied to the T2 sample image, the T1-contrast sample image and the TOF sample image, so that training data can be enhanced, and the accuracy of segmentation and identification of a trained segmentation network and the generalization capability of the segmentation network can be improved.

In the initial segmentation network training process, similar to the segmentation of the original medical image, each sample image can be directly input into the initial segmentation network for carotid artery wall segmentation; or the method can also be that each sample image is preprocessed and then input into an initial segmentation network for carotid artery wall segmentation; or may be otherwise, without specific limitation.

S304, training an initial segmentation network according to each predicted carotid wall segmentation mask and a corresponding reference carotid wall segmentation mask and according to each predicted signed distance graph and a corresponding reference signed distance graph to determine a segmentation network.

In this step, after obtaining the predicted carotid wall segmentation mask and the predicted signed distance map for each sample image, the loss may be calculated with the corresponding label to train the initial segmentation network.

In the case of network training for calculating the loss specifically, as an alternative embodiment, referring to fig. 5, the following steps may be used to calculate the loss and perform network training:

s402, calculating first loss between each predicted carotid wall segmentation mask and a corresponding reference carotid wall segmentation mask.

In this step, optionally, a similarity loss between each predicted carotid wall segmentation mask and the corresponding reference carotid wall segmentation mask may be calculated; calculating cross entropy loss between each predicted carotid wall segmentation mask and the corresponding reference carotid wall segmentation mask; the similarity loss and the cross entropy loss are taken as the first loss.

The similarity loss may be denoted as a Dice loss, and may be used to evaluate accuracy of the segmentation result, where a preset Dice loss function may be used to calculate a Dice loss between the predicted carotid wall segmentation mask and the corresponding reference carotid wall segmentation mask of each sample image, and the obtained loss may be denoted as a similarity loss or a Dice similarity coefficient (DSC, dice Similarity Cofficient).

Meanwhile, a cross entropy loss function can be used to calculate the cross entropy loss between the predicted carotid wall segmentation mask and the corresponding reference carotid wall segmentation mask of each sample image.

Then, the first loss corresponding to each sample image can be obtained after the mathematical processing through the similarity loss and the cross entropy loss of each sample image. The mathematical processing here may include, for example: weighted summation, direct summation, mean processing, etc.

S404, calculating second losses between each predicted signed distance graph and the corresponding reference signed distance graph.

In order to adapt to the visual field of various input images and the change of the blood vessel size in the images, the reference signed distance map may be normalized before the loss is calculated in this embodiment, specifically, a maximum positive value except for the carotid artery wall (or referred to as a blood vessel wall) and a minimum negative value in the carotid artery wall of the reference signed distance map, and the value of the reference signed distance map is normalized to the range (-1, 1) of each input image (original medical image).

Similarly, for the initial segmentation network when outputting the predicted signed distance map, we can also divide the predicted signed distance map by the maximum positive value outside the carotid wall and the minimum negative value inside the carotid wall, we further normalize the value of the predicted signed distance map to the range (-1, 1) of each input image (sample image) and output after activation in the output layer using the tanh activation function.

Additionally, to ensure the differentiability of all network layers in the initial segmentation network and match the predicted signed distance map to the predicted carotid wall segmentation mask, the predicted carotid wall segmentation mask may be generated by a smooth approximation using a Heaviside function. Because there is a strict mapping between the predicted signed distance map calculated from the carotid boundary surface and the binary predicted carotid wall segmentation mask, the segmentation result (i.e., the predicted carotid wall segmentation mask) can be optimized by reducing the loss of the signed distance map.

In particular, when calculating the loss of the signed distance map, the loss between the predicted signed distance map and the corresponding reference signed distance map for each sample image may be calculated and noted as the second loss. The calculation of this second loss can be seen from the following equation (1):

（1）

wherein,y* Representing a predicted signed distance map;p* Representing a reference signed distance graph;L _SDM representing the loss of the signed distance map, i.e. the second loss.

S406, training the initial segmentation network according to the first loss and the second loss, and determining the segmentation network.

In this step, after the first loss and the second loss of each sample image are obtained, the first loss and the second loss may be processed, or specifically, the similarity loss, the cross entropy loss, and the loss of the signed distance graph may be processed, to obtain a final total loss. The total loss can be calculated by the following formula (2):

（2）

Wherein,L _segmentation representing the total loss;the weight coefficients representing the losses, respectively, may be empirically determined, for example, as a set of values, respectively +.>，/>，/>；L _Dice A loss of similarity is indicated and,L _SDM a loss of the signed distance map is represented,L _CE representing cross entropy loss.

By calculating the respective losses by using the formula (2), the total loss can be obtained, and then the initial segmentation network can be trained by using the total loss, and finally the trained segmentation network can be obtained.

Further, as can be seen from the above description, in the present embodiment, the segmentation network predicts a signed distance map during the training process, and since the signed distance map SDM can provide an implicit representation of the shape and more information, it can encode more abundant structural features, so in the training process of the segmentation network of the present embodiment, in order to predict the signed distance map through the segmentation network, the segmentation network learns the volume, position and shape information of the carotid wall segmentation mask during the training process, so as to implicitly incorporate the continuity and smoothness conditions into the segmentation process, so that the carotid wall in the carotid wall segmentation mask obtained by the final segmentation network is relatively continuous and smooth, and is more in line with the actual carotid wall situation.

In this embodiment, the predicted carotid wall segmentation mask and the predicted signed distance map are obtained by segmenting each sample image based on the initial segmentation network, and the initial segmentation network is trained according to the predicted carotid wall segmentation mask and the predicted signed distance map by combining corresponding labels, so that the segmentation network combines not only the reference carotid wall segmentation mask but also the reference signed distance map in the training process, and the segmentation process can be restrained by the reference signed distance map and the reference carotid wall segmentation mask, so that the segmentation performance of the segmentation network can be improved, and the segmented carotid wall is more continuous and smooth, and is more in line with the actual situation. In addition, the training loss of the segmentation network can be definitely determined through the loss between the prediction segmentation result of the carotid artery wall and the reference segmentation result and the loss training segmentation network between the prediction signed distance graph of the carotid artery wall and the reference signed distance graph, and the accuracy and the efficiency of the segmentation network training can be improved. Further, the loss between the predicted segmentation result and the reference segmentation result of the carotid artery wall comprises similarity loss and cross entropy loss, so that the richness of the loss can be increased, and the accuracy of the trained segmentation network is further improved.

The above-described embodiments have mentioned that the segmentation process may be performed by preprocessing an original medical image and then inputting the preprocessed image into the segmentation network, and the following embodiments will explain the preprocessing process.

In another embodiment, another carotid artery wall segmentation method is provided, and the method may further include the following steps before S204, where the steps are performed:

and step A, positioning carotid arteries in the original medical image, and determining an area-of-interest image corresponding to the original medical image.

In this step, after the original medical image is obtained, the carotid artery in the original medical image may be located first, where the locating may be performed manually, by a network locating manner, or by other locating manners, and in any case, the location of the carotid artery may be located in the original medical image.

Because the carotid artery occupies only a small part in the original medical image of the head and the neck, in order to improve the segmentation accuracy, after the position of the carotid artery in the original medical image is located, the image of the carotid artery in the original medical image can be cut out based on the located position of the carotid artery, so as to obtain the region-of-interest image, and then the region-of-interest image can be input into a segmentation network, and the carotid artery wall of the carotid artery in the region-of-interest image is segmented, so as to obtain a carotid artery wall segmentation mask. The carotid artery is mainly included in the region-of-interest image, and the size of the region-of-interest image is smaller than that of the original medical image, so that the carotid artery region is focused as much as possible when the carotid artery wall is segmented by the subsequent segmentation network, and interference of other regions is reduced.

In the embodiment, the carotid artery in the original medical image is positioned, and the region-of-interest image corresponding to the original medical image is determined, so that the carotid artery region is focused as much as possible when the carotid artery wall is segmented by the subsequent segmentation network, the interference of other regions is reduced, and the segmentation accuracy is improved; and the segmentation calculation amount can be reduced, and the segmentation efficiency is improved. Further, this approach may eliminate the need for manual marking of the carotid artery, which is easier for the physician to apply, while reducing the time required.

The following example illustrates one possible implementation of the above-described localization of carotid arteries in raw medical images.

In another embodiment, another carotid artery wall segmentation method is provided, and based on the above embodiment, as shown in fig. 6, the step a may include the following steps:

s502, inputting the original medical image into a preset positioning network, positioning the carotid artery in the original medical image, and determining intra-cavity segmentation results of the carotid artery.

Wherein, as an alternative embodiment, the positioning network includes a bifurcation detecting sub-network and an intra-cavity splitting sub-network, wherein the specific architecture of each of the bifurcation detecting sub-network and the intra-cavity splitting sub-network is not particularly limited herein, for example, the bifurcation detecting sub-network may be a network constructed based on a Resnet50 network, and includes 49 convolution layers for feature extraction and one full connection layer for classification; the intra-cavity subnetwork may be a 2D U-net subnetwork comprising an encoder and a decoder.

Both the bifurcation detecting subnetwork and intra-cavity subnetwork may be pre-trained, the bifurcation detection sub-network is used for detecting the bifurcation occurrence position of the carotid artery in the original medical image, and the intra-cavity segmentation sub-network is used for carrying out segmentation processing on the intra-cavity of the carotid artery.

Taking the example that the original medical image includes a sequence image, the original medical image may include a plurality of two-dimensional slices, as an alternative embodiment, the following steps may be used to perform positioning processing on the carotid artery in the original medical image:

step 1, inputting a plurality of two-dimensional slices of an original medical image into a bifurcation detection sub-network to perform bifurcation position detection processing, and determining a target two-dimensional slice; the target two-dimensional slice is a slice at the carotid bifurcation occurrence location.

Specifically, a plurality of two-dimensional slices included in one sequence image of the original medical image may be input into a bifurcation detecting sub-network, and bifurcation positions of carotid arteries in the two-dimensional slices are detected in the bifurcation detecting sub-network, so as to obtain a slice at a carotid artery bifurcation occurrence position, and the slice may be recorded as a target two-dimensional slice.

For the original medical image which is a plurality of sequence images, each sequence comprises a plurality of two-dimensional slices, the two-dimensional slices of each sequence can be input into a bifurcation detection sub-network, and the carotid bifurcation occurrence position in each sequence is detected to obtain a target two-dimensional slice corresponding to each sequence.

It should be noted that the target two-dimensional slice of each sequence may be one slice or a plurality of slices.

Step 2 of the method, in which the step 2, inputting a target two-dimensional slice into the intra-cavity the segmentation process is performed in the cut sub-network, intra-luminal segmentation of carotid arteries on a target two-dimensional slice is determined.

In this step, after obtaining the target two-dimensional slice of each sequence, the two-dimensional slice of each sequence may be input into an intra-luminal intra-cavity network, and intra-luminal segmentation processing may be performed on the carotid artery in the target two-dimensional slice of each sequence, to obtain intra-luminal intra-cavity segmentation results of the carotid artery in the target two-dimensional slice of each sequence.

By way of example, the input to the intra-luminal segmentation sub-network may be a multi-sequence MR image of size 512×512×n, where n is the number of axial slices and the output is intra-luminal segmentation of the carotid artery.

S504, cutting the original medical image according to the intra-cavity segmentation result to obtain an interested region image.

In this step, after obtaining the intra-cavity segmentation result of each target two-dimensional slice, taking a sequence of original medical images as an example, the intra-cavity segmentation result corresponding to the sequence may be a segmentation mask in the carotid artery cavity, where coordinates of each point in the cavity may be included, and then the center point coordinates corresponding to the sequence may be obtained by calculating the center point coordinates from the coordinates of each point.

Then, the sequence in the original medical image can be cut according to a certain size by taking the center point coordinate as the center, and a cut image corresponding to the sequence can be obtained and can be recorded as an interested region image. It should be noted that the region of interest image generally includes all carotid artery regions, but other irrelevant regions may be cut off as much as possible.

For example, if the input of the intra-lumen segmentation sub-network can be a multi-sequence MR image of size 512×512×n, then a region of interest of size 128×128×30 surrounding the carotid artery can be obtained after clipping here, i.e. a region of interest image is obtained.

In this embodiment, the positioning and intra-cavity segmentation of the carotid artery are realized by inputting the original medical image into the positioning network, so that the original medical image is cut accordingly, and thus the errors and the inefficiency caused by manual positioning and segmentation can be reduced by the way of positioning and intra-cavity segmentation through the network, and the positioning and segmentation precision and the segmentation efficiency are improved. In addition, carotid bifurcation position detection and intra-cavity segmentation processing are respectively carried out through two sub-networks in the positioning network, so that carotid artery positioning and intra-cavity segmentation can be rapidly and accurately realized through the two sub-networks, and carotid artery positioning and intra-cavity segmentation efficiency and accuracy are further improved.

The above embodiment mentions that the segmentation of the carotid artery wall can be performed quickly and accurately by using the region-of-interest image, and further, the components on the carotid artery wall can be identified based on the region-of-interest image, and the following embodiment will describe the process.

In another embodiment, another carotid artery wall segmentation method is provided, and the method may further include the following steps based on the above embodiment:

and B, inputting the region-of-interest image and the carotid artery wall segmentation mask into a preset recognition network for recognition processing, and determining the composition corresponding to each region on the carotid artery wall.

Wherein the identification network may be a neural network, the specific network architecture and network type are not particularly limited herein.

The identification network may also be pre-trained for identifying the constituent elements on the carotid artery wall. Since the plaque has no fixed structure, shape or size, the loss function constructed by the recognition network during training may not include the loss of the signed distance graph, but only include the Dice loss (i.e., similarity loss) and the cross entropy loss, and the total loss can be calculated through the two loss to train the recognition network, where the formula of the total loss can be shown in the following formula (3):

（3）

Wherein,L _{indentification} representing the total loss here;the weight coefficients representing the losses, respectively, may be empirically determined, for example, as a set of values, respectively +.>，/>；L _Dice A loss of similarity is indicated and,L _CE representing cross entropy loss.

By calculating the respective losses by using the formula (3), the total loss can be obtained, and then the recognition network can be trained by using the total loss, and finally the trained recognition network can be obtained.

Specifically, after the region-of-interest image is obtained, the region-of-interest image may be input into the recognition network, and the constituent components of each region on the carotid artery wall in the region-of-interest image may be recognized to obtain constituent components corresponding to each region on the carotid artery wall.

Illustratively, the components herein may include, for example, calcified CA, lipid core LRC, bleeding HE, normal components, and the like.

It should be noted that the composition of each region on the carotid artery wall obtained here is only one intermediate data, which can be used as an input data source for subsequent image analysis, for example, the carotid artery wall in the original medical image can be further analyzed based on the composition here.

In this embodiment, the components corresponding to each region on the carotid wall are determined by inputting the region of interest image and the carotid wall segmentation mask into the preset recognition network for recognition processing, and since the components on the carotid wall are generally in the carotid and occupy only a small part of the original medical images of the head and neck, in order to improve the segmentation accuracy, the components on the carotid wall in the region of interest image can be assisted by the carotid wall segmentation mask for recognition, thereby improving the accuracy of the finally recognized components.

The following embodiments describe in detail the process of identifying the constituent components of each region on the carotid wall by specifically using the region-dividing sub-network and the classifying sub-network, taking the case that the identifying network includes the region-dividing sub-network and the classifying sub-network as examples.

In another embodiment, another carotid artery wall segmentation method is provided, and based on the above embodiment, as shown in fig. 7, the step B may include the following steps:

s602, inputting an interested region image and a carotid artery wall segmentation mask into a region segmentation sub-network to perform plaque identification and region segmentation processing, and determining a plurality of region images corresponding to carotid artery walls; each region image includes a corresponding patch.

The area division sub-network may be a neural network, and its specific structure may be similar to the above-mentioned division network, for example, may also be a U-net network. The regional division sub-network can also be trained in advance, and as for how many regions or how large the carotid wall is divided into, this can be set according to the actual situation.

Specifically, after the region-of-interest image and the carotid wall segmentation mask are obtained, the region-of-interest image and the carotid wall segmentation mask may be input into a region segmentation sub-network, in the region segmentation sub-network, a region where a corresponding carotid wall is located may be obtained in the region-of-interest image through the carotid wall segmentation mask, then a plaque on the region where the carotid wall is located may be identified, and after the plaque is identified, a segmentation process may be performed on the plaque region, so as to obtain a plurality of region images corresponding to the carotid wall.

It should be noted that, each area image may include a corresponding plaque, for example, an area without a plaque may also be calculated as an area image.

S604, inputting each area image into a classification sub-network to classify the plaque in each area image, and determining the composition of the plaque corresponding to each area image.

The classification sub-network may be a two-classification network or a multi-classification network, and specific network types may be set according to actual situations, for example, a logistic regression model, a decision tree model, a support vector machine model, a random forest model, and the like. The classification network may also be pre-trained.

In this step, after each area image is obtained, each area image may be input into a classification sub-network, and the components of the plaque in each area image may be classified to obtain the components of the plaque corresponding to each area image. Here, each area image generally corresponds to a plaque component.

In this embodiment, the plaque on the carotid artery wall in the region-of-interest image is identified and segmented by using the region segmentation sub-network, and then the plaque components in each segmented region are classified by using the classification network to obtain the plaque components in each region image, so that the plaque components are segmented by using the two networks first and the plaque components are reclassified, the difficulty in identifying the plaque components on the carotid artery wall can be simplified, and the accuracy and the efficiency of identifying the plaque components are improved.

A detailed embodiment is given below to explain the technical solution of the present application, taking an original medical image as a multi-sequence MR image as an example, the present embodiment proposes an end-to-end processing flow, referring to the overall processing flow chart shown in fig. 8, and on the basis of the above embodiment, the method may include the following steps:

s1, acquiring multi-sequence magnetic resonance MR Images (namely MR Images); each MR image comprises carotid arteries of the same object at the same moment;

s2, inputting (i.e. Input in the figure) a plurality of two-dimensional slices of each MR image into a bifurcation detection sub-network to perform bifurcation position detection processing, and determining a target two-dimensional slice corresponding to each MR image; the target two-dimensional slice is a slice at the carotid bifurcation occurrence position;

s3, inputting the target two-dimensional slice of each MR image into an intra-cavity segmentation sub-network (a positioning network Localization is formed by the target two-dimensional slice and the bifurcation detection sub-network in S2), performing segmentation processing, and determining intra-cavity segmentation results of carotid arteries on the target two-dimensional slice of each MR image;

s4, cutting (i.e. Crop) the corresponding MR image according to the intra-cavity segmentation result of each MR image to obtain a cutting image (i.e. the region of interest image) corresponding to each MR image;

S5, inputting the clipping image of each MR image into a Segmentation network (namely Segmentation) to perform Segmentation processing on carotid artery walls, and determining (namely Output) a carotid artery wall Segmentation mask (namely Vessel wall) corresponding to each MR image; the carotid artery wall segmentation mask comprises segmented carotid artery walls;

s6, inputting the clipping image corresponding to each MR image and the carotid artery wall segmentation mask into a region segmentation sub-network to perform plaque identification and region segmentation processing, and determining a plurality of region images corresponding to the carotid artery wall; each region image comprises a corresponding plaque;

s7, inputting each area image into a classification sub-network (the classification sub-network and the area segmentation sub-network in S6 form an Identification network together) to classify the plaque in each area image, and determining the Composition (namely the Composition) of the plaque corresponding to each area image; the constituent components herein may include CA, LRH, HE and the like, for example.

For example, referring to the segmented cross-sectional illustration of carotid artery wall shown in fig. 9, the first row shows the original T1 image, the second row shows the results of manual segmentation through different colored masks, the third row shows the results of automatic segmentation through different colored masks, and the fourth row shows the masks overlapped by the manual and automatic segmentation through different colored masks. It can be seen that by adopting the automatic segmentation method of the embodiment of the application, the obtained carotid wall segmentation mask is relatively close to the manual segmentation result, i.e. the accuracy of the segmentation result obtained by the automatic segmentation of the embodiment of the application is higher.

Further, referring to the segmentation example graph of carotid arteries shown in fig. 10, the first behavior is eight carotid arteries segmented manually, and the second behavior is the corresponding carotid arteries segmented automatically by the method of the present application. By comparing the carotid arteries in the two rows of corresponding locations, it can be seen that the automatically segmented carotid arteries of the present application are qualitatively very consistent with the manually segmented carotid arteries (i.e., gold standard). It can also be seen that the vessel wall surface of the carotid artery segmented by the method of the present application is smoother than the vessel wall of the carotid artery segmented manually, indicating that the method of the embodiments of the present application may be superior in terms of cross-section consistency.

Referring again to the example graph of the composition of the carotid artery regions shown in fig. 11, where the first behavior is manually labeled composition of the carotid artery regions, the second behavior is automatically determined by the method of the present application, and it can be seen that, from a visual comparison, there is a strong correspondence between the manually labeled composition and the automatically determined composition of the present application. It can also be seen that the automatically determined composition of the present application exhibits a smoother three-dimensional composition surface than the manually noted composition, and in some cases, the determined composition of the present application exhibits a more realistic plaque morphology in three dimensions, such as the lipid-rich core composition shown in the first column of fig. 11.

The following description of the training process of the above-mentioned segmentation network may include the following steps:

d1, acquiring each sample image and a corresponding label; the label comprises a reference carotid wall segmentation mask corresponding to the sample image and a reference signed distance graph, wherein the reference signed distance graph is used for representing structural characteristics of carotid walls in the sample image;

d2, inputting each sample image into an initial segmentation network to carry out carotid artery wall segmentation processing, and determining a predicted carotid artery wall segmentation mask and a predicted signed distance graph corresponding to each sample image;

d3, calculating similarity loss between each predicted carotid wall segmentation mask and a corresponding reference carotid wall segmentation mask;

d4, calculating cross entropy loss between each predicted carotid wall segmentation mask and the corresponding reference carotid wall segmentation mask;

d5, calculating signed distance map losses between each predicted signed distance map and the corresponding reference signed distance map;

and D6, training the initial segmentation network according to the similarity loss, the cross entropy loss and the signed distance graph loss, and determining the segmentation network.

At present, accurate assessment of carotid wall thickness increase and identification of dangerous plaque components is critical for early diagnosis and risk management of carotid atherosclerosis. The present application thus proposes a carotid wall segmentation method as described above for automatically segmenting the carotid wall and identifying the components of carotid plaque in a Magnetic Resonance (MR) image. A new 3D segmentation method is introduced in the segmentation of the carotid wall, which can normalize the segmentation surface in 3D and reduce erroneous segmentation. Meanwhile, the customized data enhancement operation is introduced in the segmentation network training stage of segmenting the carotid artery wall, so that training data can be enhanced, and the false positive rate of calcification and bleeding identification is reduced.

Further, the carotid wall segmentation method of the present application was trained and tested on a dataset containing 115 patients, resulting in accurate segmentation of the carotid wall (Dice value 0.8459), which is better than the best results in the published study (Dice value 0.7885). In addition, the accuracy of identification of calcified, lipid-rich core and bleeding components using the framework of carotid wall segmentation method of the present application was 0.82, 0.73 and 0.88, respectively. It follows that the framework of carotid artery wall segmentation methods presented herein is likely to be used in clinical and research settings, helping radiologists to alleviate the burdensome reading burden and enabling faster and better assessment of carotid plaque risk.

It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.

Based on the same inventive concept, the embodiment of the application also provides a carotid artery wall segmentation device for realizing the carotid artery wall segmentation method. The implementation of the solution provided by the device is similar to that described in the above method, so specific limitations in one or more embodiments of the carotid wall segmentation device provided below may be found in the limitations of the carotid wall segmentation method described above, and will not be repeated here.

In one embodiment, as shown in fig. 12, there is provided a carotid wall segmentation device comprising: the device comprises an acquisition module and a segmentation module, wherein:

In another embodiment, the structure of the codec of the split network is an asymmetric structure, and the number of filters in the encoder of the asymmetric structure is greater than the number of filters in the decoder.

In another embodiment, there is provided another carotid artery wall segmentation device, which, on the basis of the above embodiment, further includes a segmentation module including:

the segmentation unit is used for carrying out segmentation processing on carotid artery walls of carotid arteries in each sample image according to an initial segmentation network, and determining a predicted carotid artery wall segmentation mask and a predicted signed distance graph corresponding to each sample image;

the training unit is used for training the initial segmentation network according to each predicted carotid wall segmentation mask and the corresponding reference carotid wall segmentation mask and according to each predicted signed distance graph and the corresponding reference signed distance graph to determine the segmentation network.

Optionally, the training unit may include:

a first loss calculation subunit for calculating a first loss between each predicted carotid wall segmentation mask and a corresponding reference carotid wall segmentation mask;

a second loss calculation subunit for calculating a second loss between each predicted signed distance map and the corresponding reference signed distance map;

And the training subunit is used for training the initial segmentation network according to the first loss and the second loss and determining the segmentation network.

Optionally, the training subunit is specifically configured to calculate a similarity loss between each predicted carotid wall segmentation mask and a corresponding reference carotid wall segmentation mask; calculating cross entropy loss between each predicted carotid wall segmentation mask and a corresponding reference carotid wall segmentation mask; the similarity loss and the cross entropy loss are taken as the first loss.

In another embodiment, another carotid wall segmentation device is provided, based on the foregoing embodiment, the segmentation module performs segmentation processing on a carotid wall of a carotid artery in an original medical image according to a preset segmentation network, and before determining a carotid wall segmentation mask corresponding to the original medical image, the device may further include:

the positioning module is used for positioning carotid arteries in the original medical image and determining an interested region image corresponding to the original medical image; the region of interest image comprises carotid arteries, and the size of the region of interest image is smaller than that of the original medical image.

Optionally, the positioning module includes:

The positioning unit is used for inputting the original medical image into a preset positioning network, positioning the carotid artery in the original medical image and determining intra-cavity segmentation results of the carotid artery;

and the clipping unit is used for clipping the original medical image according to the intra-cavity segmentation result to obtain the region-of-interest image.

Optionally, the positioning network includes a bifurcation detecting sub-network and an intra-cavity dividing sub-network, the original medical image includes a plurality of two-dimensional slices, and the positioning unit includes:

the detection subunit is used for inputting a plurality of two-dimensional slices of the original medical image into the bifurcation detection sub-network to carry out bifurcation position detection processing and determine a target two-dimensional slice; the target two-dimensional slice is a slice at the carotid bifurcation occurrence position;

a segmentation subunit for inputting the target two-dimensional slice into the intra-cavity segmentation network for segmentation processing, intra-luminal segmentation of carotid arteries on a target two-dimensional slice is determined.

In another embodiment, another carotid artery wall segmentation device is provided, and the device may further include, based on the above embodiment:

the recognition module is used for inputting the region-of-interest image and the carotid wall segmentation mask into a preset recognition network for recognition processing and determining the composition corresponding to each region on the carotid wall.

Optionally, the identification module may include:

the identification unit is used for inputting the region-of-interest image and the carotid wall segmentation mask into the region segmentation sub-network to perform plaque identification and region segmentation processing and determining a plurality of region images corresponding to the carotid wall; each region image comprises a corresponding plaque;

and the classifying unit is used for inputting the images of each region into the classifying sub-network to classify the plaque in the images of each region, and determining the composition components of the plaque corresponding to the images of each region.

The various modules in the carotid wall segmentation apparatus described above may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

In one embodiment, a computer device is provided comprising a memory and a processor, the memory having stored therein a computer program, the processor when executing the computer program performing the steps of:

acquiring an original medical image; the original medical image comprises carotid arteries; dividing the carotid artery wall of the carotid artery in the original medical image according to a preset dividing network, and determining a carotid artery wall dividing mask corresponding to the original medical image; wherein the carotid wall segmentation mask comprises segmented carotid walls; the segmentation network is obtained by training a plurality of sample images and labels corresponding to each sample image, wherein the labels comprise a reference carotid wall segmentation mask corresponding to the sample images and a reference signed distance graph, and the reference signed distance graph is used for representing structural characteristics of carotid walls in the sample images.

In one embodiment, the processor when executing the computer program further performs the steps of:

dividing carotid artery walls of carotid arteries in each sample image according to an initial dividing network, and determining a predicted carotid artery wall dividing mask and a predicted signed distance graph corresponding to each sample image; and training the initial segmentation network according to each predicted carotid wall segmentation mask and the corresponding reference carotid wall segmentation mask and according to each predicted signed distance graph and the corresponding reference signed distance graph to determine the segmentation network.

calculating a first loss between each predicted carotid wall segmentation mask and a corresponding reference carotid wall segmentation mask; calculating a second loss between each predicted signed distance map and the corresponding reference signed distance map; the initial segmentation network is trained according to the first loss and the second loss, and the segmentation network is determined.

calculating similarity loss between each predicted carotid wall segmentation mask and a corresponding reference carotid wall segmentation mask; calculating cross entropy loss between each predicted carotid wall segmentation mask and a corresponding reference carotid wall segmentation mask; the similarity loss and the cross entropy loss are taken as the first loss.

positioning carotid arteries in an original medical image, and determining an interested region image corresponding to the original medical image; the region of interest image comprises carotid arteries, and the size of the region of interest image is smaller than that of the original medical image.

inputting an original medical image into a preset positioning network, positioning the carotid artery in the original medical image, and determining intra-cavity segmentation results of the carotid artery; and cutting the original medical image according to the intra-cavity segmentation result to obtain the region-of-interest image.

Inputting a plurality of two-dimensional slices of an original medical image into a bifurcation detection sub-network to perform bifurcation position detection processing, and determining a target two-dimensional slice; the target two-dimensional slice is a slice at the carotid bifurcation occurrence position; inputting a target two-dimensional slice into the intra-cavity the segmentation process is performed in the cut sub-network, intra-luminal segmentation of carotid arteries on a target two-dimensional slice is determined.

inputting the region-of-interest image and the carotid wall segmentation mask into a region segmentation sub-network to perform plaque identification and region segmentation processing, and determining a plurality of region images corresponding to carotid walls; each region image comprises a corresponding plaque; and inputting the images of each region into a classification sub-network to classify the plaque in the images of each region, and determining the composition of the plaque corresponding to the images of each region.

In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of:

In one embodiment, the computer program when executed by the processor further performs the steps of:

In one embodiment, a computer program product is provided comprising a computer program which, when executed by a processor, performs the steps of:

It should be noted that, the data (including, but not limited to, data for analysis, data stored, data displayed, etc.) referred to in the present application are all data fully authorized by each party, and the collection, use, and processing of the relevant data are required to meet the relevant regulations.

Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the various embodiments provided herein may include at least one of relational databases and non-relational databases. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic units, quantum computing-based data processing logic units, etc., without being limited thereto.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The above examples only represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the present application. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application shall be subject to the appended claims.

Claims

1. A method of carotid wall segmentation, the method comprising:

dividing carotid artery walls of carotid arteries in the original medical image according to a preset dividing network, and determining carotid artery wall dividing masks corresponding to the original medical image;

wherein the carotid wall segmentation mask comprises segmented carotid walls; the segmentation network is obtained by training a plurality of sample images and labels corresponding to the sample images, wherein the labels comprise a reference carotid wall segmentation mask corresponding to the sample images and a reference signed distance graph, and the reference signed distance graph is used for representing structural features of carotid walls in the sample images;

The method further comprises the steps of:

inputting the original medical image into a preset positioning network, and performing positioning treatment on carotid artery in the original medical image to obtain an interested region image; wherein the carotid artery is included in the region of interest image, and the size of the region of interest image is smaller than the size of the original medical image;

correspondingly, the segmenting the carotid artery wall of the carotid artery in the original medical image according to a preset segmenting network, and determining the carotid artery wall segmenting mask corresponding to the original medical image comprises the following steps:

inputting the region of interest image into the segmentation network, carrying out segmentation processing on carotid artery walls of carotid arteries in the region of interest image, and determining the carotid artery wall segmentation mask.

2. The method of claim 1, wherein the structure of the codec of the split network is an asymmetric structure, and the number of filters in the encoder of the asymmetric structure is greater than the number of filters in the decoder.

3. The method of claim 1, wherein the training mode of the segmentation network comprises:

the initial segmentation network is trained according to each predicted carotid wall segmentation mask and a corresponding reference carotid wall segmentation mask, and according to each predicted signed distance map and a corresponding reference signed distance map, to determine the segmentation network.

4. The method of claim 1, wherein the localization network comprises a bifurcation detection sub-network and an intra-luminal segmentation network, the original medical image comprises a plurality of two-dimensional slices, the inputting the original medical image into a preset localization network, and the localization processing is performed on carotid arteries in the original medical image to obtain the region of interest image, comprising:

inputting a plurality of two-dimensional slices of the original medical image into the bifurcation detection sub-network to perform bifurcation position detection processing, and determining a target two-dimensional slice; the target two-dimensional slice is a slice at the carotid bifurcation occurrence position;

Inputting the target two-dimensional slice into the intra-cavity segmentation sub-network for segmentation processing, and determining intra-cavity segmentation results of carotid arteries on the target two-dimensional slice;

and obtaining the region of interest image according to the intra-cavity segmentation result.

5. The method according to claim 1, wherein the method further comprises:

and inputting the region-of-interest image and the carotid wall segmentation mask into a preset recognition network for recognition processing, and determining the composition corresponding to each region on the carotid wall.

6. The method according to claim 5, wherein the identification network includes a region segmentation sub-network and a classification sub-network, the inputting the region of interest image and the carotid wall segmentation mask into a preset identification network for identification processing, and determining the composition corresponding to each region on the carotid wall includes:

inputting the region-of-interest image and the carotid wall segmentation mask into the region segmentation sub-network to perform plaque identification and region segmentation processing, and determining a plurality of region images corresponding to the carotid wall; each regional image comprises a corresponding plaque;

And inputting the regional images into the classifying sub-network to classify the plaque in the regional images, and determining the composition of the plaque corresponding to the regional images.

7. A carotid wall segmentation device, the device comprising:

the segmentation module is used for carrying out segmentation processing on carotid artery walls of carotid arteries in the original medical image according to a preset segmentation network, and determining carotid artery wall segmentation masks corresponding to the original medical image; wherein the carotid wall segmentation mask comprises segmented carotid walls; the segmentation network is obtained by training a plurality of sample images and labels corresponding to the sample images, wherein the labels comprise a reference carotid wall segmentation mask corresponding to the sample images and a reference signed distance graph, and the reference signed distance graph is used for representing structural features of carotid walls in the sample images;

the method comprises the steps of dividing carotid artery walls of carotid arteries in an original medical image according to a preset dividing network, and before determining a carotid artery wall dividing mask corresponding to the original medical image, the method further comprises the steps of:

The positioning module is used for inputting the original medical image into a preset positioning network, and performing positioning treatment on carotid artery in the original medical image to obtain an image of the region of interest; wherein the carotid artery is included in the region of interest image, and the size of the region of interest image is smaller than the size of the original medical image;

the segmentation module is further configured to input the region of interest image into the segmentation network, segment a carotid wall of a carotid artery in the region of interest image, and determine the carotid wall segmentation mask.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 6 when the computer program is executed.

9. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 6.