CN116433586A - Mammary gland ultrasonic tomography image segmentation model establishment method and segmentation method - Google Patents

Abstract

The invention discloses a method for establishing a breast ultrasound tomographic image segmentation model and a segmentation method, belonging to the field of medical image segmentation, including: constructing a training set composed of breast ultrasound tomographic images with marked lesion areas; constructing an initial breast mammary gland based on a UNet model Ultrasonic tomographic image segmentation model; take L _total = L _E + α L _vntrast + β L _uncer as the training loss function, use the initial model constructed by training set training, and complete the establishment of the breast ultrasonic tomographic image segmentation model; wherein, L _total represents the overall loss ; L _E represents the segmentation error of the breast ultrasound tomographic image segmentation model; L _contrast represents the contrast loss function between the output intermediate result of the decoding structure and the labeling result, and L _uncer is the uncertainty loss between the output intermediate result of the decoding structure and the labeling result function, α and β represent weight coefficients; preferably, the model inserts a CrossFormer module and a multi-scale attention module in UNet. The invention can effectively improve the segmentation precision of the breast ultrasound tomographic image.

Description

A method for establishing a breast ultrasound tomographic image segmentation model and a segmentation method

technical field

The invention belongs to the field of medical image segmentation, and more specifically relates to a method for establishing a segmentation model of a breast ultrasound tomographic image and a segmentation method.

Background technique

Breast cancer is the second most deadly cancer in women after lung cancer, and regular breast screening can effectively reduce breast cancer mortality. Ultrasound tomography has the advantages of high sensitivity and standardized operation compared with traditional ultrasound, and its diagnostic results are not affected by the doctor's experience, so it has broad application prospects in the screening of breast tumors. Ultrasound tomography can be divided into two modes, reflection mode and transmission mode, in which reflection mode images have higher resolution than traditional ultrasound images. Transmission images can provide functional information of lesions, thereby further assisting doctors in diagnosis. Therefore, transmission modalities are often used for breast imaging. The ultrasonic tomography system uses step-by-step scanning to obtain images of different sections of breast tissue, and then reconstructs a three-dimensional image of breast tissue, making the display of lesions more intuitive.

The frequency of the probe used in the ultrasound tomography system is usually 2-3MHz, which is much lower than the frequency of more than 10MHz used by the traditional ultrasound probe, resulting in low image resolution. Therefore, it is challenging to identify lesions in the breast ultrasound tomography image. time consuming task. At present, computer-aided diagnosis is widely used in medicine. It is mainly used to assist doctors in diagnosis and formulate follow-up treatment plans, and improve the specificity and sensitivity of doctors' diagnosis. In computer-aided breast diagnosis, automatic medical image segmentation is the most critical step to improve the efficiency and accuracy of diagnosis. By segmenting the lesion from the ultrasonic tomographic image, it is beneficial for the doctor to determine the size and extent of the lesion, reconstruct the intuitive three-dimensional shape, and facilitate the doctor to make a correct diagnosis and formulate a treatment plan accurately.

The UNet model has good results in medical image segmentation applications, so it is also widely used in breast ultrasound tomographic image segmentation. UNet is a typical encoding-decoding structure. Its structure is shown in Figure 1. The convolutional network part on the left is the encoding structure, which is responsible for feature extraction. It is mainly composed of convolutional layers and downsampling layers. You can see the size of the feature map It keeps decreasing; on the right is the decoding structure, corresponding to the upsampling process, through long jump connections (concat method) with information of different convolutional layers, to restore the size close to the original image. In order to further improve the segmentation effect of breast ultrasound tomographic images, researchers improved the traditional UNet model and proposed some improved models, such as Attention-UNet model, UNet++ model, and ResUNet model.

After the above model is trained, it can effectively segment the lesion area in the breast ultrasound tomographic image. However, unlike other medical images, the breast ultrasound tomographic image has a complex background and a low contrast between the lesion and other tissues. stage, the upsampling operation will lose a lot of information, resulting in the problem of mis-segmentation still faced in practical applications. In addition, since the lesions are often small in breast ultrasound tomographic images, the existing models cannot accurately segment small lesions. Overall, the segmentation accuracy of existing breast ultrasound tomographic image segmentation methods needs to be further improved.

Contents of the invention

Aiming at the defects and improvement needs of the prior art, the present invention provides a method for establishing a breast ultrasound tomographic image segmentation model and a segmentation method, the purpose of which is to improve the segmentation accuracy of the breast ultrasound tomographic image.

In order to achieve the above object, according to one aspect of the present invention, a method for establishing a breast ultrasound tomographic image segmentation model is provided, including:

Construct a training set; in the training set, each piece of training data is a breast ultrasound tomographic image with a marked lesion area;

Construct an initial breast ultrasound tomographic image segmentation model based on the UNet model, which is used to segment the lesion area from the breast ultrasound tomographic image;

Taking L _total = L _E + α L _contrast + β L _uncer as the training loss function, using the training set to train the initial breast ultrasound tomographic image segmentation model, and completing the establishment of the breast ultrasound tomographic image segmentation model;

Among them, L _total represents the overall loss; L _E represents the segmentation error of the breast ultrasound tomographic image segmentation model; L _contrast represents the contrast loss function between the intermediate results output by the decoding structure and the labeling results in the breast ultrasound tomographic image segmentation model, and α represents its Weight coefficient; _Luncer is an uncertainty loss function, which is used to represent the difference between the intermediate result of the decoding structure output and the labeling result in the breast ultrasound tomographic image segmentation model, and β represents its weight coefficient; α≥0, β≥0, and α and β are not 0 at the same time.

further,

表示中间结果p_j的第i个通道；D_KL表示KL散度，C表示类别数，/>

表示分配给第j个中间结果中的像素p的权重。Among them, J represents the number of intermediate outputs of the model; L _CE () represents the cross-entropy loss; l represents the labeling result; _pj represents the intermediate result of the decoding structure output in the breast ultrasound tomographic image segmentation model,

Represents the i-th channel of the intermediate result p _j ; D _KL represents the KL divergence, C represents the number of categories, />

denotes the weight assigned to pixel p in the jth intermediate result.

further,

表示同一类别的特征，/>

表示不同类别的特征；τ表示温度系数。Among them, L represents the similarity between pixels; s _o represents the feature in the intermediate result, s _l represents the feature in the labeling result; m and n represent the feature category;

Represents features of the same class, />

Indicates the characteristics of different categories; τ indicates the temperature coefficient.

Further, the breast ultrasound tomographic image segmentation model further includes: a CrossFormer module inserted between the last layer of the encoding structure and the decoding structure in the UNet model.

Further, the breast ultrasound tomographic image segmentation model also includes: a multi-scale attention module inserted in the long skip connection between the encoding structure and the decoding structure in the UNet model;

The multi-scale attention module includes: hole space convolution pooling pyramid and feature enhancement module;

The hollow convolution pooling pyramid is used to extract multi-scale features from the features output by the corresponding layers in the encoding structure to obtain multi-scale features;

The feature enhancement module, which takes multi-scale features and decoded features output by corresponding layers in the decoding structure as input, is used to enhance task-related features and suppress feature-irrelevant features.

Further, the feature enhancement module includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a ReLU activation layer, a Sigmoid activation layer, a pixel addition layer, and a pixel multiplication layer;

The first convolutional layer is used to input multi-scale features and perform convolution operations;

The second convolutional layer is used to input decoding features and perform convolution operations;

The pixel addition layer is used to add the results output by the first convolution layer and the second convolution layer pixel by pixel;

The ReLU activation layer is used to activate the output of the pixel addition layer;

The third convolutional layer is used to perform convolution operations on the results output by the ReLU activation layer;

The Sigmoid activation layer is used to activate the output of the third convolutional layer;

The pixel multiplication layer is used to multiply the results output by the Sigmoid activation layer and the multi-scale features pixel by pixel.

Further, build a training set, including:

Obtaining an original data set composed of breast ultrasound tomographic images, and marking the lesion area in each of the breast ultrasound tomographic images, and obtaining a corresponding lesion area mask image;

Stitching n consecutive breast ultrasound tomographic images in the channel dimension, and forming a training set from the spliced breast ultrasound tomographic images and corresponding lesion area mask images;

Wherein, n is a positive integer greater than 1.

Further, n=3;

In addition, three consecutive breast ultrasound tomographic images are spliced in the channel dimension, including:

Extract the information of one channel from each breast ultrasound tomographic image to obtain the information of three channels;

The information of the three channels is spliced in the channel dimension.

According to another aspect of the present invention, there is provided a breast ultrasound tomographic image segmentation method, comprising:

Input the breast ultrasound tomographic image to be segmented into the breast ultrasound tomographic image segmentation model to obtain the lesion area segmentation result;

Wherein, the breast ultrasound tomographic image segmentation model is established by the breast ultrasound tomographic image segmentation model establishment method provided by the present invention.

According to another aspect of the present invention, a computer-readable storage medium is provided, including a stored computer program; when the computer program is executed by a processor, it controls the device where the computer-readable storage medium is located to execute the above-mentioned breast ultrasound tomographic image provided by the present invention A method for establishing a segmentation model, and/or the above-mentioned method for segmenting breast ultrasound tomographic images provided by the present invention.

Generally speaking, through the above technical solutions conceived by the present invention, the following beneficial effects can be obtained:

(1) The present invention introduces at least one of the comparison loss function and the uncertainty loss function between the intermediate results output by the decoding structure and the labeling results in the training loss function when the established breast ultrasound tomographic image segmentation model is trained item; the introduction of the contrast loss function enables the model to have the ability to distinguish pixels belonging to different categories (ie, lesion area or background), and more accurately classify pixels near the boundary, thereby alleviating the impact of low contrast of breast ultrasound tomographic images on segmentation accuracy ; The introduction of the uncertainty loss function minimizes the difference between the intermediate results output by the decoding structure and the labeling results, enabling the network to learn more discriminative and reliable knowledge in the early stage, and effectively reduce the upsampling in the decoding stage The loss of detail information caused by the operation enables the model to accurately segment the lesion area even in the case of low contrast. Generally speaking, the present invention can effectively improve the segmentation accuracy of breast ultrasound tomographic images by improving the model training loss function.

(2) The difference between the model output and the label is usually measured using cross-entropy loss and Dice loss, but in breast ultrasound tomographic image segmentation, the uncertainty loss function is only constructed based on cross-entropy loss, and the detailed information caused by upsampling The mitigation effect of the loss is limited; in the preferred solution of the present invention, the designed uncertainty loss function, on the basis of the cross-entropy loss, introduces a regular term based on KL divergence, which can minimize the upsampling in the decoding stage The loss of detail information caused by the operation further improves the segmentation accuracy of the model.

(3) In the preferred solution of the present invention, the designed comparison loss function maximizes the similarity of pixels belonging to the same category in the intermediate output and labeling results, and minimizes the similarity of pixels belonging to different categories, which can further improve model differentiation. Capabilities for different classes of pixels.

(4) breast ultrasound tomographic image segmentation task belongs to intensive prediction task, and local feature and global feature are all very important for this type of task; The CrossFormer module is inserted. Based on the rich local information extracted by the convolutional layer and the pooling layer in the encoding structure, the CrossFormer module extracts the global information, and by encoding the dependency between the global information and the local information, it can make small The segmentation of lesions is more precise.

(5) In the preferred solution of the present invention, a multi-scale attention module is inserted into the long skip connection of the UNet model, which is specifically composed of an atrous spatial convolution pooling pyramid and a feature enhancement module, in which the atrous spatial convolution The pooling pyramid enables the network to retain local detail information as much as possible while increasing the receptive field and capturing multi-scale information. The feature enhancement module further processes the multi-scale features extracted by the hollow space convolution pooling pyramid, enhancing and task The relevant features and suppress the features that are not related to the features, and finally make the model have higher segmentation accuracy for small lesions.

(6) In the process of ultrasonic tomography, multiple images will be generated for the same object; in the preferred solution of the present invention, when constructing a data set for model training, multiple continuous images are spliced on the channel dimension Finally, as a model input, the interlayer information of the ultrasonic tomographic image can be effectively used to further improve the segmentation accuracy. In a further preferred solution, three consecutive images are specifically selected to extract a channel and then spliced as the model input. On the one hand, it can avoid the spliced image from including too many background region features; on the other hand, the spliced image dimension and The dimensionality of the original image is consistent, which simplifies the processing of model input in the subsequent segmentation process.

Description of drawings

Figure 1 is a schematic diagram of the existing UNet model structure;

Fig. 2 is the ultrasound tomographic image of the breast and its corresponding mask image provided by the embodiment of the present invention; wherein, (a) is the ultrasound tomogram of the breast, and (b) is the mask image;

FIG. 3 is a schematic diagram of a breast ultrasound tomographic image segmentation model provided by an embodiment of the present invention;

Figure 4 is a schematic diagram of the existing CrossFormer module;

FIG. 5 is a schematic diagram of a multi-scale attention module provided by an embodiment of the present invention;

Fig. 6 is the segmentation result of the segmentation method provided by the embodiment of the invention and other segmentation methods based on the UNet model; wherein, (a) is the lesion area label of the breast ultrasound tomographic image, (b) is the segmentation result of the UNet model, (c) (d) is the segmentation result of the UNet++ model, (e) is the segmentation result of the ResUNet model, (f) is the segmentation result of the TransUNet model, and (g) is the segmentation result of this embodiment.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not constitute a conflict with each other.

In the present invention, the terms "first", "second" and the like (if any) in the present invention and drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

In order to improve the accuracy of breast ultrasound tomographic image segmentation, the present invention provides a method for establishing a breast ultrasound tomographic image segmentation model and a segmentation method. pixels, and/or, learn more discriminative and reliable knowledge at an early stage, and reduce the loss of detail information due to upsampling operations, enabling the model to accurately segment from low-contrast breast ultrasound tomographic images out of the lesion area. On this basis, the structure of the model is further improved, so that the model can better utilize the global information and local detail information in the image, and further improve the segmentation accuracy.

The following are examples.

Example 1:

A method for establishing a breast ultrasound tomographic image segmentation model, comprising:

Build a training set; in the training set, each piece of training data is a breast ultrasound tomographic image that has marked the lesion area; Figure 2 shows an example of training data, and (a) in Figure 2 is the breast ultrasound tomographic image collected, (b) is the mask image of the marked lesion area.

Optionally, in this embodiment, constructing the training set specifically includes the following steps:

(1) Use the ultrasound tomography system of 2-3MHz to collect breast ultrasound tomography images, collect 372 images in total, and divide them into training set, verification set and test set according to the ratio of 7:1:2;

(2) Mark the lesion area in a professional and the same way on the collected 372 images, and obtain the corresponding mask image as the label during model training;

(3) Carry out data enhancement to the mammary gland ultrasound tomographic images and corresponding labels in the training set through operations such as image flipping, rotation, and zooming, and expand the training set to 8 times the original training set, with a total of 2088 pieces;

Through data enhancement, the robustness of the model can be improved and overfitting can be avoided;

(4) Normalize the size of the training image after data enhancement to 512×512 to meet the requirements of the model for the input image size, and then stitch the three consecutive images in the channel dimension. The specific stitching method is: from The information of one channel is extracted from each image, and then the information of the extracted three channels is spliced in the channel dimension, and the obtained three-channel image is used as the input of the segmentation model, that is, the input image size is 512×512×3;

In the ultrasonic tomographic imaging process, multiple (usually 30) images will be generated for the same object. In this embodiment, the continuous multiple images are spliced and used as model input, which can effectively use the interlayer information; in the breast ultrasonic tomographic image The lesions are often small, and there is often a certain layer spacing between images of different layers. In addition, the original breast ultrasound tomographic image is a three-channel image, and the signals of each channel are the same. This embodiment specifically uses three consecutive ultrasound images One channel is extracted respectively and then spliced into a three-channel image as the model input. On the basis of using inter-layer information, it not only avoids avoiding the spliced image from including too many background region features, but also keeps the spliced image dimension and The dimensionality of the original image is consistent, so that in the actual segmentation process, the channel dimension of the original breast ultrasound tomographic image is consistent with the model input requirements, which simplifies the processing of the model input in the subsequent segmentation process.

It should be noted that the construction method of the above training set is only a preferred embodiment of the present invention, and should not be understood as the only limitation of the present invention. Other data sets of breast ultrasound tomographic images marked with lesion areas can be used in the present invention It is easy to understand that in the above steps, parameters such as the number of images and image size are only exemplary descriptions, and should not be understood as the only limitation to the present invention, and can be adjusted as needed in practical applications.

This embodiment further includes: constructing an initial breast ultrasound tomographic image segmentation model based on the UNet model, for segmenting lesion areas from the breast ultrasound tomographic image.

Considering the original UNet model and the improved model based on the UNet model, since the detailed information will be lost in the up-sampling process of the decoding stage, the small lesions in the breast ultrasound tomographic image cannot be well segmented. In this embodiment, the UNet model A new breast ultrasound tomographic image segmentation model was obtained based on the improvement of the model. The selected UNet model is specifically the model shown in Figure 1. The related improvements include: the last layer of the encoding structure and the decoding structure in the UNet model Insert the CrossFormer module between them, and insert the multi-scale attention module (MFEModule) in the long skip connection of the UNet model.

The structure of the improved segmentation model is shown in Figure 3, which can be divided into twelve layers: the first layer is composed of two convolutional layers, and the second to fourth layers are composed of a 2×2 pooling layer and two A module composed of a convolutional layer. The convolutional layer consists of a 3×3 convolution, a batch normalization module, and a ReLU activation function; the output feature map size of the first layer is 512×512×64 (512 represents the feature The width and height of the map is 512, 64 represents the number of channels of the feature map), the number of channels of the output feature map of the second layer to the fourth layer is twice that of the output feature map of the previous layer, and the width and height are 1/2 of the previous layer , the size of the output feature map of the fourth layer is 64×64×512, the fifth layer consists of a pooling layer and a convolutional layer, the number of channels of the output feature map of the fifth layer is the same as that of the fourth layer, but the width and height are still Halved, that is, the output feature map size of the fifth layer is 32×32×512.

Referring to Fig. 3, in the breast ultrasound tomographic image segmentation model, the sixth layer is the inserted CrossFormer module; the CrossFormer module is composed of two continuous CrossFormerBlocks, as shown in Fig. 4 . One of the CrossFormerBlocks consists of a cross-scale embedding layer (CEL), two layers of normalization (Layer Norm), a relative position encoding layer (RPB), a short distance attention layer (SDA) and a multi-layer perceptron layer ( MLP). Another CrossFormerBlock contains two layers of normalization (Layer Norm), a relative position encoding layer (RPB), a long-distance attention layer (LDA) and a multi-layer perceptron layer (MLP). The output feature map size of the sixth layer is the same as that of the fifth layer, that is, still 32×32×512. The seventh layer is composed of a convolutional layer and an upsampling layer. The number of channels of the output feature map of the seventh layer is the same as that of the sixth layer, and the width and height are twice that of the sixth layer, that is, the output feature map size of the seventh layer is 64 ×64×512.

In this embodiment, a CrossFormer module is inserted between the last layer of the encoding structure and the decoding structure in the UNet model. On the basis of extracting rich local information from the convolutional layer and pooling layer in the encoding structure, the CrossFormer module extracts global information. , and by encoding the dependency between global information and local information, it can make the segmentation of small lesions more accurate.

Referring to Figure 3, in the breast ultrasound tomographic image segmentation model, the eighth to tenth layers are composed of two convolutional layers and an upsampling layer, and the number of channels of the output feature map of each layer becomes the number of channels of the input feature map 1/2, the width and height become twice the width and height of the input feature map. The width and height of the output feature map of the tenth layer are restored to the size of the input image, which is 512×512. The eleventh layer is composed of two convolutional layers, the twelfth layer is composed of a 1×1 convolutional layer, the number of output channels is the number of categories, and the output feature map size is 512×512×2.

The CrossFormer module is used as a distinction, the structure on the left corresponds to the encoding structure, and the structure on the right corresponds to the decoding structure.

Referring to Figure 3, in the breast ultrasound tomographic image segmentation model, the output of the first, second, third, and fourth layers in the encoding structure is inserted into the multi-scale attention module (MFEModule) and the tenth, ninth, eighth, and seventh layers in the decoding structure Layer outputs are concatenated using a long skip connection. The size of the output feature map of the first layer and the tenth layer is 512×512×64, the size of the output feature map of the second layer and the ninth layer is 256×256×128, and the output feature map of the third layer and the eighth layer The size of the output feature maps of the fourth layer and the seventh layer are both 64×64×512. That is: the output feature maps of the first layer and the tenth layer are concatenated to 512×512×128 and then the convolution operation is performed, and the output feature maps of the second layer and the ninth layer are concatenated to be 256×256×256 and then performed Convolution operation, the output feature maps of the third layer and the eighth layer are concatenated into 128×128×512 and then the convolution operation is performed, and the output feature maps of the fourth and seventh layers are concatenated into 64×64×1024 Then perform the convolution operation.

In this embodiment, the structure of the multi-scale attention module (MFEModule) is shown in Figure 5, which includes two branches, one of which is an atrous space convolution pooling pyramid (ASPPModule), which is used for the corresponding layer in the coding structure The output features are multi-scale feature extracted to obtain multi-scale features; ASPPModule contains a 1×1 convolution, three 3×3 hole space convolutions with expansion coefficients of 6, 12 and 18, and a 1×1 pool The structure of ASPP enables the network to preserve local details as much as possible while increasing the receptive field and capturing multi-scale information; the other branch is a feature enhancement module, which contains three 1×1 convolutions (that is, the first Convolutional layer, second convolutional layer and third convolutional layer), a ReLU activation layer, a Sigmoid activation layer, a pixel addition layer and a pixel multiplication layer, as shown in Figure 5:

The first convolutional layer is used to input multi-scale features and perform convolution operations;

The second convolutional layer is used to input decoding features and perform convolution operations;

The pixel addition layer is used to add the results output by the first convolution layer and the second convolution layer pixel by pixel;

The ReLU activation layer is used to activate the output of the pixel addition layer;

The third convolutional layer is used to perform convolution operations on the results output by the ReLU activation layer;

The Sigmoid activation layer is used to activate the output of the third convolutional layer;

The pixel multiplication layer is used to multiply the results output by the Sigmoid activation layer and the multi-scale features pixel by pixel; the multiplied result is used as the input of the next layer.

Through the feature enhancement module to further process the multi-scale features extracted by ASPP, the features related to the task can be enhanced and the features not related to the feature can be suppressed; it should be noted that the structural description of the feature enhancement module here is the best The solution should not be construed as the only limitation to the present invention, and other modules based on the attention mechanism that can enhance the features related to the task and suppress the features that are not related to the feature can also be used in the present invention.

In general, this embodiment can make the model have higher segmentation accuracy for small lesions by inserting the above feature enhancement module into the long skip connection of the UNet model.

It should be noted that the above model is only the preferred model of the present invention, and should not be understood as the only limitation of the present invention; in other embodiments of the present invention, when the segmentation accuracy meets the application requirements, it can only be used in the UNet model Insert the CrossFormer module at the corresponding position, or only insert the multi-scale attention module, or directly use the existing UNet model or Attention-UNet model, UNet++ model, ResUNet model and other improved models.

It is easy to understand that when the structure of the selected UNet model changes, the number of layers of the improved breast ultrasound tomographic image segmentation model and the parameters in each layer may also change accordingly. The above description of the model parameters is only a summary This exemplary description should not be construed as the only limitation of the present invention.

The above initial breast ultrasound tomographic image segmentation model needs to be trained before it can complete the actual segmentation task; the difference between the model output and the label is usually measured by cross-entropy loss and Dice loss. The contrast between the background and the background is low, and in the UNet model and its improved model, in the decoding stage, the upsampling operation will lose a lot of information, and the model trained by the existing training method will still face the problem of mis-segmentation. To solve this problem, This embodiment introduces the comparison loss function and uncertainty loss function between the intermediate results output by the decoding structure (ie O ₂ , O ₃ , O ₄ in Figure 3 ) and the marked results in the training loss function of the model, improving The final loss function is:

L _total ＝L _E +αL _contrast +βL _uncer

Among them, L _total represents the overall loss; L _E represents the segmentation error of the breast ultrasound tomographic image segmentation model; L _contrast represents the contrast loss function between the intermediate results output by the decoding structure and the labeling results in the breast ultrasound tomographic image segmentation model, and α represents its Weight coefficient; _Luncer is an uncertainty loss function, which is used to represent the difference between the intermediate result of the decoding structure output and the labeling result in the breast ultrasound tomographic image segmentation model, and β represents its weight coefficient; α≥0, β≥0, and α and β are different from 0 at the same time; the specific values of α and β can be flexibly adjusted according to the actual segmentation effect. Optionally, in this embodiment, α and β are set to 0.2 and 0.3 respectively.

In the above loss function designed in this embodiment, by introducing the uncertainty loss function, the difference between the intermediate result output by the decoding structure and the labeling result is minimized, so that the network can learn more discriminative and reliable knowledge, and effectively reduce the loss of detail information caused by the upsampling operation in the decoding stage, so that the model can accurately segment the lesion area even in the case of low contrast. Experiments have found that constructing an uncertainty loss function based only on cross-entropy loss has a limited mitigation effect on the loss of detail information caused by upsampling. In order to minimize the loss of detail information caused by upsampling operations in the decoding stage, this embodiment uses cross On the basis of entropy loss, a regular term based on KL divergence is introduced. Finally, the expression of the uncertainty loss function is specifically:

表示中间结果p_j的第i个通道；D_KL表示KL散度，C表示类别数，/>

表示分配给第j个中间结果中的像素p的权重。Among them, J represents the number of intermediate outputs of the model; L _CE () represents the cross-entropy loss; l represents the labeling result; _pj represents the intermediate result of the decoding structure output in the breast ultrasound tomographic image segmentation model,

Represents the i-th channel of the intermediate result p _j ; D _KL represents the KL divergence, C represents the number of categories, />

denotes the weight assigned to pixel p in the jth intermediate result.

The introduction of the contrast loss function enables the model to have the ability to distinguish pixels belonging to different categories (ie, lesion area or background), and more accurately classify pixels near the boundary, thereby alleviating the impact of low contrast of breast ultrasound tomographic images on segmentation accuracy; In the embodiment, the expression of the comparison loss function is specific:

表示标注结果中的第m类特征，/>

和/>

分别表示中间结果s_o中的第m、n类特征；/>

表示同一类别的特征，/>

表示不同类别的特征；τ表示温度系数，可选地，本实施例中，其值为2；Among them, L represents the similarity between pixels; s _o represents the feature in the intermediate result, s _l represents the feature in the labeling result; m and n represent the feature category,

Indicates the mth class feature in the labeling result, />

and />

Respectively represent the m and nth class features in the intermediate result s _o ; />

Represents features of the same class, />

Represents the characteristics of different categories; τ represents the temperature coefficient, optionally, in this embodiment, its value is 2;

Optionally, optionally, in this embodiment, cosine similarity is specifically used for measurement, and the corresponding expression is

It should be noted that the cosine similarity is only an optional feature similarity measurement method, and should not be understood as the only limitation to the present invention. Other measurement methods, such as Pearson correlation coefficient, Mahalanobis distance, Euclidean distance, etc., can also be used in the present invention.

Based on the comparison loss function, the similarity of pixels belonging to the same category in the intermediate output and labeling results is maximized, and the similarity of pixels belonging to different categories is minimized, which can further improve the ability of the model to distinguish pixels of different categories.

Optionally, in the above loss function, the segmentation error _LE of the breast ultrasound tomographic image segmentation model, that is, the error between the lesion region segmentation result O ₁ and the label, still uses the cross-entropy loss function L _CE and the Dice loss function L _Dice express.

Finally, the overall loss function expression is as follows:

L _CE ＝-[ylogl+(1-y)log(1-l)]

Wherein, y represents the segmentation result output by the breast ultrasound tomographic image segmentation model, and l represents the labeling result;

Based on the above analysis, after the data set and the initial breast ultrasound tomographic image segmentation model are constructed, this embodiment further includes:

Taking L _total = L _E + αL _contrast + βL _uncer as the training loss function, the initial breast ultrasound tomographic image segmentation model is trained using the training set, and the establishment of the breast ultrasound tomographic image segmentation model is completed.

It should be noted that, in some other embodiments of the present invention, one of the hyperparameters α and β may be 0.

It is easy to understand that after the model training is completed, in order to ensure that the segmentation results meet the requirements, the trained model will be further tested and verified using the test set and verification set. During the test, the Dice coefficient and IoU can be used as evaluation indicators for Segmentation results are evaluated.

Example 2:

A breast ultrasound tomographic image segmentation method, comprising:

Input the breast ultrasound tomographic image to be segmented into the breast ultrasound tomographic image segmentation model to obtain the lesion area segmentation result;

Wherein, the breast ultrasound tomographic image segmentation model is established by the breast ultrasound tomographic image segmentation model establishment method provided by the present invention.

Example 3:

A computer-readable storage medium, including a stored computer program; when the computer program is executed by a processor, it controls the device where the computer-readable storage medium is located to execute the method for establishing a breast ultrasound tomographic image segmentation model provided by the present invention, and/or, the present invention The above-mentioned breast ultrasound tomographic image segmentation method provided by the invention.

The beneficial effects obtained by the present invention will be further analyzed and described below in combination with the segmentation effects of different models.

For the breast ultrasound tomographic image segmentation model shown in Figure 3, use Pytorch as the deep learning framework, the experimental environment Nvidia GeForce RTX 2080Ti (11GB) GPU, adopt the UNet network structure provided by the above-mentioned embodiment 1 (ie the breast ultrasound shown in Figure 3 Tomographic image segmentation model) as the segmentation model, C0=2, C1=64, C2=128, C3=256, C4=512 in the model. The input image size is 512×512×3. The stochastic gradient descent algorithm (weight decay=0.0001, momentum=0.9) is used as the optimizer, the initial learning rate is set to 0.01, and the learning rate is reduced to 0.1 times the original value after every 100 epochs, and a total of 300 epochs are trained, and the batch size is set for 4. A total of 327 data sets were used. The data set was divided into training set, verification set and test set according to the ratio of 7:1:2, and the training set was expanded to 8 times of the original. The loss function uses cross-entropy function, Dice loss function, contrast loss function and uncertainty loss function, Dice coefficient and intersection-over-union ratio (IoU) as evaluation indicators.

Using the existing UNet model, Attention-UNet model, UNet++ model, ResUNet model, and TransUNet model as comparison models, different models are used to segment breast ultrasound tomographic images, and the results are shown in Figure 6. In Figure 6, (a) is the lesion area label of multiple breast ultrasound tomographic images, (b) is the segmentation result of the UNet model, (c) is the segmentation result of the Attention-UNet model, (d) is the segmentation result of the UNet++ model, ( e) is the segmentation result of the ResUNet model, (f) is the segmentation result of the TransUNet model (g) the segmentation result of this embodiment. According to the results shown in Fig. 6, it can be seen that the boundary segmented by the breast ultrasound tomographic image segmentation model based on the present invention is closer to the real boundary, and the situation of wrong segmentation is relatively less. For the convenience of description, the model established in the above-mentioned embodiment 1 is abbreviated as "MFE-DSCrossUNet" below, and the Dice coefficient and IoU coefficient of each model segmentation result are shown in Table 1.

Table 1 Dice coefficient and IoU coefficient of different model segmentation results

According to the evaluation indicators shown in Table 1, it can be seen that the Dice coefficient of the segmentation result of the present invention reaches 0.8883, and the IoU coefficient reaches 0.7990. Compared with the segmentation results of UNet, the Dice coefficient is improved by 3.68%, and the IoU coefficient is improved by 5.76%. It fully proves the validity and superiority of the breast ultrasound tomographic image segmentation model established in the present invention.

It is easy for those skilled in the art to understand that the above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention, All should be included within the protection scope of the present invention.

Claims (10)

1. A breast ultrasound tomographic image segmentation model building method, is characterized in that, comprising: Constructing a training set; in the training set, each piece of training data is a breast ultrasound tomographic image that has marked a lesion area; Construct an initial breast ultrasound tomographic image segmentation model based on the UNet model, which is used to segment the lesion area from the breast ultrasound tomographic image; Taking L _total = _LE +αL _contrast + _βLuncer as the training loss function, using the training set to train the initial breast ultrasound tomographic image segmentation model, and completing the establishment of the breast ultrasound tomographic image segmentation model; Among them, L _total represents the overall loss; L _E represents the segmentation error of the breast ultrasound tomographic image segmentation model; L _contrast represents the contrast loss function between the intermediate results output by the decoding structure and the labeling results in the breast ultrasound tomographic image segmentation model, and α represents its Weight coefficient; _Luncer is an uncertainty loss function, which is used to represent the difference between the intermediate result of the decoding structure output and the labeling result in the breast ultrasound tomographic image segmentation model, and β represents its weight coefficient; α≥0, β≥0, and α and β are not 0 at the same time. 2. the breast ultrasound tomographic image segmentation model establishment method as claimed in claim 1, is characterized in that,

Among them, J represents the number of intermediate outputs of the model; L _CE () represents the cross-entropy loss; l represents the labeling result; _pj represents the intermediate result of the decoding structure output in the breast ultrasound tomographic image segmentation model,

Represents the i-th channel of the intermediate result p _j ; D _KL represents the KL divergence, C represents the number of categories, />

denotes the weight assigned to pixel p in the jth intermediate result. 3. the breast ultrasound tomographic image segmentation model building method as claimed in claim 1 or 2, is characterized in that,

Among them, L represents the similarity between pixels; s _o represents the feature in the intermediate result, s _l represents the feature in the labeling result; m and n represent the feature category,

Represents features of the same class, />

Indicates the characteristics of different categories; τ indicates the temperature coefficient. 4. the breast ultrasound tomographic image segmentation model establishment method as claimed in claim 1, is characterized in that, described breast ultrasound tomographic image segmentation model also comprises: insert between the last layer of encoding structure and decoding structure in UNet model CrossFormer module. 5. the breast ultrasound tomographic image segmentation model establishment method as claimed in claim 1 or 4, is characterized in that, described breast ultrasound tomographic image segmentation model also comprises: insert in the long skip connection between encoding structure and decoding structure in UNet model The multi-scale attention module of ; The multi-scale attention module includes: a hollow space convolution pooling pyramid and a feature enhancement module; The hollow convolution pooling pyramid is used to perform multi-scale feature extraction on the features output by the corresponding layer in the coding structure to obtain multi-scale features; The feature enhancement module takes the multi-scale features and the decoded features output by the corresponding layer in the decoding structure as input, and is used to enhance the features related to the task and suppress the features not related to the feature. 6. the breast ultrasound tomographic image segmentation model establishment method as claimed in claim 5, is characterized in that, described feature enhancement module comprises the first convolutional layer, the second convolutional layer, the 3rd convolutional layer, ReLU activation layer, Sigmoid activation layer, pixel addition layer and pixel multiplication layer; The first convolutional layer is used to input the multi-scale features and perform a convolution operation; The second convolutional layer is used to input the decoding feature and perform a convolution operation; The pixel addition layer is used to add the results output by the first convolution layer and the second convolution layer pixel by pixel; The ReLU activation layer is used to activate the result output by the pixel addition layer; The third convolutional layer is used to perform a convolution operation on the result output by the ReLU activation layer; The Sigmoid activation layer is used to activate the result output by the third convolutional layer; The pixel multiplication layer is used to multiply the result output by the sigmoid activation layer and the multi-scale features pixel by pixel. 7. the breast ultrasound tomographic image segmentation model establishment method as claimed in claim 1, is characterized in that, constructs training set, comprises: Obtaining an original data set composed of breast ultrasound tomographic images, and marking the lesion area in each of the breast ultrasound tomographic images, and obtaining a corresponding lesion area mask image; Stitching n consecutive breast ultrasound tomographic images in the channel dimension, and forming the training set from the spliced breast ultrasound tomographic images and corresponding lesion area mask images; Wherein, n is a positive integer greater than 1. 8. the breast ultrasound tomographic image segmentation model establishment method as claimed in claim 7, is characterized in that, n=3; In addition, three consecutive breast ultrasound tomographic images are spliced in the channel dimension, including: Extract the information of one channel from each breast ultrasound tomographic image to obtain the information of three channels; The information of the three channels is spliced in the channel dimension. 9. A breast ultrasound tomographic image segmentation method, characterized in that, comprising: Input the breast ultrasound tomographic image to be segmented into the breast ultrasound tomographic image segmentation model to obtain the lesion area segmentation result; Wherein, the breast ultrasound tomographic image segmentation model is established by the method for establishing a breast ultrasound tomographic image segmentation model according to any one of claims 1-8. 10. A computer-readable storage medium, characterized by comprising a stored computer program; when the computer program is executed by a processor, it controls the device where the computer-readable storage medium is located to execute any one of claims 1-8. The method for establishing a breast ultrasound tomographic image segmentation model described above, and/or the breast ultrasound tomographic image segmentation method described in claim 9.

Images