WO2020077961A1 - Image-based breast lesion identification method and device - Google Patents

Abstract

An image-based breast lesion identification method and device, which relate to the field of machine learning technology. The method includes: acquiring a breast image (201), inputting the breast image into a feature extraction module to acquire feature images of different sizes of the breast image (202); with respect to any one of the feature images of different sizes of the breast image, determining a breast lesion identification frame (203) from that feature image; according to the breast lesion identification frame determined from each feature image, determining the breast lesion of the breast image ( 204).

Description

Method and device for identifying breast imaging lesions

This application requires the priority of the Chinese patent application filed on October 16, 2018 with the Chinese Patent Office, the application number is 201811201699.2, and the application name is "a method and device for breast imaging lesion recognition", the entire content of which is incorporated by reference in In this application.

Technical field

Embodiments of the present invention relate to the technical field of machine learning, and in particular, to a method and device for identifying breast imaging lesions.

Background technique

At present, breast imaging can use low-dose X-rays to examine human breasts. It can detect various breast tumors, cysts and other lesions, which helps to detect breast cancer early and reduce its mortality. Breast imaging is an effective detection method that can be used to diagnose a variety of female breast-related diseases. Of course, the most important use is breast cancer, especially early breast cancer screening. Therefore, if you can effectively detect the early manifestations of various breast cancers on the breast image, it will be of great help to the doctor.

After the patient takes the breast image, the doctor judges the lesion in the breast image through personal experience. This method is inefficient and subjective.

Summary of the invention

Embodiments of the present invention provide a method and device for identifying breast imaging lesions, which are used to solve the problem of low efficiency of the method for judging breast lesions in breast imaging based on doctor experience in the prior art.

Embodiments of the present invention provide a method for identifying breast imaging lesions, including:

Get breast images;

The breast image is input into a feature extraction module to obtain feature images of different sizes of the breast image; the feature extraction module includes N convolution modules; the N convolution modules are down-sampling convolution blocks and / or Or up-sampling convolution block; the size of the feature image extracted by each down-sampling convolution block or up-sampling convolution block is different, each of the N convolution modules includes a first convolution layer, The second convolutional layer; the number of feature images output by the first convolution layer is less than the number of feature images input by the first convolution layer; the number of feature images output by the second convolution layer Is greater than the number of feature images output by the first convolutional layer; N is greater than 0;

For any one of the feature images of different sizes of the breast image, determine the breast lesion identification frame from the feature image;

According to the breast lesion identification frame identified from each feature image, the breast lesion of the breast image is determined.

A possible implementation manner, the acquiring feature images of different sizes of the breast image includes:

Successively extracting the first feature image of the breast image through N / 2 down-sampling convolution blocks through the breast image;

The first feature image output from the N / 2th down-sampling convolution block is used to sequentially extract N / 2 second feature images of the mammary gland image through the N / 2 up-sampling convolution block, each up-sampling convolution block The sizes of the extracted second feature images are different;

After merging the first feature image and the second feature image of the same size, N feature images of different sizes of the breast images are determined.

A possible implementation manner, before the feature processing module, further includes a feature preprocessing module; before the input of the breast image to the feature extraction module, further includes:

Input the breast image into the feature preprocessing module, the feature preprocessing module includes a convolution layer, a BN layer, a Relu layer and a pooling layer; the convolution kernel of the feature preprocessing module The size is larger than the size of the convolution kernel in the N convolution modules;

Alternatively, the feature pre-processing module includes multiple consecutive convolutional layers, a BN layer, a Relu layer, and a pooling layer; the size of the convolution kernel of the feature pre-processing module and the N convolution modules The size of the largest convolution kernel is equal.

In a possible implementation manner, before inputting the breast image to the feature extraction module, the method further includes:

Obtain the original file of the breast image;

Selecting at least one set of window width and window levels from the original file of the breast image, and acquiring a breast image in a picture format corresponding to the at least one set of window width and window levels;

The mammary gland image according to the picture format corresponding to the at least one set of window width and window level is used as the mammary gland image input to the feature extraction module.

A possible implementation manner, the mammary gland image includes mammary gland images with different projection positions of different breasts; the inputting the mammary gland image to a feature extraction module includes:

Taking the breast image of the other breast in the same projection position of the breast image as the reference image of the breast image, and inputting it to the feature extraction module to obtain a reference feature image;

Any one of the feature images of different sizes of the breast image, the breast lesion identification frame is determined from the feature image; including:

Determining a first breast lesion recognition frame in the feature image and a second breast lesion recognition frame in the reference feature image;

If it is determined that the positions and / or sizes of the first breast lesion identification frame and the second breast lesion identification frame are the same, the first breast lesion identification frame is deleted.

An embodiment of the present invention provides a device for identifying breast imaging lesions, including:

Acquisition unit for acquiring mammary gland image;

A processing unit for inputting the breast image into a feature extraction module to obtain feature images of different sizes of the breast image; the feature extraction module includes N convolution modules; the N convolution modules are for downsampling Convolution block or up-sampling convolution block; the size of the feature image extracted by each down-sampling convolution block or up-sampling convolution block is different, and each of the N convolution modules includes a first volume Multilayer, second convolutional layer; the number of feature images output by the first convolutional layer is less than the number of feature images input by the first convolutional layer; the feature images output by the second convolutional layer The number of is greater than the number of feature images output by the first convolutional layer; N is greater than 0; for any one of the feature images of different sizes of the breast image, the breast is determined from the feature image Lesion recognition frame; according to the breast lesion recognition frame determined from each characteristic image, determine the breast lesion of the breast image.

In a possible implementation manner, the processing unit is specifically used to:

The breast image is sequentially passed through N / 2 down-sampling convolution blocks to extract N / 2 first feature images of the breast image; the first feature images output from the N / 2 down-sampling convolution block are sequentially passed N / 2 up-sampling convolution blocks extract N / 2 second feature images of the mammography image, the size of the second feature images extracted by each up-sampling convolution block are different; the first feature images of the same size After merging with the second feature image, N feature images of different sizes of the breast images are determined.

A possible implementation manner, the mammary gland image includes mammary gland images with different projection positions of different breasts; the processing unit is specifically used for:

Take the breast image of the other breast of the same projection position of the breast image as the reference image of the breast image, and input it to the feature extraction module to obtain a reference feature image; determine the first breast in the feature image The lesion identification frame and the second breast lesion identification frame in the reference feature image; if it is determined that the positions and / or sizes of the first breast lesion identification frame and the second breast lesion identification frame are the same, delete the The first breast lesion identification frame.

On the other hand, an embodiment of the present invention provides a computer device including at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the program is executed by the processing unit, The processing unit executes the steps of the method for identifying breast imaging lesions.

In still another aspect, an embodiment of the present invention provides a computer-readable storage medium that stores a computer program executable by a computer device, and when the program runs on the computer device, causes the computer device to execute the mammary gland Steps of the method of image lesion identification.

In still another aspect, an embodiment of the present invention further provides a computer program product, the computer program product includes a computer program stored on a computer-readable storage medium, the computer program includes program instructions, and when the program instructions are When the device is executed, the computer device is caused to perform the steps of the method for identifying breast imaging lesions.

In the embodiment of the present invention, since the feature image of the mammary gland image is extracted and the mammary gland in each feature image is identified, the lesion of the mammary gland can be quickly identified, and the efficiency of identifying the breast lesion is improved. In addition, in the feature extraction module, the number of channels output by the first convolution layer is reduced, and the number of channels output by the second convolution layer is increased, so that the effective information in the image is effectively retained during the convolution process. While reducing the amount of parameters, the effectiveness of feature image extraction is improved, thereby improving the accuracy of breast lesion recognition in breast images.

BRIEF DESCRIPTION

In order to more clearly explain the technical solutions in the embodiments of the present invention, the drawings required in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. Those of ordinary skill in the art can obtain other drawings based on these drawings without paying any creative labor.

FIG. 1a is a schematic diagram of a breast image provided by an embodiment of the present invention;

1b is a schematic diagram of a breast image provided by an embodiment of the present invention;

1c is a schematic diagram of a breast image provided by an embodiment of the present invention;

1d is a schematic diagram of a breast image provided by an embodiment of the present invention;

2 is a schematic flowchart of a method for identifying a breast imaging lesion according to an embodiment of the present invention;

3a is a schematic structural diagram of a feature extraction module provided by an embodiment of the present invention;

3b is a schematic structural diagram of a feature extraction module provided by an embodiment of the present invention;

3c is a schematic structural diagram of a feature extraction module provided by an embodiment of the present invention;

FIG. 3 is a schematic flowchart of a breast imaging lesion recognition provided by an embodiment of the present invention;

4 is a schematic flowchart of a breast imaging lesion recognition provided by an embodiment of the present invention;

5 is a schematic structural diagram of a device for identifying breast imaging lesions according to an embodiment of the present invention;

6 is a schematic structural diagram of a computer device according to an embodiment of the present invention.

detailed description

In order to make the objectives, technical solutions and beneficial effects of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not intended to limit the present invention.

In the embodiment of the present invention, the breast X-ray image is taken as an example for an exemplary description, and other images will not be repeated here. Breast X-ray images can be used to examine the breasts of humans (mainly women) using low-dose (about 0.7 mSv) X-rays. It can detect various breast tumors, cysts and other lesions, which helps to detect breast cancer early. And reduce its mortality. Some countries encourage older women (generally over 45 years old) to perform mammography regularly (with intervals ranging from one year to five years) to screen for early breast cancer. The mammary gland image generally includes four X-ray images, which are four mammary gland images of the two projection positions of the two breasts (head and tail CC, medial and lateral oblique MLO), as shown in Figure 1a, Figure 1b, Figure 1c, 1d.

The prior art often only detects a single type of lesions such as calcifications or masses, and cannot simultaneously detect multiple lesions, and the application range is narrow. At the same time, for many types of lesions such as calcifications, masses, asymmetry, and structural distortion, the accuracy of detection is poor and cannot meet the application requirements.

In response to the above problems, an embodiment of the present invention provides a method for identifying breast imaging lesions, as shown in FIG. 2, including:

Step 201: Obtain a breast image;

Step 202: Input the mammary gland image into a feature extraction module to obtain feature images of different sizes of the mammary gland image;

Wherein, the feature extraction module includes N convolution modules; the N convolution modules are down-sampling convolution blocks and / or up-sampling convolution blocks; each down-sampling convolution block or up-sampling convolution block extraction The size of the feature images of the two are different. Each of the N convolution modules includes a first convolution layer and a second convolution layer; the number of feature images output by the first convolution layer is less than The number of feature images input by the first convolution layer; the number of feature images output by the second convolution layer is greater than the number of feature images output by the first convolution layer; N is greater than 0;

For example, the feature extraction module may include three down-sampling convolution blocks. Each convolution module may include a first convolution layer and a second convolution layer. The first convolution layer includes a convolution layer, a normalization (BN) layer connected to the convolution layer, and a connection to the BN layer The activation function layer of Fig. 3a includes a first convolution layer and a second convolution layer.

In order to increase the depth of the feature extraction module, as shown in FIG. 3b, a possible implementation manner, the step of the feature image passing through the convolution module may include:

Step 1: input the feature image input by the convolution module to the first convolution layer to obtain the first feature image; the convolution kernel of the first convolution layer may be N1 * m * m * N2; N1 is Describe the number of channels of the feature image input by the convolution module, N2 is the number of channels of the first feature image; N1> N2;

Step 2: Input the first feature image into the second convolution layer to obtain the second feature image; the convolution kernel of the first convolution layer may be N2 * m * m * N3; N3 is the channel of the second feature image Number; N3> N2;

Step 3: After combining the feature image input by the convolution module and the second feature image, it is determined as the feature image output by the convolution module.

In a specific embodiment, the number of feature images output by the second convolution layer may be equal to the number of feature images input by the first convolution layer. That is, N1 = N2.

The method for determining the feature image corresponding to the breast image described above is only one possible implementation manner. In other possible implementation manners, the feature image corresponding to the breast image may also be determined by other methods, which is not specifically limited.

It should be noted that the activation function in the embodiment of the present invention may be multiple types of activation functions, for example, it may be a linear rectification function (Rectified Linear Unit, ReLU), which is not specifically limited;

Since the input image in the embodiment of the present invention is a two-dimensional image, the feature extraction module in the embodiment of the present invention may be a feature extraction module in a (2Dimensions, 2D) convolutional neural network. Accordingly, the first convolution layer The size of the convolution kernel can be m * m, and the size of the second convolution layer can be n * n; m and n can be the same or different, which is not limited here; where, m, n is greater than or An integer equal to 1. The number of feature images output by the first convolution layer is less than the number of feature images input by the first convolution layer; the number of feature images output by the second convolution layer is greater than the first convolution layer The number of output feature images.

Further, to optimize the feature extraction module, a possible implementation manner, as shown in FIG. 3c, further includes a third convolution layer between the first convolution layer and the second convolution layer; The feature image input by the three convolution layers is the image output by the first convolution layer, and the feature image output by the third convolution layer is the image input by the second convolution layer.

The size of the convolution kernel of the third convolutional layer may be k * k, and k may be the same as m or n, or may be different, which is not limited herein.

In a specific embodiment, the size of the convolution kernel of the first convolution layer is 3 * 3; the size of the convolution kernel of the second convolution layer is 3 * 3; the third convolution layer The size of the convolution kernel is 1 * 1.

Through the above-mentioned setting method of the convolution kernel, the perception field of feature extraction can be effectively improved, which is beneficial to improve the accuracy of breast lesion recognition.

The feature images of different sizes may be feature images of different pixels, for example, the feature image with pixels 500 × 500 and the feature image with pixels 1000 × 1000 are feature images with different sizes.

Optionally, a pre-trained breast lesion detection model is used to extract feature images of different sizes of breast images. The model is determined after training a plurality of labeled breast images using a 2D convolutional neural network.

Optionally, before extracting feature images of different sizes of the mammary gland image, the image is scaled to a specific size so that the scale of the pixels in each direction is the same as the actual length.

In another possible implementation manner, the feature extraction module includes N / 2 down-sampling convolution blocks and N / 2 up-sampling convolution blocks; and acquiring feature images of different sizes of the breast image includes:

Successively extracting the first feature image of the breast image through N / 2 down-sampling convolution blocks through the breast image;

The first feature image output from the N / 2th down-sampling convolution block is used to sequentially extract N / 2 second feature images of the mammary gland image through the N / 2 up-sampling convolution block, each up-sampling convolution block The sizes of the extracted second feature images are different;

After merging the first feature image and the second feature image of the same size, N feature images of different sizes of the breast images are determined.

In order to improve the perception field of feature extraction and improve the performance of feature extraction, a possible implementation manner, the feature extraction module also includes a feature preprocessing module before; the feature preprocessing module includes a convolution layer and a BN layer, One Relu layer and one pooling layer; the size of the convolution kernel of the feature preprocessing module is larger than that of any of the N convolution modules.

Preferably, the size of the convolution kernel of the convolution layer may be 7 * 7, and the interval is 2 pixels. The pooling layer is 2 * 2 maximum pooling. Through the feature preprocessing module, the image area can be rapidly reduced, and the side length becomes 1/4, effectively improving the perception field of the feature image, quickly extracting shallow features, and effectively reducing the loss of original information.

A possible implementation manner, the feature preprocessing module includes a plurality of continuous convolutional layers, a BN layer, a Relu layer, and a pooling layer; the size of the convolution kernel of the feature preprocessing module and the N The largest convolution kernel in each convolution module has the same size.

The step of the feature image passing through the feature preprocessing module may include: inputting the breast image to the feature preprocessing module to obtain a preprocessed feature image; and using the preprocessed feature image as an input of the feature extraction module.

Step 203: For any one of the feature images of different sizes of the breast image, determine a breast lesion recognition frame from the feature image.

Optionally, a pre-trained breast lesion detection model is used to determine the breast lesion recognition frame from the feature image. The breast lesion detection model is determined after training multiple breast images of the marked breast lesion using a 2D convolutional neural network. . The area framed by the breast lesion identification frame determined from the feature image does not necessarily contain breast lesions, so each breast lesion identification frame needs to be screened according to the breast lesion probability of the breast lesion identification frame, and the breast lesion probability is less than the preset threshold The breast lesion identification frame is deleted, where the breast lesion probability is the probability that the area framed by the breast lesion identification frame is the breast lesion.

Step 204: Determine the breast lesion of the breast image according to the breast lesion identification frame determined from each feature image.

Specifically, after identifying the breast lesion recognition frame, the recognition frame is output as the breast lesion in the breast image, and the output breast lesion parameters include the central coordinates of the breast lesion and the diameter of the breast lesion, wherein the central coordinates of the breast lesion are the breast lesion identification The center coordinate of the frame, the diameter of the breast lesion is the distance from the center of the breast lesion identification frame to one of the faces.

Because feature images of different sizes of breast images are extracted and the breast lesions in each feature image are identified, both large-sized breast lesions and small-sized breast lesions can be detected, which improves the detection of breast lesions Precision. Secondly, compared with the method of manually judging whether there is a breast lesion in the breast image, the method of automatically detecting the breast lesion in the present application effectively improves the recognition efficiency of the breast lesion.

Since the breast lesion identification frame determined from each feature image may have multiple identification frames corresponding to one breast lesion, if the number of breast lesions in the breast image is directly determined according to the number of breast lesion identification frames, the number of detected breast lesions will result There is a large deviation, so it is necessary to convert each feature image into a feature image of the same size and align it, and then screen the breast lesion recognition frame determined from each feature image, and determine the screened breast lesion recognition frame as the breast Breast lesions in the image.

In order to further improve the recognition accuracy of breast lesions, a possible implementation manner, the breast image includes breast images of different breasts with different projection positions; the input of the breast image to the feature extraction module includes:

Taking the breast image of the other breast in the same projection position of the breast image as the reference image of the breast image, and inputting it to the feature extraction module to obtain a reference feature image;

Any one of the feature images of different sizes of the breast image, the breast lesion identification frame is determined from the feature image; including:

Determining a first breast lesion recognition frame in the feature image and a second breast lesion recognition frame in the reference feature image;

If it is determined that the positions and / or sizes of the first breast lesion identification frame and the second breast lesion identification frame are the same, the first breast lesion identification frame is deleted.

Optionally, the screening process of the breast lesion identification frame includes the following steps, as shown in Figure 3:

Step 301: Determine the breast lesion identification frame with the highest probability of breast lesion from the breast lesion identification frame of each feature image.

Step 302: Calculate the intersection ratio of the breast lesion identification frame with the largest breast lesion probability and the other breast lesion identification frames.

Step 303: Delete the identification frame of other breast lesions whose cross-combination ratio is greater than a preset threshold.

Step 304: Determine the breast lesion identification frame with the highest probability of breast lesions from the remaining other breast lesion identification frames, and repeat the screening process of the breast lesion identification frame until there are no other remaining breast lesion identification frames.

The screening process of the breast lesion identification frame described above will be described in conjunction with specific examples, and the breast lesion identification frames identified in each feature image are set to A, B, C, D, E, and F, and the above breast lesion identification frames The probabilities of breast lesions are: P (A) = 0.9, P (B) = 0.85, P (C) = 0.95, P (D) = 0.75, P (E) = 0.96, P (F) = 0.65. The above breast lesion recognition frame is sorted according to the probability of breast lesion from large to small: E, C, A, B, D, F, after sorting, we can know the breast lesion with the highest probability of breast lesion in the breast lesion recognition frame of each feature image The recognition frame is E, and then calculate the intersection ratio IOU between the breast lesion recognition frame E and each other breast lesion recognition frame. The calculation method of the intersection ratio is shown in equation (1):

Among them, m is the breast lesion identification frame with the highest probability of breast lesions, n is the breast lesion identification frame compared with the breast lesion identification frame m, and IOU is the intersection ratio between the breast lesion identification frame m and the breast lesion identification frame n.

Set the preset threshold to 0.5, if the intersection ratio between the breast lesion identification frame C and the breast lesion identification frame E is greater than 0.5, the intersection ratio between the breast lesion identification frame A and the breast lesion identification frame E is greater than 0.5, the breast If the intersection ratio of the lesion identification frame B, the breast lesion identification frame D, the breast lesion identification frame F and the breast lesion identification frame E is less than 0.5, delete the breast lesion identification frame C and the breast lesion identification frame A, and identify the breast lesion Box E is identified as a breast lesion in the breast image.

Further, the remaining other breast lesion recognition frames B, D, F are sorted according to the breast lesion probability, and the breast lesion recognition frame with the highest breast lesion probability is determined as the breast lesion recognition frame B, and then the breast lesion recognition frame B and the breast lesion are calculated The cross ratio between the recognition frames D and the cross ratio between the breast lesion recognition frame B and the breast lesion recognition frame F. If the intersection ratio between the breast lesion identification frame B and the breast lesion identification frame D is greater than 0.5, and the intersection ratio between the breast lesion identification frame B and the breast lesion identification frame F is less than 0.5, then delete the breast lesion identification frame D, replace The breast lesion recognition frame B and the breast lesion recognition frame F are determined as breast lesions in the breast image. Because the breast lesion identification frame identified in each feature image is screened based on the breast lesion probability of the breast lesion identification frame and the intersection and comparison between the breast lesion identification frames, avoiding repeated detection and output of the same breast lesion in the breast image, Improve the accuracy of detecting the number of breast lesions in breast images.

The following is a detailed introduction to the process of training the multiple breast images of the marked breast lesions through the 2D convolutional neural network to determine the detection model of the breast lesions, as shown in Figure 4, including the following steps:

Step 401: Obtain a breast image as a training sample.

Specifically, the acquired multiple breast images can be used directly as training samples, or the acquired multiple breast images can be enhanced to expand the data volume of the training samples. Enhancement operations include, but are not limited to: randomly setting pixels up, down, left, and right (Such as 0-20 pixels), random rotation setting angle (such as -15-15 degrees), random zoom setting multiple (such as 0.85-1.15 times).

Step 402: Manually mark the breast lesions in the training sample.

The breast lesions in the training sample can be marked by doctors and other professionals, and the content of the marking includes the central coordinates of the breast lesions and the diameter of the breast lesions. Specifically, multiple doctors can mark the breast lesions, and determine the final breast lesions and the parameters of the breast lesions through a multiple-vote synthesis method, and the results are saved in the form of a mask. It should be noted that the manual labeling of the breast lesions in the training sample and the training sample are in no particular order of enhancement. You can manually mark the breast lesions in the training sample, and then the training sample of the labeled breast lesions can be enhanced. The training samples are enhanced, and then the training samples after the enhancement operations are manually marked.

In step 403, the training samples are input to the convolutional neural network for training to determine the breast lesion recognition model.

The structure of the convolutional neural network includes an input layer, a down-sampling convolution block, an up-sampling convolution block, a target detection network, and an output layer. Pre-process the training samples and input them into the convolutional neural network, calculate the loss function of the output breast lesions and the mask image of the pre-labeled training samples, and then iterate iteratively using the back propagation algorithm and the sgd optimization algorithm to determine the breast lesions Detection model.

Further, the process of extracting feature images of different sizes of breast images using the breast lesion detection model determined by the above training includes the following steps:

In the first step, the mammary gland image is successively passed through N / 2 down-sampling convolution blocks to extract the first feature images of the N mammary gland images.

The size of the first feature image extracted by each down-sampling convolution block is different, and N / 2 is greater than 0.

Optionally, the down-sampling convolution block includes a first convolution layer and a second convolution layer, a group connection layer, a front-back connection layer, and a down-sampling layer.

Step 2: The first feature image output from the N / 2 down-sampling convolution block is sequentially used to extract the second feature image of N / 2 mammary gland images through the N / 2 up-sampling convolution block.

The size of the second feature image extracted by each up-sampling convolution block is different.

Optionally, the up-sampling convolution block includes a convolution layer, a group connection layer, a front-back connection layer, an up-sampling layer, and a synthesis connection layer. Convolution layer includes convolution operation, batch normalization layer and RELU layer.

In step three, after combining the first feature image and the second feature image with the same size, feature images of different sizes of N / 2 breast images are determined.

The first feature image and the second feature image of the same size are combined through the up-sampling convolution block to determine feature images of different sizes. Optionally, when merging, the number of channels of the first feature image and the second feature image are combined, and the size of the feature image obtained after the merging is the same as the size of the first feature image and the second feature image.

Further, the process of determining the breast lesion recognition frame from the feature image using the breast lesion detection model determined by the above training includes the following steps:

Step 1: For any pixel in the feature image, with the pixel as the center, diffuse to the surroundings to determine the first area.

Step 2: Set multiple preset frames in the first area according to preset rules.

Due to the different shapes of breast lesions, the preset frame can be set to various shapes. The preset rule may be that the center of the preset frame coincides with the center of the first area, or that the corner of the preset frame coincides with the angle of the first area, and so on.

In a specific embodiment, the way to select the preset frame of the breast lesion is that, for each pixel of each feature map, it is considered as an anchor point. Set multiple preset frames with different aspect ratios on each anchor point.

For each preset frame, the convolution of the feature map predicts a coordinate and size offset and confidence, and the preset frame is determined based on the coordinate and size offset and confidence.

Step 3: For any preset frame, predict the position deviation of the preset frame from the first area.

Step 4: Adjust the preset frame according to the position deviation to determine the breast lesion identification frame, and predict the breast lesion probability of the breast lesion identification frame.

The breast lesion probability is the probability that the area selected by the breast lesion identification frame is the breast lesion. By predicting the position deviation between the preset frame and the first area, and then using the position deviation to adjust the preset frame to determine the recognition frame, so that the recognition frame can more frame the breast lesion area in the feature map, and improve the accuracy of breast lesion detection.

The specific training process may include: inputting the training data image to the above-mentioned convolutional neural network for calculation. During the introduction, multiple images of different window widths and positions of the lesion are introduced. During training, in the prediction frame output by the convolutional neural network, the prediction frame set with the highest confidence and the prediction frame set with the maximum coincidence with the training sample are selected. The cross-entropy of the confidence of the prediction frame and the labeling of the sample and the cross-entropy of the labeled lesion of the training sample and the offset of the prediction frame are used as the loss function. Trained by the method of back propagation, the training optimization algorithm uses the sgd algorithm with momentum and step attenuation.

In the process of using the algorithm, through the preprocessing module, the input image is preprocessed to improve the effect of feature extraction.

A possible implementation manner, the acquiring breast image includes:

Step 1: Determine the binary image of the breast image according to Gaussian filtering;

Step 2: Obtain the connected area of the binarized image, and use the area with the largest area in the connected area corresponding to the breast image as the segmented breast image;

Step 3: Add the segmented breast image to a preset image template to generate a pre-processed breast image; and use the pre-processed breast image as the breast image input to the feature extraction module.

Specifically, the input of the preprocessing module is a breast image saved in Dicom format. Preprocessing can include gland segmentation and image normalization; the main purpose of gland segmentation is to extract the breast part of the input breast image to remove other unrelated interference images; image normalization is to normalize the image into Unified format images, specifically, include:

In step one, the specific binarized threshold can be obtained by finding the maximum class interval of the grayscale histogram of the image.

In step two, the binarized result can be obtained by flooding to obtain independent regional blocks, and the area of each regional block is counted; the area on the image corresponding to the largest regional block is used as Segmented breast image.

In step three, the preset image template may be a square image of a black bottom plate; specifically, the obtained divided breast image may be expanded into a 1: 1 square image by adding a black border.

In addition, the output breast image can be scaled by pixels, for example, the image difference can be scaled to 4096 pixels × 4096 pixels.

For the mammary gland, due to the dose of the mammary gland and external factors of the shooting, the window width and position of the mammary gland can be adjusted to obtain a better identification effect of breast lesion identification. In a possible implementation manner, before inputting the breast image to the feature extraction module, the method further includes:

Obtain the original file of the breast image;

Selecting at least one set of window width and window levels from the original file of the breast image, and acquiring a breast image in a picture format corresponding to the at least one set of window width and window levels;

The mammary gland image according to the picture format corresponding to the at least one set of window width and window level is used as the mammary gland image input to the feature extraction module.

In a specific embodiment, the dicom image can be converted into a png image through three sets of window width and window levels. For example, the first set of window width is 4000 and the window level is 2000; the second set of window width is 1000; the window level is 2000 ; The third group has a window width of 1500 and a window level of 1500.

Based on the same technical concept, an embodiment of the present invention provides a device for identifying breast lesions. As shown in FIG. 5, the device can perform the flow of a method for identifying breast lesions. The device includes an acquiring unit 501 and a processing unit 502.

The obtaining unit 501 is used to obtain breast images;

The processing unit 502 is configured to input the breast image into a feature extraction module to obtain feature images of different sizes of the breast image; the feature extraction module includes N convolution modules; the N convolution modules are as follows Sampling convolutional blocks and / or upsampling convolutional blocks; the size of the feature image extracted by each downsampling convolutional block or upsampling convolutional block is different, and each of the N convolutional modules includes The first convolution layer and the second convolution layer; the number of feature images output by the first convolution layer is less than the number of feature images input by the first convolution layer; the second convolution layer outputs The number of feature images of is greater than the number of feature images output by the first convolutional layer; N is greater than 0; for any one of the feature images of different sizes of the breast image, from the feature image Identify the breast lesion recognition frame; determine the breast lesion of the breast image according to the breast lesion recognition frame determined from each feature image.

In a possible implementation manner, the processing unit 502 is specifically configured to:

The breast image is sequentially passed through N / 2 down-sampling convolution blocks to extract N / 2 first feature images of the breast image; the first feature images output from the N / 2 down-sampling convolution block are sequentially passed N / 2 up-sampling convolution blocks extract N / 2 second feature images of the mammography image, the size of the second feature images extracted by each up-sampling convolution block are different; the first feature images of the same size After merging with the second feature image, N feature images of different sizes of the breast images are determined.

A possible implementation manner, before the feature processing module further includes a feature preprocessing module; the processing unit 502 is specifically configured to:

Input the breast image into the feature preprocessing module, the feature preprocessing module includes a convolution layer, a BN layer, a Relu layer and a pooling layer; the convolution kernel of the feature preprocessing module The size is larger than the size of the convolution kernel in the N convolution modules;

Alternatively, the feature pre-processing module includes multiple consecutive convolutional layers, a BN layer, a Relu layer, and a pooling layer; the size of the convolution kernel of the feature pre-processing module and the N convolution modules The size of the largest convolution kernel is equal.

In a possible implementation manner, the obtaining unit 501 is configured to:

Obtain the original file of the breast image;

The processing unit 502 is specifically used for:

Selecting at least one group of window width and window levels from the original file of the breast image, and obtaining a breast image in a picture format corresponding to the at least one group of window width and window levels; according to the pictures corresponding to the at least one group of window width and window levels The breast image in the format is used as the breast image input to the feature extraction module.

In a possible implementation manner, the mammary gland image includes mammary gland images with different projection positions of different breasts; the acquiring unit 501 is configured to:

Take the breast image of the other breast of the same projection position of the breast image as the reference image of the breast image, and input it to the feature extraction module to obtain a reference feature image; determine the first breast in the feature image The lesion identification frame and the second breast lesion identification frame in the reference feature image; if it is determined that the positions and / or sizes of the first breast lesion identification frame and the second breast lesion identification frame are the same, delete the The first breast lesion identification frame.

Based on the same technical concept, an embodiment of the present invention provides a computer device including at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program when the program is executed by the processing unit So that the processing unit executes the steps of the method for identifying breast lesions. As shown in FIG. 6, it is a schematic diagram of the hardware structure of the computer device described in the embodiment of the present invention. The computer device may specifically be a desktop computer, a portable computer, a smart phone, a tablet computer, or the like. Specifically, the computer device may include a memory 801, a processor 802, and a computer program stored on the memory. When the processor 802 executes the program, any steps of the method for identifying a breast lesion in the foregoing embodiments are implemented. The memory 801 may include a read-only memory (ROM) and a random access memory (RAM), and provide the processor 802 with program instructions and data stored in the memory 801.

Further, the computer equipment described in the embodiments of the present application may further include an input device 803 and an output device 804. The input device 803 may include a keyboard, a mouse, a touch screen, etc .; the output device 804 may include a display device, such as a liquid crystal display (Liquid Crystal Display, LCD), a cathode ray tube (Cathode Ray Tube, CRT), a touch screen, and the like. The memory 801, the processor 802, the input device 803, and the output device 804 may be connected through a bus or in other ways. In FIG. 6, connection through a bus is used as an example. The processor 802 calls the program instructions stored in the memory 801 and executes the method for identifying breast lesions provided in the foregoing embodiments according to the obtained program instructions.

Based on the same technical concept, an embodiment of the present invention also provides a computer-readable storage medium that stores a computer program executable by a computer device, and when the program runs on the computer device, causes the computer device to execute a breast Steps of the method of lesion identification.

Based on the same technical concept, an embodiment of the present invention also provides a computer program product, the computer program product includes a computer program stored on a computer-readable storage medium, the computer program includes program instructions, when the program instructions When executed by a computer device, the computer device is caused to perform the steps of the method for identifying breast imaging lesions.

Those skilled in the art should understand that the embodiments of the present invention may be provided as methods or computer program products. Therefore, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram and a combination of the flow and / or block in the flowchart and / or block diagram may be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, special-purpose computer, embedded processing machine, or other programmable data processing device to produce a machine that enables the generation of instructions executed by the processor of the computer or other programmable data processing device A device for realizing the functions specified in one block or multiple blocks of one flow or multiple blocks of a flowchart.

These computer program instructions may also be stored in a computer readable memory that can guide a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, the instructions The device implements the functions specified in one block or multiple blocks of the flowchart one flow or multiple flows and / or block diagrams.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operating steps are performed on the computer or other programmable device to produce computer-implemented processing, which is executed on the computer or other programmable device The instructions provide steps for implementing the functions specified in one block or multiple blocks of the flowchart one flow or multiple flows and / or block diagrams.

Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications to these embodiments once they learn the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present invention. Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. In this way, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims (11)

1. A method for identifying breast imaging lesions, characterized in that it includes:

   Get breast images;

   The breast image is input into a feature extraction module to obtain feature images of different sizes of the breast image; the feature extraction module includes N convolution modules; the N convolution modules are down-sampling convolution blocks and / or Or up-sampling convolution block; the size of the feature image extracted by each down-sampling convolution block or up-sampling convolution block is different, each of the N convolution modules includes a first convolution layer, The second convolutional layer; the number of feature images output by the first convolution layer is less than the number of feature images input by the first convolution layer; the number of feature images output by the second convolution layer Is greater than the number of feature images output by the first convolutional layer; N is greater than 0;

   For any one of the feature images of different sizes of the breast image, determine the breast lesion identification frame from the feature image;

   According to the breast lesion identification frame identified from each feature image, the breast lesion of the breast image is determined.
2. The method according to claim 1, wherein the acquiring feature images of different sizes of the breast image include:

   Successively extracting the first feature image of the breast image through N / 2 down-sampling convolution blocks through the breast image;

   The first feature image output from the N / 2th down-sampling convolution block is used to sequentially extract N / 2 second feature images of the mammary gland image through the N / 2 up-sampling convolution block, each up-sampling convolution block The sizes of the extracted second feature images are different;

   After merging the first feature image and the second feature image of the same size, N feature images of different sizes of the breast images are determined.
3. The method according to claim 1, wherein before the feature processing module, a feature preprocessing module is further included; before the breast image is input to the feature extraction module, the method further includes:

   Input the breast image into the feature preprocessing module, the feature preprocessing module includes a convolution layer, a BN layer, a Relu layer and a pooling layer; the convolution kernel of the feature preprocessing module The size is larger than the size of the convolution kernel in the N convolution modules;

   Alternatively, the feature pre-processing module includes multiple consecutive convolutional layers, a BN layer, a Relu layer, and a pooling layer; the size of the convolution kernel of the feature pre-processing module and the N convolution modules The size of the largest convolution kernel is equal.
4. The method according to claim 1, wherein before the inputting the breast image to the feature extraction module, the method further comprises:

   Obtain the original file of the breast image;

   Selecting at least one set of window width and window levels from the original file of the breast image, and acquiring a breast image in a picture format corresponding to the at least one set of window width and window levels;

   The mammary gland image according to the picture format corresponding to the at least one set of window width and window level is used as the mammary gland image input to the feature extraction module.
5. The method according to any one of claims 1 to 4, wherein the mammary gland image includes mammary gland images with different projection positions of different breasts; and the inputting the mammary gland image to a feature extraction module includes:

   Taking the breast image of the other breast in the same projection position of the breast image as the reference image of the breast image, and inputting it to the feature extraction module to obtain a reference feature image;

   Any one of the feature images of different sizes of the breast image, the breast lesion identification frame is determined from the feature image; including:

   Determining a first breast lesion recognition frame in the feature image and a second breast lesion recognition frame in the reference feature image;

   If it is determined that the positions and / or sizes of the first breast lesion identification frame and the second breast lesion identification frame are the same, the first breast lesion identification frame is deleted.
6. A device for identifying breast imaging lesions, characterized in that it includes:

   Acquisition unit for acquiring mammary gland image;

   A processing unit for inputting the breast image into a feature extraction module to obtain feature images of different sizes of the breast image; the feature extraction module includes N convolution modules; the N convolution modules are for downsampling Convolution block or up-sampling convolution block; the size of the feature image extracted by each down-sampling convolution block or up-sampling convolution block is different, and each of the N convolution modules includes a first volume Multilayer, second convolutional layer; the number of feature images output by the first convolutional layer is less than the number of feature images input by the first convolutional layer; the feature images output by the second convolutional layer The number of is greater than the number of feature images output by the first convolutional layer; N is greater than 0; for any one of the feature images of different sizes of the breast image, the breast is determined from the feature image Lesion recognition frame; according to the breast lesion recognition frame determined from each characteristic image, determine the breast lesion of the breast image.
7. The apparatus according to claim 6, wherein the processing unit is specifically configured to:

   The breast image is sequentially passed through N / 2 down-sampling convolution blocks to extract N / 2 first feature images of the breast image; the first feature images output from the N / 2 down-sampling convolution block are sequentially passed N / 2 up-sampling convolution blocks extract N / 2 second feature images of the mammography image, the size of the second feature images extracted by each up-sampling convolution block are different; the first feature images of the same size After merging with the second feature image, N feature images of different sizes of the breast images are determined.
8. The device according to claim 6 or 7, wherein the mammary gland image comprises mammary gland images with different projection positions of different breasts; the processing unit is specifically used for:

   Take the breast image of the other breast of the same projection position of the breast image as the reference image of the breast image, and input it to the feature extraction module to obtain a reference feature image; determine the first breast in the feature image The lesion identification frame and the second breast lesion identification frame in the reference feature image; if it is determined that the positions and / or sizes of the first breast lesion identification frame and the second breast lesion identification frame are the same, delete the The first breast lesion identification frame.
9. A computer device, characterized by comprising at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the program is executed by the processing unit, the processing unit executes the right The method of any one of claims 1 to 5 is required.
10. A computer-readable storage medium, characterized in that it stores a computer program that can be executed by a computer device, and when the program is run on the computer device, the computer device is caused to execute any of claims 1 to 5. Describe the steps of the method.
11. A computer program product, characterized in that it includes a computer program stored on a computer-readable storage medium, the computer program includes program instructions, and when the program instructions are executed by a computer device, causes the computer device to execute the claims Steps 1 to 5 of the method.

Images