WO2020019671A1

WO2020019671A1 - Breast lump detection and classification system and computer-readable storage medium

Info

Publication number: WO2020019671A1
Application number: PCT/CN2018/124655
Authority: WO
Inventors: 徐勇; 刘宏
Original assignee: 哈尔滨工业大学（深圳）; 北京大学深圳研究生院
Priority date: 2018-07-23
Filing date: 2018-12-28
Publication date: 2020-01-30
Also published as: CN109191424B; CN109191424A

Abstract

A breast lump detection and classification system and a computer-readable storage medium. The pixels of a breast image are classified directly into three types, i.e., common pixels, benign lump pixels, and malignant lump pixels, thereby quickly and accurately performing lump detection and classification on the breast image, and resolving the technical problem in the prior art of low accuracy and low efficiency caused by two steps, i.e., lump detection and lump classification.

Description

Breast lump detection and classification system and computer-readable storage medium Ranch

Technical field

The invention relates to the medical field, in particular to a breast mass detection and classification system and a computer-readable storage medium.

Background technique

Mammography mammography images are widely used mammography images, which have the advantages of low cost, high quality, and high cost performance. This type of image mainly reflects the contours of breast tissue. Using mammography images of mammary glands, doctors can better identify breast masses and judge the nature of the masses. However, doctors' manual discrimination has the problems of relying on subjective experience and different discrimination results between different doctors. Due to the development of computer vision understanding technology, automatic identification of breast masses and their qualitative computer becomes possible.

Most of the existing methods for breast mass detection and identification are divided into two steps: breast mass detection and classification. The current method is implemented in two steps mainly as follows: It is generally believed that breast tumors have a relatively special texture structure, and tumors lack clear edges. Therefore, breast tumor detection is a task that is quite different from ordinary target detection tasks. The texture features in ordinary target edges generally have significant features. Therefore, people tend to use special methods to detect tumors before classifying them. Such a processing method has the disadvantage of high computational complexity. Moreover, these two independent steps cut off the intrinsic link between the two closely linked tasks of breast mass detection and benign and malignant mass classification; the separate use of these two steps also limits the overall performance of the method. If this The accuracy of both steps is 80%, and the overall accuracy is only 64%. Therefore, it is urgent to explore new methods and technical routes to overcome the shortcomings of current methods. Ranch

Summary of the Invention

The present invention aims to solve at least one of the technical problems in the related technology. To this end, an object of the present invention is to provide a breast mass detection and classification system and a computer-readable storage medium for implementing breast mass detection and classification.

The technical scheme adopted by the present invention is: a breast mass detection and classification system, including:

An image acquisition unit, configured to acquire a breast image;

A pixel classification unit, configured to classify each pixel of the breast image, and the types of the pixels include ordinary pixels, benign lumps pixels, and malignant lumps pixels;

A detection and classification unit is configured to detect and classify a breast mass from a pixel-classified breast image.

Further, the breast mass detection and classification system further includes a neural network unit, and the neural network unit includes:

A network construction module, configured to construct a deep neural network, which is used to classify each pixel of the breast image, and the types of the pixels include ordinary pixels, benign tumor pixels, and malignant tumor pixels; the depth The neural network is divided into six layers, which are a first convolution layer, a second convolution layer, a third convolution layer, a first fully connected layer, a second fully connected layer, and a network output layer in this order. The deep neural network The number of neurons in each layer is the same as the number of pixels in the breast image;

A network training module is configured to train the deep neural network by using a breast image sample set.

Further, the first convolutional layer, the second convolutional layer, the third convolutional layer, and the first fully connected layer adopt an improved LU activation function, and the improved LU activation function is a calculation value of the LU activation function smaller than When it is equal to a preset value, the calculated value of the LU activation function is set to 0.

Further, the second fully connected layer uses a sigmoid activation function.

Further, the breast image is calculated through the first convolution layer, the second convolution layer, the third convolution layer, the first fully connected layer, and the second fully connected layer to obtain a scalar corresponding to each pixel;

The network output layer is used to calculate the absolute value of the difference between the scalar and the pixel digital class. The pixel digital class includes a normal pixel class, a benign mass pixel class, and a malignant mass pixel class. The pixel class of the pixel corresponding to the smallest difference is output as the class of the pixel.

Further, the normal pixel class is labeled -1, the benign mass pixel class is labeled 0, and the malignant mass pixel class is labeled 1.

Further, the network training module includes

An image processing sub-module, configured to process the breast image sample set, where the breast image sample set includes multiple breast image samples, and a method for processing the breast image sample set includes:

Transforming the pixel gray value of the breast image sample includes increasing the gray value of the pixels of the breast image sample by the same proportion or reducing the gray value of the pixels of the breast image sample by the same proportion;

and / or,

Adding Gaussian noise to all or part of pixels of the breast image sample;

and / or,

Adding pepper and salt noise to all or part of the pixels of the breast image sample;

A training sub-module is configured to train the deep neural network by using a breast image sample set processed by the image processing sub-module.

Further, the detection and classification unit includes

A region division module, configured to divide a pixel-classified breast image into a plurality of partially overlapping subregions, the subregions having different sizes;

A mass detection module is configured to determine whether a proportion of benign mass pixels of the sub-region to the total pixels of the sub-region or a proportion of malignant mass pixels to the total pixels of the sub-region is greater than a preset ratio. When the determination result is yes When determining that the sub-region is a breast lump;

A tumor classification module is configured to determine whether the proportion of the benign tumor pixels in the total pixels of the sub-region is greater than the proportion of the malignant tumor pixels in the total pixels of the sub-region. A benign breast mass, and conversely, the subregion is a malignant breast mass.

Further, the preset ratio is 0.3.

Another technical solution adopted by the present invention is: a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

Obtain breast images;

Classify each pixel of the breast image, the types of the pixels include ordinary pixels, benign lumps pixels, and malignant lumps pixels;

Detection and classification of breast masses on pixel-classified breast images.

The beneficial effects of the present invention are:

The invention relates to a breast mass detection and classification system and a computer-readable storage medium. By classifying each pixel of a breast image, the pixels are directly classified into three categories: ordinary pixels, benign mass pixels, and malignant mass pixels, to achieve fast and accurate Performing mass detection and mass classification on breast images, overcoming the two steps of splitting mass detection and classification in the prior art, leads to technical problems of low accuracy and low efficiency.

BRIEF DESCRIPTION OF THE DRAWINGS

The specific embodiments of the present invention are further described below with reference to the accompanying drawings:

1 is a schematic structural diagram of a specific embodiment of a breast mass detection and classification system in the present invention;

2 is a function curve diagram of a specific embodiment of an improved LU activation function in the present invention;

3 is a schematic diagram of a specific embodiment for increasing the gray value of pixels in the same proportion according to the present invention;

FIG. 4 is a schematic diagram of a specific embodiment of reducing gray values of pixels in the same proportion according to the present invention.

detailed description

It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

A breast mass detection and classification system. Referring to FIG. 1, FIG. 1 is a schematic structural diagram of a specific embodiment of a breast mass detection and classification system according to the present invention; including:

An image acquisition unit is used to acquire a breast image. In this embodiment, the breast image is a mammography target X-ray radiographic image obtained by using an X-ray apparatus. The original mammography target X-ray radiographic image may be different in size. Therefore, the breast of the present invention The image is scaled to a size of 200 pixels by 200 pixels, and then entered into a pixel classification unit.

A pixel classification unit, configured to classify each pixel of a breast image, and the types of pixels include ordinary pixels, benign tumor pixels, and malignant tumor pixels; Ranch

A detection and classification unit is configured to detect and classify a breast mass from a pixel-classified breast image. In this embodiment, the image acquisition unit, the pixel classification unit, and the detection and classification unit may be computers. The computer uses a computer program to implement image acquisition, pixel classification, and mass detection and classification. The image acquisition unit may directly acquire a breast image stored in a computer in advance or sent to the computer in real time, or may acquire a breast image taken by the breast image by communicating with the X-ray apparatus.

The pixel classification unit in the present invention is directly based on the breast image, and performs pixel classification discrimination on a single pixel of the breast image. The detection and classification unit can further simultaneously locate a breast tumor based on the pixel discrimination result and identify the tumor type (benign breast tumor or Malignant breast mass) to achieve fast and effective breast mass detection and classification.

As a further improvement of the technical solution, in this embodiment, the pixel classification unit uses a deep neural network to classify pixels of a breast image. Therefore, before classifying a pixel of a breast image, a deep neural network needs to be designed first. The breast mass detection and classification system also includes a neural network unit. The neural network unit includes:

A network building module is used to build a deep neural network. The deep neural network is used to classify each pixel of the breast image. The types of pixels include ordinary pixels, benign lumps, and malignant lumps. The deep neural network is divided into six layers. The order is the first convolution layer, the second convolution layer, the third convolution layer, the first fully connected layer, the second fully connected layer, and the network output layer. The number of neurons in each layer of the deep neural network and The number of pixels in the breast image is the same;

A network training module for training a deep neural network using a breast image sample set.

Among them, the first convolution layer, the second convolution layer, the third convolution layer, and the first fully connected layer adopt an improved LU activation function. The improved LU activation function is that the calculated value of the LU activation function is less than or equal to a preset value. , The calculated value of the LU activation function is set to 0. The specific settings are as follows: if z> e (Z represents the result of the current convolution calculation, e is a smaller constant greater than zero), Then f (z) = z, Otherwise, f (z) = 0. Assuming e = 1, the function curve of f (z) = z when z> e is shown in FIG. 2, which is a function curve diagram of a specific embodiment of the improved LU activation function in the present invention. In addition, a large output value always plays a larger role in the network, while a small output value plays a smaller role. The method commonly used by deep networks to enhance the robustness of the network is to block the connections between some of these neurons, artificially achieving the sparsity of the connections. Blocking the connection is actually the same effect as setting the neuron's output value to 0, but the latter is simpler and does not require any additional computation. In order to enhance the network sparsity, the present invention proposes the improved LU activation function described above, which makes the network sparse in structure in a simple manner, thereby enhancing the robustness of the lump detection and classification results. The second fully connected layer uses a sigmoid activation function. The sigmoid activation function enhances the non-linear transformation capabilities of the "classifier." Different from the traditional convolutional neural network, the deep neural network in the present invention does not have a pooling layer, mainly to reduce the loss of information.

Specifically, the breast image is calculated through the first convolution layer, the second convolution layer, the third convolution layer, the first fully connected layer, and the second fully connected layer to obtain a scalar corresponding to each pixel; the network output layer is used for calculation The absolute value of the difference between a scalar and a pixel number class. The pixel number class includes ordinary pixel class, benign mass pixel class, and malignant mass pixel class, and the pixel number class corresponding to the smallest absolute value. The output is used as the pixel's class standard, and it is quantized to one of the three based on the similarity of the scalar with the three class targets. In this embodiment, the normal pixel class is labeled -1, the benign mass pixel class is labeled 0, and the malignant mass pixel class is labeled 1. This allows the steps of breast mass detection and mass classification to be combined into one, and can simultaneously improve the computational efficiency of the training and application phases. In addition, the settings of the three categories of -1, 0, and 1 are also in line with the visual description of the differences between the three pixel categories. The difference between -1 and 0 is less than the difference between -1 and 1. It is consistent with ordinary pixels and benign mass pixels. The fact that the difference between them is smaller than the difference between ordinary pixels and malignant mass pixels. For example, suppose a pixel is input to a neural network, and the final scalar output at the second fully connected layer is 3, then the network output layer calculates the absolute value of the difference between 3 and -1, 0, and 1 and compares 3 The absolute value of the absolute value, select the category corresponding to the difference with the smallest absolute value as the category of this pixel. In this embodiment, the smallest absolute value is the absolute value of the difference between 3 and 1, then this pixel The class is labeled 1. In summary, after the breast image is input into the neural network, the corresponding pixel type of the breast image is output in the network output layer; that is, the final output of the neurons in the network output layer is -1, 0, or 1. The corresponding pixels of the input breast image are respectively Discriminated as normal pixels, benign mass pixels or malignant mass pixels.

As a further improvement of the technical solution, after the deep neural network is constructed, it is trained using a network training module in order to better implement pixel classification processing of breast images. Specifically, the network training module includes

An image processing sub-module is used to process a breast image sample set. The breast image sample set includes multiple breast image samples. The breast image samples have been labeled with tumors and their types in the breast image. The method for processing the breast image sample set includes:

Transforming the pixel gray value of the breast image sample, including increasing the gray value of the pixels of the breast image sample by the same proportion or reducing the gray value of the pixels of the breast image sample by the same proportion;

and / or,

Add Gaussian noise to pixels of all or part of the breast image samples;

and / or,

Add pepper and salt noise to pixels of all or part of breast image samples;

A training sub-module is used to train a deep neural network using the breast image sample set processed by the image processing sub-module, and input the processed breast image sample set to the deep neural network for pixel classification to implement network training.

In practical applications, the images of the same mammary gland obtained by X-ray instruments from different manufacturers have great differences, and even the images obtained by the same type of X-ray apparatus have different phenomena. In addition, it is impossible to obtain the breasts of all manufacturers in practical applications It is impossible to obtain a large number of image samples of the same type of mammography target instrument with a large number of photographic images of the target instrument. In order to enhance the adaptability of the system to breast image differences, the present invention uses limited image samples during the training phase to generate as many breast image samples as possible, enhancing the diversity of breast image sample sets, as they can represent different breast molybdenum targets The image results of the instrument can enrich the "experience" of the neural network, enhance the adaptability of the training results to the samples, and the neural network after training will have better robustness. By processing and modifying the pixels of the breast image sample to obtain more breast image samples; by using the above-mentioned pixel gray value modification and noise addition modification methods at the same time, more samples are generated and the effect is better. The specific method is as follows:

1. Transform the gray value of the pixel of the breast image sample, and increase the gray value and decrease the gray value of the pixels in the same proportion to obtain new examples. In the method of increasing the gray value in the same proportion, if the modified value of the pixel is greater than 255, it is forcibly set to 255. The formula for increasing the gray value in the same proportion is a + bx, a and b are coefficients and b is greater than 1, and x is the original pixel value. The result of increasing the gray value of the pixels in the same proportion is shown in FIG. 3. FIG. 3 is a schematic diagram of a specific embodiment of increasing the gray value of the pixels in the same proportion in the present invention; When it reaches 255, the pixel value is increased by the same proportion (as shown by the y coordinate). The formula for reducing the gray value in the same proportion is c + dx, c and d are coefficients and d is greater than 0 but less than 1, and x is still the original pixel value. The result of reducing the gray value of the pixels in the same proportion is shown in FIG. 4. FIG. 4 is a schematic diagram of a specific embodiment of reducing the gray value of the pixels in the same proportion in the present invention; When it reaches 255, the pixel value becomes smaller in proportion (as shown in the y coordinate).

2. Add a certain degree of Gaussian noise to the pixels of all breast image samples.

3. Add a certain degree of salt and pepper noise to the pixels of all breast image samples.

4. Randomly add Gaussian or salt and pepper noise to the pixels in an image of a breast image sample.

The above four schemes describe the variability of the samples from different angles. The first scheme can simulate the systematic differences of different models of X-ray instruments. The two methods of increasing and decreasing the pixel value respectively make the new sample better. Representativeness and biased in two possible directions of change. The second, third, and fourth schemes can simulate the random errors of the instruments, and have better simulation capabilities for the differences between the same model of instruments. Using these four schemes at the same time makes the pixel change in the new sample generated more comprehensive. In summary, the image processing sub-module modifies the pixels of the original breast image to increase the diversity of the breast image sample set. Ranch

As a further improvement of the technical solution, in actual use, after the image acquisition unit acquires the breast image, the pixel classification unit uses the trained neural network to classify the pixels of the breast image to obtain a pixel class label corresponding to each pixel. The detection and classification unit performs breast mass detection and classification on the breast image classified by the pixels. Specifically, the detection and classification unit includes:

A region division module is used to divide a pixel-classified breast image into a plurality of partially overlapping sub-regions. The sub-regions have different sizes. In this embodiment, the sub-regions are rectangular regions, and the dimensions of the rectangular regions are different (such as 7 pixels * 7 pixels, 9 pixels * 9 pixels, 11 pixels * 11 pixels, 13 pixels * 13 pixels); because the actual breast masses are of different sizes, the detection and classification system of the present invention judges sub-regions of different sizes to make the detected breast masses more realistic. If only the input breast image is divided into non-overlapping sub-regions, and the sub-regions have the same size, when the breast mass crosses two adjacent sub-regions, it is likely to miss detection.

A mass detection module is used to determine whether the proportion of benign mass pixels in the sub-region to the total pixels of the sub-region or the proportion of malignant mass pixels in the total pixels of the sub-region is greater than a preset ratio. When the determination result is yes, the sub-region is determined to be Breast lump; in this embodiment, the preset ratio is 0.3.

A tumor classification module is used to determine whether the proportion of benign tumor pixels in the total area of the sub-region is greater than that of malignant tumor pixels in the total area of the sub-region. If the result of the determination is yes, the sub-region is a benign breast tumor. For malignant breast mass.

The mass detection module and mass classification module perform mass detection and classification on each sub-region of the breast image, and directly perform mass detection and classification based on breast image pixels, which is efficient, convenient, and highly accurate.

The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

Obtain breast images;

Classify each pixel of the breast image, the types of pixels include ordinary pixels, benign lumps pixels and malignant lumps pixels;

For the working process of a computer program stored in a computer-readable storage medium, please refer to the description of the above breast mass detection and classification system, and will not be described again.

In order to efficiently and conveniently implement breast mass detection and identification based on mammography images of mammary glands, unlike traditional methods, the present invention directly identifies pixels of breast images and determines them as ordinary pixels, benign mass pixels, There are three categories of pixels for malignant masses; the categories of pixels for the breast image output by the neural network; the location of breast masses and the discrimination results of benign or malignant masses are obtained according to the pixel class.

The above is a detailed description of the preferred implementation of the present invention, but the present invention is not limited to the described embodiments. Those skilled in the art can make various equivalent deformations or replacements without departing from the spirit of the present invention. These equivalent modifications or replacements are all included in the scope defined by the claims of this application.

Claims

A breast mass detection and classification system, comprising:

An image acquisition unit, configured to acquire a breast image;

A pixel classification unit, configured to classify each pixel of the breast image, and the types of the pixels include ordinary pixels, benign lumps pixels, and malignant lumps pixels; Ranch

A detection and classification unit is configured to detect and classify a breast mass from a pixel-classified breast image.
The breast mass detection and classification system according to claim 1, wherein the breast mass detection and classification system further comprises a neural network unit, and the neural network unit comprises:

A network construction module, configured to construct a deep neural network, which is used to classify each pixel of the breast image, and the types of the pixels include ordinary pixels, benign tumor pixels, and malignant tumor pixels; the depth The neural network is divided into six layers, which are a first convolution layer, a second convolution layer, a third convolution layer, a first fully connected layer, a second fully connected layer, and a network output layer in this order. The deep neural network The number of neurons in each layer is the same as the number of pixels in the breast image;

A network training module is configured to train the deep neural network by using a breast image sample set.
The breast mass detection and classification system according to claim 2, wherein the first convolution layer, the second convolution layer, the third convolution layer, and the first fully connected layer use an improved LU activation function, When the improved LU activation function is that the calculated value of the LU activation function is less than or equal to a preset value, the calculated value of the LU activation function is set to 0. Ranch
The breast mass detection and classification system according to claim 2, wherein the second fully connected layer uses a sigmoid activation function. Ranch
The breast mass detection and classification system according to any one of claims 2 to 4, wherein the breast image passes through the first convolution layer, the second convolution layer, the third convolution layer, the first The fully connected layer and the second fully connected layer are calculated to obtain a scalar corresponding to each pixel;

The network output layer is used to calculate the absolute value of the difference between the scalar and the pixel digital class. The pixel digital class includes a normal pixel class, a benign mass pixel class, and a malignant mass pixel class. The pixel class of the pixel corresponding to the smallest difference is output as the class of the pixel.
The breast mass detection and classification system according to claim 5, wherein the ordinary pixel class is labeled -1, the benign mass pixel class is labeled 0, and the malignant mass pixel class is labeled 1. Ranch
The breast mass detection and classification system according to any one of claims 2 to 4, wherein the network training module includes

An image processing sub-module, configured to process the breast image sample set, where the breast image sample set includes multiple breast image samples, and a method for processing the breast image sample set includes:

Transforming the pixel gray value of the breast image sample includes increasing the gray value of the pixels of the breast image sample by the same proportion or reducing the gray value of the pixels of the breast image sample by the same proportion;

and / or,

Adding Gaussian noise to all or part of pixels of the breast image sample;

and / or,

Adding pepper and salt noise to all or part of the pixels of the breast image sample;

A training sub-module is configured to train the deep neural network by using a breast image sample set processed by the image processing sub-module.
The breast mass detection and classification system according to any one of claims 1 to 4, wherein the detection and classification unit includes

A region division module, configured to divide a pixel-classified breast image into a plurality of partially overlapping subregions, the subregions having different sizes;

A mass detection module is configured to determine whether a proportion of benign mass pixels of the sub-region to the total pixels of the sub-region or a proportion of malignant mass pixels to the total pixels of the sub-region is greater than a preset ratio. When determining that the sub-region is a breast lump;

A tumor classification module is configured to determine whether the proportion of the benign tumor pixels in the total pixels of the sub-region is greater than the proportion of the malignant tumor pixels in the total pixels of the sub-region. If the determination result is yes, the sub-region is A benign breast mass, and conversely, the subregion is a malignant breast mass.
The breast mass detection and classification system according to claim 8, wherein the preset ratio is 0.3.
A computer-readable storage medium is characterized in that a computer program is stored thereon, and when the computer program is executed by a processor, the following steps are implemented:

Obtain breast images;

Classify each pixel of the breast image, the types of the pixels include ordinary pixels, benign lumps pixels, and malignant lumps pixels;

Detection and classification of breast masses on pixel-classified breast images.