CN113343995A - Image segmentation method based on reverse attention network - Google Patents
Image segmentation method based on reverse attention network Download PDFInfo
- Publication number
- CN113343995A CN113343995A CN202110493747.5A CN202110493747A CN113343995A CN 113343995 A CN113343995 A CN 113343995A CN 202110493747 A CN202110493747 A CN 202110493747A CN 113343995 A CN113343995 A CN 113343995A
- Authority
- CN
- China
- Prior art keywords
- feature
- reverse attention
- reverse
- image
- attention network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an image segmentation method based on a reverse attention network, which comprises the steps of firstly, obtaining an image data set, and constructing a training set and a test set; then constructing a reverse attention network model, wherein the processing process of the reverse attention network model comprises the steps of sequentially obtaining output feature layers of different levels by encoding the image layer by layer through a plurality of convolution layers, utilizing the output features of the different levels to be spliced and aggregated in parallel and then inputting the output features into a decoder to be decoded to obtain a global feature image, and inputting the global feature image and the output features into a reverse attention network to be processed until the reverse attention features of the low levels are obtained; and inputting the training set into a reverse attention network model for training to obtain a trained reverse attention network model and obtain an image segmentation result. The invention inputs the high-level characteristics obtained by image coding into the reverse attention network, so that the precision of image segmentation is greatly improved.
Description
Technical Field
The invention belongs to the field of medical image processing, and particularly relates to an image segmentation method based on a reverse attention network.
Background
Colorectal cancer (CRC) is the third most common cancer in the world. Therefore, prevention of colorectal tumors by pre-colorectal tumor examination has become a very important health examination worldwide. Colonoscopy can provide information on the location and appearance of colorectal polyps, enabling physicians to resect them before they develop into colorectal cancer, an effective colorectal cancer screening and prevention technique. Many studies have shown that early colonoscopy helps to reduce the incidence of CRC by 30%. Therefore, clinically, accurate polyp segmentation is very important. However, since polyps often differ in appearance, such as size, color, and texture, even if they are of the same type.
In existing colonoscopic image segmentation methods, the boundary between the polyp and its surrounding mucosa is often blurred and lacks the strong contrast required by the segmentation method. These problems lead to inaccurate segmentation of polyps and sometimes even to missed detection of polyps. Therefore, an automatic and accurate polyp segmentation method capable of finding all possible polyps at an early stage is of great significance for the prevention of colorectal cancer.
Disclosure of Invention
In order to solve the above problems in the prior art, the present invention provides an image segmentation method based on a reverse attention network, which greatly improves the precision of image segmentation by inputting high-level features obtained by image coding into the reverse attention network. The technical scheme of the invention is as follows:
an image segmentation method based on a reverse attention network comprises the following steps:
s1, acquiring an image data set, and constructing a training set and a test set;
s2, constructing a reverse attention network model, wherein the processing process of the reverse attention network model specifically comprises the following steps:
the training set image is sequentially coded by a plurality of convolutional layers layer by layer to obtain output characteristic layers f1, f2, f3, f4 and f5 of different layers, and the output characteristic layers f3, f4 and f5 of different layers are parallelly spliced and polymerized and then input to a decoder to be decoded to obtain a global characteristic image Sg;
after the global feature image Sg is subjected to downsampling operation for one time, the global feature image Sg and the output feature f5 are input into a first reverse attention network for processing, and an output feature R5 is obtained; the first reverse attention network processing process comprises the steps of carrying out region-of-interest inversion operation on the global feature image Sg, and multiplying the inverted image by the output feature f5 to obtain an output feature R5; fusing the output feature R5 and the global feature image Sg to obtain a reverse attention feature S5;
after the inverse attention feature S5 is subjected to an up-sampling operation for one time, the inverse attention feature and the output feature f4 are input into a second inverse attention network for processing, and an output feature R4 is obtained; the second reverse attention network processing procedure comprises the steps of carrying out region-of-interest inversion operation on the reverse attention feature S5, and multiplying the inverted image by the output feature f4 to obtain an output feature R4; fusing the output feature R4 and the reverse attention feature S5 to obtain a reverse attention feature S4;
after the reverse attention feature S4 is subjected to an up-sampling operation for one time, the reverse attention feature and the output feature f3 are input into a third reverse attention network for processing, and an output feature R3 is obtained; the third reverse attention network processing procedure comprises the steps of carrying out region-of-interest inversion operation on the reverse attention feature S4, and multiplying the inverted image by the output feature f3 to obtain an output feature R3; fusing the output feature R3 and the reverse attention feature S4 to obtain a reverse attention feature S3;
the reverse attention feature S3 obtains a processing result of the reverse attention network model after being activated by a Sigmoid function;
s3, inputting the training set into a reverse attention network model for training to obtain a trained reverse attention network model;
and S4, inputting the test set into the trained reverse attention network model to obtain an image segmentation result.
Further, the region of interest is a polyp region.
The invention has the beneficial effects that: the high-level features obtained by image coding can be input into the reverse attention network, so that the precision of image segmentation is greatly improved.
Drawings
FIG. 1 is a schematic structural diagram of an image segmentation method based on a reverse attention network according to the present invention;
FIG. 2 is a flow chart of the image segmentation method based on the reverse attention network of the present invention;
FIG. 3 is a schematic diagram of the structure of the reverse attention network of the present invention;
FIG. 4 is a schematic illustration of the present invention prior to a region of interest inversion operation;
FIG. 5 is a schematic illustration of the present invention after a region of interest inversion operation;
reference numerals: RA1 is the first reverse attention network, RA2 is the second reverse attention network, and RA3 is the first reverse attention network.
Detailed Description
The technical scheme of the invention is further described by combining the drawings and the embodiment:
the embodiment provides an image segmentation method based on a reverse attention network, as shown in fig. 2, including:
step one, acquiring an image data set, and constructing a training set and a testing set.
In the embodiment of the present application, an image data set is obtained from the standard data set Kvasir, and 80% of the image data in the obtained image data set is used as a training set and 20% of the image data is used as a test set, and the input is adjusted to 352 × 352.
Step two, constructing a reverse attention network model, wherein the processing process of the reverse attention network model specifically comprises the following steps:
the inverse attention network model is specifically to first roughly locate polyp regions and then accurately extract their contour templates from local features.
In this embodiment, as shown in fig. 1, the polyp training set image obtained in step 1 is sequentially subjected to layer-by-layer encoding by 5 convolutional layers to obtain output feature layers f1, f2, f3, f4, and f5 of different levels, where f1 and f2 are low-level features, f3, f4, and f5 are high-level features, and the output features f3, f4, and f5 of different levels are subjected to parallel stitching and aggregation, and then input to a decoder for decoding to obtain a global feature image Sg;
in the embodiment of the present application, the output characteristics generated by the reverse attention network are obtained by specifically using formula (1):
Ri＝fi×Ai (1)
wherein i is 3,4,5, fiAs high-level hierarchical features f3, f4 and f5, AiAttention is paid to the weight in reverse.
The specific reverse attention weight is given by:
Ai＝Θ(σ(p(Si+1))) (2)
where Θ represents the inverse operation of subtracting the input from the matrix E, all elements of the matrix E are 1, σ represents a Sigmoid function, p represents an upsampling operation, and S represents a reverse attention feature.
Specifically, in this embodiment, after a downsampling operation is performed on the global feature image Sg, the global feature image Sg and the output feature f5 are input into the first reverse attention network for processing, so as to obtain an output feature R5; the process of the first reverse attention network processing includes performing a region-of-interest reversing operation on the global feature image Sg, as shown in fig. 4, with the attention of the original polyp region (white region) being 0.9 and the attention of the non-polyp region (black region) being 0.1, obtaining the attention of the polyp region (white region) being 0.1 and the attention of the non-polyp region (black region) being 0.9 after performing the region-of-interest reversing operation, as shown in fig. 5, and multiplying the reversed image by the output feature f5 to obtain an output feature R5; fusing the output feature R5 and the global feature image Sg to obtain a reverse attention feature S5;
after the inverse attention feature S5 is subjected to an up-sampling operation for one time, the inverse attention feature and the output feature f4 are input into a second inverse attention network for processing, and an output feature R4 is obtained; the second reverse attention network processing procedure comprises the steps of carrying out region-of-interest inversion operation on the reverse attention feature S5, and multiplying the inverted image by the output feature f4 to obtain an output feature R4; fusing the output feature R4 and the reverse attention feature S5 to obtain a reverse attention feature S4;
after the reverse attention feature S4 is subjected to an up-sampling operation for one time, the reverse attention feature and the output feature f3 are input into a third reverse attention network for processing, and an output feature R3 is obtained; the third reverse attention network processing procedure comprises the steps of carrying out region-of-interest inversion operation on the reverse attention feature S4, and multiplying the inverted image by the output feature f3 to obtain an output feature R3; fusing the output feature R3 and the reverse attention feature S4 to obtain a reverse attention feature S3;
the reverse attention feature S3 obtains the processing result of the reverse attention network model after being activated by the Sigmoid function.
And step three, inputting the training set into a reverse attention network model for training to obtain a trained reverse attention network model.
In the embodiment of the present application, the loss function is specifically:
Ltotal＝LioU+LBCE (3)
wherein L isioURepresenting a weighted loss of global constraints, LBCERepresenting a locally weighted binary cross entropy loss.
Wherein G denotes the actual mask, SgRepresenting a global feature image, SiIndicating a reverse attention feature.
In this embodiment, the model is implemented in PyTorch, a multi-scale training strategy {0.75,1,1.25} is adopted, Adam optimization algorithm is used to optimize overall parameters, the learning rate is 1e-4, the whole network is trained in an end-to-end mode, and the batch size is 16.
And step four, inputting the test set into the trained reverse attention network model to obtain an image segmentation result.
The image segmentation method based on the reverse attention network realizes very high accuracy (the average dice on the Kvasir data set is 0.898), and no preprocessing or post-processing is needed, so that the image segmentation accuracy is greatly improved.
It is to be understood that the invention is not limited to the examples described above, but that modifications and variations may be effected thereto by those of ordinary skill in the art in light of the foregoing description, and that all such modifications and variations are intended to be within the scope of the invention as defined by the appended claims.
Claims (2)
1. An image segmentation method based on a reverse attention network is characterized by comprising the following steps:
s1, acquiring an image data set, and constructing a training set and a test set;
s2, constructing a reverse attention network model, wherein the processing process of the reverse attention network model specifically comprises the following steps:
the training set image is sequentially coded by a plurality of convolutional layers layer by layer to obtain output characteristic layers f1, f2, f3, f4 and f5 of different layers, and the output characteristic layers f3, f4 and f5 of the different layers are parallelly spliced and polymerized and then input to a decoder to be decoded to obtain a global characteristic image Sg;
after the global feature image Sg is subjected to downsampling operation for one time, the global feature image Sg and the output feature f5 are input into a first reverse attention network for processing, and an output feature R5 is obtained; the first reverse attention network processing process comprises the steps of carrying out region-of-interest inversion operation on the global feature image Sg, and multiplying the inverted image by the output feature f5 to obtain an output feature R5; fusing the output feature R5 and the global feature image Sg to obtain a reverse attention feature S5;
after the reverse attention feature S5 is subjected to primary up-sampling operation, the reverse attention feature S5 and the output feature f4 are input into a second reverse attention network for processing, and an output feature R4 is obtained; the second reverse attention network processing process comprises a region-of-interest inversion operation on the reverse attention feature S5, and multiplying the inverted image by the output feature f4 to obtain an output feature R4; fusing the output feature R4 and the reverse attention feature S5 to obtain a reverse attention feature S4;
after the reverse attention feature S4 is subjected to one-time up-sampling operation, the reverse attention feature S4 and the output feature f3 are input into a third reverse attention network for processing, and an output feature R3 is obtained; the third reverse attention network processing process comprises a region-of-interest inversion operation on the reverse attention feature S4, and multiplying the inverted image by the output feature f3 to obtain an output feature R3; fusing the output feature R3 and the reverse attention feature S4 to obtain a reverse attention feature S3;
the reverse attention feature S3 is activated through a Sigmoid function to obtain a processing result of a reverse attention network model;
s3, inputting the training set into the reverse attention network model for training to obtain a trained reverse attention network model;
and S4, inputting the test set into the trained reverse attention network model to obtain an image segmentation result.
2. The method of claim 1, wherein the region of interest is a polyp region.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110493747.5A CN113343995A (en) | 2021-05-07 | 2021-05-07 | Image segmentation method based on reverse attention network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110493747.5A CN113343995A (en) | 2021-05-07 | 2021-05-07 | Image segmentation method based on reverse attention network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113343995A true CN113343995A (en) | 2021-09-03 |
Family
ID=77469844
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110493747.5A Pending CN113343995A (en) | 2021-05-07 | 2021-05-07 | Image segmentation method based on reverse attention network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113343995A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114972155A (en) * | 2021-12-30 | 2022-08-30 | 昆明理工大学 | Polyp image segmentation method based on context information and reverse attention |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110189334A (en) * | 2019-05-28 | 2019-08-30 | 南京邮电大学 | The medical image cutting method of the full convolutional neural networks of residual error type based on attention mechanism |
| CN111681252A (en) * | 2020-05-30 | 2020-09-18 | 重庆邮电大学 | Medical image automatic segmentation method based on multipath attention fusion |
-
2021
- 2021-05-07 CN CN202110493747.5A patent/CN113343995A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110189334A (en) * | 2019-05-28 | 2019-08-30 | 南京邮电大学 | The medical image cutting method of the full convolutional neural networks of residual error type based on attention mechanism |
| CN111681252A (en) * | 2020-05-30 | 2020-09-18 | 重庆邮电大学 | Medical image automatic segmentation method based on multipath attention fusion |
Non-Patent Citations (1)
| Title |
|---|
| DENG-PING FAN等: "PraNet: Parallel Reverse Attention Network for Polyp Segmentation", 《INTERNATIONAL CONFERENCE ON MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION（MICCAI 2020）》, pages 263 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114972155A (en) * | 2021-12-30 | 2022-08-30 | 昆明理工大学 | Polyp image segmentation method based on context information and reverse attention |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Oda et al. | BESNet: boundary-enhanced segmentation of cells in histopathological images | |
| CN110889852B (en) | Liver segmentation method based on residual error-attention deep neural network | |
| WO2022100495A1 (en) | Method for automatically segmenting ground-glass pulmonary nodule and computer device | |
| CN115661144B (en) | Adaptive medical image segmentation method based on deformable U-Net | |
| CN111784671A (en) | Pathological image focus region detection method based on multi-scale deep learning | |
| CN111127482A (en) | CT image lung trachea segmentation method and system based on deep learning | |
| CN113674253A (en) | Rectal cancer CT image automatic segmentation method based on U-transducer | |
| CN112258488A (en) | Medical image focus segmentation method | |
| CN112396605B (en) | Network training method and device, image recognition method and electronic equipment | |
| CN110689525A (en) | Method and device for recognizing lymph nodes based on neural network | |
| CN111091575B (en) | Medical image segmentation method based on reinforcement learning method | |
| Ngo et al. | Single-image visibility restoration: A machine learning approach and its 4K-capable hardware accelerator | |
| CN113705675B (en) | Multi-focus image fusion method based on multi-scale feature interaction network | |
| CN117078930A (en) | Medical image segmentation method based on boundary sensing and attention mechanism | |
| CN115601330A (en) | Colonic polyp segmentation method based on multi-scale space reverse attention mechanism | |
| CN114689527A (en) | Rice chalkiness detection method and system | |
| CN111325671B (en) | Network training method and device, image processing method and electronic equipment | |
| CN113343995A (en) | Image segmentation method based on reverse attention network | |
| CN117197454A (en) | Liver and liver tumor data segmentation method and system | |
| CN116563189A (en) | Medical image cross-contrast synthesis method and system based on deep learning | |
| CN116597138A (en) | Polyp image semantic segmentation method based on depth convolution neural network | |
| Fan et al. | EGFNet: Efficient guided feature fusion network for skin cancer lesion segmentation | |
| CN114004795A (en) | Breast nodule segmentation method and related device | |
| CN117197156B (en) | Lesion segmentation method and system based on double decoders UNet and Transformer | |
| CN116071555B (en) | Method for establishing WMHs segmentation model, WMHs segmentation method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |