US20230110904A1 - Systems and methods for artifact reduction in tomosynthesis with deep learning image processing - Google Patents
Systems and methods for artifact reduction in tomosynthesis with deep learning image processing Download PDFInfo
- Publication number
- US20230110904A1 US20230110904A1 US17/759,737 US202117759737A US2023110904A1 US 20230110904 A1 US20230110904 A1 US 20230110904A1 US 202117759737 A US202117759737 A US 202117759737A US 2023110904 A1 US2023110904 A1 US 2023110904A1
- Authority
- US
- United States
- Prior art keywords
- unroll
- neural network
- tomosynthesis
- decoupled
- trained
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 230000009467 reduction Effects 0.000 title claims description 6
- 238000013135 deep learning Methods 0.000 title abstract description 15
- 238000012545 processing Methods 0.000 title description 5
- 210000000481 breast Anatomy 0.000 claims abstract description 44
- 238000013528 artificial neural network Methods 0.000 claims abstract description 30
- 238000012549 training Methods 0.000 claims description 37
- 208000004434 Calcinosis Diseases 0.000 claims description 15
- 238000013527 convolutional neural network Methods 0.000 claims description 14
- 238000005070 sampling Methods 0.000 claims description 12
- 238000002591 computed tomography Methods 0.000 claims description 8
- 238000003384 imaging method Methods 0.000 claims description 7
- 238000009607 mammography Methods 0.000 claims description 7
- 238000004458 analytical method Methods 0.000 claims description 5
- 230000006872 improvement Effects 0.000 abstract description 3
- 238000004891 communication Methods 0.000 description 34
- 230000002308 calcification Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 230000001413 cellular effect Effects 0.000 description 5
- 206010006187 Breast cancer Diseases 0.000 description 4
- 208000026310 Breast neoplasm Diseases 0.000 description 4
- 230000006835 compression Effects 0.000 description 4
- 238000007906 compression Methods 0.000 description 4
- 230000000762 glandular Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 238000012216 screening Methods 0.000 description 3
- 239000007787 solid Substances 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 206010028980 Neoplasm Diseases 0.000 description 1
- QXDMQSPYEZFLGF-UHFFFAOYSA-L calcium oxalate Chemical compound [Ca+2].[O-]C(=O)C([O-])=O QXDMQSPYEZFLGF-UHFFFAOYSA-L 0.000 description 1
- 201000011510 cancer Diseases 0.000 description 1
- 231100000504 carcinogenesis Toxicity 0.000 description 1
- 239000004020 conductor Substances 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000000873 masking effect Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000003325 tomography Methods 0.000 description 1
- 238000002604 ultrasonography Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
- A61B6/50—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment specially adapted for specific body parts; specially adapted for specific clinical applications
- A61B6/502—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment specially adapted for specific body parts; specially adapted for specific clinical applications for diagnosis of breast, i.e. mammography
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
- A61B6/02—Arrangements for diagnosis sequentially in different planes; Stereoscopic radiation diagnosis
- A61B6/025—Tomosynthesis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
- A61B6/52—Devices using data or image processing specially adapted for radiation diagnosis
- A61B6/5211—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data
- A61B6/5217—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data extracting a diagnostic or physiological parameter from medical diagnostic data
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
- A61B6/52—Devices using data or image processing specially adapted for radiation diagnosis
- A61B6/5258—Devices using data or image processing specially adapted for radiation diagnosis involving detection or reduction of artifacts or noise
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/60—Image enhancement or restoration using machine learning, e.g. neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10116—X-ray image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30068—Mammography; Breast
Definitions
- DBT Digital breast tomosynthesis
- Digital breast tomosynthesis provides 3D reconstruction which reduces the superposition and overlapping of breast tissues compared to mammography, leading to increased sensitivity and specificity.
- DBT Digital breast tomosynthesis
- most DBT system has large sampling interval ( ⁇ 3°) and a very limited angular sampling range of 50°.
- the large angular interval leads to sparse-view artifacts, where streaks due to high-contrast objects are visible in the DBT images.
- the artifacts cause ambiguity in some cases when they are overlapping with local structures.
- the obvious artifacts also lowered physician's confidence when making decision with the DBT images.
- DBT images also may suffer from limited in-depth resolution.
- 3D DBT also has very high resolution, which leads to huge training computational cost.
- a DBT reconstruction method to mitigate the limited angular artifacts and improve in-depth resolution while avoiding computational cost burdens.
- the present disclosure addresses the aforementioned drawbacks by providing systems and methods for a deep learning-based tomosynthesis reconstruction that mitigates the limited angular artifacts and improves in-depth resolution.
- the systems and methods can reduce the sparse-view artifacts in DBT via deep learning without clinically damaging image sharpness and contrast.
- a deep neural network may be trained, such as an unroll-type neural network, and may be used with a decoupled training for each unroll to reduce training-time computational cost.
- a method for artifact reduction in tomosynthesis imaging of a subject.
- the method includes accessing a decoupled trained, unroll-type neural network trained to suppress artifacts using decoupled training with image training data that include at least one artifact.
- the method also includes accessing tomosynthesis image data of the subject that includes a region of interest (ROI) and subjecting the tomosynthesis image data to the decoupled trained unroll-type neural network to reconstruct an image of the subject with suppressed artifacts in the region of interest.
- the method also includes displaying the image of the subject with suppressed artifacts.
- ROI region of interest
- a system for artifact reduction in tomosynthesis imaging of a subject.
- the system includes a computer system configured to access a decouple trained, unroll-type neural network trained to suppress artifacts using decoupled training with image training data that include at least one artifact.
- the computer system is also configured to access tomosynthesis image data of the subject that includes a region of interest and subject the tomosynthesis image data to the decoupled trained unroll-type neural network to reconstruct an image of the subject with suppressed artifacts in the region of interest.
- the computer system is also configured to display the image of the subject with the suppressed artifacts.
- FIG. 1 is a flow chart of non-limiting example steps for a method of tomosynthesis image reconstruction in accordance with the present disclosure.
- FIG. 2 is a non-limiting example convolutional neural network (CNN) architecture that may be used in accordance with the present disclosure.
- CNN convolutional neural network
- FIG. 3 is a block diagram of an example system that can implement a digital breast tomosynthesis reconstruction system for generating images of a subject using a machine learning or deep learning method.
- FIG. 4 is a block diagram of example hardware components of the system of FIG. 3 .
- FIG. 5 is a block diagram of a non-limiting example digital breast tomosynthesis (DBT) system.
- DBT digital breast tomosynthesis
- a deep learning-based tomosynthesis image reconstruction such as a digital breast tomosynthesis (DBT) image reconstruction
- DBT digital breast tomosynthesis
- the systems and methods may reduce the sparse-view artifacts in DBT via deep learning without losing image sharpness and contrast.
- a deep neural network may be trained, such as an unroll-type neural network, and may be used with a decoupled training for each unroll to reduce training-time computational cost.
- an ROI loss may be used for further improvement on the resolution and contrast of the images.
- a region of interest loss method on microcalcifications may be used to improve the spatial resolution and contrast of the regions of interest, such as on microcalcifications.
- the network was trained and tested on 176 realistic breast phantoms, and demonstrated improved in-plane contrast (3.17 versus 0.43, p ⁇ 0.01) and in-depth resolution (1.19 mm versus 4.96 mm, p ⁇ 0.01) as compared to an iterative reconstruction (IR) method.
- Tomosynthesis training data may be generated at step 102 .
- a neural network may be trained using the training data at step 104 .
- the resulting, trained neural network may be stored and accessed for later use.
- tomosynthesis image data of the subject may be accessed or otherwise acquired at step 106 .
- Tomosynthesis image data may include digital breast tomosynthesis image data.
- the trained neural network can be accessed and the tomosynthesis image data is subjected to the trained neural network at step 108 .
- An optional region of interest loss analysis may be performed on the tomosynthesis image data at step 110 . Images of the subject may be reconstructed from the image data that was subjected to the neural network and/or the ROI loss analysis to thereby produce, as will be described, images with suppressed artifacts.
- Training of a neural network may include using realistic breast phantoms or real patients' data.
- Numerical breast phantoms are a reliable choice for DBT simulation because of the extensive existing studies that have proven the performance of these phantoms.
- Real patients' data may yet be more precise in some aspects.
- both dense-view sampling e.g. 0.5°
- sparse-view sampling may be acquired from the scans.
- the ROIs may be annotated, such as by a human annotator. These ROIs may contain the sparse-view artifacts in the sparse-view reconstruction results.
- a deep neural network with an ROI loss may be trained to suppress the artifacts while keeping existing structures.
- a neural network is an unrolled network, which may solve the following optimization problem:
- x * argmin x ⁇ ⁇ Ax - b ⁇ w 2 + ⁇ ⁇ R ⁇ ( x ) ( 1 )
- x is the image to be reconstructed
- A is the system matrix
- b is the projection data
- w is the noise weighting matrix
- R(x) is the prior function
- p is the hyperparameter to balance between data fidelity and the prior knowledge.
- Equation (1) can be solved via proximal gradient descent:
- ⁇ is a suitable step size for the gradient descent.
- the decoupled unrolled network may replace proximal mapping with CNNs and the gradient descent step with separable quadratic surrogate (SQS).
- SQS is free of choosing step size ⁇ and has faster convergence.
- the SQS step may be noted as:
- the decoupled unrolled network can be expressed as:
- x (n) f ( g M ( x (n ⁇ 1) ), x (n ⁇ 1) ; ⁇ (n) ) (4)
- f (y, x; ⁇ (n) ) is a CNN with trainable parameters ⁇ (n) and input y and x concatenated along the channel direction.
- g M (x n ⁇ 1 ) means Mth iterations of SQS (3) being applied to input image x (n ⁇ 1) .
- the network was trained sequentially from the first unroll to the last unroll.
- the training at nth unroll may be expressed as:
- ⁇ ( n ) argmin ⁇ ⁇ ⁇ ⁇ f ⁇ ( g M ( x i ( n - 1 ) ) , x i ( n - 1 ) ; ⁇ ) - x i ref ⁇ ( 5 )
- x i ref is the label image of the ith training sample
- x i n ⁇ 1 is the output of trained networks for the ith sample.
- the decoupled training may only be optimized one OM at a time. That is, conventional training techniques suffer from large memory consumption issues when processing 3D volume data, whereas decoupled unrolled network training may train each iteration step separately to reduce memory costs.
- the feature map size may be significantly reduced, leading to less training memory and time requirement.
- the convolutional neural network is shown as a decoupled unrolled network 200 .
- the decoupled unrolled network 200 includes initial input 202 , initial separable quadratic surrogate 204 , first convolutional neural network level 206 , and initial training data 208 .
- the decoupled unrolled network 200 also includes a subsequent input 210 , a subsequent separable quadratic surrogate 212 , a subsequent convolutional neural network level 214 , and subsequent training data 216 .
- the decoupled unrolled network 200 also includes N ⁇ 1 input 218 , N ⁇ 1 separable quadratic surrogate 220 , N convolutional neural network level 222 and N training data 224 .
- the result from the decoupled unrolled network 200 is delivered via output 226 .
- microcalcifications could be early signs of breast cancer, and they usually have diameter of several hundred micrometers and significantly higher density compared to breast tissues. Because microcalcifications are extremely small compared to the whole volume, they may provide limited contributions to the training loss and may be ignored by the network during training in some cases. The micorcalcifications may be significantly different from the other breast tissues that a network was trained on, which may led to limited contrast and in-depth resolution improvement of microcalcifications compared to iterative reconstructions. In accordance with the present disclosure, these limitations may be addressed by including the ROI loss in addition to the whole-volume L2 loss. In some configurations, this may be reflected as:
- J is the total number of voxels of an image
- Jr is the total number of voxels of ROIs
- K is the number of ROIs of sample i
- E ik is the masking matrix for ROI k in sample i
- ⁇ is a hyperparameter to balance between whole-image loss and ROI loss
- a dataset consisted of 176 realistic breast phantoms derived from breast CT scans of healthy subjects.
- Each breast CT image was segmented into six classes including skin, adipose, glandular and 3 different mixtures of adipose and glandular.
- Finite element model was applied to the segmented images to simulate breast compression between 50 to 80 mm.
- the voxel size was downsampled from 0.25 ⁇ 0.25 ⁇ 0.25 mm 3 to 1 ⁇ 1 ⁇ 1 mm 3 . The downsampling did not interfere with the limited angular artifacts, because the artifacts mostly exists in low frequency.
- Calcification points were further inserted into phantoms with ⁇ from 1.011 cm ⁇ 1 to 3.034 cm ⁇ 1 , which was corresponding to 10% to 30% of the attenuation coefficient of calcium oxalate at 20 keV. Each calcification point only occupied one single voxel and each patient was randomly assigned 0 to 4 calcification points.
- the 10% to 30% scale factor was corresponding microcalcifications' diameter of 0.1 mm to 0.3 mm whereas our voxel size was 1 mm.
- a detector with pixel size of 0.5 ⁇ 0.5 mm 2 was used.
- the projecting geometry was configured where the source to detector distance was 660 mm, source to rotation center distance was 627 mm and the breast was placed 10 mm above the detector surface.
- the angular sampling was from ⁇ 21 ° to +21° with 3° interval, giving 15 views in total.
- Distance driven projector and backprojector were used for both simulation and reconstruction. Poisson noise was added to the projections assuming 105 initial photons per ray.
- Deep learning results gave significantly improved contrast for in-plane structures with less superposition.
- both axial and sagittal views showed a structure where deep learning achieved improved in-plane contrast and better in-depth resolution compared to IR.
- a calcification point was also zoomed in, and deep learning with ROI loss demonstrated significantly improved contrast compared to IR and L2 loss.
- there were obvious wave-like artifacts in the IR result which was caused by the sparse sampling. These artifacts were not present in the deep learning results.
- Root mean square errors (RMSE) and structural similarity indices (SSIM) were calculated for each testing images comparing to the ground truth. Both deep learning-based results had significantly improved metrics compared to IR (p ⁇ 0.01). Furthermore, ROI loss only led to slightly worse overall RMSEs and SSIMs compared to L2 loss. There was no substantial visual difference of the glandular reconstructed using the two losses in FIG. 2 .
- the contrast and spatial resolution were quantitatively analyzed at the calcification points. For each point, a Gaussian function was used to fit the profile along x, y and z directions in a 21-pixel window. The in-depth resolution was calculated as the full width at half maximum (FWHM) of the Gaussian function along z, and the in-plane resolution was taken as the average of FWHM along x and y direction.
- the Weber contrast was determined, which may be defined as
- Ipeak is the peak intensity as the peak of the Gaussian function
- Ibackground is the background intensity as the bias of the Gaussian function.
- the network may be scaled to larger volumes in applications without additional requirement on computational hardware.
- the trained networks may also be applied to real DBT data.
- the trained network was further applied to real DBT data from 15 patients and demonstrated improved in-plane contrast and in-depth resolution. In one of the 15 patients, the upper and lower borders of a concentrated mass can be identified in the network results, which was otherwise indistinguishable in the IR results.
- a computing device 350 can receive one or more types of data (e.g., digital breast tomography, x-ray, computed tomography, ultrasound, multiparametric MRI data, breast image data, and the like) from image source 302 .
- computing device 350 can execute at least a portion of a digital breast tomosynthesis reconstruction system 304 to generate images of a breast, or otherwise segment a region of interest from data received from the image source 302 .
- the computing device 350 can communicate information about data received from the image source 302 to a server 352 over a communication network 354 , which can execute at least a portion of the digital breast tomosynthesis reconstruction system 304 to generate images of a region of interest, or otherwise segment a region of interest from data received from the image source 302 .
- the server 352 can return information to the computing device 350 (and/or any other suitable computing device) indicative of an output of the digital breast tomosynthesis reconstruction system 304 to generate images of a region of interest, or otherwise segment a region of interest from data received from the image source 302 .
- computing device 350 and/or server 352 can be any suitable computing device or combination of devices, such as a desktop computer, a laptop computer, a smartphone, a tablet computer, a wearable computer, a server computer, a virtual machine being executed by a physical computing device, and so on.
- the computing device 350 and/or server 352 can also reconstruct images from the data.
- image source 302 can be any suitable source of image data (e.g., measurement data, images reconstructed from measurement data), such as a digital breast tomosynthesis system, another computing device (e.g., a server storing image data), and so on.
- image source 302 can be local to computing device 350 .
- image source 302 can be incorporated with computing device 350 (e.g., computing device 350 can be configured as part of a device for capturing, scanning, and/or storing images).
- image source 302 can be connected to computing device 350 by a cable, a direct wireless link, and so on.
- image source 302 can be located locally and/or remotely from computing device 350 , and can communicate data to computing device 350 (and/or server 352 ) via a communication network (e.g., communication network 354 ).
- a communication network e.g., communication network 354
- communication network 354 can be any suitable communication network or combination of communication networks.
- communication network 354 can include a Wi-Fi network (which can include one or more wireless routers, one or more switches, etc.), a peer-to-peer network (e.g., a Bluetooth network), a cellular network (e.g., a 3G network, a 4G network, etc., complying with any suitable standard, such as CDMA, GSM, LTE, LTE Advanced, WiMAX, etc.), a wired network, and so on.
- Wi-Fi network which can include one or more wireless routers, one or more switches, etc.
- peer-to-peer network e.g., a Bluetooth network
- a cellular network e.g., a 3G network, a 4G network, etc., complying with any suitable standard, such as CDMA, GSM, LTE, LTE Advanced, WiMAX, etc.
- communication network 108 can be a local area network, a wide area network, a public network (e.g., the Internet), a private or semi-private network (e.g., a corporate or university intranet), any other suitable type of network, or any suitable combination of networks.
- Communications links shown in FIG. 3 can each be any suitable communications link or combination of communications links, such as wired links, fiber optic links, Wi-Fi links, Bluetooth links, cellular links, and so on.
- computing device 350 can include a processor 402 , a display 404 , one or more inputs 406 , one or more communication systems 408 , and/or memory 410 .
- processor 402 can be any suitable hardware processor or combination of processors, such as a central processing unit (“CPU”), a graphics processing unit (“GPU”), and so on.
- display 404 can include any suitable display devices, such as a computer monitor, a touchscreen, a television, and so on.
- inputs 406 can include any suitable input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, and so on.
- communications systems 408 can include any suitable hardware, firmware, and/or software for communicating information over communication network 354 and/or any other suitable communication networks.
- communications systems 408 can include one or more transceivers, one or more communication chips and/or chip sets, and so on.
- communications systems 408 can include hardware, firmware and/or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.
- memory 410 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 402 to present content using display 404 , to communicate with server 352 via communications system(s) 408 , and so on.
- Memory 410 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof.
- memory 410 can include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on.
- memory 410 can have encoded thereon, or otherwise stored therein, a computer program for controlling operation of computing device 350 .
- processor 402 can execute at least a portion of the computer program to present content (e.g., images, user interfaces, graphics, tables), receive content from server 352 , transmit information to server 352 , and so on.
- server 352 can include a processor 412 , a display 414 , one or more inputs 416 , one or more communications systems 418 , and/or memory 420 .
- processor 412 can be any suitable hardware processor or combination of processors, such as a CPU, a GPU, and so on.
- display 414 can include any suitable display devices, such as a computer monitor, a touchscreen, a television, and so on.
- inputs 416 can include any suitable input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, and so on.
- communications systems 418 can include any suitable hardware, firmware, and/or software for communicating information over communication network 354 and/or any other suitable communication networks.
- communications systems 418 can include one or more transceivers, one or more communication chips and/or chip sets, and so on.
- communications systems 418 can include hardware, firmware and/or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.
- memory 420 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 412 to present content using display 414 , to communicate with one or more computing devices 350 , and so on.
- Memory 420 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof.
- memory 420 can include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on.
- memory 420 can have encoded thereon a server program for controlling operation of server 352 .
- processor 412 can execute at least a portion of the server program to transmit information and/or content (e.g., data, images, a user interface) to one or more computing devices 350 , receive information and/or content from one or more computing devices 350 , receive instructions from one or more devices (e.g., a personal computer, a laptop computer, a tablet computer, a smartphone), and so on.
- information and/or content e.g., data, images, a user interface
- processor 412 can execute at least a portion of the server program to transmit information and/or content (e.g., data, images, a user interface) to one or more computing devices 350 , receive information and/or content from one or more computing devices 350 , receive instructions from one or more devices (e.g., a personal computer, a laptop computer, a tablet computer, a smartphone), and so on.
- image source 302 can include a processor 422 , one or more image acquisition systems 424 , one or more communications systems 426 , and/or memory 428 .
- processor 422 can be any suitable hardware processor or combination of processors, such as a CPU, a GPU, and so on.
- the one or more image acquisition systems 424 are generally configured to acquire data, images, or both, and can include an RF transmission and reception subsystem of a digital breast tomosynthesis system.
- one or more image acquisition systems 424 can include any suitable hardware, firmware, and/or software for coupling to and/or controlling operations of an digital breast tomosynthesis system.
- one or more portions of the one or more image acquisition systems 424 can be removable and/or replaceable.
- image source 302 can include any suitable inputs and/or outputs.
- image source 302 can include input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, a trackpad, a trackball, and so on.
- image source 302 can include any suitable display devices, such as a computer monitor, a touchscreen, a television, etc., one or more speakers, and so on.
- communications systems 426 can include any suitable hardware, firmware, and/or software for communicating information to computing device 350 (and, in some embodiments, over communication network 354 and/or any other suitable communication networks).
- communications systems 426 can include one or more transceivers, one or more communication chips and/or chip sets, and so on.
- communications systems 426 can include hardware, firmware and/or software that can be used to establish a wired connection using any suitable port and/or communication standard (e.g., VGA, DVI video, USB, RS-232, etc.), Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.
- memory 428 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 422 to control the one or more image acquisition systems 424 , and/or receive data from the one or more image acquisition systems 424 ; to images from data; present content (e.g., images, a user interface) using a display; communicate with one or more computing devices 350 ; and so on.
- Memory 428 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof.
- memory 428 can include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on.
- memory 428 can have encoded thereon, or otherwise stored therein, a program for controlling operation of image source 302 .
- processor 422 can execute at least a portion of the program to generate images, transmit information and/or content (e.g., data, images) to one or more computing devices 350 , receive information and/or content from one or more computing devices 350 , receive instructions from one or more devices (e.g., a personal computer, a laptop computer, a tablet computer, a smartphone, etc.), and so on.
- any suitable computer readable media can be used for storing instructions for performing the functions and/or processes described herein.
- computer readable media can be transitory or non-transitory.
- non-transitory computer readable media can include media such as magnetic media (e.g., hard disks, floppy disks), optical media (e.g., compact discs, digital video discs, Blu-ray discs), semiconductor media (e.g., random access memory (“RAM”), flash memory, electrically programmable read only memory (“EPROM”), electrically erasable programmable read only memory (“EEPROM”)), any suitable media that is not fleeting or devoid of any semblance of permanence during transmission, and/or any suitable tangible media.
- RAM random access memory
- EPROM electrically programmable read only memory
- EEPROM electrically erasable programmable read only memory
- transitory computer readable media can include signals on networks, in wires, conductors, optical fibers, circuits, or any suitable media that is fleeting and devoid of any semblance of permanence during transmission, and/or any suitable intangible media.
- DBT system 500 may include a drive plate 518 configured to provide compression against a compression plate 516 for an organ disposed between drive plate 518 and compression plate 516 .
- Imaging detectors 512 may be coupled to a mounting system 514 .
- Radiation source 508 may be coupled to support system 502 and coupled to base 504 with rotatable coupler 506 .
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Medical Informatics (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- Optics & Photonics (AREA)
- Pathology (AREA)
- High Energy & Nuclear Physics (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Veterinary Medicine (AREA)
- Public Health (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Dentistry (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Quality & Reliability (AREA)
- Physiology (AREA)
- Apparatus For Radiation Diagnosis (AREA)
Abstract
Systems and methods are provided for a deep learning-based digital breast tomosynthesis (DBT) image reconstruction that mitigates limited angular artifacts and improves in-depth resolution of the resulting images. The systems and methods may reduce the sparse-view artifacts in DBT via deep learning without losing image sharpness and contrast. A deep neural network may be trained in a way to reduce training-time computational cost. An ROI loss method may be used for further improvement on the resolution and contrast of the images.
Description
- This application claims the benefit of U.S. Provisional patent application Ser. No. 62/968,589 filed on Jan. 31, 2020 and entitled “System and Method of Sparse-View Artifacts Reduction in Digital Breast Tomosynthesis with Deep-Learning-Based Image Processing,” which is incorporated herein by reference as if set forth in its entirety for all purposes.
- N/A
- Breast cancer is the second most common cancer in women. It is also the second leading cause of cancer death among American women. Evidence has shown that screening and early detection is the key to prevent breast cancer death. Screening is conventionally performed using mammography, which provides for 2D imaging, but the breast tissue depicted may be overlapping due to a single view angle. Digital breast tomosynthesis (DBT) is an emerging, promising new technology for breast cancer screening imaging that takes several images from different angles, which may be from different depths that may be stacked to form a 3D image. Compared to the conventional X-ray mammography, DBT has significantly improved sensitivity and specificity. Meanwhile, DBT has very similar hardware setup with mammography, which lowers the cost for both manufacturers and users to upgrade from mammography to DBT.
- Digital breast tomosynthesis (DBT) provides 3D reconstruction which reduces the superposition and overlapping of breast tissues compared to mammography, leading to increased sensitivity and specificity. However, to save dose and scanning time, most DBT system has large sampling interval (≤3°) and a very limited angular sampling range of 50°. The large angular interval leads to sparse-view artifacts, where streaks due to high-contrast objects are visible in the DBT images. The artifacts cause ambiguity in some cases when they are overlapping with local structures. The obvious artifacts also lowered physician's confidence when making decision with the DBT images. DBT images also may suffer from limited in-depth resolution.
- 3D DBT also has very high resolution, which leads to huge training computational cost. Thus, there remains a need for a DBT reconstruction method to mitigate the limited angular artifacts and improve in-depth resolution while avoiding computational cost burdens.
- The present disclosure addresses the aforementioned drawbacks by providing systems and methods for a deep learning-based tomosynthesis reconstruction that mitigates the limited angular artifacts and improves in-depth resolution. The systems and methods can reduce the sparse-view artifacts in DBT via deep learning without clinically damaging image sharpness and contrast. A deep neural network may be trained, such as an unroll-type neural network, and may be used with a decoupled training for each unroll to reduce training-time computational cost.
- In one configuration, a method is provided for artifact reduction in tomosynthesis imaging of a subject. The method includes accessing a decoupled trained, unroll-type neural network trained to suppress artifacts using decoupled training with image training data that include at least one artifact. The method also includes accessing tomosynthesis image data of the subject that includes a region of interest (ROI) and subjecting the tomosynthesis image data to the decoupled trained unroll-type neural network to reconstruct an image of the subject with suppressed artifacts in the region of interest. The method also includes displaying the image of the subject with suppressed artifacts.
- In one configuration, a system is provided for artifact reduction in tomosynthesis imaging of a subject. The system includes a computer system configured to access a decouple trained, unroll-type neural network trained to suppress artifacts using decoupled training with image training data that include at least one artifact. The computer system is also configured to access tomosynthesis image data of the subject that includes a region of interest and subject the tomosynthesis image data to the decoupled trained unroll-type neural network to reconstruct an image of the subject with suppressed artifacts in the region of interest. The computer system is also configured to display the image of the subject with the suppressed artifacts.
- The foregoing and other aspects and advantages of the present disclosure will appear from the following description. In the description, reference is made to the accompanying drawings that form a part hereof, and in which there is shown by way of illustration a preferred embodiment. This embodiment does not necessarily represent the full scope of the invention, however, and reference is therefore made to the claims and herein for interpreting the scope of the invention. Like reference numerals will be used to refer to like parts from Figure to Figure in the following description.
-
FIG. 1 is a flow chart of non-limiting example steps for a method of tomosynthesis image reconstruction in accordance with the present disclosure. -
FIG. 2 is a non-limiting example convolutional neural network (CNN) architecture that may be used in accordance with the present disclosure. -
FIG. 3 is a block diagram of an example system that can implement a digital breast tomosynthesis reconstruction system for generating images of a subject using a machine learning or deep learning method. -
FIG. 4 is a block diagram of example hardware components of the system ofFIG. 3 . -
FIG. 5 is a block diagram of a non-limiting example digital breast tomosynthesis (DBT) system. - Systems and methods are provided for a deep learning-based tomosynthesis image reconstruction, such as a digital breast tomosynthesis (DBT) image reconstruction, that mitigates limited angular artifacts and improves in-depth resolution of the resulting images. The systems and methods may reduce the sparse-view artifacts in DBT via deep learning without losing image sharpness and contrast. A deep neural network may be trained, such as an unroll-type neural network, and may be used with a decoupled training for each unroll to reduce training-time computational cost. In some configurations, an ROI loss may be used for further improvement on the resolution and contrast of the images.
- In some configurations, a region of interest loss method on microcalcifications may be used to improve the spatial resolution and contrast of the regions of interest, such as on microcalcifications. In a non-limiting example, the network was trained and tested on 176 realistic breast phantoms, and demonstrated improved in-plane contrast (3.17 versus 0.43, p<0.01) and in-depth resolution (1.19 mm versus 4.96 mm, p<0.01) as compared to an iterative reconstruction (IR) method.
- Referring to
FIG. 1 , a flow chart of non-limiting example steps for a method of tomosynthesis image reconstruction is shown. Tomosynthesis training data may be generated atstep 102. A neural network may be trained using the training data atstep 104. The resulting, trained neural network may be stored and accessed for later use. - When images of a particular subject are needed, tomosynthesis image data of the subject may be accessed or otherwise acquired at
step 106. Tomosynthesis image data may include digital breast tomosynthesis image data. Then, the trained neural network can be accessed and the tomosynthesis image data is subjected to the trained neural network atstep 108. An optional region of interest loss analysis may be performed on the tomosynthesis image data atstep 110. Images of the subject may be reconstructed from the image data that was subjected to the neural network and/or the ROI loss analysis to thereby produce, as will be described, images with suppressed artifacts. - Training of a neural network may include using realistic breast phantoms or real patients' data. Numerical breast phantoms are a reliable choice for DBT simulation because of the extensive existing studies that have proven the performance of these phantoms. Real patients' data may yet be more precise in some aspects. When training a neural network, both dense-view sampling (e.g. 0.5°) and sparse-view sampling may be acquired from the scans. In some configurations, the ROIs may be annotated, such as by a human annotator. These ROIs may contain the sparse-view artifacts in the sparse-view reconstruction results. A deep neural network with an ROI loss may be trained to suppress the artifacts while keeping existing structures.
- In one configuration, a neural network is an unrolled network, which may solve the following optimization problem:
-
- where x is the image to be reconstructed, A is the system matrix, b is the projection data, w is the noise weighting matrix, R(x) is the prior function, and p is the hyperparameter to balance between data fidelity and the prior knowledge.
- Equation (1) can be solved via proximal gradient descent:
-
x n=proxγβR {x (n−1) −γA T w(Ax n−1)−b)} (2) - where γ is a suitable step size for the gradient descent. The decoupled unrolled network may replace proximal mapping with CNNs and the gradient descent step with separable quadratic surrogate (SQS). The SQS is free of choosing step size γ and has faster convergence. The SQS step may be noted as:
-
- The decoupled unrolled network can be expressed as:
-
x (n) =f(g M(x (n−1)),x (n−1);Θ(n)) (4) - where f (y, x; Θ(n)) is a CNN with trainable parameters Θ(n) and input y and x concatenated along the channel direction. gM (xn−1) means Mth iterations of SQS (3) being applied to input image x(n−1). The network was trained sequentially from the first unroll to the last unroll. The training at nth unroll may be expressed as:
-
- where xi ref is the label image of the ith training sample, and xi n−1 is the output of trained networks for the ith sample. Compared to most unrolled networks which need simultaneously optimization of Θ(1) to Θ(N), the decoupled training may only be optimized one OM at a time. That is, conventional training techniques suffer from large memory consumption issues when processing 3D volume data, whereas decoupled unrolled network training may train each iteration step separately to reduce memory costs. The feature map size may be significantly reduced, leading to less training memory and time requirement.
- Referring to
FIG. 2 , a non-limiting example convolutional neural network architecture in accordance with the present disclosure is shown. The convolutional neural network is shown as a decoupled unrollednetwork 200. The decoupled unrollednetwork 200 includesinitial input 202, initial separablequadratic surrogate 204, first convolutionalneural network level 206, andinitial training data 208. The decoupled unrollednetwork 200 also includes asubsequent input 210, a subsequent separablequadratic surrogate 212, a subsequent convolutionalneural network level 214, andsubsequent training data 216. This framework extends such that the decoupled unrollednetwork 200 also includes N−1input 218, N−1 separablequadratic surrogate 220, N convolutionalneural network level 222 andN training data 224. The result from the decoupled unrollednetwork 200 is delivered viaoutput 226. - Non-Limiting Example Region of Interest Loss
- Detection of microcalcification is one non-limiting example application of DBT. Microcalcifications could be early signs of breast cancer, and they usually have diameter of several hundred micrometers and significantly higher density compared to breast tissues. Because microcalcifications are extremely small compared to the whole volume, they may provide limited contributions to the training loss and may be ignored by the network during training in some cases. The micorcalcifications may be significantly different from the other breast tissues that a network was trained on, which may led to limited contrast and in-depth resolution improvement of microcalcifications compared to iterative reconstructions. In accordance with the present disclosure, these limitations may be addressed by including the ROI loss in addition to the whole-volume L2 loss. In some configurations, this may be reflected as:
-
- where J is the total number of voxels of an image; Jr is the total number of voxels of ROIs; K, is the number of ROIs of sample i; Eik is the masking matrix for ROI k in sample i; λ is a hyperparameter to balance between whole-image loss and ROI loss
-
f i (n−1)(Θ)=f(g M(x i (n−1)),x i (n−1);Θ)as in equation (5). - Non-Limiting Example Breast Phantom Application
- In a non-limiting example, a dataset consisted of 176 realistic breast phantoms derived from breast CT scans of healthy subjects. Each breast CT image was segmented into six classes including skin, adipose, glandular and 3 different mixtures of adipose and glandular. Finite element model was applied to the segmented images to simulate breast compression between 50 to 80 mm. We assigned μ=0.456 cm−1 for adipose and μ=0.802 cm−1 for glandular and skin. For faster computation, the voxel size was downsampled from 0.25×0.25×0.25 mm3 to 1×1×1 mm3. The downsampling did not interfere with the limited angular artifacts, because the artifacts mostly exists in low frequency.
- Calcification points were further inserted into phantoms with μ from 1.011 cm−1 to 3.034 cm−1, which was corresponding to 10% to 30% of the attenuation coefficient of calcium oxalate at 20 keV. Each calcification point only occupied one single voxel and each patient was randomly assigned 0 to 4 calcification points. The 10% to 30% scale factor was corresponding microcalcifications' diameter of 0.1 mm to 0.3 mm whereas our voxel size was 1 mm.
- A detector with pixel size of 0.5×0.5 mm2 was used. The projecting geometry was configured where the source to detector distance was 660 mm, source to rotation center distance was 627 mm and the breast was placed 10 mm above the detector surface. The angular sampling was from −21° to +21° with 3° interval, giving 15 views in total. Distance driven projector and backprojector were used for both simulation and reconstruction. Poisson noise was added to the projections assuming 105 initial photons per ray.
- In the non-limiting example, 140 patients were randomly selected for training whereas the remaining 36 patients were used for testing. UNet with depth of 4 was used as the CNN in each unroll. An unroll number N=10 and SQS iterations M=10 were used. The initial image x(0) was 10 SQS iterations from zero images. Each unroll was trained by Adam optimizer with learning rate of 10−3 for 150 epochs. The entire training took approximately 12 hours. For the ROI loss, 9×9×9 ROIs were selected around each inserted calcification point with λ set to 0.1. IR results from 20 iterations of SQS from zero were used for comparison. The number of iterations balanced between noise level and artifacts. The thicknesses of the breast were assumed to be known for both IR and deep learning to further reduce limited angular artifacts. Thickness can be measured by the compressor of DBT scanners.
- Deep learning results gave significantly improved contrast for in-plane structures with less superposition. In the non-limiting example, both axial and sagittal views showed a structure where deep learning achieved improved in-plane contrast and better in-depth resolution compared to IR. A calcification point was also zoomed in, and deep learning with ROI loss demonstrated significantly improved contrast compared to IR and L2 loss. In addition, there were obvious wave-like artifacts in the IR result which was caused by the sparse sampling. These artifacts were not present in the deep learning results.
- Root mean square errors (RMSE) and structural similarity indices (SSIM) were calculated for each testing images comparing to the ground truth. Both deep learning-based results had significantly improved metrics compared to IR (p<0.01). Furthermore, ROI loss only led to slightly worse overall RMSEs and SSIMs compared to L2 loss. There was no substantial visual difference of the glandular reconstructed using the two losses in
FIG. 2 . - The contrast and spatial resolution were quantitatively analyzed at the calcification points. For each point, a Gaussian function was used to fit the profile along x, y and z directions in a 21-pixel window. The in-depth resolution was calculated as the full width at half maximum (FWHM) of the Gaussian function along z, and the in-plane resolution was taken as the average of FWHM along x and y direction. The Weber contrast was determined, which may be defined as
-
- where Ipeak is the peak intensity as the peak of the Gaussian function, and Ibackground is the background intensity as the bias of the Gaussian function. ROI loss significantly improved resolution and contrast over L2 loss and IR (p<0.01).
- In a non-limiting example, downsampled phantoms were used, but the network may be scaled to larger volumes in applications without additional requirement on computational hardware. The trained networks may also be applied to real DBT data. In a non-limiting example, the trained network was further applied to real DBT data from 15 patients and demonstrated improved in-plane contrast and in-depth resolution. In one of the 15 patients, the upper and lower borders of a concentrated mass can be identified in the network results, which was otherwise indistinguishable in the IR results.
- Referring to
FIG. 3 , an example of asystem 300 for generating and implementing a machine learning or deep learning routine in accordance with some embodiments of the systems and methods described in the present disclosure is shown. As shown inFIG. 3 , acomputing device 350 can receive one or more types of data (e.g., digital breast tomography, x-ray, computed tomography, ultrasound, multiparametric MRI data, breast image data, and the like) fromimage source 302. In some embodiments,computing device 350 can execute at least a portion of a digital breasttomosynthesis reconstruction system 304 to generate images of a breast, or otherwise segment a region of interest from data received from theimage source 302. - Additionally or alternatively, in some embodiments, the
computing device 350 can communicate information about data received from theimage source 302 to aserver 352 over acommunication network 354, which can execute at least a portion of the digital breasttomosynthesis reconstruction system 304 to generate images of a region of interest, or otherwise segment a region of interest from data received from theimage source 302. In such embodiments, theserver 352 can return information to the computing device 350 (and/or any other suitable computing device) indicative of an output of the digital breasttomosynthesis reconstruction system 304 to generate images of a region of interest, or otherwise segment a region of interest from data received from theimage source 302. - In some embodiments,
computing device 350 and/orserver 352 can be any suitable computing device or combination of devices, such as a desktop computer, a laptop computer, a smartphone, a tablet computer, a wearable computer, a server computer, a virtual machine being executed by a physical computing device, and so on. Thecomputing device 350 and/orserver 352 can also reconstruct images from the data. - In some embodiments,
image source 302 can be any suitable source of image data (e.g., measurement data, images reconstructed from measurement data), such as a digital breast tomosynthesis system, another computing device (e.g., a server storing image data), and so on. In some embodiments,image source 302 can be local tocomputing device 350. For example,image source 302 can be incorporated with computing device 350 (e.g.,computing device 350 can be configured as part of a device for capturing, scanning, and/or storing images). As another example,image source 302 can be connected tocomputing device 350 by a cable, a direct wireless link, and so on. Additionally or alternatively, in some embodiments,image source 302 can be located locally and/or remotely fromcomputing device 350, and can communicate data to computing device 350 (and/or server 352) via a communication network (e.g., communication network 354). - In some embodiments,
communication network 354 can be any suitable communication network or combination of communication networks. For example,communication network 354 can include a Wi-Fi network (which can include one or more wireless routers, one or more switches, etc.), a peer-to-peer network (e.g., a Bluetooth network), a cellular network (e.g., a 3G network, a 4G network, etc., complying with any suitable standard, such as CDMA, GSM, LTE, LTE Advanced, WiMAX, etc.), a wired network, and so on. In some embodiments,communication network 108 can be a local area network, a wide area network, a public network (e.g., the Internet), a private or semi-private network (e.g., a corporate or university intranet), any other suitable type of network, or any suitable combination of networks. Communications links shown inFIG. 3 can each be any suitable communications link or combination of communications links, such as wired links, fiber optic links, Wi-Fi links, Bluetooth links, cellular links, and so on. - Referring now to
FIG. 4 , an example ofhardware 400 that can be used to implementimage source 302,computing device 350, andserver 354 in accordance with some embodiments of the systems and methods described in the present disclosure is shown. As shown inFIG. 4 , in some embodiments,computing device 350 can include aprocessor 402, adisplay 404, one ormore inputs 406, one ormore communication systems 408, and/ormemory 410. In some embodiments,processor 402 can be any suitable hardware processor or combination of processors, such as a central processing unit (“CPU”), a graphics processing unit (“GPU”), and so on. In some embodiments,display 404 can include any suitable display devices, such as a computer monitor, a touchscreen, a television, and so on. In some embodiments,inputs 406 can include any suitable input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, and so on. - In some embodiments,
communications systems 408 can include any suitable hardware, firmware, and/or software for communicating information overcommunication network 354 and/or any other suitable communication networks. For example,communications systems 408 can include one or more transceivers, one or more communication chips and/or chip sets, and so on. In a more particular example,communications systems 408 can include hardware, firmware and/or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on. - In some embodiments,
memory 410 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, byprocessor 402 to presentcontent using display 404, to communicate withserver 352 via communications system(s) 408, and so on.Memory 410 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof. For example,memory 410 can include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. In some embodiments,memory 410 can have encoded thereon, or otherwise stored therein, a computer program for controlling operation ofcomputing device 350. In such embodiments,processor 402 can execute at least a portion of the computer program to present content (e.g., images, user interfaces, graphics, tables), receive content fromserver 352, transmit information toserver 352, and so on. - In some embodiments,
server 352 can include aprocessor 412, adisplay 414, one ormore inputs 416, one ormore communications systems 418, and/ormemory 420. In some embodiments,processor 412 can be any suitable hardware processor or combination of processors, such as a CPU, a GPU, and so on. In some embodiments,display 414 can include any suitable display devices, such as a computer monitor, a touchscreen, a television, and so on. In some embodiments,inputs 416 can include any suitable input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, and so on. - In some embodiments,
communications systems 418 can include any suitable hardware, firmware, and/or software for communicating information overcommunication network 354 and/or any other suitable communication networks. For example,communications systems 418 can include one or more transceivers, one or more communication chips and/or chip sets, and so on. In a more particular example,communications systems 418 can include hardware, firmware and/or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on. - In some embodiments,
memory 420 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, byprocessor 412 to presentcontent using display 414, to communicate with one ormore computing devices 350, and so on.Memory 420 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof. For example,memory 420 can include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. In some embodiments,memory 420 can have encoded thereon a server program for controlling operation ofserver 352. In such embodiments,processor 412 can execute at least a portion of the server program to transmit information and/or content (e.g., data, images, a user interface) to one ormore computing devices 350, receive information and/or content from one ormore computing devices 350, receive instructions from one or more devices (e.g., a personal computer, a laptop computer, a tablet computer, a smartphone), and so on. - In some embodiments,
image source 302 can include aprocessor 422, one or moreimage acquisition systems 424, one ormore communications systems 426, and/ormemory 428. In some embodiments,processor 422 can be any suitable hardware processor or combination of processors, such as a CPU, a GPU, and so on. In some embodiments, the one or moreimage acquisition systems 424 are generally configured to acquire data, images, or both, and can include an RF transmission and reception subsystem of a digital breast tomosynthesis system. Additionally or alternatively, in some embodiments, one or moreimage acquisition systems 424 can include any suitable hardware, firmware, and/or software for coupling to and/or controlling operations of an digital breast tomosynthesis system. In some embodiments, one or more portions of the one or moreimage acquisition systems 424 can be removable and/or replaceable. - Note that, although not shown,
image source 302 can include any suitable inputs and/or outputs. For example,image source 302 can include input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, a trackpad, a trackball, and so on. As another example,image source 302 can include any suitable display devices, such as a computer monitor, a touchscreen, a television, etc., one or more speakers, and so on. - In some embodiments,
communications systems 426 can include any suitable hardware, firmware, and/or software for communicating information to computing device 350 (and, in some embodiments, overcommunication network 354 and/or any other suitable communication networks). For example,communications systems 426 can include one or more transceivers, one or more communication chips and/or chip sets, and so on. In a more particular example,communications systems 426 can include hardware, firmware and/or software that can be used to establish a wired connection using any suitable port and/or communication standard (e.g., VGA, DVI video, USB, RS-232, etc.), Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on. - In some embodiments,
memory 428 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, byprocessor 422 to control the one or moreimage acquisition systems 424, and/or receive data from the one or moreimage acquisition systems 424; to images from data; present content (e.g., images, a user interface) using a display; communicate with one ormore computing devices 350; and so on.Memory 428 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof. For example,memory 428 can include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. In some embodiments,memory 428 can have encoded thereon, or otherwise stored therein, a program for controlling operation ofimage source 302. In such embodiments,processor 422 can execute at least a portion of the program to generate images, transmit information and/or content (e.g., data, images) to one ormore computing devices 350, receive information and/or content from one ormore computing devices 350, receive instructions from one or more devices (e.g., a personal computer, a laptop computer, a tablet computer, a smartphone, etc.), and so on. - In some embodiments, any suitable computer readable media can be used for storing instructions for performing the functions and/or processes described herein. For example, in some embodiments, computer readable media can be transitory or non-transitory. For example, non-transitory computer readable media can include media such as magnetic media (e.g., hard disks, floppy disks), optical media (e.g., compact discs, digital video discs, Blu-ray discs), semiconductor media (e.g., random access memory (“RAM”), flash memory, electrically programmable read only memory (“EPROM”), electrically erasable programmable read only memory (“EEPROM”)), any suitable media that is not fleeting or devoid of any semblance of permanence during transmission, and/or any suitable tangible media. As another example, transitory computer readable media can include signals on networks, in wires, conductors, optical fibers, circuits, or any suitable media that is fleeting and devoid of any semblance of permanence during transmission, and/or any suitable intangible media.
- Referring to
FIG. 5 , a non-limiting example digital breast tomosynthesis (DBT)system 500 is shown that may be used in accordance with the present disclosure.DBT system 500 may include adrive plate 518 configured to provide compression against acompression plate 516 for an organ disposed betweendrive plate 518 andcompression plate 516.Imaging detectors 512 may be coupled to a mountingsystem 514.Radiation source 508 may be coupled to supportsystem 502 and coupled tobase 504 withrotatable coupler 506. - The present disclosure has described one or more preferred embodiments, and it should be appreciated that many equivalents, alternatives, variations, and modifications, aside from those expressly stated, are possible and within the scope of the invention.
Claims (23)
1. A method for artifact reduction in tomosynthesis imaging of a subject, the method comprising:
a) accessing a decoupled trained, unroll-type neural network trained to suppress artifacts using decoupled training with image training data that include at least one artifact;
b) accessing tomosynthesis image data of the subject that includes a region of interest;
c) subjecting the tomosynthesis image data to the decoupled trained unroll-type neural network to reconstruct an image of the subject with suppressed artifacts in the region of interest;
d) displaying the image of the subject with the suppressed artifacts.
2. The method of claim 1 , wherein decoupled training includes optimizing one training parameter at a time.
3. The method of claim 2 , wherein the decoupled trained unroll-type neural network is a convolutional neural network (CNN) with separable quadratic surrogate.
4. The method of claim 3 , wherein the tomosynthesis image data is subjected to the decoupled trained unroll-type neural network over a plurality of iterations.
5. The method of claim 1 , wherein the unroll-type neural network is trained sequentially from a first unroll to a last unroll.
6. The method of claim 5 , wherein the image training data include dense-view sampling data and sparse-view sampling data.
7. The method of claim 6 , wherein the sparse-view sampling data includes sparse-view artifacts, and wherein the at least one artifact included with the image training data is the sparse-view artifact.
8. The method of claim 1 , wherein the tomosynthesis image data includes digital breast tomosynthesis image data.
9. The method of claim 8 , wherein the digital breast tomosynthesis image data includes microcalcifications.
10. The method of claim 1 , further comprising performing a region of interest loss analysis that applies a region-of-interest loss.
11. The method of claim 1 , wherein the image training data includes image data of realistic numerical breast phantoms generated from data acquired with at least one of digital breast tomosynthesis, mammography, computed tomography (CT), breast CT, magnetic resonance (MR) or breast MR.
12. A system for artifact reduction in tomosynthesis imaging of a subject, the system comprising:
a computer system configured to:
i) access a decoupled trained, unroll-type neural network trained to suppress artifacts using decoupled training with image training data that include at least one artifact;
ii) access tomosynthesis image data of the subject that includes a region of interest;
iii) subject the tomosynthesis image data to the decoupled trained unroll-type neural network to reconstruct an image of the subject with suppressed artifacts in the region of interest;
iv) display the image of the subject with the suppressed artifacts.
13. The system of claim 12 , wherein the computer system is further configured to decoupled train the unroll-type neural network by optimizing one training parameter at a time.
14. The system of claim 13 , wherein the decoupled trained unroll-type neural network is a convolutional neural network (CNN) with separable quadratic surrogate.
15. The system of claim 14 , wherein the computer system is further configured to subject the tomosynthesis image data to the decoupled trained unroll-type neural network over a plurality of iterations.
16. The system of claim 12 , wherein the computer system is further configured to train the unroll-type neural network sequentially from a first unroll to a last unroll.
17. The system of claim 16 , wherein the image training data include dense-view sampling data and sparse-view sampling data.
18. The system of claim 17 , wherein the sparse-view sampling data includes sparse-view artifacts, and wherein the at least one artifact included with the image training data is the sparse-view artifact.
19. The system of claim 12 , wherein the tomosynthesis image data includes digital breast tomosynthesis image data.
20. The system of claim 19 , wherein the digital breast tomosynthesis image data includes microcalcifications.
21. The system of claim 12 , wherein the computer system is further configured to perform a region of interest loss analysis.
22. The system of claim 21 , wherein the region of interest loss analysis applies a region of interest loss that is separate from a whole-volume L2 loss applied by the decoupled trained unroll-type neural network.
23. The method of claim 12 , wherein the image training data includes image data of realistic numerical breast phantoms generated from data acquired with at least one of digital breast tomosynthesis, mammography, computed tomography (CT), breast CT, magnetic resonance (MR) or breast MR.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US17/759,737 US20230110904A1 (en) | 2020-01-31 | 2021-01-29 | Systems and methods for artifact reduction in tomosynthesis with deep learning image processing |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US202062968589P | 2020-01-31 | 2020-01-31 | |
US17/759,737 US20230110904A1 (en) | 2020-01-31 | 2021-01-29 | Systems and methods for artifact reduction in tomosynthesis with deep learning image processing |
PCT/US2021/015677 WO2021155123A1 (en) | 2020-01-31 | 2021-01-29 | Systems and methods for artifact reduction in tomosynthesis with deep learning image processing |
Publications (1)
Publication Number | Publication Date |
---|---|
US20230110904A1 true US20230110904A1 (en) | 2023-04-13 |
Family
ID=77079955
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/759,737 Pending US20230110904A1 (en) | 2020-01-31 | 2021-01-29 | Systems and methods for artifact reduction in tomosynthesis with deep learning image processing |
Country Status (2)
Country | Link |
---|---|
US (1) | US20230110904A1 (en) |
WO (1) | WO2021155123A1 (en) |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10610182B2 (en) * | 2014-01-15 | 2020-04-07 | Alara Systems, Inc | Converting low-dose to higher dose 3D tomosynthesis images through machine-learning processes |
WO2017223560A1 (en) * | 2016-06-24 | 2017-12-28 | Rensselaer Polytechnic Institute | Tomographic image reconstruction via machine learning |
US10489907B2 (en) * | 2017-11-13 | 2019-11-26 | Siemens Healthcare Gmbh | Artifact identification and/or correction for medical imaging |
-
2021
- 2021-01-29 WO PCT/US2021/015677 patent/WO2021155123A1/en active Application Filing
- 2021-01-29 US US17/759,737 patent/US20230110904A1/en active Pending
Also Published As
Publication number | Publication date |
---|---|
WO2021155123A1 (en) | 2021-08-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11948314B2 (en) | Systems and methods for image processing | |
US10839567B2 (en) | Systems and methods for correcting mismatch induced by respiratory motion in positron emission tomography image reconstruction | |
JP7245364B2 (en) | sCT Imaging Using CycleGAN with Deformable Layers | |
US20230085203A1 (en) | Systems and methods for image reconstruction | |
US10690782B2 (en) | Systems and methods for positron emission tomography image reconstruction | |
US20200167929A1 (en) | Image processing method, image processing apparatus, and computer-program product | |
US11270478B2 (en) | System and method for reconstructing a computed tomography image | |
US11250601B2 (en) | Learning-assisted multi-modality dielectric imaging | |
US9775582B2 (en) | Medical image photographing apparatus and method of processing medical image | |
CN106999138B (en) | Diagnostic imaging method and apparatus and recording medium therefor | |
US8049752B2 (en) | Systems and methods of determining sampling rates for volume rendering | |
US20200242744A1 (en) | Forecasting Images for Image Processing | |
US10032295B2 (en) | Tomography apparatus and method of processing tomography image | |
Zeng et al. | Cerebral perfusion computed tomography deconvolution via structure tensor total variation regularization | |
US11995745B2 (en) | Systems and methods for correcting mismatch induced by respiratory motion in positron emission tomography image reconstruction | |
US20150117742A1 (en) | Medical imaging apparatus and method of reconstructing medical image | |
US20230061863A1 (en) | Systems and methods for artifact reduction in tomosynthesis with multi-scale deep learning image processing | |
US20230419455A1 (en) | System and method for image correction | |
US20230110904A1 (en) | Systems and methods for artifact reduction in tomosynthesis with deep learning image processing | |
US10685461B1 (en) | Apparatus and method for context-oriented iterative reconstruction for computed tomography (CT) | |
WO2023205726A1 (en) | Deep learning based image reconstruction | |
US20220383491A1 (en) | Tissue boundary determination apparatus and method | |
US11986337B2 (en) | Dose reduction for cardiac computed tomography | |
US20160166218A1 (en) | System and method for expectation maximization reconstruction for gamma emission breast tomosynthesis | |
CN115375840A (en) | Image reconstruction method, device, system, computer equipment and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: GENERAL HOSPITAL CORPORATION, THE, MASSACHUSETTS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:WU, DUFAN;KIM, KYUNGSANG;LI, QUANZHENG;SIGNING DATES FROM 20220427 TO 20220502;REEL/FRAME:060691/0832 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |