US20230110904A1 - Systems and methods for artifact reduction in tomosynthesis with deep learning image processing - Google Patents

Systems and methods for artifact reduction in tomosynthesis with deep learning image processing Download PDF

Info

Publication number
US20230110904A1
US20230110904A1 US17/759,737 US202117759737A US2023110904A1 US 20230110904 A1 US20230110904 A1 US 20230110904A1 US 202117759737 A US202117759737 A US 202117759737A US 2023110904 A1 US2023110904 A1 US 2023110904A1
Authority
US
United States
Prior art keywords
unroll
neural network
tomosynthesis
decoupled
trained
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
US17/759,737
Inventor
Dufan Wu
Kyungsang Kim
Quanzheng Li
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
General Hospital Corp
Original Assignee
General Hospital Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by General Hospital Corp filed Critical General Hospital Corp
Priority to US17/759,737 priority Critical patent/US20230110904A1/en
Assigned to GENERAL HOSPITAL CORPORATION, THE reassignment GENERAL HOSPITAL CORPORATION, THE ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: LI, QUANZHENG, KIM, Kyungsang, WU, Dufan
Publication of US20230110904A1 publication Critical patent/US20230110904A1/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/70Denoising; Smoothing
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/50Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment specially adapted for specific body parts; specially adapted for specific clinical applications
    • A61B6/502Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment specially adapted for specific body parts; specially adapted for specific clinical applications for diagnosis of breast, i.e. mammography
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/02Arrangements for diagnosis sequentially in different planes; Stereoscopic radiation diagnosis
    • A61B6/025Tomosynthesis
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/52Devices using data or image processing specially adapted for radiation diagnosis
    • A61B6/5211Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data
    • A61B6/5217Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data extracting a diagnostic or physiological parameter from medical diagnostic data
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/52Devices using data or image processing specially adapted for radiation diagnosis
    • A61B6/5258Devices using data or image processing specially adapted for radiation diagnosis involving detection or reduction of artifacts or noise
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/60Image enhancement or restoration using machine learning, e.g. neural networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0012Biomedical image inspection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10116X-ray image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30068Mammography; Breast

Definitions

  • DBT Digital breast tomosynthesis
  • Digital breast tomosynthesis provides 3D reconstruction which reduces the superposition and overlapping of breast tissues compared to mammography, leading to increased sensitivity and specificity.
  • DBT Digital breast tomosynthesis
  • most DBT system has large sampling interval ( ⁇ 3°) and a very limited angular sampling range of 50°.
  • the large angular interval leads to sparse-view artifacts, where streaks due to high-contrast objects are visible in the DBT images.
  • the artifacts cause ambiguity in some cases when they are overlapping with local structures.
  • the obvious artifacts also lowered physician's confidence when making decision with the DBT images.
  • DBT images also may suffer from limited in-depth resolution.
  • 3D DBT also has very high resolution, which leads to huge training computational cost.
  • a DBT reconstruction method to mitigate the limited angular artifacts and improve in-depth resolution while avoiding computational cost burdens.
  • the present disclosure addresses the aforementioned drawbacks by providing systems and methods for a deep learning-based tomosynthesis reconstruction that mitigates the limited angular artifacts and improves in-depth resolution.
  • the systems and methods can reduce the sparse-view artifacts in DBT via deep learning without clinically damaging image sharpness and contrast.
  • a deep neural network may be trained, such as an unroll-type neural network, and may be used with a decoupled training for each unroll to reduce training-time computational cost.
  • a method for artifact reduction in tomosynthesis imaging of a subject.
  • the method includes accessing a decoupled trained, unroll-type neural network trained to suppress artifacts using decoupled training with image training data that include at least one artifact.
  • the method also includes accessing tomosynthesis image data of the subject that includes a region of interest (ROI) and subjecting the tomosynthesis image data to the decoupled trained unroll-type neural network to reconstruct an image of the subject with suppressed artifacts in the region of interest.
  • the method also includes displaying the image of the subject with suppressed artifacts.
  • ROI region of interest
  • a system for artifact reduction in tomosynthesis imaging of a subject.
  • the system includes a computer system configured to access a decouple trained, unroll-type neural network trained to suppress artifacts using decoupled training with image training data that include at least one artifact.
  • the computer system is also configured to access tomosynthesis image data of the subject that includes a region of interest and subject the tomosynthesis image data to the decoupled trained unroll-type neural network to reconstruct an image of the subject with suppressed artifacts in the region of interest.
  • the computer system is also configured to display the image of the subject with the suppressed artifacts.
  • FIG. 1 is a flow chart of non-limiting example steps for a method of tomosynthesis image reconstruction in accordance with the present disclosure.
  • FIG. 2 is a non-limiting example convolutional neural network (CNN) architecture that may be used in accordance with the present disclosure.
  • CNN convolutional neural network
  • FIG. 3 is a block diagram of an example system that can implement a digital breast tomosynthesis reconstruction system for generating images of a subject using a machine learning or deep learning method.
  • FIG. 4 is a block diagram of example hardware components of the system of FIG. 3 .
  • FIG. 5 is a block diagram of a non-limiting example digital breast tomosynthesis (DBT) system.
  • DBT digital breast tomosynthesis
  • a deep learning-based tomosynthesis image reconstruction such as a digital breast tomosynthesis (DBT) image reconstruction
  • DBT digital breast tomosynthesis
  • the systems and methods may reduce the sparse-view artifacts in DBT via deep learning without losing image sharpness and contrast.
  • a deep neural network may be trained, such as an unroll-type neural network, and may be used with a decoupled training for each unroll to reduce training-time computational cost.
  • an ROI loss may be used for further improvement on the resolution and contrast of the images.
  • a region of interest loss method on microcalcifications may be used to improve the spatial resolution and contrast of the regions of interest, such as on microcalcifications.
  • the network was trained and tested on 176 realistic breast phantoms, and demonstrated improved in-plane contrast (3.17 versus 0.43, p ⁇ 0.01) and in-depth resolution (1.19 mm versus 4.96 mm, p ⁇ 0.01) as compared to an iterative reconstruction (IR) method.
  • Tomosynthesis training data may be generated at step 102 .
  • a neural network may be trained using the training data at step 104 .
  • the resulting, trained neural network may be stored and accessed for later use.
  • tomosynthesis image data of the subject may be accessed or otherwise acquired at step 106 .
  • Tomosynthesis image data may include digital breast tomosynthesis image data.
  • the trained neural network can be accessed and the tomosynthesis image data is subjected to the trained neural network at step 108 .
  • An optional region of interest loss analysis may be performed on the tomosynthesis image data at step 110 . Images of the subject may be reconstructed from the image data that was subjected to the neural network and/or the ROI loss analysis to thereby produce, as will be described, images with suppressed artifacts.
  • Training of a neural network may include using realistic breast phantoms or real patients' data.
  • Numerical breast phantoms are a reliable choice for DBT simulation because of the extensive existing studies that have proven the performance of these phantoms.
  • Real patients' data may yet be more precise in some aspects.
  • both dense-view sampling e.g. 0.5°
  • sparse-view sampling may be acquired from the scans.
  • the ROIs may be annotated, such as by a human annotator. These ROIs may contain the sparse-view artifacts in the sparse-view reconstruction results.
  • a deep neural network with an ROI loss may be trained to suppress the artifacts while keeping existing structures.
  • a neural network is an unrolled network, which may solve the following optimization problem:
  • x * argmin x ⁇ ⁇ Ax - b ⁇ w 2 + ⁇ ⁇ R ⁇ ( x ) ( 1 )
  • x is the image to be reconstructed
  • A is the system matrix
  • b is the projection data
  • w is the noise weighting matrix
  • R(x) is the prior function
  • p is the hyperparameter to balance between data fidelity and the prior knowledge.
  • Equation (1) can be solved via proximal gradient descent:
  • is a suitable step size for the gradient descent.
  • the decoupled unrolled network may replace proximal mapping with CNNs and the gradient descent step with separable quadratic surrogate (SQS).
  • SQS is free of choosing step size ⁇ and has faster convergence.
  • the SQS step may be noted as:
  • the decoupled unrolled network can be expressed as:
  • x (n) f ( g M ( x (n ⁇ 1) ), x (n ⁇ 1) ; ⁇ (n) ) (4)
  • f (y, x; ⁇ (n) ) is a CNN with trainable parameters ⁇ (n) and input y and x concatenated along the channel direction.
  • g M (x n ⁇ 1 ) means Mth iterations of SQS (3) being applied to input image x (n ⁇ 1) .
  • the network was trained sequentially from the first unroll to the last unroll.
  • the training at nth unroll may be expressed as:
  • ⁇ ( n ) argmin ⁇ ⁇ ⁇ ⁇ f ⁇ ( g M ( x i ( n - 1 ) ) , x i ( n - 1 ) ; ⁇ ) - x i ref ⁇ ( 5 )
  • x i ref is the label image of the ith training sample
  • x i n ⁇ 1 is the output of trained networks for the ith sample.
  • the decoupled training may only be optimized one OM at a time. That is, conventional training techniques suffer from large memory consumption issues when processing 3D volume data, whereas decoupled unrolled network training may train each iteration step separately to reduce memory costs.
  • the feature map size may be significantly reduced, leading to less training memory and time requirement.
  • the convolutional neural network is shown as a decoupled unrolled network 200 .
  • the decoupled unrolled network 200 includes initial input 202 , initial separable quadratic surrogate 204 , first convolutional neural network level 206 , and initial training data 208 .
  • the decoupled unrolled network 200 also includes a subsequent input 210 , a subsequent separable quadratic surrogate 212 , a subsequent convolutional neural network level 214 , and subsequent training data 216 .
  • the decoupled unrolled network 200 also includes N ⁇ 1 input 218 , N ⁇ 1 separable quadratic surrogate 220 , N convolutional neural network level 222 and N training data 224 .
  • the result from the decoupled unrolled network 200 is delivered via output 226 .
  • microcalcifications could be early signs of breast cancer, and they usually have diameter of several hundred micrometers and significantly higher density compared to breast tissues. Because microcalcifications are extremely small compared to the whole volume, they may provide limited contributions to the training loss and may be ignored by the network during training in some cases. The micorcalcifications may be significantly different from the other breast tissues that a network was trained on, which may led to limited contrast and in-depth resolution improvement of microcalcifications compared to iterative reconstructions. In accordance with the present disclosure, these limitations may be addressed by including the ROI loss in addition to the whole-volume L2 loss. In some configurations, this may be reflected as:
  • J is the total number of voxels of an image
  • Jr is the total number of voxels of ROIs
  • K is the number of ROIs of sample i
  • E ik is the masking matrix for ROI k in sample i
  • is a hyperparameter to balance between whole-image loss and ROI loss
  • a dataset consisted of 176 realistic breast phantoms derived from breast CT scans of healthy subjects.
  • Each breast CT image was segmented into six classes including skin, adipose, glandular and 3 different mixtures of adipose and glandular.
  • Finite element model was applied to the segmented images to simulate breast compression between 50 to 80 mm.
  • the voxel size was downsampled from 0.25 ⁇ 0.25 ⁇ 0.25 mm 3 to 1 ⁇ 1 ⁇ 1 mm 3 . The downsampling did not interfere with the limited angular artifacts, because the artifacts mostly exists in low frequency.
  • Calcification points were further inserted into phantoms with ⁇ from 1.011 cm ⁇ 1 to 3.034 cm ⁇ 1 , which was corresponding to 10% to 30% of the attenuation coefficient of calcium oxalate at 20 keV. Each calcification point only occupied one single voxel and each patient was randomly assigned 0 to 4 calcification points.
  • the 10% to 30% scale factor was corresponding microcalcifications' diameter of 0.1 mm to 0.3 mm whereas our voxel size was 1 mm.
  • a detector with pixel size of 0.5 ⁇ 0.5 mm 2 was used.
  • the projecting geometry was configured where the source to detector distance was 660 mm, source to rotation center distance was 627 mm and the breast was placed 10 mm above the detector surface.
  • the angular sampling was from ⁇ 21 ° to +21° with 3° interval, giving 15 views in total.
  • Distance driven projector and backprojector were used for both simulation and reconstruction. Poisson noise was added to the projections assuming 105 initial photons per ray.
  • Deep learning results gave significantly improved contrast for in-plane structures with less superposition.
  • both axial and sagittal views showed a structure where deep learning achieved improved in-plane contrast and better in-depth resolution compared to IR.
  • a calcification point was also zoomed in, and deep learning with ROI loss demonstrated significantly improved contrast compared to IR and L2 loss.
  • there were obvious wave-like artifacts in the IR result which was caused by the sparse sampling. These artifacts were not present in the deep learning results.
  • Root mean square errors (RMSE) and structural similarity indices (SSIM) were calculated for each testing images comparing to the ground truth. Both deep learning-based results had significantly improved metrics compared to IR (p ⁇ 0.01). Furthermore, ROI loss only led to slightly worse overall RMSEs and SSIMs compared to L2 loss. There was no substantial visual difference of the glandular reconstructed using the two losses in FIG. 2 .
  • the contrast and spatial resolution were quantitatively analyzed at the calcification points. For each point, a Gaussian function was used to fit the profile along x, y and z directions in a 21-pixel window. The in-depth resolution was calculated as the full width at half maximum (FWHM) of the Gaussian function along z, and the in-plane resolution was taken as the average of FWHM along x and y direction.
  • the Weber contrast was determined, which may be defined as
  • Ipeak is the peak intensity as the peak of the Gaussian function
  • Ibackground is the background intensity as the bias of the Gaussian function.
  • the network may be scaled to larger volumes in applications without additional requirement on computational hardware.
  • the trained networks may also be applied to real DBT data.
  • the trained network was further applied to real DBT data from 15 patients and demonstrated improved in-plane contrast and in-depth resolution. In one of the 15 patients, the upper and lower borders of a concentrated mass can be identified in the network results, which was otherwise indistinguishable in the IR results.
  • a computing device 350 can receive one or more types of data (e.g., digital breast tomography, x-ray, computed tomography, ultrasound, multiparametric MRI data, breast image data, and the like) from image source 302 .
  • computing device 350 can execute at least a portion of a digital breast tomosynthesis reconstruction system 304 to generate images of a breast, or otherwise segment a region of interest from data received from the image source 302 .
  • the computing device 350 can communicate information about data received from the image source 302 to a server 352 over a communication network 354 , which can execute at least a portion of the digital breast tomosynthesis reconstruction system 304 to generate images of a region of interest, or otherwise segment a region of interest from data received from the image source 302 .
  • the server 352 can return information to the computing device 350 (and/or any other suitable computing device) indicative of an output of the digital breast tomosynthesis reconstruction system 304 to generate images of a region of interest, or otherwise segment a region of interest from data received from the image source 302 .
  • computing device 350 and/or server 352 can be any suitable computing device or combination of devices, such as a desktop computer, a laptop computer, a smartphone, a tablet computer, a wearable computer, a server computer, a virtual machine being executed by a physical computing device, and so on.
  • the computing device 350 and/or server 352 can also reconstruct images from the data.
  • image source 302 can be any suitable source of image data (e.g., measurement data, images reconstructed from measurement data), such as a digital breast tomosynthesis system, another computing device (e.g., a server storing image data), and so on.
  • image source 302 can be local to computing device 350 .
  • image source 302 can be incorporated with computing device 350 (e.g., computing device 350 can be configured as part of a device for capturing, scanning, and/or storing images).
  • image source 302 can be connected to computing device 350 by a cable, a direct wireless link, and so on.
  • image source 302 can be located locally and/or remotely from computing device 350 , and can communicate data to computing device 350 (and/or server 352 ) via a communication network (e.g., communication network 354 ).
  • a communication network e.g., communication network 354
  • communication network 354 can be any suitable communication network or combination of communication networks.
  • communication network 354 can include a Wi-Fi network (which can include one or more wireless routers, one or more switches, etc.), a peer-to-peer network (e.g., a Bluetooth network), a cellular network (e.g., a 3G network, a 4G network, etc., complying with any suitable standard, such as CDMA, GSM, LTE, LTE Advanced, WiMAX, etc.), a wired network, and so on.
  • Wi-Fi network which can include one or more wireless routers, one or more switches, etc.
  • peer-to-peer network e.g., a Bluetooth network
  • a cellular network e.g., a 3G network, a 4G network, etc., complying with any suitable standard, such as CDMA, GSM, LTE, LTE Advanced, WiMAX, etc.
  • communication network 108 can be a local area network, a wide area network, a public network (e.g., the Internet), a private or semi-private network (e.g., a corporate or university intranet), any other suitable type of network, or any suitable combination of networks.
  • Communications links shown in FIG. 3 can each be any suitable communications link or combination of communications links, such as wired links, fiber optic links, Wi-Fi links, Bluetooth links, cellular links, and so on.
  • computing device 350 can include a processor 402 , a display 404 , one or more inputs 406 , one or more communication systems 408 , and/or memory 410 .
  • processor 402 can be any suitable hardware processor or combination of processors, such as a central processing unit (“CPU”), a graphics processing unit (“GPU”), and so on.
  • display 404 can include any suitable display devices, such as a computer monitor, a touchscreen, a television, and so on.
  • inputs 406 can include any suitable input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, and so on.
  • communications systems 408 can include any suitable hardware, firmware, and/or software for communicating information over communication network 354 and/or any other suitable communication networks.
  • communications systems 408 can include one or more transceivers, one or more communication chips and/or chip sets, and so on.
  • communications systems 408 can include hardware, firmware and/or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.
  • memory 410 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 402 to present content using display 404 , to communicate with server 352 via communications system(s) 408 , and so on.
  • Memory 410 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof.
  • memory 410 can include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on.
  • memory 410 can have encoded thereon, or otherwise stored therein, a computer program for controlling operation of computing device 350 .
  • processor 402 can execute at least a portion of the computer program to present content (e.g., images, user interfaces, graphics, tables), receive content from server 352 , transmit information to server 352 , and so on.
  • server 352 can include a processor 412 , a display 414 , one or more inputs 416 , one or more communications systems 418 , and/or memory 420 .
  • processor 412 can be any suitable hardware processor or combination of processors, such as a CPU, a GPU, and so on.
  • display 414 can include any suitable display devices, such as a computer monitor, a touchscreen, a television, and so on.
  • inputs 416 can include any suitable input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, and so on.
  • communications systems 418 can include any suitable hardware, firmware, and/or software for communicating information over communication network 354 and/or any other suitable communication networks.
  • communications systems 418 can include one or more transceivers, one or more communication chips and/or chip sets, and so on.
  • communications systems 418 can include hardware, firmware and/or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.
  • memory 420 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 412 to present content using display 414 , to communicate with one or more computing devices 350 , and so on.
  • Memory 420 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof.
  • memory 420 can include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on.
  • memory 420 can have encoded thereon a server program for controlling operation of server 352 .
  • processor 412 can execute at least a portion of the server program to transmit information and/or content (e.g., data, images, a user interface) to one or more computing devices 350 , receive information and/or content from one or more computing devices 350 , receive instructions from one or more devices (e.g., a personal computer, a laptop computer, a tablet computer, a smartphone), and so on.
  • information and/or content e.g., data, images, a user interface
  • processor 412 can execute at least a portion of the server program to transmit information and/or content (e.g., data, images, a user interface) to one or more computing devices 350 , receive information and/or content from one or more computing devices 350 , receive instructions from one or more devices (e.g., a personal computer, a laptop computer, a tablet computer, a smartphone), and so on.
  • image source 302 can include a processor 422 , one or more image acquisition systems 424 , one or more communications systems 426 , and/or memory 428 .
  • processor 422 can be any suitable hardware processor or combination of processors, such as a CPU, a GPU, and so on.
  • the one or more image acquisition systems 424 are generally configured to acquire data, images, or both, and can include an RF transmission and reception subsystem of a digital breast tomosynthesis system.
  • one or more image acquisition systems 424 can include any suitable hardware, firmware, and/or software for coupling to and/or controlling operations of an digital breast tomosynthesis system.
  • one or more portions of the one or more image acquisition systems 424 can be removable and/or replaceable.
  • image source 302 can include any suitable inputs and/or outputs.
  • image source 302 can include input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, a trackpad, a trackball, and so on.
  • image source 302 can include any suitable display devices, such as a computer monitor, a touchscreen, a television, etc., one or more speakers, and so on.
  • communications systems 426 can include any suitable hardware, firmware, and/or software for communicating information to computing device 350 (and, in some embodiments, over communication network 354 and/or any other suitable communication networks).
  • communications systems 426 can include one or more transceivers, one or more communication chips and/or chip sets, and so on.
  • communications systems 426 can include hardware, firmware and/or software that can be used to establish a wired connection using any suitable port and/or communication standard (e.g., VGA, DVI video, USB, RS-232, etc.), Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.
  • memory 428 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 422 to control the one or more image acquisition systems 424 , and/or receive data from the one or more image acquisition systems 424 ; to images from data; present content (e.g., images, a user interface) using a display; communicate with one or more computing devices 350 ; and so on.
  • Memory 428 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof.
  • memory 428 can include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on.
  • memory 428 can have encoded thereon, or otherwise stored therein, a program for controlling operation of image source 302 .
  • processor 422 can execute at least a portion of the program to generate images, transmit information and/or content (e.g., data, images) to one or more computing devices 350 , receive information and/or content from one or more computing devices 350 , receive instructions from one or more devices (e.g., a personal computer, a laptop computer, a tablet computer, a smartphone, etc.), and so on.
  • any suitable computer readable media can be used for storing instructions for performing the functions and/or processes described herein.
  • computer readable media can be transitory or non-transitory.
  • non-transitory computer readable media can include media such as magnetic media (e.g., hard disks, floppy disks), optical media (e.g., compact discs, digital video discs, Blu-ray discs), semiconductor media (e.g., random access memory (“RAM”), flash memory, electrically programmable read only memory (“EPROM”), electrically erasable programmable read only memory (“EEPROM”)), any suitable media that is not fleeting or devoid of any semblance of permanence during transmission, and/or any suitable tangible media.
  • RAM random access memory
  • EPROM electrically programmable read only memory
  • EEPROM electrically erasable programmable read only memory
  • transitory computer readable media can include signals on networks, in wires, conductors, optical fibers, circuits, or any suitable media that is fleeting and devoid of any semblance of permanence during transmission, and/or any suitable intangible media.
  • DBT system 500 may include a drive plate 518 configured to provide compression against a compression plate 516 for an organ disposed between drive plate 518 and compression plate 516 .
  • Imaging detectors 512 may be coupled to a mounting system 514 .
  • Radiation source 508 may be coupled to support system 502 and coupled to base 504 with rotatable coupler 506 .

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Medical Informatics (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Optics & Photonics (AREA)
  • Pathology (AREA)
  • High Energy & Nuclear Physics (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Veterinary Medicine (AREA)
  • Public Health (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Dentistry (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Quality & Reliability (AREA)
  • Physiology (AREA)
  • Apparatus For Radiation Diagnosis (AREA)

Abstract

Systems and methods are provided for a deep learning-based digital breast tomosynthesis (DBT) image reconstruction that mitigates limited angular artifacts and improves in-depth resolution of the resulting images. The systems and methods may reduce the sparse-view artifacts in DBT via deep learning without losing image sharpness and contrast. A deep neural network may be trained in a way to reduce training-time computational cost. An ROI loss method may be used for further improvement on the resolution and contrast of the images.

Description

    CROSS-REFERENCE TO RELATED APPLICATIONS
  • This application claims the benefit of U.S. Provisional patent application Ser. No. 62/968,589 filed on Jan. 31, 2020 and entitled “System and Method of Sparse-View Artifacts Reduction in Digital Breast Tomosynthesis with Deep-Learning-Based Image Processing,” which is incorporated herein by reference as if set forth in its entirety for all purposes.
  • STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH
  • N/A
  • BACKGROUND
  • Breast cancer is the second most common cancer in women. It is also the second leading cause of cancer death among American women. Evidence has shown that screening and early detection is the key to prevent breast cancer death. Screening is conventionally performed using mammography, which provides for 2D imaging, but the breast tissue depicted may be overlapping due to a single view angle. Digital breast tomosynthesis (DBT) is an emerging, promising new technology for breast cancer screening imaging that takes several images from different angles, which may be from different depths that may be stacked to form a 3D image. Compared to the conventional X-ray mammography, DBT has significantly improved sensitivity and specificity. Meanwhile, DBT has very similar hardware setup with mammography, which lowers the cost for both manufacturers and users to upgrade from mammography to DBT.
  • Digital breast tomosynthesis (DBT) provides 3D reconstruction which reduces the superposition and overlapping of breast tissues compared to mammography, leading to increased sensitivity and specificity. However, to save dose and scanning time, most DBT system has large sampling interval (≤3°) and a very limited angular sampling range of 50°. The large angular interval leads to sparse-view artifacts, where streaks due to high-contrast objects are visible in the DBT images. The artifacts cause ambiguity in some cases when they are overlapping with local structures. The obvious artifacts also lowered physician's confidence when making decision with the DBT images. DBT images also may suffer from limited in-depth resolution.
  • 3D DBT also has very high resolution, which leads to huge training computational cost. Thus, there remains a need for a DBT reconstruction method to mitigate the limited angular artifacts and improve in-depth resolution while avoiding computational cost burdens.
  • SUMMARY OF THE DISCLOSURE
  • The present disclosure addresses the aforementioned drawbacks by providing systems and methods for a deep learning-based tomosynthesis reconstruction that mitigates the limited angular artifacts and improves in-depth resolution. The systems and methods can reduce the sparse-view artifacts in DBT via deep learning without clinically damaging image sharpness and contrast. A deep neural network may be trained, such as an unroll-type neural network, and may be used with a decoupled training for each unroll to reduce training-time computational cost.
  • In one configuration, a method is provided for artifact reduction in tomosynthesis imaging of a subject. The method includes accessing a decoupled trained, unroll-type neural network trained to suppress artifacts using decoupled training with image training data that include at least one artifact. The method also includes accessing tomosynthesis image data of the subject that includes a region of interest (ROI) and subjecting the tomosynthesis image data to the decoupled trained unroll-type neural network to reconstruct an image of the subject with suppressed artifacts in the region of interest. The method also includes displaying the image of the subject with suppressed artifacts.
  • In one configuration, a system is provided for artifact reduction in tomosynthesis imaging of a subject. The system includes a computer system configured to access a decouple trained, unroll-type neural network trained to suppress artifacts using decoupled training with image training data that include at least one artifact. The computer system is also configured to access tomosynthesis image data of the subject that includes a region of interest and subject the tomosynthesis image data to the decoupled trained unroll-type neural network to reconstruct an image of the subject with suppressed artifacts in the region of interest. The computer system is also configured to display the image of the subject with the suppressed artifacts.
  • The foregoing and other aspects and advantages of the present disclosure will appear from the following description. In the description, reference is made to the accompanying drawings that form a part hereof, and in which there is shown by way of illustration a preferred embodiment. This embodiment does not necessarily represent the full scope of the invention, however, and reference is therefore made to the claims and herein for interpreting the scope of the invention. Like reference numerals will be used to refer to like parts from Figure to Figure in the following description.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • FIG. 1 is a flow chart of non-limiting example steps for a method of tomosynthesis image reconstruction in accordance with the present disclosure.
  • FIG. 2 is a non-limiting example convolutional neural network (CNN) architecture that may be used in accordance with the present disclosure.
  • FIG. 3 is a block diagram of an example system that can implement a digital breast tomosynthesis reconstruction system for generating images of a subject using a machine learning or deep learning method.
  • FIG. 4 is a block diagram of example hardware components of the system of FIG. 3 .
  • FIG. 5 is a block diagram of a non-limiting example digital breast tomosynthesis (DBT) system.
  • DETAILED DESCRIPTION
  • Systems and methods are provided for a deep learning-based tomosynthesis image reconstruction, such as a digital breast tomosynthesis (DBT) image reconstruction, that mitigates limited angular artifacts and improves in-depth resolution of the resulting images. The systems and methods may reduce the sparse-view artifacts in DBT via deep learning without losing image sharpness and contrast. A deep neural network may be trained, such as an unroll-type neural network, and may be used with a decoupled training for each unroll to reduce training-time computational cost. In some configurations, an ROI loss may be used for further improvement on the resolution and contrast of the images.
  • In some configurations, a region of interest loss method on microcalcifications may be used to improve the spatial resolution and contrast of the regions of interest, such as on microcalcifications. In a non-limiting example, the network was trained and tested on 176 realistic breast phantoms, and demonstrated improved in-plane contrast (3.17 versus 0.43, p<0.01) and in-depth resolution (1.19 mm versus 4.96 mm, p<0.01) as compared to an iterative reconstruction (IR) method.
  • Referring to FIG. 1 , a flow chart of non-limiting example steps for a method of tomosynthesis image reconstruction is shown. Tomosynthesis training data may be generated at step 102. A neural network may be trained using the training data at step 104. The resulting, trained neural network may be stored and accessed for later use.
  • When images of a particular subject are needed, tomosynthesis image data of the subject may be accessed or otherwise acquired at step 106. Tomosynthesis image data may include digital breast tomosynthesis image data. Then, the trained neural network can be accessed and the tomosynthesis image data is subjected to the trained neural network at step 108. An optional region of interest loss analysis may be performed on the tomosynthesis image data at step 110. Images of the subject may be reconstructed from the image data that was subjected to the neural network and/or the ROI loss analysis to thereby produce, as will be described, images with suppressed artifacts.
  • Training of a neural network may include using realistic breast phantoms or real patients' data. Numerical breast phantoms are a reliable choice for DBT simulation because of the extensive existing studies that have proven the performance of these phantoms. Real patients' data may yet be more precise in some aspects. When training a neural network, both dense-view sampling (e.g. 0.5°) and sparse-view sampling may be acquired from the scans. In some configurations, the ROIs may be annotated, such as by a human annotator. These ROIs may contain the sparse-view artifacts in the sparse-view reconstruction results. A deep neural network with an ROI loss may be trained to suppress the artifacts while keeping existing structures.
  • In one configuration, a neural network is an unrolled network, which may solve the following optimization problem:
  • x * = argmin x Ax - b w 2 + β R ( x ) ( 1 )
  • where x is the image to be reconstructed, A is the system matrix, b is the projection data, w is the noise weighting matrix, R(x) is the prior function, and p is the hyperparameter to balance between data fidelity and the prior knowledge.
  • Equation (1) can be solved via proximal gradient descent:

  • x n=proxγβR {x (n−1) −γA T w(Ax n−1)−b)}  (2)
  • where γ is a suitable step size for the gradient descent. The decoupled unrolled network may replace proximal mapping with CNNs and the gradient descent step with separable quadratic surrogate (SQS). The SQS is free of choosing step size γ and has faster convergence. The SQS step may be noted as:
  • g ( x ) = x - A T w ( A x - b ) A T A 1 ( 3 )
  • The decoupled unrolled network can be expressed as:

  • x (n) =f(g M(x (n−1)),x (n−1)(n))  (4)
  • where f (y, x; Θ(n)) is a CNN with trainable parameters Θ(n) and input y and x concatenated along the channel direction. gM (xn−1) means Mth iterations of SQS (3) being applied to input image x(n−1). The network was trained sequentially from the first unroll to the last unroll. The training at nth unroll may be expressed as:
  • Θ ( n ) = argmin Θ f ( g M ( x i ( n - 1 ) ) , x i ( n - 1 ) ; Θ ) - x i ref ( 5 )
  • where xi ref is the label image of the ith training sample, and xi n−1 is the output of trained networks for the ith sample. Compared to most unrolled networks which need simultaneously optimization of Θ(1) to Θ(N), the decoupled training may only be optimized one OM at a time. That is, conventional training techniques suffer from large memory consumption issues when processing 3D volume data, whereas decoupled unrolled network training may train each iteration step separately to reduce memory costs. The feature map size may be significantly reduced, leading to less training memory and time requirement.
  • Referring to FIG. 2 , a non-limiting example convolutional neural network architecture in accordance with the present disclosure is shown. The convolutional neural network is shown as a decoupled unrolled network 200. The decoupled unrolled network 200 includes initial input 202, initial separable quadratic surrogate 204, first convolutional neural network level 206, and initial training data 208. The decoupled unrolled network 200 also includes a subsequent input 210, a subsequent separable quadratic surrogate 212, a subsequent convolutional neural network level 214, and subsequent training data 216. This framework extends such that the decoupled unrolled network 200 also includes N−1 input 218, N−1 separable quadratic surrogate 220, N convolutional neural network level 222 and N training data 224. The result from the decoupled unrolled network 200 is delivered via output 226.
  • Non-Limiting Example Region of Interest Loss
  • Detection of microcalcification is one non-limiting example application of DBT. Microcalcifications could be early signs of breast cancer, and they usually have diameter of several hundred micrometers and significantly higher density compared to breast tissues. Because microcalcifications are extremely small compared to the whole volume, they may provide limited contributions to the training loss and may be ignored by the network during training in some cases. The micorcalcifications may be significantly different from the other breast tissues that a network was trained on, which may led to limited contrast and in-depth resolution improvement of microcalcifications compared to iterative reconstructions. In accordance with the present disclosure, these limitations may be addressed by including the ROI loss in addition to the whole-volume L2 loss. In some configurations, this may be reflected as:
  • Θ ( n ) = argmin Θ 1 J f i ( n - 1 ) ( Θ ) - x i ref 2 2 + λ i k = 1 K i 1 J r K i E i k ( f i ( n - 1 ) ( Θ ) - x i ref ) 2 2 ( 6 )
  • where J is the total number of voxels of an image; Jr is the total number of voxels of ROIs; K, is the number of ROIs of sample i; Eik is the masking matrix for ROI k in sample i; λ is a hyperparameter to balance between whole-image loss and ROI loss

  • f i (n−1)(Θ)=f(g M(x i (n−1)),x i (n−1);Θ)as in equation  (5).
  • Non-Limiting Example Breast Phantom Application
  • In a non-limiting example, a dataset consisted of 176 realistic breast phantoms derived from breast CT scans of healthy subjects. Each breast CT image was segmented into six classes including skin, adipose, glandular and 3 different mixtures of adipose and glandular. Finite element model was applied to the segmented images to simulate breast compression between 50 to 80 mm. We assigned μ=0.456 cm−1 for adipose and μ=0.802 cm−1 for glandular and skin. For faster computation, the voxel size was downsampled from 0.25×0.25×0.25 mm3 to 1×1×1 mm3. The downsampling did not interfere with the limited angular artifacts, because the artifacts mostly exists in low frequency.
  • Calcification points were further inserted into phantoms with μ from 1.011 cm−1 to 3.034 cm−1, which was corresponding to 10% to 30% of the attenuation coefficient of calcium oxalate at 20 keV. Each calcification point only occupied one single voxel and each patient was randomly assigned 0 to 4 calcification points. The 10% to 30% scale factor was corresponding microcalcifications' diameter of 0.1 mm to 0.3 mm whereas our voxel size was 1 mm.
  • A detector with pixel size of 0.5×0.5 mm2 was used. The projecting geometry was configured where the source to detector distance was 660 mm, source to rotation center distance was 627 mm and the breast was placed 10 mm above the detector surface. The angular sampling was from −21° to +21° with 3° interval, giving 15 views in total. Distance driven projector and backprojector were used for both simulation and reconstruction. Poisson noise was added to the projections assuming 105 initial photons per ray.
  • In the non-limiting example, 140 patients were randomly selected for training whereas the remaining 36 patients were used for testing. UNet with depth of 4 was used as the CNN in each unroll. An unroll number N=10 and SQS iterations M=10 were used. The initial image x(0) was 10 SQS iterations from zero images. Each unroll was trained by Adam optimizer with learning rate of 10−3 for 150 epochs. The entire training took approximately 12 hours. For the ROI loss, 9×9×9 ROIs were selected around each inserted calcification point with λ set to 0.1. IR results from 20 iterations of SQS from zero were used for comparison. The number of iterations balanced between noise level and artifacts. The thicknesses of the breast were assumed to be known for both IR and deep learning to further reduce limited angular artifacts. Thickness can be measured by the compressor of DBT scanners.
  • Deep learning results gave significantly improved contrast for in-plane structures with less superposition. In the non-limiting example, both axial and sagittal views showed a structure where deep learning achieved improved in-plane contrast and better in-depth resolution compared to IR. A calcification point was also zoomed in, and deep learning with ROI loss demonstrated significantly improved contrast compared to IR and L2 loss. In addition, there were obvious wave-like artifacts in the IR result which was caused by the sparse sampling. These artifacts were not present in the deep learning results.
  • Root mean square errors (RMSE) and structural similarity indices (SSIM) were calculated for each testing images comparing to the ground truth. Both deep learning-based results had significantly improved metrics compared to IR (p<0.01). Furthermore, ROI loss only led to slightly worse overall RMSEs and SSIMs compared to L2 loss. There was no substantial visual difference of the glandular reconstructed using the two losses in FIG. 2 .
  • The contrast and spatial resolution were quantitatively analyzed at the calcification points. For each point, a Gaussian function was used to fit the profile along x, y and z directions in a 21-pixel window. The in-depth resolution was calculated as the full width at half maximum (FWHM) of the Gaussian function along z, and the in-plane resolution was taken as the average of FWHM along x and y direction. The Weber contrast was determined, which may be defined as
  • c = I p e a k - I background I background ( 7 )
  • where Ipeak is the peak intensity as the peak of the Gaussian function, and Ibackground is the background intensity as the bias of the Gaussian function. ROI loss significantly improved resolution and contrast over L2 loss and IR (p<0.01).
  • In a non-limiting example, downsampled phantoms were used, but the network may be scaled to larger volumes in applications without additional requirement on computational hardware. The trained networks may also be applied to real DBT data. In a non-limiting example, the trained network was further applied to real DBT data from 15 patients and demonstrated improved in-plane contrast and in-depth resolution. In one of the 15 patients, the upper and lower borders of a concentrated mass can be identified in the network results, which was otherwise indistinguishable in the IR results.
  • Referring to FIG. 3 , an example of a system 300 for generating and implementing a machine learning or deep learning routine in accordance with some embodiments of the systems and methods described in the present disclosure is shown. As shown in FIG. 3 , a computing device 350 can receive one or more types of data (e.g., digital breast tomography, x-ray, computed tomography, ultrasound, multiparametric MRI data, breast image data, and the like) from image source 302. In some embodiments, computing device 350 can execute at least a portion of a digital breast tomosynthesis reconstruction system 304 to generate images of a breast, or otherwise segment a region of interest from data received from the image source 302.
  • Additionally or alternatively, in some embodiments, the computing device 350 can communicate information about data received from the image source 302 to a server 352 over a communication network 354, which can execute at least a portion of the digital breast tomosynthesis reconstruction system 304 to generate images of a region of interest, or otherwise segment a region of interest from data received from the image source 302. In such embodiments, the server 352 can return information to the computing device 350 (and/or any other suitable computing device) indicative of an output of the digital breast tomosynthesis reconstruction system 304 to generate images of a region of interest, or otherwise segment a region of interest from data received from the image source 302.
  • In some embodiments, computing device 350 and/or server 352 can be any suitable computing device or combination of devices, such as a desktop computer, a laptop computer, a smartphone, a tablet computer, a wearable computer, a server computer, a virtual machine being executed by a physical computing device, and so on. The computing device 350 and/or server 352 can also reconstruct images from the data.
  • In some embodiments, image source 302 can be any suitable source of image data (e.g., measurement data, images reconstructed from measurement data), such as a digital breast tomosynthesis system, another computing device (e.g., a server storing image data), and so on. In some embodiments, image source 302 can be local to computing device 350. For example, image source 302 can be incorporated with computing device 350 (e.g., computing device 350 can be configured as part of a device for capturing, scanning, and/or storing images). As another example, image source 302 can be connected to computing device 350 by a cable, a direct wireless link, and so on. Additionally or alternatively, in some embodiments, image source 302 can be located locally and/or remotely from computing device 350, and can communicate data to computing device 350 (and/or server 352) via a communication network (e.g., communication network 354).
  • In some embodiments, communication network 354 can be any suitable communication network or combination of communication networks. For example, communication network 354 can include a Wi-Fi network (which can include one or more wireless routers, one or more switches, etc.), a peer-to-peer network (e.g., a Bluetooth network), a cellular network (e.g., a 3G network, a 4G network, etc., complying with any suitable standard, such as CDMA, GSM, LTE, LTE Advanced, WiMAX, etc.), a wired network, and so on. In some embodiments, communication network 108 can be a local area network, a wide area network, a public network (e.g., the Internet), a private or semi-private network (e.g., a corporate or university intranet), any other suitable type of network, or any suitable combination of networks. Communications links shown in FIG. 3 can each be any suitable communications link or combination of communications links, such as wired links, fiber optic links, Wi-Fi links, Bluetooth links, cellular links, and so on.
  • Referring now to FIG. 4 , an example of hardware 400 that can be used to implement image source 302, computing device 350, and server 354 in accordance with some embodiments of the systems and methods described in the present disclosure is shown. As shown in FIG. 4 , in some embodiments, computing device 350 can include a processor 402, a display 404, one or more inputs 406, one or more communication systems 408, and/or memory 410. In some embodiments, processor 402 can be any suitable hardware processor or combination of processors, such as a central processing unit (“CPU”), a graphics processing unit (“GPU”), and so on. In some embodiments, display 404 can include any suitable display devices, such as a computer monitor, a touchscreen, a television, and so on. In some embodiments, inputs 406 can include any suitable input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, and so on.
  • In some embodiments, communications systems 408 can include any suitable hardware, firmware, and/or software for communicating information over communication network 354 and/or any other suitable communication networks. For example, communications systems 408 can include one or more transceivers, one or more communication chips and/or chip sets, and so on. In a more particular example, communications systems 408 can include hardware, firmware and/or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.
  • In some embodiments, memory 410 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 402 to present content using display 404, to communicate with server 352 via communications system(s) 408, and so on. Memory 410 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof. For example, memory 410 can include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. In some embodiments, memory 410 can have encoded thereon, or otherwise stored therein, a computer program for controlling operation of computing device 350. In such embodiments, processor 402 can execute at least a portion of the computer program to present content (e.g., images, user interfaces, graphics, tables), receive content from server 352, transmit information to server 352, and so on.
  • In some embodiments, server 352 can include a processor 412, a display 414, one or more inputs 416, one or more communications systems 418, and/or memory 420. In some embodiments, processor 412 can be any suitable hardware processor or combination of processors, such as a CPU, a GPU, and so on. In some embodiments, display 414 can include any suitable display devices, such as a computer monitor, a touchscreen, a television, and so on. In some embodiments, inputs 416 can include any suitable input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, and so on.
  • In some embodiments, communications systems 418 can include any suitable hardware, firmware, and/or software for communicating information over communication network 354 and/or any other suitable communication networks. For example, communications systems 418 can include one or more transceivers, one or more communication chips and/or chip sets, and so on. In a more particular example, communications systems 418 can include hardware, firmware and/or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.
  • In some embodiments, memory 420 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 412 to present content using display 414, to communicate with one or more computing devices 350, and so on. Memory 420 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof. For example, memory 420 can include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. In some embodiments, memory 420 can have encoded thereon a server program for controlling operation of server 352. In such embodiments, processor 412 can execute at least a portion of the server program to transmit information and/or content (e.g., data, images, a user interface) to one or more computing devices 350, receive information and/or content from one or more computing devices 350, receive instructions from one or more devices (e.g., a personal computer, a laptop computer, a tablet computer, a smartphone), and so on.
  • In some embodiments, image source 302 can include a processor 422, one or more image acquisition systems 424, one or more communications systems 426, and/or memory 428. In some embodiments, processor 422 can be any suitable hardware processor or combination of processors, such as a CPU, a GPU, and so on. In some embodiments, the one or more image acquisition systems 424 are generally configured to acquire data, images, or both, and can include an RF transmission and reception subsystem of a digital breast tomosynthesis system. Additionally or alternatively, in some embodiments, one or more image acquisition systems 424 can include any suitable hardware, firmware, and/or software for coupling to and/or controlling operations of an digital breast tomosynthesis system. In some embodiments, one or more portions of the one or more image acquisition systems 424 can be removable and/or replaceable.
  • Note that, although not shown, image source 302 can include any suitable inputs and/or outputs. For example, image source 302 can include input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, a trackpad, a trackball, and so on. As another example, image source 302 can include any suitable display devices, such as a computer monitor, a touchscreen, a television, etc., one or more speakers, and so on.
  • In some embodiments, communications systems 426 can include any suitable hardware, firmware, and/or software for communicating information to computing device 350 (and, in some embodiments, over communication network 354 and/or any other suitable communication networks). For example, communications systems 426 can include one or more transceivers, one or more communication chips and/or chip sets, and so on. In a more particular example, communications systems 426 can include hardware, firmware and/or software that can be used to establish a wired connection using any suitable port and/or communication standard (e.g., VGA, DVI video, USB, RS-232, etc.), Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.
  • In some embodiments, memory 428 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 422 to control the one or more image acquisition systems 424, and/or receive data from the one or more image acquisition systems 424; to images from data; present content (e.g., images, a user interface) using a display; communicate with one or more computing devices 350; and so on. Memory 428 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof. For example, memory 428 can include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. In some embodiments, memory 428 can have encoded thereon, or otherwise stored therein, a program for controlling operation of image source 302. In such embodiments, processor 422 can execute at least a portion of the program to generate images, transmit information and/or content (e.g., data, images) to one or more computing devices 350, receive information and/or content from one or more computing devices 350, receive instructions from one or more devices (e.g., a personal computer, a laptop computer, a tablet computer, a smartphone, etc.), and so on.
  • In some embodiments, any suitable computer readable media can be used for storing instructions for performing the functions and/or processes described herein. For example, in some embodiments, computer readable media can be transitory or non-transitory. For example, non-transitory computer readable media can include media such as magnetic media (e.g., hard disks, floppy disks), optical media (e.g., compact discs, digital video discs, Blu-ray discs), semiconductor media (e.g., random access memory (“RAM”), flash memory, electrically programmable read only memory (“EPROM”), electrically erasable programmable read only memory (“EEPROM”)), any suitable media that is not fleeting or devoid of any semblance of permanence during transmission, and/or any suitable tangible media. As another example, transitory computer readable media can include signals on networks, in wires, conductors, optical fibers, circuits, or any suitable media that is fleeting and devoid of any semblance of permanence during transmission, and/or any suitable intangible media.
  • Referring to FIG. 5 , a non-limiting example digital breast tomosynthesis (DBT) system 500 is shown that may be used in accordance with the present disclosure. DBT system 500 may include a drive plate 518 configured to provide compression against a compression plate 516 for an organ disposed between drive plate 518 and compression plate 516. Imaging detectors 512 may be coupled to a mounting system 514. Radiation source 508 may be coupled to support system 502 and coupled to base 504 with rotatable coupler 506.
  • The present disclosure has described one or more preferred embodiments, and it should be appreciated that many equivalents, alternatives, variations, and modifications, aside from those expressly stated, are possible and within the scope of the invention.

Claims (23)

1. A method for artifact reduction in tomosynthesis imaging of a subject, the method comprising:
a) accessing a decoupled trained, unroll-type neural network trained to suppress artifacts using decoupled training with image training data that include at least one artifact;
b) accessing tomosynthesis image data of the subject that includes a region of interest;
c) subjecting the tomosynthesis image data to the decoupled trained unroll-type neural network to reconstruct an image of the subject with suppressed artifacts in the region of interest;
d) displaying the image of the subject with the suppressed artifacts.
2. The method of claim 1, wherein decoupled training includes optimizing one training parameter at a time.
3. The method of claim 2, wherein the decoupled trained unroll-type neural network is a convolutional neural network (CNN) with separable quadratic surrogate.
4. The method of claim 3, wherein the tomosynthesis image data is subjected to the decoupled trained unroll-type neural network over a plurality of iterations.
5. The method of claim 1, wherein the unroll-type neural network is trained sequentially from a first unroll to a last unroll.
6. The method of claim 5, wherein the image training data include dense-view sampling data and sparse-view sampling data.
7. The method of claim 6, wherein the sparse-view sampling data includes sparse-view artifacts, and wherein the at least one artifact included with the image training data is the sparse-view artifact.
8. The method of claim 1, wherein the tomosynthesis image data includes digital breast tomosynthesis image data.
9. The method of claim 8, wherein the digital breast tomosynthesis image data includes microcalcifications.
10. The method of claim 1, further comprising performing a region of interest loss analysis that applies a region-of-interest loss.
11. The method of claim 1, wherein the image training data includes image data of realistic numerical breast phantoms generated from data acquired with at least one of digital breast tomosynthesis, mammography, computed tomography (CT), breast CT, magnetic resonance (MR) or breast MR.
12. A system for artifact reduction in tomosynthesis imaging of a subject, the system comprising:
a computer system configured to:
i) access a decoupled trained, unroll-type neural network trained to suppress artifacts using decoupled training with image training data that include at least one artifact;
ii) access tomosynthesis image data of the subject that includes a region of interest;
iii) subject the tomosynthesis image data to the decoupled trained unroll-type neural network to reconstruct an image of the subject with suppressed artifacts in the region of interest;
iv) display the image of the subject with the suppressed artifacts.
13. The system of claim 12, wherein the computer system is further configured to decoupled train the unroll-type neural network by optimizing one training parameter at a time.
14. The system of claim 13, wherein the decoupled trained unroll-type neural network is a convolutional neural network (CNN) with separable quadratic surrogate.
15. The system of claim 14, wherein the computer system is further configured to subject the tomosynthesis image data to the decoupled trained unroll-type neural network over a plurality of iterations.
16. The system of claim 12, wherein the computer system is further configured to train the unroll-type neural network sequentially from a first unroll to a last unroll.
17. The system of claim 16, wherein the image training data include dense-view sampling data and sparse-view sampling data.
18. The system of claim 17, wherein the sparse-view sampling data includes sparse-view artifacts, and wherein the at least one artifact included with the image training data is the sparse-view artifact.
19. The system of claim 12, wherein the tomosynthesis image data includes digital breast tomosynthesis image data.
20. The system of claim 19, wherein the digital breast tomosynthesis image data includes microcalcifications.
21. The system of claim 12, wherein the computer system is further configured to perform a region of interest loss analysis.
22. The system of claim 21, wherein the region of interest loss analysis applies a region of interest loss that is separate from a whole-volume L2 loss applied by the decoupled trained unroll-type neural network.
23. The method of claim 12, wherein the image training data includes image data of realistic numerical breast phantoms generated from data acquired with at least one of digital breast tomosynthesis, mammography, computed tomography (CT), breast CT, magnetic resonance (MR) or breast MR.
US17/759,737 2020-01-31 2021-01-29 Systems and methods for artifact reduction in tomosynthesis with deep learning image processing Pending US20230110904A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/759,737 US20230110904A1 (en) 2020-01-31 2021-01-29 Systems and methods for artifact reduction in tomosynthesis with deep learning image processing

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US202062968589P 2020-01-31 2020-01-31
US17/759,737 US20230110904A1 (en) 2020-01-31 2021-01-29 Systems and methods for artifact reduction in tomosynthesis with deep learning image processing
PCT/US2021/015677 WO2021155123A1 (en) 2020-01-31 2021-01-29 Systems and methods for artifact reduction in tomosynthesis with deep learning image processing

Publications (1)

Publication Number Publication Date
US20230110904A1 true US20230110904A1 (en) 2023-04-13

Family

ID=77079955

Family Applications (1)

Application Number Title Priority Date Filing Date
US17/759,737 Pending US20230110904A1 (en) 2020-01-31 2021-01-29 Systems and methods for artifact reduction in tomosynthesis with deep learning image processing

Country Status (2)

Country Link
US (1) US20230110904A1 (en)
WO (1) WO2021155123A1 (en)

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10610182B2 (en) * 2014-01-15 2020-04-07 Alara Systems, Inc Converting low-dose to higher dose 3D tomosynthesis images through machine-learning processes
WO2017223560A1 (en) * 2016-06-24 2017-12-28 Rensselaer Polytechnic Institute Tomographic image reconstruction via machine learning
US10489907B2 (en) * 2017-11-13 2019-11-26 Siemens Healthcare Gmbh Artifact identification and/or correction for medical imaging

Also Published As

Publication number Publication date
WO2021155123A1 (en) 2021-08-05

Similar Documents

Publication Publication Date Title
US11948314B2 (en) Systems and methods for image processing
US10839567B2 (en) Systems and methods for correcting mismatch induced by respiratory motion in positron emission tomography image reconstruction
JP7245364B2 (en) sCT Imaging Using CycleGAN with Deformable Layers
US20230085203A1 (en) Systems and methods for image reconstruction
US10690782B2 (en) Systems and methods for positron emission tomography image reconstruction
US20200167929A1 (en) Image processing method, image processing apparatus, and computer-program product
US11270478B2 (en) System and method for reconstructing a computed tomography image
US11250601B2 (en) Learning-assisted multi-modality dielectric imaging
US9775582B2 (en) Medical image photographing apparatus and method of processing medical image
CN106999138B (en) Diagnostic imaging method and apparatus and recording medium therefor
US8049752B2 (en) Systems and methods of determining sampling rates for volume rendering
US20200242744A1 (en) Forecasting Images for Image Processing
US10032295B2 (en) Tomography apparatus and method of processing tomography image
Zeng et al. Cerebral perfusion computed tomography deconvolution via structure tensor total variation regularization
US11995745B2 (en) Systems and methods for correcting mismatch induced by respiratory motion in positron emission tomography image reconstruction
US20150117742A1 (en) Medical imaging apparatus and method of reconstructing medical image
US20230061863A1 (en) Systems and methods for artifact reduction in tomosynthesis with multi-scale deep learning image processing
US20230419455A1 (en) System and method for image correction
US20230110904A1 (en) Systems and methods for artifact reduction in tomosynthesis with deep learning image processing
US10685461B1 (en) Apparatus and method for context-oriented iterative reconstruction for computed tomography (CT)
WO2023205726A1 (en) Deep learning based image reconstruction
US20220383491A1 (en) Tissue boundary determination apparatus and method
US11986337B2 (en) Dose reduction for cardiac computed tomography
US20160166218A1 (en) System and method for expectation maximization reconstruction for gamma emission breast tomosynthesis
CN115375840A (en) Image reconstruction method, device, system, computer equipment and storage medium

Legal Events

Date Code Title Description
AS Assignment

Owner name: GENERAL HOSPITAL CORPORATION, THE, MASSACHUSETTS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:WU, DUFAN;KIM, KYUNGSANG;LI, QUANZHENG;SIGNING DATES FROM 20220427 TO 20220502;REEL/FRAME:060691/0832

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION