EP4626319A1

EP4626319A1 - System and method for medical imaging

Info

Publication number: EP4626319A1
Application number: EP22969868.3A
Authority: EP
Inventors: Liyi Zhao; Wenjing Cao
Original assignee: Shanghai United Imaging Healthcare Co Ltd
Current assignee: Shanghai United Imaging Healthcare Co Ltd
Priority date: 2022-12-30
Filing date: 2022-12-30
Publication date: 2025-10-08
Also published as: EP4626319A4; CN120456863A; WO2024138726A1

Abstract

The present disclosure provides a method for multi-energy computed tomography. The method includes: obtaining raw date acquired using a multi-energy computed tomography(CT) device; determining, based on the raw data, a cost function including a data fidelity item and a regularization item, the data fidelity item correlating at least one candidate image with the raw data, and the regularization item being configured to reduce noise in the at least one candidate image and being determined through a deep learning model; and generating at least one target image by iteratively optimizing the cost function.

Description

SYSTEM AND METHOD FOR MEDICAL IMAGING

TECHNICAL FIELD

The present disclosure generally relates to medical imaging, and in particular, to systems and methods for multi-energy computed tomography (MECT) .

BACKGROUND

Medical imaging, such as MECT is widely used in disease diagnosis and/or treatment for various medical conditions (e.g., tumors, coronary heart diseases, or brain diseases) . Image reconstruction is a key technology for the MECT. Taking a base material image as an example, conventionally, the base material image can be reconstructed based on reconstruction methods. However, a computation load may be significantly large during the reconstruction, and the quality of the reconstructed base material image can hardly be satisfying. Therefore, it is desirable to provide systems and methods for image reconstruction with improved image quality, thereby improving the efficiency and accuracy of medical analysis and/or diagnosis.
SUMMARY
According to another aspect of the present disclosure, a method is provided. The method may be implemented on a computing device having a processor and a computer-readable storage device. The method may comprise obtaining raw data acquired using a multi-energy CT device; determining, based on the raw data, a cost function including a data fidelity item and a regularization item, the data fidelity item correlating at least one candidate image with the raw data, and the regularization item being configured to reduce noise in the at least one candidate image and being determined through a deep learning model; and generating at least one target image by iteratively optimizing the cost function.
In some embodiments, the at least one candidate image includes at least one candidate base material image, and the at least one target image includes at least one target base material image; or the at least one candidate image includes at least one candidate single energy bin image, and the at least one target image includes at least one target single energy bin image.
In some embodiments, the raw data includes beam intensity data.
In some embodiments, the data fidelity item relates to at least one of spectrum information of X-rays generated by the multi-energy CT device or response information of detectors of the multi-energy CT device.
In some embodiments, the raw data includes raw data of base materials, the raw data being determined by performing a base material decomposition in a data domain based on beam intensity data generated based on the multi-energy CT device and spectrum information of X-rays generated by the multi-energy CT device.
In some embodiments, the raw data includes raw data of base materials, the raw data being determined by performing a base material decomposition in a data domain based on beam intensity data generated based on the multi-energy CT device and response information of detectors of the multi-energy CT device.
In some embodiments, the raw data is generated by performing a data correction on original scan data generated by the multi-energy CT device.
In some embodiments, the iteratively optimizing the cost function includes: iteratively optimizing the data fidelity item and the regularization item simultaneously or alternately.
In some embodiments, the regularization item includes a deep learning network.
In some embodiments, the deep learning network is trained offline.
In some embodiments, the deep learning network is trained using training inputs that include material decomposition images of one or more base materials.
In some embodiments, each of the one or more base materials corresponds to a deep learning sub-network of the deep learning network.
In some embodiments, the deep learning network is trained using a plurality of training samples, each training sample includes a training target that includes first images generated by iterative image reconstruction on high dose data acquired by high dose scans.
In some embodiments, each training sample further include a training input that is generated by combining the training target of the training sample and noise images, each of the noise images represents a difference between one of the first images of the training sample and a second image, the second image being generated by: determining low dose data by performing a low dose simulation on the high dose data, and reconstructing the second image by performing one or more non-regularization iterations on the low dose data.
According to one aspect of the present disclosure, a system is provided. The system may include at least one storage device storing a set of instructions and at least one processor configured to communicate with the at least one storage device. When executing the set of instructions, the at least one processor may be configured to direct the system to perform the following operations. The following operations may include obtaining raw data acquired using a multi-energy CT device; determining, based on the raw data, a cost function including a data fidelity item and a regularization item, the data fidelity item correlating at least one candidate image with the raw data, and the regularization item being configured to reduce noise in the at least one candidate image and being determined through a deep learning model; and generating at least one target image by iteratively optimizing the cost function.
According to a further aspect of the present disclosure, a non-transitory computer-readable storage medium may be provided. The non-transitory computer-readable storage medium may include instructions, that, when accessed by at least one processor of a system, causes the system to perform a method. The method may comprise obtaining raw data acquired using a multi-energy CT device; determining, based on the raw data, a cost function including a data fidelity item and a regularization item, the data fidelity item correlating at least one candidate image with the raw data, and the regularization item being configured to reduce noise in the at least one candidate image and being determined through a deep learning model; and generating at least one target image by iteratively optimizing the cost function.
Additional features will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following and the accompanying drawings or may be learned by production or operation of the examples. The features of the present disclosure may be realized and attained by practice or use of various aspects of the methodologies, instrumentalities and combinations set forth in the detailed examples discussed below.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is further described in terms of exemplary embodiments. These exemplary embodiments are described in detail with reference to the drawings. These embodiments are non-limiting exemplary embodiments, in which like reference numerals represent similar structures throughout the several views of the drawings, and wherein:
FIG. 1 is a schematic diagram illustrating an exemplary imaging system according to some embodiments of the present disclosure;
FIG. 2 is a schematic diagram illustrating exemplary hardware and/or software components of an exemplary computing device according to some embodiments of the present disclosure;
FIG. 3 is a schematic diagram illustrating hardware and/or software components of an exemplary mobile device according to some embodiments of the present disclosure;
FIG. 4 is a block diagram illustrating an exemplary processing device according to some embodiments of the present disclosure;
FIG. 5 is a flowchart illustrating an exemplary process for generating at least one target base material image according to some embodiments of the present disclosure;
FIG. 6A is a flowchart illustrating an exemplary process of an iteration in the optimization process according to some embodiments of the present disclosure;
FIG. 6B is a schematic diagram illustrating an exemplary process for generating at least one target base material image according to some embodiments of the present disclosure;
FIG. 7 is a flowchart illustrating an exemplary process for determining a deep learning network according to some embodiment of the present disclosure; and
FIG. 8 illustrates exemplary training samples for training a deep learning network according to some embodiments of the present disclosure.

DETAILED DESCRIPTION

In the following detailed description, numerous specific details are set forth by way of examples in order to provide a thorough understanding of the relevant disclosure. However, it should be apparent to those skilled in the art that the present disclosure may be practiced without such details. In other instances, well-known methods, procedures, systems, components, and/or circuitry have been described at a relatively high-level, without detail, in order to avoid unnecessarily obscuring aspects of the present disclosure. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the present disclosure. Thus, the present disclosure is not limited to the embodiments shown, but to be accorded the widest scope consistent with the claims.
The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting. As used herein, the singular forms “a, ” “an, ” and “the” may be intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprise, ” “comprises, ” and/or “comprising, ” “include, ” “includes, ” and/or “including, ” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It will be understood that the terms “system, ” “engine, ” “unit, ” “module, ” and/or “block” used herein are one method to distinguish different components, elements, parts, sections, or assemblies of different levels in ascending order. However, the terms may be displaced by another expression if they achieve the same purpose.
Generally, the word “module, ” “unit, ” or “block, ” as used herein, refers to logic embodied in hardware or firmware, or to a collection of software instructions. A module, a unit, or a block described herein may be implemented as software and/or hardware and may be stored in any type of non-transitory computer-readable medium or another storage device. In some embodiments, a software module/unit/block may be compiled and linked into an executable program. It will be appreciated that software modules can be callable from other modules/units/blocks or from themselves, and/or may be invoked in response to detected events or interrupts. Software modules/units/blocks configured for execution on computing devices (e.g., processor 210 as illustrated in FIG. 2) may be provided on a computer-readable medium, such as a compact disc, a digital video disc, a flash drive, a magnetic disc, or any other tangible medium, or as a digital download (and can be originally stored in a compressed or installable format that needs installation, decompression, or decryption prior to execution) . Such software code may be stored, partially or fully, on a storage device of the executing computing device, for execution by the computing device. Software instructions may be embedded in firmware, such as an EPROM. It will be further appreciated that hardware modules/units/blocks may be included in connected logic components, such as gates and flip-flops, and/or can be included of programmable units, such as programmable gate arrays or processors. The modules/units/blocks or computing device functionality described herein may be implemented as software modules/units/blocks, but may be represented in hardware or firmware. In general, the modules/units/blocks described herein refer to logical modules/units/blocks that may be combined with other modules/units/blocks or divided into sub-modules/sub-units/sub-blocks despite their physical organization or storage. The description may be applicable to a system, an engine, or a portion thereof.
It will be understood that when a unit, engine, module or block is referred to as being “on, ” “connected to, ” or “coupled to, ” another unit, engine, module, or block, it may be directly on, connected or coupled to, or communicate with the other unit, engine, module, or block, or an intervening unit, engine, module, or block may be present, unless the context clearly indicates otherwise. As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items.
These and other features, and characteristics of the present disclosure, as well as the methods of operation and functions of the related elements of structure and the combination of parts and economies of manufacture, may become more apparent upon consideration of the following description with reference to the accompanying drawings, all of which form a part of this disclosure. It is to be expressly understood, however, that the drawings are for the purpose of illustration and description only and are not intended to limit the scope of the present disclosure. It is understood that the drawings are not to scale.
The flowcharts used in the present disclosure illustrate operations that systems implement according to some embodiments of the present disclosure. It is to be expressly understood the operations of the flowcharts may be implemented not in order. Conversely, the operations may be implemented in an inverted order, or simultaneously. Moreover, one or more other operations may be added to the flowcharts. One or more operations may be removed from the flowcharts.
Provided herein are systems and methods for non-invasive imaging, such as for disease diagnosis, treatment, and/or research purposes. In some embodiments, the imaging system may include a single modality system and/or a multi-modality system. The term “modality” used herein broadly refers to an imaging or treatment method or technology that gathers, generates, processes, and/or analyzes imaging information of a subject or treatments the subject. The single modality system may include a computed tomography (CT) system. The multi-modality system may include a positron emission tomography-computed tomography (PET-CT) system, a magnetic resonance imaging-computed tomography (MRI-CT) system, or the like, or any combination thereof.
In the present disclosure, the term “image” may refer to a two-dimensional (2D) image, a three-dimensional (3D) image, or a four-dimensional (4D) image. In some embodiments, the term “image” may refer to an image of a region (e.g., a region of interest (ROI) ) of a subject. In some embodiments, the image may be a CT image, etc.
An aspect of the present disclosure relates to systems and methods for medical imaging. During the medical imaging process, raw data acquired using an MECT device may be obtained. A cost function including a data fidelity item and a regularization item may be determined based on the raw data. The data fidelity item may correlate at least one candidate image with the raw data, and the regularization item may be determined through a deep learning model. Reconstructed images, such as at least one target image may be generated by iteratively optimizing the cost function. In some embodiments, the deep learning model may be used to determine the regularization item, which is configured to reduce noise (e.g., artifacts) in the at least one candidate image, thereby improving the image quality of the at least one target image. In addition, the data fidelity item and the regularization item may be optimized in a plurality of iterations independently, thus improving the efficiency of the imaging process.
In some embodiments, the at least one candidate image may include at least one candidate base material image, and the at least one target image may include at least one target base material image. Additionally or alternatively, the at least one candidate image may include at least one candidate single energy bin image, and the at least one target image may include at least one target single energy bin image. For illustration purposes, the generation of the at least one target base material image is described hereinafter as an example. It should be noted that the systems and methods disclosed herein can also be used to generate at least one target single energy bin image. For example, in the descriptions hereinafter, the term “candidate base material image” may be replaced by “candidate single energy bin image, ” and the term “target base material image” may be replaced by “target single energy bin image. ”
FIG. 1 is a schematic diagram illustrating an exemplary imaging system 100 according to some embodiments of the present disclosure. As illustrated in FIG. 1, the imaging system 100 may include a scanner 110, a processing device 120, a storage device 130, a terminal device 140, and a network 150. In some embodiments, two or more components of the imaging system 100 may be connected to and/or communicate with each other via a wireless connection, a wired connection, or a combination thereof. The connection among the components of the imaging system 100 may be variable. Merely by way of example, the scanner 110 may be connected to the processing device 120 through the network 150 or directly. As another example, the storage device 130 may be connected to the processing device 120 through the network 150 or directly.
The scanner 110 may be configured to scan a subject or a portion thereof that is located within its detection region and generate scanning data/signals relating to the (portion of) subject.
In some embodiments, the scanner 110 may include a single modality device. For example, the scanner 110 may include a CT scanner. In some embodiments, the scanner 110 may be a multi-modality device. For example, the scanner 110 may include a PET-CT scanner. The following descriptions are provided, unless otherwise stated expressly, with reference to a CT scanner for illustration purposes and are not intended to be limiting.
As illustrated, the CT scanner may include a gantry 111, a detector 112, a detecting region 113, a table 114, and a radiation source 115. The gantry 111 may support the detector 112 and the radiation source 115. The subject may be placed on the table 114 for scanning. The radiation source 115 may emit x-rays. The x-rays may be emitted from a focal spot using a high-intensity magnetic field to form an x-ray beam. The x-ray beam may travel toward the subject. The detector 112 may detect x-ray photons from the detecting region 113. In some embodiments, the detector 112 may include one or more detector units. The detector unit (s) may be and/or include single-row detector elements and/or multi-row detector elements.
In some embodiments, the CT scanner may be an MECT device. The MECT device may also be referred to as a spectral CT device. The MECT device may irradiate x-rays having different energy spectra. In some embodiments, an emission voltage of the radiation source 115 of the MECT device may be switched among two or more different emission voltages (e.g., 80 kilovolt peak (kVp) , 100 kVp, 120 kVp, 140 kVp, etc. ) such that x-rays of different energy spectra may be irradiated. In some embodiments, the radiation source 115 of the MECT device may include multiple x-ray tubes. The multiple x-ray tubes may emit x-rays at different emission voltages such that x-rays of different energy spectra may be irradiated. In some embodiments, the radiation source 115 of the MECT device may include a broad-spectrum x-ray tube. The broad-spectrum x-ray tube may irradiate x-rays of different energy spectra.
In some embodiments, the detector 112 of the MECT device may generate a signal for each of the different emission voltages. In some embodiments, as for the broad-spectrum x-ray tube, the detector 112 may include an energy-resolving detector that generates a signal for each of the different energy spectra. The energy-resolving detector may include, e.g., a multi-layered scintillator/photodiode, a direct conversion photon counting detector, etc.
The processing device 120 may process data and/or information. The data and/or information may be obtained from the scanner 110 or retrieved from the storage device 130, the terminal device 140, and/or an external device (external to the imaging system 100) via the network 150. In some embodiments, the processing device 120 may be a single server or a server group. The server group may be centralized or distributed. In some embodiments, the processing device 120 may be local or remote. For example, the processing device 120 may access information and/or data stored in the scanner 110, the terminal device 140, and/or the storage device 130 via the network 150. As another example, the processing device 120 may be directly connected to the scanner 110, the terminal device 140, and/or the storage device 130 to access stored information and/or data. In some embodiments, the processing device 120 may be implemented on a cloud platform. Merely by way of example, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter- cloud, a multi-cloud, or the like, or any combination thereof. In some embodiments, the processing device 120 may be implemented by a computing device 200 having one or more components as illustrated in FIG. 2.
The storage device 130 may store data, instructions, and/or any other information. In some embodiments, the storage device 130 may store data obtained from the scanner 110, the terminal device 140, and/or the processing device 120. In some embodiments, the storage device 130 may store data and/or instructions that the processing device 120 may execute or use to perform exemplary methods described in the present disclosure. In some embodiments, the storage device 130 may include a mass storage device, a removable storage device, a volatile read-and-write memory, a read-only memory (ROM) , or the like, or any combination thereof. Exemplary mass storage may include a magnetic disk, an optical disk, a solid-state drive, etc. Exemplary removable storage may include a flash drive, a floppy disk, an optical disk, a memory card, a zip disk, a magnetic tape, etc. Exemplary volatile read-and-write memory may include a random access memory (RAM) . Exemplary RAM may include a dynamic RAM (DRAM) , a double date rate synchronous dynamic RAM (DDR SDRAM) , a static RAM (SRAM) , a thyristor RAM (T-RAM) , and a zero-capacitor RAM (Z-RAM) , etc. Exemplary ROM may include a mask ROM (MROM) , a programmable ROM (PROM) , an erasable programmable ROM (EPROM) , an electrically erasable programmable ROM (EEPROM) , a compact disk ROM (CD-ROM) , and a digital versatile disk ROM, etc. In some embodiments, the storage device 130 may be implemented on a cloud platform. Merely by way of example, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud, a multi-cloud, or the like, or any combination thereof.
In some embodiments, the storage device 130 may be connected to the network 150 to communicate with one or more other components (e.g., the processing device 120, the terminal device 140) of the imaging system 100. One or more components of the imaging system 100 may access the data or instructions stored in the storage device 130 via the network 150. In some embodiments, the storage device 130 may be directly connected to or communicate with one or more other components (e.g., the processing device 120, the terminal device 140) of the imaging system 100. In some embodiments, the storage device 130 may be part of the processing device 120.
The terminal device 140 may input/output signals, data, information, etc. In some embodiments, the terminal device 140 may enable a user interaction with the processing device 120. For example, the terminal device 140 may display an image of the subject on a screen. As another example, the terminal device 140 may obtain a user’s input information through an input device (e.g., a keyboard, a touch screen, a brain wave monitoring device) , and transmit the input information to the processing device 120 for further processing. The terminal device 140 may be a mobile device, a tablet computer, a laptop computer, a desktop computer, or the like, or any combination thereof. In some embodiments, the mobile device may include a home device, a wearable device, a virtual reality device, an augmented reality device, or the like, or any combination thereof. The home device may include a lighting device, a control device of an intelligent electrical apparatus, a monitoring device, a television, a video camera, an interphone, or the like, or any combination thereof. The wearable device may include a bracelet, a footgear, eyeglasses, a helmet, a watch, clothing, a backpack, an accessory, or the like, or any combination thereof. The virtual reality device and/or the augmented reality device may include a virtual reality helmet, virtual reality glasses, a virtual reality patch, an augmented reality helmet, augmented reality glasses, an augmented reality patch, or the like, or any combination thereof. For example, the virtual reality device and/or the augmented reality device may include a Google Glass ^TM, an Oculus Rift ^TM, a Hololens ^TM, a Gear VR ^TM, etc. In some embodiments, the terminal device 140 may be part of the processing device 120 or a peripheral device of the processing device 120 (e.g., a console connected to and/or communicating with the processing device 120) .
The network 150 may include any suitable network that can facilitate the exchange of information and/or data for the imaging system 100. In some embodiments, one or more components (e.g., the scanner 110, the terminal device 140, the processing device 120, the storage device 130) of the imaging system 100 may communicate information and/or data with one or more other components of the imaging system 100 via the network 150. The network 150 may be and/or include a public network (e.g., the Internet) , a private network (e.g., a local area network (LAN) , a wide area network (WAN) ) ) , a wired network (e.g., an Ethernet network) , a wireless network (e.g., an 802.11 network, a Wi-Fi network) , a cellular network (e.g., a Long Term Evolution (LTE) network, 4G network, 5G network) , a frame relay network, a virtual private network (VPN) , a satellite network, a telephone network, routers, hubs, switches, server computers, and/or any combination thereof. Merely by way of example, the network 150 may include a cable network, a wireline network, a fiber-optic network, a telecommunications network, an intranet, a wireless local area network (WLAN) , a metropolitan area network (MAN) , a public telephone switched network (PSTN) , a Bluetooth ^TM network, a ZigBee ^TM network, a near field communication (NFC) network, or the like, or any combination thereof. In some embodiments, the network 150 may include one or more network access points. For example, the network 150 may include wired and/or wireless network access points such as base stations and/or internet exchange points through which one or more components of the imaging system 100 may be connected to the network 150 to exchange data and/or information.
It should be noted that the above description regarding the imaging system 100 is merely provided for the purposes of illustration, and not intended to limit the scope of the present disclosure. For persons having ordinary skills in the art, multiple variations and modifications may be made under the teachings of the present disclosure. However, those variations and modifications do not depart from the scope of the present disclosure. In some embodiments, the imaging system 100 may include one or more additional components and/or one or more components of the imaging system 100 described above may be omitted. In some embodiments, a component of the imaging system 100 may be implemented on two or more sub-components. Two or more components of the imaging system 100 may be integrated into a single component.
FIG. 2 is a schematic diagram illustrating exemplary hardware and/or software components of an exemplary computing device according to some embodiments of the present disclosure. The computing device 200 may be configured to implement any component of the imaging system 100. For example, the scanner 110, the processing device 120, the storage device 130, and/or the terminal device 140 may be implemented on the computing device 200. Although only one such computing device is shown for convenience, the computer functions relating to the imaging system 100 as described herein may be implemented in a distributed fashion on a number of similar platforms, to distribute the processing load. As illustrated in FIG. 2, the computing device 200 may include a processor 210, a storage 220, an input/output (I/O) 230, and a communication port 240.
The processor 210 may execute computer instructions (e.g., program codes) and perform functions of the processing device 120 in accordance with the techniques described herein. The computer instructions may include, for example, routines, programs, objects, components, signals, data structures, procedures, modules, and functions, which perform particular functions described herein. In some embodiments, the processor 210 may perform instructions obtained from the terminal device 140 and/or the storage device 130. In some embodiments, the processor 210 may include one or more hardware processors, such as a microcontroller, a microprocessor, a reduced instruction set computer (RISC) , an application-specific integrated circuits (ASICs) , an application-specific instruction-set processor (ASIP) , a central processing unit (CPU) , a graphics processing unit (GPU) , a physics processing unit (PPU) , a microcontroller unit, a digital signal processor (DSP) , a field-programmable gate array (FPGA) , an advanced RISC machine (ARM) , a programmable logic device (PLD) , any circuit or processor capable of executing one or more functions, or the like, or any combinations thereof.
Merely for illustration, only one processor is described in the computing device 200. However, it should be noted that the computing device 200 in the present disclosure may also include multiple processors. Thus operations and/or method steps that are performed by one processor as described in the present disclosure may also be jointly or separately performed by the multiple processors. For example, if in the present disclosure the processor of the computing device 200 executes both operation A and operation B, it should be understood that operation A and operation B may also be performed by two or more different processors jointly or separately in the computing device 200 (e.g., a first processor executes operation A and a second processor executes operation B, or the first and second processors jointly execute operations A and B) .
The storage 220 may store data/information obtained from the scanner 110, the terminal device 140, the storage device 130, or any other component of the imaging system 100. In some embodiments, the storage 220 may include a mass storage device, a removable storage device, a volatile read-and-write memory, a read-only memory (ROM) , or the like, or any combination thereof. In some embodiments, the storage 220 may store one or more programs and/or instructions to perform exemplary methods described in the present disclosure.
The I/O 230 may input or output signals, data, and/or information. In some embodiments, the I/O 230 may enable user interaction with the processing device 120. In some embodiments, the I/O 230 may include an input device and an output device. Exemplary input devices may include a keyboard, a mouse, a touch screen, a microphone, a camera capturing gestures, or the like, or a combination thereof. Exemplary output devices may include a display device, a loudspeaker, a printer, a projector, a 3D hologram, a light, a warning light, or the like, or a combination thereof. Exemplary display devices may include a liquid crystal display (LCD) , a light-emitting diode (LED) -based display, a flat panel display, a curved screen, a television device, a cathode ray tube (CRT) , or the like, or a combination thereof.
The communication port 240 may be connected with a network (e.g., the network 150) to facilitate data communications. The communication port 240 may establish connections between the processing device 120 and the scanner 110, the terminal device 140, or the storage device 130. The connection may be a wired connection, a wireless connection, or a combination of both that enables data transmission and reception. The wired connection may include an electrical cable, an optical cable, a telephone wire, or the like, or any combination thereof. The wireless connection may include a Bluetooth network, a Wi-Fi network, a WiMax network, a WLAN, a ZigBee network, a mobile network (e.g., 3G, 4G, 5G) , or the like, or any combination thereof. In some embodiments, the communication port 240 may be a standardized communication port, such as RS232, RS485, etc. In some embodiments, the communication port 240 may be a specially designed communication port. For example, the communication port 240 may be designed in accordance with the digital imaging and communications in medicine (DICOM) protocol.
FIG. 3 is a schematic diagram illustrating exemplary hardware and/or software components of an exemplary mobile device according to some embodiments of the present disclosure. In some embodiments, the processing device 120 or the terminal device 140 may be implemented on the mobile device 300. As illustrated in FIG. 3, the mobile device 300 may include a communication module 310, a display 320, a graphics processing unit (GPU) 330, a central processing unit (CPU) 340, an I/O 350, a memory 360, and storage 390. The CPU 340 may include interface circuits and processing circuits similar to the processor 210. In some embodiments, any other suitable component, including but not limited to a system bus or a controller (not shown) , may also be included in the mobile device 300. In some embodiments, a mobile operating system 370 (e.g., iOS ^TM, Android ^TM, Windows Phone ^TM) and one or more applications 380 may be loaded into the memory 360 from the storage 390 in order to be executed by the CPU 340. The applications 380 may include a browser or any other suitable mobile apps for receiving and rendering information relating to imaging from the imaging system 100 on the mobile device 300. User interactions with the information stream may be achieved via the I/O devices 350 and provided to the processing device 120 and/or other components of the imaging system 100 via the network 150.
To implement various modules, units, and their functionalities described in the present disclosure, computer hardware platforms may be used as the hardware platform (s) for one or more of the elements described herein. A computer with user interface elements may be used to implement a personal computer (PC) or any other type of work station or terminal device. A computer may also act as a server if appropriately programmed.
FIG. 4 is a block diagram illustrating an exemplary processing device according to some embodiments of the present disclosure. As illustrated in FIG. 4, the processing device 120 may include an obtaining module 410, a determination module 420, and an optimizing module 430, and a training module 440.
The obtaining module 410 may be configured to obtain data and/or information. The obtaining module 410 may obtain data and/or information from the scanner 110, the storage device 130, the terminal (s) 140, or any devices or components capable of storing data via the network 150. In some embodiments, the obtaining module 410 may obtain raw data acquired using an MECT device. In some embodiments, the obtaining module 410 may obtain the raw data from the MECT device directly. For example, during a scan of the subject using the MECT device, the raw data may be obtained from the MECT device in real time. In some embodiments, the obtaining module 410 may obtain the raw data from a storage device (e.g., the storage device 130, the storage 220, the storage 390, a cloud storage, etc. ) .
The determination module 420 may determine a cost function based on the raw data. The cost function may include a data fidelity item and a regularization item. The data fidelity item may characterize a fidelity of at least one candidate base material image. As used herein, a candidate base material image refers to a candidate image of a base material generated during an optimization process of the cost function. The data fidelity item may correlate the at least one candidate base material image with the raw data. For example, the data fidelity item may relate to a difference between the raw data and projection data of the at least one candidate base material image, or a difference between the raw data and beam intensity data of the at least one candidate base material image. The regularization item refers to an item that may be configured to regularize the at least one candidate base material image during the optimization process of the cost function. For example, the regularization item may be used to reduce noise in the at least one candidate base material image.
The optimizing module 430 may generate at least one target base material image by iteratively optimizing the cost function. The optimization process of the cost function may include a plurality of iterations. After a final iteration of the optimization process is complete, at least one candidate base material image generated in the final iteration may be designated as the at least one target base material image. In some embodiments, each of the one or more base materials may correspond to one of the at least one target base material image.
The training module 440 may determine a deep learning network by training a preliminary deep learning network based on the plurality of training samples. In some embodiments, the processing device 120 may obtain training samples (e.g., training samples including a training input 802 and a training target 801) , and train the preliminary deep learning network iteratively based on the training samples until a termination condition is satisfied.
The modules in the processing device 120 may be connected to or communicated with each other via a wired connection or a wireless connection. The wired connection may include a metal cable, an optical cable, a hybrid cable, or the like, or any combination thereof. The wireless connection may include a Local Area Network (LAN) , a Wide Area Network (WAN) , a Bluetooth, a ZigBee, a Near Field Communication (NFC) , or the like, or any combination thereof. Two or more of the modules may be combined into a single module, and any one of the modules may be divided into two or more units. For example, the above-mentioned modules may be integrated into a console (not shown) . Via the console, a user may set parameters for scanning a subject, controlling imaging processes, controlling parameters for reconstruction of an image, etc. As another example, the processing device 120 may include a storage module (not shown) configured to store information and/or data (e.g., scanning data, images) associated with the above-mentioned modules. As yet another example, the training module 440 may be implemented on a processing device different from the processing device 120, such as a processing device of a vendor of a deep learning network disclosed herein.
FIG. 5 is a flowchart illustrating an exemplary process for generating at least one target base material image according to some embodiments of the present disclosure. In some embodiments, the process 500 may be executed by the imaging system 100. For example, the process 500 may be implemented as a set of instructions (e.g., an application) stored in a storage device (e.g., the storage device 130) . The processing device 120 (e.g., the modules described in FIG. 4) and/or the processor 210 may execute the set of instructions and may accordingly be directed to perform the process 500. The operations of the illustrated process presented below are intended to be illustrative. In some embodiments, the process 500 may be accomplished with one or more additional operations not described and/or without one or more of the operations discussed. Additionally, the order of the operations of process 500 illustrated in FIG. 5 and described below is not intended to be limiting.
In 510, the processing device 120 (e.g., the processor 210, the obtaining module 410) may obtain raw data acquired using a multi-energy CT device.
As set forth above, the MECT device may also be referred to as a spectral CT device. The MECT device may be configured to emit multi-energy x-rays for performing a multi-energy scan of the subject. The raw data may be acquired via the multi-energy scan using the MECT device. The subject may include a biological subject and/or a non-biological subject. The biological subject may be a human being, an animal, a plant, or a specific portion, an organ, and/or tissue thereof. For example, the subject may include a lesion (e.g., a tumor) in the brain, the thorax, the stomach, soft tissue, etc., of a patient. In some embodiments, the subject may be a man-made composition of organic and/or inorganic matters that are with or without life.
In some embodiments, the raw data may include beam intensity data generated based on the MECT device. The beam intensity data refers to data regarding a beam intensity of the multi-energy x-rays collected by the MECT device. The beam intensity data may be data in an intensity domain. In some embodiments, the beam intensity data may be original data regarding a beam intensity of the multi-energy x-rays collected by the MECT device or data generated by processing the original data (e.g., correcting the original data) . In some embodiments, the beam intensity data may also be referred to as intensity domain data.
In some embodiments, the raw data may include data in a log domain. For example, the raw data may include data relating to attenuation of X-rays by the scanned subject along a transmission path, which may be determined by performing log operation on photon numbers collected by the MECT device.
In some embodiments, the raw data may include raw data of base materials. In some embodiments, the base materials may include materials of different densities. For example, a pair of materials including a low density material (e.g., water) and a high density material (e.g., iodine) may be used as the base materials. The attenuation of x-rays having different energy spectra by different base materials may be transformed into raw data in the intensity domain, and a difference between the raw data in the intensity domain and intensity domain data transformed from images may be determined. In some embodiments, the raw data may be determined by performing a base material decomposition (BMD) in a data domain on the beam intensity data and spectrum information of x-rays generated by the MECT device. In some embodiments, the raw data may be data in an attenuation domain. In some embodiments, the raw data may also be referred to as attenuation domain data. For example, raw data of a specific base material may reflect an attenuation capacity of the base material in the subject to X-rays having different energy spectra.
In some embodiments, the raw data may be obtained from the MECT device directly. For example, during a scan of the subject using the MECT device, the raw data may be obtained from the MECT device in real time. In some embodiments, the raw data may be retrieved from a storage device (e.g., the storage device 130, the storage 220, the storage 390, a cloud storage, etc. ) .
In some embodiments, the raw data (e.g., the beam intensity data) may be generated by performing a data correction on original scan data collected by the MECT device. The data correction may be used to calibrate the responses of different detectors of the MECT device. Exemplary data corrections on may include air correction, beam hard correction, pileup correction, or the like, or any combination thereof.
In 520, the processing device 120 (e.g., the processor 210, the determination module 420) may determine a cost function based on the raw data.
The cost function may include a data fidelity item and a regularization item. The data fidelity item may characterize a fidelity of at least one candidate base material image. As used herein, a candidate base material image refers to a candidate image of a base material generated during an optimization process of the cost function. The data fidelity item may correlate the at least one candidate base material image with the raw data. For example, the data fidelity item may relate to a difference between the raw data and projection data of the at least one candidate base material image, or a difference between the raw data and beam intensity data corresponding to the at least one candidate base material image.
The regularization item refers to an item that may be configured to regularize the at least one candidate base material image during the optimization process of the cost function. For example, the regularization item may be used to reduce noise in the at least one candidate base material image.
In some embodiments, the regularization item may be determined through a deep learning model. Exemplary deep learning models may include a deep neural network (DNN) model, a convolutional neural network (CNN) model, a recurrent neural network (RNN) model, a feature pyramid network (FPN) model, etc. Exemplary CNN models may include a V-Net model, a U-Net model, a FB-Net model, a Link-Net model, or the like, or any combination thereof. Details regarding the determination of the regularization item may be found elsewhere in the present disclosure. See, for example, FIG. 7 and the descriptions thereof.
In some embodiments, the regularization item may include a deep learning network. The deep learning network may be trained offline (e.g., previously generated and stored in a storage device) . The deep learning network may be trained using a plurality of training samples. A training sample may include a pair of a training input and a training target.
In some embodiments, the raw data may include beam intensity data. Correspondingly, the cost function may be determined according to Equation (1) :
E (U ₁, …, U _M) =∑ _n‖∑ _m∫dES _n (E) exp [-FP (U _m) μ _m (E) ] -Y _n‖ ²+R (U ₁, …, U _M) , (1)
where Y _n denotes the beam intensity data, n denotes a serial number of an energy spectrum, M denotes a count of base materials, m denotes a serial number of a base material among M base materials, U _m denotes a candidate base material image of the m ^th base material, E denotes energy of x-rays generated by the scanner 110, S _n (E) denotes spectrum information of the x-rays generated by the scanner or response information of detectors of the scanner 110, FP denotes a forward projection operator, μ _m (E) denotes a spectrum absorptivity of a base material, ∑ _m∫dES _n (E) exp [-FP (U _m) μ _m (E) ] denotes beam intensity data corresponding to the at least one candidate base material image, and R (U ₁, …, U _M) denotes the regularization item. As used herein, the spectrum absorptivity refers to an absorptivity with respect to x-ray of an energy spectrum. The serial number of a base material m may be 1, 2, …, or M. The count of base materials M may be an integer larger than 1.
In the equation (1) provided above, ∑ _n‖∑ _m∫dES _n (E) exp [-FP (U _m) μ _m (E) ] -Y _n‖ ² may represent the data fidelity item. According to Equation (1) , the data fidelity item relates to S _n (E) , that is, in cases where the raw data includes the beam intensity data, the data fidelity item may relate to at least one of spectrum information of x-rays generated by the MECT device or response information of detectors of the multi-energy CT device.
In some embodiments, the raw data may include raw data of base materials. Correspondingly, the cost function may be determined according to Equation (2) :
E (U ₁, …, U _M) =∑ _m‖FP (U _m) -Y _m‖ ²+R (U ₁, …, U _M) , (2)
where Y _m denotes the raw data of the m ^th base material, FP (U _m) denotes projection data corresponding to the at least one candidate base material image.
In 530, the processing device 120 (e.g., the processor 210, the optimizing module 430) may generate at least one target base material image by iteratively optimizing the cost function.
The optimization process of the cost function may include a plurality of iterations. After a final iteration of the optimization process is complete, at least one candidate base material image generated in the final iteration may be designated as the at least one target base material image. In some embodiments, each of the one or more base materials may correspond to one of the at least one target base material image.
During the optimization process of the cost function, the data fidelity item and the regularization item may be optimized simultaneously or alternately. The optimization of the data fidelity item may also be referred to as a first optimization operation. The optimization of the regularization item may also be referred to as a second optimization operation. The optimization process may include at least one first optimization operation and at least one second optimization operation. In some embodiments, the at least one first optimization operation and the at least one second optimization operation may be executed simultaneously. In some embodiments, the at least one first optimization operation and the at least one second optimization operation may be executed alternately.
As for a specific iteration (e.g., a current iteration) , the processing device 120 may determine whether a termination condition is satisfied in the current iteration. Exemplary termination conditions may include that a certain count of iterations has been performed, at least one candidate base material image generated in the current iteration has reached a desired image quality (e.g., a noise rate is less than a threshold) , the value of the cost function is lower than a threshold value, a difference between the values of the cost function in consecutive iterations is lower than a threshold difference, etc. The certain count, the threshold value, and the threshold difference may be default values of the imaging system 100, manually set by a user, determined by the processing device 120 according to an actual need, etc. In some embodiments, whether the termination condition is satisfied may be determined manually by a user. For example, the at least one candidate base material image may be displayed on an interface implemented on, for example, the terminal device 140, and the user may input an evaluation result regarding whether the at least one candidate base material image has reached desired image quality.
If it is determined that the termination condition is satisfied in the current iteration, the processing device 120 may determine that the cost function has been optimized, and designate the at least one candidate base material image generated in the current iteration as the at least one target base material image. If it is determined that the termination condition is not satisfied in the current iteration, the processing device 120 may determine that the cost function has not been optimized, and proceed to a next iteration until the termination condition is satisfied.
It should be noted that the above description regarding the process 500 is merely provided for the purposes of illustration, and not intended to limit the scope of the present disclosure. For persons having ordinary skills in the art, multiple variations and modifications may be made under the teachings of the present disclosure. However, those variations and modifications do not depart from the scope of the present disclosure. In some embodiments, the process 500 may be accomplished with one or more additional operations not described and/or without one or more of the operations discussed above. For example, the process 500 may include an additional transmitting operation in which the processing device 120 may transmit the reconstructed image to the terminal device 140. As another example, the process 500 may include an additional storing operation in which the processing device 120 may store information and/or data (e.g., the preliminary image, the at least one candidate base material image, the at least one target base material image) in a storage device (e.g., the storage device 130, the storage 220, the storage 390) .
In some embodiments, a process similar to the process 500 may be performed to generate at least one target single energy bin image. A single energy bin image refers to an image corresponding to a specific energy bin. For example, in 520, a second cost function including a second data fidelity item and a second regularization item may be determined. The second data fidelity item may correlate at least one candidate single energy bin image with the raw data, and the regularization item may be configured to reduce noise in the at least one candidate single bin image and being determined through a deep learning model. In 530, the at least one target single energy bin image may be generated by iteratively optimizing the second cost function. The optimization of the second cost function may be performed in a similar manner to that of the cost function as described above.
FIG. 6A is a flowchart illustrating an exemplary process of an iteration in the optimization process according to some embodiments of the present disclosure. In some embodiments, one or more operations of the process 600A may be performed to achieve at least part of operation 530 as described in connection with FIG. 5.
As described in connection with FIG. 5, a cost function including a data fidelity item and a regularization item may be optimized to generate at least one target base material image, and the optimization process of the cost function may include a plurality of iterations. If the regularization item implemented using the deep learning network is optimized together with the data fidelity item, e.g., in a single equation, the minimization (e.g., iterative optimization) of the equation may be difficult. Besides, a training process of the deep learning network involves operations such as forward projection and gradient transfer, which may need a large amount of computation. Therefore, the data fidelity item and the regularization item may be optimized separately.
In some embodiments, at least one iteration (e.g., each iteration of the iterations) may be performed by the process 600A. For illustration purposes, the implementation of a current iteration is described hereinafter. As shown in FIG. 6A, the current iteration may include a first optimization operation 610 for optimizing the data fidelity item and at least one second optimization operation 620 for optimizing the regularization item. The first optimization operation 610 and the at least one second optimization operation 620 may be independent optimization operations so as to reduce the computation load of the processing device 120 and improve the efficiency of the iterative optimization. The first optimization operation 610 and the at least one second optimization operation 620 may be performed simultaneously or alternately. The first optimization operation 610 may include operation 611 and operation 612. The at least one second optimization operation 620 may include operation 621 and operation 622. Since there is no regularization involved in the first optimization operation 610, the first optimization operation 610 may also be referred to as a non-regularization iteration.
In 611, the processing device 120 (e.g., the processor 210, the obtaining module 410) may obtain at least one candidate base material image to be processed in a current iteration. For example, for a first iteration, the at least one candidate base material image to be processed may be at least one preliminary base material image corresponding to the raw data (e.g., reconstructed based on the raw data) ; for a subsequent iteration, the at least one candidate base material image to be processed may be at least one candidate base material image determined in a previously adjacent iteration.
In 612, the processing device 120 (e.g., the processor 210, the optimizing module 430) may determine at least one updated base material image by optimizing the data fidelity item based on the candidate base material image to be processed.
In some embodiments, the processing device 120 may determine the at least one updated base material image by optimizing the data fidelity item according to Equation (3) :
where U _m, k denotes the at least one updated base material image U _m in a k ^th iteration (i.e., the current iteration) .
In some embodiments, the processing device 120 may determine the at least one updated base material image by solving Equation (3) , and the at least one candidate base material image to be processed in the current iteration may be designated as the initial value (s) of U _m for solving Equation (3) . By solving Equation (3) , a difference between beam intensity data corresponding to the at least one updated base material image and the beam intensity data is minimized. In some embodiments, the processing device 120 may solve Equation (3) by using Newton’s algorithm or a gradient descent algorithm. Since the regularization item is not incorporated into Equation (3) , the beam intensity data Y _n may introduce both information regarding the subject scanned by the MECT device and noise information into the at least one updated base material image.
In 621, the processing device 120 (e.g., the processor 210, the optimizing module 430) may determine at least one optimized base material image by optimizing the regularization item based on the at least one updated base material image.
After the preliminary optimization in the current iteration is performed, the processing device 120 may perform a further optimization to reduce (or eliminate) the noise information in the at least one updated base material image to generate the at least one optimized base material image with better image quality. The noise information used herein may refer to information related to a noise feature in an image, for example, a noise distribution, a noise intensity, a global noise intensity, a noise rate, etc. The noise intensity refers to a value of a noise pixel that reflects an amplitude of the noise in the noise pixel. The noise distribution may reflect the probability densities of noises with different noise intensities in the image. The global noise intensity may reflect an average noise intensity or a weighted average noise intensity in the image. The noise rate may reflect a dispersion degree of the noise distribution. In some embodiments, the processing device 120 may determine the noise feature based on a statistical noise model and/or a probability density function (PDF) corresponding to the statistical noise model. For example, the processing device 120 may determine a representation (e.g., a curve, a value, a vector, a matrix) of the noise distribution according to the statistical noise model and/or the PDF. Exemplary statistical noise models may include a Gaussian noise model, an impulse noise model, a Rayleigh noise model, an exponential distribution noise model, a uniform distribution noise model, or the like, or any combination thereof.
In some embodiments, the at least one optimized base material image may be determined by optimizing a regularization item according to Equation (4) :
where V _m, k denotes the at least one optimized base material image V _m in the k ^th iteration, N denotes a denoising function (e.g., the deep learning network as set forth above) , and θ denotes a parameter set of the denoising function. In some embodiments, Equation (4) may be implemented via the deep learning network. The at least one optimized base material image may be determined by inputting the at least one updated base material image determined in 621 into the deep learning network. In some embodiments, the deep learning network may be pre-trained and stored in a storage device (e.g., the storage device 130) . The processing device 120 may retrieve the deep learning network from the storage device. In some embodiments, the deep learning network may be trained based on a plurality of training samples. More descriptions regarding the training of the deep learning network may be found elsewhere in the present disclosure (e.g., FIG. 7 and the description thereof) . In some embodiments, the at least one optimized base material image V _m in the k ^th iteration may also be referred to as at least one denoised base material image in the k ^th iteration.
In some embodiments, the processing device 120 may execute a plurality of noise information reduction operations (i.e., perform operation 621 multiple times) on the at least one updated base material image to determine the at least one optimized base material image. The plurality of noise information reduction operations may be executed based on the deep learning network. In some embodiments, the deep learning network may include a plurality of sub-networks (also referred to as deep learning sub-networks) . The sub-networks may be networks of different types or the same type. In some embodiments, each of a plurality of base materials may correspond to a sub-network of the deep learning network. In some embodiments, the sub-networks may be networks of the same type with the same structure or different structures. For example, the sub-networks may be deep neural networks with different numbers (or counts) of convolutional layers and/or different numbers (or counts) of neurons. As another example, the sub-networks may be deep neural networks with different activation modes and/or different structures. In some embodiments, each of the plurality of sub-networks may correspond to a base material. At least one update base material image regarding a specific base material may be input into a corresponding sub-network for reducing or eliminating noise in the at least one update base material image, and at least one optimized base material image regarding the specific base material may be output by the sub-network.
In some embodiments, the processing device 120 may execute the plurality of noise information reduction operations on the at least one updated base material image sequentially. For example, the processing device 120 may designate at least one result base material image (also referred to as result image) obtained in the current noise information reduction operation as the at least one updated base material image in a next noise information reduction operation. Further, the processing device 120 may designate at least one result base material image obtained in the last noise information reduction operation as the at least one optimized base material image.
In some embodiments, the processing device 120 may execute the plurality of noise information reduction operations on the at least one updated base material image in parallel. For example, the processing device 120 may determine a plurality of result base material images in the plurality of noise information reduction operations. Further, the processing device 120 may determine an average result or a weighted average result of the plurality of result base material images as the at least one optimized base material image.
In 622, the processing device 120 (e.g., the processor 210, the optimizing module 430) may designate the at least one optimized base material image as at least one candidate base material image to be processed in a next iteration.
In some embodiments, the processing device 120 may determine whether a termination condition is satisfied after the at least one optimized base material image is determined. If it is determined that the termination condition is satisfied, operation 622 may be omitted, and the processing device 120 may designate the at least one optimized base material image as the at least one target base material image. If it is determined that the termination condition is not satisfied, operation 622 may be performed, and the processing device 120 may execute the first optimization operation 610 again in the next iteration.
FIG. 6B is a schematic diagram illustrating an exemplary process 600B for generating at least one target base material image according to some embodiments of the present disclosure.
As shown in FIG. 6B, raw data may be obtained. The raw data may include data in the intensity domain or data in the log domain. More descriptions regarding the raw data may be found elsewhere in the present disclosure. See, e.g., operation 510 and relevant descriptions thereof.
At least one seed image (e.g., preliminary base material image (s) ) may be generated by performing filtered back-projection on the raw data. A plurality of iterations may be performed based on the at least one seed image to generate the at least one target base material image. In the first iteration, the at least one seed image may be used as at least one candidate base material image to be processed.
In each iteration, optimizations of a first sub-problem and optimizations of a second sub-problem may be performed. The first sub-problem may relate to the data fidelity item, and the optimizations of the first sub-problem may also be referred as to the first optimization operation. The second sub-problem may relate to the regularization item, and the optimizations of the second sub-problem may also be referred to as the second optimization operation. More descriptions regarding the first optimization operation and the second optimization operation may be found elsewhere in the present disclosure. See, e.g., FIG. 6A and relevant descriptions thereof. The iterations may be terminated if N iterations have been performed. N may be equal to any integer greater than 1, such as 5, 10, etc.
In some embodiments, acquisition parameters of the raw data (e.g., scanning parameters of the multi-energy CT device) may be obtained, and raw statistical model estimation may be performed on the acquisition parameters. Merely by way of example, the multi-energy CT device may include a plurality of detector units, and the raw data may include multiple subsets collected by the detector units. The raw statistical model estimation may be performed on the acquisition parameters to determine a noise level of each subset of the raw data. Then, in the iterations for generating the at least one target base material image, different weights may be assigned to the subsets of the raw data based on their corresponding noise levels. For example, the higher the noise level of a subset, the lower weight the subset has. Merely by way of example, Y _n in Equation (1) provided above may be modified into a weighted sum of the subsets collected by different detector units. By taking the acquisition parameters and the noise levels of different detector units into consideration, the resulting target base material image (s) may have an improved accuracy.
FIG. 7 is a flowchart illustrating an exemplary process for determining a deep learning network according to some embodiment of the present disclosure. In some embodiments, the deep learning network as described in connection with operation 621 may be generated by performing process 700 of FIG. 7. In some embodiments, the process 700 may be performed by the processing device 120 or another processing device (e.g., a processing device of a vendor of the deep learning network) . For illustration purposes, the implementation of the process 700 by the processing device 120 is described below.
In 710, the processing device 120 (e.g., the processor 210, the obtaining module 410) may obtain a plurality of training samples. In some embodiments, at least part of the plurality of training samples may be previously generated and stored in a storage device (e.g., the storage device 130, the storage 220, the storage 390, or an external database) . The processing device 120 may retrieve the training samples directly from the storage device.
In some embodiments, a training sample may include a training input and a training target. In some embodiments, the training input of the training sample may include material decomposition images of multiple base materials. In some embodiments, the material decomposition images may include high noise images with relatively high noise intensities and low noise images with relatively low noise intensities. The high noise images may be generated by, for example, performing a low dose scan, adding noise in material decomposition images, etc. The low noise images may be generated by, for example, performing a high dose scan, reducing noise in material decomposition images, etc. As used herein, a low dose scan refers to a scan radiating x-rays at a relatively low dose (e.g., lower than a first threshold dose) . A high dose scan refers to a scan radiating x-rays at a relatively high dose (e.g., higher than a second threshold dose) .
In some embodiments, the training sample may include a training target 801 and a training input 802 as shown in FIG. 8. In some embodiments, the training target 801 may include first base material images {U _m} _C generated by iterative image reconstruction (e.g., one or more non-regularization iterations) on high dose data acquired by a high dose scan. The first base material images {U _m} _C may be images generated after the iterative image reconstruction converges. As aforementioned, m refers to a serial number of a base material among M base materials. The first base material images {U _m} _C may include a first base material image corresponding to each of the M base materials.
In some embodiments, the training input 802 may be generated by combining the training target 801 and noise images {U _m} _Noise. As used herein, a noise image refers to an image that contains noise and does not have any effective signals. In some embodiments, each of the noise images may correspond to one base material and represent a difference between a first base material image corresponding to the base material and a second base material image corresponding to the base material. In some embodiments, the processing device 120 may determine low dose data by performing a low dose simulation on the high dose data. Second base material images {U _m} _low of the M base materials may be reconstructed by performing one or more non-regularization iterations on the low dose data. In this way, training inputs corresponding to various doses may be generated randomly, thereby improving a robustness of the deep learning network.
Merely for illustration, the training input 802 may be determined according to Equations (5) and (6):
{U _m} _Input= {U _m} _Gold+α× {U _m} _Noise, (5)
{U _m} _Noise= {U _m} _Low- {U _m} _High, (6)
where {U _m} _Input denotes the training input, {U _m} _Gold denotes the training target 801, {U _m} _Noise denotes the noise images, and α denotes a noise amplification coefficient. The parameter α may be determined by a user, or determined according to default settings of the imaging system 100, or randomly set, etc. The parameter α may be the same or different for different training samples.
In some embodiments, at least a portion of the plurality of training samples may be generated by the processing device 120. Merely by way of example, the processing device 120 may obtain at least one qualified image (e.g., an image with its quality satisfying a quality condition) , and generate a plurality of training inputs based on the at least one qualified image. For example, the processing device 120 may generate the plurality of training inputs by preprocessing (e.g., performing a noise addition, an artifact addition on) the at least one qualified image.
In some embodiments, the training input of a training sample may include sample base material images (e.g., determined based on Equations (5) and (6) above) of different base materials. The sample base material images of different base materials may correspond to the same anatomical structure and have certain correlations, and such correlations may be used to train the deep learning network with an improved accuracy.
In 720, the processing device 120 (e.g., the processor 210, the training module 440) may determine the deep learning network by training a preliminary deep learning network based on the plurality of training samples.
In some embodiments, the preliminary deep learning network may be deep neural network (DNN) , a convolutional neural network (CNN) , a recurrent neural network (RNN) , a feature pyramid network (FPN) , etc. In some embodiments, the preliminary deep learning network may include at least one preliminary network parameter. The at least one preliminary network parameter may be set by a user, according to default settings of the imaging system 100, etc. In some embodiments, the at least one preliminary network parameter may be adjustable under different situations. Taking a CNN as an example, the at least one preliminary network parameter may include a count of convolutional layers, a count of kernels, a kernel size, a stride, a padding of each convolutional layer, or the like, or any combination thereof.
In some embodiments, the preliminary deep learning network may include one or more preliminary sub-networks. Each of the one or more preliminary sub-networks may correspond to a base material. Material decomposition images of a specific base material may be used to train a preliminary sub-network corresponding to the specific base material. For example, one or more first material decomposition images (e.g., the first base material images, the second base material image) of a first base material may be used to train a first preliminary sub-network. One or more second material decomposition images of a second base material may be used to train a second preliminary sub-network. In some embodiments, different preliminary sub-networks may be trained jointly or separately.
In some embodiments, the processing device 120 may train the preliminary deep learning network based on one or more gradient descent algorithms. Exemplary gradient descent algorithms may include an Adam optimization algorithm, a stochastic gradient descent (SGD) + Momentum optimization algorithm, a Nesterov accelerated gradient (NAG) algorithm, an Adaptive Gradient (Adagrad) algorithm, an Adaptive Delta (Adadelta) algorithm, a Root Mean Square Propagation (RMSprop) algorithm, an AdaMax algorithm, a Nadam (Nesterov-accelerated Adaptive Moment Estimation) algorithm, an AMSGrad (Adam+SGD) algorithm, or the like, or any combination thereof.
In some embodiments, the processing device 120 may train the preliminary deep learning network iteratively based on the training samples until a termination condition is satisfied. In response to that the termination condition is satisfied, the deep learning network may be finalized. In some embodiments, the termination condition may relate to a value of an objective function. For example, in the current iteration, the training input 802 of each training sample may be input into an intermediate preliminary model to be trained in the current iteration. The intermediate preliminary model may be the preliminary model if the current iteration is the first iteration, or an updated preliminary model if the current iteration is an iteration other than the first iteration. The intermediate preliminary model may output a predicted base material image of each training sample. The objective function may measure a difference between the predicted base material image and the training target 801 (e.g., the first base material image) of each training sample.
Merely by way of example, the termination condition may be satisfied if the value of the objective function is minimal or smaller than a predetermined threshold. As another example, the termination condition may be satisfied if the value of the objective function reaches a convergence. In some embodiments, the “convergence” refers that the variation of the values of the objective function in two or more consecutive iterations is equal to or smaller than a predetermined threshold. In some embodiments, the “convergence” refers that a difference between the value of the objective function and a target value is equal to or smaller than a predetermined threshold. In some embodiments, the termination condition may be satisfied when a specified count of iterations have been performed in the training process of the preliminary model.
It should be noted that the above description regarding the process 700 is merely provided for the purposes of illustration, and not intended to limit the scope of the present disclosure. For persons having ordinary skills in the art, multiple variations and modifications may be made under the teachings of the present disclosure. However, those variations and modifications do not depart from the scope of the present disclosure. In some embodiments, one or more operations may be added or omitted. For example, the processing device 120 may update the deep learning network periodically or irregularly based on one or more newly-generated training samples. As another example, the processing device 120 may divide the plurality of training samples into a training set and a test set. The training set may be used to train the deep learning network, and the test set may be used to determine whether the training process is complete.
In some embodiments, if the process 700 is used to generate a deep learning network relating to single energy bin images, the training target may include first single energy bin images generated by iterative image reconstruction on high dose data acquired by high dose scans, and the training input may be determined by combining the first single energy bin images and noise images. The noise images may be determined based on the first single energy bin images and second single energy bin images reconstructed based on low dose data. The first single energy bin images and the first base material images are collectively referred to as first images, and the second single energy bin images and the second base material images are collectively referred to as second images.
Having thus described the basic concepts, it may be rather apparent to those skilled in the art after reading this detailed disclosure that the foregoing detailed disclosure is intended to be presented by way of example only and is not limiting. Various alterations, improvements, and modifications may occur and are intended to those skilled in the art, though not expressly stated herein. These alterations, improvements, and modifications are intended to be suggested by this disclosure, and are within the spirit and scope of the exemplary embodiments of this disclosure.
Moreover, certain terminology has been used to describe embodiments of the present disclosure. For example, the terms “one embodiment, ” “an embodiment, ” and/or “some embodiments” mean that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Therefore, it is emphasized and should be appreciated that two or more references to “an embodiment” or “one embodiment” or “an alternative embodiment” in various portions of this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures or characteristics may be combined as suitable in one or more embodiments of the present disclosure.
Further, it will be appreciated by one skilled in the art, aspects of the present disclosure may be illustrated and described herein in any of a number of patentable classes or context including any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof. Accordingly, aspects of the present disclosure may be implemented entirely hardware, entirely software (including firmware, resident software, micro-code, etc. ) or combining software and hardware implementation that may all generally be referred to herein as a “unit, ” “module, ” or “system. ” Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer readable media having computer readable program code embodied thereon.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including electro-magnetic, optical, or the like, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that may communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable signal medium may be transmitted using any appropriate medium, including wireless, wireline, optical fiber cable, RF, or the like, or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB. NET, Python or the like, conventional procedural programming languages, such as the “C” programming language, Visual Basic, Fortran 2103, Perl, COBOL 2102, PHP, ABAP, dynamic programming languages such as Python, Ruby and Groovy, or other programming languages. The program code may execute entirely on the user’s computer, partly on the user’s computer, as a stand-alone software package, partly on the user’s computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user’s computer through any type of network, including a local area network (LAN) or a wide area network (WAN) , or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider) or in a cloud computing environment or offered as a service such as a Software as a Service (SaaS) .
Furthermore, the recited order of processing elements or sequences, or the use of numbers, letters, or other designations therefore, is not intended to limit the claimed processes and methods to any order except as may be specified in the claims. Although the above disclosure discusses through various examples what is currently considered to be a variety of useful embodiments of the disclosure, it is to be understood that such detail is solely for that purpose, and that the appended claims are not limited to the disclosed embodiments, but, on the contrary, are intended to cover modifications and equivalent arrangements that are within the spirit and scope of the disclosed embodiments. For example, although the implementation of various components described above may be embodied in a hardware device, it may also be implemented as a software only solution, e.g., an installation on an existing server or mobile device.
Similarly, it should be appreciated that in the foregoing description of embodiments of the present disclosure, various features are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure aiding in the understanding of one or more of the various inventive embodiments. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed subject matter requires more features than are expressly recited in each claim. Rather, inventive embodiments lie in less than all features of a single foregoing disclosed embodiment.
In some embodiments, the numbers expressing quantities or properties used to describe and claim certain embodiments of the application are to be understood as being modified in some instances by the term “about, ” “approximate, ” or “substantially. ” For example, “about, ” “approximate, ” or “substantially” may indicate ±20%variation of the value it describes, unless otherwise stated. Accordingly, in some embodiments, the numerical parameters set forth in the written description and attached claims are approximations that may vary depending upon the desired properties sought to be obtained by a particular embodiment. In some embodiments, the numerical parameters should be construed in light of the number of reported significant digits and by applying ordinary rounding techniques. Notwithstanding that the numerical ranges and parameters setting forth the broad scope of some embodiments of the application are approximations, the numerical values set forth in the specific examples are reported as precisely as practicable.
Each of the patents, patent applications, publications of patent applications, and other material, such as articles, books, specifications, publications, documents, things, and/or the like, referenced herein is hereby incorporated herein by this reference in its entirety for all purposes, excepting any prosecution file history associated with same, any of same that is inconsistent with or in conflict with the present document, or any of same that may have a limiting affect as to the broadest scope of the claims now or later associated with the present document. By way of example, should there be any inconsistency or conflict between the description, definition, and/or the use of a term associated with any of the incorporated material and that associated with the present document, the description, definition, and/or the use of the term in the present document shall prevail.
In closing, it is to be understood that the embodiments of the application disclosed herein are illustrative of the principles of the embodiments of the application. Other modifications that may be employed may be within the scope of the application. Thus, by way of example, but not of limitation, alternative configurations of the embodiments of the application may be utilized in accordance with the teachings herein. Accordingly, embodiments of the present application are not limited to that precisely as shown and described.

Claims

A method implemented on a computing device having a processor and a computer-readable storage device, the method comprising:

obtaining raw data acquired using a multi-energy computed tomography (CT) device;

determining, based on the raw data, a cost function including a data fidelity item and a regularization item, the data fidelity item correlating at least one candidate image with the raw data, and the regularization item being configured to reduce noise in the at least one candidate image and being determined through a deep learning model; and

generating at least one target image by iteratively optimizing the cost function.
The method of claim 1, wherein:

the at least one candidate image includes at least one candidate base material image, and the at least one target image includes at least one target base material image; or

the at least one candidate image includes at least one candidate single energy bin image, and the at least one target image includes at least one target single energy bin image.
The method of claim 1, wherein the raw data includes beam intensity data.
The method of claim 3, wherein the data fidelity item relates to at least one of spectrum information of X-rays generated by the multi-energy CT device or response information of detectors of the multi-energy CT device.
The method of claim 1, wherein the raw data includes raw data of base materials, the raw data being determined by performing a base material decomposition in a data domain based on beam intensity data generated based on the multi-energy CT device and spectrum information of X-rays generated by the multi-energy CT device.
The system of claim 1, wherein the raw data includes raw data of base materials, the raw data being determined by performing a base material decomposition in a data domain based on beam intensity data generated based on the multi-energy CT device and response information of detectors of the multi-energy CT device.
The method of claim 1, wherein the raw data is generated by performing a data correction on original scan data generated by the multi-energy CT device.
The method of claim 1, wherein the iteratively optimizing the cost function includes:

iteratively optimizing the data fidelity item and the regularization item simultaneously or alternately.
The method of claim 1, wherein the regularization item includes a deep learning network.
The method of claim 9, wherein the deep learning network is trained offline.
The method of claim 9, wherein the deep learning network is trained using training inputs that include material decomposition images of one or more base materials.
The method of claim 11, wherein each of the one or more base materials corresponds to a deep learning sub-network of the deep learning network.
The method of claim 9, wherein the deep learning network is trained using a plurality of training samples, each training sample includes a training target that includes first images generated by iterative image reconstruction on high dose data acquired by high dose scans.
The method of claim 13, wherein each training sample further include a training input that is generated by combining the training target of the training sample and noise images,

each of the noise images represents a difference between one of the first images of the training sample and a second image, the second image being generated by:

determining low dose data by performing a low dose simulation on the high dose data, and

reconstructing the second image by performing one or more non-regularization iterations on the low dose data.
A system, comprising:

at least one storage device storing a set of instructions; and

at least one processor configured to communicate with the at least one storage device, wherein when executing the set of instructions, the at least one processor is configured to direct the system to perform operations including:

obtaining raw data acquired using a multi-energy computed tomography (CT) device;

determining, based on the raw data, a cost function including a data fidelity item and a regularization item, the data fidelity item correlating at least one candidate image with the raw data, and the regularization item being configured to reduce noise in the at least one candidate image and being determined through a deep learning model; and

generating at least one target image by iteratively optimizing the cost function.
The system of claim 15, wherein:

the at least one candidate image includes at least one candidate base material image, and the at least one target image includes at least one target base material image; or

the at least one candidate image includes at least one candidate single energy bin image, and the at least one target image includes at least one target single energy bin image.
The system of claim 15, wherein the raw data includes raw data of base materials, the raw data being determined by performing a base material decomposition in a data domain based on beam intensity data generated based on the multi-energy CT device and spectrum information of X-rays generated by the multi-energy CT device.
The system of claim 15, wherein the raw data includes raw data of base materials, the raw data being determined by performing a base material decomposition in a data domain based on beam intensity data generated based on the multi-energy CT device and response information of detectors of the multi-energy CT device.
The system of claim 15, wherein the regularization item includes a deep learning network.
A non-transitory computer readable medium, comprising a set of instructions, wherein when executed by at least one processor, the set of instructions direct the at least one processor to effectuate a method, the method comprising:

obtaining raw data acquired using a multi-energy computed tomography (CT) device;

determining, based on the raw data, a cost function including a data fidelity item and a regularization item, the data fidelity item correlating at least one candidate image with the raw data, and the regularization item being configured to reduce noise in the at least one candidate image and being determined through a deep learning model; and

generating at least one target image by iteratively optimizing the cost function.