EP3703004A2 - Image quality improvement methods for optical coherence tomography - Google Patents
Image quality improvement methods for optical coherence tomography Download PDFInfo
- Publication number
- EP3703004A2 EP3703004A2 EP20159835.6A EP20159835A EP3703004A2 EP 3703004 A2 EP3703004 A2 EP 3703004A2 EP 20159835 A EP20159835 A EP 20159835A EP 3703004 A2 EP3703004 A2 EP 3703004A2
- Authority
- EP
- European Patent Office
- Prior art keywords
- noise
- image
- input image
- reduced
- images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000012014 optical coherence tomography Methods 0.000 title claims description 26
- 238000003384 imaging method Methods 0.000 claims abstract description 29
- 238000009826 distribution Methods 0.000 claims description 31
- 238000001914 filtration Methods 0.000 claims description 16
- 238000010801 machine learning Methods 0.000 claims description 15
- 238000012935 Averaging Methods 0.000 claims description 11
- 238000002583 angiography Methods 0.000 claims description 9
- 210000001525 retina Anatomy 0.000 claims description 7
- 238000003672 processing method Methods 0.000 claims description 3
- 230000001427 coherent effect Effects 0.000 abstract description 9
- 230000009467 reduction Effects 0.000 description 32
- 238000013135 deep learning Methods 0.000 description 15
- 230000002207 retinal effect Effects 0.000 description 12
- 230000011218 segmentation Effects 0.000 description 11
- 210000001519 tissue Anatomy 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000017531 blood circulation Effects 0.000 description 3
- 210000004969 inflammatory cell Anatomy 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 210000004027 cell Anatomy 0.000 description 2
- 210000003583 retinal pigment epithelium Anatomy 0.000 description 2
- 238000012800 visualization Methods 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 210000001775 bruch membrane Anatomy 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 210000003743 erythrocyte Anatomy 0.000 description 1
- 210000004379 membrane Anatomy 0.000 description 1
- 239000012528 membrane Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011946 reduction process Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 238000013179 statistical model Methods 0.000 description 1
- 210000003462 vein Anatomy 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7203—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/60—Image enhancement or restoration using machine learning, e.g. neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
- G06T5/75—Unsharp masking
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0062—Arrangements for scanning
- A61B5/0066—Optical coherence imaging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10101—Optical tomography; Optical coherence tomography [OCT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20076—Probabilistic image processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20216—Image averaging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30041—Eye; Retina; Ophthalmic
Definitions
- Coherent imaging modalities are subject to noise, which can degrade final image quality.
- OCT optical coherence tomography
- the presence of noise can significantly degrade imaging fidelity and can lead to errors during segmentation of retinal layers (e.g., during segmentation of B-scans or like images including a depth dimension).
- segmentation errors can further lead to errors in analysis of the images based on the segmentation, and thus ultimately, errors in diagnosis and treatment of an imaged subject.
- noise is treated by applying a common filter to an underlying set of data or images produced from that data.
- these filters are unable to remove all noise at appropriate levels.
- noise reduction systems, and methods that utilize a large number of sample data and images to normalize noise are still subject to noises and errors that persist throughout the sample sets. Therefore, the images can still lack quality, and segmentation and analysis can still be prone to errors.
- an image processing method comprises: applying a first filter to an input image, thereby generating a first noise-reduced image; applying a second filter to the input image, thereby generating a second noise-reduced image; and combining the first noise-reduced image and the second noise-reduced image, thereby generating a final noise-reduced image, wherein the first filter is configured to suppress a first type of noise from the input image and the second filter is configured to suppress a second type of noise from the input image, the first and second types of noise being different.
- the first noise-reduced image and the second noise-reduced images are combined by weighted averaging, the first noise-reduced image being weighted according to a level of the first type of noise in the input image and the second noise-reduced image being weighted according to a level of the second type of noise in the input image; the first noise-reduced image and the second noise-reduced images are combined, or the first and/or second noise-reduced images are produced, by a machine learning system; an intensity of at least one pixel of the first noise-reduced image is set by the first filter to correspond to a maximum intensity of a probability distribution of intensities of pixels at a corresponding location of an object in the image; the probability distribution is determined by a machine learning system; the first type of noise is random noise or noise caused by an imaging system that captured the input image; the first filter is a machine learning system trained with images taken from the same or substantially the same location as the input image; the first type of noise is speckle
- an image processing method comprises: filtering a first input image; filtering a second input image; and combining the filtered first input image and the filtered second input image, thereby generating a final noise-reduced volume, wherein the first input image and the second input image are different 2D images of a 3D volume, and wherein the first input image and the second input image are from different planes and/or en face images of different reference layers of the 3D volume.
- filtering the first input image comprises: applying a first filter to the first input image, thereby generating a first noise-reduced image; applying a second filter to the first input image, thereby generating a second noise-reduced image; and combining the first noise-reduced image and the second noise-reduced image, thereby generating a final noise-reduced image, wherein the first filter is configured to suppress a first type of noise from the first input image and the second filter is configured to suppress a second type of noise from the first input image, the first and second types of noise being different.
- the present disclosure is directed to improvements in noise reduction for coherent imaging modalities. It is now recognized that more than one type of noise exists in images from coherent imaging modalities, and the types and levels of the noise can vary between pixels of an image. These types of noise include: 1) random noise from the system; and 2) speckle variation (noise) caused by the coherent imaging modalities or objects being imaged (e.g., a subject's eye or other biological tissue).
- speckle noise may arise from interference of light waves having random phases, for example, as light scattered from various points of turbid object being imaged.
- existing noise reductions methods and systems exist they are directed to treating the different types of noise with the same filter. And other machine learning based methods improperly rely on a substantial number of averaged images that can induce errors in the filtering process.
- Fig. 1 illustrates an image showing the absolute difference between each pixel of two OCT B-scan images generated from scans of essentially the same location of a subject's eye at different times.
- the reference to OCT images is merely an example and that the systems and methods described by the present application are applicable to images generated by any coherent imaging technique. Because the general structure at the location is unchanged, the difference between the two images represents noise, where the brightness of each pixel corresponds to the level of noise.
- the first region 100 is an area above the retina, which includes random noise from the imaging system.
- the second region 102 in the retinal tissue includes speckle variation due to blood flow as well as the random noise in the system.
- the third region 104 in the retinal tissue similarly includes other speckle variation due to live biological tissues (such as blood flow) as well as the random noise in the system.
- a plurality of noise reduction filters 202 (1 to N) are applied to a noisy input image 200.
- Each noise reduction filter 202 produces a corresponding noise reduced image 204 (1 to N).
- the filters 202 may be applied in parallel such that each noise reduced image 204 only has the noise reduced that corresponds to the applied filter 202, and thus may still include the other types of noise not reduced by the corresponding filter 202.
- actual processing may occur sequentially in time while retaining the parallel nature of the application.
- the plurality of noise reduced images 204 can then combined 206 to produce a final noise reduced image 208.
- the combination of noise reduced images results in the final noise reduced image 208 having each type of noise reduced.
- the filters 212-1 to 212-N may be applied sequentially such that a first type of noise is removed by a first filter 212-1 to produce a first noise-reduced image 214-1 in which the first type of noise is removed but other types of noise remain.
- a second type of filter 212-2 may then be applied to the first noise-reduced image 214-1 to further remove a second type of noise, while still other types of noise remain. This process can be repeated with additional noise reduction filters 212-N until all desired noise is removed and a final noise-reduced image 216 produced.
- Each filter may be designed and applied based on the probability distribution for the relative intensity for the pixels in an image without a given type of noise.
- the intensity of each pixel in an image, when subject to random noise can be modeled as shown in Fig. 3 .
- the "true value" (represented by the dashed vertical line) corresponds to the actual intensity of the pixel without any noise.
- the 'true value' may be determined as the intensity where probability is a maximum.
- Each pixel may have its own unique probability distribution.
- the noise type and level
- the intensity probability distribution 408 for a pixel 404 above the retina (having only random system noise) in a B-scan image 400 is skewed toward zero relative to the intensity probability distribution 410 for a pixel 406 in a retinal layer (having random system noise and speckle noise) of the image 400.
- the 'true value' intensity at the maximum of the probability distribution is shown in a corresponding noise-reduced B-scan image 402 where the intensity of the pixel 404 above the retina is much lower (darker) than that of the retinal pixel 406.
- probability distributions may be different for each type of noise.
- the intensity probability distribution for pixels without many types of noise may be Gaussian; however, some removal of other noises may cause the intensity probability distributions to follow different statistical models.
- removal of each type of noise may result in probability distributions having a different mean, sigma value, or like statistical property.
- each filter may set each pixel of an image to have its true intensity value (e.g., the intensity corresponding to the maximum probability).
- Artificial intelligence systems such as deep learning (and other machine learning) models/systems can be used to determine the intensity probability distributions for each pixel (e.g., each location of the retina) to more accurately estimate the most probable intensity value of each pixel.
- the deep learning systems of each filter are particularly designed and trained to estimate the intensity probability distribution of each pixel.
- the design and training of the deep learning systems are based on an understanding of the fundamental physics in OCT imaging (or the other coherent imaging methods) so that the systems can be trained with images demonstrating the correct intensity probability distributions.
- the deep learning systems can be trained with data sets having two or more images.
- the pixel intensity at the same corresponding location among different images is used to construct the intensity probability distribution of that pixel.
- a perfectly large set will include at least one image having every possible intensity for a given pixel location, and the relative number of images in the set having a given intensity represents the corresponding probability of that intensity.
- the relative number of images for every intensity value will produce the intensity probability distribution.
- the same process can be used to generate a probability distribution by fitting the relative number of images of each intensity to a known type of distribution curve.
- the approach alone does not necessarily suppress the different types of noise separately.
- differences between the images of a set can be compared, and adjusted based on the comparison, prior to developing the probability distribution.
- the effect of random noise can be derived by considering differences between images of a set of B-scan images taken at the same cross-sectional location through the retina. This is because the difference between images of the set will primarily represent random noise, since the underlying structure at the same cross-sectional location generally remains unchanged. Adjusting each image based on these differences has the effect of removing the random noise from the adjusted images.
- a probability distribution derived based on the adjusted images is a probability distribution for the image where random noise is suppressed but speckle is largely preserved.
- the difference may be determined according to any statistical method, such as by simple subtraction, an average of the differences between each successive scan, or the like; and the adjustments may be made in any statistical manner, such as by subtracting the average difference from each image.
- the effect of speckle noise can be found from sets of images of B-scans taken from adjacent or nearby locations within a transverse resolution of the imaging system.
- the differences between the images of such sets will primarily represent speckle noise.
- the images may be adjusted according to any statistical determination of the difference, so that a resulting intensity probability distribution is one in which the pixel intensity at the maximum probability represents the most probable intensity without speckle noise (e.g., where both speckle noise and random noise are suppressed).
- a deep learning system trained to suppress random noise may correspond to noise reduction filter 1 (a sharp filter) 202-1, 212-1 and a deep learning system trained to suppress speckle noise may correspond to noise reduction filter 2 (a smooth filter) 202-2, 212-2.
- the original noisy image is input to the trained system (filter), and the output of the trained system is an image in which each pixel is set to the intensity corresponding to the maximum probability.
- a trained deep learning system may be one that has been trained to know the intensity probability distribution (or otherwise knows a 'true value' intensity) for each pixel without one type of noise.
- FIG. 5 Such a configuration where separately trained deep learning systems filter an original input B-scan image to produce a combined noise-reduced image is illustrated in Fig. 5 .
- an original input image e.g., an OCT B-scan
- the sharp filter 502 produces an image 506 with reduced random noise
- the smooth filter produces an image 508 with reduced speckle noise.
- These reduced noise images 506, 508 are then combined to produce a final noise-reduced image 510 with both random and speckle noises reduced.
- any type of filter capable of reducing a desired type of noise may be used.
- filters may include median filters, Gaussian filters, spectral domain filters, and the like.
- each filter can be combined according to any statistical combination method.
- each noise-reduced image can be combined through a weighted average technique where the weight of each image is adjusted to match the relative noise level of the corresponding noise type.
- each noise-reduced image can be weighted according to the amount of the corresponding noise in the original input image.
- the output weights are 0.2 for the sharp filter (meaning 20% of the noise is random) 502 and 0.8 for the smooth filter (meaning 80% of the noise is speckle noise) 508.
- combining the outputs of each filter can produce a final noise-reduced image comparable to an image produced by averaging 128 images taken at the same location (a traditional technique for suppressing noise).
- Fig. 6 shows such a comparison, where an original input image (without any noise reduction) 600, an image produced by averaging 128 images from substantially the same location (including the original input image) 602, and an a noise reduced image 604 resulting from the above-described combination of outputs of a smooth filter and a sharp filter.
- an original input image (without any noise reduction) 600 an image produced by averaging 128 images from substantially the same location (including the original input image) 602
- an a noise reduced image 604 resulting from the above-described combination of outputs of a smooth filter and a sharp filter.
- the noise-reduced output image 508 of the smooth filter 504 (removing speckle noise) has smoother retinal boundaries than image 506 output by the sharp filter 502. Therefore, the smoothed image 508 (rather than the output from the sharp filter, or the combined output image) may be best used in tasks such as visualization/display and layer segmentation.
- Fig. 7 illustrates this concept with respect to 1-pixel depth en-face images of the lamina cribrosa, where the primary noise component is speckle noise.
- the noise-reduced image 702 filtered only with a smooth filter to reduce speckle noise is smoother and more visually appealing than the original image 704.
- the speckle noise-reduced image 704 output from the smooth filter may be beneficial for improving visualization applications.
- Fig. 8 illustrates an original B-scan image 800 and a noise-reduced B-scan image 800 output by a sharp filter to remove random noise.
- an inflammatory cell(s) (circled) 804 is visible.
- the cell 804 is more identifiable in the random noise-reduced image 802 as it is against a clearer background.
- the deep learning systems for noise-reduction filtering, additional information such as the retinal layers, are implicitly learned. This results from the different statistical properties of noise associated with pixels of each retinal layer. As the deep learning system is trained to reduce the different noises (types and levels) in different retinal layers, it thus also learns which pixels are associated with each layer. Put another way, for the deep learning systems to remove proper noise types and levels from each retinal layer, it should know to which layer each pixel belongs. With such knowledge, layer boundaries can be identified, and segmentation realized, by determining the pixels at which the associated layers change.
- noise-reduction filters and segmentation detectors can be trained simultaneously as part of the same deep learning system.
- Such an output is illustrated in Fig. 9 , where an original image 900 is input into a trained deep learning system 902, which outputs a noise-reduced image 904 and a retinal segmentation 906 of the image.
- the segmentation 906 includes (from top to bottom) boundaries of the inner limiting membrane (ILM), retinal pigment epithelium (RPE), and Bruch's membrane layers.
- the input may be a single optical coherence tomography (OCT) (or like coherent imaging modality) B-scan, a 3D volumetric scan, an en face image or C-scan, an angiographic B-scan image, an angiography en face image or C-scan, or like images from other imaging modalities.
- OCT optical coherence tomography
- the noise reduction can be performed in different planes of 2D images that collectively form the 3D volumes, as shown in Fig. 10 .
- the noise (types and levels) may be different depending on the location in the volume (either at different depths, or different locations).
- different noise reduction filters may be applied to different portions of the volume. These filters, if trained machine learning systems, may be trained for different types of noise and/or the different effects of noise at different locations in the volume. Weights used to combine differently filtered images, or other combination techniques, may also be variable and change depending on where in the volume the images are located.
- volumetric data may be noise reduced according to the above-described methods.
- collections of B-scans in the X-Z plane 1000 as described above can be individually noise reduced to reduce noise of an entire volume.
- the data may be sampled (or re-sampled) in the Y-Z plane and noise reduction as described above applied on Y-Z B-scans 1002.
- the noise reduction process can be applied on X-Y plane C-scans or en face images flattened over any depth/thickness and/or with respect to any reference layer.
- a reference layer may be the separation between two retinal tissue layers, for example, the ILM.
- filtering can be applied to different planes of 2D images of a 3D volume or other collection of 2D images (e.g., from non-adjacent locations of a larger 3D volume).
- the application of filters to different planes can be in parallel and combined, or sequential.
- Fig. 11A illustrates a first example of sequential noise reduction of a 3D volume (e.g., an OCT-angiography (OCT-A) volume).
- OCT-A OCT-angiography
- a first step of noise reducing a 3D OCT-A volume 1100 involves noise reduction of OCT-angiography images in X-Z plane 1102 of the volume 1100, and a second step involves noise reduction of en face images 1104 of the volume 1100, to produce a final noise-reduced 3D volume 1106.
- Fig. 12 illustrates a comparison of an en face image 1200 of the deep plexiform layer taken from the original OCT-A volume 1100, and of a en face image 1202 of deep plexiform layer noise-reduced by the method of Fig. 11A .
- vessel connectivity is greatly enhanced after noise reduction.
- Figs. 11B and 11C illustrate similar sequential methods with other combinations of planes.
- X-Z plane images 1112 from an original 3D volume 1110 are first filtered, followed by Y-Z plane images 1114 and then X-Y plane images 1116, to produce a final noise-reduced volume 1118.
- X-Y plane images 1122 from an original 3D volume 1120 are first filtered, and then X-Z plane images 1124 are filtered to produce a final noise-reduced volume 1126.
- the different filterings may be directed to a type of noise and/or the plane in which it is applied.
- Each filtering step may be performed by a filter trained or otherwise designed to remove a different type of noise and/or to identify noise in the respective plane in which it is applied. Removing a different type of noise in each filter step may be beneficial where each type of noise in more apparent in a different plane, whereas removing the same type of noise in multiple steps may be beneficial where one type of noise expresses itself differently in different planes.
- the images of the plane in a subsequent filtering step may be generated from the original volume after a preceding noise reduction step. Accordingly, the subsequently generated images may already have the preceding type of noise removed.
- FIG. 11D An example of noise reduction for different planes in parallel is shown in Fig. 11D .
- noise reduction is separately performed in parallel on images of the X-Z plane 1132 and the Y-Z plane 1136 of an original 3D volume.
- performing noise reduction in parallel refers to performing the noise-reduction on images of a 3D volume not subject to the other noise-reduction application. Accordingly, the actual application of the noise-reduction may still be performed sequentially in time.
- the resulting noise-reduced volumes 1134, 1138 may be averaged (e.g., by weighted averaging or other combination techniques as described above) to produce a final noise-reduced volume 1142.
- different planes and combinations may be used in parallel in other embodiments.
- the present disclosure provides various improvements compared to traditional noise reduction techniques (such as averaging 128 images taken from a single location). For example, it can take a relatively long time to obtain 128 images (or any number of images being averaged), which limits the total imaging speed and makes the variously averaged images more susceptible to motion artifact. Additionally, it is practically difficult to take so many images at a single location; therefore, each image of the average may include different information. Put another way, an imaging target at that location may not present in every image, or other elements may be confused as the imaging target. Further, to the extent any other machine learning systems are trained with the above-described average images as supervisory training data, the inherent limitations of averaging are imputed into the trained machine learning system.
- the present noise reduction technique preserves the quantitative information available in the original images.
- the present noise reduction technique enables applications which were previously only achievable with high-speed devices (e.g. variable interscan time analysis (VISTA) systems), where the high-speed is used to take multiple repeated scans to suppress noise, possible on low-speed devices with fewer repeats.
- high-speed devices e.g. variable interscan time analysis (VISTA) systems
- VISTA variable interscan time analysis
- OCT-A angiographic imaging
- dynamic (non-static) structures e.g., movement of cells, such as red blood cells.
- the noise reduction described herein may be used in at or near real-time imaging applications.
- the input images may be obtained with high speed OCT imaging (e.g., operating at a 400 kHz A-line scanning rate), such that the filtering according to the present disclosure provides high fidelity images at that high speed.
- OCT imaging e.g., operating at a 400 kHz A-line scanning rate
- noise-reduced 3D volumes, cross-sectional images (B-scan or C-scan images), or en face images can be displayed concurrent (or nearly concurrently) with their imaging.
- a system for executing the above-described methods is also contemplated within the scope of the present disclosure.
- a system may include a computer having one or more processors (e.g., in the form of an integrated circuit(s), discrete circuitry, or the like) for executing the method, storage (such as a hard disk, memory, RAM, or the like) and an input/output interface (e.g., display, keyboard, mouse, and the like).
- the above-described filters may be implemented via software executed by the processor(s) or hardware (e.g., via circuitry, optical filters, or the like) depending on the desired filtering.
- the storage may be located locally with the computer, or remotely, for example at a centralized database, and store the software instructions for executing the method and filtering, the filtered images and/or volumes, and/or the resulting noise-reduced images and/or volumes.
- the system may also be integrated or separate from a system used to obtain the images of the object to be processed.
- the computer may be the same as that used to control an OCT system.
- OCT ophthalmological imaging is used as an example modality and application thereof herein, the present disclosure is applicable to all types of coherent imaging modalities for any application producing multiple types of noise (e.g., random and speckle noise).
- additional filters may be used and any combination thereof may be used as a final output image.
- the filtering may be applied to any combination of 2D images (from any combination of planes), 3D volumes, and/or collections of 2D images from different locations in a 3D volume.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Signal Processing (AREA)
- Pathology (AREA)
- Molecular Biology (AREA)
- Psychiatry (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biophysics (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Physiology (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Eye Examination Apparatus (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
- Investigating Or Analysing Materials By Optical Means (AREA)
Abstract
Description
- This application claims priority to
U.S. Provisional Application Serial No. 62/812,728, filed on March 1, 2019 - Coherent imaging modalities (e.g., optical coherence tomography (OCT)) are subject to noise, which can degrade final image quality. In particular with ophthalmological imaging, the presence of noise can significantly degrade imaging fidelity and can lead to errors during segmentation of retinal layers (e.g., during segmentation of B-scans or like images including a depth dimension). These segmentation errors can further lead to errors in analysis of the images based on the segmentation, and thus ultimately, errors in diagnosis and treatment of an imaged subject.
- Currently, noise is treated by applying a common filter to an underlying set of data or images produced from that data. However, these filters are unable to remove all noise at appropriate levels. Further, noise reduction systems, and methods that utilize a large number of sample data and images to normalize noise are still subject to noises and errors that persist throughout the sample sets. Therefore, the images can still lack quality, and segmentation and analysis can still be prone to errors.
- According to a first example, an image processing method comprises: applying a first filter to an input image, thereby generating a first noise-reduced image; applying a second filter to the input image, thereby generating a second noise-reduced image; and combining the first noise-reduced image and the second noise-reduced image, thereby generating a final noise-reduced image, wherein the first filter is configured to suppress a first type of noise from the input image and the second filter is configured to suppress a second type of noise from the input image, the first and second types of noise being different.
- In various embodiments of the first example, the first noise-reduced image and the second noise-reduced images are combined by weighted averaging, the first noise-reduced image being weighted according to a level of the first type of noise in the input image and the second noise-reduced image being weighted according to a level of the second type of noise in the input image; the first noise-reduced image and the second noise-reduced images are combined, or the first and/or second noise-reduced images are produced, by a machine learning system; an intensity of at least one pixel of the first noise-reduced image is set by the first filter to correspond to a maximum intensity of a probability distribution of intensities of pixels at a corresponding location of an object in the image; the probability distribution is determined by a machine learning system; the first type of noise is random noise or noise caused by an imaging system that captured the input image; the first filter is a machine learning system trained with images taken from the same or substantially the same location as the input image; the first type of noise is speckle noise; the first filter is a machine learning system trained with images taken from locations adjacent to or nearby a location of the input image; the input image is an optical coherence tomography (OCT) or OCT-angiography B-scan image; the input image is an en face optical coherence tomography (OCT) or OCT-angiography image; the input image is obtained by an optical coherence tomography imaging system configured to operate at least at a 400 kHz A-line rate; the input image is an image of a retina; the method further comprises segmenting the input image as a result of applying the first filter and the second filter; and/or the method further comprises displaying the final noise-reduced image in real-time with the capturing of the input image.
- According to a second example, an image processing method comprises: filtering a first input image; filtering a second input image; and combining the filtered first input image and the filtered second input image, thereby generating a final noise-reduced volume, wherein the first input image and the second input image are different 2D images of a 3D volume, and wherein the first input image and the second input image are from different planes and/or en face images of different reference layers of the 3D volume.
- In various embodiments of the second example, filtering the first input image comprises: applying a first filter to the first input image, thereby generating a first noise-reduced image; applying a second filter to the first input image, thereby generating a second noise-reduced image; and combining the first noise-reduced image and the second noise-reduced image, thereby generating a final noise-reduced image, wherein the first filter is configured to suppress a first type of noise from the first input image and the second filter is configured to suppress a second type of noise from the first input image, the first and second types of noise being different.
-
-
Figure 1 illustrates noise as an image of the absolute difference between each pixel of two OCT B-scan images generated from scans of essentially the same location of a subject's eye at different times. -
Figures 2A and2B illustrate example noise reduction methods according to the present disclosure. -
Figure 3 illustrates an example probability distribution of the intensity of a given pixel. -
Figure 4 illustrates different intensity probability distributions for different areas of a B-scan. -
Figure 5 illustrates parallel filtering of B-scan images to produce a combined noise-reduced image. -
Figure 6 illustrates a comparison of a noise reduction method according to the present disclosure and a noise reduction method based on averaging. -
Figure 7 illustrates original and noise-reduced 1-pixel depth en-face images of the lamina cribrosa. -
Figure 8 illustrates original and noise-reduced B-scan images showing an inflammatory cell. -
Figure 9 illustrates an output of trained deep learning system including a noise-reduced image and a segmentation of the image. -
Figure 10 illustrates 3D noise reduction according to the present disclosure with different planes. -
Figures 11A ,11B ,11C , and11D illustrate example volumetric noise reduction methods according to the present disclosure. -
Figure 12 illustrates an example of noise reduction according to the method ofFig. 11A . - In view of the above, the present disclosure is directed to improvements in noise reduction for coherent imaging modalities. It is now recognized that more than one type of noise exists in images from coherent imaging modalities, and the types and levels of the noise can vary between pixels of an image. These types of noise include: 1) random noise from the system; and 2) speckle variation (noise) caused by the coherent imaging modalities or objects being imaged (e.g., a subject's eye or other biological tissue). The speckle noise may arise from interference of light waves having random phases, for example, as light scattered from various points of turbid object being imaged. As suggested above, while existing noise reductions methods and systems exist they are directed to treating the different types of noise with the same filter. And other machine learning based methods improperly rely on a substantial number of averaged images that can induce errors in the filtering process.
- The aforementioned two types of noise can be seen with respect to
Fig. 1 , which illustrates an image showing the absolute difference between each pixel of two OCT B-scan images generated from scans of essentially the same location of a subject's eye at different times. It is noted that the reference to OCT images is merely an example and that the systems and methods described by the present application are applicable to images generated by any coherent imaging technique. Because the general structure at the location is unchanged, the difference between the two images represents noise, where the brightness of each pixel corresponds to the level of noise. - Three regions of noise are identified in
Fig. 1 . Thefirst region 100 is an area above the retina, which includes random noise from the imaging system. Thesecond region 102 in the retinal tissue includes speckle variation due to blood flow as well as the random noise in the system. Thethird region 104 in the retinal tissue similarly includes other speckle variation due to live biological tissues (such as blood flow) as well as the random noise in the system. - Because the different types of noise, due to their difference in source, follow different statistical patterns, they should be processed separately. This process is illustrated in the flow chart of
Fig. 2A , showing an example noise reduction method according to the present disclosure. As seen therein, a plurality of noise reduction filters 202 (1 to N) are applied to anoisy input image 200. Eachnoise reduction filter 202 produces a corresponding noise reduced image 204 (1 to N). Thefilters 202 may be applied in parallel such that each noise reducedimage 204 only has the noise reduced that corresponds to the appliedfilter 202, and thus may still include the other types of noise not reduced by thecorresponding filter 202. Of course, actual processing may occur sequentially in time while retaining the parallel nature of the application. The plurality of noise reducedimages 204 can then combined 206 to produce a final noise reducedimage 208. The combination of noise reduced images results in the final noise reducedimage 208 having each type of noise reduced. - In other embodiments such as those corresponding to
Fig. 2B , however, the filters 212-1 to 212-N may be applied sequentially such that a first type of noise is removed by a first filter 212-1 to produce a first noise-reduced image 214-1 in which the first type of noise is removed but other types of noise remain. A second type of filter 212-2 may then be applied to the first noise-reduced image 214-1 to further remove a second type of noise, while still other types of noise remain. This process can be repeated with additional noise reduction filters 212-N until all desired noise is removed and a final noise-reducedimage 216 produced. - Each filter may be designed and applied based on the probability distribution for the relative intensity for the pixels in an image without a given type of noise. For example, the intensity of each pixel in an image, when subject to random noise, can be modeled as shown in
Fig. 3 . The "true value" (represented by the dashed vertical line) corresponds to the actual intensity of the pixel without any noise. In the probability distribution example ofFig. 3 , the 'true value' may be determined as the intensity where probability is a maximum. Each pixel may have its own unique probability distribution. In other words, the noise (type and level) may not be uniform throughout an entire image. For example, as seen inFig. 4 , theintensity probability distribution 408 for apixel 404 above the retina (having only random system noise) in a B-scan image 400 is skewed toward zero relative to theintensity probability distribution 410 for apixel 406 in a retinal layer (having random system noise and speckle noise) of theimage 400. The 'true value' intensity at the maximum of the probability distribution is shown in a corresponding noise-reduced B-scan image 402 where the intensity of thepixel 404 above the retina is much lower (darker) than that of theretinal pixel 406. - These probability distributions may be different for each type of noise. For example, the intensity probability distribution for pixels without many types of noise may be Gaussian; however, some removal of other noises may cause the intensity probability distributions to follow different statistical models. For noises resulting in the same type of distribution model, removal of each type of noise may result in probability distributions having a different mean, sigma value, or like statistical property.
- Therefore, if the intensity probability distribution is understood for the types of noise in each pixel of an image, the noise-free value of each pixel can be reconstructed. For example, the application of each filter may set each pixel of an image to have its true intensity value (e.g., the intensity corresponding to the maximum probability).
- Artificial intelligence systems such as deep learning (and other machine learning) models/systems can be used to determine the intensity probability distributions for each pixel (e.g., each location of the retina) to more accurately estimate the most probable intensity value of each pixel. The deep learning systems of each filter are particularly designed and trained to estimate the intensity probability distribution of each pixel. The design and training of the deep learning systems are based on an understanding of the fundamental physics in OCT imaging (or the other coherent imaging methods) so that the systems can be trained with images demonstrating the correct intensity probability distributions.
- For example, the deep learning systems can be trained with data sets having two or more images. In each set, the pixel intensity at the same corresponding location among different images is used to construct the intensity probability distribution of that pixel. Put another way, a perfectly large set will include at least one image having every possible intensity for a given pixel location, and the relative number of images in the set having a given intensity represents the corresponding probability of that intensity. Thus, the relative number of images for every intensity value will produce the intensity probability distribution. For sets of fewer images, the same process can be used to generate a probability distribution by fitting the relative number of images of each intensity to a known type of distribution curve. However, as the above description derives probability distribution models based on pixel intensities that retain noise information, the approach alone does not necessarily suppress the different types of noise separately.
- To account for the different types of noise, differences between the images of a set can be compared, and adjusted based on the comparison, prior to developing the probability distribution. For example, the effect of random noise can be derived by considering differences between images of a set of B-scan images taken at the same cross-sectional location through the retina. This is because the difference between images of the set will primarily represent random noise, since the underlying structure at the same cross-sectional location generally remains unchanged. Adjusting each image based on these differences has the effect of removing the random noise from the adjusted images. As a result, a probability distribution derived based on the adjusted images is a probability distribution for the image where random noise is suppressed but speckle is largely preserved. In these embodiments, the difference may be determined according to any statistical method, such as by simple subtraction, an average of the differences between each successive scan, or the like; and the adjustments may be made in any statistical manner, such as by subtracting the average difference from each image.
- Similarly, the effect of speckle noise can be found from sets of images of B-scans taken from adjacent or nearby locations within a transverse resolution of the imaging system. The differences between the images of such sets will primarily represent speckle noise. Again, the images may be adjusted according to any statistical determination of the difference, so that a resulting intensity probability distribution is one in which the pixel intensity at the maximum probability represents the most probable intensity without speckle noise (e.g., where both speckle noise and random noise are suppressed).
- Referring back to
Figs. 2A and2B , a deep learning system trained to suppress random noise may correspond to noise reduction filter 1 (a sharp filter) 202-1, 212-1 and a deep learning system trained to suppress speckle noise may correspond to noise reduction filter 2 (a smooth filter) 202-2, 212-2. Where deep learning systems are the filters, the original noisy image is input to the trained system (filter), and the output of the trained system is an image in which each pixel is set to the intensity corresponding to the maximum probability. In other words, a trained deep learning system may be one that has been trained to know the intensity probability distribution (or otherwise knows a 'true value' intensity) for each pixel without one type of noise. - Such a configuration where separately trained deep learning systems filter an original input B-scan image to produce a combined noise-reduced image is illustrated in
Fig. 5 . More particularly, an original input image (e.g., an OCT B-scan) 500 is input to both a trainedmachine learning system 502 providing a sharp filter and a trainedmachine learning system 504 providing a smooth filter. Thesharp filter 502 produces animage 506 with reduced random noise and the smooth filter produces animage 508 with reduced speckle noise. These reducednoise images image 510 with both random and speckle noises reduced. - In still other embodiments, any type of filter capable of reducing a desired type of noise may be used. Such filters may include median filters, Gaussian filters, spectral domain filters, and the like.
- The noise-reduced images output by each filter can be combined according to any statistical combination method. For example, each noise-reduced image can be combined through a weighted average technique where the weight of each image is adjusted to match the relative noise level of the corresponding noise type. In other words, each noise-reduced image can be weighted according to the amount of the corresponding noise in the original input image. Considering the example of
Fig. 5 , the output weights are 0.2 for the sharp filter (meaning 20% of the noise is random) 502 and 0.8 for the smooth filter (meaning 80% of the noise is speckle noise) 508. - By way of comparison, combining the outputs of each filter can produce a final noise-reduced image comparable to an image produced by averaging 128 images taken at the same location (a traditional technique for suppressing noise).
Fig. 6 shows such a comparison, where an original input image (without any noise reduction) 600, an image produced by averaging 128 images from substantially the same location (including the original input image) 602, and an a noise reducedimage 604 resulting from the above-described combination of outputs of a smooth filter and a sharp filter. As can be seen in the entire B-scan, and in the enlarged portions, separately applying different types noise reduction and then combining the outputs of the filters produces comparable or better results to the averaging, both clearly having less noise than the input image. However, with the method described herein, only one B-scan image is needed to be filtered, rather than the many needed if reducing noise by averaging or the perpetuated errors introduced if reducing noise based on a system trained by averaging. - Other embodiments may use additional filters, or use the filters individually. For example, the noise-reduced
output image 508 of the smooth filter 504 (removing speckle noise) has smoother retinal boundaries thanimage 506 output by thesharp filter 502. Therefore, the smoothed image 508 (rather than the output from the sharp filter, or the combined output image) may be best used in tasks such as visualization/display and layer segmentation.Fig. 7 illustrates this concept with respect to 1-pixel depth en-face images of the lamina cribrosa, where the primary noise component is speckle noise. As seen inFig. 7 , the noise-reducedimage 702 filtered only with a smooth filter to reduce speckle noise is smoother and more visually appealing than the original image 704. Thus, the speckle noise-reduced image 704 output from the smooth filter may be beneficial for improving visualization applications. - Similarly, because the noise reduced
output image 506 of thesharp filter 502 preserves speckle information, it may be best used in tasks such as abnormality detection (identifying inflammatory cells) or OCT angiography (which relies on differences between images at a common location to indicate blood flow). For example,Fig. 8 illustrates an original B-scan image 800 and a noise-reduced B-scan image 800 output by a sharp filter to remove random noise. In bothimages cell 804 is more identifiable in the random noise-reducedimage 802 as it is against a clearer background. - During the training of the deep learning systems for noise-reduction filtering, additional information such as the retinal layers, are implicitly learned. This results from the different statistical properties of noise associated with pixels of each retinal layer. As the deep learning system is trained to reduce the different noises (types and levels) in different retinal layers, it thus also learns which pixels are associated with each layer. Put another way, for the deep learning systems to remove proper noise types and levels from each retinal layer, it should know to which layer each pixel belongs. With such knowledge, layer boundaries can be identified, and segmentation realized, by determining the pixels at which the associated layers change.
- Thus, in some embodiments, noise-reduction filters and segmentation detectors can be trained simultaneously as part of the same deep learning system. Such an output is illustrated in
Fig. 9 , where anoriginal image 900 is input into a traineddeep learning system 902, which outputs a noise-reducedimage 904 and aretinal segmentation 906 of the image. Thesegmentation 906 includes (from top to bottom) boundaries of the inner limiting membrane (ILM), retinal pigment epithelium (RPE), and Bruch's membrane layers. - While 2D images such as B-scans have generally been described, the above method and corresponding system could be applied to any type of image. For example, the input may be a single optical coherence tomography (OCT) (or like coherent imaging modality) B-scan, a 3D volumetric scan, an en face image or C-scan, an angiographic B-scan image, an angiography en face image or C-scan, or like images from other imaging modalities.
- When using the methods described herein to reduce noise for 3D volumetric data (e.g., 3D OCT structural data, 3D OCT-angiography data), the noise reduction can be performed in different planes of 2D images that collectively form the 3D volumes, as shown in
Fig. 10 . In these embodiments, the noise (types and levels) may be different depending on the location in the volume (either at different depths, or different locations). Accordingly, different noise reduction filters may be applied to different portions of the volume. These filters, if trained machine learning systems, may be trained for different types of noise and/or the different effects of noise at different locations in the volume. Weights used to combine differently filtered images, or other combination techniques, may also be variable and change depending on where in the volume the images are located. - Any of the volumetric data may be noise reduced according to the above-described methods. For example, collections of B-scans in the
X-Z plane 1000 as described above can be individually noise reduced to reduce noise of an entire volume. In other embodiments, the data may be sampled (or re-sampled) in the Y-Z plane and noise reduction as described above applied on Y-Z B-scans 1002. Similarly, the noise reduction process can be applied on X-Y plane C-scans or en face images flattened over any depth/thickness and/or with respect to any reference layer. Such a reference layer may be the separation between two retinal tissue layers, for example, the ILM. - In some embodiments, filtering can be applied to different planes of 2D images of a 3D volume or other collection of 2D images (e.g., from non-adjacent locations of a larger 3D volume). Just as with the application of individual filters to the 2D images, the application of filters to different planes can be in parallel and combined, or sequential.
Fig. 11A illustrates a first example of sequential noise reduction of a 3D volume (e.g., an OCT-angiography (OCT-A) volume). According to the example ofFig. 11A , a first step of noise reducing a 3D OCT-A volume 1100 involves noise reduction of OCT-angiography images inX-Z plane 1102 of thevolume 1100, and a second step involves noise reduction of enface images 1104 of thevolume 1100, to produce a final noise-reduced3D volume 1106.Fig. 12 illustrates a comparison of an enface image 1200 of the deep plexiform layer taken from the original OCT-A volume 1100, and of a enface image 1202 of deep plexiform layer noise-reduced by the method ofFig. 11A . As can be seen inFig. 12 , vessel connectivity is greatly enhanced after noise reduction. -
Figs. 11B and 11C illustrate similar sequential methods with other combinations of planes. In the example ofFig. 11B ,X-Z plane images 1112 from anoriginal 3D volume 1110 are first filtered, followed byY-Z plane images 1114 and thenX-Y plane images 1116, to produce a final noise-reducedvolume 1118. And in the example ofFig. 11C ,X-Y plane images 1122 from anoriginal 3D volume 1120 are first filtered, and thenX-Z plane images 1124 are filtered to produce a final noise-reducedvolume 1126. As above, the different filterings may be directed to a type of noise and/or the plane in which it is applied. - Each filtering step may be performed by a filter trained or otherwise designed to remove a different type of noise and/or to identify noise in the respective plane in which it is applied. Removing a different type of noise in each filter step may be beneficial where each type of noise in more apparent in a different plane, whereas removing the same type of noise in multiple steps may be beneficial where one type of noise expresses itself differently in different planes. Where different types of noise are removed, the images of the plane in a subsequent filtering step may be generated from the original volume after a preceding noise reduction step. Accordingly, the subsequently generated images may already have the preceding type of noise removed. By sequentially applying the filters, a volume reconstructed from the finally filtered images represents a noise-reduced volume.
- While each of
Figs. 11A-11C represent the sequential application of filters where subsequent filters are applied to images in which some noise has already been reduced, other embodiments may apply filters in parallel. An example of noise reduction for different planes in parallel is shown inFig. 11D . Therein, noise reduction is separately performed in parallel on images of theX-Z plane 1132 and theY-Z plane 1136 of an original 3D volume. As above, performing noise reduction in parallel refers to performing the noise-reduction on images of a 3D volume not subject to the other noise-reduction application. Accordingly, the actual application of the noise-reduction may still be performed sequentially in time. The resulting noise-reducedvolumes volume 1142. Of course, different planes and combinations may be used in parallel in other embodiments. - The present disclosure provides various improvements compared to traditional noise reduction techniques (such as averaging 128 images taken from a single location). For example, it can take a relatively long time to obtain 128 images (or any number of images being averaged), which limits the total imaging speed and makes the variously averaged images more susceptible to motion artifact. Additionally, it is practically difficult to take so many images at a single location; therefore, each image of the average may include different information. Put another way, an imaging target at that location may not present in every image, or other elements may be confused as the imaging target. Further, to the extent any other machine learning systems are trained with the above-described average images as supervisory training data, the inherent limitations of averaging are imputed into the trained machine learning system.
- In addition, it has been empirically observed that the presently described noise reduction preserves the quantitative information available in the original images. Thus, the present noise reduction technique enables applications which were previously only achievable with high-speed devices (e.g. variable interscan time analysis (VISTA) systems), where the high-speed is used to take multiple repeated scans to suppress noise, possible on low-speed devices with fewer repeats. In particular such applications include angiographic imaging (e.g., OCT-A) and the like where imaging is of, or based on, dynamic (non-static) structures (e.g., movement of cells, such as red blood cells). Further in this vein, the noise reduction described herein may be used in at or near real-time imaging applications. In some embodiments, the input images may be obtained with high speed OCT imaging (e.g., operating at a 400 kHz A-line scanning rate), such that the filtering according to the present disclosure provides high fidelity images at that high speed. As a result, noise-reduced 3D volumes, cross-sectional images (B-scan or C-scan images), or en face images can be displayed concurrent (or nearly concurrently) with their imaging.
- A system for executing the above-described methods is also contemplated within the scope of the present disclosure. Such a system may include a computer having one or more processors (e.g., in the form of an integrated circuit(s), discrete circuitry, or the like) for executing the method, storage (such as a hard disk, memory, RAM, or the like) and an input/output interface (e.g., display, keyboard, mouse, and the like). The above-described filters may be implemented via software executed by the processor(s) or hardware (e.g., via circuitry, optical filters, or the like) depending on the desired filtering. The storage may be located locally with the computer, or remotely, for example at a centralized database, and store the software instructions for executing the method and filtering, the filtered images and/or volumes, and/or the resulting noise-reduced images and/or volumes. The system may also be integrated or separate from a system used to obtain the images of the object to be processed. For example, the computer may be the same as that used to control an OCT system.
- It is again noted that while OCT ophthalmological imaging is used as an example modality and application thereof herein, the present disclosure is applicable to all types of coherent imaging modalities for any application producing multiple types of noise (e.g., random and speckle noise). Additionally, although two filters are described herein, additional filters may be used and any combination thereof may be used as a final output image. Still further, in some embodiments it may be desirable to apply two or more filters in series, the series filters still being a parallel filtering path to other filters. In some embodiments, the filtering may be applied to any combination of 2D images (from any combination of planes), 3D volumes, and/or collections of 2D images from different locations in a 3D volume.
Claims (16)
- An image processing method comprising:applying a first filter to an input image, thereby generating a first noise-reduced image;applying a second filter to the input image, thereby generating a second noise-reduced image; andcombining the first noise-reduced image and the second noise-reduced image, thereby generating a final noise-reduced image,wherein the first filter is configured to suppress a first type of noise from the input image and the second filter is configured to suppress a second type of noise from the input image, the first and second types of noise being different.
- The method of claim 1, wherein the first noise-reduced image and the second noise-reduced images are combined by weighted averaging, the first noise-reduced image being weighted according to a level of the first type of noise in the input image and the second noise-reduced image being weighted according to a level of the second type of noise in the input image.
- The method of any of the above claims, wherein the first noise-reduced image and the second noise-reduced images are combined, or the first and/or second noise-reduced images are produced, by a machine learning system.
- The method of any of the above claims, wherein an intensity of at least one pixel of the first noise-reduced image is set by the first filter to correspond to a maximum intensity of a probability distribution of intensities of pixels at a corresponding location of an object in the image.
- The method of claim 4, wherein the probability distribution is determined by a machine learning system.
- The method of any of the above claims, wherein the first type of noise is random noise or noise caused by an imaging system that captured the input image.
- The method of claim 6, wherein the first filter is a machine learning system trained with images taken from the same or substantially the same location as the input image.
- The method of any of the above claims, wherein the first type of noise is speckle noise.
- The method of claim 8, wherein the first filter is a machine learning system trained with images taken from locations adjacent to or nearby a location of the input image.
- The method of any of the above claims, wherein the input image is an optical coherence tomography (OCT) or OCT-angiography B-scan image.
- The method of any of the above claims, wherein the input image is an en face optical coherence tomography (OCT) or OCT-angiography image.
- The method of any of the above claims, wherein the input image is obtained by an optical coherence tomography imaging system configured to operate at least at a 400 kHz A-line rate.
- The method of any of the above claims, wherein the input image is an image of a retina.
- The method of any of the above claims, further comprising:
segmenting the input image as a result of applying the first filter and the second filter. - The method of any of the above claims, further comprising:
displaying the final noise-reduced image in real-time with the capturing of the input image. - The method of any of the above claims, further comprising:filtering a second input image; andcombining the filtered second input image and the final noise-reduced image, thereby generating a final noise-reduced volume,wherein the input image and the second input image are different 2D images of a 3D volume, andwherein the input image and the second input image are from different planes, and/or are en face images of different reference layers, of the 3D volume.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE20159835.6T DE20159835T1 (en) | 2019-03-01 | 2020-02-27 | IMAGE QUALITY IMPROVEMENT PROCESS FOR OPTICAL COHERENCE TOMOGRAPHY |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201962812728P | 2019-03-01 | 2019-03-01 | |
US16/797,848 US11257190B2 (en) | 2019-03-01 | 2020-02-21 | Image quality improvement methods for optical coherence tomography |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3703004A2 true EP3703004A2 (en) | 2020-09-02 |
EP3703004A3 EP3703004A3 (en) | 2020-10-14 |
Family
ID=69742796
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP20159835.6A Pending EP3703004A3 (en) | 2019-03-01 | 2020-02-27 | Image quality improvement methods for optical coherence tomography |
Country Status (4)
Country | Link |
---|---|
US (2) | US11257190B2 (en) |
EP (1) | EP3703004A3 (en) |
JP (3) | JP7010983B2 (en) |
DE (1) | DE20159835T1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2022099851A1 (en) * | 2020-11-12 | 2022-05-19 | 生物岛实验室 | Particle beam imaging noise reduction method and apparatus |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB2536650A (en) | 2015-03-24 | 2016-09-28 | Augmedics Ltd | Method and system for combining video-based and optic-based augmented reality in a near eye display |
US11980507B2 (en) | 2018-05-02 | 2024-05-14 | Augmedics Ltd. | Registration of a fiducial marker for an augmented reality system |
US11766296B2 (en) | 2018-11-26 | 2023-09-26 | Augmedics Ltd. | Tracking system for image-guided surgery |
US11980506B2 (en) | 2019-07-29 | 2024-05-14 | Augmedics Ltd. | Fiducial marker |
US11382712B2 (en) | 2019-12-22 | 2022-07-12 | Augmedics Ltd. | Mirroring in image guided surgery |
US20210319551A1 (en) * | 2020-04-10 | 2021-10-14 | Topcon Corporation | 3d analysis with optical coherence tomography images |
US11896445B2 (en) | 2021-07-07 | 2024-02-13 | Augmedics Ltd. | Iliac pin and adapter |
Family Cites Families (26)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050157796A1 (en) | 2004-01-20 | 2005-07-21 | Victor Company Of Japan, Ltd. | Block noise reducing apparatus |
US7657098B2 (en) | 2005-05-02 | 2010-02-02 | Samsung Electronics Co., Ltd. | Method and apparatus for reducing mosquito noise in decoded video sequence |
EP3515063A1 (en) * | 2007-04-02 | 2019-07-24 | Esight Corp. | A method for augmenting sight |
US9347765B2 (en) * | 2007-10-05 | 2016-05-24 | Volcano Corporation | Real time SD-OCT with distributed acquisition and processing |
JP5035029B2 (en) | 2008-03-03 | 2012-09-26 | ソニー株式会社 | Signal processing apparatus and method, and program |
JP5147903B2 (en) | 2010-07-12 | 2013-02-20 | キヤノン株式会社 | Image processing apparatus, image processing method, and program |
JP5882576B2 (en) * | 2010-12-01 | 2016-03-09 | キヤノン株式会社 | Image processing apparatus, image processing apparatus control method, and program |
JP2013063166A (en) * | 2011-09-16 | 2013-04-11 | Fujifilm Corp | Radiographic apparatus and image processing method |
JP5903305B2 (en) * | 2012-03-26 | 2016-04-13 | 東芝デジタルメディアエンジニアリング株式会社 | Image processing apparatus and image processing method |
JP2014025804A (en) * | 2012-07-26 | 2014-02-06 | Sanyo Electric Co Ltd | Information acquisition device and object detection device |
US9639915B2 (en) * | 2012-08-08 | 2017-05-02 | Samsung Electronics Co., Ltd. | Image processing method and apparatus |
JP6381895B2 (en) | 2012-11-20 | 2018-08-29 | キヤノンメディカルシステムズ株式会社 | Image processing apparatus, image processing method, and X-ray diagnostic apparatus |
JP6408916B2 (en) * | 2013-01-21 | 2018-10-17 | 興和株式会社 | Image processing apparatus, image processing method, image processing program, and recording medium storing the program |
JP5991438B2 (en) * | 2013-08-08 | 2016-09-14 | 株式会社島津製作所 | Image processing device |
JP5784105B2 (en) * | 2013-12-27 | 2015-09-24 | キヤノン株式会社 | Image processing apparatus and image processing method |
KR102165610B1 (en) * | 2014-04-23 | 2020-10-14 | 삼성전자주식회사 | X-ray Imaging Apparatus and Image Processing Method of X-ray Imaging Apparatus |
JP6731919B2 (en) * | 2014-11-20 | 2020-07-29 | エージェンシー フォー サイエンス,テクノロジー アンド リサーチ | Speckle reduction in optical coherence tomography images |
US9591240B1 (en) * | 2014-12-01 | 2017-03-07 | The Florida State University Research Foundation, Inc. | System and method for generating a dataset for real noise reduction evaluation |
US10117568B2 (en) * | 2015-01-15 | 2018-11-06 | Kabushiki Kaisha Topcon | Geographic atrophy identification and measurement |
US9984459B2 (en) * | 2015-04-15 | 2018-05-29 | Kabushiki Kaisha Topcon | OCT angiography calculation with optimized signal processing |
GB2549515A (en) * | 2016-04-20 | 2017-10-25 | Michelson Diagnostics Ltd | Processing optical coherence tomography scans |
US20180012359A1 (en) * | 2016-07-06 | 2018-01-11 | Marinko Venci Sarunic | Systems and Methods for Automated Image Classification and Segmentation |
JP6581068B2 (en) | 2016-11-11 | 2019-09-25 | 株式会社東芝 | Image processing apparatus, image processing method, program, operation control system, and vehicle |
WO2018159689A1 (en) * | 2017-02-28 | 2018-09-07 | 地方独立行政法人神奈川県立産業技術総合研究所 | Method for observing inner structure of ceramic, method for producing ceramic, analysis system, and system for producing ceramic |
EP3404611A1 (en) | 2017-05-19 | 2018-11-21 | RetinAI Medical GmbH | Reducing noise in an image |
JP6873831B2 (en) * | 2017-06-07 | 2021-05-19 | キヤノンメディカルシステムズ株式会社 | Medical image diagnostic equipment, medical image processing equipment and medical image processing program |
-
2020
- 2020-02-21 US US16/797,848 patent/US11257190B2/en active Active
- 2020-02-27 DE DE20159835.6T patent/DE20159835T1/en active Pending
- 2020-02-27 EP EP20159835.6A patent/EP3703004A3/en active Pending
- 2020-02-27 JP JP2020032136A patent/JP7010983B2/en active Active
-
2021
- 2021-07-29 JP JP2021123867A patent/JP7222038B2/en active Active
- 2021-07-29 JP JP2021123868A patent/JP7154353B2/en active Active
-
2022
- 2022-01-10 US US17/647,527 patent/US20220130021A1/en active Pending
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2022099851A1 (en) * | 2020-11-12 | 2022-05-19 | 生物岛实验室 | Particle beam imaging noise reduction method and apparatus |
Also Published As
Publication number | Publication date |
---|---|
EP3703004A3 (en) | 2020-10-14 |
US20220130021A1 (en) | 2022-04-28 |
DE20159835T1 (en) | 2021-02-25 |
US11257190B2 (en) | 2022-02-22 |
JP7222038B2 (en) | 2023-02-14 |
JP7010983B2 (en) | 2022-01-26 |
JP2020140715A (en) | 2020-09-03 |
JP2021176101A (en) | 2021-11-04 |
JP2021176102A (en) | 2021-11-04 |
JP7154353B2 (en) | 2022-10-17 |
US20200279352A1 (en) | 2020-09-03 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3703004A2 (en) | Image quality improvement methods for optical coherence tomography | |
Duan et al. | Denoising optical coherence tomography using second order total generalized variation decomposition | |
US20200234080A1 (en) | Reducing noise in an image | |
Smith et al. | Automated, depth-resolved estimation of the attenuation coefficient from optical coherence tomography data | |
Kafieh et al. | An accurate multimodal 3-D vessel segmentation method based on brightness variations on OCT layers and curvelet domain fundus image analysis | |
JP2023520001A (en) | Correction of Flow Projection Artifacts in OCTA Volumes Using Neural Networks | |
Chithra et al. | Hybrid denoising technique for suppressing Gaussian noise in medical images | |
JP7116763B2 (en) | A signal processing method for reducing three-dimensional shadows in optical coherence tomography (OCT) images | |
Tsai et al. | Adaptive segmentation of vessels from coronary angiograms using multi-scale filtering | |
Yu et al. | Generative adversarial networks with dense connection for optical coherence tomography images denoising | |
CN115456890A (en) | Method for generating anti-medical CT image denoising based on multi-scale dual-domain discriminator | |
Esmaeili et al. | 3D segmentation of retinal cysts from SD-OCT images by the use of three dimensional curvelet based K-SVD | |
JP2022082541A (en) | 3d analysis with optical coherence tomography images | |
Sharif et al. | Extraction and analysis of RPE layer from OCT images for detection of age related macular degeneration | |
Anantrasirichai et al. | Adaptive-weighted bilateral filtering for optical coherence tomography | |
Marciniak et al. | Measurement of retina vessels by segmentation of images reconstructed from optical coherence tomography data | |
Stankiewicz et al. | Volumetric segmentation of human eye blood vessels based on OCT images | |
Bindhya et al. | A Review on Methods of Enhancement And Denoising in Retinal Fundus Images | |
Rowedder et al. | Denoising of home OCT images using Noise2Noise trained on artificial eye data | |
Chieng et al. | Efficient morphological-based edge detection algorithm for computed tomography cardiac images | |
Li et al. | Oct image blind despeckling based on gradient guided filter with speckle statistical prior | |
Ahmed et al. | Framework of Unsupervised based Denoising for Optical Coherence Tomography | |
Kumar et al. | New Ultrasound Image Watermarking Enhancement algorithm for cardiovascular disease | |
de Moura et al. | Automatic vessel detection by means of brightness profile characterization in OCT images | |
Rizki et al. | Active Contour-Based Analysis of Stent Characterization in Cardiac Coronary Arteries from 2D Intravascular Ultrasound Images |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED |
|
AK | Designated contracting states |
Kind code of ref document: A2 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
PUAL | Search report despatched |
Free format text: ORIGINAL CODE: 0009013 |
|
AK | Designated contracting states |
Kind code of ref document: A3 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G06T 5/00 20060101AFI20200904BHEP Ipc: A61B 5/00 20060101ALI20200904BHEP Ipc: G06T 5/20 20060101ALI20200904BHEP Ipc: A61B 3/10 20060101ALI20200904BHEP |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: ZAIXING, MAO Inventor name: KINPUI, CHAN Inventor name: ZHENGUO, WANG |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20210414 |
|
RBV | Designated contracting states (corrected) |
Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
17Q | First examination report despatched |
Effective date: 20221207 |