EP3417401A1 - Methods and apparatus for reducing artifacts in oct angiography using machine learning techniques - Google Patents
Methods and apparatus for reducing artifacts in oct angiography using machine learning techniquesInfo
- Publication number
- EP3417401A1 EP3417401A1 EP17754001.0A EP17754001A EP3417401A1 EP 3417401 A1 EP3417401 A1 EP 3417401A1 EP 17754001 A EP17754001 A EP 17754001A EP 3417401 A1 EP3417401 A1 EP 3417401A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- octa
- oct
- data
- base unit
- artifacts
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 238000002583 angiography Methods 0.000 title description 12
- 238000010801 machine learning Methods 0.000 title description 3
- 238000007781 pre-processing Methods 0.000 claims abstract description 11
- 230000004044 response Effects 0.000 claims abstract description 3
- FCKYPQBAHLOOJQ-UHFFFAOYSA-N Cyclohexane-1,2-diaminetetraacetic acid Chemical compound OC(=O)CN(CC(O)=O)C1CCCCC1N(CC(O)=O)CC(O)=O FCKYPQBAHLOOJQ-UHFFFAOYSA-N 0.000 claims abstract 17
- 238000012549 training Methods 0.000 claims description 14
- 239000000203 mixture Substances 0.000 claims description 4
- 238000001514 detection method Methods 0.000 claims 1
- 239000011159 matrix material Substances 0.000 claims 1
- 238000007670 refining Methods 0.000 claims 1
- 238000012014 optical coherence tomography Methods 0.000 description 49
- 238000003384 imaging method Methods 0.000 description 12
- 238000012545 processing Methods 0.000 description 10
- 238000000605 extraction Methods 0.000 description 9
- 210000001525 retina Anatomy 0.000 description 9
- 210000001519 tissue Anatomy 0.000 description 9
- 208000005590 Choroidal Neovascularization Diseases 0.000 description 7
- 206010060823 Choroidal neovascularisation Diseases 0.000 description 7
- 230000002207 retinal effect Effects 0.000 description 7
- 230000002792 vascular Effects 0.000 description 7
- 238000012800 visualization Methods 0.000 description 7
- 230000017531 blood circulation Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000005259 measurement Methods 0.000 description 5
- 238000012360 testing method Methods 0.000 description 5
- 206010064930 age-related macular degeneration Diseases 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 4
- 238000013534 fluorescein angiography Methods 0.000 description 4
- 208000002780 macular degeneration Diseases 0.000 description 4
- 230000009467 reduction Effects 0.000 description 4
- 230000011218 segmentation Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000001575 pathological effect Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 210000005166 vasculature Anatomy 0.000 description 3
- 210000004204 blood vessel Anatomy 0.000 description 2
- MOFVSTNWEDAEEK-UHFFFAOYSA-M indocyanine green Chemical compound [Na+].[O-]S(=O)(=O)CCCCN1C2=CC=C3C=CC=CC3=C2C(C)(C)C1=CC=CC=CC=CC1=[N+](CCCCS([O-])(=O)=O)C2=CC=C(C=CC=C3)C3=C2C1(C)C MOFVSTNWEDAEEK-UHFFFAOYSA-M 0.000 description 2
- 229960004657 indocyanine green Drugs 0.000 description 2
- 239000012528 membrane Substances 0.000 description 2
- 230000004286 retinal pathology Effects 0.000 description 2
- 239000000523 sample Substances 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000003325 tomography Methods 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 208000002177 Cataract Diseases 0.000 description 1
- 208000001351 Epiretinal Membrane Diseases 0.000 description 1
- 208000034699 Vitreous floaters Diseases 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 238000010521 absorption reaction Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000002238 attenuated effect Effects 0.000 description 1
- 210000003161 choroid Anatomy 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000004424 eye movement Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical compound [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 230000031700 light absorption Effects 0.000 description 1
- 238000007477 logistic regression Methods 0.000 description 1
- 230000008397 ocular pathology Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000007170 pathology Effects 0.000 description 1
- 108091008695 photoreceptors Proteins 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
- 238000004445 quantitative analysis Methods 0.000 description 1
- 238000007637 random forest analysis Methods 0.000 description 1
- 238000011946 reduction process Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000004232 retinal microvasculature Effects 0.000 description 1
- 239000000790 retinal pigment Substances 0.000 description 1
- 210000003583 retinal pigment epithelium Anatomy 0.000 description 1
- 210000001210 retinal vessel Anatomy 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000004218 vascular function Effects 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7203—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
- A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
- A61B3/102—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for optical coherence tomography [OCT]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
- A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
- A61B3/12—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for looking at the eye fundus, e.g. ophthalmoscopes
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0062—Arrangements for scanning
- A61B5/0066—Optical coherence imaging
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
- A61B5/02007—Evaluating blood vessel condition, e.g. elasticity, compliance
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7203—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
- A61B5/7207—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal of noise induced by motion artifacts
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7253—Details of waveform analysis characterised by using transforms
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
- A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/30—Noise filtering
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
- A61B5/026—Measuring blood flow
- A61B5/0261—Measuring blood flow using optical means, e.g. infrared light
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/02—Preprocessing
- G06F2218/04—Denoising
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10101—Optical tomography; Optical coherence tomography [OCT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Definitions
- Embodiments of the present invention relate generally to the field of optical coherence tomography (OCT) angiography and applications thereof, and specifically methods and apparatus for improved processing of OCT angiography (OCTA) data and reducing the effects of various errors caused by, for example, projection artifacts, low OCT signal, and noise.
- OCT optical coherence tomography
- OCTA OCT angiography
- Optical coherence tomography angiography is a non-invasive vascular imaging modality to visualize flow by detecting motion contrast using repeated OCT measurements at the same location.
- OCTA optical coherence tomography angiography
- FA fluorescein angiography
- ICG indocyanine green
- OCTA imaging is injection-free and provides depth- resolved three dimensional (3D) vascular information of the blood flow or vasculature in the tissue such as in an eye.
- 3D three dimensional
- a method of reducing artifacts includes obtaining OCT/OCTA data from an OCT/OCTA imager; preprocessing OCTA/OCT volume data; extracting features from the preprocessed OCTA/OCT volume data; classifying the OCTA/OCT volume data to provide a probability determination data; determining a percentage data from the probability determination data; and reducing artifacts in response to the percentage data.
- Figure 1 illustrates an exemplary image of the OCTA B-scan with projection artifacts.
- Figures 2A through 2D illustrate exemplary images of OCTA imaging of a normal subject.
- Figure 3 shows a block diagram illustrating the steps to reduce projection artifacts in OCTA 3D volume.
- Figure 4 illustrates an exemplary image of the OCTA B-scan at the same location as Fig. 1 after projection artifacts are reduced.
- Figures 5 A through 5D illustrate exemplary images of OCTA imaging of the same normal subject shown in Fig. 2 after the projection artifacts are reduced.
- Figures 6A and 6B illustrate exemplary images of the outer retina of an age-related macular degeneration (AMD) patient with choroidal neovascularization (CNV) before and after projection artifacts are reduced in the OCTA volume.
- AMD age-related macular degeneration
- CNV choroidal neovascularization
- Figure 7 shows a block diagram illustrating the steps to train the classifier used for projection artifacts reduction.
- OCTA imaging detects vessels with blood flow.
- flow and "vessel” are therefore used interchangeably in the following descriptions.
- OCTA employs motion contrast imaging in order to generate images that show flow, in particular blood flow.
- an OCTA imager compares the differences in the backscattered OCT signal intensity between sequential OCT B-scans taken at the same cross-section of the sample in order to construct a map of blood flow.
- the OCT scans can be corrected for eye movement between sequential images.
- both the OCT images and the derivative OCTA images can be provided.
- Projection or decorrelati on-tail artifacts are one of the most important artifacts that could limit the clinical utility and accuracy of OCTA results.
- Current OCTA processing techniques can generate false motion contrast signals in tissue that falls underneath a blood flow region, even when the underlying tissue is static.
- OCTA techniques are based on the principle of obtaining motion contrast, i.e. identifying and quantifying the change in OCT signal at different depths in the tissue. When the light passes through a blood vessel or a flow region, various factors such as forward scattering, refraction, absorption and path length variations cause unpredictable changes to the light field (and signal) at subsequent depths.
- the backscattered light (and hence the signal) that comes from underneath a region of flow inherits the changes in light field and signal from above, and hence may show a false motion contrast signal, depending on the level of backscattered light and change imparted by the disturbance above. It is very difficult to quantify or correct these changes as they are variable in nature and change in each
- Figure 1 illustrates an exemplary image of the OCTA B-scan illustrating projection artifacts.
- some projection artifacts are illustrated by arrows.
- Figure 1 further shows the internal limiting membrane (ILM) and retinal pigment epithelial (RPE).
- Figure 1 shows OCTA signal in the human retinal layers.
- the arrows indicate the projection artifacts at different retinal levels, whereas the true location of the blood vessels is in the retina above. Hence, any quantitative analysis that occurs without removing the projection artifacts will be misleading, inaccurate, or sub-optimal at best.
- Shadowing artifacts occur when the OCT signal is attenuated behind an absorbing of scattering opacity or obstruction. No or low OCT signal results in no or low OCTA signal. These artifacts can be due to the pathologies of patients such as epi-retinal membranes (floaters) and cataracts. The artifacts can also be due to strong light absorption in the upper tissue layers. Some imaging and processing techniques may be applied to alleviate the shadowing effect. Subsequent image processing and analysis for OCTA can be adjusted accordingly to offset the shadowing effect.
- OCTA noise Another artifact is noise.
- System noise and fluctuations in OCT incident light intensity can result in high OCTA signal even at locations of static tissue with no flow.
- OCTA noise, or false-positive flow can be visually identified by its short segment and isolation from neighboring structured vessels.
- the presence of noise affects subsequent quantification and visualization of small capillaries.
- OCTA volume data may consist of artificial signals that are not related to flow.
- the errors in the OCTA data caused by factors including projection artifacts, shadowing, and noise can be detected and reduced through several methods and techniques according to some embodiments and discussed in this disclosure. This reduction in the artifacts can result in improving the image quality of retinal microvasculature visualization and accuracy of the subsequent quantitative measurements for blood flow.
- the methods used to reduce OCTA artifacts can be generalized to process both OCT and OCTA 3D volume, 2D plane (B-scan), and ID line (A-line) data.
- the artifacts-reduced volume can be used for true 3D visualization.
- the artifacts-reduced volume can be used to generate 2D en face projection images.
- the methods to generate en face images have been disclosed in previous applications. See, e.g., John Davis et al. "Enhanced imaging for optical coherence tomography," US Patent US 8,781,214 B2, July 2014, which is herein incorporated by reference in its entirety.
- the OCTA data can be visualized in 3D and/or 2D by using a different color scheme for pre-processed original signals and artifact signals. For example, voxels/pixels with true signals can be color-coded in grayscale, while projection artifacts color- coded in red, and shadowing artifacts in blue.
- vascular parameters can be calculated from the artifacts-reduced OCTA volume.
- quantitative measurements can be calculated with 3D volume- based parameters and/or 2D en face image-based parameters.
- the parameters include, but are not limited to, flow volume/area, non-flow volume/area, flow density (volume/area/length density), vessel caliber, vessel branching, and tortuosity.
- Figures 1 and 2 A through 2D illustrate projection artifacts in a normal subject with no retinal pathologies based on clinical evaluation, as demonstrated by the B-scan (Fig. 1) and en face (Figs. 2A through 2D) images.
- Figures 2A through 2D illustrate exemplary images of OCTA imaging of a normal subject, with Figure 2 A illustrating en face images of four retinal layers, superficial capillary plexus, with Figure 2B illustrating the deep capillary plexus, with Figure 2C illustrating the outer retina, and with Figure 2D illustrating the choriocapillaris.
- Figures 2A through 2D have been generated from the pre-processed OCTA volume before projection artifacts are reduced.
- the projection artifacts appear at different retinal layers, as indicated by the arrows in Figure 1.
- the projection artifacts coming from the superficial capillary plexus (Figure 2 A) are most noticeable, causing false OCTA signals with similar vascular pattern in the deep capillary plexus ( Figure 2B), outer retina ( Figure 2C), and choriocapillaris (Figure 2D) layers, where no capillaries actually exist.
- FIG. 3 illustrates an exemplary flow diagram demonstrating the steps to reduce artifacts in an OCTA 3D volume.
- An OCTA imager (block 301) generates OCTA volume from OCT data using methods described in previously filed applications. See, e.g., Yali Jia et al. "Split-spectrum amplitude-decorrelation angiography with optical coherence tomography," Optics Express, Feb 2012, which is herein incorporated by reference in its entirety.
- an OCT imager can also be used to provide the structural OCT volume for additional information.
- the OCTA and OCT imager can also be combined to a single
- the OCTA volume and OCT volume data 302 is first passed to an optional preprocessing processer 303.
- the pre-processing processer 303 first detects regions with OCT or OCTA signals above background noise. Background regions can be excluded in the later processing steps to speed up the processing time.
- landmarks are detected along each OCT/OCTA A-line (depth-direction). These landmarks may include peaks and valleys along the ID A-line signal profile, and are often associated with retinal layer boundaries. For example, inner limiting membrane (ILM), junction of inner and outer photoreceptor segments (IS/OS), and retinal pigment epithelium (RPE) usually have stronger OCT intensities and appear as peak points along OCT A-lines.
- ILM inner limiting membrane
- IS/OS junction of inner and outer photoreceptor segments
- RPE retinal pigment epithelium
- the locations or depths of these landmarks can be further refined by averaging over neighboring landmarks (across A-lines and across B-scans). Next, flattening is performed to align all A-scans to a chosen landmark in depth. This is a common step performed for retina segmentation and has been disclosed previously. See, e.g., Mona K. Garvin et al. "Automated 3-D Intraretinal Layer Segmentation of Macular Spectral -Domain Optical
- the OCTA and OCT volume 302 are passed to a feature extraction processer 304. If the optional pre-processing processer 303 is applied, the pre-processed OCTA and OCT volume, along with outputs from the preprocessing processer 303 (for example, detected landmarks) are passed to the feature extraction processer 304.
- Feature-extraction processer 304 extracts features for each base unit.
- the base unit can be one single voxel or a localized region formed by a small number of voxels.
- Feature extraction involving spatial location or depth of the current base unit can include, for example, distance to landmarks (measured in pixels or in microns). Such extraction may also include relative distance (RD) to landmarks, for example, the relative distance from the current base unit (z current ) to landmark A (z A ) can be computed by normalizing with the distance between landmark A and B (z B ). This can be given by the following relation:
- RDA( z ) ⁇ z current ⁇ Z A ⁇ / ⁇ Z A ⁇ Z B ⁇ -
- Feature extraction involving pre-processed OCT and OCTA intensity can include the OCT intensity of the current base unit and the OCTA intensity of the current base unit. Furtherm ore, derivatives (1 st , 2 nd , . . . ) of OCT intensity in each x-, y-, z-direction from the current base uni t and derivatives (1 st , 2 nd , . . . ) of OCTA intensity in each x-, y-, z-direction from the current base unit can be included. Furthermore, intensities and derivatives neighboring base units can be used The kernel size of the neighboring base units to be included as features can be fixed.
- the surrounding 26 voxels in a 3 x3 x3 kernel can be defined as neighbors.
- the kernel size can also be dynamically determined. For example, a bigger kernel s ize can be assigned to a voxel with a higher OCTA intensity.
- Such features may also include a c orresponding OCT intensity at the same depth location where maximum OCTA intensity along A -line is detected.
- Yet another example of these features includes One-dimensional (ID) derivativ e oi OCTA max (z) .
- Feature extraction may also include information related to vessel caliber.
- Such features include the distance to a closet base unit with half the OCTA intensity of the current base unit in the x-direction, the distance to the closet base unit with half the OCTA intensity of the current base unit in the y-direction, the distance to the closet base unit with half the OCTA intensity of the current base unit in the +z-direction, or the distance to the closet base unit with half the OCTA intensity of the current base unit in -z-direction.
- the extracted features are passed to a classifier (block 305).
- the classifier is trained with a sufficiently large dataset where each OCTA voxels are manually labeled by human experts to indicate the presence of different types of artifacts, including projection artifacts. The details of the how the classifier can be trained is described in the Training classifier section and Figure 7.
- the classifier can also be designed with observational criteria.
- the classifier then returns the probability or score of each base unit belonging to one of the classification categories. For example, three categories can be used: a purely true flow signal, a purely artifact signal, and a mixture of both true flow and artifact.
- the classifier can return a hard classification which predicts which categories the base unit belongs to, without providing the probability.
- the probability volume or categorical results provided by the classifier is passed to a transform processer (block 306).
- the processer transforms the probability or categorical results to the percentage of true signal in each base unit.
- the transform function can be a linear transformation determined empirically by phantom studies or by optimizing human retinal scan data to meet clinical understanding. For example,
- Percentage ⁇ wi ⁇ Prob tme + w 2 ⁇ Probe d + w 0 , where Percentage ⁇ is the percentage of true signal in the base unit, Prob s and Prob ⁇ is the probability of belonging to a purely true flow signal group and the probability of belonging to a mixed signal group, respectively.
- the parameters w 0 , wi, and w 2 are the linear weighting factors, which may be determined empirically.
- OC73 ⁇ 4 post (x,y,z) OCTA (x,y,z) ⁇ Percentage ⁇ (x,y,z), and the post-processed artifacts-reduced OCTA volume (OCTA post ) is obtained.
- the artifacts- reduced OCTA data can then be utilized for display (block 308) including but not limited to 3D visualization with volume rendering, 2D visualization of en face projection images and B-scans.
- the artifacts-reduced OCTA data can also be used for further analysis (block 309) to calculate flow or vasculature-related quantitative parameters.
- Figure 4 illustrates an exemplary image of the OCTA B-scan at the same location as Fig 1 after projection artifacts are reduced.
- the arrows indicate a few locations where the projection artifacts are reduced after processing.
- Elongated inner retinal vessels which appear in the pre- processed B-scan ( Figure 1) are shortened. This circular shape of vessels is more consistent with their physical dimensions. Projection artifacts at the IS/OS and RPE layers are also significantly reduced.
- Figures 5 A through 5D illustrate exemplary images of OCTA imaging of the same normal subject shown in Figures 2 A through 2D after the projection artifacts are reduced.
- the four en face images include superficial capillary plexus (Figure 5A), deep capillary plexus (Figure 5B), outer retina (Figure 5C), and choriocapillaris ( Figure 5D).
- Figures 5 A through 5D show the post-processed en face images as compared to the pre-processed en face images in Figures 2A through 2D.
- the duplicated vascular networks are removed from the bottom layers, while the remaining networks are preserved and well-connected.
- Figures 6A and 6B illustrates exemplary images of the outer retina of an AMD patient with CNV before ( Figure 6 A) and after ( Figure 6B) projection artifacts are reduced in the OCTA volume. After processing, the projection artifacts are reduced and the CNV network is better visualized (Fig. 6B). The CNV boundaries are also easier to outline which allows more reliable patient follow-up to assess treatment outcome.
- Figure 7 shows a block diagram illustrating the steps to train the classifier used for projection artifacts reduction.
- the classifier used in the projection artifacts reduction process illustrated as block 305, can be pre-trained with a sufficiently large amount of data.
- Figure 7 is an exemplary flow diagram demonstrating the steps to train the classifier.
- First a training dataset with co-acquired OCT and OCTA volume data from subjects with varying ages, genders and retinal pathologies are collected by an OCT/OCTA imager in block 701.
- A-lines are randomly selected from the OCT/OCTA volume for normal subjects, and randomly selected within pathological areas in scans of pathological patients.
- human experts grade every base unit of these A-lines in block 702.
- Each base unit is labeled with a category.
- the categories can include pure flow signal, pure projection artifacts signal, mixed signal, noise, and unknown signal.
- a subset of dataset is used as testing dataset in block 707 and not used during the training process.
- the OCT and OCTA volume data goes through the pre-processing and feature extraction step in block 703 as described in the previous sections.
- the volume data, features and the human graded label are then passed to a classifier in block 704.
- the machine learning model for example, can be based on logistic regression, ensemble models such as random forest, naive bayes, support vector machine, or combinations of different models.
- the training error is calculated during the training process in block 705.
- the testing dataset (block 707) is inputted to the classifier and the testing error is calculated in block 706.
- the training error (block 705) and testing error (block 706) are then used to refine the classifier in block 708. During this step, the parameters and features in the classifier are refined to minimize while balancing the error from the training dataset and from the testing dataset.
- the method described herein is applied to reduce projection artifacts in OCTA volume, but other artifacts such as noise and shadowing artifacts, can also be reduced through the same processing.
- the method can also be applied to detect artifacts in OCT volume, such as shadowing artifacts.
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Veterinary Medicine (AREA)
- Public Health (AREA)
- Surgery (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Pathology (AREA)
- Artificial Intelligence (AREA)
- Signal Processing (AREA)
- Physiology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Psychiatry (AREA)
- Ophthalmology & Optometry (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Evolutionary Computation (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Vascular Medicine (AREA)
- Cardiology (AREA)
- Multimedia (AREA)
- Eye Examination Apparatus (AREA)
Abstract
Description
Claims
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PL17754001T PL3417401T3 (en) | 2016-02-19 | 2017-02-17 | Method for reducing artifacts in oct using machine learning techniques |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201662297649P | 2016-02-19 | 2016-02-19 | |
PCT/US2017/018521 WO2017143300A1 (en) | 2016-02-19 | 2017-02-17 | Methods and apparatus for reducing artifacts in oct angiography using machine learning techniques |
Publications (3)
Publication Number | Publication Date |
---|---|
EP3417401A1 true EP3417401A1 (en) | 2018-12-26 |
EP3417401A4 EP3417401A4 (en) | 2019-07-31 |
EP3417401B1 EP3417401B1 (en) | 2022-01-19 |
Family
ID=64325909
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP17754001.0A Active EP3417401B1 (en) | 2016-02-19 | 2017-02-17 | Method for reducing artifacts in oct using machine learning techniques |
Country Status (4)
Country | Link |
---|---|
EP (1) | EP3417401B1 (en) |
CN (1) | CN108885687A (en) |
ES (1) | ES2908188T3 (en) |
PL (1) | PL3417401T3 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113543695A (en) * | 2019-03-11 | 2021-10-22 | 佳能株式会社 | Image processing apparatus, image processing method, and program |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110033861B (en) * | 2019-03-26 | 2021-03-23 | 西安交通大学 | Blood vessel and macular avascular region quantitative analysis method and system suitable for OCTA image |
WO2022037517A1 (en) * | 2020-08-17 | 2022-02-24 | The University Of Hong Kong | Methods and system for automated cloud-based quantitative assessment of retinal microvasculature using optical coherence tomography angiography images |
CN115919362B (en) * | 2023-03-15 | 2023-05-30 | 深圳英美达医疗技术有限公司 | Method, device, equipment and storage medium for removing artifacts of ultrasonic imaging system |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5785042A (en) * | 1995-02-14 | 1998-07-28 | Duke University | Magnetic resonance imaging method providing for correction of striation artifacts |
EP2742460A4 (en) * | 2011-08-09 | 2015-09-09 | Optovue Inc | Motion correction and normalization of features in optical coherence tomography |
US9241626B2 (en) * | 2013-03-14 | 2016-01-26 | Carl Zeiss Meditec, Inc. | Systems and methods for improved acquisition of ophthalmic optical coherence tomography data |
US10497124B2 (en) * | 2013-03-15 | 2019-12-03 | Kabushiki Kaisha Topcon | Optic disc image segmentation method and apparatus |
US9778021B2 (en) * | 2013-08-29 | 2017-10-03 | Carl Zeiss Meditec, Inc. | Evaluation of optical coherence tomographic data prior to segmentation |
WO2015060897A1 (en) * | 2013-10-22 | 2015-04-30 | Eyenuk, Inc. | Systems and methods for automated analysis of retinal images |
CN104143087A (en) * | 2014-07-24 | 2014-11-12 | 苏州大学 | SD-OCT-based contused retina intramembrane and extraarticular deletion detection method |
-
2017
- 2017-02-17 CN CN201780013438.5A patent/CN108885687A/en active Pending
- 2017-02-17 EP EP17754001.0A patent/EP3417401B1/en active Active
- 2017-02-17 ES ES17754001T patent/ES2908188T3/en active Active
- 2017-02-17 PL PL17754001T patent/PL3417401T3/en unknown
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113543695A (en) * | 2019-03-11 | 2021-10-22 | 佳能株式会社 | Image processing apparatus, image processing method, and program |
Also Published As
Publication number | Publication date |
---|---|
EP3417401B1 (en) | 2022-01-19 |
PL3417401T3 (en) | 2022-05-16 |
CN108885687A (en) | 2018-11-23 |
EP3417401A4 (en) | 2019-07-31 |
ES2908188T3 (en) | 2022-04-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10194866B2 (en) | Methods and apparatus for reducing artifacts in OCT angiography using machine learning techniques | |
Wood et al. | Retinal and choroidal thickness in early age-related macular degeneration | |
Girard et al. | In vivo 3-dimensional strain mapping of the optic nerve head following intraocular pressure lowering by trabeculectomy | |
US9098742B2 (en) | Image processing apparatus and image processing method | |
US9418423B2 (en) | Motion correction and normalization of features in optical coherence tomography | |
US10149610B2 (en) | Methods and systems for automatic detection and classification of ocular inflammation | |
US20160183783A1 (en) | Systems and methods for automated classification of abnormalities in optical coherence tomography images of the eye | |
Sawides et al. | The organization of the cone photoreceptor mosaic measured in the living human retina | |
US9060711B2 (en) | Automated detection of uveitis using optical coherence tomography | |
EP2779095B1 (en) | Optic disc image segmentation method | |
CA2776437C (en) | Diagnostic method and apparatus for predicting potential preserved visual acuity | |
Belghith et al. | A hierarchical framework for estimating neuroretinal rim area using 3D spectral domain optical coherence tomography (SD-OCT) optic nerve head (ONH) images of healthy and glaucoma eyes | |
WO2020137678A1 (en) | Image processing device, image processing method, and program | |
EP3417401B1 (en) | Method for reducing artifacts in oct using machine learning techniques | |
Garcia-Marin et al. | Patch-based CNN for corneal segmentation of AS-OCT images: Effect of the number of classes and image quality upon performance | |
AU2015245909B2 (en) | Method for the analysis of image data representing a three-dimensional volume of biological tissue | |
CN117099130A (en) | Method and system for detecting vascular structures | |
Girard et al. | 3D structural analysis of the optic nerve head to robustly discriminate between papilledema and optic disc drusen | |
Linderman | Quantifying the Retinal Vasculature in Development and Disease | |
WO2022232555A1 (en) | Techniques for automatically segmenting ocular imagery and predicting progression of age-related macular degeneration | |
Fabritius et al. | Automated retinal pigment epithelium identification from optical coherence tomography images |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20180816 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: KO, TONY H. Inventor name: JANG, BEN K. Inventor name: HSIAO, YI-SING Inventor name: WEI, JAY Inventor name: ZHOU, QIENYUAN Inventor name: SHARMA,UTKARSH |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
A4 | Supplementary search report drawn up and despatched |
Effective date: 20190703 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G06K 9/00 20060101AFI20190627BHEP |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R079 Ref document number: 602017052513 Country of ref document: DE Free format text: PREVIOUS MAIN CLASS: G06K0009000000 Ipc: A61B0005000000 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: A61B 6/02 20060101AFI20210817BHEP |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: A61B 5/02 20060101ALI20210831BHEP Ipc: A61B 5/00 20060101AFI20210831BHEP |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
INTG | Intention to grant announced |
Effective date: 20211007 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602017052513 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 1463306 Country of ref document: AT Kind code of ref document: T Effective date: 20220215 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: ES Ref legal event code: FG2A Ref document number: 2908188 Country of ref document: ES Kind code of ref document: T3 Effective date: 20220428 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG9D |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20220119 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 1463306 Country of ref document: AT Kind code of ref document: T Effective date: 20220119 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220119 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220119 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220119 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220519 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220419 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220119 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220119 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220419 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220119 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220420 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220119 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220119 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220519 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602017052513 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20220228 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220119 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220119 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220119 Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220119 Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20220217 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220119 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220119 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220119 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220119 |
|
26N | No opposition filed |
Effective date: 20221020 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20220228 Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20220217 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20220228 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220119 Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20220228 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20170217 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: ES Payment date: 20240301 Year of fee payment: 8 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220119 Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20220119 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20240228 Year of fee payment: 8 Ref country code: GB Payment date: 20240227 Year of fee payment: 8 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: PL Payment date: 20240201 Year of fee payment: 8 Ref country code: IT Payment date: 20240222 Year of fee payment: 8 Ref country code: FR Payment date: 20240226 Year of fee payment: 8 |