Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T3/00—Geometric image transformations in the plane of the image
  • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
  • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
  • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
  • A61B3/12—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for looking at the eye fundus, e.g. ophthalmoscopes
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T5/00—Image enhancement or restoration
  • G06T5/90—Dynamic range modification of images or parts thereof
  • G06T5/94—Dynamic range modification of images or parts thereof based on local image properties, e.g. for local contrast enhancement
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/10—Segmentation; Edge detection
  • G06T7/11—Region-based segmentation
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10072—Tomographic images
  • G06T2207/10101—Optical tomography; Optical coherence tomography [OCT]
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/30—Subject of image; Context of image processing
  • G06T2207/30004—Biomedical image processing
  • G06T2207/30041—Eye; Retina; Ophthalmic
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/30—Subject of image; Context of image processing
  • G06T2207/30004—Biomedical image processing
  • G06T2207/30101—Blood vessel; Artery; Vein; Vascular

Definitions

• the present disclosure relates to an image processing method, an image processing device, and a program.
• US Pat. No. 1,023,8281 discloses a technique for generating volume data of an eye to be examined using an optical coherence tomography. Conventionally, it has been desired to visualize blood vessels based on volume data of an eye to be examined.
• a first aspect is an image processing method performed by a processor, which includes a step of acquiring an image in which the choroid is shown, a step of performing an enhancement process to enhance the contrast of the acquired image, and a step of performing an enhancement process to enhance the contrast of the acquired image;
• This image processing method includes the steps of: performing a binarization process on the image; and extracting a region corresponding to a choroidal blood vessel in the choroid from the binarized image.
• a second aspect includes an image acquisition unit that acquires an image showing the choroid, an enhancement processing unit that performs enhancement processing to enhance the contrast of the acquired image, and binarization of the image that has been subjected to the enhancement processing.
• the image processing apparatus includes a binarization processing unit that performs processing, and a region extraction unit that extracts a region corresponding to a choroidal blood vessel in the choroid from the binarized image.
• a third aspect is a program that performs image processing, and the program includes the steps of: acquiring an image showing the choroid; performing enhancement processing to enhance the contrast of the acquired image;
• the present invention is a program that processes the steps of performing a binarization process on the image and extracting a region corresponding to a choroidal blood vessel in the choroid from the binarized image.
• FIG. 1 is a schematic configuration diagram of an ophthalmologic system according to an embodiment.
• FIG. 1 is a schematic configuration diagram of an ophthalmologic apparatus according to an embodiment.
• FIG. 2 is a schematic configuration diagram of a server.
• FIG. 2 is an explanatory diagram of functions realized by an image processing program in a CPU of a server.
• 3 is a flowchart illustrating an example of the flow of image processing by a server.
• 2 is a flowchart illustrating an example of the flow of image formation processing of choroidal blood vessels.
• 12 is a flowchart illustrating an example of the flow of first image processing by first blood vessel extraction processing.
• 12 is a flowchart illustrating an example of the flow of second image processing by second blood vessel extraction processing.
• FIG. 12 is a flowchart illustrating an example of the flow of third image processing by third blood vessel extraction processing.
• FIG. 2 is a schematic diagram showing the relationship between the eyeball and the position of the vortex vein.
• FIG. 3 is a diagram showing the relationship between OCT volume data and en-face images.
• FIG. 3 is a diagram showing an example of a fundus image of a choroidal blood vessel including vortex veins. It is a conceptual diagram of a three-dimensional image of a vortex vein.
• FIG. 3 is an explanatory diagram regarding contrast enhancement processing.
• FIG. 3 is an explanatory diagram regarding fine region connection processing.
• FIG. 3 is a diagram showing an example of a three-dimensional image of choroidal blood vessels around vortex veins.
• FIG. 3 is a diagram showing an example of a display screen using a three-dimensional image of vortex veins.
• FIG. 1 shows a schematic configuration of an ophthalmologic system 100.
• the ophthalmology system 100 includes an ophthalmology apparatus 110, a server device (hereinafter referred to as “server”) 140, and a display device (hereinafter referred to as "viewer”) 150.
• the ophthalmologic apparatus 110 acquires fundus images.
• the server 140 stores a plurality of fundus images obtained by photographing the fundus of a plurality of patients by the ophthalmological device 110 and an axial length measured by an axial length measuring device (not shown) in a manner corresponding to the patient ID. memorize it.
• the viewer 150 displays fundus images and analysis results obtained by the server 140.
• the server 140 is an example of the "image processing device" of the present disclosure.
• the ophthalmological apparatus 110, server 140, and viewer 150 are interconnected via a network 130.
• the network 130 is any network such as a LAN, WAN, the Internet, or a wide area Ethernet network.
• a LAN can be used as the network 130.
• the viewer 150 is a client in a client server system, and a plurality of viewers are connected via a network. Furthermore, a plurality of servers 140 may be connected via a network to ensure system redundancy.
• the ophthalmologic apparatus 110 has an image processing function and an image viewing function of the viewer 150, the ophthalmologic apparatus 110 can acquire fundus images, process images, and view images in a standalone state.
• the server 140 is provided with the image viewing function of the viewer 150, the configuration of the ophthalmological apparatus 110 and the server 140 enables fundus image acquisition, image processing, and image viewing.
• ophthalmological equipment inspection equipment such as visual field measurement and intraocular pressure measurement
• diagnostic support device that performs image analysis using AI (Artificial Intelligence) are connected to the ophthalmological equipment 110, server 140, and viewer via the network 130. 150.
• AI Artificial Intelligence
• SLO scanning laser ophthalmoscope
• OCT optical coherence tomography
• the horizontal direction is the "X direction”
• the vertical direction to the horizontal plane is the "Y direction” connecting the center of the pupil in the anterior segment of the eye 12 to be examined and the center of the eyeball.
• Let the direction be the "Z direction”. Therefore, the X, Y, and Z directions are perpendicular to each other.
• the ophthalmological apparatus 110 includes an imaging device 14 and a control device 16.
• the photographing device 14 includes an SLO unit 18 and an OCT unit 20, and acquires a fundus image of the eye 12 to be examined.
• the two-dimensional fundus image acquired by the SLO unit 18 will be referred to as an SLO image.
• a tomographic image, en-face image, etc. of the retina created based on OCT data acquired by the OCT unit 20 is referred to as an OCT image.
• the control device 16 has a CPU (Central Processing Unit) 16A, a RAM (Random Access Memory) 16B, a ROM (Read-Only Memory) 16C, and an input/output (I/O) port 16D. equipped with a computer ing.
• CPU Central Processing Unit
• RAM Random Access Memory
• ROM Read-Only Memory
• I/O input/output
• the control device 16 includes an input/display device 16E connected to the CPU 16A via an I/O port 16D.
• the input/display device 16E has a graphic user interface that displays an image of the eye 12 to be examined and receives various instructions from the user.
• An example of a graphic user interface is a touch panel display.
• the control device 16 also includes an image processor 17 connected to the I/O port 16D.
• the image processor 17 generates an image of the eye 12 based on the data obtained by the imaging device 14. Note that the control device 16 is connected to the network 130 via a communication interface (I/F) 16F.
• I/F communication interface
• the control device 16 of the ophthalmological device 110 includes an input/display device 16E
• the present disclosure is not limited thereto.
• the control device 16 of the ophthalmologic device 110 may not include the input/display device 16E, but may include a separate input/display device that is physically independent of the ophthalmologic device 110.
• the display device includes an image processing processor unit that operates under the control of the display control section 204 (see FIG. 4) of the CPU 16A of the control device 16.
• the image processing processor unit may display the SLO image or the like based on the image signal outputted by the display control unit 204.
• the photographing device 14 operates under the control of the CPU 16A of the control device 16.
• the imaging device 14 includes an SLO unit 18, an imaging optical system 19, and an OCT unit 20.
• the photographing optical system 19 includes an optical scanner 22 and a wide-angle optical system 30.
• the optical scanner 22 two-dimensionally scans the light emitted from the SLO unit 18 in the X direction and the Y direction.
• the optical scanner 22 may be any optical element that can deflect a light beam, and for example, a polygon mirror, a galvano mirror, or the like can be used. Alternatively, a combination thereof may be used.
• the wide-angle optical system 30 combines the light from the SLO unit 18 and the light from the OCT unit 20.
• the wide-angle optical system 30 may be a reflective optical system using a concave mirror such as an elliptical mirror, a refractive optical system using a wide-angle lens, or a catadioptric optical system combining a concave mirror and lenses.
• a wide-angle optical system using an elliptical mirror, a wide-angle lens, or the like it is possible to photograph not only the central part of the fundus but also the retina in the peripheral part of the fundus.
• the wide-angle optical system 30 allows observation in a wide field of view (FOV) 12A at the fundus.
• the FOV 12A indicates the range that can be photographed by the photographing device 14.
• FOV12A may be expressed as a viewing angle.
• the viewing angle may be defined by an internal illumination angle and an external illumination angle.
• the external irradiation angle is an irradiation angle that defines the irradiation angle of the light beam irradiated from the ophthalmological apparatus 110 to the eye 12 to be examined, with the pupil 27 as a reference.
• the internal illumination angle is an illumination angle that defines the illumination angle of the light beam irradiated to the fundus of the eye with the eyeball center O as a reference.
• the external illumination angle and the internal illumination angle have a corresponding relationship. For example, if the external illumination angle is 120 degrees, the internal illumination angle corresponds to approximately 160 degrees. In this embodiment, the internal illumination angle is 200 degrees.
• UWF-SLO fundus image an SLO fundus image obtained by photographing at an internal illumination angle of 160 degrees or more is referred to as a UWF-SLO fundus image.
• UWF is an abbreviation for UltraWide Field.
• FOV ultra-wide field of view
• the ophthalmological apparatus 110 can photograph a region 12A with an internal illumination angle of 200° using the eyeball center O of the subject's eye 12 as a reference position.
• the internal illumination angle of 200° is 110° in terms of the external illumination angle with respect to the pupil of the eyeball of the eye 12 to be examined. That is, the wide-angle optical system 30 irradiates a laser beam from the pupil with an external illumination angle of 110° and photographs a fundus region of 200° with an internal illumination angle.
• the SLO system is realized by the control device 16, the SLO unit 18, and the photographing optical system 19 shown in FIG. Since the SLO system includes the wide-angle optical system 30, it is possible to photograph the fundus with a wide FOV 12A.
• the SLO unit 18 includes a light source 40 for B light (blue light), a light source 42 for G light (green light), a light source 44 for R light (red light), and a light source 44 for IR light (infrared light (for example, near infrared light)). It includes a light source 46 and optical systems 48, 50, 52, 54, and 56 that reflect or transmit light from the light sources 40, 42, 44, and 46 and guide it into one optical path.
• Optical systems 48, 56 are mirrors, and optical systems 50, 52, 54 are beam splitters.
• the B light is reflected by the optical system 48, transmitted through the optical system 50, and reflected by the optical system 54, the G light is reflected by the optical systems 50 and 54, and the R light is transmitted through the optical systems 52 and 54.
• IR light is reflected by optical systems 52 and 56 and guided to one optical path, respectively.
• the SLO unit 18 is configured to be able to switch between a light source that emits laser light of different wavelengths or a combination of light sources that emit light, such as a mode that emits R light and G light and a mode that emits infrared light.
• a light source that emits laser light of different wavelengths or a combination of light sources that emit light
• FIG. 2 includes four light sources: a B light source 40, a G light source 42, an R light source 44, and an IR light source 46
• the present disclosure is not limited thereto.
• the SLO unit 18 may further include a white light source and emit light in various modes, such as a mode in which G light, R light, and B light are emitted, and a mode in which only white light is emitted.
• the light incident on the photographing optical system 19 from the SLO unit 18 is scanned in the X direction and the Y direction by the optical scanner 22.
• the scanning light passes through the wide-angle optical system 30 and the pupil 27 and is irradiated onto the fundus of the eye.
• the light reflected by the fundus enters the SLO unit 18 via the wide-angle optical system 30 and the optical scanner 22.
• the SLO unit 18 includes a beam splitter 64 that reflects B light and transmits light other than B light from the posterior segment (fundus) of the subject's eye 12, and G light of the light that has passed through the beam splitter 64.
• a beam splitter 58 is provided that reflects light and transmits light other than G light.
• the SLO unit 18 includes a beam splitter 60 that reflects R light among the light transmitted through the beam splitter 58 and transmits light other than the R light.
• the SLO unit 18 includes a beam splitter 62 that reflects IR light out of the light that has passed through the beam splitter 60.
• the SLO unit 18 includes a B light detection element 70 that detects the B light reflected by the beam splitter 64, a G light detection element 72 that detects the G light reflected by the beam splitter 58, and a G light detection element 72 that detects the R light reflected by the beam splitter 60. It includes an R light detection element 74 and an IR light detection element 76 that detects the IR light reflected by the beam splitter 62.
• the light incident on the SLO unit 18 via the wide-angle optical system 30 and the optical scanner 22 is reflected by the beam splitter 64 and received by the B light detection element 70.
• the beam splitter 58 is reflected by the beam splitter 58 and received by the G light detection element 72.
• the incident light is R light
• the beam splitter 58 passes through the beam splitter 58, is reflected by the beam splitter 60, and is received by the R light detection element 74.
• the incident light passes through the beam splitters 58 and 60, is reflected by the beam splitter 62, and is received by the IR light detection element 76.
• the image processor 17 operating under the control of the CPU 16A generates a UWF-SLO image using the signals detected by the B light detection element 70, the G light detection element 72, the R light detection element 74, and the IR light detection element 76. generate.
• the UWF-SLO image generated using the signal detected by the B light detection element 70 is referred to as a B-UWF-SLO image (B-color fundus image).
• the UWF-SLO image generated using the signal detected by the G light detection element 72 is referred to as a G-UWF-SLO image (G color fundus image).
• the UWF-SLO image generated using the signal detected by the R light detection element 74 is referred to as an R-UWF-SLO image (R-color fundus image).
• the UWF-SLO image generated using the signal detected by the IR light detection element 76 is referred to as an IR-UWF-SLO image (IR fundus image).
• the UWF-SLO image includes these R color fundus images, G color fundus images, B color fundus images to IR fundus images. It also includes UWF-SLO images of fluorescence.
• control device 16 controls the light sources 40, 42, and 44 so that they emit light at the same time.
• a G-color fundus image, an R-color fundus image, and a B-color fundus image whose respective positions correspond to each other are obtained.
• An RGB color fundus image is obtained from the G color fundus image, the R color fundus image, and the B color fundus image.
• the control device 16 controls the light sources 42 and 44 to emit light at the same time, and the fundus of the subject's eye 12 is simultaneously photographed using the G light and the R light, thereby creating a G-color fundus image and an R-color fundus image whose respective positions correspond to each other.
• a fundus image is obtained.
• An RG color fundus image is obtained from the G color fundus image and the R color fundus image. Further, a full-color fundus image may be generated using the G-color fundus image, the R-color fundus image, and the B-color fundus image.
• the wide-angle optical system 30 makes it possible to set the field of view (FOV) of the fundus to an ultra-wide angle, and to photograph a region beyond the equator from the posterior pole of the fundus of the eye 12 to be examined.
• FOV field of view
• the OCT system is realized by the control device 16, OCT unit 20, and imaging optical system 19 shown in FIG. Since the OCT system includes the wide-angle optical system 30, it is possible to perform OCT imaging of the peripheral part of the fundus, similar to the imaging of the SLO fundus image described above. In other words, by using the wide-angle optical system 30 that sets the viewing angle (FOV) of the fundus to an ultra-wide angle, it is possible to perform OCT imaging of a region beyond the equator 178 from the posterior pole of the fundus of the eye 12 to be examined.
• FOV viewing angle
• OCT data of structures existing in the periphery of the fundus such as vortex veins
• a tomographic image of the vortex veins and a 3D structure of the vortex veins can be obtained by image processing the OCT data.
• the OCT unit 20 includes a light source 20A, a sensor (detection element) 20B, a first optical coupler 20C, a reference optical system 20D, a collimating lens 20E, and a second optical coupler 20F.
• the light emitted from the light source 20A is branched by the first optical coupler 20C.
• One of the branched lights is made into parallel light by a collimating lens 20E as measurement light, and then enters the photographing optical system 19.
• the measurement light passes through the wide-angle optical system 30 and the pupil 27 and is irradiated onto the fundus of the eye.
• the measurement light reflected by the fundus enters the OCT unit 20 via the wide-angle optical system 30, and enters the second optical coupler 20F via the collimating lens 20E and the first optical coupler 20C.
• the other light emitted from the light source 20A and branched by the first optical coupler 20C enters the reference optical system 20D as a reference light, and then enters the second optical coupler 20F via the reference optical system 20D. do.
• the interference light is received by sensor 20B.
• the image processor 17, which operates under the control of the image processor 206 (see FIG. 4), generates OCT data detected by the sensor 20B. It is also possible for the image processor 17 to generate OCT images such as tomographic images and en-face images based on the OCT data.
• the OCT unit 20 can scan a predetermined range (for example, a rectangular range of 6 mm x 6 mm) in one OCT imaging.
• the predetermined range is not limited to 6 mm x 6 mm, but may be a square range of 12 mm x 12 mm or 23 mm x 23 mm, or a rectangular range such as 14 mm x 9 mm, 6 mm x 3.5 mm, or any rectangular range. can.
• the diameter may be in a range of 6 mm, 12 mm, 23 mm, etc.
• the ophthalmological apparatus 110 can scan an area 12A with an internal illumination angle of 200°. That is, by controlling the optical scanner 22, OCT imaging of a predetermined range including vortex veins is performed. The ophthalmologic apparatus 110 can generate OCT data through the OCT imaging.
• the ophthalmologic apparatus 110 can generate OCT images such as tomographic images (B-scan images) of the fundus including vortex veins, OCT volume data including vortex veins, and en-face images (OCT) that are cross sections of the OCT volume data.
• OCT images such as tomographic images (B-scan images) of the fundus including vortex veins, OCT volume data including vortex veins, and en-face images (OCT) that are cross sections of the OCT volume data.
• a frontal image generated based on volume data can be generated.
• the OCT image includes an OCT image of the center of the fundus (the posterior pole of the eyeball where the macula, optic disc, etc. are present).
• OCT data (or image data of an OCT image) is sent from the ophthalmological apparatus 110 to the server 140 via the communication interface 16F, and is stored in the storage device 254 described in FIG. 3.
• the light source 20A is a wavelength-swept type SS-OCT (Swept-Source OCT), but various types such as SD-OCT (Spectral-Domain OCT), TD-OCT (Time-Domain OCT), etc.
• SS-OCT Session-Coupled OCT
• SD-OCT Spectral-Domain OCT
• TD-OCT Time-Domain OCT
• the OCT system may be of any type.
• the server 140 includes a computer main body 252.
• the computer main body 252 has a CPU 262, a RAM 266, a ROM 264, and an input/output (I/O) port 268.
• a storage device 254, a display 256, a mouse 255M, a keyboard 255K, and a communication interface (I/F) 258 are connected to the input/output (I/O) port 268.
• the storage device 254 is composed of, for example, nonvolatile memory.
• Input/output (I/O) port 268 is connected to network 130 via communication interface (I/F) 258 . Accordingly, server 140 can communicate with ophthalmological device 110 and viewer 150.
• An image processing program (FIGS. 5 to 9) is stored in the ROM 264 or the storage device 254.
• the ROM 264 or the storage device 254 is an example of the "memory" of the present disclosure.
• CPU 262 is an example of a “processor” in the present disclosure.
• the image processing program is an example of the "program” of the present disclosure.
• the server 140 stores each data received from the ophthalmological device 110 in the storage device 254.
• the image processing program executed by the CPU 262 includes a display control function, an image processing function, and a processing function.
• the CPU 262 functions as the display control section 204, the image processing section 206, and the processing section 208 by executing the image processing program having each of these functions.
• the image processing unit 206 is an example of an “image acquisition unit”, an “emphasis processing unit”, and an “area extraction unit” of the present disclosure.
• FIG. 5 The image processing (image processing method) shown in FIG. 5 is realized by the CPU 262 of the server 140 executing the image processing program.
• the image processing unit 206 acquires a fundus image from the storage device 254.
• the fundus image includes data related to the vortex veins to be displayed stereoscopically based on the user's instructions.
• step S20 the image processing unit 206 acquires OCT volume data including the choroid corresponding to the fundus image from the storage device 254.
• the image processing unit 206 extracts the choroidal blood vessels based on the OCT volume data and performs image formation processing (details will be described later) of the choroidal blood vessels to generate a three-dimensional image (3D image) of the vortex vein blood vessels. Execute.
• step S40 the processing unit 208 outputs the generated three-dimensional image (3D image) of the vortex vein blood vessel, specifically, stores it in the RAM 266 or the storage device. 254, and end the image processing.
• a display screen (an example of a display screen is shown in FIG. 17, which will be described later) including a stereoscopic image of vortex veins is generated by the display control unit 204 based on the user's instructions.
• the generated display screen is output as an image signal by the processing unit 208 to the viewer 150.
• a display screen appears on the display of viewer 150.
• step S30 the image forming process of the choroidal blood vessels that generates the stereoscopic image regarding the vortex veins (VV) in step S30 will be described in detail using FIG. 6.
• FIG. 10 shows the positional relationship between the choroid 12M and the vortex veins 12V1 and V2 in the eyeball.
• the mesh pattern indicates the choroidal blood vessels of the choroid 12M. Choroidal blood vessels circulate blood throughout the choroid. Then, blood flows out of the eyeball from a plurality of (usually four to six) vortex veins present in the eye 12 to be examined.
• FIG. 10 shows an upper vortex vein 12V1 and a lower vortex vein 12V2 that exist on one side of the eyeball. Vortex veins often exist near the equator. Therefore, in order to photograph the vortex vein and the choroidal blood vessels around the vortex vein present in the eye 12 to be examined, the ophthalmologic apparatus 110 that can scan at an internal illumination angle of 200 degrees is used, for example.
• step S10 the image processing unit 206 acquires a fundus image and identifies a vortex vein (VV) to be displayed stereoscopically.
• VV vortex vein
• a UWF-SLO image is acquired from the storage device 254 as a UWF fundus image.
• the image processing unit 206 creates a choroidal blood vessel image, which is a binarized image, from the acquired UWF-SLO image. Then, the part designated by the user is specified as the vortex vein to be displayed in three dimensions.
• FIG. 12 is a fundus image of choroidal blood vessels including vortex veins.
• the fundus image shown in FIG. 12 is an example of a choroidal blood vessel image that is a binarized image created from the UWF-SLO image.
• the choroidal blood vessel image is a binarized image in which pixels corresponding to choroidal blood vessels and vortex veins are white, and pixels in other areas are black.
• FIG. 12 is an image 302 showing the presence of choroidal blood vessels connected to the vortex veins.
• Image 302 shows a case where vortex vein 310V1, which is an image of upper vortex vein 12V1 included in region 310A designated by the user, is specified as a vortex vein (VV) to be displayed stereoscopically, and a region including a choroidal blood vessel is specified. ing.
• Choroidal blood vessel images including vortex veins are composed of an R-UWF-SLO image (R-color fundus image) taken with red light (laser light with a wavelength of 630 to 660 nm) and a green light (laser light with a wavelength of 500 to 550 nm). ) is generated by image processing the image data of the G-UWF-SLO image (G color fundus image). Specifically, a choroidal blood vessel image is generated by extracting retinal blood vessels from the G-color fundus image, removing the retinal blood vessels from the R-color fundus image, and performing image processing to emphasize the choroidal blood vessels. Regarding the method of generating a choroidal blood vessel image, the disclosure of International Publication WO2019/181981 is incorporated herein by reference in its entirety.
• the position of the vortex vein to be displayed stereoscopically may be detected manually or automatically.
• the position indicated by the user may be detected by visually observing the displayed choroidal blood vessels.
• the choroidal blood vessels may be extracted from the choroidal blood vessel image, the moving direction of each choroidal blood vessel (vessel running direction) may be estimated, and the position of the vortex vein may be estimated based on the position where the choroidal blood vessels gather. .
• step S31 of FIG. 6 the image processing unit 206 extracts a region corresponding to the choroid from the OCT volume data 400 (see FIG. 11) acquired in step S20, and based on the extracted region, the image processing unit 206 extracts a region corresponding to the choroid. Extract (obtain) partial OCT volume data.
• the OCT volume data 400 is a predetermined area including a vortex vein VV, for example, 6 mm, obtained by performing OCT imaging of one of the plurality of vortex veins VV in the subject's eye using the ophthalmological apparatus 110.
• This is OCT volume data 400 of a rectangular area of 6 mm.
• N planes having different depths from the first plane f401 to the Nth plane f40N are set for the OCT volume data 400.
• the OCT volume data 400 may be obtained by performing OCT imaging of each of a plurality of vortex veins VV in the subject's eye using the ophthalmologic apparatus 110.
• OCT volume data 400D including vortex veins and choroidal blood vessels around the vortex veins will be described as an example of OCT volume data 400D.
• the choroidal blood vessels refer to the vortex veins and the choroidal blood vessels surrounding the vortex veins.
• the image processing unit 206 extracts retinal pigment from OCT volume data 400 of a region where choroidal blood vessels exist, from OCT volume data scanned to include vortex veins and choroidal blood vessels around the vortex veins.
• OCT volume data 400D of the region below the epithelial cell layer 400R (Retinal Pigment Epithelium, hereinafter referred to as RPE layer) is extracted.
• the image processing unit 206 performs image processing on the OCT volume data 400 to identify the boundary surfaces of each layer, thereby identifying the RPE layer 400R.
• the highest brightness layer in the OCT volume data may be specified as the RPE layer 400R.
• the image processing unit 206 extracts pixel data of a region of the choroid in a predetermined range deeper than the RPE layer 400R (a predetermined range farther than the RPE layer when viewed from the center of the eyeball) as OCT volume data 400D. Since OCT volume data in deep regions may not be uniform, the image processing unit 206 analyzes the region between the RPE layer 400R and the bottom surface 400E obtained by the above image processing to identify the boundary surface, as shown in FIG. may be extracted as OCT volume data 400D. A region of the choroid in a predetermined range deeper than the RPE layer 400R is an example of the "choroidal portion" of the present disclosure.
• OCT volume data 400D for generating a three-dimensional image of the choroidal blood vessels is extracted.
• the image processing unit 206 executes a first blood vessel extraction process (ampulla extraction) using the OCT volume data 400D.
• the first blood vessel extraction process is a process of extracting a choroidal blood vessel (hereinafter referred to as the ampulla) that forms the ampullae, which is the first blood vessel.
• the ampulla a choroidal blood vessel
• first image processing shown in FIG. 7 is executed.
• step S322 the image processing unit 206 performs a binarization process on the OCT volume data 400D as preprocessing for the first blood vessel extraction process (ampulla extraction). Specifically, by setting the binarization threshold to a predetermined threshold that leaves the vascular ampulla, the OCT volume data D becomes black pixels for the vascular ampullae and white pixels for the other parts. .
• step S324 the image processing unit 206 executes noise removal processing to delete noise regions in the binarized OCT volume data 400D.
• the image processing unit 206 extracts the first choroidal blood vessel, which is the ampulla, from the OCT volume data by deleting the noise region in the binarized OCT volume data 400D.
• the noise region may be an isolated region of black pixels or a region corresponding to a small blood vessel.
• the image processing unit 206 performs median filtering, opening processing, contraction processing, etc. on the binarized OCT volume data 400D to delete the noise regions.
• step S326 the image processing unit 206 performs segmentation processing (such as dynamic contour, graph cut, or U-net) on the OCT volume data from which the noise region has been removed in order to smooth the surface of the extracted ampullae. image processing).
• segmentation processing such as dynamic contour, graph cut, or U-net
• This step S326 can be omitted.
• segmentation refers to image processing that performs binarization processing to separate the background and foreground of an image to be analyzed.
• a stereoscopic image 680B of blood vessels in the ampullae shown in FIG. 13 is generated.
• the image data of the stereoscopic image 680B of the blood vessel in the ampulla is stored in the RAM 266 by the processing unit 208.
• the blood vessel in the ampullae shown in FIG. 13 is an example of the "first choroidal blood vessel” of the present disclosure
• the stereoscopic image 680B of the blood vessel in the ampullae is an example of the "first stereoscopic image” of the present disclosure.
• the image processing unit 206 executes a second blood vessel extraction process (thick blood vessel extraction) using the OCT volume data 400D in step S33 shown in FIG.
• the second blood vessel extraction process extracts choroidal blood vessels (hereinafter referred to as thick blood vessels) that exceed a predetermined threshold, that is, a predetermined diameter, which are thick linear second blood vessels that extend from the ampullae. It is processing.
• a predetermined threshold that is, a predetermined diameter
• the thick blood vessels mainly indicate blood vessels arranged in the Haller layer.
• second image processing shown in FIG. 8 is executed.
• the predetermined threshold value (that is, the predetermined diameter) can be a predetermined value such that blood vessels with a diameter of several hundred ⁇ m are left as thick blood vessels.
• the threshold value determined to leave small blood vessels which will be described later, may be a value less than several 100 ⁇ m in diameter, which is determined to be left as large blood vessels, or a value smaller than the predetermined value to be left as large blood vessels. Good too.
• the image processing unit 206 first performs image processing to perform preprocessing on the OCT volume data 400D in step S331 shown in FIG.
• An example of preprocessing includes blurring processing that performs noise removal and the like.
• the blurring process can be performed by removing the influence of speckle noise and extracting a linear blood vessel that accurately reflects the blood vessel shape.
• Speckle noise processing includes Gaussian blur processing and the like.
• the image processing unit 206 performs line extraction processing (thick linear blood vessel extraction) on the preprocessed OCT volume data 400D, thereby extracting thick linear parts from the OCT volume data 400D.
• line extraction processing thin linear blood vessel extraction
• the image processing unit 206 performs image processing using, for example, an eigenvalue filter, a Gabor filter, etc., and extracts a linear blood vessel region from the OCT volume data 400D.
• the blood vessel region is a low-luminance pixel (darkish pixel), and an area in which low-luminance pixels are continuous remains as a blood vessel portion.
• step S333 the image processing unit 206 performs a binarization process on the OCT volume data 400D. Specifically, by setting the binarization threshold to a predetermined threshold that leaves large blood vessels, the OCT volume data D has large blood vessels as black pixels and other parts as white pixels.
• step S334 the image processing unit 206 performs processing for deleting isolated regions that are not connected to surrounding blood vessels, median filter processing, for the extracted and binarized linear blood vessel region.
• Image processing such as opening processing and shrinking processing is performed to remove discrete minute regions.
• the processing unit 208 By performing the second blood vessel extraction process described above, only the region of the large blood vessel remains from the OCT volume data 400D, and a stereoscopic image 680L of the large blood vessel shown in FIG. 13 is generated.
• the image data of the three-dimensional image 680L of the large blood vessel is stored in the RAM 266 by the processing unit 208.
• FIG. 16 shows an example of a three-dimensional image of the choroidal blood vessels around the vortex vein VV obtained by the above-described image processing (FIG. 5).
• FIG. 16 shows an example of a three-dimensional image of the choroidal blood vessels around the vortex vein VV obtained by the above-described image processing (FIG. 5).
• the image processing unit 206 aligns the stereoscopic image 680B of the ampullae and the stereoscopic image 680L of the linear blood vessel, and calculates the logical sum of both images.
• the image 680L and the 3D image 680B of the ampulla are combined. Thereby, it is possible to generate a stereoscopic image 680M (FIG. 13) of choroidal blood vessels including vortex veins, which are large blood vessels.
• a stereoscopic image 680M FOG. 13
• the present disclosure provides processing for extracting choroidal blood vessels (hereinafter referred to as thin blood vessels) having a diameter equal to or less than a predetermined threshold value, that is, a thin linear third blood vessel extending from the ampullae. include.
• the image processing unit 206 executes the third blood vessel extraction process (thin blood vessel extraction) using the OCT volume data 400D in step S34 shown in FIG.
• the third blood vessel extraction process extracts choroidal blood vessels (hereinafter referred to as thin blood vessels) that are thin linear third blood vessels that extend from the ampullae and have a diameter equal to or less than a predetermined threshold value, that is, a predetermined diameter. It is processing.
• a linear third blood vessel extending from the ampullae is extracted.
• the thin blood vessels mainly refer to blood vessels located in the Sattler layer.
• third image processing shown in FIG. 9 is executed.
• the image processing unit 206 performs preprocessing for small blood vessels, including first preprocessing and second preprocessing, on the OCT volume data 400D in the process of extracting a third blood vessel that is a small blood vessel.
• image processing is performed to perform first preprocessing on the OCT volume data 400D.
• An example of the first preprocessing includes a blurring process similar to step S331 described above, as an example of a process for removing noise.
• the image processing unit 206 performs image processing to perform the second preprocessing on the OCT volume data 400D that has been subjected to the first preprocessing.
• Contrast enhancement processing is applied as an example of the second preprocessing. Contrast enhancement processing works effectively when extracting small blood vessels. Contrast enhancement processing is processing that increases the contrast of an image compared to before processing, that is, increases the difference between brightness and darkness. For example, the difference between the maximum value and the minimum value of the degree of brightness (for example, luminance) is increased by a predetermined value from the difference value before processing. The predetermined value can be set as appropriate.
• the contrast enhancement process is an example of the "enhancement process" of the present disclosure.
• FIG. 14 shows an example of an image related to contrast enhancement processing applied to the second preprocessing.
• images of small blood vessels are shown as white images.
• the image G10 containing thin blood vessels in the OCT volume data 400D has a lower contrast than the image containing thick blood vessels, and when binarized after noise removal, the thin blood vessels may not be depicted, as shown in image G11. Therefore, when image G10 containing small blood vessels is subjected to contrast enhancement processing (image G12) and binarized, the small blood vessels appear as a continuous line as shown in image G13. This makes it possible to reduce separation.
• the image processing unit 206 performs image processing using, for example, an eigenvalue filter, a Gabor filter, etc., and can extract a region of a linear blood vessel, which is a thin blood vessel, from the OCT volume data 400D. be.
• step S343 shown in FIG. 9 the image processing unit 206 performs image processing to perform binarization processing on the OCT volume data 400D that has been subjected to contrast enhancement processing. Specifically, by setting the binarization threshold to a predetermined threshold that leaves small blood vessels, the OCT volume data D has small blood vessels as black pixels and other parts as white pixels.
• step S344 the image processing unit 206 removes discrete minute regions from the binarized image (region including small blood vessels), similarly to step S333 (FIG. 8).
• image processing is performed to remove speckle noise and isolated areas separated by a predetermined distance that are estimated not to be continuous with surrounding blood vessels, thereby removing discrete minute areas.
• the removal of a minute area can be applied by removing an area smaller than a predetermined area. Further, it is also applicable to remove a region having a predetermined shape as a minute region.
• the image processing unit 206 performs micro region connection processing on the OCT volume data 400D from which the micro regions have been removed as post-processing, so that the thin linear portions of the OCT volume data 400D are A third choroidal blood vessel, which is a blood vessel, is extracted. Specifically, the image processing unit 206 performs image processing using, for example, morphological processing such as closing processing, and connects discretely detected small blood vessels, thereby extracting small blood vessels from the OCT volume data 400D. A third choroidal blood vessel, which is a blood vessel, is extracted. Specifically, a third choroidal blood vessel within a predetermined distance is connected.
• the fine region connection process is an example of the "connection process" of the present disclosure.
• FIG. 15 shows an example of an image related to the fine region connection process.
• an image of a small blood vessel is shown as a white image.
• Small blood vessels may have greater curvature than larger blood vessels.
• the line extraction process step S332 shown in FIG. 8 is performed on an image including a thin blood vessel with a larger curvature than the thick blood vessel, the line structure may not be extracted. Therefore, when the image G20 including the small blood vessels in the OCT volume data 400D is binarized, as shown in the image G21, parts of the small blood vessels with large curvature may not be depicted. Therefore, by performing fine region connection processing on image G21, even small blood vessels with large curvature parts appear as continuous lines, as shown in image G22. This makes it possible to reduce separation.
• step S346 the image processing unit 206 performs segmentation processing (such as dynamic contour, graph cut, or U-net) on the OCT volume data to which the fine regions are connected, in order to smooth the surface of the extracted thin blood vessels.
• segmentation processing such as dynamic contour, graph cut, or U-net
• the linear blood vessel which is a thin blood vessel shown in FIG. 16, is an example of the "third choroidal blood vessel" of the present disclosure
• the three-dimensional image 681S of the thin blood vessel is an example of the "third three-dimensional image” of the present disclosure. .
• steps S32, S33, and S34 is not limited to the processing order described above, and any one of the processing may be performed first, or may be performed in parallel at the same time.
• step S35 the image processing unit 206 reads out a stereoscopic image of the ampullae, a stereoscopic image of a large blood vessel, and a stereoscopic image of a small blood vessel from the RAM 266. Then, by aligning these three-dimensional images and calculating the logical sum of each image, a three-dimensional image of the ampullae, a three-dimensional image of a large blood vessel, and a three-dimensional image of a small blood vessel are synthesized. As a result, a stereoscopic image 681M (see FIG. 16) of choroidal blood vessels including vortex veins is generated.
• the image data of the stereoscopic image 681M is stored in the RAM 266 or the storage device 254 by the processing unit 208.
• the stereoscopic image 681M of the choroidal blood vessels including the vortex veins is an example of the "stereoscopic image of the choroidal blood vessels" of the present disclosure.
• the display screen is generated by the display control unit 204 of the server 140 based on a user's instruction, and is output by the processing unit 208 to the viewer 150 as an image signal.
• Viewer 150 displays a display screen on the display based on the image signal.
• a display screen 500A is shown. As shown in FIG. 17, display screen 500A has an information area 502 and an image display area 504A. Image display area 504A includes a comment field 506 that displays the patient's treatment history.
• the information area 502 includes a patient ID display field 512, a patient name display field 514, an age display field 516, an acuity display field 518, a right eye/left eye display field 520, and an axial length display field 522.
• the viewer 150 displays information in each display area from the patient ID display field 512 to the axial length display field 522 based on the information received from the server 140.
• the image display area 504A is an area that mainly displays images of the eye to be examined.
• the image display area 504A is provided with the following display fields, specifically including a UWF fundus image display field 542 and a choroidal blood vessel stereoscopic image display field 548.
• an OCT volume data conceptual diagram display field and a tomographic image display field 546 can be displayed in a superimposed manner on the image display area 504A.
• the comment field 506 included in the image display area 504A functions as a comment field in which the patient's treatment history can be displayed, and the results of observation and diagnosis by the ophthalmologist, who is the user, can be arbitrarily input.
• a UWF-SLO fundus image 542B captured by the ophthalmologic apparatus 110 of the fundus of the eye to be examined is displayed.
• a range 542A indicating the position where the OCT volume data was acquired is displayed in a superimposed manner. If there is a plurality of OCT volume data associated with the UWF-SLO image, the plurality of ranges may be displayed in a superimposed manner, and the user may select one position from the plurality of ranges.
• FIG. 17 shows that the range including the vortex vein in the upper right corner of the UWF-SLO image was scanned.
• a stereoscopic image (3D image) 548B of the choroidal blood vessel obtained by image processing the OCT volume data is displayed in the choroidal blood vessel stereoscopic image display field 548.
• the stereoscopic image 548B can be rotated around three axes by user operation.
• the stereoscopic image 548B of the choroidal blood vessels includes an image of the second choroidal blood vessel (stereoscopic image of a large blood vessel) extending from the ampullae 548X and an image of the third choroidal blood vessel (a stereoscopic image of a thin blood vessel) in different display formats. Can be displayed.
• a solid line represents a stereoscopic image 548L of a large blood vessel extending from the ampullae 548X
• a dotted line represents a stereoscopic image 548S of a thin blood vessel.
• the stereoscopic image 548L of a large blood vessel and the stereoscopic image 548S of a thin blood vessel may be displayed in different colors, or the background (filling) of the images may be different.
• FIG. 17 shows an example in which a layer boundary 548T is displayed with a long dotted line. This boundary 548T can be used as a guide to confirm the Hara layer and the Satra layer.
• a three-dimensional image of choroidal blood vessels including large blood vessels and small blood vessels can be confirmed.
• the vortex veins and surrounding choroidal vessels, including large and small blood vessels can be displayed in a three-dimensional image, allowing the user to obtain more information for diagnosis. becomes.
• the image display area 504A it is possible to grasp the position of OCT volume data on the UWF-SLO image.
• the cross section of the stereoscopic image can be arbitrarily selected, and by displaying the tomographic image, the user can obtain detailed information on the choroidal blood vessels.
• the three-dimensional display of the choroidal blood vessels can be performed without using OCT-A (OCT-angiography). It becomes possible to generate a three-dimensional image of a choroidal blood vessel without performing complicated calculation-intensive processing such as taking a difference between OCT volume data and obtaining motion contrast.
• OCT-A requires OCT volume data to be obtained multiple times at different times in order to obtain a difference, but in this embodiment, choroidal blood vessels are detected based on one OCT volume data without performing motion contrast extraction processing. can generate 3D images.
• choroidal blood vessels including vortex veins and surrounding large and small blood vessels are extracted based on OCT volume data including the choroid, and a three-dimensional image of each choroidal blood vessel is generated. , it becomes possible to three-dimensionally visualize the choroid, including large and small blood vessels.
• a three-dimensional image of choroidal blood vessels including large blood vessels and small blood vessels is generated based on OCT volume data without using OCT-A (OCT-angiography). Therefore, in this embodiment, it is possible to generate a three-dimensional image of choroidal blood vessels including large blood vessels and small blood vessels without performing complex calculation-intensive processing of taking differences between OCT volume data and extracting motion contrast. , the amount of calculation can be reduced.
• the image processing (FIG. 5) is executed by the server 140, but the present disclosure is not limited to this, and the additional image processing further provided in the ophthalmological apparatus 110, viewer 150, or network It may be performed by the device.
• each component may exist as long as there is no contradiction.
• image processing is realized by a software configuration using a computer, but the present disclosure is not limited to this, and at least a part of the processing can be realized by a hardware configuration. It's okay.
• a CPU is used as an example of a general-purpose processor, but a processor refers to a processor in a broad sense, and includes a general-purpose processor (for example, CPU: Central Processing Unit, etc.) and a dedicated processor ( For example, GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Ar ray, programmable logic device, etc.). Therefore, image processing may be performed only by the hardware configuration, or part of the image processing may be performed by the software configuration, and the remaining processing may be performed by the hardware configuration. It's okay.
• the operation of the processor described above may be performed not only by one processor, but also by multiple processors working together, or by multiple processors located at physically separate locations working together. It may be something that is done.
• a program in which the above-described processes are written in code that can be processed by a computer may be stored and distributed in a storage medium such as an optical disk.
• the present disclosure includes cases in which image processing is implemented by a software configuration using a computer and cases in which it is not implemented, and thus includes the following techniques.
• An image processing device comprising:
• the acquisition unit acquiring OCT volume data including the choroid; a generation unit extracting choroidal blood vessels exceeding a predetermined diameter and choroidal blood vessels having a diameter equal to or less than the predetermined diameter based on the OCT volume data, and generating a three-dimensional image of the choroidal blood vessels; image processing methods including;
• the image processing unit 206 is an example of an “acquisition unit” and a “generation unit” of the present disclosure. Based on the above disclosure, the following technology is proposed.
• a computer program product for image processing comprises a computer readable storage medium that is not itself a transitory signal;
• a program is stored in the computer readable storage medium, The program is to the processor, acquiring OCT volume data including the choroid; Based on the OCT volume data, extracting choroidal blood vessels exceeding a predetermined diameter and choroidal blood vessels having a diameter smaller than the predetermined diameter, and generating a three-dimensional image of the choroidal blood vessels; to process, computer program product.
• Server 140 is an example of a "computer program product" of this disclosure.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Life Sciences & Earth Sciences (AREA)
• Health & Medical Sciences (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Molecular Biology (AREA)
• Biophysics (AREA)
• Ophthalmology & Optometry (AREA)
• Biomedical Technology (AREA)
• Heart & Thoracic Surgery (AREA)
• Medical Informatics (AREA)
• Surgery (AREA)
• Animal Behavior & Ethology (AREA)
• General Health & Medical Sciences (AREA)
• Public Health (AREA)
• Veterinary Medicine (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Eye Examination Apparatus (AREA)

Priority Applications (4)

Applications Claiming Priority (2)

Related Child Applications (1)

Publications (1)

Family

ID=88329639

Family Applications (1)

Country Status (4)

Citations (5)

• 2023
  • 2023-04-06 JP JP2024514932A patent/JP7758171B2/ja active Active
  • 2023-04-06 WO PCT/JP2023/014303 patent/WO2023199847A1/ja not_active Ceased
  • 2023-04-06 CN CN202380033100.1A patent/CN118984672A/zh active Pending
• 2024
  • 2024-10-09 US US18/910,779 patent/US20250029222A1/en active Pending
• 2025
  • 2025-10-07 JP JP2025169700A patent/JP2026010021A/ja active Pending

Patent Citations (5)

Also Published As

Similar Documents

Legal Events