JP7419946B2 - Image processing method, image processing device, and image processing program - Google Patents

Description

The present invention relates to an image processing method, an image processing device, and an image processing program for detecting vortex veins suspected of being related to retinal lesions from fundus images.

It is desired to analyze vortex veins from wide-angle fundus images.

US Patent No. 8356901

The image processing method of the first aspect of the technology of the present disclosure is image processing performed by a processor, which includes the steps of: detecting a predetermined region of the fundus from a choroidal blood vessel image; and detecting a plurality of vortex veins from the choroidal blood vessel image. and identifying a first vortex vein related to the predetermined region among the plurality of vortex veins detected from the choroidal blood vessel image.

An image processing device according to a second aspect of the technology of the present disclosure includes a memory and a processor connected to the memory, the processor detects a predetermined region of the fundus from a choroidal blood vessel image, and detects a vortex from the choroidal blood vessel image. Veins are detected, and among the vortex veins detected from the choroidal blood vessel image, a first vortex vein related to the predetermined region is detected.

A program according to a third aspect of the technology of the present disclosure causes a computer to detect a predetermined region of the fundus from a choroidal blood vessel image, detect vortex veins from the choroidal blood vessel image, and detect vortex veins detected from the choroidal blood vessel image. Among them, detecting a first vortex vein related to the predetermined area is performed.

FIG. 1 is a schematic configuration diagram of an ophthalmologic system according to the present embodiment. FIG. 1 is a schematic configuration diagram of an ophthalmologic apparatus according to the present embodiment. FIG. 2 is a schematic configuration diagram of a server. FIG. 2 is an explanatory diagram of functions realized by an image processing program in a CPU of a server. FIG. 2 is an explanatory diagram of the equator of the eyeball. FIG. 2 is a schematic diagram showing a UWF-SLO image obtained by photographing a wide area of the fundus of the eye. FIG. 2 is a schematic diagram showing the positional relationship between the choroid and vortex veins in the eyeball. 5 is a flowchart showing image processing by the server. 7 is a flowchart showing the data analysis process of step 606 in FIG. 6. FIG. 3 is a schematic diagram showing a first vortex vein that has vascular connectivity with a region of interest indicating a lesion in a choroidal blood vessel image. This is an image showing the connectivity between the region of interest indicating the lesion and the first vortex vein. FIG. 7 is an explanatory diagram showing an example of measuring the diameter of a blood vessel that intersects with an analysis circle centered on a vortex vein in a choroidal blood vessel image. It is an enlarged view of an analytical circle. 1 is a schematic diagram showing a display screen displayed on a display of a viewer; FIG. FIG. 2 is an example of a display screen displaying the average position of vortex veins in a normal eye. This is an example of a display screen when the coordinates of the fundus image are transformed so that the centers of the first vortex vein and the second vortex vein become the optic disc. 3 is an example of a choroidal blood vessel image displayed in an image display area of a display screen. It is a modification of the choroidal blood vessel image displayed in the image display area of the display screen.

Hereinafter, an ophthalmologic system 100 according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 shows a schematic configuration of an ophthalmologic system 100. As shown in FIG. 1, the ophthalmology system 100 includes an ophthalmology apparatus 110, a server device (hereinafter referred to as "server") 140, and a display device (hereinafter referred to as "viewer") 150. The ophthalmologic apparatus 110 acquires fundus images. The server 140 stores a plurality of fundus images obtained by photographing the fundus of a plurality of patients using the ophthalmological device 110 and the axial length separately measured by an axial length measuring device (not shown), in correspondence with the patient ID. and memorize it. The viewer 150 displays fundus images and analysis results obtained by the server 140.

Ophthalmologic apparatus 110, server 140, and viewer 150 are interconnected via network 130. The viewer 150 is a client in a client server system, and a plurality of viewers 150 are connected via a network. Furthermore, a plurality of servers 140 may be connected via a network to ensure system redundancy. Alternatively, if the ophthalmologic apparatus 110 has an image processing function and an image viewing function of the viewer 150, the ophthalmologic apparatus 110 can acquire fundus images, process images, and view images in a standalone state. Further, if the server 140 is provided with the image viewing function of the viewer 150, the configuration of the ophthalmological apparatus 110 and the server 140 enables fundus image acquisition, image processing, and image viewing.

Note that other ophthalmological equipment (inspection equipment such as visual field measurement and intraocular pressure measurement) and a diagnostic support device that performs image analysis using AI (Artificial Intelligence) are connected to the ophthalmological apparatus 110, server 140, and viewer via the network 130. 150.

Next, the configuration of the ophthalmologic apparatus 110 will be described with reference to FIG. 2.

For convenience of explanation, a scanning laser ophthalmoscope will be referred to as an "SLO". Moreover, optical coherence tomography (Optical Coherence Tomography) is called "OCT".

Note that when the ophthalmological device 110 is installed on a horizontal plane, the horizontal direction is the "X direction", and the vertical direction to the horizontal plane is the "Y direction", connecting the center of the pupil in the anterior segment of the eye 12 to be examined and the center of the eyeball. Let the direction be the "Z direction". Therefore, the X, Y, and Z directions are perpendicular to each other.

Ophthalmological apparatus 110 includes an imaging device 14 and a control device 16. The photographing device 14 includes an SLO unit 18 and an OCT unit 20, and acquires a fundus image of the fundus of the eye 12 to be examined. Hereinafter, the two-dimensional fundus image acquired by the SLO unit 18 will be referred to as an SLO image. Furthermore, a tomographic image, en-face image, etc. of the retina created based on OCT data acquired by the OCT unit 20 is referred to as an OCT image. The OCT image corresponds to a "tomographic image" in the technology of the present disclosure.

The control device 16 includes a computer having a CPU (Central Processing Unit) 16A, a RAM (Random Access Memory) 16B, a ROM (Read-Only memory) 16C, and an input/output (I/O) port 16D. ing.

Control device 16 includes an input/display device 16E connected to CPU 16A via I/O port 16D. The input/display device 16E has a graphic user interface that displays an image of the eye 12 to be examined and receives various instructions from the user. An example of a graphic user interface is a touch panel display.

The control device 16 also includes an image processing device 17 connected to an I/O port 16D. The image processing device 17 generates an image of the eye 12 based on the data obtained by the imaging device 14 . Note that the control device 16 is connected to the network 130 via a communication interface 16F.

As mentioned above, although in FIG. 2 the control device 16 of the ophthalmological device 110 includes an input/display device 16E, the techniques of the present disclosure are not limited thereto. For example, the control device 16 of the ophthalmologic device 110 may not include the input/display device 16E, but may include a separate input/display device that is physically independent of the ophthalmologic device 110. In this case, the display device includes an image processing processor unit that operates under the control of the display control section 204 of the CPU 16A of the control device 16. The image processing processor unit may display the SLO image or the like based on the image signal outputted by the display control unit 204.

The photographing device 14 operates under the control of the CPU 16A of the control device 16. The imaging device 14 includes an SLO unit 18, an imaging optical system 19, and an OCT unit 20. The photographing optical system 19 includes an optical scanner 22 and a wide-angle optical system 30.

The optical scanner 22 two-dimensionally scans the light emitted from the SLO unit 18 in the X direction and the Y direction. The optical scanner 22 may be any optical element that can deflect a light beam, and for example, a polygon mirror, a galvano mirror, or the like can be used. Alternatively, a combination thereof may be used.

The wide-angle optical system 30 combines the light from the SLO unit 18 and the light from the OCT unit 20.

Note that the wide-angle optical system 30 may be a reflective optical system using a concave mirror such as an elliptical mirror, a refractive optical system using a wide-angle lens, or a catadioptric optical system combining a concave mirror and a lens. By using a wide-angle optical system using an elliptical mirror, a wide-angle lens, or the like, it is possible to photograph not only the central part of the fundus but also the retina in the peripheral part of the fundus.

When using a system including an elliptical mirror, a configuration using a system using an elliptical mirror described in International Publication WO2016/103484 or International Publication WO2016/103489 may be used. Each of the disclosures of International Publication WO2016/103484 and International Publication WO2016/103489 is incorporated herein by reference in its entirety.

The wide-angle optical system 30 realizes observation in a wide field of view (FOV) 12A at the fundus. The FOV 12A indicates the range that can be photographed by the photographing device 14. FOV12A may be expressed as a viewing angle. In this embodiment, the viewing angle may be defined by an internal illumination angle and an external illumination angle. The external irradiation angle is an irradiation angle that defines the irradiation angle of the light beam irradiated from the ophthalmological apparatus 110 to the eye 12 to be examined, with the pupil 27 as a reference. Further, the internal illumination angle is an illumination angle that defines the illumination angle of the light beam irradiated to the fundus F with the eyeball center O as a reference. The external illumination angle and the internal illumination angle have a corresponding relationship. For example, if the external illumination angle is 120 degrees, the internal illumination angle corresponds to approximately 160 degrees. In this embodiment, the internal illumination angle is 200 degrees.

Here, an SLO fundus image obtained by photographing at an internal illumination angle of 160 degrees or more is referred to as a UWF-SLO fundus image. Note that UWF is an abbreviation for UltraWide Field. The wide-angle optical system 30, which has an ultra-wide field of view (FOV) of the fundus, can photograph the region beyond the equator from the posterior pole of the fundus of the eye 12 to be examined, and can capture images of peripheral areas of the fundus such as vortex veins. You can take pictures of existing structures.

The SLO system is realized by the control device 16, the SLO unit 18, and the photographing optical system 19 shown in FIG. Since the SLO system includes the wide-angle optical system 30, it is possible to photograph the fundus with a wide FOV 12A.

The SLO unit 18 includes a B (blue light) light source 40, a G light (green light) light source 42, an R light (red light) light source 44, and an IR light (infrared light (for example, near infrared light)) light source. 46, and optical systems 48, 50, 52, 54, 56 that reflect or transmit light from the light sources 40, 42, 44, 46 and guide it to one optical path. Optical systems 48, 56 are mirrors, and optical systems 50, 52, 54 are beam splitters. The B light is reflected by the optical system 48, transmitted through the optical system 50, and reflected by the optical system 54, the G light is reflected by the optical systems 50 and 54, and the R light is transmitted through the optical systems 52 and 54. , IR light is reflected by optical systems 52 and 56 and guided to one optical path, respectively.

The SLO unit 18 is configured to be able to switch between a light source that emits laser light of different wavelengths or a combination of light sources that emit laser light, such as a mode that emits R light and G light and a mode that emits infrared light. Although the example shown in FIG. 2 includes four light sources: a B light source 40, a G light source 42, an R light source 44, and an IR light source 46, the technology of the present disclosure is not limited thereto. For example, the SLO unit 18 may further include a white light source and emit light in various modes, such as a mode that emits G light, R light, and B light, or a mode that emits only white light. good.

The light incident on the photographing optical system 19 from the SLO unit 18 is scanned in the X direction and the Y direction by the optical scanner 22. The scanning light passes through the wide-angle optical system 30 and the pupil 27 and is irradiated onto the fundus of the eye. The light reflected by the fundus enters the SLO unit 18 via the wide-angle optical system 30 and the optical scanner 22.

The SLO unit 18 includes a beam splitter 64 that reflects B light and transmits light other than B light among the light from the posterior segment (fundus) of the subject's eye 12, and a beam splitter 64 that reflects G light among the light that has passed through the beam splitter 64. A beam splitter 58 is provided that reflects light and transmits light other than G light. The SLO unit 18 includes a beam splitter 60 that reflects R light among the light transmitted through the beam splitter 58 and transmits light other than the R light. The SLO unit 18 includes a beam splitter 62 that reflects IR light out of the light that has passed through the beam splitter 60. The SLO unit 18 includes a B light detection element 70 that detects the B light reflected by the beam splitter 64, a G light detection element 72 that detects the G light reflected by the beam splitter 58, and a G light detection element 72 that detects the R light reflected by the beam splitter 60. It includes an R light detection element 74 and an IR light detection element 76 that detects the IR light reflected by the beam splitter 62.

In the case of B light, the light incident on the SLO unit 18 via the wide-angle optical system 30 and the optical scanner 22 (reflected light reflected by the fundus) is reflected by the beam splitter 64 and received by the B light detection element 70. In the case of G light, it is reflected by the beam splitter 58 and received by the G light detection element 72. If the incident light is R light, it passes through the beam splitter 58, is reflected by the beam splitter 60, and is received by the R light detection element 74. In the case of IR light, the incident light passes through the beam splitters 58 and 60, is reflected by the beam splitter 62, and is received by the IR light detection element 76. The image processing device 17 operating under the control of the CPU 16A generates a UWF-SLO image using the signals detected by the B light detection element 70, the G light detection element 72, the R light detection element 74, and the IR light detection element 76. generate.

Further, the control device 16 controls the light sources 40, 42, and 44 so that they emit light simultaneously. By simultaneously photographing the fundus of the subject's eye 12 using B light, G light, and R light, a G-color fundus image, an R-color fundus image, and a B-color fundus image whose respective positions correspond to each other are obtained. An RGB color fundus image is obtained from the G color fundus image, the R color fundus image, and the B color fundus image. The control device 16 controls the light sources 42 and 44 to emit light at the same time, and the fundus of the subject's eye 12 is simultaneously photographed using the G light and the R light, thereby creating a G-color fundus image and an R-color fundus image whose respective positions correspond to each other. A fundus image is obtained. An RG color fundus image is obtained from the G color fundus image and the R color fundus image.

The wide-angle optical system 30 makes it possible to set the field of view (FOV) of the fundus to an ultra-wide angle, and to photograph a region beyond the equator from the posterior pole of the fundus of the eye 12 to be examined.

The equatorial portion 178 will be explained using FIG. 5A. The eyeball (tested eye 12) is a spherical structure with a diameter of about 24 mm and an eyeball center 170. The straight line connecting the anterior pole 175 and the posterior pole 176 is called an eyeball axis 172, and the line where a plane perpendicular to the eyeball axis 172 intersects with the eyeball surface is called a latitude, the largest of which is the equator 174. The portion of the retina or choroid that corresponds to the position of the equator 174 is defined as an equator section 178.

The ophthalmological apparatus 110 can image an area with an internal illumination angle of 200° using the eyeball center 170 of the eye 12 as a reference position. Note that the internal illumination angle of 200° is 110° in terms of the external illumination angle with respect to the pupil of the eyeball of the eye 12 to be examined. That is, the wide-angle optical system 30 irradiates a laser beam from the pupil with an external illumination angle of 110° and photographs a fundus region of 200° with an internal illumination angle.

FIG. 5B shows a UWF-SLO image 179 taken by the ophthalmological apparatus 110 that can scan with an internal illumination angle of 200°. As shown in FIG. 5B, the equatorial portion 178 corresponds to an internal illumination angle of 180°, and in the UWF-SLO image 179, a location indicated by a dotted line 178a corresponds to the equatorial portion 178. In this way, the ophthalmologic apparatus 110 can image the fundus region from the posterior pole region beyond the equator region 178.

FIG. 5C is a diagram showing the positional relationship between the choroid 12M and the vortex veins 12V1 and V2 in the eyeball. In FIG. 5C, the mesh pattern indicates the choroidal blood vessels of the choroid 12M. Choroidal blood vessels circulate blood throughout the choroid. Then, blood flows out of the eyeball from a plurality of (usually four to six) vortex veins present in the eye 12 to be examined. In FIG. 5C, the upper vortex vein V1 and the lower vortex vein V2 that are present on one side of the eyeball are shown. Vortex veins often exist near the equator 178. Therefore, in order to photograph the vortex veins existing in the eye 12 to be examined and the choroidal blood vessels around the vortex veins, the above-mentioned ophthalmologic apparatus 110 capable of scanning at an internal illumination angle of 200 degrees is used.

The OCT system is realized by the control device 16, OCT unit 20, and imaging optical system 19 shown in FIG. Since the OCT system includes the wide-angle optical system 30, it is possible to perform OCT imaging of the peripheral part of the fundus, similar to the imaging of the SLO fundus image described above. In other words, by using the wide-angle optical system 30 that sets the viewing angle (FOV) of the fundus to an ultra-wide angle, it is possible to perform OCT imaging of a region beyond the equator 178 from the posterior pole of the fundus of the eye 12 to be examined. OCT data of structures existing in the periphery of the fundus, such as vortex veins, can be obtained, and a tomographic image of the vortex veins and a 3D structure of the vortex veins can be obtained by image processing the OCT data.

The OCT unit 20 includes a light source 20A, a sensor (detection element) 20B, a first optical coupler 20C, a reference optical system 20D, a collimating lens 20E, and a second optical coupler 20F.

The light emitted from the light source 20A is split by a first optical coupler 20C. One of the branched lights is made into parallel light by a collimating lens 20E as measurement light, and then enters the photographing optical system 19. The measurement light passes through the wide-angle optical system 30 and the pupil 27 and is irradiated onto the fundus of the eye. The measurement light reflected by the fundus enters the OCT unit 20 via the wide-angle optical system 30, and enters the second optical coupler 20F via the collimating lens 20E and the first optical coupler 20C.

The other light emitted from the light source 20A and branched by the first optical coupler 20C enters the reference optical system 20D as a reference light, and then enters the second optical coupler 20F via the reference optical system 20D. do.

These lights incident on the second optical coupler 20F, that is, the measurement light reflected from the fundus and the reference light, are interfered with each other by the second optical coupler 20F to generate interference light. The interference light is received by sensor 20B. The image processing device 17, which operates under the control of the image processing unit 206, generates OCT images such as tomographic images and en-face images based on the OCT data detected by the sensor 20B.

Here, an OCT image obtained by photographing at an internal illumination angle of 160 degrees or more or an OCT image obtained by scanning the peripheral part of the fundus is referred to as a UWF-OCT image. OCT images include tomographic images of the fundus obtained by B-scan, stereoscopic images (three-dimensional images) based on OCT volume data, and enface images (two-dimensional images) that are cross sections of the OCT volume data.

Image data of the UWF-OCT image is sent from the ophthalmological apparatus 110 to the server 140 via the communication interface (I/F) 16F, and is stored in the storage device 254.

In this embodiment, the light source 20A is a wavelength swept type SS-OCT (Swept-Source OCT), but various types such as SD-OCT (Spectral-Domain OCT) and TD-OCT (Time-Domain OCT) are used. The OCT system may be of any type.

Next, the configuration of the electrical system of the server 140 will be described with reference to FIG. 3. As shown in FIG. 3, the server 140 includes a computer main body 252. The computer main body 252 has a CPU 262, a RAM 266, a ROM 264, and an input/output (I/O) port 268. A storage device 254, a display 256, a mouse 255M, a keyboard 255K, and a communication interface (I/F) 258 are connected to the input/output (I/O) port 268. The storage device 254 is composed of, for example, nonvolatile memory. Input/output (I/O) port 268 is connected to network 130 via communication interface (I/F) 258 . Accordingly, server 140 can communicate with ophthalmological device 110 and viewer 150.

Server 140 stores each data received from ophthalmological device 110 in storage device 254.

Various functions realized by the CPU 262 of the server 140 executing the image processing program will be described. As shown in FIG. 4, the image processing program includes a display control function, an image processing function, and a processing function. The CPU 262 functions as the display control section 204, the image processing section 206, and the processing section 208 by executing the image processing program having each of these functions.

Next, image processing by the server 140 will be described in detail using FIG. 6. The image processing (image processing method) shown in the flowchart of FIG. 6 is realized by the CPU 262 of the server 140 executing the image processing program.

In step 600, the image processing unit 206 acquires a UWF-SLO image 179 as shown in FIG. 5B from the storage device 254 as a UWF fundus image. In step 602, the image processing unit 206 creates a choroidal blood vessel image, which is a binarized image, from the acquired UWF-SLO image 179 as follows. The choroidal blood vessel image is a binarized image in which pixels corresponding to choroidal blood vessels and vortex veins are white, and pixels in other areas are black.

A case will be explained in which the choroidal blood vessel image is generated from an R-color fundus image and a G-color fundus image. First, information included in the R-color fundus image and the G-color fundus image will be explained.

The structure of the eye is such that the vitreous body is covered by multiple layers with different structures. The layers include, from the innermost vitreous side to the outermost side, the retina, choroid, and sclera. The R light passes through the retina and reaches the choroid. Therefore, the first fundus image (R-color fundus image) includes information on blood vessels present in the retina (retinal blood vessels) and information on blood vessels present in the choroid (choroidal blood vessels). In contrast, G light only reaches the retina. Therefore, the second fundus image (G color fundus image) includes only information about blood vessels present in the retina (retinal blood vessels).

The image processing unit 206 of the CPU 262 extracts retinal blood vessels from the second fundus image (G color fundus image) by applying black hat filter processing to the second fundus image (G color fundus image). Next, the image processing unit 206 removes retinal blood vessels from the first fundus image (R color fundus image) by inpainting processing using the retinal blood vessels extracted from the second fundus image (G color fundus image). . In other words, the retinal blood vessel structure in the first fundus image (R color fundus image) is filled with the same value as the surrounding pixels using the position information of the retinal blood vessels extracted from the second fundus image (G color fundus image). . Then, the image processing unit 206 performs a Contrast Limited Adaptive Histogram Equalization (CLAHE) on the image data of the first fundus image (R-color fundus image) from which the retinal blood vessels have been removed. In one fundus image (R color fundus image), the choroidal blood vessels are emphasized. As a result, as shown in FIG. 8, a choroidal blood vessel image 300 is obtained in which the background is represented by black pixels and the choroidal blood vessels are represented by white pixels. The generated choroidal blood vessel image 300 is stored in the storage device 254.

Further, the choroidal blood vessel image 300 is generated from the first fundus image (R-color fundus image) and the second fundus image (G-color fundus image), but the image processing unit 206 generates the choroidal blood vessel image 300 from the first fundus image (R-color fundus image). ) Alternatively, the choroidal blood vessel image 300 may be generated using an IR fundus image photographed with IR light.

Regarding the method of generating the choroidal blood vessel image 300, the disclosure of International Publication WO2019/181981 is incorporated herein by reference in its entirety.

In step 602, the image processing unit 206 determines the area of interest (or area of interest) related to the lesion area on the fundus, such as CSC (central serous chorioretinopathy), the macula, which is a characteristic part of the fundus, and the optic disc. position). In step 602, for example, in addition to the UWF fundus image obtained in step 600, a separately obtained fundus fluorescence image, OCT data, and AF (autofluorescence image) images are used to determine the location of the lesion on the retina. and the macula as a region of interest, which is automatically detected or manually detected by the user's visual observation. In automatic detection of a region of interest, for example, in a fluorescent image or AF, a region exhibiting a color tone change specific to a lesion is extracted. Furthermore, since the macula is displayed as a dark area in the G-color fundus image, in this embodiment, a region of a predetermined number of pixels with the lowest luminance value in the G-color fundus image is detected as the position of the macula.

In step 604, the location of the vortex veins (VV) is analyzed as follows. The image processing unit 206 sets the moving direction (vessel running direction) of each choroidal blood vessel in the choroidal blood vessel image 300. Specifically, first, the image processing unit 206 executes the following process for each pixel of the choroidal blood vessel image. That is, the image processing unit 206 sets, for each pixel, an area (cell) centered around the pixel, and creates a histogram in the gradient direction of brightness for each pixel within the cell. Next, the image processing unit 206 sets the gradient direction with the smallest count in the histogram in each cell as the movement direction of the pixel in each cell. This gradient direction corresponds to the blood vessel running direction. The reason why the gradient direction with the least number of counts is the blood vessel running direction is as follows. The brightness gradient is small in the blood vessel running direction, while the brightness gradient is large in other directions (for example, there is a large difference in brightness between blood vessels and objects other than blood vessels). Therefore, when a histogram of the brightness gradient of each pixel is created, the number of counts in the blood vessel running direction becomes smaller. Through the above processing, the blood vessel traveling direction in each pixel of the choroidal blood vessel image is set.

The image processing unit 206 sets initial positions of M (natural number)×N (natural number) (=L) particles. Specifically, the image processing unit 206 sets a total of L initial positions, M in the vertical direction and N in the horizontal direction, at equal intervals on the choroidal blood vessel image.

The image processing unit 206 estimates the vortex vein position. Specifically, the image processing unit 206 performs the following processing for each of the L positions. That is, the image processing unit 206 acquires the blood vessel running direction at the first position (any of L), moves the particle by a predetermined distance along the acquired blood vessel running direction, and then moves the particle again at the moved position. The blood vessel running direction is acquired, and the particles are moved a predetermined distance along the acquired blood vessel running direction. This movement of a predetermined distance along the blood vessel running direction is repeated a preset number of times. The above processing is executed at all L positions. A point where a certain number or more of particles have gathered at that point is defined as a vortex vein position. Other methods for detecting vortex veins include image processing that recognizes as vortex veins positions on choroidal blood vessel images where the feature quantity of the radial pattern is greater than a predetermined value, and detection of vortex vein ampullae from choroidal blood vessel images. The position of the vortex vein may be detected by this method. Regarding the method of detecting vortex veins, the disclosure of International Publication WO2019/203309 is incorporated herein by reference in its entirety.

The vortex vein position information (the number of vortex veins, their coordinates on the choroidal blood vessel image, etc.) is stored in the storage device 254.

In step 606, data analysis is performed. Details of step 606 will be described later. Then, in step 608, the obtained data is saved in the storage device 254, and the process ends.

FIG. 7 is a flowchart showing the data analysis process of step 606 in FIG. In step 700, vortex veins connected to the region of interest are identified. FIG. 8 is a schematic diagram showing a first vortex vein 310V1 having vascular connectivity with a region of interest 310 indicating a lesion in the choroidal blood vessel image 300. In step 700, the region of interest 310 and the first vortex vein 310V1 are determined using graph theory that connects nodes (nodes) using the vascular structure (network structure consisting of contact points of blood vessels and links connecting the nodes) from the choroidal blood vessel image. Assess vascular connectivity. Graph theory is a calculation process that determines which nodes are mathematically rational to connect. In this embodiment, one node is set as the region of interest 310, the other node is set as the first vortex vein 310V1, Verify connectivity between the two. An example of graph theory is the application of graph cut theory. In this case, the nodes are branch points and vortex veins of choroidal blood vessels, and the edges are blood vessels connecting nodes. On an edge, define a data term such as blood vessel length and a smoothing term such as a term corresponding to the smooth connection of blood vessels, and select nodes and edges that minimize the energy function defined by the sum of the data term and smoothing term. By searching for the minimum cut that selects , it is possible to select a blood vessel that connects the vortex vein V1 and the region of interest 310 and verify its connectivity.

Further, the number of first vortex veins identified in step 700 is not limited to one, and two or more may be identified. When vortex veins are detected as a group, the region of interest 310 may be connected to multiple vortex veins forming the group by choroidal blood vessels, and two or more vortex veins in the group may be connected to the region of interest 310. It will have vascular connectivity with region 310.

Furthermore, when the region of interest 310 is the macula, it is often seen that two vortex veins, the upper nasal vortex vein and the lower nasal vortex vein, are connected by a choroidal blood vessel.

Note that the "predetermined area" described in the claims corresponds to the "region of interest 310" in this embodiment.

In this embodiment, when applying graph theory, morphological transformation may be performed to contract and expand the entire choroidal blood vessel image in order to eliminate the influence of minute blood vessels and branch blood vessels.

Furthermore, by applying morphological transformation and graph theory to the composite image data obtained by combining the choroidal blood vessel image with the OCTA (OCT angiography) enface image, the first vortex vein 310V1 connected to the region of interest 310 is identified. You may.

Note that the distance between the region of interest 310 and the detected vortex vein may be calculated, and the first vortex vein may be identified based on the distance. Furthermore, a vortex vein that has blood vessel connectivity with the region of interest 310 and satisfies the condition that the distance to the region of interest 310 is a predetermined distance or less may be set as the first vortex vein.

The distance between the region of interest 310 and the vortex vein on the image or the distance on the spherical surface of the eyeball model can be used as the distance. When a plurality of first vortex veins having vascular connectivity are identified, the first vortex veins can be narrowed down to first vortex veins in which the distance between the region of interest 310 and the vortex veins is equal to or less than a predetermined distance.

Furthermore, the above-mentioned definition of distance includes not only the distance between two points but also a feature quantity using the weight of the edge of the route determined by the above-mentioned graph theory. When a plurality of first vortex veins having vascular connectivity are identified, among the routes connecting the region of interest 310 and the vortex veins, the first vortex veins connected by a route whose feature amount is equal to or less than a predetermined value are selected. You can narrow it down.

FIG. 9 is an image 302 showing the presence of a choroidal blood vessel (dotted line in the figure) to which the region of interest 310 indicating the lesion and the first vortex vein 310V1 are connected. In the image 302, the region of interest 310 is shown to be connected to the first vortex vein 310V1 and the choroidal blood vessels indicated by dotted lines. However, this blood vessel is not connected to second vortex veins 310V2, 310V3, and 310V4, which are vortex veins other than the first vortex vein 310V1. By performing such image processing on the choroidal blood vessels connected to the region of interest, the first vortex vein, which is a vortex vein related to the region of interest, is identified.

In step 702, the first vortex vein is excluded from the plurality of vortex veins detected in step 604 of FIG. 6, and second vortex veins 310V2, 310V3, and 310V4 are identified. Information such as the positions of the identified second vortex veins 310V2, 310V3, and 310V4 is stored in the storage device 254.

Next, in step 704, the position of the first vortex vein 310V1 is analyzed. In the position analysis, the position of the first vortex vein 310V1 is specified using the same method as in step 604 described above. Then, the distance between the identified first vortex vein 310V1 and the region of interest 310 such as a lesion or the macula is calculated. The distance may be calculated from the pixel size on the image, or may be calculated as a distance on the eyeball surface.

Further, in step 704, as shown in FIGS. 10A and 10B, analysis circles 312A, 312B, 312C, and 312D are provided in the choroidal blood vessel image 304, with analysis circles 312A, 312B, 312C, and 312D centered on the vortex veins. The diameter of the blood vessel that intersects with the blood vessel may be measured. FIG. 10B is an enlarged view of the analytic circle 312A. In FIG. 10B, the diameter of the intersection between the analysis circle 312A and the blood vessel is shown in μm. In step 702, the diameters of blood vessels that intersect the analysis circles 312A, 312B, 312C, and 312D are calculated, taking into account that the fundus is approximately spherical.

Furthermore, in step 704, a comparison may be made between the normal eye and the eye 12 to be examined. For comparison with a normal eye, for example, as shown in FIG. 10A, the choroidal blood vessel image 304 is divided into four quadrants 322A, 322B, 322C, and 322D with the optic disc 320 as the center, and multiple vortex veins within the same quadrant are The representative position (e.g., center position) determined from is compared with the data of the center position of the vortex vein in a normal eye. The center position of the vortex vein is represented by the distance and angle from the optic disc 320, for example, as shown in FIGS. 11 to 13, which will be described later. Since the optic disc 320 is displayed as the brightest area in the G-color fundus image, in this embodiment, a region of a predetermined number of pixels with the largest luminance value in the G-color fundus image is detected as the position of the optic disc 320. do.

The data on the center position of the vortex vein in normal eyes is managed as a database that collects data on the center position and the diameter of the vortex vein in multiple specimens of normal eyes, and statistics such as the average value and standard deviation of the data are collected. indicators are calculated in advance. In step 704, the position of the first vortex vein 310V1 of the subject's eye 12 is determined by comparing the center position of the vortex vein in the choroidal blood vessel image 304 of the subject's eye 12 with the average value of the center position of the vortex vein of a normal eye. It is possible to determine how far the eye deviates from the normal eye. Further, in step 704, the average value and standard deviation of the center position of the vortex vein are calculated when the center position of the vortex vein in the choroidal blood vessel image 304 of the subject eye 12 is added to the data of the center position of the vortex vein of the normal eye. Then, the calculated average value and standard deviation are compared with the average value and standard deviation of the center position of the vortex vein in the normal eye calculated in advance, and the center position of the vortex vein in the subject eye 12 is compared to the normal eye. You may estimate how much they differ.

Then, in step 704, the vascular connectivity between the first vortex vein 310V1 and the region of interest 310 performed in step 700, the distance between the first vortex vein 310V1 and the region of interest 310, the first vortex vein 310V1 and the vortex vein in the normal eye. The degree of involvement of the first vortex vein 310V1 in the lesion is determined based on the deviation from the position of the first vortex vein 310V1 and the diameter of the first vortex vein 310V1. For example, if the first vortex vein 310V1 and the region of interest 310 are connected, and the distance between the first vortex vein 310V1 and the region of interest 310 is less than a predetermined distance, the first vortex vein 310V1 and the vortex in the normal eye This applies either when the deviation from the position of the vein is larger than a predetermined positional deviation, or when the deviation between the diameter of the first vortex vein 310V1 and the diameter of the vortex vein in a normal eye is larger than a predetermined diameter deviation. If so, it can be determined that the first vortex vein 310V1 is a vortex vein related to the lesion.

When the region of interest 310 is a lesion, the first vortex vein connected to the lesion can be identified, and images of the area surrounding the first vortex vein can be intensively analyzed or progress observation can be performed. Ophthalmologists and others can identify the target vortex vein.

Furthermore, when the region of interest 310 is the macula, the macula and the first vortex vein can be identified, and images of the region around the first vortex vein as well as images of the region around the macula can be intensively analyzed. , ophthalmologists and others can identify vortex veins as targets for follow-up observation. Diseases that occur in the macula include age-related macular degeneration, pachycholloid diseases such as polypoidal choroidal vasculopathy (PCV), and diabetic macular degeneration. Obtaining information regarding the positional relationship with the vortex veins related to the macula will lead to obtaining valuable information regarding the intrachoroidal circulation, which is useful for diagnosis and treatment selection.

In step 706, the image processing unit 206 generates a display screen showing the analysis results. The display screen generated in step 706 includes an image that distinguishes and displays the first vortex vein 310V1 and second vortex veins 310V2, 310V3, and 310V4 as shown in FIG. 11, which will be described later, and a choroidal blood vessel image as shown in FIG. includes an image in which a mark indicating the first vortex vein 310V1 and a choroidal blood vessel connecting the first vortex vein 310V1 and the region of interest 310 are highlighted and displayed superimposed on the choroidal blood vessel image. In step 706, the processing unit 208 stores the generated display screen and the data analyzed in step 704 in the storage device 254. Then, the process returns.

The display screen displays the respective positions of the first vortex vein 310V1 and second vortex veins 310V2, 310V3, and 310V4, which are vortex veins other than the first vortex vein 310V1, as shown in FIGS. 11 to 13, which will be described later. . In addition, as shown in FIG. 12, which will be described later, the average position of the vortex veins in the normal eye is also displayed to show how far the position of the first vortex vein 310V1 deviates from the average position of the vortex veins in the normal eye. You can also do this. Alternatively, the position of the vortex vein may be quantitatively evaluated. A quantitative evaluation of the position of the vortex vein is, for example, the distance from the optic disc 320. Since each of the vortex veins is considered to exist at equal distances from the optic disc 320, a vortex vein whose distance from the optic disc 320 is different from other vortex veins is suspected to be related to the lesion. In the present embodiment, a vortex vein whose distance from the optic disc 320 is different from that of other vortex veins may be further detected as a vortex vein related to a lesion. Further, an additional OCT scan may be performed on vortex veins such as the first vortex vein 310V1 that are considered to be related to the lesion.

FIG. 11 is a schematic diagram showing a display screen 500 that is a display screen displayed on the display of the viewer 150.

Display screen 500 has an information area 502 and an image display area 504, as shown in FIG. Information area 502 includes patient ID display field 512, patient name display field 514, age display field 516, visual acuity display field 518, right eye/left eye display field 520, and axial length display field 522. Viewer 150 displays each piece of information in each display area from patient ID display field 512 to axial length display field 522 based on the received information.

The image display area 504 is an area for displaying fundus images and the like. A comment field 506 provided in the image display area 504 is a comment field into which the user, an ophthalmologist, can arbitrarily input observation results or diagnostic results.

In FIG. 11, the image display area 504 includes a region of interest 310 indicating a lesion, a first vortex vein 310V1 connected to the region of interest 310, and second vortex veins 310V2 and 310V3 that are vortex veins other than the first vortex vein 310V1. Each fundus structure such as 310V4 is displayed, and a position display image 330 showing the distance and angle of the fundus structure with respect to the optic disc 320 is displayed in the form of a radar chart. In FIG. 11, one region of interest 310 and one first vortex vein 310V1 are displayed, but the present invention is not limited to this. Depending on the pathological condition of the eye 12 to be examined, a plurality of each of the region of interest 310 and the first vortex vein 310V1 may be detected. Although angles are shown in degrees in FIG. 11 and FIGS. 12 and 13, which will be described later, they may also be shown in radians.

FIG. 12 is a variation of a display screen 500 having a position display image 332 displaying the average position 310A of the vortex veins in a normal eye. By displaying the average position of the vortex vein in the normal eye, it is possible to intuitively understand how far the position of the first vortex vein 310V1 of the eye 12 to be examined deviates from the normal eye. In FIG. 12, the average position of the vortex vein in the normal eye is displayed in the quadrant where the first vortex vein 310V1 exists, but the average position of the vortex vein in the normal eye is also displayed in the quadrant where the second vortex vein 310V2, 310V3, and 310V4 exist. may be displayed.

FIG. 13 shows the position 310V1A of the first vortex vein 310V1 when the coordinates of the fundus image are transformed so that the center of the first vortex vein 310V1 and the second vortex veins 310V2, 310V3, and 310V4 becomes the optic disc (ONH) 320. 5 is an example of a display screen 500 having a position display image 334 showing a location display image 334. In a normal eye, each of the vortex veins is generally considered to exist within an appropriate range from the optic disc 320, so if the deviation between the first vortex vein 310V1 and the position 310V1A is large in the position display image 334, the first vortex vein It can be inferred that the single vortex vein 310V1 is involved in some kind of lesion.

FIG. 14 is a choroidal vessel image 316 that is a UWF-SLO image displayed in the image display area 504 of the display screen 500. As shown in FIG. 14, analysis circles 312A, 312B, 312C, and 312D centered on the vortex veins are displayed, and the diameters of blood vessels that intersect with the analysis circles 312A, 312B, 312C, and 312D are also displayed. In FIG. 14, a circle mark indicating the first vortex vein 310V1 and a choroidal blood vessel connecting the first vortex vein 310V and the region of interest 310 may be highlighted and displayed superimposed on the choroidal blood vessel image 316.

FIG. 15 is a modification of the choroidal blood vessel image 316 shown in FIG. 14, in which a region of interest 310 indicating a lesion, a first vortex vein 310V1 connected to the region of interest 310, and an analysis circle 312B surrounding the first vortex vein 310V1 are displayed. ing. In FIG. 15, unlike in FIG. 14, the analysis circles 312A, 312C, and 312D are not displayed, so the choroidal blood vessel image can be observed focusing on the first vortex vein 310V1 and the region of interest 310.

As explained above, in this embodiment, the first vortex vein 310V1 having blood vessel connectivity with the region of interest 310 indicating a lesion is detected using graph theory, and the detected first vortex vein 310V1 and the region of interest 310 is less than a predetermined distance, when the deviation between the first vortex vein 310V1 and the position of the vortex vein in a normal eye is larger than a predetermined positional deviation, and when the diameter of the first vortex vein 310V1 and the vortex in a normal eye are If the deviation from the diameter of the vein is larger than a predetermined diameter deviation, it can be determined that the first vortex vein 310V1 is a vortex vein related to a lesion.

In this embodiment, the processing shown in FIGS. 6 and 7 is performed by the server 140, but may be performed by the image processing device 17 of the ophthalmological apparatus 110.

The image processing in each embodiment described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be changed within the scope of the main idea.

Although each of the embodiments described above assumes image processing using a software configuration using a computer, the technology of the present disclosure is not limited to this. For example, instead of using a software configuration using a computer, image processing may be performed only by a hardware configuration such as an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). Part of the image processing may be performed by a software configuration, and the remaining processing may be performed by a hardware configuration.

As described above, the technology of the present disclosure includes cases in which image processing is implemented by a software configuration using a computer and cases in which it is not implemented, and therefore includes the following technology.
(First technology)
a step in which the lesion detection unit detects a lesion in the fundus from the choroidal blood vessel image;
a vortex vein detection unit detecting vortex veins from the choroidal blood vessel image;
a step in which a vortex vein identification unit detects a first vortex vein related to the lesioned part and a second vortex vein other than the first vortex vein from among the vortex veins detected from the choroidal blood vessel image;
image processing methods including;
(Second technology)
a lesion detection unit that detects a lesion in the fundus from a choroidal blood vessel image;
a vortex vein detection unit that detects vortex veins from the choroidal blood vessel image;
a vortex vein identification unit that detects a first vortex vein related to the lesion and a second vortex vein other than the first vortex vein among the vortex veins detected from the choroidal blood vessel image;
An image processing device comprising:
(Third technology)
A computer program product for image processing,
The computer program product comprises a computer readable storage medium that is not itself a transitory signal;
A program is stored in the computer readable storage medium,
The program is installed on a computer,
Detecting a lesion in the fundus from the choroidal blood vessel image;
detecting vortex veins from the choroidal blood vessel image;
A computer program product for detecting a first vortex vein related to the lesion and a second vortex vein other than the first vortex vein among the vortex veins detected from the choroidal blood vessel image.

12 Eye to be examined 14 Photographing device 16 Control device 17 Image processing device 100 Ophthalmology system 110 Ophthalmology device 140 Server 150 Viewer 179 UWF-SLO image 204 Display control section 206 Image processing section 208 Processing section 252 Computer body 254 Storage device 256 Display 262 CPU
264 ROM
266 RAM
300, 304 Choroidal blood vessel image 310 Region of interest 310A Average position 310V1A Position 310V1 First vortex vein 310V2, 310V3, 310V4 Second vortex vein 312A, 312B, 312C, 312D Analysis circle 316 Choroidal blood vessel image 320 Optic disc 500 Display screen

Claims (17)

Image processing performed by a processor,
detecting a predetermined region of the fundus from the choroidal blood vessel image;
detecting a plurality of vortex veins from the choroidal blood vessel image;
calculating a feature amount related to blood vessel connectivity with the predetermined region for the plurality of vortex veins;
identifying a first vortex vein having vascular connectivity in the predetermined region from among the plurality of vortex veins detected from the choroidal blood vessel image based on the feature amount ;
image processing methods, including The feature amount is calculated based on the structure of a blood vessel connected to the plurality of vortex veins and the predetermined region.
The image processing method according to claim 1. The structure of the blood vessel includes the length of the blood vessel or the smoothness of the connection of the blood vessel.
The image processing method according to claim 2. The image processing method according to claim 1, further comprising the step of identifying a second vortex vein other than the first vortex vein among the plurality of vortex veins. The image processing method according to claim 1, wherein the predetermined area is a lesion. The image processing method according to claim 1, wherein the predetermined region is the macula. 2. The first vortex vein is a vortex vein in which the vortex vein has vascular connectivity with the predetermined region, and the distance between the vortex vein and the predetermined region is equal to or less than a predetermined distance. Image processing method. 8. The image processing method according to claim 7, wherein the first vortex vein is a vortex vein having the closest distance between the vortex vein and the predetermined area. The image processing method according to claim 1, further comprising the step of generating a display screen for displaying a positional relationship between the first vortex vein and a second vortex vein other than the first vortex vein. The image processing method according to claim 9 , comprising the step of generating a display screen that displays the first vortex vein and the second vortex vein in a distinguished manner. The image processing method according to claim 9 or 10, wherein the display screen is a radar chart. 11. The image processing method according to claim 9, wherein the display screen is a choroidal blood vessel image in which the first vortex vein and the second vortex vein are displayed separately. On the display screen, a mark indicating the first vortex vein and a choroidal blood vessel connecting the first vortex vein and the predetermined area are highlighted and displayed superimposed on the choroidal blood vessel image. The image processing method according to claim 12 . 11. The image processing method according to claim 9 , wherein the display screen is an image in which the first vortex vein and the second vortex vein are displayed separately in a UWF-SLO image. On the display screen, a mark indicating the first vortex vein and a choroidal blood vessel connecting the first vortex vein and the predetermined region are highlighted and displayed superimposed on the UWF-SLO image. The image processing method according to claim 14 . comprising a memory and a processor connected to the memory;
The processor includes:
Detects a predetermined area of the fundus from the choroidal blood vessel image,
detecting vortex veins from the choroidal blood vessel image;
Calculating feature amounts related to connectivity with the predetermined region for the plurality of vortex veins,
An image processing device that detects a first vortex vein having connectivity to the predetermined region from among the vortex veins detected from the choroidal blood vessel image based on the feature amount . to the computer,
Detects a predetermined area of the fundus from the choroidal blood vessel image,
detecting vortex veins from the choroidal blood vessel image;
Calculating feature amounts related to connectivity with the predetermined region for the plurality of vortex veins,
An image processing program for detecting a related first vortex vein having connectivity to the predetermined region from among vortex veins detected from the choroidal blood vessel image based on the feature amount .