JP2020054812A - Image processing device, image processing method and program - Google Patents

Abstract

To provide a mechanism that, when a blood vessel image of a subject has multiple candidates of avascular area, can accurately detect those avascular areas.SOLUTION: An image processing device specifies multiple avascular areas corresponding to multiple positions that are set to a blood vessel image of a subject (S350), and determines whether there is any overlapped avascular area in the specified multiple avascular areas (S370).SELECTED DRAWING: Figure 3

Description

  The present invention relates to an image processing device, an image processing method, and a program.

  2. Description of the Related Art Conventionally, when a fundus of an eye to be examined is photographed as a subject using, for example, a tomographic image photographing apparatus such as an optical coherence tomography (OCT), a state inside a retinal layer can be three-dimensionally observed. . Since this tomographic imaging apparatus is useful for more accurately diagnosing a disease, it is widely used particularly in the field of ophthalmic medical care. As a form of the OCT, for example, there is a TD-OCT (Time Domain OCT) in which a broadband light source and a Michelson interferometer are combined. In this TD-OCT, the position of a reference mirror that reflects the reference light branched from the measurement light is moved at a constant speed to obtain the interference light between the measurement light and the reference light, and the light in the depth direction of the subject is obtained. It is configured to obtain an intensity distribution. However, in this TD-OCT, it is difficult to obtain a tomographic image at high speed because mechanical driving is required. Therefore, as a method of acquiring a tomographic image at a higher speed, a spectroscope uses a broadband light source to acquire an interference signal based on the interference light, or a spectral domain OCT, or a temporally spectral source using a high-speed wavelength sweep light source. SS-OCT (Swept Source OCT) has been developed.

  In recent years, OCT Angiography (hereinafter, referred to as “OCTA”) that non-invasively renders a blood vessel three-dimensionally using OCT to generate a blood vessel image of a subject (for example, a fundus blood vessel image of an eye to be examined). ) Technology is being used. In this OCTA, the same position of the subject is scanned a plurality of times with measurement light, and a motion contrast obtained by the interaction between the displacement of the red blood cells and the measurement light is imaged.

  In addition, for example, for a disease diagnosis of diabetic retinopathy, identification of an avascular region such as a foveal avascular region or an unperfused region is performed from a two-dimensional frontal OCTA image. For example, in Patent Literature 1, in order to specify an avascular region on an OCTA image which is a blood vessel image, a process of specifying an avascular region based on position information of a seed point set by an instruction from a user is performed. I have.

JP 2017-47127 A

  In the technique described in Patent Document 1 described above, when there is one candidate for an avascular region in a blood vessel image of a subject, one seed point is set to specify the one avascular region. For this reason, the technique described in Patent Literature 1 has a problem that when there are a plurality of candidates for an avascular region in a blood vessel image of a subject, it is difficult to accurately detect these avascular regions. Was.

  The present invention has been made in view of such a problem, and provides a mechanism that can accurately detect these avascular regions when there are a plurality of candidate avascular regions in a blood vessel image of a subject. It is one of the purposes.

One of the image processing apparatuses according to the present invention includes a specifying unit that specifies a plurality of avascular regions corresponding to a plurality of positions set with respect to a blood vessel image of a subject, and an avascular region in the plurality of avascular regions. Determining means for determining the overlap between them.
Further, the present invention includes an image processing method by the above-described image processing apparatus, and a program for causing a computer to function as each unit of the above-described image processing apparatus.

  According to an embodiment of the present invention, when there are a plurality of candidates for an avascular region in a blood vessel image of a subject, these avascular regions can be accurately detected.

FIG. 1 is a diagram illustrating an example of a schematic configuration of an image processing system including an image processing device according to a first embodiment. FIG. 2 is a diagram illustrating an example of a schematic configuration inside the image processing apparatus according to the first embodiment. 5 is a flowchart illustrating an example of a processing procedure of an image processing method performed by the image processing apparatus according to the first embodiment. FIG. 3 shows the first embodiment and is a diagram for explaining OCTA imaging. FIG. 4 is a diagram illustrating an example of information including an image obtained by the processing of the flowchart illustrated in FIG. 3 according to the first embodiment. FIG. 4 is a diagram illustrating an example of information including an image obtained by the processing of the flowchart illustrated in FIG. 3 according to the first embodiment. FIG. 4 shows the first embodiment, and is a diagram for explaining the process of step S350 in FIG. FIG. 4 illustrates the first embodiment, and is a diagram illustrating an example of a mask image of an NPA (avascular region) obtained in step S355 in FIG. 3C. FIG. 4 shows the first embodiment, and is a diagram for explaining the processing in step S380 in FIG. FIG. 6 shows the first embodiment, and is a diagram for explaining the processing of step S386 in FIG. FIG. 9 is a diagram for describing a first modification. FIG. 9 is a diagram for explaining a modification 2; 9 is a flowchart illustrating an example of a processing procedure of an image processing method performed by the image processing apparatus according to the second embodiment. FIG. 14 shows the second embodiment and is a diagram for describing the processing of the flowchart shown in FIG. 13. 13 is a flowchart illustrating an example of a processing procedure of an image processing method by an image processing device according to a third embodiment. FIG. 16 shows the third embodiment, and is a diagram for describing the processing in the flowchart shown in FIG. 15. FIG. 14 shows the fourth embodiment, and is a diagram illustrating an example of information indicating a temporal change of NPA information displayed on the display device in FIG. 1 in a time series. FIG. 14 is a diagram for explaining a modification 6; It is a figure for explaining modification 7 and modification 8.

  Hereinafter, an embodiment (embodiment) for carrying out the present invention will be described with reference to the drawings.

[First Embodiment]
First, a first embodiment will be described.

  In the first embodiment, the input (specifically, in the present embodiment, the fundus of the subject's eye) blood vessel image (specifically, in the present embodiment, the fundus blood vessel image), the foveal avascular area, As a method of specifying an NPA (avascular region) in a region of interest such as a non-perfusion region, a method of setting a seed point in each of a plurality of NPA candidates and specifying each NPA based on the information will be described. I do. At this time, the seed point corresponds to a start point of detection of an NPA (avascular region).

  Specifically, in the image processing apparatus according to the present embodiment, there are a plurality of NPA candidates in the fundus blood vessel image, the fundus blood vessel image is presented to the user (operator), and each NPA candidate is input from the user's operation input. A seed point is set, and an NPA is specified based on the set seed point. Then, in the image processing apparatus according to the present embodiment, when an overlap of the specified NPAs (duplication of the specified NPAs) occurs, a warning is presented to the user, and the overlap is performed from the user's operation input. An example will be described in which NPA editing processing is performed to specify each NPA.

  FIG. 1 is a diagram illustrating an example of a schematic configuration of an image processing system 10 including an image processing apparatus 100 according to the first embodiment. As shown in FIG. 1A, the image processing system 10 includes an image processing device 100, a tomographic image capturing device 200, an input device 300, and a display device 400. The image processing device 100 is connected to the tomographic image capturing device 200, the input device 300, and the display device 400 via, for example, an interface.

  The tomographic image photographing apparatus 200 is an apparatus for photographing various image monks including a tomographic image of the eye (eye E to be examined in FIG. 1B). In the present embodiment, the tomographic image capturing apparatus 200 uses SD-OCT. However, in the present invention, the tomographic image capturing apparatus 200 is not limited to the SD-OCT, and may use, for example, the SS-OCT. Apparatus is also applicable. As shown in FIG. 1A, the tomographic imaging apparatus 200 includes a measurement optical system 200-1, a stage 200-2, and a base 200-3. The measurement optical system 200-1 is an optical system for capturing an anterior eye image of the eye E, an SLO fundus image of the eye E, and a tomographic image of the eye E shown in FIG. The stage section 200-2 is a component configured to be able to move the measuring optical system 200-1 in the front, rear, left, and right directions (xyz directions in FIG. 1B). The base unit 200-3 is a component that incorporates a spectroscope described later.

  The image processing device 100 is a device that processes various images including a tomographic image captured by the tomographic image capturing device 200 based on information input from the input device 300, for example. The image processing apparatus 100 is a computer that controls the stage unit 200-2, controls the alignment operation, reconstructs a tomographic image, and the like. The image processing apparatus 100 includes a storage unit 120. The storage unit 120 includes, for example, a program for capturing tomographic images, patient information, imaging data, fundus vascular images, image data of past examinations, Measurement data such as area (NPA) information is stored.

  The input device 300 is a device that inputs various information to the image processing device 100. The input device 300 includes, for example, a keyboard, a mouse, and a touch operation screen, and a user issues an instruction to the image processing device 100 or the tomographic image capturing device 200 via the input device 300. .

  The display device 400 is a device that displays various images and various information processed by the image processing device 100 under the control of the image processing device 100, for example. The display device 400 includes, for example, a monitor.

(Schematic configuration of tomographic imaging apparatus 200)
Next, a schematic configuration of the tomographic imaging apparatus 200 will be described with reference to FIG. FIG. 1B shows a schematic configuration inside the measurement optical system 200-1 shown in FIG. 1A and a schematic configuration inside the base unit 200-3 shown in FIG. 1A. I have.

First, a schematic configuration inside the measurement optical system 200-1 will be described.
In the measurement optical system 200-1, an objective lens 201 is installed so as to face the eye E, and a first dichroic mirror 202 and a second dichroic mirror 203 are arranged on the optical axis. By these dichroic mirrors 202 and 203, light is branched into an optical path 231 of the OCT optical system, an optical path 232 for the SLO optical system and the fixation lamp, and an optical path 233 for anterior eye observation for each wavelength band. Here, the SLO optical system is an optical system of a confocal laser scanning ophthalmoscope (Scanning Laser Ophthalmoscope: SLO).

  In the optical path 232 for the SLO optical system and the fixation light, an SLO scanning means 204, lenses 205 and 206, a mirror 207, a third dichroic mirror 208, an SLO light source 209, a fixation light 210, and an APD (Avalanche Photodiode) 211 are arranged. Have been. The mirror 207 is, for example, a prism on which a perforated mirror or a hollow mirror is deposited, and separates the illumination light from the SLO light source 209 from the return light from the eye E. The third dichroic mirror 208 separates the optical path of the SLO light source 209 and the optical path of the fixation lamp 210 for each wavelength band. The SLO scanning unit 204 scans the light emitted from the SLO light source 209 on the eye E, and includes an X scanner that scans in the X direction and a Y scanner that scans in the Y direction. In the present embodiment, for example, the X scanner of the SLO scanning unit 204 needs to perform high-speed scanning, and is configured by a polygon mirror, and the Y scanner of the SLO scanning unit 204 is configured by a galvanometer mirror. The lens 205 is driven by a motor (not shown) for focusing the SLO optical system and the fixation lamp 210. The SLO light source 209 generates light having a wavelength near 780 nm. The fixation lamp 210 generates visible light to promote fixation of the subject's eye E. The APD 211 detects the return light from the subject's eye E.

  The light emitted from the SLO light source 209 is reflected by the third dichroic mirror 208, passes through the mirror 207, passes through the lenses 206 and 205, and is examined by the SLO scanning unit 204 (specifically, in this embodiment, the subject's eye E). It is scanned on the fundus Er) of the optometry E. The return light from the subject's eye E returns along the same path as the illumination light, is reflected by the mirror 207, is guided to the APD 211, and an SLO fundus image is obtained. The light emitted from the fixation lamp 210 passes through the third dichroic mirror 208 and the mirror 207, passes through the lenses 206 and 205, and has a predetermined shape at an arbitrary position on the eye E by the SLO scanning means 204. To promote fixation of the eye E.

  In the optical path 233 for anterior eye observation, a lens 212, a split prism 213, a lens 214, and a CCD 215 for anterior eye observation for detecting infrared light are arranged. The CCD 215 has a sensitivity at a wavelength of irradiation light (not shown) for anterior ocular segment observation, specifically, around 970 nm. The split prism 213 is disposed at a position conjugate with the pupil of the eye E to be inspected, and the distance in the Z-axis direction (optical axis direction) of the measuring optical system 200-1 to the eye E to be inspected is defined as a split image of the anterior eye part. Can be detected.

  An OCT optical system is disposed in the optical path 231 of the OCT optical system. The OCT optical system is used to capture a tomographic image of the eye E (specifically, in this embodiment, the fundus Er of the eye E). Things. More specifically, the OCT optical system obtains an interference signal for forming a tomographic image. The XY scanner 216 is an OCT scanning unit that scans the measurement light with respect to the eye E (specifically, in this embodiment, the fundus Er of the eye E). Although the XY scanner 216 is illustrated as a single mirror in FIG. 1B, it is actually a galvano mirror that performs scanning in two axial directions in the x and y directions shown in FIG. 1B. . The lens 217 of the lenses 217 and 218 is driven by a motor (not shown) to focus light from the OCT light source 220 emitted from the optical fiber 225 connected to the optical coupler 219 to the eye E. By this focusing, the return light from the eye E is simultaneously focused on the tip of the optical fiber 225 in the form of a spot and is incident. Next, the optical path from the OCT light source 220 to the eye E, the reference optical system, and the spectroscope 240, which constitute the optical path 231 of the OCT optical system, will be described. An OCT light source 220, an optical fiber 224, an optical coupler 219, an optical fiber 225, a polarization adjusting unit 228 provided on the optical fiber 225, a lens 218, a lens 217, and an XY scanner 216 are arranged on the optical path from the OCT light source 220 to the eye E. ing. As the reference optical system, an optical fiber 226 connected to the optical coupler 219, a polarization adjusting unit 229 provided on the optical fiber 226, a lens 223, a dispersion compensation glass 222, and a reference mirror 221 are provided. The spectroscope 240 is provided inside the base unit 200-3, and receives light from the optical fiber 227 connected to the optical coupler 219. The optical fibers 224 to 227 are, for example, single-mode optical fibers connected to and integrated with the optical coupler 219. In the present embodiment, these configurations constitute a Michelson interferometer.

  The light emitted from the OCT light source 220 is split through the optical fiber 224 into measurement light traveling toward the optical fiber 225 via the optical coupler 219 and reference light traveling toward the optical fiber 226. The measurement light branched by the optical coupler 219 passes through the dichroic mirrors 203 and 202 and the objective lens 201 through the above-described optical path from the OCT light source 220 to the eye E to be inspected. Then, the light is irradiated on the fundus Er) of the eye E to be examined. Then, the measurement light applied to the eye E reaches the optical coupler 219 through the same optical path due to reflection and scattering by the eye E.

  On the other hand, the reference light branched by the optical coupler 219 reaches the reference mirror 221 via the optical fiber 226, the lens 223, and the dispersion compensation glass 222 inserted for adjusting the wavelength dispersion of the measurement light and the reference light, and is reflected. . Then, the reference light reflected by the reference mirror 221 returns along the same optical path and reaches the optical coupler 219.

  The measurement light returned via the eye E and the reference light returned via the reference mirror 221 are multiplexed by the optical coupler 219 into interference light. Here, interference occurs when the optical path length of the measurement light and the optical path length of the reference light become substantially the same. The reference mirror 221 is held so as to be adjustable in the optical axis direction by a motor and a driving mechanism (not shown), and is configured to be able to adjust the optical path length of the reference light to the optical path length of the measurement light. The polarization adjusters 228 and 229 are provided in the optical fibers 225 and 226, respectively, and perform polarization adjustment. These polarization adjusting sections 228 and 229 have several portions where the corresponding optical fibers are drawn in a loop. By rotating the loop portion about the longitudinal direction of the optical fiber, the optical fiber is twisted, and the polarization states of the measurement light and the reference light can be adjusted and matched. Then, the interference light generated by the optical coupler 219 is guided to the spectroscope 240 via the optical fiber 227.

  The spectroscope 240 includes a lens 241, a diffraction grating 242, a lens 243, and a line sensor 244. The spectroscope 240 is obtained by multiplexing the return light from the eye E to which the measurement light is irradiated (specifically, in this embodiment, the fundus Er of the eye E to be inspected) with the reference light corresponding to the measurement light. It is a detecting means for detecting interference light (combined light). The interference light emitted from the optical fiber 227 is converted into parallel light through the lens 241, is then separated by the diffraction grating 242, and is imaged on the line sensor 244 by the lens 243. Then, the line sensor 244 generates an interference signal based on the formed interference light.

Next, the periphery of the OCT light source 220 will be described.
The OCT light source 220 is, for example, an SLD (Super Luminescent Diode) that is a typical low coherent light source. In this case, the center wavelength of the light emitted from the OCT light source 220 is about 855 nm, and its wavelength bandwidth is about 100 nm. Here, the bandwidth is an important parameter because it affects the resolution of the obtained tomographic image in the optical axis direction. Although the SLD is used here as the OCT light source 220, the present invention is not limited to this. Any light source that can emit low coherent light can be used. For example, ASE (Amplified Spontaneous Emission) or the like can be used. . The near-infrared light is suitable for the center wavelength of the light emitted from the OCT light source 220 in consideration of measuring the eye E to be inspected. Further, the center wavelength of the light emitted from the OCT light source 220 affects the resolution in the lateral direction of the obtained tomographic image. For both reasons, in the present embodiment, the center wavelength of the light emitted from the OCT light source 220 is set to about 855 nm. Further, in the present embodiment, a Michelson interferometer is used as an interferometer, but a Mach-Zehnder interferometer may be used, for example. For example, it is desirable to use a Mach-Zehnder interferometer when the light amount difference is large and to use a Michelson interferometer when the light amount difference is relatively small, according to the light amount difference between the measurement light and the reference light.

(Schematic Configuration of Image Processing Apparatus 100)
Next, a schematic configuration inside the image processing apparatus 100 shown in FIG. 1A will be described.
FIG. 2 is a diagram illustrating an example of a schematic configuration inside the image processing apparatus 100 according to the first embodiment. In FIG. 2, in addition to the image processing device 100, a tomographic image capturing device 200, an input device 300, and a display device 400 shown in FIG.

  The image processing device 100 is, for example, a personal computer (PC) communicably connected to the tomographic image capturing device 200, and as shown in FIG. 2, an image acquiring unit 110, a storage unit 120, a capturing control unit 130, an image processing unit And a display control unit 150.

  In the image processing device 100, for example, the functions of the image acquisition unit 110, the imaging control unit 130, the image processing unit 140, and the display control unit 150 are realized by a CPU that is an arithmetic processing device executing a program of a software module. It can take the form. Note that the present invention is not limited to this mode. For example, the image processing unit 140 may be realized by dedicated hardware such as an ASIC, or the display control unit 150 may be implemented by a GPU or the like different from a CPU. It may be realized by using a dedicated processor. Further, the communication connection between the tomographic imaging apparatus 200 and the image processing apparatus 100 may be configured via a network.

  The image acquiring unit 110 acquires signal data of an SLO fundus image and a tomographic image captured by the tomographic image capturing apparatus 200. The image acquisition unit 110 includes a tomographic image generation unit 111 and a motion contrast data generation unit 112. The tomographic image generation unit 111 acquires signal data (interference signal) of a tomographic image captured by the tomographic image capturing apparatus 200 and performs signal processing on the signal data (interference signal) to obtain the subject (the fundus E of the eye E in the present embodiment). An Er) tomographic image is generated. Then, the tomographic image generation unit 111 stores the data of the generated tomographic image in the storage unit 120. The motion contrast data generation unit 112 repeatedly captures the same position of the subject, and generates motion contrast data that is data obtained by detecting a temporal change of the subject during the capturing.

  The storage unit 120 stores information on the eye E (patient's name, age, gender, and the like), images (tomographic images, SLO images, OCTA images, and the like) obtained by imaging, composite images, imaging parameters, NPA (avascular region) ) Information, measurement values, parameters set by the user, etc. are stored in association with each other.

  The imaging control unit 130 performs imaging control on the tomographic imaging apparatus 200. The photographing control includes instructing the tomographic image photographing apparatus 200 about setting of photographing parameters and instructing start or end of photographing. At this time, for example, the imaging control unit 130 may adopt a form in which imaging control for the tomographic imaging apparatus 200 is performed using the imaging parameters stored in the storage unit 120.

  The image processing unit 140 performs image processing on various images (tomographic images, SLO images, OCTA images, and the like). As shown in FIG. 2, the image processing unit 140 includes a positioning unit 141, a synthesizing unit 142, a correcting unit 143, an image feature acquiring unit 144, a projecting unit 145, and an analyzing unit 146. .

  The alignment unit 141 generates alignment parameters used when the synthesizing unit 142 performs the synthesizing process, and the like.

  The combining unit 142 combines a plurality of pieces of motion contrast data generated by the motion contrast data generating unit 112 based on the positioning parameters obtained by the positioning unit 141, and generates a motion contrast image and the like.

  The correction unit 143 performs correction processing such as two-dimensionally or three-dimensionally suppressing projection artifacts occurring in the motion contrast image (projection artifacts will be described in S314 of FIG. 3B).

  From the tomographic image generated by the tomographic image generating unit 111, the image feature obtaining unit 144 obtains image features such as the layer boundaries of the retina and choroid in the fundus Er of the eye E, the position of the fovea, and the center of the optic disc.

  The projection unit 145 projects a motion contrast image in a depth range based on the position of the layer boundary acquired by the image feature acquisition unit 144, and generates a front motion contrast image, and the like.

  The analysis unit 146 performs analysis processing on various images. For example, the analysis unit 146 specifies and measures NPA (avascular region) from the front motion contrast image obtained by the projection unit 145. As shown in FIG. 2, the analyzing unit 146 includes an emphasizing unit 1461, an extracting unit 1462, a measuring unit 1463, and a coordinate converting unit 1464.

  The enhancement unit 1461 performs a blood vessel enhancement process on the subject (the fundus Er of the eye E in the present embodiment). The extracting unit 1462 extracts a boundary line of the NPA based on the blood vessel enhanced image, and extracts the NPA. The measurement unit 1463 generates a mask image of the NPA extracted by the extraction unit 1462, and calculates a measurement value such as the area and volume of the NPA. The coordinate conversion unit 1464 performs coordinate conversion of the front motion contrast image. Examples of coordinate conversion by the coordinate conversion unit 1464 include coordinate conversion from rectangular coordinates to polar coordinates and coordinate conversion from polar coordinates to rectangular coordinates.

(Image processing method by image processing apparatus 100)
Next, an image processing method by the image processing apparatus 100 shown in FIGS. 1 and 2 will be described.
FIG. 3 is a flowchart illustrating an example of a processing procedure of an image processing method by the image processing apparatus 100 according to the first embodiment. Specifically, FIG. 3A is a flowchart illustrating an overall processing procedure of the image processing method according to the first embodiment, and FIGS. 3B to 3D respectively illustrate FIGS. 4 is a flowchart illustrating a detailed processing procedure of steps S310, S350, and S380 illustrated in FIG.

Here, before describing FIG. 3, OCTA imaging will be described with reference to FIG.
FIG. 4 is a diagram illustrating the first embodiment and illustrating OCTA imaging. Specifically, in FIG. 4, the main scanning direction is a horizontal direction (x direction in FIG. 1B), and each position (yi; y) in the vertical sub scanning direction (y direction in FIG. 1B). In (1 ≦ i ≦ n), an example of OCTA imaging in which B scans are performed r times in succession is shown. In OCTA imaging, scanning a plurality of times at the same position is referred to as cluster scanning, and a plurality of tomographic images obtained at the same position is referred to as a cluster. It is known that when motion contrast data is generated in cluster units and the number of tomographic images per cluster (the number of scans at substantially the same position) is increased, the contrast of an OCTA image, which is a type of motion contrast image, is improved. ing.

  The overall processing procedure of the image processing method according to the first embodiment shown in FIG.

<Step S310>
First, in step S310 in FIG. 3A, the image acquisition unit 110 and the image processing unit 140 acquire an OCTA image that is a motion contrast image as a fundus blood vessel image. At this time, the image processing apparatus 100 may acquire a fundus blood vessel image already stored in the storage unit 120. However, in the present embodiment, the image processing apparatus 100 controls the measurement optical system 200-1. An example in which a motion contrast image is generated from an acquired tomographic image and acquired is shown. That is, in the present embodiment, the image acquisition unit 110 and the image processing unit 140 use the plurality of tomographic images obtained by controlling the measurement light to scan the same position of the fundus Er as the subject, and use the plurality of tomographic images to determine the motion contrast. A mode of generating and acquiring a fundus blood vessel image is adopted. In the present invention, the present invention is not limited to this acquisition method, and any other method may be used as long as a fundus blood vessel image is acquired.

FIG. 5 is a diagram illustrating the first embodiment and illustrating an example of information including an image obtained by the processing of the flowchart illustrated in FIG. 3.
Specifically, for example, in the process of step S310, a motion contrast image as shown in FIG. 5A is obtained as a fundus blood vessel image. In the present invention, the present invention is not limited to this, and other fundus blood vessel images such as a fluorescein angiographic image, an indocyanine green (ICG) angiographic image, and an OCT angiographic image are obtained. You may do so.

<Step S320>
Subsequently, in step S320, the enhancement unit 1461 of the analysis unit 146 performs image enhancement processing on the fundus blood vessel image acquired in step S310 while minimizing the influence of noise. A specific example of this processing will be described below.
-S320-1) A "Gaussian Low Pass Filter (LPF)" of "narrow window size" is applied to the fundus blood vessel image acquired in step S310 to generate a Light-Filtered OCTA image (LF-OCTA image). Here, “narrow window size” is three pixels.
-S320-2) A "Gaussian Low Pass Filter (LPF)" of "wide window size" is applied to the fundus blood vessel image acquired in step S310 to generate a Strong-Filtered OCTA image (SF-OCTA image). Here, “size of wide window” is set to 85 pixels.
-S320-3) The LF-OCTA image is divided by the SF-OCTA image to generate a Relative Variation OCTA (RV-OCTA) image.
-S320-4) Binarization of the RV-OCTA image is performed using a specific threshold value RL (Ratio Limit). Here, the threshold RL is set to 1. If each pixel value of the RV-OCTA image is equal to or less than RL, the pixel is set to zero. Otherwise, set the pixel to one.
-S320-5) Morphological filter is applied to the binarized image. Here, open-close processing of kernel size = 3 is performed. This process reduces salt noise on the image and improves the continuity of the blood vessel structure.
By performing the processing in step S320, for example, a fundus blood vessel image shown in FIG. 5B is obtained.

<Step S330>
Subsequently, in step S330, the analysis unit 146 sets a seed point for each of the plurality of NPA candidates in the fundus blood vessel image obtained in step S320. That is, here, there are a plurality of NPA candidates in the fundus blood vessel image, and a plurality of seed points corresponding to the plurality of NPA candidates are set.

  Here, as a method of setting a seed point in step S330, it is assumed that analysis unit 146 sets a plurality of seed points based on information input by the user via input device 300. In this case, specifically, first, the image processing device 100 causes the display device 400 to display an OCTA image that is a fundus blood vessel image, and the user inputs, via the input device 300, each of the plurality of NPA candidates in the OCTA image. Select and specify one point. Then, the analysis unit 146 acquires input information of the user via the input device 300, and sets a seed point for each designated point in the OCTA image that is a fundus blood vessel image. The present invention is not limited to this mode, and a mode in which the analysis unit 146 automatically sets a seed point is also applicable to the present invention.

An example of a mode in which the analysis unit 146 automatically sets a seed point will be described below.
The structural analysis of the OCTA image is performed, and the fovea of the fundus of the eye E is detected.
-A strong low pass filter is applied to the OCTA image to detect a minimum intensity area.
Detection is performed using an analysis result of a fundus image of another modality (for example, an SLO fundus image or a fundus camera image).
Through the above processing, one point in each of the detected regions is automatically set as a seed point.

FIG. 6 illustrates the first embodiment, and is a diagram illustrating an example of information including an image obtained by the processing of the flowchart illustrated in FIG. 3.
FIG. 6A is a diagram illustrating an example of the processing result of step S330. In FIG. 6A, a fundus blood vessel image 601 is an image after the processing in step S320, and an NPA (avascular region) 602 is shown in the fundus blood vessel image 601. The fundus blood vessel image 601 shows the seed points 603 and 604 set in step S330.

  In the subsequent processing, the NPA corresponding to each of the plurality of seed points set in step S330 is specified. Therefore, initially, the target area J is set to 0.

<Step S340>
Subsequently, in step S340, the analysis unit 146 selects a seed point of the region J from the plurality of seed points set in step S330.

<Step S350>
Subsequently, in step S350, the analysis unit 146 performs an NPA specifying process including the seed point using the seed point of the region J selected in step S340.
FIGS. 6B and 6C are diagrams illustrating an example of the processing result of step S350. Specifically, FIG. 6B illustrates an NPA (avascular region) 605 specified based on the seed point 603, and FIG. 6C illustrates an NPA (avascular region) specified based on the seed point 604. NPA (avascular region) 606 is shown.

<Step S360>
Subsequently, in step S360, the analysis unit 146 determines whether or not the specification of the NPA (avascular region) has been completed from all of the plurality of seed points set in step S330. Specifically, if there is a seed point for which the NPA has not yet been specified and there is a next NPA (S360 / YES), the region J is set to J + 1, the process returns to step S340, and the process after step S340 is performed. Perform processing. On the other hand, if the specification of the NPA has been completed from all the seed points (if there is no next NPA) (S360 / NO), the process proceeds to step S370.

<Step S370>
In step S370, the analysis unit 146 determines whether an overlap of the NPA (avascular region) specified in step S350 has occurred. That is, in step S370, it is determined whether or not there is an overlapping common area in a plurality of NPAs (avascular areas) specified from different seed points. Here, the description has been made assuming that the processing of step S370 is performed by the analysis unit 146, but the processing may be performed by the synthesis unit 142, for example.

  FIG. 6D is a diagram illustrating an example of the processing result of step S370. Specifically, FIG. 6D shows an overlapping region (common region) 607 where the NPA (avascular region) 605 and the NPA (avascular region) 606 specified in step S350 overlap. . Here, in the present embodiment, when detecting the overlapping area (common area) 607, the analyzing unit 146, for example, replaces the pixels within the boundary line of the NPA (avascular area) 605 shown in FIG. 6A. A mask image of the NPA 605 is generated by setting pixels as pixels and other pixels as black pixels (the mask image of the NPA 605 may be a mask image generated in step S355 described later), and FIG. The pixels within the boundary of the NPA (avascular region) 606 shown in FIG. 6 are white pixels, and the other pixels are black pixels to generate a mask image of the NPA 606 (this mask image of the NPA 606 is described later in step S355). A generated mask image may be used), and an overlapping area (common area) between the mask image of NPA 605 and the mask image of NPA 606 The range), may take the form of detecting the overlap region (common region) 607. If the result of determination in step S370 is that there is an overlap area (common area) 607 shown in FIG. 6D, it is determined that overlap has occurred (S370 / YES), and the flow proceeds to step S380. On the other hand, when the overlapping area (common area) 607 shown in FIG. 6D does not exist and an independent NPA (avascular area) is specified from a different seed point, it is determined that no overlap has occurred. (S370 / NO), the process proceeds to step S390.

<Step S380>
In step S380, the image processing apparatus 100 performs a process for dealing with an NPA (avascular region) in which an overlap has occurred. In the present embodiment, for example, countermeasures such as presenting a warning display to the user and editing the NPA (avascular region) based on the user's operation input are performed. Thereafter, the process proceeds to step S390.

<Step S390>
In step S390, the image processing unit 140 transmits the NPA information specified by the above processing to the storage unit 120, and the storage unit 120 stores and stores the NPA information. Further, in this step, the display control unit 150 performs a process of causing the display device 400 to display information related to the determination result in step S370. Specifically, the display control unit 150 transmits, for example, the NPA information stored in the storage unit 120 and the fundus blood vessel image acquired by the image processing unit 140 to the display device 400, and the display device 400 The NPA information is superimposed and displayed. Here, as the display method, the NPA (avascular region) for each seed point is determined by at least one of a display mode of displaying in a different color and a display mode of displaying in a different pattern. Can be adopted. The present invention is not limited to this mode, and for example, a mode in which all NPAs are displayed in the same color is also applicable to the present invention. Further, the display control unit 150 displays, on the display device 400, a display mode in which the areas of the plurality of NPAs (avascular regions) identified by the image processing unit 140 are displayed, and a total area obtained by summing the areas of the plurality of NPAs. A display mode to be displayed on the display device 400 can be adopted. In the present embodiment, any display mode may be used as long as at least one area is displayed among the respective areas of a plurality of NPAs (avascular regions) and the total area obtained by adding the respective areas. In this case, these areas are calculated by the measuring unit 1463, for example.
Then, when the processing in step S390 ends, the processing in the flowchart illustrated in FIG. 3A ends.

  Next, an example of a detailed processing procedure in the processing of acquiring a fundus blood vessel image (OCTA image) shown in step S310 of FIG. 3A will be described with reference to FIG.

{Step S311}
First, in step S311 in FIG. 3B, the image processing apparatus 100 acquires a fundus blood vessel image that instructs the tomographic image capturing apparatus 200 based on information input by the user via the input device 300. The photographing conditions of the OCTA image are set.
In particular,
S311-1) Selection or registration of an examination set S311-2) Selection or addition of a scan mode in the selected examination set S311-3) Procedures for setting imaging parameters corresponding to the scan mode are performed.
In the present embodiment, the following settings are made, and OCTA imaging (under the same imaging condition) is repeatedly executed a predetermined number of times while appropriately taking a break.
・ S311-1) Register a Molecular Disease inspection set ・ S311-2) Select OCTA scan mode ・ S311-3) Set the following imaging parameters (A) to (H) (A) Scan pattern: Small Square
(B) Scanning area size: 3 mm x 3 mm
(C) Main scanning direction: horizontal direction (X direction)
(D) Scan interval: 0.01 mm
(E) Fixation light position: fovea (F) Number of B scans per cluster: 4
(G) Coherence gate position: Vitreous body side (H) Default display report type: Single inspection report Note that an inspection set is an imaging procedure set for each inspection purpose (including a scan mode) and OCT acquired in each scan mode. Refers to the default display method for images and OCTA images. As a result, an examination set including the OCTA scan mode set for the macular disease eye is registered, for example, under the name “Macular Disease”. The registered test set is stored in the storage unit 120.

{Step S312}
Subsequently, in step S312, when an instruction to start imaging is input from the user via the input device 300, the image processing apparatus 100 repeatedly starts OCTA imaging according to the imaging conditions set in step S311.

  Specifically, the imaging control unit 130 controls the tomographic imaging apparatus 200 to repeatedly perform OCTA imaging based on the imaging conditions set in step S311. Thus, the tomographic imaging apparatus 200 acquires a corresponding OCT tomographic image. In the present embodiment, the number of times of repeated imaging in OCTA imaging in step S312 is set to three, but the number of times of repeated imaging is not limited to this and can be set to an arbitrary number. Further, in the present embodiment, the present invention is not limited to the case where the photographing time interval between repetitive photographing is longer than the photographing time interval of tomographic images in each repetitive photographing. Included in the form.

  In step S312, the tomographic image capturing apparatus 200 also acquires an SLO image, and executes a tracking process based on the SLO moving image. In the present embodiment, the reference SLO image used in the tracking processing in the repeated OCTA imaging is the reference SLO image set in the first repeated OCTA imaging, and a common reference SLO image is used in all the repeated OCTA imagings. During repeated OCTA imaging, the same setting values are used (not changed) for the selection of the left and right eyes and the execution of tracking processing in addition to the imaging conditions set in step S311.

{Step S313}
Subsequently, in step S313, the image acquiring unit 110 and the image processing unit 140 generate an OCT tomographic image and a motion contrast image based on the information acquired by the tomographic image capturing apparatus 200 in step S312.

  Specifically, first, the tomographic image generation unit 111 performs wave number conversion, fast Fourier transform (FFT), and absolute value conversion (acquisition of amplitude) on the interference signal acquired from the tomographic image capturing apparatus 200, and A tomographic image for the cluster is generated.

  Next, the positioning unit 141 performs positioning of the tomographic images belonging to the same cluster generated by the tomographic image generating unit 111, and performs a superposition process. Next, the image feature acquiring unit 144 acquires layer boundary data from the tomographic images superimposed by the positioning unit 141. Here, in the present embodiment, a variable shape model is used as a method of obtaining a layer boundary, but any known layer boundary obtaining method may be used. In the present embodiment, the acquisition process of the layer boundary is not essential. For example, when the generation of the motion contrast image is performed only in three dimensions and the two-dimensional motion contrast image projected in the depth direction is not generated, The process of acquiring a layer boundary can be omitted.

Next, the motion contrast data generation unit 112 calculates a motion contrast between adjacent tomographic images in the same cluster. In the present embodiment, the decorrelation value Mxy is obtained as the motion contrast based on the following equation (1).
In the equation (1), Axy indicates the amplitude (of the complex number data after the FFT processing) at the position (x, y) of the tomographic image data A, and Bxy indicates the amplitude at the same position (x, y) of the tomographic data B. The amplitude is shown. Further, in the equation (1), 0 ≦ Mxy ≦ 1, and the value becomes closer to 1 as the difference between the two amplitude values increases. The decorrelation calculation processing shown in equation (1) is performed between arbitrary adjacent tomographic images (belonging to the same cluster), and the average of the obtained (number of tomographic images per cluster minus one) motion contrast values is calculated as a pixel. An image having a value is generated as a final motion contrast image.
Here, although the motion contrast is calculated based on the amplitude of the complex number data after the FFT processing, the calculation method of the motion contrast is not limited to this. For example, the motion contrast may be calculated based on the phase information of the complex data, or the motion contrast may be calculated based on both the amplitude and the phase information. Further, for example, the motion contrast may be calculated based on the real part or the imaginary part of the complex data. In the present embodiment, the decorrelation value is calculated as the motion contrast, but the method of calculating the motion contrast is not limited to this. For example, the motion contrast may be calculated based on a difference between two values, or the motion contrast may be calculated based on a ratio of the two values. Further, in the present embodiment, the final motion contrast image is obtained by calculating the average value of the plurality of acquired decorrelation values, but the present invention is not limited to this embodiment. For example, an image having the pixel value of the median value or the maximum value of the acquired plurality of decorrelation values may be generated as the final motion contrast image.

{Step S314}
Subsequently, in step S314, the image processing unit 140 generates a high-contrast motion contrast image synthesized by performing three-dimensional positioning of the group of motion contrast images obtained through repeated OCTA imaging and performing averaging. Specifically, the generation of the motion contrast image is performed by a combining process by the combining unit 142. At this time, the combining process by the combining unit 142 is not limited to the simple averaging. For example, in the synthesizing process, a luminance value of each motion contrast image may be arbitrarily weighted and averaged, or may be an average value such as a median value. The present invention also includes a case where the above-described positioning process performed by the positioning unit 141 is performed two-dimensionally. The combining unit 142 may determine whether or not a motion contrast image inappropriate for the combining process is included, and then perform the combining process excluding the motion contrast image determined to be inappropriate. For example, if the evaluation value (for example, the average value of the decorrelation values or fSNR) of each motion contrast image is out of a predetermined range, it may be determined that the motion contrast image is not suitable for the combination processing.

In the present embodiment, after the synthesis unit 142 synthesizes the motion contrast image three-dimensionally, the correction unit 143 performs a process of three-dimensionally suppressing projection artifacts occurring in the motion contrast image. Here, the projection artifact is a phenomenon in which the motion contrast in the superficial blood vessels of the retina is reflected on the deep side (the deep retina, the outer retina and the choroid), and a high decorrelation value is generated in the deep area where no blood vessels actually exist. Point. The correction unit 143 performs a process of suppressing projection artifacts generated on the three-dimensional motion contrast image. Here, any known projection artifact suppression technique may be used, but in the present embodiment, “Step-down Exponential Filtering” is used. In this “Step-down Exponential Filtering”, projection artifacts are suppressed by executing a process represented by the following expression (2) on each A-scan data on a three-dimensional motion contrast image.
In the equation (2), γ indicates a damping coefficient having a negative value, D (x, y, z) indicates a decorrelation value before the projection artifact suppression processing, and D _E (x, y, z) ) Indicates the decorrelation value after the suppression processing.

  Next, the projection unit 145 projects a motion contrast image in a depth range based on the position of the layer boundary acquired by the image feature acquisition unit 144 in step S313, and generates a front motion contrast image. Here, projection may be performed in an arbitrary depth range. In the present embodiment, two types of front motion contrast images are generated in the depth range of the retinal surface layer and the retinal deep layer. In addition, as the projection method, any one of a maximum intensity projection (MIP) and an average intensity projection (AIP) can be selected. In the present embodiment, the projection is performed by the maximum intensity projection.

{Step S315}
Subsequently, in step S315, the image processing apparatus 100 acquires the acquired image group (SLO image or tomographic image), the imaging condition data of the image group, and the generated three-dimensional and frontal motion contrast image and the accompanying generation condition data. Is stored in the storage unit 120 in association with the examination date and time and information for identifying the eye E.

  In the present embodiment, the processing of step S310 shown in FIG. 3A is executed by performing the processing of steps S311 to S315 described above.

  Next, an example of a detailed processing procedure in the process of specifying an NPA (avascular region) shown in step S350 of FIG. 3A will be described with reference to FIG.

  In the present embodiment, the first edge (end) is found from the seed point in the NPA as the starting point in the fundus blood vessel image represented by the polar coordinates based on the conversion from the rectangular coordinate fundus blood vessel image to the polar coordinate fundus blood vessel image. An example of a method of specifying a boundary line or an area of NPA by using the following will be described.

{Step S351}
First, in step S351 of FIG. 3C, the coordinate conversion unit 1464 of the analysis unit 146 sets the seed point of the region J selected in step S340 as a pole, and based on the pole, the fundus blood vessel image in rectangular coordinates. From the OCTA image is converted to an OCTA image in polar coordinates. The angle step for generating “loss-less image” in polar coordinates can be calculated by the cosine method as in the following equation (3).
The value of the angle step δθ shown in the equation (3) depends on the size S and the pixel size p of the OCTA image. In Expression (3), acos is an arccosine function, and size S is a distance in mm from the center of the OCTA image to the farthest point. For example, when the resolution of a 3 mm × 3 mm OCTA image is 232 pixels × 232 pixels, S = 0.5 · {2.3 mm} 2.1 mm and p = 13 μm, and the angle step δθ is about 0. 35 degrees. Then, the sample angle θ (i) is set to θ (i) = δθ × i. Here, i is the index of the sample. The first sample angle, θ (0) = 0 degrees. FIG. 5C is a diagram showing the processing result of step S351. Here, as an example of the coordinate transformation, a rectangular coordinate to a polar coordinate transformation has been described. However, in the present embodiment, the present invention is not limited to this transformation method. For example, non-orthogonal coordinates (for example, spherical or cylindrical coordinates) may be used. (Coordination). Further, in the present embodiment, an example in which a plurality of NPAs (avascular region) are specified by performing coordinate conversion will be described. However, the present invention is not limited to this embodiment. ) Can be specified, it is not essential to perform the coordinate conversion.

{Step S352}
Subsequently, in step S352, the extraction unit 1462 of the analysis unit 146 extracts the boundary line of the NPA (avascular region) from the polar coordinate OCTA image generated in step S351. Here, starting from the polar point (the bottom end of the OCTA image in polar coordinates), the first edge found is set as an NPA boundary line candidate.
FIG. 7 illustrates the first embodiment, and is a diagram for explaining the process of step S350 in FIG. 3A. Specifically, the edge found first in step S352 of FIG. 3C is represented by a dotted line in FIG.

{Step S353}
Subsequently, in step S353, the extraction unit 1462 of the analysis unit 146 performs a smoothing process on the NPA boundary line candidate extracted in step S352. In the smoothing process here, a “moving median filter” (moving median filter) of “window size” of 10 degrees is used. The purpose of this smoothing process is to delete the spike (spike, singularity) of the boundary line candidate extracted in step S352. These spikes originate from discontinuities in the vasculature and are not suitable as NPA boundaries. The solid line in FIG. 7 indicates the NPA boundary line after the smoothing process in step S353. Here, an example in which “moving median filter” is used as the smoothing process has been described. However, the present invention is not limited to this method. For example, smoothing using a moving average method, a Savitzky-Golay filter, a filter based on a Fourier transform method, or the like. May be used.

{Step S354}
Subsequently, in step S354, the coordinate conversion unit 1464 of the analysis unit 146 converts the image of the NPA boundary line of the polar coordinates extracted in step S353 into rectangular coordinates.

{Step S355}
Subsequently, in step S355, the measurement unit 1463 of the analysis unit 146 generates a mask image of NPA (avascular region). Specifically, the measurement unit 1463 uses the image of the NPA boundary line of the rectangular coordinates generated in step S354 to set pixels within the NPA boundary line to white pixels, and to set other pixels to black pixels, and Generate a mask image.
FIG. 8 illustrates the first embodiment, and is a diagram illustrating an example of an NPA (avascular region) mask image obtained in the process of step S355 in FIG. 3C. In FIG. 8, the white area is the area of the NPA mask image.

  In the present embodiment, the processing of step S350 illustrated in FIG. 3A is performed by performing the processing of steps S351 to S355 described above.

  Next, an example of a detailed processing procedure in the overlap countermeasure processing shown in step S380 of FIG. 3A will be described with reference to FIG.

  FIG. 9 illustrates the first embodiment, and is a diagram for explaining the process of step S380 in FIG. 3A. Hereinafter, the flowchart shown in FIG. 3D will be described with reference to FIG.

{Step S381}
First, in step S381 of FIG. 3D, the display control unit 150 causes the display device 400 to display an NPA (avascular region) in which the fundus blood vessel image overlaps (region overlap) occurs. At this time, in order to draw the user's attention, the display control unit 150 performs control to make the display color of the overlapped NPA different from the color of the peripheral region, for example, and to easily display the NPA. . However, in the present embodiment, the present invention is not limited to this mode, and other display methods for causing the user to pay attention, for example, a display mode in which the entire area of the NPA where the overlap occurs is displayed in a blinking manner, A display mode in which the outline of the NPA blinks is also applicable.

{Step S382}
Subsequently, in step S382, the display control unit 150 causes the display device 400 to display a warning display for notifying the user that overlap (overlap) between NPAs (avascular regions) has occurred.

  FIG. 9A is a diagram illustrating an example of the warning display screen 901 displayed on the display device 400 by the process of step S382. The warning display screen 901 shown in FIG. 9A includes a close button 902 for hiding the warning display, a manual edit button 903, an automatic response button 904, and an ignore button 905. Then, the input device 300 inputs information of the button (902 to 905) selected by the user to the image processing device 100. Then, in the present embodiment, the image processing apparatus 100 terminates the warning display based on the information of the button (902 to 905) selected by the user input through the input device 300, and performs the processing after step S 383. I do. However, the present embodiment is not limited to this method. For example, a key of a keyboard corresponds to each button, and a button selected when a key is detected to be pressed is specified. Etc. may be used. Further, the warning display in step S382 is not limited to the warning display on the warning display screen 901 shown in FIG. 9A, and may include, for example, other layouts and additional buttons.

{Step S383}
Subsequently, in step S383, the display control unit 150 determines whether or not the user has selected the manual edit button 903 on the warning display screen 901 displayed in step S382. If the result of this determination is that the user has selected the manual edit button 903 (S383 / YES), the flow proceeds to step S384. On the other hand, if the result of determination in step S383 is that the user has not selected the manual edit button 903 (S383 / NO), the flow proceeds to step S385.

{Step S384}
In step S384, the display control unit 150 performs an NPA (avascular region) editing process based on information input by the user via the input device 300.

A processing example of step S384 will be described with reference to FIGS. 9B to 9D.
In step S384, the display control unit 150 causes the display device 400 to display the area 906 of the NPAs where the overlap (overlap) has occurred and the editing tool 907, as shown in FIG. 9B. The editing tool 907 allows the user to perform operations such as movement via the input device 300. Here, it is assumed that the user performs an operation input of the editing tool 907 using the mouse of the input device 300, but the present embodiment is not limited to this method. The display control unit 150 moves the editing tool 907 on the area 906 based on an operation input performed by the user using the mouse of the input device 300. Further, the display control unit 150 presents an NPA boundary line 908 based on, for example, a click operation or a drag operation performed by the user using the mouse of the input device 300, as illustrated in FIG. 9C. In addition, the display control unit 150 presents and corrects the outline of the NPA using the editing tool 907 based on, for example, an operation input performed by the user via the input device 300, in addition to the boundary line 908. FIG. 9D is a diagram showing the processing result of the manual editing in step S384. As shown in FIG. 9D, the area 906 between the NPAs where the overlap (overlap) has occurred is divided into two NPAs 909 and 910 by setting the boundary line 908. In the example shown in FIG. 9D, the two NPAs 909 and 910 are identified by hatching that changes the direction of an oblique line. However, the present embodiment is not limited to this example. Alternatively, the identification may be performed using different colors or different patterns. Then, in the processing example of step S384 described with reference to FIGS. 9B to 9D, the display control unit 150 determines that no overlap has occurred in response to an instruction from the user. It is determined whether or not to separate the blood vessel regions.
When the process in step S384 ends, the process proceeds to step S387.

{Step S385}
In step S385, the display control unit 150 determines whether the user has selected the automatic response button 904 on the warning display screen 901 displayed in step S382. If the result of this determination is that the user has selected the automatic response button 904 (S385 / YES), the flow proceeds to step S386. On the other hand, if the result of determination in step S385 is that the user has not selected the automatic action button 904 (S385 / NO), for example, if the ignore button 905 or the close button 902 has been selected, the flow proceeds to step S387.

{Step S386}
In step S386, the analysis unit 146 performs an automatic handling process on the area between the NPAs where the overlap (overlap) has occurred.

A processing example of step S386 will be described with reference to FIG.
FIG. 10 is a diagram illustrating the first embodiment and illustrating the process of step S386 in FIG. 3D. Here, FIG. 10A illustrates a region between NPAs where overlap (overlap) has occurred as a black region.
In step S386-A, the analysis unit 146 performs a blurring process using a low-pass filter such as a Gaussian filter on the area between the NPAs where the overlap (overlap) illustrated in FIG. FIG. 10B shows an example of the processing result of step S386-A. Here, a standard deviation of 5 pixels is used, but for example, a standard deviation of 1 / 25th of the NPA image may be used.
Next, in step S386-B, the analysis unit 146 performs a process of calculating contour lines, for example, for the pixel values of the Gaussian filter result image in step S386-A. FIG. 10C shows an example of the processing result of step S386-B.
Next, in step S386-C, the analysis unit 146 determines the number of regions to be divided based on the number of seed points. Then, the analysis unit 146 sets a position where the pixel value becomes the lowest along the ridge line connecting the peaks of the contour lines obtained in step S386-B as a division point of the region, and sets a valley passing through the division point. Set the line as the boundary of the NPA region. FIG. 10D shows an example of the processing result of step S386-C. Specifically, in FIG. 10D, the display control unit 150 presents the set boundary line, and displays two different NPAs defined by the boundary line by hatching display (different inclination) in which the direction of the oblique line is changed. (Display of a striped pattern). In step S386C, the analysis unit 146 determines the number of regions to be divided based on the seed points. However, the present invention is not limited to this method. For example, the peak position of the high contour obtained in step S386-B may be determined. However, a method of determining the number of regions to be divided or another method may be used.
When the process in step S386 ends, the process proceeds to step S387.

{Step S387}
In step S387, the image processing apparatus 100 determines whether or not the coping process has been completed for the area of all the NPAs where the overlap (duplication) has occurred. As a result of this determination, the coping process has not yet been completed for all the NPA areas where overlap (duplication) has occurred, and if there is a next area (S387 / YES), the process proceeds to step S381. Then, the process returns to step S381. On the other hand, as a result of the determination in step S387, if the coping process for all the NPA regions in which overlap (duplication) has occurred has been completed (S387 / NO), the process of the flowchart in FIG. finish.

  In the present embodiment, the processing of step S380 shown in FIG. 3A is executed by performing the processing of steps S381 to S387 described above.

  Next, various modifications will be described.

(Modification 1)
In the above-described embodiment, an example in which the boundary line of the NPA (avascular region) is extracted in step S352 of FIG. 3C has been described. However, the present invention is not limited to this.

FIG. 11 is a diagram for explaining the first modification.
Specifically, FIG. 11A shows an example in which a fundus blood vessel image (OCTA image) has noise. When the pole 10 is assigned to the noise 20 (two-pixel noise) as in the example shown in FIG. 11A, the noise 20 is converted into the polar coordinate and then converted to the polar coordinate as shown in FIG. Radius = 0 in the fundus blood vessel image (OCTA image). Since the first edge has R = 0, a correct NPA boundary cannot be extracted in this case. In order to avoid this problem, in the first modification, as the process in step S352, it is determined whether or not the pixel is a non-signal pixel (the pixel value is equal to zero), and the following process is performed. Specifically, when searching for the first edge, if Radius = 0 is a signal pixel (a pixel has a signal), the Radius value is sequentially incremented until a non-signal pixel is found. Then, after the non-signal pixel is found, the Radius value is sequentially incremented until a further signal pixel is found, and when the first signal pixel is found, it is set as the NPA boundary. Further, as another processing method, for example, when a pole is set, if there is a signal at that pixel, a method of searching for a nearby non-signal pixel and using the non-signal pixel as a pole may be used. .

(Modification 2)
In the above-described embodiment, an example in which a fixed “window size moving median filter” is used in the boundary line smoothing process in step S353 of FIG. 3C has been described. It is not limited.

  FIG. 12 is a diagram for explaining the second modification. Specifically, in Modification Example 2, an example in which “window size” is determined based on the distance from the pole to the NPA boundary line will be described with reference to FIG.

FIG. 12A shows an NPA (avascular region) of a fundus blood vessel image (OCTA image) in a rectangular coordinate system. In FIG. 12A, a solid line 100 and a solid line 120 are NPA boundary lines. FIG. 12B shows a polar coordinate image of the same image. In the rectangular coordinates shown in FIG. 12A, the gap 110 and the gap 130 have the same size. On the other hand, in the polar coordinate image shown in FIG. 12B, the sizes of the gap 110 and the gap 130 are largely different depending on the distance from the pole 140 (that is, the gap 110 is larger than the gap 130). Here, the “window size” of the “moving median filter” is represented by the following equation (4).
In the equation (4), R is a distance from the pole. In Modification 2, μ = 0.05 mm will be described as an example, but the value is not limited to this value. In addition, for example, the “window size” may be determined by a Savitzky-Golay filter or another expression.

(Modification 3)
In the above-described embodiment, the description has been made using the OCTA image as the fundus blood vessel image, but the present invention is not limited to this. In the present invention, it is also possible to use a fundus blood vessel image other than the OCTA image, for example, a fluorescein angiographic image, an ICGA angiographic image, or another contrast fundus image.

(Modification 4)
In the above-described embodiment, the processing example using the low-pass filter in the automatic coping process in step S386 in FIG. 3D has been described, but the present invention is not limited to this. For example, using the Voronoi diagram (Voronoi diagram) or the Watershed algorithm, respective boundaries of the regions of the NPAs where the overlap (overlap) has occurred may be set. Further, using Morphological image processing, respective boundaries of the areas of the NPAs where the overlap (overlap) has occurred may be set. Further, morphological image processing may be used to repeatedly execute erosion and opening until the regions of the NPAs where the overlap (overlap) has occurred become independent, and set the respective boundaries.

(Modification 5)
In the embodiment described above, the example in which the enhancing unit 1461 determines the parameter for enhancing the image for each NPA candidate in step S320 of FIG. 3A has been described, but the present invention is not limited to this. is not. For example, after analyzing noise characteristics for each of a plurality of NPA candidates or for all NPA candidates, a process of emphasizing an image with unified parameters may be performed.

As described above, the image processing apparatus 100 according to the above-described embodiment or each modification acquires a fundus blood vessel image of the fundus of the eye E to be inspected (S310 in FIG. 3), and acquires the acquired fundus blood vessel image. A plurality of avascular regions corresponding to a plurality of seed points set for are identified (S350 in FIG. 3), and the overlap of the avascular regions in the identified avascular regions is determined ( S370 in FIG. 3). Here, the image acquisition unit 110 and the image processing unit 140 that perform the acquisition processing in step S310 in FIG. 3 are an example of an acquisition unit according to the present invention. Further, the analysis unit 146 that performs the specifying process in S350 in FIG. 3 is an example of a specifying unit according to the present invention. In addition, the analysis unit 146 that performs the determination process of S370 in FIG. 3 is an example of a determination unit according to the present invention.
According to this configuration, a plurality of avascular images are added to the fundus vascular image by taking action on the regions of the avascular regions determined to have overlapped (specifically, the region where the avascular regions are combined). When there are region candidates, these avascular regions can be detected with high accuracy.

[Second embodiment]
Next, a second embodiment will be described. In the following description of the second embodiment, description of matters common to the above-described first embodiment will be omitted, and matters different from the above-described first embodiment will be described.

  In the first embodiment described above, for a plurality of seed points, a fundus blood vessel image (OCTA image) in polar coordinates is generated from each seed point, and each NPA boundary line is extracted, and the NPA (avascular area The example in which the specific processing is performed is described above. However, in this case, since the specified NPA is calculated based on one pole, it is easily affected by image noise. Therefore, in the second embodiment, in order to perform the NPA specifying process more robustly, an additional pole is set as an additional seed point (additional position), and the NPA is determined based on the plurality of poles (a plurality of seed points). An example of generating the weight NPA using the NPA identification results will be described. Then, in the second embodiment, each NPA is specified based on the generated weight NPA, and an overlap countermeasure is performed based on the weight NPA.

  The schematic configuration of the image processing system according to the second embodiment is the same as the schematic configuration of the image processing system 10 according to the first embodiment shown in FIG. The schematic configuration inside the image processing apparatus according to the second embodiment is the same as the schematic configuration inside the image processing apparatus 100 according to the first embodiment shown in FIG.

  FIG. 3A is a flowchart illustrating an overall processing procedure of the image processing method by the image processing apparatus 100 according to the second embodiment. This is basically the same as the flowchart showing the procedure. However, in the second embodiment, step S350 (NPA specifying process) shown in FIG. 3A is different from the first embodiment. Accordingly, in the second embodiment, step S386 (automatic processing) shown in FIG. 3D is different from that of the first embodiment.

  First, in the overall processing of the image processing method according to the second embodiment, the processing of steps S310 to S340 in FIG. Then, using the seed point of the region J selected in step S340, the analysis unit 146 specifies an NPA (avascular region) including the seed point. At this time, in the second embodiment, an initial NPA is specified, and then additional poles are acquired to specify a weight NPA. An image processing method according to the second embodiment will be described with reference to FIG.

FIG. 13 is a flowchart illustrating an example of a processing procedure of an image processing method by the image processing apparatus 100 according to the second embodiment. The process of the flowchart of FIG. 13 is a process performed in the second embodiment instead of step S350 of FIG. 3A.
FIG. 14 illustrates the second embodiment and is a diagram for describing the processing of the flowchart illustrated in FIG. 13. Hereinafter, the flowchart shown in FIG. 13 will be described with reference to FIG.

{Step S1311}
First, in step S1311 of FIG. 13, the coordinate conversion unit 1464 of the analysis unit 146 sets the seed point of the region J selected in step S340 as an initial pole.

{Step S1312}
Subsequently, in step S1312, the coordinate conversion unit 1464 of the analysis unit 146 performs a process of specifying an NPA (avascular region) based on the initial pole set in step S1311. More specifically, the processing in step S1312 is the same as the processing in step S350 in the first embodiment, and a description thereof will not be repeated. FIG. 14A illustrates an example of the overlapped NPA in step S1312.

{Step S1313}
Subsequently, in step S1313, the analysis unit 146 acquires a pole candidate as an additional seed point from the NPA specified in step S1312. Here, an example using reduced NPA will be described. In the present embodiment, as shown in FIG. 14B, the analysis unit 146 first sets the reduction ratio to 0.7, and sets the reduction center as the center of gravity of the region. However, in the present embodiment, the determination of the reduction ratio and the center is not limited to these methods. For example, the reduction ratio may be set to 0.9, and the reduction center may be made random within the area. Next, the analysis unit 146 selects a plurality of points from the outline of the reduced area. Here, for example, 30 equally spaced points on the contour line are considered as pole (center point) candidates. Here, the fixed pole number will be described as an example, but the present invention is not limited to these. For example, the point number may be set according to the size of the contour line. In the present embodiment, the method of setting a plurality of pole candidates is not limited to the above-described method. For example, pole candidates may be set two-dimensionally at equal intervals in the NPA. May be set by other methods.

{Step S1314}
Subsequently, in step S1314, the analysis unit 146 selects one pole candidate from the plurality of pole candidates acquired in step S1313. Here, the pole candidate is referred to as a pole (J) or a J-th candidate. The first candidate is J = 1.

{Step S1315}
Subsequently, in step S1315, the analysis unit 146 performs an NPA (J) specifying process using the pole (J) selected in step S1314. The process in step S1315 is the same as the process in step S350 in the first embodiment, and a description thereof will not be repeated. The NPA specified using the pole (J) is referred to as NPA (J) or intermediate NPA. FIGS. 14C and 14D show an example of the processing result of step S1315. Specifically, FIG. 14C shows the intermediate NPA 1402 specified based on the additional pole 1401, and FIG. 14D shows the intermediate NPA 1404 specified based on the additional pole 1403.

{Step S1316}
Subsequently, in step S1316, the image processing unit 140 stores and stores the intermediate NPA (J) specified using the pole (J) in step S1315 in the storage unit 120.

{Step S1317}
Subsequently, in step S1317, the analysis unit 146 determines whether there is a next pole that has not been processed yet among the plurality of poles acquired in step S1313. If there is a next pole that has not been processed yet (S1317 / YES), it is set as the pole (J + 1), and the process returns to step S1314 to perform the processing from step S1314. On the other hand, if the specification of the intermediate NPA has been completed from all the poles acquired in step S1313 (if there is no next pole) (S1317 / NO), the process proceeds to step S1318.

{Step S1318}
In step S1318, the image processing unit 140 performs a weight NPA combining process using all the intermediate NPAs stored in step S1316. In the present embodiment, as a combining method, the sum of the intermediate NPAs is performed, and the number of NPAs common to each pixel (a position in the NPA) is set as a weight. The combined information is also called weight NPA information. FIG. 14E shows an example of the processing result of step S1318.

{Step S1319}
Subsequently, in step S1319, the image processing unit 140 transmits the weight NPA information specified by the above processing to the storage unit 120, and the storage unit 120 stores and stores the weight NPA information.
When the process in step S1319 ends, the process in the flowchart illustrated in FIG. 13 ends.

  Next, an automatic handling process when an overlap occurs in the second embodiment will be described. This corresponds to step S386 (automatic processing) shown in FIG. 3D, but the details of the processing are different from those of the first embodiment, and therefore will be described below.

  In the second embodiment, the analysis unit 146 performs a process for coping with the overlap of the weight NPAs specified in the flowchart of FIG. Specifically, in the present embodiment, each NPA is specified using the weighted NPA information. Hereinafter, the detailed processing will be described.

In the second embodiment, in step S386 shown in FIG. 3D, the analysis unit 146 performs an automatic handling process of the overlapped weight NPA.
(A) Specifically, first, the analysis unit 146 calculates a contour line for a pixel value of a result image with respect to the overlapped weighted NPA image. FIG. 14E shows an example of the result of the process (A).
(B) Next, the analysis unit 146 specifies the number of regions to be divided based on the number of seed points. Then, along the ridge line connecting the peaks of the contour lines described above, the position where the pixel value becomes the lowest value is set as the division point of the region, and the valley line passing through the division point is set as the boundary line of the NPA region. FIG. 14F shows an example of the result of the process (B). Specifically, in FIG. 14F, two different NPAs obtained in the process (B) are identified by hatching display (display of a stripe pattern having a different inclination) in which the direction of an oblique line is changed. In the process (B), the analysis unit 146 determines the number of regions to be divided based on the seed points. However, the present invention is not limited to this method. For example, the high contour obtained in (A) may be used. A method of specifying the number of regions to be divided or another method may be used for the peak location.
With the above processing, the automatic handling processing in the case where an overlap has occurred in the second embodiment ends.

As described above, in the image processing apparatus 100 according to the second embodiment, in addition to the processing of the first embodiment, additional seed points are added according to the plurality of avascular regions specified in step S1312 in FIG. Are further set (S1313 in FIG. 13), the avascular region is specified using the additional seed point (S1315 in FIG. 13), and the overlap between the avascular regions is determined according to the specification. .
According to this configuration, when there are a plurality of candidate avascular areas in the fundus vascular image, these avascular areas can be detected with higher accuracy.

[Third Embodiment]
Next, a third embodiment will be described. In the following description of the third embodiment, description of matters common to the above-described first and second embodiments will be omitted, and matters different from the above-described first and second embodiments will be omitted. Give an explanation.

  In the above-described first and second embodiments, a fundus blood vessel image (OCTA image) is generated based on information input from the tomographic image capturing apparatus 200, and an NPA (avascular region) is identified using the fundus blood vessel image. The example of performing the processing has been described. On the other hand, in the third embodiment, the NPA specifying process for the fundus blood vessel image generated based on the information input from the tomographic image capturing apparatus 200 using the result of the past NPA specifying process in the subject Will be described.

  The schematic configuration of the image processing system according to the third embodiment is the same as the schematic configuration of the image processing system 10 according to the first embodiment shown in FIG. The schematic configuration inside the image processing apparatus according to the third embodiment is the same as the schematic configuration inside the image processing apparatus 100 according to the first embodiment shown in FIG.

FIG. 15 is a flowchart illustrating an example of a processing procedure of an image processing method performed by the image processing apparatus 100 according to the third embodiment. Specifically, FIG. 15 is a flowchart illustrating an overall processing procedure of the image processing method according to the third embodiment.
FIG. 16 illustrates the third embodiment and is a diagram for describing the processing of the flowchart illustrated in FIG. 15. Hereinafter, the flowchart shown in FIG. 15 will be described with reference to FIG.

<Step S1510>
First, in step S1510 in FIG. 15, the analysis unit 146 acquires past information of a subject to be examined from the storage unit 120. Specifically, the past information of the present embodiment includes a past fundus blood vessel image of the subject, the past imaging parameters (imaging area, imaging pattern, and the like) and processing parameters, the past NPA information, and the like. FIG. 16A shows an example of a past fundus blood vessel image (OCTA image) 1601 and the past NPA 1602 and NPA 1603.

<Step S1520>
Subsequently, in step S1520, the analysis unit 146 acquires an OCTA image that is a motion contrast image as the current fundus blood vessel image. The process in step S1520 is the same as the process in step S310 in FIG. 3A in the first embodiment, and a detailed description thereof will be omitted. However, as the shooting parameters, parameters used in the past shooting from the past information acquired in step S1510 are used. Note that the photographing parameters input by the user via the input device 300 may be used.

<Step S1530>
Subsequently, in step S1530, the enhancement unit 1461 of the analysis unit 146 performs an image enhancement process on the fundus blood vessel image acquired in step S1520, for example, while minimizing the influence of noise. The process in step S1530 is the same as the process in step S320 in FIG. 3A in the first embodiment, and thus a detailed description is omitted. However, the processing parameters used in the past from the past information acquired in step S1510 may be used as the parameters used for the image enhancement processing. FIG. 16A shows an example of a fundus blood vessel image (OCTA image) 1604 that has undergone image enhancement processing in step S1530. Further, an area 1605 corresponding to the NPA is shown in the fundus blood vessel image 1604. Furthermore, in step S1530, edge enhancement processing or contrast enhancement processing may be performed for the alignment processing in the next step S1540.

<Step S1540>
Subsequently, in step S1540, the positioning unit 141 performs positioning of the past fundus blood vessel image of the subject acquired in step S1510 and the current fundus blood vessel image acquired in step S1530. In the present embodiment, the positioning of the fundus blood vessel image is performed using the past NPA information, excluding the past NPA. Although the pattern matching is used as the alignment processing here, other than that, registration using similarity, control point registration, or the like may be used.

<Step S1550>
Subsequently, in step S1550, analysis unit 146 sets a seed point for each of the plurality of NPA candidates in the current fundus vascular image. Since the processing in step 155 is the same as the processing in step S330 in FIG. 3A in the first embodiment, detailed description will be omitted. However, the setting of the seed point in the present embodiment is as follows. First, the analyzing unit 146 specifies the position of the past NPA on the current fundus blood vessel image based on the alignment with the past information obtained in step S1540. FIG. 16C shows the positions of the mask images 1606 and 1607 based on the past NPAs 1602 and 1603 shown in FIG. 16A with respect to the current fundus blood vessel image 1604. Next, the analysis unit 146 specifies the past center of gravity position of the NPA, and sets those barycentric positions as the positions of the seed points. FIG. 16D shows an example of the processing result, and shows that the centroid positions in the past NPA 1602 and NPA 1603 are set as seed points 1608 and 1609, respectively.

<Step S1560>
Subsequently, in step S1560, analysis unit 146 selects a seed point of region J from the plurality of seed points set in step S1550. The process in step S1560 is the same as the process in step S340 in FIG. 3A in the first embodiment, and thus a detailed description is omitted.

<Step S1570>
Subsequently, in step S1570, the analysis unit 146 uses the seed point of the region J selected in step S1560 to perform an NPA specifying process including the seed point. Since the processing in step S1570 is the same as the processing in step S350 in FIG. 3A in the first embodiment, a detailed description will be omitted.

<Step S1580>
Subsequently, in step S1580, the analysis unit 146 determines whether or not the specification of the NPA (avascular region) has been completed from all of the plurality of seed points set in step S1550. Specifically, if there is a seed point for which the NPA has not yet been specified and there is the next NPA (S1580 / YES), the region J is set to J + 1, the process returns to step S1560, and the process after step S1560 is performed. Perform processing. On the other hand, if the specification of the NPA has been completed from all the seed points (if there is no next NPA) (S1580 / NO), the process proceeds to step S150.

<Step S1590>
In step S1590, the analysis unit 146 determines whether the NPA (avascular region) specified in step S1570 has overlapped. That is, in step S1590, it is determined whether there is an overlapping common area in a plurality of NPAs (avascular areas) specified from different seed points. Since the processing in step S1590 is the same as the processing in step S370 in FIG. 3A in the first embodiment, a detailed description will be omitted. As a result of the determination in step S1590, if an overlap has occurred (S1590 / YES), the process proceeds to step S1591, and if no overlap has occurred (S1590 / NO), the process proceeds to step S1592.

<Step S1591>
In step S1591, the image processing apparatus 100 performs a process for dealing with an NPA (avascular region) in which an overlap has occurred. The processing in step S1591 is the same as the processing in step S380 in FIG. 3A in the first embodiment, and thus a detailed description is omitted. Thereafter, the process proceeds to step S1592.

<Step S1592>
In step S1592, the image processing unit 140 transmits the NPA information specified by the above processing to the storage unit 120, and the storage unit 120 stores and stores the NPA information. This step S1592 is the same as the process of step S390 in FIG. 3A in the first embodiment, and thus a detailed description is omitted.
When the processing in step S1592 ends, the processing in the flowchart illustrated in FIG. 15 ends.

  In the process of step S1591 in FIG. 15, the process of dealing with NPA when an overlap occurs is the same as the process of step S380 in FIG. 3A in the first embodiment, but in the present embodiment, It is not limited to this. For example, a coping process using the past information acquired in step S1510 may be performed. The details will be described below.

First, in step S1591-A, the image processing apparatus 100 performs positioning between the past information NPA and the current NPA candidate based on the positioning result in step S1540. FIG. 16C shows an example of the processing.
Next, in step S1591-B, the image processing unit 140 performs expansion processing (enlargement processing) on the past NPAs until the past NPAs are in contact with each other and the current NPA candidate is satisfied. Then, the expansion process is stopped where the expanded past NPA contacts. An example in which the expansion rate in this expansion processing is made to be proportional to the area of each NPA can be considered, but the present invention is not limited to this.
Here, the process of specifying the current NPA candidate based on the past NPA expansion process has been described, but the present embodiment is not limited to this. For example, the current NPA may be specified by using Morphological image processing or Voronoi diagram (Voronoi diagram) method, or another method may be used.

Further, in the present embodiment, the display control unit 150 determines the past avascular region in the region of the avascular regions where the overlap (overlap) occurs (specifically, the region where the avascular regions are combined). On the other hand, a mode in which the changed portion is displayed on the display device 400 so as to be identifiable can be adopted. For example, in a region 1065 shown in white in FIG. 16C, which corresponds to a region between avascular regions in which overlap (overlap) occurs (specifically, a region in which avascular regions are combined). In this embodiment, portions that have changed from mask images 1606 and 1607 corresponding to past avascular regions are displayed so as to be identifiable.
In the present embodiment, an example in which a plurality of NPAs exist in the past NPA information included in the subject's past information has been described. However, the present invention is not limited to this case, and only one NPA information is included in the past NPA information. Applicable even when NPA is present.

As described above, in the image processing apparatus 100 according to the third embodiment, the seed point is set using the past information on the past avascular region in the fundus Er of the subject's eye E as the subject. (S1550 in FIG. 15).
According to this configuration, when there are a plurality of candidates for the avascular region in the fundus vascular image, these avascular regions can be detected with high accuracy.

[Fourth embodiment]
Next, a fourth embodiment will be described. In the following description of the fourth embodiment, description of matters common to the above-described first to third embodiments will be omitted, and matters different from the above-described first to third embodiments will be omitted. Give an explanation.

  In the above-described third embodiment, an example has been described in which a portion that has changed with respect to a past NPA (avascular region) is displayed on the display device 400 in an identifiable manner (for example, FIG. 16C). On the other hand, in the fourth embodiment, an example is shown in which a graph of a temporal change is displayed on the display device 400 using past NPA information and specified NPA information, and the change is clearly indicated. explain.

  The schematic configuration of the image processing system according to the fourth embodiment is the same as the schematic configuration of the image processing system 10 according to the first embodiment shown in FIG. The schematic configuration inside the image processing apparatus according to the fourth embodiment is the same as the schematic configuration inside the image processing apparatus 100 according to the first embodiment shown in FIG.

  FIG. 17 illustrates the fourth embodiment, and is a diagram illustrating an example of information indicating a chronological change of NPA information displayed on the display device 400 in FIG. 1 in a time series.

First, FIG. 17A shows, as a graph 1710, information indicating time-dependent change in the area of one NPA when the horizontal axis represents elapsed time and the vertical axis represents NPA area. It has been described. Here, the unit of the area of the NPA according to the vertical axis in FIG. 17A is mm ² , but is not limited to this in the present embodiment. , A virtual unit may be used, or another unit may be used. The area of the NPA (avascular region) described here is an example of a measurement value in the NPA. The measurement value in the NPA may be, for example, circularity, diameter, radius, or the like, in addition to the area. When a three-dimensional NPA is specified using volume data, the measurement value in the NPA may be, for example, a volume. The elapsed time on the horizontal axis in FIG. 17A indicates a date, but is not limited to this in the present embodiment. For example, other times such as time and quarter may be used. It may be. Further, the elapsed time on the horizontal axis in FIG. 17A indicates a date at a fixed time interval, but is not limited to this in the present embodiment. The dates may be shown unequally in intervals. In the graph 1710 shown in FIG. 17A, each index 1711 indicates the area of the NPA at a certain date (elapsed time) corresponding to the index.

  Next, FIG. 17B illustrates, as a table, information indicating the values (dates and corresponding NPA areas) related to the respective indexes 1711 of the graph 1710 illustrated in FIG. It has been described. In the table shown in FIG. 17B, one set of numerical values of the date and the area corresponds to one index 1711 of the graph 1710 shown in FIG. 17A.

  Next, FIG. 17C shows a fundus blood vessel image including an NPA (avascular region) 1720 corresponding to each set (each set having one date and area) in the table shown in FIG. 17B. Are illustrated in chronological order. In the example of FIG. 17C, the area of the NPA 1720 corresponding to each date does not change as much as the area shown in the table of FIG. 17B. A mode in which a fundus blood vessel image including the NPA 1720 corresponding to the area shown in the table of FIG. Here, the plurality of fundus blood vessel images shown in FIG. 17C are obtained by capturing the first blood vessel image obtained by imaging the subject at the first time and the first time (first examination date and time). It is an example of a plurality of blood vessel images including a second blood vessel image obtained by imaging the subject at a different second time (second examination date and time). In addition, in the plurality of fundus blood vessel images shown in FIG. 17C, one NPA specified in the first blood vessel image and the second blood vessel image is illustrated in time series, but two or more NPAs are used. There may be. That is, at least one NPA among the plurality of NPAs may be specified in each image. At this time, the analysis unit 146, which is an example of the specifying unit, uses the information on the position set in the first blood vessel image (for example, the relative position coordinates of the seed point with respect to the characteristic portion in the image) to generate the second blood vessel image. May be specified. Then, the display control unit 150 may cause the display device 400 to display information indicating a chronological change of the measurement value in the NPA specified in the first blood vessel image and the second blood vessel image in a time series. Thus, for example, the user can easily grasp the change in the progress of the NPA, and thus the diagnostic efficiency can be improved.

  Also, for example, past NPA information is stored in the storage unit 120 in advance, and the graph 1710 in FIG. 17A, the table in FIG. 17B, and the fundus blood vessel image group in FIG. At this time, it is assumed that the display control unit 150 acquires the information from the storage unit 120. However, in the present embodiment, the acquisition of the past NPA information is not limited to the acquisition from the storage unit 120. For example, the past NPA information stored in a network server (not shown) may be acquired. The past NPA information stored in another storage device may be acquired. In the present embodiment, the display control unit 150 displays the graph 1710 in FIG. 17A, the table in FIG. 17B, and the fundus blood vessel image group in FIG. Although a form of displaying on the (same screen) can be adopted, the present invention is not limited to this form. For example, the display control unit 150 may display at least one of the graph 1710 in FIG. 17A, the table in FIG. 17B, and the fundus blood vessel image group in FIG. The present invention can be applied to the present invention as long as it is displayed in the form shown in FIG.

  Further, in the present embodiment, the display method is not limited to the display method shown in FIG. 17, and other arrangements and further information (for example, the number of NPAs) may be displayed. Further, in the example illustrated in FIG. 17, the change with time of one piece of NPA information is shown in time series. However, the present embodiment is not limited to this. For example, the total area of a plurality of NPAs A graph, a table, or a fundus blood vessel image group showing a change with time in a time series may be displayed.

(Modification 6)
In the above-described embodiment, an example is described in which information indicating a temporal change of one piece of NPA information is displayed in time series. However, the present invention is not limited to this. In the sixth modification, a method of displaying information indicating a temporal change of each of a plurality of pieces of NPA information in a time series will be described.

FIG. 18 is a diagram for explaining the sixth modification.
Specifically, FIG. 18A is a graph showing information indicating the time-dependent change in the area of two (plural) NPAs in a time series, where the horizontal axis is the elapsed time and the vertical axis is the NPA area. What is illustrated as 1810 and graph 1820 is described. In the sixth modification, as shown in FIG. 18A, the line types of the graphs 1810 and 1820 are changed for each NPA so that information indicating a time-dependent change in the area of each NPA can be identified. Therefore, they are displayed in different display modes. Specifically, the change over time indicating the area in the first NPA of the two NPAs in time series is illustrated as a solid line graph as illustrated in a graph 1810. Further, the time-dependent change in area of the second NPA of the two NPAs in time series is shown as a dotted line graph as shown in a graph 1820.

  Next, FIG. 18B illustrates a fundus blood vessel image 1830 including NPAs 1831 and 1832 corresponding to the graphs 1810 and 1820 shown in FIG. 18A. Specifically, the NPA that represents the time-dependent change of the area in a time series by the graph 1810 illustrated in FIG. 18A is the first NPA 1831 indicated by the arrow 1841 in the fundus blood vessel image 1830 illustrated in FIG. . 18A is a second NPA 1832 indicated by an arrow 1842 in a fundus blood vessel image 1830 shown in FIG. 18B.

  In Modification 6, the display control unit 150 displays the graph 1810 and the graph 1820 in FIG. 18A and the fundus blood vessel image 1830 in FIG. 18B on one screen (the same screen) of the display device 400. It can take the form. At this time, the display control unit 150 can perform identification display for associating the graphs 1810 and 1820 shown in FIG. 18A with the NPAs 1831 and 1832 shown in FIG. 18B. For example, the display control unit 150 sets the arrow 1841 indicating the first NPA 1831 in FIG. 18B as a line type of the graph 1810 in FIG. A hatched line corresponding to a certain solid line is attached, and an arrow 1842 pointing to the second NPA 1832 in FIG. 18B is a graph 1820 in FIG. A hatching corresponding to the dotted line which is the line type may be displayed by associating the graphs 1810 and 1820 shown in FIG. 18A with the NPAs 1831 and 1832 shown in FIG. 18B. Note that the identification display method is not limited to the mode described here, and the colors of the NPAs 1831 and 1832 shown in FIG. 18A and the corresponding graphs 1810 and 1820 shown in FIG. Identification display may be performed by using the same color. In addition, for example, in the graph 1810 and the graph 1820, an identification display may be performed by attaching an arrow or a circle used for identifying the corresponding NPA, or other identification display methods may be used. Good.

  Note that, in Modification Example 6 illustrated in FIG. 18, an example has been described in which a graph indicating a time-dependent change in the area related to two NPAs is displayed, but the number of NPAs is not limited to two and may be three or more. There may be.

(Modification 7)
In the above-described modified example, an example has been described in which a graph showing time-dependent changes with respect to a plurality of independent NPAs is identified and displayed, but the present invention is not limited to this. Modification 7 describes a method of displaying a time-series graph showing time-dependent changes related to NPA when a plurality of independent NPAs are united with an elapsed time and an overlap occurs.

  FIG. 19 is a diagram for describing Modification 7 and Modification 8. Specifically, FIG. 19A is a diagram for explaining Modification Example 7 of the fourth embodiment, and illustrates a change with time of the area of the NPA when a plurality of independent NPAs are combined with the elapsed time. It is the figure which illustrated the information shown in time series as a graph.

  In the seventh modification, the display control unit 150 may adopt a form in which the graph of FIG. 19A is displayed on the screen of the display device 400. Specifically, FIG. 19A is a graph showing time-series information indicating changes over time in the area of two (plural) NPAs when the horizontal axis represents elapsed time and the vertical axis represents NPA area. What is illustrated as 1910 and graph 1920 is described. Also, a graph 1930 indicated by a double line from the index 1931 to the index 1932 indicates information indicating a time-series change in the area (total area) of the combined NPA when the two NPAs are combined. . In the example of FIG. 19A, the two NPAs are independent before the index 1931, the area of the united NPA is reduced at the time of the index 1932, and the area after the index 1932 is Again, each NPA independently shows a further reduction in its area.

  Note that, in Modification Example 7 shown in FIG. 19A, an example is described in which a graph showing a time-dependent change in the area of two NPAs is displayed, but the number of NPAs is not limited to two but is three. There may be more than one.

(Modification 8)
Subsequently, in a modified example 8, at the time of periodic measurement, a method of displaying a time-series graph showing a temporal change of the NPA when the NPA once disappears (when the area becomes zero) with the elapsed time is displayed. explain.

  FIG. 19B is a diagram for explaining Modification Example 8 of the fourth embodiment, and shows the time lapse of the NPA area when the NPA once disappears (when the area becomes zero) with the lapse of time. It is the figure which illustrated the information which shows a change in time series as a graph.

  In the eighth modification, the display control unit 150 may adopt a form in which the graph of FIG. 19B is displayed on the screen of the display device 400. Specifically, FIG. 19 (b) shows, as a graph, information indicating the time-dependent change in the area of the NPA in a time series when the horizontal axis represents the elapsed time and the vertical axis represents the NPA area. doing. In FIG. 19B, a graph 1940 represents information indicating a time-dependent change in the area of a certain NPA in time series. In FIG. 19B, the index 1941 indicates the time when the area of the certain NPA becomes 0 (the time when the certain NPA disappears). Then, in FIG. 19B, another new NPA is generated again at the time of the index 1951, and the graph 1950 represents information indicating a time-dependent change in the area of the new one NPA in a time series. .

  Note that, in Modification 8 shown in FIG. 19B, an example is described in which a graph showing a time-dependent change in the area of one NPA is displayed in a time-series manner. A graph, a table, or a fundus blood vessel image group showing the change over time of the total area in time series may be displayed.

  As described above, according to the image processing apparatus 100 according to the fourth embodiment, the measured NPA information for each elapsed time is stored, and the time-dependent change of the NPA information is displayed in a time-series manner. The change with time of the NPA information can be presented in an easily understandable manner.

[Other Embodiments]
In each of the embodiments of the present invention described above, an example is described in which the fundus Er of the eye E to be inspected is applied as a subject, but the present invention is not limited to this. In the present invention, any object other than the fundus Er of the subject's eye E can be applied as a subject as long as it can acquire a blood vessel image. Further, in each of the above-described embodiments, an example has been described in which a plurality of “seed points” are applied as a plurality of “positions” set for a blood vessel image of a subject. However, the present invention is not limited to this. For example, a form in which a plurality of “seed regions” are applied by applying a region that is a concept including a point as a plurality of “positions” set with respect to a blood vessel image of a subject is also included in the present invention. is there. That is, in the present invention, the plurality of “positions” may be set in any way to the blood vessel image as long as a plurality of avascular regions can be specified.

The present invention supplies a program for realizing one or more functions of the above-described embodiments to a system or an apparatus via a network or a storage medium, and one or more processors in a computer of the system or the apparatus read and execute the program. This processing can be realized. Further, it can be realized by a circuit (for example, an ASIC) that realizes one or more functions.
This program and a computer-readable storage medium storing the program are included in the present invention.

  It should be noted that each of the above-described embodiments is merely an example of a concrete example for carrying out the present invention, and the technical scope of the present invention should not be interpreted in a limited manner. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features.

100: image processing device, 110: image acquisition unit, 111: tomographic image generation unit, 112: motion contrast data generation unit, 120: storage unit, 130: imaging control unit, 140: image processing unit, 141: positioning unit, 142: synthesis unit, 143: correction unit, 144: image feature acquisition unit, 145: projection unit, 146: analysis unit, 1461: enhancement unit, 1462: extraction unit, 1363: measurement unit, 1464: coordinate conversion unit, 150: Display control unit, 200: tomographic image capturing device, 300: input device, 400: display device

Claims (30)

Specifying means for specifying a plurality of avascular regions corresponding to a plurality of positions set for the blood vessel image of the subject,
Determining means for determining overlap between avascular regions in the plurality of avascular regions,
An image processing apparatus comprising:   The image processing apparatus according to claim 1, further comprising a display control unit configured to display information on a result of the determination by the determination unit on a display device.   The information according to the determination result includes at least one of information indicating that the overlap has occurred and information indicating an area obtained by combining the avascular areas in which the overlap has occurred. 3. The image processing device according to 2. The display control means, when the determination means determines the overlap between the avascular regions, display information indicating that the overlap has occurred on the display device,
The image processing apparatus according to claim 3, further comprising a determination unit that determines whether or not to separate the avascular regions from each other in accordance with an instruction from a user.   The display control unit, when the determination unit determines the overlap between the avascular regions, causes the display device to display information indicating a region in which the overlapped avascular regions are combined. The image processing apparatus according to claim 3 or 4, wherein   6. The image processing apparatus according to claim 5, wherein the display control unit presents a boundary line of the avascular region in a region where the avascular regions in which the overlap occurs are combined.   The image processing according to claim 5, further comprising a calculation unit configured to perform image processing on an area in which the overlapping avascular areas are combined and calculate a boundary line of the avascular area. apparatus.   The calculating means, after performing a blurring process on the combined region of the avascular region where the overlap occurs, calculates a contour line based on a pixel value on the image of the blurring process, and calculates the contour line. The image processing apparatus according to claim 7, wherein the boundary line is calculated using the image data.   The image processing apparatus according to claim 7, wherein the calculation unit calculates the boundary using Morphological image processing as the image processing.   10. The display control device according to claim 6, wherein each of the avascular regions defined by the boundary line is displayed on the display device in a different display mode. An image processing apparatus according to the item.   The display control means may determine at least one area of an area of each of the avascular regions between the avascular regions defined by the boundary line and a total area obtained by summing the areas of the respective avascular regions. The image processing device according to claim 6, wherein the image is displayed on the display device. The specifying means may use information about a plurality of positions set in a first blood vessel image obtained by imaging the subject at a first time, at a second time different from the first time. Specifying a plurality of avascular regions in a second blood vessel image obtained by imaging the subject,
The display control unit is configured to measure at least one avascular region among the plurality of avascular regions identified in the first blood vessel image and the second blood vessel image, and to measure the at least one avascular region. The image processing apparatus according to claim 2, wherein information indicating a change with time of the value over time is further displayed on the display device.   The display control means causes the display device to display at least one of a graph, a table, and the blood vessel image including the avascular region as the information shown in the time series. Item 13. The image processing device according to item 12.   The display control means is for displaying, on the display device, information indicating a chronological change of a measurement value in each of the avascular regions in the plurality of avascular regions on the display device, and 14. The image processing apparatus according to claim 12, wherein information displayed in a time series is displayed in different display modes.   The display control means, when displaying the time-series information in the respective avascular regions on the display device, also displays the blood vessel image including the respective avascular regions on the display device. The image processing apparatus according to claim 14, wherein   The display control means, when the plurality of avascular regions are united together with the elapsed time, displays, as the time-series information, information indicating the temporal change of the measurement value in the united avascular region in a time-series manner. The image processing apparatus according to claim 14, wherein: The blood vessel image further includes a setting unit that sets the plurality of positions,
17. The image processing apparatus according to claim 1, wherein the specifying unit specifies the plurality of avascular regions corresponding to the plurality of positions set by the setting unit.   18. The image processing apparatus according to claim 17, wherein the setting unit sets the plurality of positions according to a designation from a user. The setting means further sets an additional position, which is the position to be added, according to the plurality of avascular regions specified by the specifying means,
19. The image processing apparatus according to claim 17, wherein the specifying unit further specifies the avascular region using the additional position. Further having a storage means for storing past information related to the past avascular region in the subject,
20. The image processing apparatus according to claim 17, wherein the setting unit sets the position using the past information.   21. The image processing apparatus according to claim 20, wherein the setting unit sets the position at a center of gravity of a region corresponding to the past avascular region in the blood vessel image. Further having a display control means for displaying on the display device an area obtained by combining the avascular areas where the overlap occurs,
The display control means displays on the display device a portion that has changed from the past avascular region in a region where the overlapping avascular regions are combined with each other. 22. The image processing device according to 20 or 21.   23. The method according to claim 1, wherein the determining unit determines the overlap using a mask image of the plurality of avascular regions when there is an overlapping common region in the mask image. 2. The image processing device according to claim 1. An avascular region corresponding to a position set in a first blood vessel image obtained by imaging the subject at a first time is specified, and the subject is identified at a second time different from the first time. Specifying means for specifying an avascular region in a second blood vessel image obtained by imaging using information on the set position;
Display control means for displaying, on a display device, information indicating a chronological change of measured values in the specified avascular region in the first blood vessel image and the second blood vessel image in a time series;
An image processing apparatus comprising:   25. The apparatus according to claim 1, further comprising a generating unit configured to generate a motion contrast as the blood vessel image using a plurality of tomographic images obtained by controlling the measurement light to scan the same position of the subject. The image processing device according to any one of claims 1 to 4. The image processing device includes:
Scanning means for scanning the measurement light with respect to the subject; detecting combined light obtained by combining return light from the subject irradiated with the measurement light and reference light corresponding to the measurement light; 26. The image processing apparatus according to claim 25, wherein the image processing apparatus is communicably connected to a photographing apparatus having:   27. The image processing apparatus according to claim 1, wherein the subject is a fundus of a subject's eye. A specifying step of specifying a plurality of avascular regions corresponding to a plurality of positions set for a blood vessel image of the subject,
A judging step of judging an overlap between the avascular regions in the plurality of avascular regions;
An image processing method comprising: An avascular region corresponding to a position set in a first blood vessel image obtained by imaging the subject at a first time is specified, and the subject is identified at a second time different from the first time. A specifying step of specifying an avascular region in the second blood vessel image obtained by imaging using the information on the set position;
A display control step of displaying, on a display device, information indicating a chronological change of the measurement value in the specified avascular region in the first blood vessel image and the second blood vessel image in a time series;
An image processing method comprising:   A program for causing a computer to function as each unit of the image processing apparatus according to any one of claims 1 to 27.