JP2019154715A - Image processing apparatus, image processing method, and program - Google Patents

Abstract

To allow an operator to more easily obtain a more accurate analysis result even if at least two conditions of a plurality of conditions suited for analysis are not satisfied.SOLUTION: An image processing apparatus includes: analysis means which performs analysis of at least one partial region of a motion contrast image of an eye part; and notification means which, when an image displayed on display means in response to an instruction from an operator to show an analysis result is obtained with at least two conditions of a plurality of conditions suited for analysis not being satisfied, makes notification about information of the at least two conditions in the order of priority of the plurality of conditions.SELECTED DRAWING: Figure 12

Description

  The disclosed technology relates to an image processing apparatus, an image processing method, and a program.

  If a tomographic imaging apparatus for an eye such as an optical coherence tomography (OCT) is used, the state inside the retinal layer can be observed three-dimensionally. This tomographic imaging apparatus is widely used in ophthalmic practice because it is useful for more accurately diagnosing diseases. As a form of OCT, for example, there is TD-OCT (Time domain OCT) in which a broadband light source and a Michelson interferometer are combined. This is configured to measure the interference light with the backscattered light acquired by the signal arm by moving the position of the reference mirror at a constant speed to obtain the reflected light intensity distribution in the depth direction. However, since such TD-OCT requires mechanical scanning, high-speed image acquisition is difficult. Therefore, SS-OCT (Swept Source) that spectrally disperses temporally by using a broadband light source as a faster image acquisition method and using SD-OCT (Spectral domain OCT) that acquires an interference signal with a spectrometer or a high-speed wavelength swept light source. OCT) has been developed, and a tomographic image having a wider angle of view can be acquired.

  On the other hand, in the ophthalmic practice, invasive fluorescent fundus contrast examination has been performed so far in order to grasp the pathology of the fundus blood vessel. In recent years, OCT Angiography (hereinafter referred to as OCTA) technology for non-invasively drawing a fundus blood vessel in three dimensions using OCT has come to be used. In OCTA, the same position is scanned a plurality of times with measurement light, and the motion contrast obtained by the interaction between the displacement of red blood cells and the measurement light is imaged. FIG. 4 shows an example of OCTA imaging in which the main scanning direction is the horizontal (x-axis) direction and the B-scan is continuously performed r times at each position (yi; 1 ≦ i ≦ n) in the sub-scanning direction (y-axis direction). ing. In OCTA imaging, scanning a plurality of times at the same position is called cluster scanning, and a plurality of tomographic images obtained at the same position is called a cluster. It is known that when the motion contrast data is generated in cluster units and the number of tomographic images per cluster (the number of scans at substantially the same position) is increased, the contrast of the OCTA image is improved.

  Here, a technique relating to display of a blood vessel analysis map calculated using motion contrast data is disclosed in Patent Literature.

JP 2017-77414 A

  By the way, in order to accurately analyze the blood vessel region and the like on the OCTA image, it is desired that the image indicating the analysis result is an image obtained in a state where a plurality of conditions suitable for the analysis are satisfied. Here, a case is considered where the image indicating the analysis result is an image obtained in a state where at least two of the plurality of conditions suitable for analysis are not satisfied. At this time, for example, there is a possibility that an operator unfamiliar with photographing or analysis may make a diagnosis using an image indicating the analysis result without recognizing that the image is such an image. Also, consider a case where all conditions that are not satisfied are simply transmitted to the operator. At this time, for example, an operator unfamiliar with shooting or analysis may make a diagnosis using an image showing the analysis result without knowing which condition has a relatively large influence on the analysis. There is.

  It is an object of the disclosed technique to make it easier for an operator to deal with a more accurate analysis result even when at least two of a plurality of conditions suitable for analysis are not satisfied. I will.

  The present invention is not limited to the above-described object, and is a function and effect derived from each configuration shown in the embodiment for carrying out the invention, which will be described later. It can be positioned as one.

In order to achieve the above object, one of the disclosed image processing apparatuses includes: an analysis unit that performs analysis on at least a partial region of a motion contrast image of an eye;
A state in which at least two of the plurality of conditions suitable for the analysis are not satisfied in the image displayed on the display unit according to the instruction from the operator and showing the result of analyzing the at least a part of the region In the case of the image obtained in step (b), there is provided an informing means for informing information on the at least two conditions according to the priority order of the plurality of conditions.

  According to one of the disclosed techniques, even when at least two conditions among a plurality of conditions suitable for analysis are not satisfied, an operator can easily cope with the analysis result so as to obtain a more accurate analysis result. be able to.

1 is a block diagram illustrating a configuration of an image processing apparatus according to a first embodiment of the present invention. It is a figure explaining the measurement optical system contained in the image processing system which concerns on embodiment of this invention, and the tomographic imaging apparatus which comprises this image processing system. It is a flowchart of the process which the image processing system which concerns on 1st embodiment of this invention can perform. It is a figure explaining the scanning method of OCTA imaging in the embodiment of the present invention. It is a figure explaining the process performed by S307 of 1st embodiment of this invention. It is a figure explaining the process performed by S308 of 1st embodiment of this invention. It is a figure explaining the selection screen and imaging | photography screen of the reference | standard inspection displayed on a display means in 1st embodiment of this invention. It is a figure explaining the report screen displayed on the display means in S305 and the image processing content in S304 of 1st embodiment of this invention. It is a figure explaining the measurement operation screen displayed on a display means in 1st embodiment of this invention, and the measurement report screen displayed in S308. It is a figure explaining the operation procedure and the image processing content performed when a user corrects the blood vessel region specified in the first embodiment of the present invention. It is a figure explaining the time-dependent change measurement report screen displayed on a display means in S311 of 1st embodiment of this invention. It is a figure explaining the measurement report screen on which the warning message of the first embodiment of the present invention was displayed.

[First embodiment]
One of the image processing apparatuses according to this embodiment is a retina generated from an OCTA superimposed image acquired under substantially the same imaging conditions (conditions for follow-up imaging for follow-up observation) at different dates and times for the same eye to be examined. Using the front motion contrast images of the surface layer and the deep retina, blood vessel region specification and blood vessel density measurement processing are performed. A case will be described in which the composite image and the measurement value obtained by the specifying process and the measurement process are displayed in time series in a plurality of depth ranges. In the present invention, follow-up shooting for the purpose of follow-up observation, image overlay processing, and the like are not essential.

  Hereinafter, an image processing system including an image processing apparatus according to a first embodiment of the present invention will be described with reference to the drawings.

  FIG. 2 is a diagram illustrating a configuration of the image processing system 10 including the image processing apparatus 101 according to the present embodiment. As shown in FIG. 2, in the image processing system 10, an image processing apparatus 101 is connected to a tomographic imaging apparatus 100 (also referred to as OCT), an external storage unit 102, an input unit 103, and a display unit 104 via an interface. It is constituted by.

  The tomographic image capturing apparatus 100 is an apparatus that captures a tomographic image of the eye. In the present embodiment, SD-OCT is used as the tomographic imaging apparatus 100. For example, SS-OCT may be used.

  In FIG. 2A, a measurement optical system 100-1 is an optical system for acquiring an anterior ocular segment image, an SLO fundus image of a subject eye, and a tomographic image. The stage unit 100-2 enables the measurement optical system 100-1 to move back and forth and right and left. The base unit 100-3 incorporates a spectroscope described later.

  The image processing apparatus 101 is a computer that executes control of the stage unit 100-2, alignment operation control, tomographic image reconstruction, and the like. The external storage unit 102 stores programs for tomographic imaging, patient information, imaging data, past examination image data, measurement data, and the like.

  The input unit 103 instructs the computer, and specifically includes a keyboard and a mouse. The display unit 104 includes a monitor, for example.

(Configuration of tomographic imaging system)
The configuration of the measurement optical system and the spectroscope in the tomographic imaging apparatus 100 of the present embodiment will be described with reference to FIG.

  First, the inside of the measurement optical system 100-1 will be described. An objective lens 201 is installed facing the eye 200, and a first dichroic mirror 202 and a second dichroic mirror 203 are arranged on the optical axis. By these dichroic mirrors, the optical path 250 of the OCT optical system, the SLO optical system and the optical path 251 for the fixation lamp, and the optical path 252 for anterior eye observation are branched for each wavelength band.

  The optical path 251 for the SLO optical system and the fixation lamp includes SLO scanning means 204, lenses 205 and 206, a mirror 207, a third dichroic mirror 208, an APD (Avalanche Photodiode) 209, an SLO light source 210, and a fixation lamp 211. ing.

  The mirror 207 is a prism on which a perforated mirror or a hollow mirror is deposited, and separates illumination light from the SLO light source 210 and return light from the eye to be examined. The third dichroic mirror 208 separates the optical path of the SLO light source 210 and the optical path of the fixation lamp 211 for each wavelength band.

  The SLO scanning means 204 scans the light emitted from the SLO light source 210 on the eye 200 to be examined, and includes an X scanner that scans in the X direction and a Y scanner that scans in the Y direction. In this embodiment, since the X scanner needs to perform high-speed scanning, it is a polygon mirror, and the Y scanner is a galvanometer mirror.

  The lens 205 is driven by a motor (not shown) for focusing the SLO optical system and the fixation lamp 211. The SLO light source 210 generates light having a wavelength near 780 nm. The APD 209 detects return light from the eye to be examined. The fixation lamp 211 generates visible light to promote fixation of the subject.

  The light emitted from the SLO light source 210 is reflected by the third dichroic mirror 208, passes through the mirror 207, passes through the lenses 206 and 205, and is scanned on the eye 200 by the SLO scanning unit 204. The return light from the eye 200 to be examined returns to the same path as the illumination light, and is then reflected by the mirror 207 and guided to the APD 209 to obtain an SLO fundus image.

  The light emitted from the fixation lamp 211 passes through the third dichroic mirror 208 and the mirror 207, passes through the lenses 206 and 205, and forms a predetermined shape on the eye 200 by the SLO scanning unit 204, Encourage the patient to fixate.

  In the optical path 252 for observing the anterior eye, lenses 212 and 213, a split prism 214, and a CCD 215 for observing the anterior eye part that detects infrared light are arranged. The CCD 215 has sensitivity at a wavelength of irradiation light for anterior ocular segment observation (not shown), specifically, around 970 nm. The split prism 214 is disposed at a position conjugate with the pupil of the eye 200 to be examined, and the distance in the Z-axis direction (optical axis direction) of the measurement optical system 100-1 with respect to the eye 200 is used as a split image of the anterior eye part. It can be detected.

  The optical path 250 of the OCT optical system constitutes the OCT optical system as described above, and is for taking a tomographic image of the eye 200 to be examined. More specifically, an interference signal for forming a tomographic image is obtained. The XY scanner 216 is for scanning light on the eye 200 and is shown as a single mirror in FIG. 2B, but is actually a galvanometer mirror that performs scanning in the XY biaxial directions.

  Among the lenses 217 and 218, the lens 217 is driven by a motor (not shown) in order to focus the light from the OCT light source 220 emitted from the fiber 224 connected to the optical coupler 219 on the eye 200 to be examined. By this focusing, the return light from the eye 200 is simultaneously incident on the tip of the fiber 224 in the form of a spot. Next, the configuration of the optical path from the OCT light source 220, the reference optical system, and the spectrometer will be described. 220 is an OCT light source, 221 is a reference mirror, 222 is a dispersion compensating glass, 223 is a lens, 219 is an optical coupler, 224 to 227 are connected to the optical coupler and integrated into a single mode, and 230 is a spectroscope. .

  These configurations constitute a Michelson interferometer. The light emitted from the OCT light source 220 is split into the measurement light on the optical fiber 224 side and the reference light on the optical fiber 226 side through the optical coupler 219 through the optical fiber 225. The measurement light is irradiated to the eye 200 to be observed through the above-mentioned OCT optical system optical path, and reaches the optical coupler 219 through the same optical path due to reflection and scattering by the eye 200 to be observed.

  On the other hand, the reference light reaches the reference mirror 221 and is reflected through the optical fiber 226, the lens 223, and the dispersion compensation glass 222 inserted to match the wavelength dispersion of the measurement light and the reference light. Then, it returns on the same optical path and reaches the optical coupler 219.

  The measurement light and the reference light are combined by the optical coupler 219 and become interference light.

  Here, interference occurs when the optical path length of the measurement light and the optical path length of the reference light are substantially the same. The reference mirror 221 is held so as to be adjustable in the optical axis direction by a motor and a driving mechanism (not shown), and the optical path length of the reference light can be adjusted to the optical path length of the measurement light. The interference light is guided to the spectroscope 230 via the optical fiber 227.

  The polarization adjusting units 228 and 229 are provided in the optical fibers 224 and 226, respectively, and perform polarization adjustment. These polarization adjusting units have several portions where the optical fiber is looped. By rotating the loop-shaped portion around the longitudinal direction of the fiber, the fiber can be twisted, and the polarization states of the measurement light and the reference light can be adjusted and matched.

  The spectroscope 230 includes lenses 232 and 234, a diffraction grating 233, and a line sensor 231. The interference light emitted from the optical fiber 227 becomes parallel light through the lens 234, and then is split by the diffraction grating 233 and imaged by the lens 232 on the line sensor 231.

  Next, the periphery of the OCT light source 220 will be described. The OCT light source 220 is an SLD (Super Luminescent Diode) which is a typical low coherent light source. The center wavelength is 855 nm and the wavelength bandwidth is about 100 nm. Here, the bandwidth is an important parameter because it affects the resolution of the obtained tomographic image in the optical axis direction.

  As the type of light source, SLD is selected here, but it is sufficient that low-coherent light can be emitted, and ASE (Amplified Spontaneous Emission) or the like can be used. Near-infrared light is suitable for the center wavelength in view of measuring the eye. Moreover, since the center wavelength affects the lateral resolution of the obtained tomographic image, it is desirable that the center wavelength be as short as possible. For both reasons, the center wavelength was 855 nm.

  In this embodiment, a Michelson interferometer is used as an interferometer, but a Mach-Zehnder interferometer may be used. It is desirable to use a Mach-Zehnder interferometer when the light amount difference is large and a Michelson interferometer when the light amount difference is relatively small according to the light amount difference between the measurement light and the reference light.

(Configuration of image processing apparatus)
The configuration of the image processing apparatus 101 of this embodiment will be described with reference to FIG.

  The image processing apparatus 101 is a personal computer (PC) connected to the tomographic imaging apparatus 100, and includes an image acquisition unit 101-01, a storage unit 101-02, an imaging control unit 101-03, an image processing unit 101-04, and a display. A control unit 101-05 is provided. The image processing apparatus 101 also functions when the arithmetic processing unit CPU executes software modules that implement the image acquisition unit 101-01, the imaging control unit 101-03, the image processing unit 101-04, and the display control unit 101-05. To realize. The present invention is not limited to this. For example, the image processing unit 101-04 may be realized by dedicated hardware such as ASIC, or the display control unit 101-05 is used by a dedicated processor such as GPU different from the CPU. May be realized. Further, the connection between the tomographic imaging apparatus 100 and the image processing apparatus 101 may be configured via a network.

  The image acquisition unit 101-01 acquires signal data of an SLO fundus image and a tomographic image captured by the tomographic image capturing apparatus 100. The image acquisition unit 101-01 includes a tomographic image generation unit 101-11 and a motion contrast data generation unit 101-12. The tomographic image generation unit 101-11 acquires signal data (interference signal) of the tomographic image captured by the tomographic imaging apparatus 100, generates a tomographic image by signal processing, and stores the generated tomographic image in the storage unit 101-02. Store.

  The imaging control unit 101-03 performs imaging control for the tomographic imaging apparatus 100. The imaging control includes instructing the tomographic imaging apparatus 100 regarding the setting of imaging parameters and instructing the start or end of imaging.

  The image processing unit 101-04 includes an alignment unit 101-41, a synthesis unit 101-42, a correction unit 101-43, an image feature acquisition unit 101-44, a projection unit 101-45, and an analysis unit 101-46. The image acquisition unit 101-01 and the synthesis unit 101-42 described above are examples of acquisition units. The synthesizing unit 101-42 synthesizes the plurality of motion contrast data generated by the motion contrast data generating unit 101-12 based on the alignment parameter obtained by the alignment unit 101-41, and generates a synthesized motion contrast image. . The correcting unit 101-43 performs processing to suppress projection artifacts generated in the motion contrast image in two dimensions or three dimensions (the projection artifact will be described in S304). The image feature acquisition unit 101-44 acquires the layer boundary of the retina and choroid, the position of the fovea and the center of the optic disc from the tomographic image. The projection unit 101-45 projects a motion contrast image in a depth range based on the layer boundary position acquired by the image feature acquisition unit 101-44, and generates a front motion contrast image. The analysis unit 101-46 includes an enhancement unit 101-461, an extraction unit 101-462, a measurement unit 101-463, and a comparison unit 101-464, and performs blood vessel region extraction and measurement processing from the front motion contrast image. Here, the analysis unit 101-46 performs analysis on at least one of the first region and the second region including at least a region narrower than the first region in the eye motion contrast image. It is an example. The second area is an example of a sector area. The first area is an example of an area wider than the sector area (for example, the entire image). The analysis unit 101-46 may perform analysis on at least a partial region of the motion contrast image of the eye. The emphasis unit 101-461 executes blood vessel emphasis processing. In addition, the extraction unit 101-462 extracts a blood vessel region based on the blood vessel emphasized image. Furthermore, the measurement unit 101-463 calculates a measurement value such as a blood vessel density using the extracted blood vessel region and blood vessel centerline data acquired by thinning the blood vessel region. The comparison unit 101-464 reads the combined motion contrast image of the same eye and the associated measurement data acquired from different examination dates from the storage unit 101-02 or the external storage unit 102, and generates temporal comparison data.

  The display control unit 101-05 uses the information indicating the type of analysis selected for either the first area or the second area in accordance with an instruction from the operator, It is preferable to display an image indicating the obtained result on the display unit 104. Here, the type of analysis selected is, for example, a blood vessel density relating to the area of the blood vessel region (Vessel Area Density; VAD), a blood vessel density relating to the blood vessel length (Vessel Length Density; VLD), or the like. Further, the image indicating the analyzed result is, for example, a two-dimensional image indicating the result analyzed for at least a partial region of the motion contrast front image. The two-dimensional image indicating the analysis result includes, for example, a VAD map, a VLD map, a VAD sector map, a VLD sector map, and an image in which each of these analysis maps is superimposed on a motion contrast front image. Also, an image in which a plurality of analysis maps of the same type are superimposed or an image in which a plurality of analysis maps of the same type are superimposed on a motion contrast front image may be used. For example, a two-dimensional image in which a VAD sector map is superimposed on a VAD map, a two-dimensional image in which a VAD sector map and a VAD map are superimposed on a motion contrast image, a two-dimensional image in which a VLD sector map is superimposed on a VLD map, and a VLD sector There is a two-dimensional image in which a map and a VLD map are superimposed on a motion contrast image. Note that the timing of executing the analysis by the analysis unit 101-46 may be the timing at which the type of analysis is selected according to an instruction from the operator, or the type of analysis assumed before the type of analysis is selected. The analysis corresponding to may be completed in advance.

  Here, a case is considered where the type of analysis selected for the other after selection for one and the type of analysis selected for the other differ according to instructions from the operator. At this time, the display control unit 101-05 indicates the type of analysis selected with respect to the display of the image indicating the analysis result of one of the display areas of the image indicating the analysis result of the one. It is preferable to execute control to change to display of an image indicating the result of analyzing one and the other using information. As a result, when an image showing the results of the analysis performed on a plurality of analysis regions on the motion contrast image is displayed, it is easy to select the types corresponding to the analysis types for the plurality of analysis regions. Can be configured. For example, when the VAD sector is selected in the Sector button group 903 after the VLD map is selected in the Map button group 902 on the right side of FIG. 9A, the two-dimensional image in which the VLD map is superimposed on the motion contrast front image. It is preferable to execute a control for changing the display of VAD to a two-dimensional image in which the VAD sector map and the VAD map are superimposed on the motion contrast front image. Thus, for example, unlike the two-dimensional image in which the VAD sector map and the VLD map are superimposed on the motion contrast image, the analysis types are not displayed in different states. In other words, for example, since a plurality of analysis maps to be superimposed are surely selected as the same analysis type, the analysis result can be easily confirmed. At this time, as a matter of course, it is preferable to execute control for changing the display of the information indicating the type of analysis. That is, the display control unit 101-05 determines that the type of analysis selected for the other after the selection for one differs from the type of analysis selected for the one according to an instruction from the operator. Preferably, another control is executed to change the display of the information indicating the type of analysis selected for one to the display of the information indicating the type of analysis selected for the other. The information indicating the selected analysis type may be displayed on the display unit 104 so that the selected analysis type can be identified. For example, the Map button on the right side of FIG. There are a group 902 and a Sector button group 903. Further, in follow-up photography for follow-up observation, in a display area in which a plurality of motion contrast images corresponding to a plurality of examination days are displayed in time series, one of the plurality of motion contrast images is displayed. It is preferable that the control executed in this way is executed for other images. In addition, it is preferable that information indicating the type of analysis selected for one image among a plurality of motion contrast images corresponding to a plurality of examination dates is applied to other images. Thus, for example, convenience in follow-up shooting for the purpose of follow-up observation can be improved. The sector area is preferably divided into a plurality of areas, and each area has a value indicating the analysis result of the area (for example, the average value of the area) as the unit of the type of analysis. It is preferable to display in an identifiable state. Here, in the Map button group 902 and the Sector button group 903, when “None” is selected in accordance with an instruction from the operator, the analysis type is preferably in a non-selected state. At this time, it is preferable that an image indicating the analyzed result is not displayed in the display area, and the motion contrast image is displayed in the display area. In addition, when the type of analysis selected for one is changed to non-selection in accordance with an instruction from the operator, the type of analysis selected for the other is preferably not changed.

  Also, consider a case where an analysis is performed on the first region in the motion contrast image of the eye using information indicating the type of analysis selected in accordance with an instruction from the operator. At this time, the display control unit 101-05 is selected when the display of the image indicating the analysis result for the second region in the motion contrast image is selected according to the instruction from the operator. An image indicating the result of analysis for the second region using information indicating the type of analysis is displayed on the display unit 104 in a state of being superimposed on an image indicating the result of analysis for the first region. It may be made to do. Thereby, for example, when VAD is selected as the type of analysis and “On” is selected as the display of the sector area, a two-dimensional image in which the VAD sector map is superimposed on the VAD map is displayed in the display area. Can do. For this reason, for example, since a plurality of analysis maps to be superimposed are surely selected as the same analysis type, the analysis result can be easily confirmed.

  In addition, the display control unit 101-05 is in a state where the image indicating the analysis result displayed on the display unit 104 according to the instruction from the operator does not satisfy at least two conditions among a plurality of conditions suitable for the analysis. In the case of the image obtained in the above, it may be an example of an informing means for informing information on at least two conditions according to the priority order of a plurality of conditions. Thereby, even when at least two conditions among a plurality of conditions suitable for the analysis are not satisfied, the operator can easily cope with it so that a more accurate analysis result can be obtained. Here, the plurality of conditions suitable for the analysis include, for example, a plurality of three-dimensional motion contrast images obtained by controlling a motion contrast image so that measurement light is scanned at the same position of the eye. It is preferable that the condition that the image is an image is included as a condition having a higher priority than other conditions. Thereby, for example, the operator can be advised to confirm the analysis result using the high-quality image. The display control unit 101-05 preferably displays information on at least two conditions on the display unit 104. At this time, the display control unit 101-05 uses the information indicating the type of analysis selected according to the instruction from the operator to display the analysis result on the display unit 104 along with information on at least two conditions. It is preferable to display. Moreover, it is preferable that it is a warning message regarding the high priority condition among at least two conditions. For example, as described in the lower right part of FIG. 12, if a warning message “Average OCTA is recommended in calculating VAD or VLD.” Is displayed at the end of the display screen on which an image indicating the analysis result is displayed. Good. In addition, the warning message may be displayed, for example, in a state where the warning message is superimposed on the image indicating the analysis result, for example, at an edge on the display area where the image indicating the analysis result is displayed. Thus, for example, on the display screen of the display unit 104, an image indicating the analysis result that the operator wants to confirm most is displayed, and the remaining space is effectively used to advise the operator on conditions suitable for the analysis. can do. Of course, the notification means may notify warning messages regarding at least two conditions as information regarding at least two conditions in priority order.

  The external storage unit 102 also stores information on the eye to be examined (patient name, age, sex, etc.), captured images (tomographic images and SLO images / OCTA images), composite images, imaging parameters, blood vessel regions and blood vessel centerlines. Position data, measured values, and parameters set by the operator are stored in association with each other. The input unit 103 is, for example, a mouse, a keyboard, a touch operation screen, and the like, and the operator gives an instruction to the image processing apparatus 101 and the tomographic imaging apparatus 100 via the input unit 103.

  Next, the processing procedure of the image processing apparatus 101 of this embodiment will be described with reference to FIG. FIG. 3 is a flowchart showing a flow of operation processing of the entire system in the present embodiment.

<Step 301>
The operator selects the reference examination for the eye to be examined in which past examination data is stored. Further, the image processing apparatus 101 sets the imaging conditions for OCTA imaging so that the imaging conditions are the same as the selected reference inspection.

  In the present embodiment, on the patient screen 700 as shown in FIG. 7A, the operator selects the subject 701 from the patient list by operating the input unit 103. Further, the operator selects a reference examination (Baseline) in the follow-up examination from the examination list (Examination List) of the subject (702 in FIG. 7). Regarding the selection of the inspection set and the scan mode, the image processing apparatus 101 selects the follow-up inspection set by opening the imaging screen (OCTCapture) with the operator selecting the reference inspection, and the scan mode is the same as the reference inspection. Set to scan mode. In the present embodiment, as shown in the imaging screen 710 in FIG. 7B, “Follow-up” (711) is selected as the inspection set, and “OCTA” mode 712 is selected as the scan mode. Here, the inspection set refers to an imaging procedure (including a scan mode) set for each inspection purpose, and a default display method for an OCT image or an OCTA image.

  The image processing apparatus sets an OCTA image capturing condition to be instructed to the tomographic image capturing apparatus 100. In addition, there are setting items as shown in the following 1) to 7) as imaging conditions regarding individual OCTA imaging, and after setting these setting items to the same values as in the reference examination, a break is appropriately inserted in S302. However, OCTA imaging (with the same imaging conditions) is repeatedly executed a predetermined number of times.

In this embodiment, it is assumed that 7) OCTA imaging in which the number of B scans per cluster is four is repeated three times.
1) Scanning pattern (Scan Pattern)
2) Scanning area size (Scan Size)
3) Main scanning direction (Scanning Direction)
4) Scan interval (Distance between B-scans)
5) Fixation light position (Fixation Position)
6) Coherence gate position (C-Gate Orientation)
7) Number of B scans per cluster (B-scans per Cluster)

<Step 302>
The operator operates the input unit 103 and presses a capturing start (Capture) button 713 in the capturing screen 710 shown in FIG. 7B, thereby starting repeated OCTA capturing under the capturing conditions specified in S301.

  The imaging control unit 101-03 instructs the tomographic imaging apparatus 100 to repeatedly perform OCTA imaging based on the setting instructed by the operator in S301, and the tomographic imaging apparatus 100 acquires a corresponding OCT tomographic image. To do.

  In this step, the tomographic imaging apparatus 100 also acquires an SLO image and executes a tracking process based on the SLO moving image. In this embodiment, the reference SLO image used for the tracking process in repeated OCTA imaging is the reference SLO image set in the first repeated OCTA imaging, and a common reference SLO image is used in all repeated OCTA imaging.

  In the present embodiment, as cluster scanning, B-scan imaging is performed four times continuously in each position in the vertical direction (sub-scanning direction) in the 3 × 3 mm rectangular region with the fovea as the imaging center in the horizontal direction as the main scanning direction. The interval between adjacent cluster scans is 0.01 mm, and an OCT tomographic image is acquired by setting a coherence gate on the vitreous side. In this embodiment, it is assumed that one B-scan is configured with 300 A scan.

During OCTA repeated imaging, in addition to the imaging conditions set in S301, the same set values are used (not changed) for selection of left and right eyes and whether or not tracking processing is executed.

<Step 303>
The image acquisition unit 101-01 and the image processing unit 101-04 generate a motion contrast image based on the OCT tomographic image acquired in S302.

  First, the tomographic image generation unit 101-11 performs wave number conversion, fast Fourier transform (FFT), and absolute value conversion (amplitude acquisition) on the interference signal acquired by the image acquisition unit 101-01, thereby obtaining a tomographic image for one cluster. Generate an image.

  Next, the alignment unit 101-41 aligns the tomographic images belonging to the same cluster and performs an overlay process. The image feature acquisition unit 101-44 acquires layer boundary data from the superimposed tomographic image. In this embodiment, a variable shape model is used as a layer boundary acquisition method, but any known layer boundary acquisition method may be used. The layer boundary acquisition process is not essential. For example, when the motion contrast image is generated only in three dimensions and the two-dimensional motion contrast image projected in the depth direction is not generated, the layer boundary acquisition process can be omitted. The motion contrast data generation unit 101-12 calculates the motion contrast between adjacent tomographic images in the same cluster. In this embodiment, a decorrelation value Mxy is obtained as motion contrast based on the following equation (1).

  Here, Axy represents the amplitude (of the complex number data after FFT processing) at the position (x, y) of the tomographic image data A, and Bxy represents the amplitude at the same position (x, y) of the tomographic data B. 0 ≦ Mxy ≦ 1, and the closer the amplitude value is, the closer the value is to 1. The decorrelation calculation process as in equation (1) is performed between any adjacent tomographic images (belonging to the same cluster), and the average of the obtained (the number of tomographic images per cluster minus 1) motion contrast values is calculated. An image having pixel values is generated as a final motion contrast image.

  Here, the motion contrast is calculated based on the amplitude of the complex data after the FFT processing, but the method of calculating the motion contrast is not limited to the above. For example, motion contrast may be calculated based on phase information of complex number data, or motion contrast may be calculated based on both amplitude and phase information. Alternatively, the motion contrast may be calculated based on the real part and the imaginary part of the complex data.

  In the present embodiment, the decorrelation value is calculated as the motion contrast, but the method of calculating the motion contrast is not limited to this. For example, the motion contrast may be calculated based on the difference between the two values, or the motion contrast may be calculated based on the ratio between the two values.

  Further, in the above description, the final motion contrast image is obtained by obtaining an average value of a plurality of acquired decorrelation values, but the present invention is not limited to this. For example, an image having a median value or a maximum value of a plurality of acquired decorrelation values as pixel values may be generated as a final motion contrast image.

<Step 304>
The image processing unit 101-04 three-dimensionally aligns the motion contrast image group (FIG. 8 (a)) obtained through repeated OCTA imaging, and performs averaging to obtain a high contrast as shown in FIG. 8 (b). A contrast synthetic motion contrast image is generated. Note that the composition process is not limited to simple addition averaging. For example, it may be an average value after arbitrarily weighting the luminance value of each motion contrast image, or an arbitrary statistical value including a median value may be calculated. Further, the case where the alignment process is performed two-dimensionally is also included in the present invention.

  The composition unit 101-42 may determine whether or not a motion contrast image inappropriate for the composition process is included, and then perform the composition process by removing the motion contrast image determined to be inappropriate. For example, when an evaluation value (for example, an average value of decorrelation values or fSNR) is out of a predetermined range with respect to each motion contrast image, it may be determined to be unsuitable for the synthesis process.

  In the present embodiment, after the synthesizing unit 101-42 synthesizes the motion contrast image three-dimensionally, the correction unit 101-43 performs a process of three-dimensionally suppressing projection artifacts generated in the motion contrast image.

  Here, the projection artifact is a phenomenon in which the motion contrast in the retinal surface blood vessels is reflected on the deep layer side (the deep retinal layer, the outer retinal layer, and the choroid), and a high decorrelation value is actually generated in the deep layer region where no blood vessel exists. . FIG. 8C shows an example in which 3D motion contrast data is superimposed and displayed on a 3D OCT tomographic image. A region 802 having a high decorrelation value is generated on the deep side (photocell layer) of the region 801 having a high decorrelation value corresponding to the retinal surface blood vessel region. Although the blood vessel originally does not exist in the photoreceptor cell layer, the flickering of the blood vessel shadow generated in the surface layer of the retina is reflected in the photoreceptor cell layer, and the luminance value of the photoreceptor cell layer changes, thereby generating an artifact 802.

  The correcting unit 101-43 executes a process for suppressing the projection artifact 802 generated on the three-dimensional synthesized motion contrast image. Although any known projection artifact suppression technique may be used, Step-down Exponential Filtering is used in the present embodiment. In Step-down Exponential Filtering, projection artifacts are suppressed by executing the processing represented by Expression (2) for each A scan data on the three-dimensional motion contrast image.

Here, γ is a negative attenuation coefficient, D (x, y, z) is a decorrelation value before the projection artifact suppression process, and D _E (x, y, z) is a decorrelation value after the suppression process. Represents.

  FIG. 8D shows an example in which the three-dimensional synthesized motion contrast data (gray) after the projection artifact suppression processing is superimposed and displayed on the tomographic image. It can be seen that the artifacts seen on the photoreceptor layer before the projection artifact suppression processing (FIG. 8C) were removed by the suppression processing.

  Next, the projection unit 101-45 projects a motion contrast image in a depth range based on the layer boundary position acquired by the image feature acquisition unit 101-44 in S303, and generates a front motion contrast image. In this embodiment, two types of two-dimensional synthesized motion contrast images are generated in the depth range of the retina surface layer and the deep retina. In addition, as a projection method, either maximum value projection (MIP) or average value projection (AIP) can be selected, and in this embodiment, projection is performed with maximum value projection.

  Finally, the image processing apparatus 101 uses the acquired image group (SLO image or tomographic image), the imaging condition data of the image group, and the generated motion contrast image and the generated generation condition data associated with the examination date and information for identifying the eye to be examined. And stored in the external storage unit 102.

<Step 305>
The display control unit 101-05 causes the display unit 104 to display the tomographic image generated in step S303, the motion contrast image combined in step S304, the shooting conditions, and the combination conditions.

  FIG. 8E shows an example of the report screen 803. In this embodiment, an SLO image and a tomographic image, a front motion contrast image in a different depth range generated by combining and projecting in S304, and a corresponding front OCT image are displayed.

  The projection range of the front motion contrast image can be changed by the operator selecting from a predetermined depth range set (805 and 809) displayed in the list box. Also, the type and offset position of the layer boundary used to specify the projection range can be changed from the user interface such as 806 or 810, or the layer boundary data (807 and 811) superimposed on the tomographic image can be operated from the input unit 103. To change the projection range.

  Furthermore, the presence or absence of an image projection method or projection artifact suppression processing may be changed by selecting from a user interface such as a context menu.

<Step 306>
The operator uses the input unit 103 to instruct the start of the OCTA measurement process.

  In the present embodiment, double-clicking on the motion contrast image on the report screen 803 in FIG. 8E shifts to an OCTA measurement screen as shown in FIG. The motion contrast image is enlarged and displayed, and the type of image projection method (maximum value projection (MIP) or average value projection (AIP)), the projection depth range, and whether or not to perform the projection artifact removal process are selected as appropriate. Next, when the operator selects an appropriate item from the map button group 902, the Sector button group 903, and the selection button 904 displayed via the measurement button 904 on the right side of FIG. The target area is selected, and the analysis unit 101-46 starts the measurement process. At the time when the OCTA measurement screen is displayed, no measurement target area is set (none is selected in both the Map button group 902 and the Sector button group, and the selection screen 905 is not displayed).

As a type of measurement processing, in this embodiment, from the Map button group 902 or the Sector button group 903 i) None (not measured)
ii) VAD (blood vessel density calculated based on the area occupied by blood vessels)
iii) VLD (blood vessel density calculated based on the total length of blood vessels)
Select one of the following. For example, Fractal Dimension that quantifies the complexity of the vascular structure, or Vessel Diameter Index that represents the distribution of the vascular diameter (distribution of vascular aneurysm or stenosis) may be selected. From the selection screen 905 displayed via the Measurement button 904, i) distance measurement between any two points ii) area measurement of an avascular region iii) VAD
iv) VLD
You can select either of these.

  In this embodiment, the entire image can be set by selecting a region other than None from the Map button group 902 as the measurement processing target region. By selecting a region other than None from the Sector button group 903, a sector region (fixation) A minimum circular area and a sector area defined by a plurality of concentric circles having a center at the position and different radii and a plurality of straight lines having different angles passing through the fixation position can be set. Further, after selecting a desired measurement type from a selection screen 905 displayed via the Measurement button 904, the input unit 103 is used to select a boundary position of an arbitrary shape on the motion contrast image (the gray line portion 1001 in FIG. 10B). ) And press the OK button, an arbitrary shape measurement target area can be set. A numerical value shown in the area indicates a value measured in the area (in this case, a VAD value). When the region of interest is manually set, a round control point indicating that the border position can be edited is superimposed on the designated border position (gray line portion 1001), and when the OK button is pressed, the round control point is displayed. The point disappears and only the gray line portion 1001 is displayed, and the boundary position is in an uneditable state.

  In this embodiment, by selecting VAD from the Map button group 902 and the Sector button group 903, a VAD map (measurement type is VAD, a measurement target area is the entire image) and a VAD sector map (measurement type is VAD, measurement). A case where the target area is a sector area corresponding to the ETDRS grid will be described.

Here, VAD is an abbreviation for Vessel Area Density and is a blood vessel density (unit:%) defined by the ratio of the blood vessel region included in the measurement target. That is, VAD is an example of the blood vessel density related to the area of the blood vessel region specified in the motion contrast image. VLD is an abbreviation for Vessel Length Density, and is a blood vessel density defined by the sum of the lengths of blood vessels contained in a unit area (unit: mm ⁻¹ ). That is, the VLD is an example of the blood vessel density related to the length of the blood vessel region specified in the motion contrast image. VAD and VLD are examples of parameters related to a blood vessel region specified in a motion contrast image. The parameters relating to the blood vessel region include the area of the blood vessel region, the blood vessel length, the curvature of the blood vessel, and the like. Here, the blood vessel density is an index for quantifying the occlusion range of the blood vessel and the degree of density of the blood vessel network, and VAD is most often used. However, in VAD, the contribution of the large blood vessel region to the measurement value increases, so if you want to measure by paying attention to the pathology of capillaries like diabetic retinopathy (as an index more sensitive to capillary blockage) VLD is used. In addition to the parameters related to the blood vessel region, the types of analysis include, for example, parameters related to a non-blood vessel region (Non Perfusion Area; NPA) specified in the motion contrast image. The parameters related to the avascular region include the area and shape (length and circularity) of the avascular region. Further, the present invention is not limited thereto, and for example, Fractal Dimension for quantifying the complexity of the vascular structure, or Vessel Diameter Index representing the distribution of the vascular diameter (distribution of vascular aneurysm or stenosis) may be measured.

  A plurality of measurement target areas may be set for the same motion contrast image. Examples of the plurality of measurement target areas include at least two of the entire image, the sector area, and the arbitrarily shaped area, two or more depth ranges, or a combination thereof. When different types of measurement are selected for the plurality of measurement target regions, the analysis type selected last (for the specified analysis target region) is also applied to other measurement target regions. You may comprise so that it may measure, after applying in conjunction and to display the measurement result. For example, when an instruction to change to a VLD map is made with the VAD map and VAD sector map selected, the VLD sector map is automatically selected, and VLD measurement for the entire image and VLD measurement for the ETDRS sector region are executed. By such an interlocking selection operation, it is possible to prevent the operator from misunderstanding and confusing the display contents due to different types of measurement values superimposed on the same image. Note that among the types of measurement, None (not measured) is selected independently in each measurement target region (when “None” is selected for a certain measurement target region, the selection of “None” is other than Not applied in conjunction with the measurement target area). Further, the present invention is not limited to applying the measurement type that is selected at the end to all the analysis target areas in conjunction with the measurement / display. For example, linked to multiple measurement target areas in the in-plane direction and not linked to multiple target areas in the depth direction, or vice versa (linked to multiple measurement target areas in the in-plane direction). Alternatively, the measurement may be adaptively applied so as to be linked to a plurality of target areas in the depth direction), and a corresponding measurement result may be displayed.

  Next, the analysis unit 101-46 performs image enlargement and top hat filter processing as preprocessing of measurement processing. By applying the top hat filter, luminance unevenness of the background component can be reduced. In this embodiment, it is assumed that the image is enlarged using Bicubic interpolation so that the pixel size of the synthesized motion contrast image is about 3 μm, and the top hat filter process is performed using a circular structural element.

<Step 307>
The analysis unit 101-46 performs a blood vessel region specifying process. In the present embodiment, the enhancement unit 101-461 performs blood vessel enhancement processing based on a Hessian filter and edge selective sharpening. Next, the extraction unit 101-462 performs binarization processing using two types of blood vessel emphasized images, and specifies a blood vessel region by performing shaping processing.

  Details of the blood vessel region specifying process will be described in S510 to S560.

<Step 308>
The measuring units 101 to 463 measure the blood vessel density of the single examination image based on the information related to the measurement target region designated by the operator. Subsequently, the display control unit 101-05 displays the measurement result on the display unit 104.

  There are two types of indexes of blood vessel density, VAD and VLD, and in the present embodiment, a procedure for calculating VAD will be described as an example. The procedure for calculating the VLD will also be described later.

  When the operator inputs an instruction to correct the blood vessel region or the blood vessel centerline data from the input unit 103, the analysis unit 101-46 uses the position information designated by the operator via the input unit 103 to select the blood vessel. Correct the area or blood vessel centerline data and recalculate the measured values.

  When measurement is performed without satisfying the predetermined condition in this step, a message (warning display) indicating that the display control unit 101-05 should perform measurement in a state where the predetermined measurement condition is satisfied is displayed on the display unit. To 104.

  Details of the VAD measurement process will be described in S810 to S830, and details of the VLD measurement process will be described in S840 to S870.

<Step 309>
The analysis unit 101-46 obtains an instruction from the outside as to whether or not to correct the blood vessel region and the blood vessel centerline data specified in S307. This instruction is input by the operator via the input unit 103, for example. If the correction process is instructed, the process proceeds to S308. If the correction process is not instructed, the process proceeds to S310.

<Step 310>
The comparison unit 101-464 executes a change with time measurement (Progression measurement) process. FIG. 11 shows an example of the Progression measurement report. By designating the Progression mode tab 1101, the report screen is displayed, and the time-dependent change measurement process based on the measurement type and measurement target area selected in S306 is started. In the present embodiment, four examinations are automatically selected in order from the newest examination date as the Progression measurement target image. However, the present invention is not limited to this. For example, an image having the same inspection interval may be selected from the image of the oldest inspection date, the image of the latest inspection, and the images taken on both inspection days. Here, the selection conditions for the measurement target image are: i) images with the same fixation position in descending order of priority ii) motion contrast images with a large number of tomographic images acquired at substantially the same position (for example, 4 or more) or equivalent It is a composite motion contrast image obtained by performing OCTA overlay processing so as to become a motion contrast image, and an image that satisfies the above selection conditions is selected with priority.

  Note that the measurement target image is not limited to this. For example, the selection button 1107 in FIG. 11 may be selected to display a selection screen, and may be selected from an image list displayed on the selection screen.

  Next, the comparison unit 101-464 acquires from the external storage unit 102 data related to the past examination image and measurement values corresponding to the measurement contents of the single examination performed in S 309. Further, the alignment unit 101-41 performs alignment between the single inspection image measured in S308 and the past inspection image, and the measurement data (measurement value, measurement value map, difference map, At least one of the trend graphs). The difference map is generated by designating a “Show Difference” check box as shown by 1108 in FIG. 11, for example.

<Step 311>
The display control unit 101-05 displays on the display unit 104 a report related to progression measurement performed in S310.

  In the present embodiment, the VAD map and VAD sector map measured on the retina surface layer are superimposed and displayed on the upper stage of the Progression measurement report shown in FIG. 11, and the VAD map and VAD sector map measured on the deep retina are superimposed and displayed on the lower stage. This makes it possible to list and grasp time-series changes in vascular pathology at different depth positions. In the VAD measurement result displayed in time-series juxtaposition in FIG. 11, an initial lesion occurs in the deep retina, A list of vascular occlusions spreading from the fovea to the parafovea can be seen.

  Further, regarding each measurement target image, the display unit 104 may display information on the number of tomographic images at substantially the same position, information on the execution conditions of the OCTA overlay processing, and information on the evaluation value (image quality index) of the OCT tomographic image or motion contrast image. Good. In FIG. 11, a mark (“Averaged OCTA” in the upper left) indicating that the OCTA overlay processing has been performed is displayed. The arrow 1104 displayed at the top of FIG. 11 is a mark indicating that the examination is currently selected, and the reference examination (Baseline) is the examination selected during Follow-up imaging (one of FIG. 11). Image of the leftmost image). Of course, a mark indicating the reference inspection may be displayed on the display unit 104. If the “Show Difference” check box 1108 is designated in S310, the measurement value distribution (map or sector map) for the reference image is displayed on the reference image, and the reference image is displayed in an area corresponding to the other inspection date. The difference measurement value map with the measurement value distribution calculated with respect to is displayed. As a measurement result, a trend graph (a graph of measured values with respect to an image of each inspection date obtained by measurement with time) may be displayed on a report screen. A regression line (curve) of the trend graph or a corresponding mathematical expression may be displayed on the report screen.

  In this embodiment, images and measurement values of the retina surface layer and the deep retina are displayed in time series as different depth ranges. However, the present invention is not limited to this. For example, four types of depth ranges of the retina surface layer, the deep retina layer, the outer retina layer, and the choroid can be used. Images and measurement values may be displayed in time series.

  Alternatively, measurement results of different indexes may be juxtaposed and displayed in time series. For example, a time series display of the VAD map may be performed in the upper stage, and a time series juxtaposition display of the VLD map (or the shape value of the avascular region) may be performed in the lower stage.

  Note that the projection depth range in the case of time-series juxtaposed display is the same as that of the user interfaces (805, 806, 809, and 810) in FIG. Can be changed using the user interface. Similarly, the projection method (MIP / AIP) and the projection artifact suppression processing may be changed by, for example, selecting from a context menu. Furthermore, by changing the items related to the measurement type and the measurement target area from the shortcut menu to different values, the measurement type and the measurement target area can be changed and remeasured.

  For example, an item corresponding to the Map button group 902 and an item corresponding to the Sector button group 903 in FIG. 9A are displayed on the shortcut menu, and one item each (for example, “VLD Map” and “VLD Sector”). Select). As in the case of S306, when a plurality of measurement target areas are selected and the measurement type for one area is changed, the same measurement type is applied to the other area in conjunction with it, Measurement is performed. Further, an instruction not to set a measurement target area (select “None”) is not applied in conjunction with the other measurement target area.

  Further, the external storage unit 102 stores the motion contrast image displayed on the display unit 104, the binary image, the measurement value, and the measurement map related to the blood vessel region and the blood vessel center line generated by the extraction unit 101-462 and the measurement unit 101-463. You may output and save as a file. In order to facilitate comparative observation, it is desirable that the motion contrast image to be output as a file, the binary image related to the blood vessel region and the blood vessel center line, and the image size and pixel size of the measurement value map are the same.

  In addition, a warning message is displayed when displaying the measurement result performed in a state not satisfying the predetermined condition on the measurement report screen in the same manner as in the case of measurement for a single inspection (details will be described in S830). It's okay. Note that the recommended condition is not limited to the condition shown in S830. For example, “the difference in the number of acquired tomographic images or the composition of motion contrast images or the image quality index value at substantially the same position between the selected temporal change measurement target images. May be set as a recommended condition, and a warning may be displayed when the condition is not satisfied.

<Step 312>
The image processing apparatus 101 obtains an instruction from the outside as to whether or not to end a series of processing from S301 to S312. This instruction is input by the operator via the input unit 103. If an instruction to end the process is acquired, the process ends. On the other hand, if an instruction to continue the process is acquired, the process returns to S302, and the process for the next observing eye (or reprocessing for the same observing eye) is performed.

  Further, details of the process executed in S307 will be described with reference to the flowchart shown in FIG.

<Step 510>
The enhancement unit 101-461 performs blood vessel enhancement filter processing based on the eigenvalues of the Hessian matrix on the motion contrast image that has been subjected to the preprocessing of Step 306. Such enhancement filters are generically referred to as Hessian filters, and examples thereof include a Vesselness filter and a multi-scale line filter. In this embodiment, a multi-scale line filter is used, but any known blood vessel enhancement filter may be used.

  The Hessian filter smoothes the image with a size suitable for the diameter of the blood vessel to be emphasized, calculates a Hessian matrix having the second derivative of the luminance value as an element in each pixel of the smoothed image, and the eigenvalue of the matrix The local structure is emphasized based on the magnitude relation of. The Hessian matrix is a square matrix as given by Equation (3), and each element of the matrix is, for example, the second derivative of the luminance value Is of the image obtained by smoothing the luminance value I of the image as shown in Equation (4). Represented by value. In the Hessian filter, when “one of the eigenvalues (λ1, λ2) is close to 0 and the other is negative and has a large absolute value” of such a Hessian matrix, it is regarded as a linear structure and emphasized. This is to emphasize pixels that have the characteristics of a blood vessel region on a motion contrast image, that is, pixels that satisfy the characteristic that “the luminance change is small in the traveling direction and the luminance value decreases greatly in the direction orthogonal to the traveling direction” as a linear structure. It corresponds to.

  Since the motion contrast image includes blood vessels of various diameters from capillaries to fibrillation veins, a line-enhanced image is generated using a Hessian matrix for an image smoothed by a Gaussian filter at a plurality of scales. Next, as shown in the equation (5), the square of the smoothing parameter σ of the Gaussian filter is multiplied as a correction coefficient and then combined by the maximum value calculation, and the combined image Ihesian is used as the output of the Hessian filter.

  The Hessian filter has the advantage of being resistant to noise and improving the continuity of blood vessels. On the other hand, since the maximum blood vessel diameter included in the image is often unknown in advance, the emphasized blood vessel region tends to become thick, especially when the smoothing parameter is too large for the maximum blood vessel diameter in the image. There are drawbacks.

  Therefore, in the present embodiment, the blood vessel region is prevented from becoming too thick by calculating an image in which the blood vessel region is emphasized by another blood vessel enhancement method described in S530.

<Step 520>
The extraction unit 101-462 binarizes the blood vessel emphasized image (hereinafter, referred to as a Hessian emphasized image) generated by the Hessian filter generated in S510.

  When binarization is performed using luminance statistics (average value, median, etc.) of a Hessian-weighted image as a threshold value, for example, in the optic nerve head, the threshold value increases due to the influence of the high-intensity region of the large blood vessel, and the capillaries around the teat RPC (Radial Peripillary Capillary) may be insufficiently extracted. Also, in a region where an avascular region is likely to expand, such as the deep retina, the threshold is too low and the avascular region may be erroneously detected as a blood vessel.

  Therefore, in this embodiment, the threshold value is prevented from becoming too high in the optic nerve head by using the average value of the Hessian-weighted image synthesized with only the low-scale weighted image as a threshold value. Further, by setting a lower limit value of the threshold value, erroneous detection in an avascular region is suppressed.

Here, the method for preventing the threshold from becoming too high in the optic papilla is not limited to binarization using the statistical value of the low-scale enhanced image as the threshold. For example, if the luminance value on the Hessian-weighted image is equal to or higher than a predetermined value, the same effect can be expected even if the average value when it is regarded as the predetermined value is binarized as a threshold value. Alternatively, for example, a robust estimation value such as M-estimator may be binarized using a threshold value.
In this embodiment, since the synthesized motion contrast image is enhanced with the Hessian filter, the continuity of the binarized blood vessel region is further improved as compared with the case where the single motion contrast image is enhanced with the Hessian filter.

<Step 530>
The enhancement unit 101-461 performs edge selection sharpening processing on the combined motion contrast image after applying the top hat filter generated in S306.

  Here, the edge selective sharpening processing refers to performing weighted sharpening processing after setting a large weight to the edge portion of the image. In the present embodiment, edge selection sharpening processing is performed by performing unsharp mask processing using an image obtained by applying a Sobel filter as a weight to the synthesized motion contrast image.

  When sharpening processing is performed with a small filter size, a blood vessel region can be specified more accurately when a thin blood vessel edge is emphasized and binarized (a phenomenon in which the blood vessel region becomes thicker can be prevented). On the other hand, particularly in the case of a motion contrast image with a small number of tomographic images at the same imaging position, there is a large amount of noise. Therefore, noise enhancement is suppressed by performing edge selective sharpening.

<Step 540>
The extraction unit 101-462 binarizes the sharpened image to which the edge selective sharpening process generated in S530 is applied. Although any known binarization method may be used, in this embodiment, binarization is performed using a luminance statistical value (average value or median value) calculated in each local region on the sharpened image as a threshold value.

  However, since the threshold value set in the large blood vessel region of the optic nerve head is too high, and many holes are formed in the blood vessel region on the binary image, the upper limit value of the threshold value is set particularly in the optic nerve head. Prevent the threshold from becoming too high.

  Similarly to the case of S520, when the proportion of the avascular region in the image is large, the threshold is too low, and a part of the avascular region may be erroneously detected as a blood vessel. Therefore, erroneous detection is suppressed by setting a lower limit value of the threshold.

  Since the synthesized motion contrast image is sharpened by edge selection as in the case of S520, a noise-like false detection region when binarized compared to a case where a single motion contrast image is sharpened by edge selection is further increased. Can be reduced.

<Step 550>
The extraction unit 101-462 selects the blood vessel candidate region when both the luminance value of the binary image of the Hessian-weighted image generated in S520 and the luminance value of the binary image of the edge selective sharpened image generated in S540 are greater than zero. Extract as This calculation process suppresses both the region overestimating the vessel diameter seen in the Hessian-weighted image and the noise region seen on the edge-selective sharpened image so that the blood vessel boundary position is accurate and the blood vessel continuity is improved. A good binary image can be acquired.

  In addition, since both binary images are binary images based on the combined motion contrast image, noise-like false detection areas when binarized are reduced compared to a binary image based on a single motion contrast image. In particular, the continuity of the capillary region is improved. In addition, since it is a composite motion contrast image, the image quality and luminance level between examinations are stable, and the blood vessel extraction ability is easily stabilized between examinations.

<Step 560>
The extraction units 101 to 462 perform a binary image opening process (performing the expansion process after the contraction process) and a closing process (performing the contraction process after the expansion process) as the blood vessel region shaping process. The shaping process is not limited to this, and for example, small area removal based on the area of each label when a binary image is labeled may be performed.

  Note that the method for adaptively determining the scale for blood vessel enhancement in a motion contrast image including blood vessels of various diameters is not limited to the method described in S510 to S560. For example, as shown in S610 to S650 in FIG. 5B, a luminance statistical value (for example, an average value) for an image to which an operation of multiplying the luminance value of the Hessian-enhanced image and the luminance value of the blood vessel-enhanced image by edge selective sharpening is applied. The blood vessel region may be specified by binarizing with the threshold value. A lower limit value or an upper limit value can be set as the threshold value.

  Alternatively, as shown in S710 to S740 of FIG. 5C, the Hessian filter is applied after adaptively changing the range of the smoothing parameter σ when applying the Hessian filter according to the fixation position and depth range of the image, Blood vessel enhancement may be performed by binarization. For example, σ = 1 to 10 can be set for the retinal surface layer of the papillae, σ = 1 to 8 for the surface layer of the macular region, and σ = 1 to 6 for the deep layer of the macular region.

  The binarization process is not limited to the threshold process, and any known segmentation technique may be used.

  Further, details of the process executed in S308 will be described with reference to the flowchart shown in FIG.

<Step 810>
The operator sets a region of interest in the measurement process via the input unit 103. In the present embodiment, the measurement contents (measurement type and target area) are VAD maps (measurement type is VAD and measurement target area is the entire image) and VAD sector map (measurement type is VAD and measurement target area is SAD) in S306. The sector area corresponding to the ETDRS grid) is selected. Therefore, as a region of interest, (i) the entire image (ii) a sector region centered on the fixation lamp position (in a circular region defined by an inner circle with a diameter of 1 mm and an outer circle with a diameter of 3 mm, Superior, Inferior, Nasal, Temporal Are divided into four fan-shaped areas and the inner circle area).

<Step 820>
The measurement unit 101-463 performs measurement processing based on the binary image of the blood vessel region obtained in S307. In the present embodiment, the proportion of non-zero pixels (white pixels) occupying the vicinity region centered on the pixel at each pixel position of the binary image is calculated as the blood vessel density (VAD) in the pixel. Further, an image (VAD map) having a value of blood vessel density (VAD) calculated for each pixel is generated.

  Further, the ratio of non-zero pixels (white pixels) in each sector area (set in S810) on the binary image is calculated as the blood vessel density (VAD) in the sector. Further, a map (VAD sector map) having a value of blood vessel density (VAD) calculated in each sector region is generated.

<Step 830>
The display control unit 101-05 displays the VAD map and the VAD sector map generated in S820 as the measurement result on the display unit 104. In this embodiment, the VAD map of the retina surface layer is displayed in 906 of FIG. 9B, and the VAD map of the deep retina is displayed in 908. Further, a VAD sector map on the retina surface layer is displayed in 907, and a VAD sector map in the deep retina layer is displayed on 909 in a superimposed manner.

In this embodiment, the recommended conditions for measurement performed in FIG. 9B are as follows: i) Motion contrast images in which the number of acquired tomographic images at substantially the same position in the selected measurement target images is greater than or equal to a predetermined value or greater than or equal to the predetermined value It must be a measurement for an equivalent composite motion contrast image or a motion contrast image whose image quality index value (Quality Index) is greater than or equal to a predetermined value ii) It must be a measurement for a motion contrast image generated by maximum value projection iii) Projection artifact removal Iv) The processing has been performed. Iv) Measurement is set for a motion contrast image generated in one of the projection depth ranges including the retinal surface layer, the deep retinal layer, and the projection depth range including radial peripapillary capillaries (RPC). As above ) To iv) When displaying a measurement result performed in a state that does not satisfy at least one of the conditions on the measurement report screen (assuming that measurement was performed under conditions where accurate measurement cannot be expected), warning Display.

  For example, when displaying a result measured in a state where i) is not satisfied, a warning message such as “Averaged OCTA is recommended in calculating VAD or VLD” is displayed in the lower right part of FIG. Good. (For example, the lower right part of FIG. 12)

  Further, when displaying the result measured in a state where ii) is not satisfied, a warning message may be displayed in the lower right part of FIG. 9B, such as “MIP is recommended in calculating VAD or VLD.” .

  Similarly, when displaying the result measured in a state where iii) is not satisfied, a warning message such as “PAR is recommended in calculating VAD or VLD” is displayed at the lower right part of FIG. 9B. Good.

  Further, when displaying a result measured in a state where iv) is not satisfied, a warning message such as “Superficial Capillary, Deep Capillary, RPC can be analyzed in VCAL or VLD.” Is displayed. By displaying the warning message, it is known that there is a risk that the measurement result obtained by measurement that does not satisfy the recommended measurement condition will be a measurement result with low reliability, and by indicating the recommended measurement condition, Make it easier to perform more reliable measurements.

  In order to avoid a large number of warning messages occupying the report screen, priorities are assigned to the recommended conditions (for example, i) is the highest priority, ii) is the second, iii) is the third, and iv) is the fourth. It may be configured to display a warning regarding the highest priority condition among recommended conditions that are not satisfied. When a plurality of measurement results are displayed as shown in FIG. 9B, a warning message may be displayed for each individual measurement, or a warning with the highest priority among the warning messages to be displayed. Only a message may be displayed. Or, in order to make it easy to understand conditions that have a large impact on the reliability of measurement results and to display warnings about unsatisfied conditions without omission, warnings about unsatisfied measurement conditions can be identified in order of priority. You may display on the display part 104 (changing a color, a magnitude | size, etc.). Examples of displaying multiple measurement results include displaying the measurement results at the top and bottom of the report screen, and displaying the measurement results by setting multiple measurement target areas for the same image. .

  The warning message may be displayed in the same report screen or may be displayed as a separate screen. The warning message is not limited to a character string, and a still image or a moving image may be displayed on the display unit 104 or output as sound. The present invention includes a case where the report screen on which the warning message is displayed is output as a file or printed out.

  Further, the operator can select a warning message to be deleted from the warning messages displayed on the display unit 104 by using the input unit 103, change the priority order of warnings, or designate a warning message to be excluded from display. A user interface may be provided.

  In the above description, the procedure for measuring VAD as a blood vessel density has been described as an example. However, when generating a VLD map or a VLD sector map as a measured value, S840 to S840 shown in FIG. 6B is used instead of S810 to 830. 870 is executed.

<Step 840>
The measurement unit 101-463 generates a binary image (hereinafter referred to as a skeleton image) having a line width of 1 pixel corresponding to the center line of the blood vessel by thinning the binary image of the blood vessel region generated in S307. To do. Any thinning method or skeleton processing may be used, but in this embodiment, the thinning method of Hilditch is used as the thinning method.

<Step 850>
The operator sets a region of interest similar to that in S810 via the input unit 103. In this embodiment, the VLD map and the VLD sector map are calculated as the measurement contents (measurement type and target area). While VAD is selected in S810, VLD is selected in this step. It is only different. If the VLD map or the VLD sector map is not to be superimposed and displayed on the motion contrast image, the Map or Sector item in FIG. 9A may be set to “None”.

<Step 860>
The measurement unit 101-463 performs measurement processing based on the skeleton image obtained in S840. In the present embodiment, the total sum [mm ⁻¹ ] of the lengths of non-zero pixels (white pixels) per unit area in the vicinity region centered on the pixel at each pixel position of the skeleton image is calculated as the blood vessel density ( VLD). Further, an image (VLD map) having a blood vessel density (VLD) value calculated for each pixel is generated.

Further, the sum [mm ⁻¹ ] of the lengths of non-zero pixels (white pixels) per unit area in each sector area (set in S850) on the skeleton image is calculated as the blood vessel density (VLD) in the sector. . Further, a map (VLD sector map) having a value of blood vessel density (VLD) calculated in each sector region is generated.

<Step 870>
The display control unit 101-05 displays the VLD map and the VLD sector map generated in S860 as measurement results on 906/907 or 908/909 in FIG. 9B.

  As in the case of S830, a warning message is displayed on the display unit 104 when a measurement result that is performed in a state that does not satisfy a condition suitable for a predetermined analysis is displayed on the measurement report screen.

  Moreover, although this embodiment demonstrated the case where a measurement map was superimposed and displayed on a front motion contrast image as a blood vessel region specification in a single examination and a display method of a measurement result, it is not limited to this. For example, a binary image or a skeleton image of the specified blood vessel region may be displayed on 906 or 908 in FIG. Alternatively, the present invention also includes a case in which a motion contrast image is displayed on 906 and 908, and a binary image or a skeleton image of the specified blood vessel region is superimposed and displayed on the image after appropriately adjusting the color or transparency. include. The binary image is not limited to being displayed as a front image. For example, a binary image or a skeleton image of a blood vessel region specified on a B-scan tomographic image may be superimposed and displayed with appropriate adjustment of color or transparency. Good.

  When the operator inputs an instruction to correct the blood vessel region or the blood vessel centerline data from the input unit 103 in S309, the correction is performed according to the following procedure.

  That is, when a binary image including an overextracted region as shown in FIG. 10C is obtained for a synthesized motion contrast image as shown in FIG. 10A, the operator passes through the input unit 103. Then, the analysis unit 101-46 deletes the white pixel at the designated position. As an example of a method for specifying an addition / deletion position, for example, a method of clicking with the mouse while pressing the “d” key for deletion, or pressing the “a” key for addition is given. Alternatively, as shown in FIG. 6D, the binary image (blood vessel region or blood vessel centerline) to be corrected is superimposed on the image based on the motion contrast image, and is over-extracted or extracted. Make sure that the missing area is easy to identify. An enlarged image of the rectangular area 1002 in FIG. 4D is shown in FIG. Gray is an overextracted region, and white indicates the decorrelation value of the original motion contrast image. The operator may specify the overextracted / underextracted region using the input unit 103 so that the blood vessel or the blood vessel centerline region on the binary image is corrected accurately and efficiently. The binary image correction process is not limited to the front image. For example, motion contrast data, binary data of a blood vessel region, or a blood vessel centerline region is superimposed on a B-scan tomographic image at an arbitrary slice position as indicated by reference numeral 910 in FIG. 9A after adjustment of color and transparency. In this way, after the overextracted or underextracted region is easily discriminated, the input unit 103 is used to input the three-dimensional position (x, y, z coordinate) of the binary data to be corrected (added / deleted) by the operator It may be specified and corrected.

  Further, information indicating that the binary image (binary image or skeleton image of the blood vessel region) has been corrected or information regarding the correction position is stored in the external storage unit 102 in association with the binary image, and S870 or When the blood vessel identification result and the measurement result are displayed on the display unit 104 in S <b> 311, information indicating that the blood vessel has been corrected or information on the correction position may be displayed on the display unit 104.

  In the present embodiment, the case where the synthesis unit 101-42 repeatedly generates a composite motion contrast image at the end of OCTA imaging has been described, but the procedure for generating a composite motion contrast image is not limited to this. For example, a composite motion contrast image generation instruction button 812 is arranged on the report screen 803 in FIG. When the operator explicitly presses the generation instruction button 812 after completion of the OCTA imaging (or a date after the imaging date), the synthesis unit 101-42 causes the image processing apparatus 101 to generate a synthesized motion contrast image. It may be configured. When the operator explicitly presses the composite image generation instruction button 812 to generate a composite image, a composite motion contrast image 804, composite condition data, and inspection image list are displayed on a report screen 803 as shown in FIG. Items related to the composite image are displayed on the top.

  When the operator explicitly presses the generation instruction button 812, the display control unit 101-05 performs the following processing. That is, when a synthesis target image selection screen is displayed, the operator operates the input unit 103 to specify a synthesis target image group and presses an allow button, the synthesis unit 101-42 generates a synthesized motion contrast image, It is displayed on the display unit 104. Note that the case where a synthesized motion contrast image that has already been generated is selected and synthesized is also included in the present invention.

  When the operator presses the composite image generation instruction button 812, a two-dimensional composite image may be generated by combining two-dimensional images projected with a three-dimensional motion contrast image, or a three-dimensional composite image is generated. A two-dimensional composite image may be generated by projecting later.

  According to the configuration described above, the image processing apparatus 101 uses the front motion contrast images of the retinal surface layer and the deep retinal layer generated from the OCTA superimposed images acquired under substantially the same imaging conditions at different dates and times for the same eye to be examined. The region identification and blood vessel density measurement processing is performed. The composite image and the measurement value obtained by the specifying process and the measurement process are displayed in time series in a plurality of depth ranges. Thereby, for example, it is possible to accurately identify and measure a change in vascular pathological condition (vascular occlusion, new blood vessel, aneurysm, etc.) while suppressing the influence of variations in signal intensity and image quality of OCT tomographic images for each examination. Also, by analyzing one OCTA image, for example, the distribution of vascular lesions can be grasped quantitatively.

[Second Embodiment]
One of the image processing apparatuses according to this embodiment performs the blood vessel region specification and measurement processing in the first embodiment in three dimensions, and obtains the obtained image and measurement data (blood vessel region and blood vessel centerline / measurement data). It is configured to display juxtaposed in time series.

  Specifically, motion artifact suppression processing is performed on a three-dimensional synthetic motion contrast image including choroidal neovascularization (CNV). Next, a three-dimensional morphological filter and a blood vessel enhancement filter are applied and binarized to identify a blood vessel region including CNV in three dimensions. Furthermore, the case where the blood vessel density calculated in the retina surface layer and the deep retina, and the binary image and volume value of the choroidal neovascular region specified and measured in the outer retina are displayed in time series will be described.

  Since the configuration and image processing flow of the image processing system 10 including the image processing apparatus 101 according to the present embodiment are the same as those in the first embodiment, a description thereof will be omitted.

  Also, in the image processing flow in this embodiment, steps other than S306 to S308 and S310 to S311 in FIG. 3 are the same as in the case of the first embodiment, and a description thereof will be omitted.

<Step 306>
The operator uses the input unit 103 to instruct the start of the OCTA measurement process.

  In the present embodiment, double-clicking on the motion contrast image on the report screen 803 in FIG. 8E shifts to an OCTA measurement screen as shown in FIG. The motion contrast image is enlarged and displayed, and the type of image projection method (maximum value projection (MIP) or average value projection (AIP)), the projection depth range, and whether or not to perform the projection artifact removal process are selected as appropriate. Next, when the operator selects an appropriate item from the map button group 902, the Sector button group 903, and the selection button 904 displayed via the measurement button 904 on the right side of FIG. The target area is selected, and the analysis unit 101-46 starts the measurement process.

As a type of measurement processing, i) None (not measured) from the Map button group or Sector button group in this embodiment.
ii) VAD (blood vessel density calculated based on the area occupied by blood vessels)
iii) VLD (blood vessel density calculated based on the total length of blood vessels)
iv) Volume (volume of blood vessel region)
Select one of the following. Not limited to this, any type of measurement may be performed.

  For example, instead of the Volume of iv), a blood vessel region (obtained by specifying on a two-dimensional motion contrast image or by projecting the specified three-dimensional blood vessel region in a predetermined depth range (for example, the outer retina)) The case of measuring the area of choroidal capillaries is also included in the present invention.

In addition, from the selection screen displayed via the Measurement button: i) Area measurement of avascular region ii) Blood vessel density (VAD)
iii) Blood vessel density (VLD)
iv) Volume of blood vessel region (Volume)
Select one of the following. Not limited to this, for example, a blood vessel region (for example, a choroidal capillary vessel obtained by specifying on a two-dimensional motion contrast image or projecting the specified three-dimensional blood vessel region in a predetermined depth range (for example, outer retina)) ) May be measured.

The measurement by three-dimensional image processing is largely: 1) Two-dimensional measurement for the blood vessel region or blood vessel centerline data specified on the emphasized image projected in two dimensions and two-dimensionally projected 2) The blood vessel emphasized and specified in three dimensions 2D Measurement when Region or Blood Vessel Centerline Data is Projected 3) It can be broadly divided into 3D measurement for blood vessel region or blood vessel centerline data emphasized and specified in 3D.

  Examples of 1) and 2) include measuring the area of the avascular region, the blood vessel density, the area, diameter, length, and curvature of the blood vessel region on the projection image. Although the measurement contents are the same as the measurement for the front motion contrast image, the blood vessel extraction ability is improved as compared with the case where measurement is performed by emphasizing and specifying the front motion contrast image, so that the measurement accuracy is improved.

As an example of 3) 3-1) Volume measurement of blood vessels 3-2) Measurement on cross-sectional images or curved cross-sectional images in any direction (including blood vessel diameter or cross-sectional area measurement)
3-3) Measuring the length and curvature of blood vessels.

  In this embodiment, after performing blood vessel enhancement and blood vessel region specifying processing in three dimensions, VAD on each binary image projected in the depth range of the retinal surface layer and the deep layer of the retina, and the blood vessel region (choroid in the depth range of the outer retina layer) The volume of the neovascular area) is measured.

  As in the case of the first embodiment, when one of the measurement type selected from the Map button group and the measurement type selected from the Sector button group is changed, the other is also linked (same measurement). May be configured to be changed). Next, the analysis unit 101-46 performs image enlargement and top hat filter processing as preprocessing of measurement processing. In the present embodiment, three-dimensional bicubic interpolation and top hat filter processing are executed in both cases.

<Step 307>
The analysis unit 101-46 performs a blood vessel region specifying process. In the present embodiment, the enhancement unit 101-461 performs blood vessel enhancement processing based on the three-dimensional Hessian filter and the three-dimensional edge selective sharpening filter processing. Next, as in the case of the first embodiment, the extraction unit 101-462 performs binarization processing using two types of blood vessel emphasized images, and specifies a blood vessel region by performing shaping processing. Details of the blood vessel region specifying process will be described in S510 to S560.

<Step 308>
The measurement unit 101-463 performs measurement on a single examination image based on information on the measurement target region designated by the operator. Subsequently, the display control unit 101-05 displays the measurement result on the display unit 104. As in the case of the first embodiment, a warning message is displayed on the display unit 104 when displaying a measurement result performed in a state that does not satisfy a condition suitable for a predetermined analysis on the measurement report screen. To do. When the operator inputs an instruction to correct the blood vessel region or the blood vessel centerline data from the input unit 103, the analysis unit 101-46 causes the input unit 103 to be input from the operator as in the first embodiment. The blood vessel region or blood vessel centerline data is corrected based on the position information specified via the measurement information, and the measurement value is recalculated. S810 to S830 for VAD measurement in the surface layer and deep layer of the retina, volume measurement of the choroidal neovascularization in the outer layer of the retina, VLD measurement in the surface layer and deep layer of the retina, and S840 to S870 for the total blood vessel length measurement of the choroidal neovascularization in the outer layer of the retina Each will be described.

<Step 310>
The comparison units 101 to 464 perform a temporal change measurement (Progression measurement) process by the same operation as in the first embodiment.

<Step 311>
The display control unit 101-05 displays on the display unit 104 a report related to progression measurement performed in S310. In this embodiment, the VAD map measured in the retinal surface layer is displayed at the top of the Progression measurement report, the VAD map measured in the deep retinal layer in the second row, and i in the choroidal neovascular region measured in the outer layer of the retina in the third row. ) Binary image (or difference image from the reference image of the binary image)
ii) Volume value or total blood vessel length (or a difference value from the reference image of the binary image)
Are displayed side by side in time series. For example, the blood vessel density (VAD or VLD) map in the choroid may be displayed side by side in time series on the fourth stage. Thereby, it is possible to list and grasp time-series changes related to the three-dimensional pathological condition of the fundus blood vessel.

  In addition, as in the first embodiment, the number of tomographic images at substantially the same position for each measurement target image, whether or not OCTA overlay processing is performed, conditions for performing OCTA overlay processing, evaluation values of OCT tomographic images or motion contrast images Information may be displayed on the display unit 104. Furthermore, by changing the items related to the measurement type and the measurement target area from the shortcut menu to different values, the measurement type and the measurement target area can be changed and remeasured. Similar to the first embodiment, when a plurality of measurement target areas are selected and the measurement type for one area is changed, the same measurement type is applied to the other area in conjunction with the measurement. Is executed. Furthermore, a warning message may be displayed when displaying a measurement result performed in a state that does not satisfy the predetermined condition on the measurement report screen in the same manner as in the first embodiment.

  The present invention is not limited to the front image in different depth ranges and the time series display of the measurement value distribution for the front image. For example, the measurement value for the image orthogonal to the front image and the image orthogonal to the front image. The distribution, the volume-rendered three-dimensional image, and the measurement value distribution for the three-dimensional image may be displayed in time series.

  Further, details of the process executed in S307 will be described with reference to the flowchart shown in FIG.

<Step 510>
The enhancement unit 101-461 performs a three-dimensional blood vessel enhancement filter process based on the eigenvalues of the Hessian matrix on the three-dimensional motion contrast image that has been subjected to the preprocessing of Step 306. In this embodiment, a three-dimensional multi-scale line filter is used, but any known blood vessel enhancement filter may be used. In the three-dimensional Hessian filter, one of “the eigenvalues (λ1, λ2, λ3) of the Hessian matrix (equation (6)) calculated for each pixel on the three-dimensional image is close to 0, and the remaining two are negative and absolute. When the value is large, it is regarded as a linear structure and emphasized.

When a three-dimensional Hessian filter is used, a blood vessel bent in the depth direction has the property that “the luminance change in the blood vessel running direction is small and the luminance in two directions orthogonal to the blood vessel running direction is greatly reduced”. There is an advantage that can be emphasized. Some of the fundus blood vessels are blood vessels bent in the depth direction, for example: • Choroidal neovascularization (CNV) that enters the retina from the choroid side
-Blood vessels in the optic nerve head-Connections between retinal surface capillaries and deep retinal capillaries. When a two-dimensional Hessian filter is applied to the blood vessel on the front motion contrast image, the change in luminance in the blood vessel traveling direction in the two-dimensional plane is small, and the luminance in the direction orthogonal to the blood vessel traveling direction is There is a problem that it cannot be specified as a blood vessel region because it is not sufficiently emphasized because the property of “significantly reduced” does not hold. When a three-dimensional Hessian filter is used, the blood vessels can be favorably enhanced and the blood vessel detection ability is improved.

<Step 520>
The extraction unit 101-462 binarizes the blood vessel emphasized image (hereinafter referred to as a three-dimensional Hessian emphasized image) generated by the three-dimensional Hessian filter generated in S510. The procedure for binarization is the same as in the first embodiment, but is different from the case of the first embodiment in that it is binarization of three-dimensional data. Further, since the composite motion contrast image is an image obtained by emphasizing with a Hessian filter, the continuity of the binarized blood vessel region is improved as compared with a case where a single motion contrast image is enhanced with the Hessian filter.

<Step 530>
The enhancement unit 101-461 performs a three-dimensional edge selection sharpening process on the combined motion contrast image after applying the top hat filter generated in S306. In the present embodiment, edge selection sharpening processing is performed by performing three-dimensional unsharp mask processing using an image obtained by applying a three-dimensional Sobel filter as a weight to a three-dimensional motion contrast image.

<Step 540>
The extraction unit 101-462 binarizes the sharpened image to which the edge selective sharpening process generated in S530 is applied. Any known binarization method may be used. In the present embodiment, binary values are used with the luminance statistical value (average value or median value) calculated in each three-dimensional local region on the three-dimensionally sharpened image as a threshold value. Turn into. As in the case of the first embodiment, by setting the upper and lower limits of the threshold, insufficient extraction in the blood vessel region and erroneous extraction in the avascular region are suppressed. Similarly to the case of S520, since the synthesized motion contrast image is an edge-selected sharpened image, a noise-like false detection region when binarized compared to a single motion contrast image edge-selected sharpened Decrease.

<Step 550>
When both the luminance value of the binary image of the three-dimensional Hessian emphasized image generated in S520 and the luminance value of the binary image of the three-dimensional edge selection enhanced image generated in S540 are greater than zero, the extraction unit 101-462 Extracted as a blood vessel candidate region. This calculation process suppresses both the overestimated region of the vessel diameter seen in the Hessian-weighted image and the noise region seen on the edge-selection-enhanced image, ensuring accurate blood vessel boundary positions and good blood vessel continuity A binary image can be acquired. In addition, since both binary images are binary images based on the synthesized motion contrast image, noise-like false detection areas when binarized are reduced as compared to a binary image based on a single motion contrast image. At the same time, especially the continuity of the capillary region is improved. In addition, since it is a composite motion contrast image, the image quality and luminance level between examinations are stable, and the blood vessel extraction ability is easily stabilized between examinations.

<Step 560>
The extraction units 101 to 462 perform a three-dimensional opening process (performing the expansion process after the contraction process) and a closing process (performing the contraction process after the expansion process) as the blood vessel region shaping process. The shaping process is not limited to this, and small area removal based on the area of each label when a binary image is labeled may be performed.

  As in the case of the first embodiment, the method for adaptively determining the scale for blood vessel enhancement in a motion contrast image including blood vessels of various diameters is not limited to the method described in S510 to S560. For example, as shown in S610 to S650 of FIG. 5B, a luminance statistical value (for example, an image) applied to an image to which a calculation is performed by multiplying the luminance value of the three-dimensional Hessian emphasized image and the luminance value of the blood vessel emphasized image by the three-dimensional edge selective sharpening. The blood vessel region may be specified by binarization using an average value) as a threshold value. A lower limit value or an upper limit value can be set as the threshold value. Alternatively, as shown in S710 to S740 in FIG. 5C, the smoothing filter parameters (Gaussian filter) when applying the Hessian filter based on the three-dimensional position of each pixel (may be the fixation position or depth range data). The blood vessel enhancement may be performed by adaptively changing the smoothing parameter σ) and applying a Hessian filter to binarize. Also, the binarization process is not limited to the threshold process, and any known segmentation technique can be used.

  Further, details of the process executed in S308 will be described with reference to the flowchart shown in FIG.

<Step 810>
The operator sets a region of interest in the measurement process via the input unit 103.

In this embodiment, the measurement contents are as follows: 1) VAD map and VAD sector map in the retina surface layer and deep retina layer 2) The volume of choroidal neovascularization in the outer retina layer is calculated. Therefore, a sector area centering on the entire image and the fixation lamp position is selected as the region of interest in the retina surface layer and the deep retina layer. In the outer retina, a layer boundary corresponding to the outer retina (the range surrounded by the OPL / ONL boundary and the position where the Bruch's membrane boundary is moved to the boundary deep layer side by 20 μm) is designated.

<Step 820>
The measurement unit 101-463 performs measurement processing based on the binary image of the blood vessel region obtained in S307. The measurement contents (VAD map and VAD sector map generation) in the retina surface layer and the deep retina are basically the same as in the first embodiment, but the three-dimensional blood vessel region specified in the retina surface layer and the deep retina layer is projected as a front image. The point to measure after is different. In the outer retina, the volume of non-zero pixels (white pixels) in the region of interest corresponding to the outer retina set in S810 is calculated.

<Step 830>
The display control unit 101-05 displays on the display unit 104 the VAD map and VAD sector map in the retinal surface layer and deep retinal layer, the binary image of the blood vessel region in the outer retina and the volume value of the blood vessel region generated in S820 as measurement results. To do.

  In the same way as in the first embodiment, when displaying a measurement result that has been performed in a state that does not satisfy at least one of the recommended measurement conditions on the measurement report screen (as it cannot be expected to measure accurately) Warning display may be performed (assuming that measurement was performed under various conditions).

  In the above description, the procedure for measuring the volume based on the specified three-dimensional blood vessel region has been described as an example. However, in the case of measuring based on the three-dimensional blood vessel center line, FIG. 6 is used instead of S810 to 830 above. S840 to 870 shown in (b) are executed.

<Step 840>
The measurement unit 101-463 generates a skeleton image having a line width of 1 pixel corresponding to the center line of the blood vessel by performing a three-dimensional thinning process on the binary image of the blood vessel region generated in S820.

<Step 850>
The operator sets a region of interest similar to that in S810 via the input unit 103. In this embodiment, the measurement contents are as follows: 1) VLD map and VLD sector map in the retina surface layer and deep retina layer 2) The total blood vessel length of the choroidal neovascularization in the outer retina layer is calculated.

<Step 860>
The measurement unit 101-463 performs measurement processing based on the skeleton image obtained in S840. The measurement contents (VLD map and VLD sector map generation) in the retina surface layer and the deep retina are basically the same as in the first embodiment, but a three-dimensional skeleton specified in the retina surface layer and the deep retina layer is projected as a front image. The point to measure from is different. Note that the two-dimensional thinning process may be executed after the three-dimensional blood vessel region specified in S307 is projected as a front image. In the outer retina, the sum of the lengths of non-zero pixels (white pixels) in the region of interest corresponding to the outer retina set in S810 is calculated.

<Step 870>
The display control unit 101-05 displays, on the display unit 104, the VLD map and VLD sector map in the retinal surface layer and deep retinal layer, the skeleton image in the outer retina layer, and the total length of the skeleton generated in S860 as the measurement results. Similarly to the case of S830, a warning message is displayed when a measurement result that is performed in a state that does not satisfy a condition suitable for a predetermined analysis is displayed on the measurement report screen.

  In the present embodiment, the procedure in the case where the choroidal neovascularization is three-dimensionally extracted and the total volume and blood vessel length are displayed in time series has been described, but the present invention is not limited to this. For example, by pressing the “Show Difference” check box in FIG. 11, a difference image and a difference value between the volume of the new blood vessel in the reference image and the volume of the new blood vessel in other images are generated and displayed in time series. Also good. Alternatively, the arteriovenous region bent in the depth direction of the optic nerve head may be specified in the same manner as the procedure described in the present embodiment, and the blood vessel shape such as the blood vessel diameter, the blood vessel cross-sectional area, and the curvature of the blood vessel center line may be measured. . Alternatively, the connecting portion between the capillary of the retina surface layer and the capillary of the deep retina may be extracted three-dimensionally and highlighted, or the number of connecting portions may be counted.

  According to the configuration described above, the image processing apparatus 101 performs motion artifact suppression processing on a three-dimensional synthesized motion contrast image, and then applies a three-dimensional morphology filter and a blood vessel enhancement filter to perform binarization. A blood vessel region is specified in three dimensions. Further, the volume of the specified blood vessel region is calculated, and the binary image and volume value of the blood vessel region are displayed in time series. Thereby, for example, a change in vascular pathological condition can be accurately identified and measured while suppressing the influence of signal intensity and image quality variation of OCT tomographic images for each examination.

[Other Embodiments]
In each of the above embodiments, the present invention is realized as the image processing apparatus 101. However, the embodiment of the present invention is not limited to the image processing apparatus 101 alone. For example, the present invention can take an embodiment as a system, apparatus, method, program, storage medium, or the like.

Claims (16)

An analysis means for performing an analysis on at least a part of the eye motion contrast image;
A state in which at least two of the plurality of conditions suitable for the analysis are not satisfied in the image displayed on the display unit according to the instruction from the operator and showing the result of analyzing the at least a part of the region In the case of the image obtained in the above, the notifying means for notifying information on the at least two conditions according to the priority order of the plurality of conditions;
An image processing apparatus comprising: The notification means is display control means for displaying information on the at least two conditions on the display means,
The display control means includes an image showing a result obtained by analyzing the at least a part of the area using information indicating an analysis type selected according to an instruction from an operator, and information on the at least two conditions. The image processing apparatus according to claim 1, wherein the image processing apparatuses are arranged and displayed on the display unit.   3. The display control unit according to claim 2, wherein the display control unit causes the display unit to display a warning message regarding a condition having a higher priority of the at least two conditions as information regarding the at least two conditions. Image processing device.   The image processing apparatus according to claim 2, wherein the selected analysis type is any one of a plurality of types including at least a parameter related to a blood vessel region specified in the motion contrast image.   The selected analysis type is any one of a plurality of types including at least a blood vessel density related to an area of a blood vessel region specified in the motion contrast image and a blood vessel density related to the length of the blood vessel region. The image processing apparatus according to any one of claims 2 to 4.   6. The selected analysis type is any one of a plurality of types including at least a parameter related to an avascular region specified in the motion contrast image. Image processing apparatus. The display control means includes at least a part of the plurality of motion contrast front images of the eye corresponding to different depth ranges set according to an instruction from the operator in the three-dimensional motion contrast image of the eye. A plurality of images showing the result of analyzing the region are displayed side by side on the display control means,
In the case where at least one image of the plurality of images is an image obtained in a state where the at least two conditions are not satisfied, the notifying unit displays information on the at least two conditions. The image processing apparatus according to claim 2, wherein notification is performed according to a priority order of conditions.   The image processing apparatus according to claim 1, wherein the notification unit notifies a warning message regarding each of the at least two conditions as information regarding the at least two conditions in the priority order.   According to the plurality of conditions, the motion contrast image is an image obtained by combining a plurality of three-dimensional motion contrast images obtained by controlling the same position of the eye so that the measurement light is scanned. The image processing apparatus according to claim 1, wherein the condition is included as a condition having a higher priority than other conditions.   In the display area in which a plurality of motion contrast images of the eye corresponding to a plurality of examination days are displayed side by side in time series, the analyzing means includes the plurality of motion contrast images in response to an instruction from an operator. 10. The information indicating the type of analysis selected for at least a partial area of one image is applied to another image as well. Image processing apparatus.   The plurality of conditions suitable for the analysis of the motion contrast image include the number of tomographic images acquired at substantially the same position or the condition regarding the composition of the motion contrast or the image quality index value, the condition regarding the image projection method, and the condition regarding whether or not the projection artifact removal processing is performed. 11. The image processing apparatus according to claim 1, wherein at least two of the conditions relating to the analysis target region are set, and a priority order is set among the plurality of conditions.   The notification means displays a warning message regarding the at least two conditions as a character string, a still image or a moving image on the display unit as information regarding the at least two conditions, and outputs a report screen on which the warning message is displayed as a file The image processing apparatus according to claim 1, wherein at least one of printing and outputting a warning message as sound is executed.   The notification means displays the result of analysis in a state where there are a plurality of unsatisfied conditions among a plurality of conditions suitable for analysis of the motion contrast image of the eye based on a preset priority order. 2. Only a warning regarding a condition having the highest priority among a plurality of unsatisfied conditions is notified, or a warning regarding a plurality of unsatisfied conditions is notified in a manner in which a high priority can be identified. 13. The image processing device according to any one of 1 to 12.   Based on an instruction for selecting a warning to be deleted from the display unit among warnings notified by the notification unit, an instruction for changing the priority order of warnings, or an instruction for selecting a type of warning not to be displayed on the display unit, 14. The method according to claim 1, wherein the selected warning is deleted from the display unit, the priority of the warning is changed, or the selected type of warning is not displayed on the display unit. The image processing apparatus according to claim 1. Performing an analysis on at least a portion of the eye motion contrast image;
A state in which at least two of the plurality of conditions suitable for the analysis are not satisfied in the image displayed on the display unit according to the instruction from the operator and showing the result of analyzing the at least a part of the region In the case of the image obtained in the step, informing the information on the at least two conditions according to the priority order of the plurality of conditions,
An image processing method comprising:   A program causing a computer to execute each step of the image processing method according to claim 15.