JP7086683B2 - Image processing equipment, image processing methods and programs - Google Patents

Description

The disclosed technique relates to an image processing apparatus, an image processing method and a program.

Using an optical tomography device such as an optical coherence tomography (OCT), the state inside the retinal layer can be observed three-dimensionally. This tomographic imaging device is widely used in ophthalmic medical care because it is useful for more accurately diagnosing a disease. As a form of OCT, for example, there is TD-OCT (Time domain OCT) in which a wide band light source and a Michelson interferometer are combined. This is configured to move the position of the reference mirror at a constant speed, measure the interference light with the backscattered light acquired by the signal arm, and obtain the reflected light intensity distribution in the depth direction. However, such TD-OCT requires mechanical scanning, so high-speed image acquisition is difficult. Therefore, as a faster image acquisition method, a wideband light source is used, and an SD-OCT (Spectral domain OCT) that acquires an interference signal with a spectroscope or an SS-OCT (Swept Source) that separates time by using a high-speed wavelength sweep light source. OCT) has been developed to enable the acquisition of tomographic images with a wider angle of view.

On the other hand, in ophthalmic practice, invasive fluorescein angiography has been performed to understand the pathophysiology of fundus blood vessels. In recent years, the OCT Angiography (hereinafter referred to as OCTA) technique for non-invasively depicting fundus blood vessels in three dimensions using OCT has been used. In OCTA, the same position is scanned multiple times with the measurement light, and the motion contrast obtained by the interaction between the displacement of red blood cells and the measurement light is imaged. FIG. 3 shows an example of OCTA imaging in which the main scanning direction is the horizontal (x-axis) direction and B scans are continuously performed r times at each position (yi: 1 ≦ i ≦ n) in the sub-scanning direction (y-axis direction). ing. In OCTA imaging, scanning multiple times at the same position is called cluster scanning, and multiple tomographic images obtained at the same position are called clusters. It is known that the contrast of OCTA images is improved by generating motion contrast data for each cluster and increasing the number of tomographic images per cluster (the number of scans at substantially the same position).

Here, a technique for displaying a blood vessel analysis map calculated using motion contrast data is disclosed in the patent document.

JP-A-2017-77414

Here, consider a case where the analysis results of a plurality of types are displayed on the display unit for each of a plurality of regions in the motion contrast image of the eye unit. At this time, for example, if all the analysis results can be simply displayed on the display unit, there is a problem that the convenience of the operator is low.

One of the purposes of the disclosed technique is to improve the convenience of the operator in displaying the analysis result of the motion contrast image of the eye portion.

It should be noted that the other purpose of the present invention is not limited to the above-mentioned purpose, but is an action and effect derived by each configuration shown in the embodiment for carrying out the invention described later, and the action and effect which cannot be obtained by the conventional technique can be obtained. It can be positioned as one.

One of the disclosed image processing devices is a selection means for selecting a first type of analysis or a second type of analysis according to an instruction from an operator.
A setting means for setting a first region and a second region, which are partial regions of the motion contrast image of the eye portion, according to an instruction from the operator .
An analysis means for analyzing the set first region and the second region, and
The set first region and the second region were displayed on the display means in a state of being superimposed on the motion contrast image, and the first region was analyzed by the first type of analysis. It has a result and a display control means for causing the display means to display the result of analysis of the second region by the second type of analysis .
When the first type of analysis is selected, the display control means displays the result of the analysis by the first type of analysis in the first region in a state of being superimposed on the motion contrast image. At the same time, information indicating the first type is displayed .

According to one of the disclosed techniques, it is possible to improve the convenience of the operator in displaying the analysis result of the motion contrast image of the eye portion.

It is a figure which shows an example of the image processing apparatus of 1st Embodiment. It is a figure which shows an example of the image processing apparatus of 1st Embodiment. It is a figure explaining the scanning method of OCTA photography. It is a flowchart of 1st Embodiment. It is an example of the user interface of the first embodiment. This is an example of how to display the analysis results. It is an example of the user interface of the first embodiment. It is a flowchart of the modification of 1st Embodiment. This is an example of how to display the analysis results. This is an example of how to select the analysis range. It is an example of the user interface of the first embodiment. It is a flowchart of 2nd Embodiment. It is an example of the user interface of the second embodiment. It is a figure explaining the method of image processing in 1st Embodiment. This is an example of how to select an analysis method. It is an example of the user interface of the third embodiment. This is an example of how to display the analysis results. This is an example of how to display the analysis results.

(First Embodiment)
The image processing apparatus according to the present embodiment describes a method of displaying the analysis result when the image to be analyzed is analyzed. Hereinafter, an image processing system including an image processing apparatus according to the first embodiment of the present invention will be described with reference to the drawings.

FIG. 2 is a diagram showing a configuration of an image processing system 10 including an image processing device 101 according to the present embodiment. As shown in FIG. 2, in the image processing system 10, the image processing device 101 is connected to the tomographic imaging device 100 (also referred to as OCT), the external storage unit 102, the input unit 103, and the display unit 304 via an interface. It is composed of.

The tomographic image capturing device 100 is a device that captures a tomographic image of the eye portion. In this embodiment, SD-OCT is used as the tomographic imaging apparatus 100. Not limited to this, for example, SS-OCT may be used for configuration.

In FIG. 2A, the measurement optical system 100-1 is an optical system for acquiring an anterior eye portion image, an SLO fundus image of the eye to be inspected, and a tomographic image. The stage unit 100-2 makes the measurement optical system 100-1 movable back and forth and left and right. The base portion 100-3 has a built-in spectroscope described later.

The image processing device 101 is a computer that controls the stage unit 100-2, controls the alignment operation, reconstructs a tomographic image, and the like. The external storage unit 102 stores a program for tomographic imaging, patient information, imaging data, image data of past examinations, measurement data, and the like.

The input unit 103 gives an instruction to the computer, and specifically includes a keyboard and a mouse. The display unit 304 includes, for example, a monitor.

(Configuration of tomographic imaging device)
The configuration of the measurement optical system and the spectroscope in the tomographic imaging apparatus 100 of the present embodiment will be described with reference to FIG. 2 (b).

First, the inside of the measurement optical system 100-1 will be described. The objective lens 201 is installed facing the eye 200 to be inspected, and the first dichroic mirror 202 and the second dichroic mirror 203 are arranged on the optical axis thereof. These dichroic mirrors are branched into an optical path 250 for the OCT optical system, an optical path 251 for the SLO optical system and the fixation lamp, and an optical path 252 for frontal eye observation for each wavelength band.

The optical path 251 for the SLO optical system and the fixative lamp has an SLO scanning means 204, lenses 205 and 206, a mirror 207, a third dichroic mirror 208, an APD (Avalanche photodiode) 209, an SLO light source 210, and a fixative lamp 211. ing. The mirror 207 is a prism on which a perforated mirror or a hollow mirror is vapor-deposited, and separates the illumination light from the SLO light source 210 and the return light from the eye to be inspected. The third dichroic mirror 208 separates the optical path of the SLO light source 210 and the optical path of the fixative lamp 211 for each wavelength band. The SLO scanning means 204 scans the light emitted from the SLO light source 210 on the eye 200 to be inspected, and is composed of an X scanner that scans in the X direction and a Y scanner that scans in the Y direction. In the present embodiment, the X scanner is a polygon mirror because it is necessary to perform high-speed scanning, and the Y scanner is composed of a galvano mirror. The lens 205 is driven by a motor (not shown) for focusing the SLO optical system and the fixative lamp 211. The SLO light source 210 generates light having a wavelength near 780 nm. APD209 detects the return light from the eye to be inspected. The fixative lamp 211 generates visible light to promote the fixative of the subject. The light emitted from the SLO light source 210 is reflected by the third dichroic mirror 208, passes through the mirror 207, passes through the lenses 206 and 205, and is scanned by the SLO scanning means 204 on the eye 200 to be inspected. The return light from the eye 200 to be inspected returns to the same path as the illumination light, is reflected by the mirror 207, and is guided to the APD209 to obtain an SLO fundus image. The light emitted from the fixative lamp 211 passes through the third dichroic mirror 208 and the mirror 207, passes through the lenses 206 and 205, and forms a predetermined shape at an arbitrary position on the eye 200 to be inspected by the SLO scanning means 204. Encourage the subject to fix.

In the optical path 252 for frontal eye observation, lenses 212 and 213, a split prism 214, and a CCD 215 for frontal eye observation for detecting infrared light are arranged. The CCD215 has a sensitivity in the wavelength of the irradiation light for observing the anterior eye portion (not shown), specifically, in the vicinity of 970 nm. The split prism 214 is arranged at a position conjugate with the pupil of the eye to be inspected 200, and the distance in the Z-axis direction (optical axis direction) of the measurement optical system 100-1 with respect to the eye to be inspected 200 is used as a split image of the anterior eye portion. Can be detected.

The optical path 250 of the OCT optical system constitutes the OCT optical system as described above, and is for taking a tomographic image of the eye 200 to be inspected. More specifically, it obtains an interference signal for forming a tomographic image. The XY scanner 216 is for scanning light on the eye 200 to be inspected, and is shown as a single mirror in FIG. 2B, but is actually a galvano mirror that scans in the XY2 axis direction. Of the lenses 217 and 218, the lens 217 is driven by a motor (not shown) to focus the light emitted from the OCT light source 220 emitted from the fiber 224 connected to the optical coupler 219 to the eye 200 to be inspected. By this focusing, the return light from the eye 200 to be inspected is simultaneously imaged and incident on the tip of the fiber 224 in a spot shape. Next, the configuration of the optical path from the OCT light source 220, the reference optical system, and the spectroscope will be described. 220 is an OCT light source, 221 is a reference mirror, 222 is a dispersion compensating glass, 223 is a lens, 219 is an optical coupler, 224 to 227 are single-mode optical fibers connected to an optical coupler and integrated, and 230 is a spectroscope. .. These configurations make up the Michelson interferometer. The light emitted from the OCT light source 220 is divided into the measurement light on the optical fiber 224 side and the reference light on the optical fiber 226 side via the optical coupler 219 through the optical fiber 225. The measurement light is applied to the eye 200 to be observed through the above-mentioned OCT optical path, and reaches the optical coupler 219 through the same optical path due to reflection or scattering by the eye 200 to be observed. On the other hand, the reference light reaches the reference mirror 221 and is reflected through the optical fiber 226, the lens 223, and the dispersion compensating glass 222 inserted to match the wavelength dispersion of the measurement light and the reference light. Then, it returns to the same optical path and reaches the optical coupler 219. The optical coupler 219 combines the measurement light and the reference light into interference light. Here, interference occurs when the optical path length of the measurement light and the optical path length of the reference light are almost the same. The reference mirror 221 is held adjustable in the optical axis direction by a motor and a drive mechanism (not shown), and it is possible to match the optical path length of the reference light with the optical path length of the measurement light. The interfering light is guided to the spectroscope 230 via the optical fiber 227. Further, the polarization adjusting units 228 and 229 are provided in the optical fibers 224 and 226, respectively, to adjust the polarization. These polarization adjusting parts have some parts in which the optical fiber is wound in a loop shape. By rotating this loop-shaped portion around the longitudinal direction of the fiber, the fiber can be twisted to adjust and match the polarization states of the measurement light and the reference light. The spectroscope 230 is composed of a lens 232, 234, a diffraction grating 233, and a line sensor 231. The interference light emitted from the optical fiber 227 becomes parallel light through the lens 234, is separated by the diffraction grating 233, and is imaged on the line sensor 231 by the lens 232.

Next, the periphery of the OCT light source 220 will be described. The OCT light source 220 is an SLD (Super Luminescent Diode), which is a typical low coherent light source. The center wavelength is 855 nm and the wavelength bandwidth is about 100 nm. Here, the bandwidth is an important parameter because it affects the resolution of the obtained tomographic image in the optical axis direction. Although SLD was selected as the type of light source here, ASE (Amplified Spontaneous Emission) or the like can be used as long as low coherent light can be emitted. Near-infrared light is suitable for the center wavelength in view of measuring the eye. Further, since the central wavelength affects the lateral resolution of the obtained tomographic image, it is desirable that the wavelength is as short as possible. For both reasons, the center wavelength was set to 855 nm.

Although the Michelson interferometer is used as the interferometer in this embodiment, a Mach-Zehnder interferometer may be used. It is desirable to use a Mach-Zehnder interferometer when the light amount difference is large and a Michelson interferometer when the light amount difference is relatively small, depending on the light amount difference between the measured light and the reference light.

(Configuration of image processing device)
The configuration of the image processing apparatus 101 of the present embodiment will be described with reference to FIG. The image processing device 101 is a personal computer (PC) connected to the tomographic image capturing device 100, and is an image acquisition unit 301, a storage unit 101-02, an imaging control unit 101-03, an image processing unit 101-04, and a display control unit. It is equipped with 101-05. Further, the image processing device 101 realizes a function by the arithmetic processing unit CPU executing a software module that realizes an image acquisition unit 301, a shooting control unit 101-03, an image processing unit 101-04, and a display control unit 101-05. do. The present invention is not limited to this, and for example, the image processing unit 101-04 may be realized by dedicated hardware such as an ASIC, or the display control unit 101-05 may be realized by using a dedicated processor such as a GPU different from the CPU. May be realized. Further, the connection between the tomographic image capturing device 100 and the image processing device 101 may be configured via a network.

The image acquisition unit 301 acquires the signal data of the SLO fundus image and the tomographic image taken by the tomographic image capturing apparatus 100. Further, the image acquisition unit 301 has a tomographic image generation unit 101-11 and a motion contrast data generation unit 101-12. The tomographic image generation unit 101-11 acquires the signal data (interference signal) of the tomographic image taken by the tomographic image capturing apparatus 100, generates a tomographic image by signal processing, and stores the generated tomographic image in the storage unit 101-02. Store. The imaging control unit 101-03 performs imaging control for the tomographic imaging apparatus 100. The imaging control also includes instructing the tomographic imaging apparatus 100 regarding the setting of imaging parameters and instructing the start or end of imaging.

The image processing unit 101-04 includes an alignment unit 101-41, a composition unit 101-42, a correction unit 101-43, an image feature acquisition unit 101-44, a projection unit 101-45, an analysis range designation unit 701, and an analysis method selection unit. It has a unit 302 and an analysis unit 303. The image acquisition unit 301 and the composition unit 101-42 described above are examples of the acquisition means according to the present invention. The compositing unit 101-42 synthesizes a plurality of motion contrast data generated by the motion contrast data generation unit 101-12 based on the alignment parameters obtained by the alignment unit 101-41, and generates a composite motion contrast image. .. The correction unit 101-43 performs a process of suppressing projection artifacts generated in the motion contrast image two-dimensionally or three-dimensionally. The image feature acquisition unit 101-44 acquires the positions of the layer boundaries of the retina and choroid, the fovea centralis, and the center of the optic nerve head from the tomographic image. The projection unit 101-45 projects a motion contrast image in a depth range based on the position of the layer boundary acquired by the image feature acquisition unit 101-44, and generates a frontal motion contrast image. The analysis range designation unit 701 designates the analysis range in the front motion contrast image. The analysis method selection unit 302 specifies (selects) an analysis method for the entire image or each analysis range. The storage unit 101-02 can also store the designated analysis range and analysis method in association with the motion contrast image. The analysis unit 303 has an emphasis unit 101-461, an extraction unit 101-462, a measurement unit 101-463, and an adjustment unit 101-464, and extracts and measures a blood vessel region from a three-dimensional or frontal motion contrast image. Here, the analysis unit 303 is an example of analysis means for analyzing the first region and the second region, which are partial regions of the motion contrast image of the eye portion. Further, it is preferable that the first region and the second region are partial regions selected according to an instruction from the operator for the motion contrast image. Here, the first region and the second region may be completely overlapping regions or may be partially overlapping regions. Further, the first region and the second region may be completely different regions. In this case, the first region and the second region may be regions in contact with each other or regions completely separated from each other. The method of selecting the first region and the second region will be described later with reference to FIG. 9 as a method of designating the analysis range. The emphasis section 101-461 performs a process of emphasizing the blood vessel region or the edge of the blood vessel. Extraction unit 101-462 extracts the blood vessel region based on the blood vessel-enhanced image. Further, the measuring unit 101-463 calculates the measured value such as the blood vessel density by using the extracted blood vessel region and the blood vessel center line data acquired by thinning the blood vessel region. Further, the adjustment unit 101-464 changes the applicable range of the smoothing parameter σ used when the emphasis unit 101-461 emphasizes the blood vessel based on the position on the eye portion, and the extraction unit 101-462 extracts the blood vessel region. The binarization method at the time of binarization is changed based on the brightness value of the image to be binarized. As the configuration of the image processing apparatus 101 in the present invention, all the configurations described above are not essential, and for example, the alignment unit 101-41, the composition unit 101-42, the correction unit 101-43, and the like are omitted. Is also good.

Further, the display control unit 101-05 superimposes the information indicating the first region and the information indicating the second region, which are partial regions of the motion contrast image of the eye portion, on the motion contrast image on the display unit 304. Display. Further, the display control unit 101-05 analyzes the first region using the information indicating at least one analysis type (for example, the first type) selected in response to the first instruction from the operator. Information indicating the result is displayed on the display unit 304. Further, the display control unit 101-05 uses information indicating at least one analysis type (for example, a second type different from the first type) selected in response to the second instruction from the operator. Information indicating the result of analysis of the second region is displayed on the display unit 304. As a result, the operator can display the analysis result corresponding to the type of analysis selected according to the purpose such as diagnosis. Therefore, unlike the conventional configuration in which all the analysis results can be simply displayed on the display unit, it is possible to improve the convenience of the operator in displaying the analysis results of the motion contrast image of the eye unit.

Here, the information indicating the region may be displayed in any manner as long as the partial region on the motion contrast image can be identified. For example, it is preferable that a line indicating the outer edge of the region and a color indicating the inside are superimposed on the motion contrast front image. The type of analysis is, for example, the blood vessel density (VAD) related to the area of the blood vessel region, the blood vessel density (VLD) related to the blood vessel length, and the like. The information indicating the analysis result is, for example, a value indicating the analysis result, and it is preferable to display the selected analysis type unit in an identifiable state.

Further, when the first type is selected, the display control unit 101-05 is selected for the first image showing the result of analysis using the information indicating the first type. Consider a situation in which the display unit 304 displays the information indicating the area superimposed on the first image. At this time, it is preferable that the display control unit 101-05 displays information indicating the result of analysis of the first area in the display area where the first image is output. Further, the display control unit 101-05 displays the first image and the information indicating the second type in the display area when the second type is selected after the first type is selected. It is preferred to perform control to change the display to a second image showing the results analyzed using. Further, the display control unit 101-05 causes the display unit 304 to display the information indicating the second area selected for the second image in a state of being superimposed on the second image, and the second area is displayed. It is preferable to display the information indicating the analyzed result in the display area. The image showing the analysis result (first image, second image) is, for example, a two-dimensional image showing the analysis result with respect to the motion contrast front image. Further, the two-dimensional image showing the analysis result includes, for example, a VAD map, a VLD map, a VAD sector map, a VLD sector map, and an image in which each of these analysis maps is superimposed on a motion contrast front image. It should be noted that these analysis maps may be any as long as the analysis results can be identified, and are preferably two-dimensional images represented by colors corresponding to the analysis results, for example. Further, an image in which a plurality of analysis maps of the same type are superimposed on each other or an image in which a plurality of analysis maps of the same type are superimposed on a motion contrast front image may be used. For example, a two-dimensional image in which a VAD sector map is superimposed on a VAD map, a two-dimensional image in which a VAD sector map and a VAD map are superimposed on a motion contrast image, a two-dimensional image in which a VLD sector map is superimposed on a VLD map, and a VLD sector. There is a two-dimensional image in which a map and a VLD map are superimposed on a motion contrast image. The timing of executing the analysis by the analysis unit 101-46 may be the timing at which the analysis type is selected according to the instruction from the operator, or the type of analysis assumed before the analysis type is selected. The analysis corresponding to may be completed in advance. Here, consider a situation in which control for changing the display of the first image to the display of the second image is executed. At this time, it is preferable that the display control unit 101-05 executes a control for changing the information indicating the first region (or the information indicating the result of analysis of the first region) to be hidden. As a result, the analysis region (or analysis result) corresponding to the second image can be selectively displayed on the second image. Therefore, it is possible to improve the convenience of the operator in displaying the analysis result of the motion contrast image of the eye portion. The display control unit 101-05 may display the information indicating the first region on the display unit 304 in a state of being superimposed on the second image. Further, it is preferable that the display control unit 101-05 displays in the display area information indicating the result of analysis of the first area using the information indicating the second type. Further, the display control unit 101-05 displays information indicating the result of analysis of the first type using the information indicating the first type in the display area, and uses the information indicating the second type. Control may be executed to change the display of information indicating the result of analysis of the first region. The case where the analysis area is selected after the analysis type is selected has been described, but conversely, the analysis type is selected after the analysis area is selected. May be.

The external storage unit 102 contains information on the eye to be inspected (patient's name, age, gender, etc.), captured images (tomographic image, SLO image, OCTA image), synthetic image, imaging parameter, position of blood vessel region and blood vessel center line. Parameters such as data, measured values, analysis range and analysis method set according to instructions from the operator are associated and held. The input unit 103 is, for example, a mouse, a keyboard, a touch operation screen, or the like, and the operator gives an instruction to the image processing device 101 or the tomographic image capturing device 100 via the input unit 103.

Next, a method of generating a motion contrast image based on the OCT tomographic image will be described. Here, the motion ton trust is a visualization of the temporal change of the OCT tomographic image in the same cluster, and the temporal change indicates the blood flow.

First, the tomographic image generation unit 101-11 performs wave number conversion, fast Fourier transform (FFT), and absolute value conversion (amplitude acquisition) on the interference signal acquired by the image acquisition unit 301 to obtain a tomographic image for one cluster. Generate. Next, the alignment unit 101-41 aligns the tomographic images belonging to the same cluster with each other and performs superposition processing. The image feature acquisition unit 101-44 acquires layer boundary data from the superposition tomographic image. In this embodiment, the variable shape model is used as the layer boundary acquisition method, but any known layer boundary acquisition method may be used. It should be noted that the layer boundary acquisition process is not essential. For example, when the motion contrast image is generated only in three dimensions or when the two-dimensional motion contrast image projected in the depth direction is not generated, the layer boundary acquisition process is omitted. can. The motion contrast data generation unit 101-12 calculates the motion contrast between adjacent tomographic images in the same cluster. In the present embodiment, the decorrelation value Mxy is obtained as the motion contrast based on the following equation (1).

Here, Axy indicates the amplitude (of the complex number data after FFT processing) at the position (x, y) of the tomographic image data A, and Bxy indicates the amplitude of the tomographic data B at the same position (x, y). 0 ≦ Mxy ≦ 1, and the larger the difference between the two amplitude values, the closer to 1. Decorrelation calculation processing as in equation (1) is performed between arbitrary adjacent tomographic images (belonging to the same cluster), and the average of the obtained (number of tomographic images per cluster-1) motion contrast values is averaged. An image having as a pixel value is generated as a final motion contrast image.

Here, the motion contrast is calculated based on the amplitude of the complex number data after the FFT processing, but the calculation method of the motion contrast is not limited to the above. For example, the motion contrast may be calculated based on the phase information of the complex number data, or the motion contrast may be calculated based on both the amplitude and the phase information. Alternatively, the motion contrast may be calculated based on the real part or the imaginary part of the complex number data.

Further, in the present embodiment, the decorrelation value is calculated as the motion contrast, but the calculation method of the motion contrast is not limited to this. For example, the motion contrast may be calculated based on the difference between the two values, or the motion contrast may be calculated based on the ratio of the two values.

Further, in the above, the final motion contrast image is obtained by obtaining the average value of the obtained plurality of decorrelation values, but the present invention is not limited to this. For example, an image having the median value or the maximum value of a plurality of acquired decorrelation values as pixel values may be generated as a final motion contrast image. Further, a high-contrast composite motion contrast image may be generated by three-dimensionally aligning the motion contrast image group obtained through repeated OCTA photography and averaging them.

The compositing unit 101-42 three-dimensionally aligns the motion contrast image group (FIG. 14 (a)) obtained through repeated OCTA imaging, and adds and averages them to achieve high contrast as shown in FIG. 14 (b). Generates a composite motion contrast image. The composition process is not limited to the simple addition average. For example, the brightness value of each motion contrast image may be arbitrarily weighted and then averaged, or an arbitrary statistical value including the median may be calculated. The present invention also includes a case where the alignment process is performed two-dimensionally.

The compositing unit 101-42 may be configured to determine whether or not a motion contrast image unsuitable for the compositing process is included, and then perform the compositing process excluding the motion contrast image determined to be unsuitable. For example, when the evaluation value (for example, the average value of the decorrelation value or fSNR) for each motion contrast image is out of a predetermined range, it may be determined that the image is unsuitable for the composition process.

Next, a method of three-dimensionally suppressing projection artifacts generated in a motion contrast image will be described. Here, the projection artifact refers to a phenomenon in which the motion contrast in the blood vessels on the surface of the retina is reflected on the deep side (deep layer of the retina, outer layer of the retina, choroid), and a high decorrelation value occurs in the deep region where the blood vessels do not actually exist. ..

FIG. 14C shows an example in which 3D motion contrast data is superimposed and displayed on a 3D OCT tomographic image. A region 2202 having a high decorrelation value is generated on the deep side (photoreceptor layer) of the region 2201 having a high decorrelation value corresponding to the retinal surface blood vessel region. Although blood vessels do not originally exist in the photoreceptor layer, the blinking of blood vessel shadows occurring on the surface layer of the retina is reflected in the photoreceptor layer, and the brightness value of the photoreceptor layer changes, resulting in artifact 2202.

The correction unit 101-43 executes a process of suppressing the projection artifact 802 generated on the three-dimensional synthetic motion contrast image. Any known projection artifact suppression technique may be used, but in this embodiment, Step-down Exponential Filtering is used. In Step-down Exponential Filtering, projection artifacts are suppressed by executing the process represented by the equation (2) for each A scan data on the three-dimensional motion contrast image.

Here, γ is an attenuation coefficient having a negative value, D (x, y, z) is a decorrelation value before the projection artifact suppression process, and DE (x, y, z) is a decorrelation value after the suppression process. show.

FIG. 14 (d) shows an example in which the three-dimensional synthetic motion contrast data (gray) after the projection artifact suppression process is superimposed and displayed on the tomographic image. It can be seen that the artifacts found on the photoreceptor layer before the projection artifact suppression treatment (FIG. 14 (c)) were removed by the suppression treatment.

Next, a method for specifying the blood vessel region in the present embodiment will be described. The analysis unit 303 performs preprocessing for measurement processing. Any known image processing can be applied as preprocessing, but in the present embodiment, image enlargement and morphology calculation (top hat filter processing) are performed as preprocessing. By applying the top hat filter, it is possible to reduce the uneven brightness of the background component. Specifically, the image is enlarged by using three-dimensional Bicubic interpolation so that the pixel size of the composite motion contrast image becomes about 3 μm, and the top hat filter processing is performed using the spherical structural element.

The analysis unit 303 performs a process for specifying the blood vessel region. In the present embodiment, the emphasis section 101-461 performs a blood vessel enhancement process and an edge selection sharpening process based on a hesian filter. Next, the extraction unit 101-462 performs binarization processing using two types of blood vessel-enhanced images, and performs shaping processing to specify the blood vessel region.

In this embodiment, blood vessel area density analysis and blood vessel length density analysis are defined as analysis methods. The blood vessel area density (VAD: Vessel Area Density) is a blood vessel density (unit:%) defined by the ratio of the blood vessel region included in the measurement target. That is, VAD is an example of blood vessel density with respect to the area of the blood vessel region specified in the motion contrast image. Further, the blood vessel length density (VLD: Vessel Length Density) is a blood vessel density defined by the total length of blood vessels (unit: mm-1) contained in a unit area. That is, VLD is an example of blood vessel density with respect to the length of the blood vessel region identified in the motion contrast image. Further, VAD and VLD are examples of parameters related to the blood vessel region specified in the motion contrast image. Parameters related to the blood vessel region include the area of the blood vessel region, the blood vessel length, the curvature of the blood vessel, and the like. Here, the blood vessel density is an index for quantifying the occlusion range of the blood vessel and the degree of sparseness of the blood vessel network, and the blood vessel area density analysis is most often used. However, in the blood vessel area density analysis, the contribution of the large blood vessel region to the measured value is large, so if you want to focus on the pathophysiology of capillaries such as diabetic retinopathy (more sensitive to capillary occlusion). (As an index) vessel length density analysis is used. In addition to the parameters related to the blood vessel region, the type of analysis includes, for example, parameters related to the non-perfusion region (NPA) specified in the motion contrast image. Parameters related to the avascular region include the area and shape (length and circularity) of the avascular region. Not limited to this, for example, a Fractal dimension that quantifies the complexity of the blood vessel structure or a Vessel Diameter Index that represents the distribution of the blood vessel diameter (distribution of aneurysm or stenosis of the blood vessel) may be measured.

The apparatus configuration in this embodiment will be described with reference to FIG. The image acquisition unit 301 acquires an analysis target image to be analyzed, and stores the analysis target image in the storage unit 101-02. The analysis unit 303 obtains an analysis result based on the analysis target image acquired from the storage unit 101-02 and the analysis method selected by the analysis method selection unit 302, and stores the analysis result in the storage unit 101-02. The display control unit 101-05 acquires the analysis target image and the analysis result from the storage unit 101-02, and notifies the display unit 304.

The flow of processing in this embodiment will be described with reference to the flowchart of FIG. In S401, the user selects an image to be analyzed. In S402, the user selects an analysis method (blood vessel area density analysis, blood vessel length density analysis, no analysis). When the selected analysis method is blood vessel area density analysis, the analysis result of the area density analysis is displayed in S403 and S404. When the selected analysis method is blood vessel length density analysis, the analysis result of the blood vessel length density analysis is displayed in S405 and S406. If you select Do not analyze, do not analyze. In S407, when the user changes the analysis method, the process of S402 to S407 is repeated. The image to be analyzed may be an image taken by the image acquisition unit 301, or may be an image stored in advance in the external storage unit 102 or the storage unit 101-02. Further, in the present embodiment, as the type of image, a front image such as a luminance Enface image, an OCTA image, an SLO image, and a fundus image is assumed, but it can also be applied to a configuration targeting a tomographic image or the like.

Further, FIG. 5 shows an example of the user interface in the present embodiment. By switching the analysis method shown in 501, the analysis result displayed on the 502 is changed. 6 (a) and 6 (b) show an example of the analysis result displayed in 502, respectively. Further, the values shown in FIGS. 6A and 6B are values analyzed by blood vessel area density analysis for the entire image. When the blood vessel area density is selected as the analysis method in 501, the analysis result displayed in 502 may be displayed as a composite image in which the analysis target image and the analysis result are combined as shown in FIG. 6 (a). As shown in 6 (b), a numerical value, a graph, a color map, or the like may be displayed separately from the image to be analyzed. Further, when the blood vessel length density is selected as the analysis method in 501, the analysis result displayed on the 502 is switched to the analysis result analyzed by the blood vessel length density analysis. When displaying the analysis result of the blood vessel length density analysis, the display method equivalent to that of FIGS. 6 (a) and 6 (b) may be used. It may be displayed.

In addition, all the figures introduced below as analysis results are applicable to 502. The image to be analyzed is not limited to the captured image, but may be an image stored in advance. Further, the configuration is such that the user selects the analysis method, but the present invention is not limited to this. For example, if the image to be analyzed is around the papilla, blood vessel length density analysis is selected, and if it is around the yellow spot, blood vessel area density analysis is automatically selected. The analysis method may be automatically selected according to the type.

In this embodiment, an example of analyzing the entire image based on the selected analysis method has been described. A modified example in which a part of the configuration of the present embodiment is changed, an analysis range is specified for each analysis method, and an analysis result is displayed will be described with reference to FIG. When analyzing an image, it may be desired to analyze not the entire image but a range including a site such as the macula or the optic disc, or a characteristic range such as a lesion. At this time, local analysis is possible by designating the analysis range and performing the analysis. The analysis range designation unit 701 designates an analysis range in the image to be analyzed and notifies the storage unit 101-02, and the storage unit 101-02 stores the combination of the analysis range and the analysis method. The analysis unit 303 obtains an analysis result based on the combination of the analysis range and the analysis method in addition to the analysis target image and the analysis method acquired from the storage unit 101-02.

The processing flow in this modification will be described using the flowchart of FIG. 8 and the user interface of FIG. 7.

In S401 to S406, the analysis result based on the analysis method selected in the analysis method 501 is displayed in 502, and in S407, the analysis method 501 is changed as necessary. In this modification, when the analysis range is added or changed in S404 and S801 after S406, the analysis range 901 is specified at an arbitrary position in S802. In S803, the combination of the specified analysis range and the analysis method selected by the analysis method 501 is stored as an analysis setting. After that, in S404 and S406, the analysis result of the analysis range corresponding to each analysis method is displayed in 502.

Specifically, FIG. 7 is an example when the analysis range 901 is specified with the blood vessel area density analysis selected in the analysis method 501. At this time, the analysis range 901 is stored as an analysis setting in combination with the blood vessel area density analysis. Further, FIG. 11 is an example when the analysis method 501 is switched to the blood vessel length density from the state of FIG. 7 and the analysis range 902 is specified. At this time, the analysis range 902 is stored as an analysis setting in combination with the blood vessel length density analysis. At this time, the analysis range 901-2 specified by the blood vessel area density is not displayed. On the contrary, when the analysis range 501 is switched to the blood vessel area density, the analysis range 902 specified by the blood vessel length density is not displayed. Note that 901-2 may be displayed faintly so that it can be seen that the analysis range is set, or it may be configured so that it is not displayed at all.

Note that FIGS. 9 (a) and 9 (b) are examples in which a plurality of analysis ranges are specified by each analysis method, and the same analysis range may be specified by each analysis method as in the analysis range 903. At this time, in S402, when the blood vessel area density is selected by the analysis method 501, FIG. 9A is displayed in the analysis result 502, and when the blood vessel length density is selected by the analysis method 501, FIG. 9B is shown. It is displayed in the analysis result 502. Further, when "None" is selected in the analysis method 501, all the analysis results may not be displayed, or conversely, all the analysis results may be displayed as shown in FIG. 9 (c). .. In addition, the method of specifying the analysis range may be specified by drawing a closed area on the analysis target image by freehand using a mouse, or by specifying a plurality of points and specifying them using a straight line or a Bezier curve connecting them. You may. Further, it may be specified by deforming and moving a shape prepared in advance such as a circle or a grid (sector). Further, the specified analysis range is not limited to the two-dimensional plane. Depending on the type of analysis method, the analysis range may be specified by a straight line or a three-dimensional volume. Further, in the present embodiment and the modified example, the user specifies the analysis range and the analysis method, but the present invention is not limited to this. For example, as shown in FIG. 10, in each analysis target image, structural parts such as the optic nerve head (FIG. 10 (a)) and macula (FIG. 10 (b)), parts where the retina is thinned, surgical scars, and the like. A range that covers lesions and the like and a range that is divided into grids around them (at least one area of a sector divided into a plurality of areas) may be automatically set as an analysis range. Further, in the present embodiment and the modified examples, the blood vessel area density analysis and the blood vessel length density analysis have been described as examples, but another analysis method may be added. By the above method, the analysis results obtained by a plurality of analysis methods can be displayed suitable for observation for each analysis method, which is a very preferable result.

(Second Embodiment)
In the first embodiment, an example of designating an analysis range for each analysis method and displaying the analysis result has been described. In this embodiment, an example will be described in which a part of the configuration shown in the first embodiment is changed, and an analysis method is specified for each analysis range and the analysis result is displayed, contrary to the first embodiment. The purpose and the result are substantially the same as those of the first embodiment, but in the present embodiment, a plurality of analysis methods can be associated with one analysis range. The apparatus configuration in this embodiment will be described with reference to FIG. The analysis method selection unit 302 designates an analysis method for each analysis range designated by the analysis range designation unit 701, and notifies the storage unit 101-02.

The difference between the flow of processing in the present embodiment and the flowchart of the first embodiment (FIG. 4) will be described with reference to the flowchart of FIG. When the analysis target image is selected in S401 and then the analysis range is added or changed in S801, the analysis range is specified in S802. In S1201, the analysis method of the analysis range is specified. In S803, the combination of the analysis range and the analysis method is stored as the analysis setting. After that, the analysis method is selected in S402, and the analysis results of the selected analysis method are displayed in S403 to S407. Specifically, it will be described with reference to the user interface of FIG. 13 and the analysis result of FIG. In S802, the analysis ranges 901, 902, and 903 are specified regardless of the selection of the analysis method 501. In S1201, an analysis method is associated with each analysis range. Here, there are check boxes for blood vessel area density and blood vessel length density for each analysis range, and the configuration is such that the target analysis method is checked. The analysis range 901 is associated with blood vessel area density analysis only, the analysis range 902 is associated with blood vessel length density analysis, and the analysis range 903 is associated with blood vessel area density analysis and blood vessel length density analysis. At this time, in S402, when the blood vessel area density is selected by the analysis method 501, FIG. 9A is displayed in the analysis result 502, and when the blood vessel length density is selected by the analysis method 501, FIG. 9B is shown. It is displayed in the analysis result 502. Further, when "None" is selected in the analysis method 501, all the analysis results may not be displayed, or conversely, all the analysis results may be displayed as shown in FIG. 9 (c). ..

Here, the difference between the first embodiment and the present embodiment is the difference between setting the analysis range after selecting the analysis method (first embodiment) and selecting the analysis method after selecting the analysis range (this embodiment). Is. Of course, in the present embodiment, in combination with the configuration of the first embodiment, the configuration may be such that the analysis range can be added or changed after the analysis method is selected. The analysis method may be automatically selected based on the specified analysis range. For example, when the analysis range is around the optic nerve head as shown in FIG. 15 (a), the blood vessel length density analysis is automatically selected, and when the analysis range is around the macula as shown in FIG. 15 (b), it is automatically selected. The blood vessel area density analysis may be selected. By the above method, the analysis results obtained by a plurality of analysis methods can be displayed suitable for observation for each analysis method, which is a very preferable result.

(Third Embodiment)
When analyzing an image, it may be desired to check the analysis result not only in a specific analysis method but also in a state where a plurality of analysis methods are mixed. For example, this corresponds to the case where one analysis range A is to be displayed simultaneously by blood vessel area density analysis and another analysis range B is to be displayed simultaneously by blood vessel length density analysis. In this embodiment, variations of the display method when there are a plurality of analysis ranges in the image to be analyzed or when a plurality of analysis methods correspond to one analysis range will be described.

FIG. 16 shows an example of a user interface for switching the display / non-display of the analysis result at an arbitrary timing. A user interface for switching the display / non-display setting 1601 of each analysis result may be prepared, and the analysis result may be displayed according to the display setting 1601. As a result, the analysis results of the plurality of analysis methods can be switched and confirmed at any timing. Further, the initial value of each display setting may be determined according to the analysis method. For example, when the blood vessel area density analysis is selected for the analysis method 501, the display setting 1601 displays the blood vessel area density and hides the blood vessel length density, and when the blood vessel length density analysis is selected for the analysis method 501, the display setting. The 1601 may be set so that the blood vessel area density is not displayed and the blood vessel length density is displayed.

Further, FIG. 17 is a display example of the analysis result when a plurality of analysis methods are associated with one analysis range. As shown in FIG. 17, the analysis results of each analysis method may be displayed together. At this time, in order to make it easy to identify which analysis method each analysis result is based on, for example, ■ in the analysis result of the blood vessel area density analysis, ★ in the analysis result of the blood vessel length density analysis, etc. The identifier of may be given. In addition, the analysis result by the selected analysis method may be highlighted in bold or colored. Further, the analysis result may be switched and displayed at regular intervals. Further, it may be possible to select which analysis result is to be displayed for each analysis range and to switch between display and non-display. Further, FIG. 18 is an example in which FIG. 17 is tabulated. As the analysis result, a table as shown in FIG. 17 may be displayed. In addition, a graph or other format converted may be displayed as an analysis result. By the above method, the analysis results obtained by a plurality of analysis methods can be displayed suitable for observation for each analysis method, which is a very preferable result.

(Modification example)
As one of the modifications of the embodiment, it is preferable that one of the image processing devices according to the embodiment includes a function of follow-up (follow-up) of the eye to be inspected. For example, it is preferable that the display control unit 101-05 displays a plurality of motion contrast images corresponding to a plurality of inspections performed on different days side by side on the display unit 304. Further, the display control unit 101-05 may display a plurality of analysis maps (images showing the analysis results) corresponding to a plurality of inspections performed on different days side by side on the display unit 304. That is, the display control unit 101-05 may display a plurality of two-dimensional images (a plurality of motion contrast front images and a plurality of analysis maps) corresponding to a plurality of inspections performed on different days side by side on the display unit 304. I hope I can. As the plurality of analysis maps, it is preferable to select an analysis map analyzed by the same type of analysis method (type of analysis). Further, a plurality of inspections corresponding to a plurality of two-dimensional images to be displayed on the display unit 304 may be configured to be selected according to an instruction of the operator. Further, in the display control unit 101-05, at least one analysis region (for example, a first region or a second region which is a partial region) selected for each of the plurality of inspections according to the instruction of the operator is provided. A plurality of two-dimensional images may be displayed on the display unit 304 in the superimposed state. Further, when the setting related to the reference inspection is selected (changed) according to the instruction of the operator, it is preferable that the setting related to other inspections is selected (changed) at the same time (in conjunction with). For example, the display control unit 101-05 is configured to apply at least one analysis area selected for the reference inspection selected according to the instruction of the operator to other inspections. Is also good. That is, the position of the analysis region on the two-dimensional image corresponding to the reference inspection may be reflected in the corresponding position on the two-dimensional image corresponding to the other inspection. The analysis area may be an area manually set for the two-dimensional image according to the instruction of the operator, or at least one area selected from the sectors divided into a plurality of areas. It may be an area (updated area) selected by moving a sector with respect to a two-dimensional image. Further, the setting to be changed may be an analysis method (analysis type) other than the analysis area. Further, the display control unit 101-05 may execute control to change the display of the plurality of analysis maps in conjunction with the display of the plurality of analysis maps of different analysis types. At this time, even if the display control unit 101-05 executes control to hide the information indicating the analysis area (or the information indicating the analysis result) superimposed and displayed on the plurality of analysis maps. good. Further, the display control unit 101-05 may display the time series data of the analysis results corresponding to the plurality of inspections on the display unit 304. Here, the time-series data is, for example, a time-series graph, but any data expressed in time (numerical values or charts) may be used. Further, the display control unit 101-05 may display the time series data of the integrated analysis result showing the ratio or difference of the analysis results of the plurality of analysis regions on the display unit 304. As a result, for example, an operator such as a doctor can efficiently follow up (follow-up) the eye to be inspected.

(Other embodiments)
The present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiment is supplied to the system or device via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or device reads the program. This is the process to be executed.

Claims (16)

A selection means for selecting the first type of analysis or the second type of analysis according to the instruction from the operator, and
A setting means for setting a first region and a second region, which are partial regions of the motion contrast image of the eye portion, according to an instruction from the operator .
An analysis means for analyzing the set first region and the second region, and
The set first region and the second region were displayed on the display means in a state of being superimposed on the motion contrast image, and the first region was analyzed by the first type of analysis. It has a result and a display control means for causing the display means to display the result of analysis of the second region by the second type of analysis .
When the first type of analysis is selected, the display control means causes the result of the analysis by the first type of analysis to be displayed in the first region in a state of being superimposed on the motion contrast image. An image processing apparatus characterized by displaying information indicating the first type . The analysis means analyzes the motion contrast image by a second type of analysis different from the first type of analysis , which is selected according to the instruction of the operator .
The display control means is the result of analysis by the first type of analysis in the display area when the second type of analysis is selected after the first type of analysis is selected. The image processing apparatus according to claim 1 , wherein the control is performed to change the display to the display of the result analyzed by the second type of analysis . The display control means is characterized in that an image in which the second region is superimposed on the motion contrast image is displayed on the display means, and the result of analysis of the second region is displayed . The image processing apparatus according to claim 2 . The image according to claim 2 or 3 , wherein the display control means executes a control for changing the first area to be hidden when the control for changing the display is executed. Processing equipment. 2 . _ _ _ _ _ _ The image processing device described. The image processing apparatus according to claim 5 , wherein the display control means displays the result of analysis of the second region by the second type of analysis . The display control means displays the result of the analysis of the first region by the first type of analysis , and the result of the analysis of the second region by the second type of analysis . The image processing apparatus according to claim 5 or 6 , wherein the control for changing the display is executed. The display control means is a two-dimensional image of the motion contrast image, and a plurality of two-dimensional images corresponding to a plurality of inspections performed on different days are arranged and displayed on the display means.
7 . The image processing apparatus according to claim 1. The image processing apparatus according to claim 8 , wherein when the setting related to the reference inspection is changed according to the instruction of the operator, the setting related to other inspections is changed at the same time. The image processing apparatus according to claim 8 or 9 , wherein the display control means causes the display means to display time-series data as a result of analyzing the partial region. After the partial region is selected for the motion contrast image, the first type of analysis and the first type of analysis are performed in order to analyze the partial region by the first type of analysis and the second type of analysis. The image processing apparatus according to any one of claims 1 to 10 , wherein one of two types of analysis is configured to be selected. One of claims 1 to 11 , wherein the first type and the second type are any of a plurality of types including at least a parameter relating to a blood vessel region specified in the motion contrast image. The image processing apparatus described in the section. The first type and the second type may be any one of a plurality of types including at least a blood vessel density relating to the area of the blood vessel region specified in the motion contrast image and a blood vessel density relating to the length of the blood vessel region. The image processing apparatus according to any one of claims 1 to 12 , wherein the image processing apparatus is provided. One of claims 1 to 13 , wherein the first type and the second type are any of a plurality of types including at least a parameter relating to an avascular region specified in the motion contrast image. The image processing apparatus according to item 1. The process of selecting the first type of analysis or the second type of analysis,
The process of setting the first region and the second region, which are partial regions of the motion contrast image of the eye part, and
The step of analyzing the set first region and the second region, and
The set first region and the set second region are displayed on the display means in a state of being superimposed on the motion contrast image, and the first region is obtained by the first type of analysis. It has a control step of displaying the analysis result and the result of analysis of the second region by the second type of analysis on the display means.
When the first type of analysis is selected in the control step, the result of the analysis by the first type of analysis is displayed in the first region in a state of being superimposed on the motion contrast image. , An image processing method comprising displaying information indicating the first type . A program comprising causing a computer to execute each step of the image processing method according to claim 15 .

Images