JP6849776B2 - Information processing device and information processing method - Google Patents

Description

The present invention relates to an information processing device and an information processing method .

Optical coherence tomography (hereinafter referred to as OCT) has been put into practical use as a non-destructive and non-invasive method for acquiring a tomographic image of a living body or the like. OCT is widely used in ophthalmic diagnosis of the retina because a tomographic image of the retina in the fundus of the eye to be examined is acquired, especially in the field of ophthalmology.

OCT obtains a tomographic image by interfering the light reflected from the measurement target with the light reflected from the reference mirror and analyzing the time dependence or wave number dependence of the interfering light intensity. As such an optical coherence tomography acquisition device, a time domain OCT, a spectral domain OCT, or a wavelength sweep OCT is known. In the time domain OCT, the depth information of the measurement target is obtained by changing the position of the reference mirror. In a spectral domain OCT (SD-OCT: Spectral Domine Optical Coherence Tomography) using a wideband light source, interference light is separated for each wavelength by a spectroscope to obtain depth information of a measurement target. The wavelength sweep optical coherence tomography (SS-OCT) device uses a wavelength variable light source device capable of changing the oscillation wavelength. In addition, SD-OCT and SS-OCT are collectively called (FD-OCT: Fourier Optical Coherence Tomography).

In recent years, a pseudo-angiography method using this FD-OCT has been proposed and is called OCT angiography (OCTA) (Non-Patent Document 1). Fluorescence imaging, which is a common angiography method in modern clinical medicine, requires injection of a fluorescent dye (for example, fluorescein or indocyanine green) into the body, and displays the blood vessels through which the fluorescent dye passes in two dimensions. On the other hand, OCTA enables non-invasive and pseudo angiography and enables three-dimensional display of the network of blood flow sites. Furthermore, OCTA is attracting attention because it has a higher resolution than fluorescence contrast and can visualize microvessels or blood flow in the fundus.

A plurality of OCTA methods have been proposed depending on the difference in blood flow detection method. For example, Patent Document 2 proposes a method of separating an OCT signal from a bloodstream by extracting only a time-modulated signal from an OCT signal. Non-Patent Document 3 proposes a method utilizing phase variation due to blood flow, and Non-Patent Document 4 and Patent Document 1 propose a method utilizing intensity variation due to blood flow.

USP Pub. No. 2014/221827 A1

Makita et al., “Optical Coherence Angiography,” Optics Express, 14 (17), 7821-7840 (2006) An et al. “Optical microangiography provides correlation between microstructure and microvasculature of optic nerve head in human subjects,” J. Biomed. Opt. 17, 116018 (2012) Fingler et al. “Mobility and transverse flow visualization using phase variance contrast with spectral domain optical coherence tomography” Optics Express. Vol. 15, No. 20. pp 12637-12653 (2007) Mariampillai et al., “Optimized speckle variance OCT imaging of microvasculature,” Optics Letters 35, 1257-1259 (2010)

However, since the above-mentioned OCTA can obtain detailed blood flow sites, it may be difficult for the examiner to understand the connection and running of specific blood vessels. Further, the image regarding the blood flow site obtained by the above-mentioned OCTA and the image regarding the structural information obtained by the OCTA are obtained as independent and separate images. Therefore, in the actual diagnosis, it is necessary for the examiner to compare these two images alternately. However, since the detailed blood flow site obtained by OCTA is detailed, it is difficult to understand the correspondence with the structural information of the eye to be inspected obtained from the intensity image of OCT, the photograph of the fundus, and the like.

The present invention has been made in view of such a situation, and an object of the present invention is to facilitate correspondence between an image relating to a blood flow site obtained by OCTA and structural information obtained by OCT or the like.

In order to solve the above problems, the information processing device according to one aspect of the present invention is
Using each of the data of a plurality of 3-dimensional motion contrast data obtained by photographing a subject's eye at different times, information acquisition means for acquiring measurement information for at least one blood vessel,
A classification means for classifying at least one blood vessel using the acquired measurement information, and
It is a motion contrast Enface image generated by using at least a part range of each data of the plurality of three-dimensional motion contrast data in the depth direction of the eye to be inspected, and corresponds to the plurality of three-dimensional motion contrast data. A display control means for displaying a plurality of motion contrast Enface images side by side on a display means, and displaying display information indicating positions corresponding to each other in the plurality of motion contrast Enface images on the display means.
To be equipped.

According to the present invention, it is possible to easily obtain a correspondence between an image relating to a blood flow site obtained by OCTA and structural information obtained by OCT or the like.

It is a figure which shows the outline of the whole structure of the apparatus in 1st Embodiment of this invention. It is a figure which shows an example of the scan pattern in 1st Embodiment. It is a flowchart which shows the whole processing procedure of image generation in 1st Embodiment. It is a flowchart which shows the interference signal acquisition procedure in 1st Embodiment. It is a flowchart which shows the signal processing procedure in 1st Embodiment. It is a flowchart which shows the 3D blood flow part information acquisition and display procedure in 1st Embodiment. It is explanatory drawing of the segmentation result in 1st Embodiment. It is a figure which shows an example of the display screen in 1st Embodiment. It is a figure which shows an example of the display method in 1st Embodiment. The figure which shows an example of the depth selection method in 2nd Embodiment. The figure which shows an example of the method of generating the 2D intensity image in 2nd Embodiment. The figure which shows an example of the display method of the selected part in 2nd Embodiment. It is a flowchart which shows an example of the area division processing procedure. It is a figure which shows an example of the display method which displays two images. It is a flowchart which shows an example of the comparison procedure performed at the time of estimating a blood vessel. It is a figure which shows an example of the display method of the analysis result of the blood vessel estimation. It is a figure which shows an example of the display screen in 3rd Embodiment. It is a figure which shows an example of the display screen in 4th Embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. It should be noted that the following embodiments do not limit the present invention relating to the scope of claims, and that all combinations of features described in the following embodiments are essential for the means for solving the present invention. Not exclusively.

Further, in the present specification, an image in which an OCT signal is displayed as an image is referred to as an intensity image. Further, a signal obtained by displaying a time-modulated signal from the OCT signal as an image is called a motion contrast image, the pixel value thereof is called a motion contrast, and the data set is called a motion contrast data.

(First Embodiment)
In this embodiment, an example is shown in which a tomographic image is generated from a captured three-dimensional optical interference signal, motion contrast is calculated, and three-dimensional blood flow site information is acquired.

[Structure of the entire image forming apparatus]
FIG. 1 is a diagram showing a configuration example of an image forming method and an apparatus using the optical coherence tomography method according to the embodiment of the present invention. The image forming method and device are composed of an OCT device 100 and a control unit 143, which are optical coherence tomography acquisition units that acquire optical coherence tomography signals. As the OCT apparatus, for example, the SD-OCT and SS-OCT described above can be applied to the present invention. In the embodiment described below, the configuration when the OCT apparatus is SS-OCT is shown.

<Configuration of OCT device 100>
The configuration of the OCT apparatus 100 will be described below with reference to FIG.
The light source 101 is a wavelength sweep type (Swept Source: SS) light source, and emits light while sweeping at a sweep center wavelength of 1050 nm and a sweep width of 100 nm, for example.

The light emitted from the light source 101 is guided to the beam splitter 110 via the optical fiber 102 and split into measurement light (also referred to as OCT measurement light) and reference light (also referred to as reference light corresponding to OCT measurement light). .. The branching ratio of the beam splitter 110 is 90 (reference light): 10 (measurement light). The branched measurement light is emitted to the measurement optical path via the optical fiber 111. In the measurement optical path, the collimator lens 112, the galvano scanner 114, the scan lens 115, and the focus lens are arranged in this order from the optical fiber 111 to the eye to be inspected 118. 116 is placed.

The measurement light emitted into the measurement optical path is made parallel light by the collimator lens 112. The parallel light is incident on the eye 118 via the galvano scanner 114, the scan lens 115, and the focus lens 116 that scan the measurement light on the fundus Er of the eye 118. Here, the galvano scanner 114 is described as a single mirror, but is actually composed of two galvano scanners (for example, an X-axis scanner and a Y-axis scanner) (not shown) so as to raster scan the fundus Er of the eye 118 to be inspected. ing.

The focus lens 116 is fixed on the stage 117, and the focus can be adjusted by the focus lens 116 by moving the stage 117 in the optical axis direction. The galvano scanner 114 and the stage 117 are controlled by the signal acquisition control unit 145, which will be described later, in a desired range of the fundus Er of the eye to be inspected 118 (also referred to as a tomographic image acquisition range, a tomographic image acquisition position, and a measurement light irradiation position). The measurement light can be scanned.

Although detailed description is not given in this embodiment, it is desirable that a tracking function for detecting the movement of the fundus Er and scanning the mirror of the galvano scanner 114 by following the movement of the fundus Er is provided. The tracking method can be performed using a general technique, and can be performed in real time or by post-processing.

As a method of obtaining a fundus image for detecting the movement of the fundus Er, for example, there is a method of using a scanning laser ophthalmoscope (SLO). In this method, with respect to the image of the fundus Er, an in-plane image (fundus surface image) perpendicular to the optical axis is acquired over time using SLO, and characteristic points such as blood vessel branches in the image are extracted. Real-time tracking can be performed by calculating how the feature portion in the acquired two-dimensional image moves as the movement amount of the fundus Er, and feeding back the calculated movement amount to the galvano scanner 114.

As described above, the measurement light is incident on the eye to be inspected 118 via the focus lens 116 mounted on the stage 117, and is focused on the fundus Er by the focus lens 116. The measurement light that irradiates the fundus Er is reflected and scattered in each retinal layer, and returns to the beam splitter 110 via the above-mentioned measurement optical path. The return light of the measurement light incident on the beam splitter 110 passes through the optical fiber 126 and is incident on the beam splitter 128.

On the other hand, the reference light branched by the beam splitter 110 is emitted to the reference optical path via the optical fiber 119a, the polarization controller 150, and the optical fiber 119b. 120, dispersion compensating glass 122, ND filter 123, and collimator lens 124 are arranged.

The reference light emitted from the optical fiber 119b is collimated by the collimator lens 120. The polarization controller 150 can change the polarization of the reference light to a desired polarization state. The reference light enters the optical fiber 127 via the dispersion compensating glass 122, the ND filter 123, and the collimator lens 124. One end of the collimator lens 124 and the optical fiber 127 is fixed on the coherence gate stage 125, and the collimator lens 124 or the like is driven in the optical axis direction in response to a difference in the axial length of the subject. , It is controlled by the signal acquisition control unit 145, which will be described later. Although the optical path length of the reference light is changed in this embodiment, it is sufficient that the optical path length difference between the optical path of the measurement light and the optical path of the reference light can be changed.

The reference light that has passed through the optical fiber 127 enters the beam splitter 128. In the beam splitter 128, the return light and the reference light of the measurement light described above are combined to form interference light, and then split into two. The divided interference light is interference light having phases that are inverted from each other (hereinafter, referred to as a positive component and a negative component). The positive component of the divided interference light enters one input port of the detector 141 via the optical fiber 129. On the other hand, the negative component of the interference light enters the other input port of the detector 141 via the optical fiber 130. The detector 141 is a differential detector, and when two interference signals whose phases are inverted by 180 ° are input, the DC component is removed and only the interference component is output.

The interference signal detected by the detector 141 is output as an electric signal according to the intensity of light, and is input to the signal processing unit 144, which is an example of the tomographic image generation unit.

<Structure of control unit 143>
The control unit 143 for controlling the entire apparatus will be described.
The control unit 143 is composed of a signal processing unit 144, a signal acquisition control unit 145, a display unit 146, and a display control unit 149. Further, the signal processing unit 144 further includes an image generation unit 147 and a map generation unit 148. The image generation unit 147 has a function of generating a luminance image and a motion contrast image from the transmitted electric signal, and the map generation unit 148 has a function of generating layer information (segmentation of the retina) from the luminance image.

The signal acquisition control unit 145 controls each unit such as the stage 117 and the coherence gate stage 125 as described above. The signal processing unit 144 generates an image, analyzes the generated image, and generates visible information of the analysis result based on the signal output from the detector 141.

The image and analysis result generated by the signal processing unit 144 are sent to the display control unit 149, and the display control unit 149 displays the image and the analysis result on the display screen of the display unit 146. Here, the display unit 146 is a display such as a liquid crystal display. The image data generated by the signal processing unit 144 may be transmitted to the display control unit 149 and then to the display unit 146 by wire or wirelessly. Further, in the present embodiment, the display unit 146 and the like are included in the control unit 143, but the present invention is not limited to this, and the display unit 146 and the like may be provided separately from the control unit 143, for example, an example of a device that can be carried by the user. It may be a tablet that is. In this case, it is preferable that the display unit is equipped with a touch panel function so that the display position of the image can be moved, enlarged / reduced, the displayed image can be changed, and the like can be operated on the touch panel.

[Scan pattern]
<Scanning mode of measurement light in OCT device>
Here, a mode for scanning the measurement light in the OCT apparatus will be described. As described above, the interference light is obtained from the return light of the measurement light irradiated to an arbitrary point on the fundus of the eye 118 to be inspected and the corresponding reference light. The signal processing unit 144 processes an electric signal according to the intensity of the interference light detected by the detector 141 to obtain image data in the depth direction at the arbitrary point. The above is the description of the process of acquiring information on the tomography at one point of the eye to be inspected 118. Acquiring information on the tomographic fault in the depth direction of the eye 118 in this way is called A-scan.

Further, acquisition of information on the tomography of the eye 118 to be inspected in the direction orthogonal to A-scan, that is, information on a plurality of tomography in the depth direction described above in the scanning direction for acquiring a two-dimensional image is called B-scan. Call. Further, acquiring information on a plurality of faults in the depth direction in the scanning direction orthogonal to both the scanning directions of A-scan and B-scan is called C-scan. When two-dimensional raster scanning is performed in the bottom surface of the eye when acquiring a three-dimensional tomographic image, the high-speed scanning direction is the B-scan direction, and the low-speed scanning direction in which B-scan is arranged in the orthogonal direction is the C-scan direction. Called. A two-dimensional tomographic image can be obtained by performing A-scan and B-scan, and a three-dimensional tomographic image can be obtained by performing A-scan, B-scan and C-scan. These B-scan and C-scan are performed by the above-mentioned galvano scanner 114.

As described above, the galvano scanner 114 is composed of an X-axis scanner and a Y-axis scanner (not shown), each of which is composed of deflection mirrors arranged so that their rotation axes are orthogonal to each other. For example, the X-axis scanner scans the measurement light in the X-axis direction on the fundus Er, and the Y-axis scanner scans in the Y-axis direction. Each of the X-axis direction and the Y-axis direction is a direction perpendicular to the eye axis direction of the eyeball and is a direction perpendicular to each other.

Further, the line scanning direction such as B-scan and C-scan does not have to coincide with the X-axis direction or the Y-axis direction. Therefore, the line scanning direction of B-scan and C-scan can be appropriately determined according to the two-dimensional tomographic image or the three-dimensional tomographic image to be imaged.

<Scanning mode of measurement light in this embodiment>
Since OCTA measures the time change of the OCT interference signal due to blood flow, it is necessary to measure it multiple times at the same place. In the present embodiment, the OCT apparatus repeats the B scan (scan in the X-axis direction) at the same location m times, and at n positions (the position where the irradiation position of the measurement light is moved in the Y-axis direction) at the scan position. Perform a scan to move. A specific scan pattern is shown in FIG. As shown in the figure, in the present embodiment, B scan is repeated m times for each of n positions of y1 to yn on the fundus plane.

Here, when the value of m is large, the number of measurements at the same place increases, so that the accuracy of detecting blood flow is improved. On the other hand, the scanning time becomes long, and there arises a problem that motion artifacts occur in the image due to eye movement (fixation fine movement) during scanning and a problem that the burden on the subject increases. In this embodiment, m = 4 is set in consideration of the balance between the two. The control unit 143 may automatically change m according to the A scan speed of the OCT apparatus and the motion analysis of the fundus surface image of the eye to be inspected 118.

Further, in FIG. 2, p indicates the number of samples of A scan in one B scan. That is, in one B scan, the A scan is performed at each of the positions x1 to xp, and the plane image size is determined by pxn. If the value of p × n is large, a wide range can be scanned at the same measurement pitch, but the scanning time becomes long, and the above-mentioned motion artifact and patient burden problems occur. In this embodiment, n = p = 300 in consideration of the balance between the two. The values of n and p described above can be freely changed as appropriate.

Further, Δx in FIG. 2 is the interval (x pitch) between adjacent x positions, and Δy is the interval (y pitch) between adjacent y positions. In the present embodiment, the x-pitch and y-pitch are determined as 1/2 of the beam spot diameter of the irradiation light on the fundus and set to 10 μm. By setting the x-pitch and y-pitch to 1/2 of the diameter of the beam spot on the fundus, it is possible to form a high-definition image to be generated. Even if the x-pitch and y-pitch are made smaller than 1/2 of the fundus beam spot diameter, the effect of further increasing the definition of the generated image is small.

On the contrary, if the x-pitch and the y-pitch are made larger than 1/2 of the fundus beam spot diameter, the definition is deteriorated, but a wide range of images can be acquired with a small data capacity. However, the x-pitch and the y-pitch may be freely changed according to clinical requirements.
The scan range of this embodiment is p × Δx = 3 mm in the x direction and n × Δy = 3 mm in the y direction.

<Acquisition mode of OCT interference signal and OCTA signal>
The acquisition of the OCT interference signal and the acquisition of the OCTA signal may be the same step or different steps. Here, a case where the acquisition of the OCT interference signal and the acquisition of the OCTA signal are acquired in the same process will be described. When acquiring in the same process, the signal acquisition control unit 145 acquires an interference signal set for a plurality of frames used when forming a three-dimensional tomographic image by measuring the same position multiple times in scanning with measurement light. To do. Here, the signal acquisition control unit 145 functions as a signal acquisition means. The interference signal set is a set of interference signals obtained by the above-mentioned one B scan, and means a set of interference signals capable of generating a tomographic image by one frame by the B scan. In addition, since the eye to be inspected constantly performs fixation tremor, even if an attempt is made to construct an image of the same cross section of the eye to be inspected by scanning the same position in the eye to be inspected multiple times to obtain an interference signal set for a plurality of frames, they are actually the same. It is difficult to obtain an accurate cross section. Therefore, the interference signal set for a plurality of frames described here is grasped as an interference signal set for a plurality of frames intentionally acquired with the same cross section.

The signal processing unit 144 calculates the OCT interference signal and the OCTA signal by processing these interference signal sets. By using the interference signal set acquired in the same process, each pixel of the intensity image by OCT and the motion contrast image by OCTA can be set to the same position. That is, in the present embodiment, the interference signal set used for generating the OCT intensity image is included in the interference signal set for a plurality of frames acquired to generate the motion contrast image by OCTA. Further, the OCT intensity image displayed as a two-dimensional image is averaged (superposed) using at least two or more interference signal sets of the above-mentioned acquired interference signal sets for a plurality of frames, that is, two frames. You may get it. With such a configuration, the random noise included in the interference signal set is averaged, and the noise of the intensity image can be reduced. Here, the signal processing unit 144 functions as a motion contrast image generation means for generating a motion contrast image by using the pixel data corresponding to each frame in the interference signal set for a plurality of frames constituting the same cross section.

Next, a case where the acquisition of the OCT interference signal and the acquisition of the OCTA signal are acquired in separate steps will be described. In the case of separate steps, the signal acquisition control unit 145 causes, for example, the acquisition of the OCTA interference signal to be followed by the acquisition of the OCTA signal. In the scan pattern at the time of acquiring the OCT interference signal, the values such as the number of repetitions and the pitch corresponding to the scan pattern are used. For example, the acquisition of the OCT interference signal does not necessarily have to repeat the B scan m times. Further, the scan range does not have to be the same for the acquisition of the OCT interference signal and the acquisition of the OCTA signal. Therefore, if the scan time is the same, the acquisition of the OCT interference signal can be acquired in a wider range as the first predetermined range. In addition, when separate processes are used, OCTA signals are acquired in detail only in a predetermined area of interest as a second predetermined range included in the first predetermined range while observing a wide range with a three-dimensional tomographic image by OCT. Can be used.

Next, a specific processing procedure of the image forming method of the present embodiment will be described with reference to the flowchart shown in FIG. A detailed description of each process shown in the flow will be described later. In FIG. 3, an example in which the acquisition of the OCT interference signal and the acquisition of the OCTA signal are acquired in separate steps will be described.

In step S101, which is the first signal acquisition step, that is, the subject image acquisition step displayed as a two-dimensional image, the signal acquisition control unit 145 controls the OCT device 100 to acquire an optical interference tomographic signal.

In step S102, which is the second signal acquisition step, that is, the signal acquisition step, the signal acquisition control unit 145 controls the OCT device 100 and acquires a plurality of frames of optical coherence tomographic signals. More specifically, the interference signal set based on the measurement light controlled to scan the same position a plurality of times in the sub-scanning direction is acquired for a plurality of frames. Further, this operation is executed a plurality of times to obtain an interference signal set from a plurality of different cross sections while shifting the scanning position in the sub-scanning direction, and an interference signal set sufficient to form a three-dimensional tomographic image is acquired.

In step S103, which is an intensity image generation step, the control unit 143 calculates the three-dimensional tomographic image data of the eye to be inspected 118 based on the optical interference tomographic signal acquired in the first signal acquisition step, and generates an intensity image. .. Further, the control unit 143 calculates the three-dimensional motion contrast data from the optical interference tomographic signals of a plurality of frames acquired in the second signal acquisition step, and generates a three-dimensional motion contrast image.

In step S104, which is a display step, the control unit 143 generates and displays display information from the intensity image and the three-dimensional motion contrast image. The control unit 143 reconstructs and redisplays the display information according to the operation of the examiner. A detailed description of the reconstruction process will be described later.
By carrying out the above steps, the procedure for processing the image forming method of the present embodiment is completed.

[Interference signal acquisition procedure]
Next, the specific processing procedure of the first and second signal acquisition steps S101 and S102 of the above-described embodiment will be described with reference to FIG.

In step S109, the signal acquisition control unit 145 sets the index i of the position yi in FIG. 2 to 1. In step S110, the OCT apparatus 100 moves the scan position to yi. In step S119, the signal acquisition control unit 145 repeatedly sets the index j of the B scan to 1. In step S120, the OCT apparatus 100 performs a B scan.

In step S130, the detector 141 detects an interference signal for each A scan, and the above-mentioned interference signal is stored in the signal processing unit 144 via an A / D converter (not shown). The signal processing unit 144 acquires the interference signal of the A scan as a p-sample to obtain the interference signal for the 1B scan.

In step S139, the signal acquisition control unit 145 repeatedly increments the index j of the B scan.

In step S140, the signal acquisition control unit 145 determines whether j is larger than the predetermined number of times (m). That is, it is determined whether the B scan at the position y is repeated m times. If it is not repeated, the process returns to S120 and the B scan measurement at the same position is repeated. If it is repeated a predetermined number of times, the process proceeds to S149. In step S149, the signal acquisition control unit 145 increments the index i of the position yi. In step S150, the signal acquisition control unit 145 determines whether i is larger than the number of measurements (n) at a predetermined Y position, that is, whether the B scan is performed at all y positions at n locations. If the number of measurements at the predetermined Y position is less than (no), the process returns to S110 and the measurement at the next measurement position is repeated. When the number of measurements at the predetermined Y position is completed (yes), the process proceeds to the next step S160. In the first and second signal acquisition steps S101 and S102 described here, i = 1 and m = 4, respectively.

In step S160, the OCT apparatus 100 acquires background data. The OCT device 100 executes 100 A scans with the shutter (not shown) closed, and the signal acquisition control unit 145 averages and stores the signals obtained in the 100 A scans. The number of background measurements is not limited to 100 as illustrated here.

By carrying out each of the above processes, the interference signal acquisition procedure of the present invention is completed.

In the acquisition of the OCTA interference signal, the OCTA interference signal can be acquired in the same procedure as the OCTA signal acquisition by setting the number of measurements and the position to the values corresponding to the acquisition of the OCTA interference signal.

In the present embodiment, in order to acquire the OCT signal and form a three-dimensional tomographic image, a subject image displayed as two-dimensional is generated based on the data obtained from the interference signal set by the OCT signal. There is. As described above, the range for acquiring the OCTA signal does not have to be the same as the range for acquiring the OCTA signal, and therefore, as described above, this subject image includes the second range for acquiring the OCTA signal in the eye 118 to be examined. Generated as an image of the first range.

[Signal processing procedure]
Next, with reference to FIG. 5, a specific process for generating display information including the three-dimensional image information of the OCT of step S103 and the three-dimensional blood flow site information of the OCTA shown in FIG. 3 as the present embodiment will be described. In the present invention, the motion contrast of OCTA is calculated in order to generate three-dimensional blood flow site information from OCTA information.

In step S210, the signal processing unit 144 sets the index i of the position yi to 1. In step S220, the signal processing unit 144 extracts the repeated B scan interference signal (m times) at the position yi (here, i = 1). In step S230, the signal processing unit 144 sets the index j of the repeated B scan to 1. In step S240, the signal processing unit 144 extracts the j-th (here, j = 1) B scan data.

In step S250, the signal processing unit 144 generates a luminance image of the tomographic image by performing reconstruction processing on the interference signal of the B scan data in step S240. First, the image generation unit 147 removes fixed pattern noise composed of background data from the interference signal. Fixed pattern noise removal is performed by extracting fixed pattern noise by averaging the A scan signals of a plurality of detected background data and subtracting the fixed pattern noise from the input interference signal. Next, the image generation unit 147 performs a desired window function processing in order to optimize the depth resolution and the dynamic range, which are in a trade-off relationship when the Fourier transform is performed in a finite interval. After that, a luminance image of a tomographic image is generated by performing FFT processing.

In step S260, the signal processing unit 144 repeatedly increments the index j of the B scan. In step S270, the signal processing unit 144 determines whether or not the incremented j is greater than m. That is, it is determined whether the luminance calculation of the B scan at the position y is repeated m times. In the case of no, the process returns to S240, and the brightness calculation of the repeated B scan at the same Y position is repeated. If yes, the flow proceeds to the next step.

In step S280, the signal processing unit 144 aligns the m-frame of the repeated B scan at a certain y position. Specifically, first, the signal processing unit 144 selects any one of the m frames as a template. For the frames selected as the template, the correlation may be calculated for all combinations of each other, the sum of the correlation coefficients may be obtained for each frame, and the frame having the maximum sum may be selected. Next, the template is used to collate each frame, and the amount of misalignment (δX, δY, δθ) for each is obtained. Specifically, Normalized Cross-Correlation (NCC), which is an index showing the degree of similarity while changing the position and angle of the template image, is calculated, and the difference in image position when this value is maximized is calculated as the amount of misalignment. ..

In the present invention, the index indicating the degree of similarity can be variously changed as long as it is a scale indicating the similarity between the features of the image in the template and the frame. For example, Sum of Abusolute Difference (SAD), Sum of Squared Difference (SSD), Zero-means Normalized Cross-Correlation (ZNCC), Phase Only Correlation (POC), Rotation Invariant Phase Only Correlation (RIPOC) and the like may be used.

Next, the signal processing unit 144 applies the position correction to the m-1 frame other than the template according to the amount of misalignment (δX, δY, δθ), and aligns the m frame. Further, when the acquisition of the OCT 3D image information and the OCTA 3D blood flow site information is performed in separate steps as in the present embodiment, even between the 3D image information and the 3D blood flow site information. Align. That is, it was calculated based on the 3D tomographic image data calculated based on the interference signal set acquired for the purpose of generating the OCT intensity image and the interference signal set acquired for the purpose of generating the motion contrast image. Alignment with the data of the 3D tomographic image is performed. By performing these alignments, it becomes easy to correspond the three-dimensional image information and the three-dimensional blood flow site information. Alignment can be performed in the same manner as alignment between frames.

In step S290, the signal processing unit 144 averages the aligned luminance images calculated in step S280 to generate a luminance averaged image.

In step S300, the map generation unit 148 executes retina segmentation (site information acquisition) from the luminance averaging image generated by the signal processing unit 144 in step S290. Since this step is not used in the first embodiment described here, this step is skipped. The description will be given in the second embodiment.

In step S310, the image generation unit 147 calculates the motion contrast. In the present embodiment, the variance value is calculated for each pixel at the same position from the luminance image of each tomographic image of the m frame output by the signal processing unit 144 in step S250, and the variance value is used as the motion contrast.

There are various ways to obtain the motion contrast. For example, the type of feature amount used as motion contrast in the present invention is a change in the brightness value of each pixel of a plurality of B scan images at the same Y position. Therefore, any index showing the change in the brightness value can be applied to obtain the motion contrast. Further, for the motion contrast, another method may be used instead of the dispersion value for each pixel at the same position from the luminance image of the tomographic image of the m frame. For example, it is also possible to use a coefficient of variation normalized by the average value for each pixel of each frame.

In step S320, the signal processing unit 144 executes the first threshold processing of the motion contrast output by the signal processing unit 144. The value of the first threshold value is obtained from the luminance averaging image output by the signal processing unit 144 in step S290. Specifically, an area where only random noise is displayed on the noise floor is extracted from this luminance averaged image, the standard deviation σ is calculated, and the average luminance + 2σ of the noise floor is set as the first threshold value. The signal processing unit 144 sets the value of the motion contrast corresponding to the region where each brightness is equal to or less than the first threshold value to 0.

Noise can be reduced by removing the motion contrast caused by the change in brightness due to random noise by the first threshold processing of S320.

The smaller the value of the first threshold value, the higher the detection sensitivity of the motion contrast, but also the noise component. Also, the larger the value, the less noise, but the lower the sensitivity of motion contrast detection. In the present embodiment, the threshold value is set as the average brightness of the noise floor + 2σ, but the threshold value is not limited to this.

In step S330, the signal processing unit 144 increments the index i at position yi.

In step S340, the signal processing unit 144 determines whether i is larger than n. That is, it is determined whether or not the alignment is performed at all n y positions, the luminance image averaging calculation, the motion contrast calculation, and the threshold processing are performed.

In the case of no, the process returns to S220. In the case of yes, the flow proceeds to the next step S350.

At the end of S340, the three-dimensional data (three-dimensional data) of the motion contrast of each pixel of the B scan image (Z depth vs. X direction data) at all Y positions has been acquired. In step S350, the signal processing unit 144 performs display information generation processing from these three-dimensional image information.

The display information generation process of this embodiment will be described below. FIG. 6 shows the details of the process executed in step S350 as a flowchart.

In generating the display information, in step S351, the signal processing unit 144 acquires the three-dimensional data of the motion contrast previously determined.

In step S352, the signal processing unit 144 performs smoothing processing on the motion contrast three-dimensional data in order to remove noise while leaving the blood flow site information. The optimum smoothing process differs depending on the nature of the motion contrast, but the following can be considered, for example.

A smoothing method that outputs the maximum value of motion contrast from nx × ny × nz voxels in the vicinity of a pixel of interest.

Alternatively, a smoothing method that outputs the average value of the motion contrasts of nx × ny × nz voxels in the vicinity of the pixel of interest.

Alternatively, a smoothing method that outputs the median value of the motion contrast of nx × ny × nz voxels in the vicinity of the pixel of interest.

Alternatively, a smoothing method in which the motion contrast of nx × ny × nz voxels in the vicinity of the pixel of interest is weighted by a distance.

Alternatively, a smoothing method in which the motion contrast of nx × ny × nz voxels in the vicinity of the pixel of interest is weighted according to the difference between the weight due to the distance and the pixel value of the pixel of interest.

Alternatively, a smoothing method that outputs a value using weights according to the similarity between the motion contrast pattern of a small area around the pixel of interest and the motion contrast pattern of a small area around the peripheral pixels.

In addition, a method of smoothing while leaving other blood flow site information may be used.

After the above smoothing process, in step S353, the signal processing unit 144 obtains from the display control unit 149 the threshold value used when determining the pixel to be displayed in step S354 and the initial value of the range in the depth direction to be displayed. .. The initial value of the display range is usually about 1/4 in the depth direction, and the position includes the range of the surface layer of the retina. Here, the reason why the initial value of the display range is not the entire range in the depth direction is that the main blood vessels and the capillary network in the surface layer are to be displayed in an easy-to-see manner. That is, if the surface layer portion including the main blood vessels and the capillary network and the RPE layer having no blood vessels and having a large noise are displayed at the same time, the discrimination of the main blood vessels and the capillary network in the surface layer portion is hindered. The initial value of the display threshold value may be set by obtaining a representative value in advance for a healthy eye or the like. Alternatively, when performing repeated shooting, the previous shooting data may be used. Further, the display threshold value may be adjusted in a step described later.

Next, in step 354, display threshold processing for displaying pixels exceeding the smoothed three-dimensional data using the initial value of the display threshold is performed. FIG. 7 shows an example of conversion from motion contrast to display pixel values by this processing. FIG. 7A shows an example in which the pixel value below the display threshold value is set to zero, and the display pixel value proportional to the pixels from the pixel value above the threshold value to the maximum intensity is assigned. In FIG. 7B, the pixel value below the display threshold value is assigned. Is multiplied by zero, and a display value obtained by multiplying the pixel values beyond that by 1 is assigned. In any case, if the motion contrast is less than or equal to the display threshold value, the motion contrast is invalidated, and the area having the motion contrast with the displayed connection is displayed in a separated form.

Step 355 executed by the display control unit 149 shown in FIG. 6 is a step of displaying the motion contrast image subjected to the display threshold processing shown in FIG. 7.

Next, step 356 is a step of changing the display conditions. The display conditions that can be changed will be described later. When the display condition is changed, the process returns to step 354, the display condition is changed, and the updated image is displayed. If the display condition is not changed, the display information generation process is terminated.

Next, FIG. 8 shows a display example of an image generated by the display information generation process of the present embodiment. On the display screen, the intensity image 400 and the motion contrast image 401 displaying the OCT signal as an image are displayed side by side. In this display example, a slider 402 for operating the motion contrast display threshold value is provided beside the motion contrast image 401.

Further, a tomographic image 403 in a part of the intensity image 400 is displayed beside the intensity image 400. The tomographic image is displayed with xz cross section 403a and yz cross section 403b. Further, the line 404 indicating the display position of the tomographic image may be displayed on the intensity image 400 or the motion contrast image 401. The line 404 indicating the display position may be automatically determined from the image information under specific conditions, or may be specified by the examiner. For example, as an example of a specific condition, a tomographic image 403 centered on the macula may be specified. Alternatively, the examiner may specify an arbitrary position and length from the intensity image 400 or the motion contrast image 401, and display the tomographic image correspondingly.

Further, a slider 405 operated to indicate the display range in the depth direction of the intensity image 400 and the motion contrast image 401 from the tomographic image may be provided. In the slider 405, the slide portion 405a specifies the depth at which the intensity image 400 and the motion contrast image 401 are displayed on the bar 405b indicating the adjustable range in the depth direction, or the display range in the depth direction. Further, it may have a GUI operation unit 406 that operates the display position. The details of the GUI operation unit 406 will be described later. Further, the cursor 407 indicating the selected position may be displayed. In this case, it is preferable that the cursor 407 is displayed so as to simultaneously point to the corresponding positions of the intensity image 400 and the motion contrast image 401. The display is instructed by the display control unit 149. In the example of FIG. 8, although the example displayed as a two-dimensional plane image is shown, a three-dimensional image, an arbitrary tomographic image, or a combination thereof may be used.

Further, in the present embodiment, the image displayed on the two-dimensional plane shown in FIG. 8 is generated from the OCT intensity signal without using a fundus camera or a laser scanning ophthalmoscope. Such an image is called an Enface image. Here, a method for generating an Enface image will be briefly described.

The information obtained by A-scan is arranged from the front side in the optical axis direction, and the position where a strong signal can be obtained first from the position where there is no signal is the information on the fundus surface layer (retinal surface). Therefore, the fundus surface layer is two-dimensionally formed by first connecting the reflection intensities of the strong signals for the A scan signals at each measurement position (imaging position) on the fundus when the measurement light is scanned in two axes. It is possible to acquire an Enface image of the surface layer of the fundus expressed in. Further, by connecting the reaction intensities of the signals having a specific intensity from the signal, an Enface image at a specific layer or depth in the retina can be obtained.

Further, the tomographic image acquired by the B scan is constructed by continuously acquiring the information by A-scan in the uniaxial direction. Therefore, by finding the position where a part of the image signal of the Enface image and the image signal of the surface layer of the fundus of the acquired tomographic image collate, the A-scan is the tomographic image of which position on the Enface image. Can be detected accurately.

Next, the display operation will be described. The display operation is shown in FIG. FIG. 9 shows the intensity image 400 and the motion contrast image 401 of FIG. As described above, in the present embodiment, the acquisition of the OCT interference signal and the acquisition of the OCTA signal are performed in the same process, and the alignment of these signals on the same scanning line is completed when the display image is generated. ing. Therefore, the correspondence between the pixels of the intensity image 400 and the motion contrast image 401 is known in advance.

In the display operation of the present embodiment, when a part of one image (for example, the blood vessel portion of the intensity image) is selected, the corresponding area of the other image (corresponding blood vessel portion of the motion contrast image) is changed to the original image (intensity image). ) And display it. In the example of FIG. 9, when the blood vessel portion 450 of the intensity image 400 is selected, the corresponding blood vessels and the connecting blood vessels 450a in the motion contrast image 401 are superimposed and displayed on the corresponding positions of the intensity image 400. More specifically, blood vessels are first selected in one of the images. Here, the position information of each pixel or blood vessel is already associated with the other corresponding image by alignment. In this state, the position information of the blood vessel selected in one image is acquired. Subsequently, the corresponding position information in the other image is selected based on the acquired position information. Based on the selected location information, the blood vessel in the other image is designated as the corresponding blood vessel. Through the above procedure, the blood vessel of one image is superimposed and displayed on the other blood vessel image based on the position information of the corresponding blood vessel in each image.
It is assumed that the connected state of blood vessels (a thick blood vessel and a group of thin blood vessels branched from the thick blood vessel) is grasped in advance in each image by the region division processing described later. Therefore, the selection and superposition of blood vessels are carried out in consideration of this connection state (in a manner of associating blood vessel groups with each other). In this superimposed display, as will be described later, information (information about blood vessels) of one image of the intensity image and the motion contrast image selected by input from the cursor 407 or the like is displayed in the area where the information is displayed in the other image. It is executed so as to overlap. This superimposed display is executed by the display control unit 149, which is a display control means for displaying the superimposed image on the display unit 146, which is the display means.

As a specific selection method, a specific blood vessel may be selected as shown in FIG. 9, or a region may be specified. The designated area may be a rectangle or a circle at a certain distance from the center of the cursor 407. Further, when the measurement area of OCTA is narrower than the measurement area of OCT, the entire motion contrast image may be overlapped. Further, the display color of the blood vessel that is the source of superimposition in the other image (motion contrast image) may be changed so that it can be known which blood vessel is superposed on the original image (intensity image).

Although the example of superimposing the motion contrast image on the intensity image has been described, the reverse may be performed. When superimposing an intensity image on a motion contrast image, when a specific blood vessel portion of the motion contrast image 401 is selected, the corresponding blood vessels and the connecting blood vessels in the intensity image 400 are superimposed and displayed at the corresponding positions of the motion contrast image 401. Will be done. In this case, in order to improve visibility, it is preferable to represent the portion corresponding to the blood vessel portion that is superimposed and displayed by a line in which a thick line and a dotted line are overlapped. Alternatively, the display color of the overlapping blood vessels may be changed. Generally, the blood vessels that can be identified by the intensity image are relatively thick blood vessels, and it is easy to take a positional correspondence with the structural information of the intensity image. Therefore, by superimposing the intensity image on the motion contrast image in this way, a thick blood vessel can be used as a mark, and it becomes easy to compare with structural information other than the blood vessel. Further, by using the intensity image 400 as a base, it is possible to display not only blood vessels but also structural information such as macula or lesions.

As described above, by configuring the configuration in which one image information of the OCT intensity image and the OCTA motion contrast image can be superimposed and displayed on the other, it becomes easy to correspond the intensity image and the motion contrast image. Therefore, it is possible to provide an image that is intuitively easy to understand.

A cursor display will be described as an example of the display of the present embodiment. As shown in FIGS. 8 and 9, the cursor 407 may be displayed on both the intensity image 400 and the motion contrast image 401. The shape of the cursor 407 may be, for example, an arrow as shown in the figure, or may be a circle or a square. Further, both images may have different shapes. Alternatively, it may be a cross line that intersects in the X direction and the Y direction. The cursor points to the corresponding positions in both images at the same time. It is desirable that the cursor can be freely moved by operating a mouse (not shown) attached to the image forming apparatus. Further, the cutting position of the tomographic image may be changed as the cursor moves. By displaying the corresponding positions at the same time, it is possible to easily compare both images.

Next, as an example of changing the display conditions of the present embodiment, the change of the display range and the viewpoint will be described. The display magnification, display position, or viewpoint position of both images may be changed at the same time. For example, the GUI operation unit 406 of the display screen of FIG. 8 changes the display conditions. The enlargement / reduction button 406a enlarges (+) and reduces (-) both images at the same time. Further, when the image is enlarged, the translation movement button 406b is used to simultaneously perform the translation movement of the display range of both images. Similarly, the rotation button 406c simultaneously rotates and moves the display viewpoints of both images. Rotational movement can be applied when both images are displayed three-dimensionally. That is, with these configurations, it is possible to change at least one of the magnification, the display position, and the viewpoint position for both the intensity image and the motion contrast image at the same time.

Although the GUI operation unit 406 has been described here with an example of three types of buttons, the number, shape, and type of buttons may be different. Alternatively, it may be another method. For example, it may be performed by operating the mouse attached to the image forming apparatus. Place the mouse cursor on either of the two images and operate the wheel attached to the mouse to zoom in / out, or move the mouse while holding down the mouse button to move (translate or rotate) the image. You may do it. In the example other than the mouse, if it is a touch panel, the image displayed on the panel may be operated with a finger. In any case, the display range and viewpoint of the intensity image and the motion contrast image displayed side by side may be changed at the same time. According to this configuration, since the image is changed at the same time for both the intensity image and the motion contrast image, it is possible to easily understand which position is being viewed even if the image display method is changed.

The display threshold value of the motion contrast pixel value will be described. In FIG. 8, a slider 402 for adjusting the threshold value of the pixel to be displayed is provided. The initial position of the display threshold value may be a representative value set in advance. When the examiner drags this slider with the mouse, the signal processing unit 144 determines that the display condition has been changed (YES) in step 356 of FIG. The signal processing unit 144 changes the display threshold value, returns the processing to step S354, and updates the three-dimensional motion contrast image to be displayed. At this time, if the threshold value is set so that it can be changed by a relative value with respect to the initial value, an equivalent effect can be obtained for different data of different objects such as different eyes and parts to be examined.

Next, a modified example of the method of displaying blood vessel information will be described. Here, a step of acquiring blood vessel information is performed for a two-dimensional motion contrast image. The step of acquiring blood vessel information may be performed at the same timing as step S352, which is the smoothing processing step described with reference to FIG. 6, or may be performed as a subsequent step. Specifically, the motion contrast data is subjected to a step of adding a smoothing process and a step of dividing a region corresponding to a blood vessel. By performing these treatments, a smooth blood vessel region can be extracted.

Next, the area division process will be described. An example of the flow of the area division processing is shown in FIG.
First, in step S601, which is a blood vessel candidate extraction step, the signal processing unit 144 extracts a pixel value having a predetermined threshold value or more as a pixel of the blood vessel candidate with respect to the pixel value representing the volume data corresponding to the motion contrast of the pixel of interest. Run.

Next, in step S602, which is a blood vessel connection estimation step, the signal processing unit 144 executes a process of estimating the presence or absence of a blood vessel connection relationship, which is a connection relationship between these blood vessels, for each of the extracted blood vessel candidate pixels. To do. In the blood vessel connection estimation step, if the adjacent pixels have a pixel value equal to or higher than a predetermined threshold value for each pixel of the blood vessel candidate, or if the adjacent pixel is also a blood vessel candidate in advance, these pixels are connected. Judged as a blood vessel candidate. When a specific pixel is selected as a blood vessel candidate to be connected in this way, the evaluation range is further expanded to adjacent pixels. When the adjacent pixel is smaller than the threshold value (or is not a blood vessel candidate), the adjacent pixel is excluded from the blood vessel candidate, and other adjacent pixels are evaluated. The operation of estimating blood vessel connection may be performed on all or a specified range of blood vessel candidates.

After the evaluation of all the adjacent pixels or all the pixels of the blood vessel candidate is completed, the signal processing unit 144 performs the process of step S603, which is the blood vessel recognition processing step, based on the size of the connected blood vessel candidate. In the blood vessel recognition process, blood vessel candidates having a predetermined number or more of pixel connections are identified as the same relatively thick or major blood vessel.

Finally, in step S604, which is the blood vessel connection data generation step, the signal processing unit 144 executes a process of generating the first blood vessel connection data. The first blood vessel connection relationship data has at least three-dimensional distribution information of blood vessels to be connected and blood vessel connection information. For example, volume data having information on the presence or absence of blood vessels for each pixel and distinguishing each blood vessel may be used as the first blood vessel connection relationship data.

The threshold value for extracting the blood vessel candidates described above may be a predetermined value or may be changed. For example, the examiner may adjust the threshold value by measuring the OCTA signal for the eye to be inspected and observing how the region is divided as a blood vessel. Further, a typical value may be used as the initial value of the threshold value. In the region division step, when the motion contrast data is binarized in the blood vessel region and other regions, the blood vessel candidates may be extracted using the result of the binarization treatment. Similarly, the threshold value of the predetermined number of connections to be determined to be a blood vessel may be a predetermined value or may be changed. For example, when a plurality of pixels are connected, the blood vessel may be excluded, and when a single pixel is used, the candidate may be excluded. By performing such processing, an isolated region can be removed from the motion contrast image, and connected blood vessels can be extracted. Examples of isolated regions include local and random variations in reflection intensity such as speckle noise.

Further, although the intensity image and the motion contrast image have the same size as described above, as described above, the motion contrast image can be displayed in the same manner even in a narrow range. An example in which the sizes of the intensity image and the motion contrast image are different is shown in FIG. FIG. 14A shows an example in which the size of the motion contrast image is changed so as to be aligned at the same scale. FIG. 14B shows an example in which the sizes of the images are aligned and the scale is changed. A frame line 416 indicating the other display area may be provided on the side having a wider angle of view. Further, the enlargement / reduction button 406a or the like shown in FIG. 8 may be used to enlarge / reduce the intensity image and the motion contrast image, move the viewpoint, or superimpose the other state on one of them.

Next, an example of comparing the blood vessels obtained by the intensity image and the blood vessels obtained by the motion contrast image will be described. The operation performed in this case will be described with reference to the flowchart shown in FIG.

First, in step S651, which is the first blood vessel connection estimation step, the signal processing unit 144 executes a process of estimating the first blood vessel connection relationship from the two-dimensional motion contrast image. In this step S651, the first blood vessel connection relationship can be estimated by the same method as the method for estimating the blood vessel connection relationship performed in step S602 described above.

Next, in step S652, which is the second blood vessel connection estimation step, the signal processing unit 144 executes a process of estimating the second blood vessel connection relationship from the data of the three-dimensional tomographic image. The second estimation of the vascular connection relationship may be performed for a specific layer. For example, if a two-dimensional intensity image (for example, intensity image 400) is generated from the three-dimensional tomographic image data at a depth corresponding to the retinal layer from the three-dimensional tomographic image data, and the second vascular connection relationship is estimated. Good. In the intensity image, the intensity of the signal obtained from the blood vessel becomes relatively weaker than that of the surroundings due to absorption by blood or the like. In this case, by selecting the retinal layer, it becomes easy to extract the pixels corresponding to the retinal arteriovenous blood vessels as blood vessels.

The region corresponding to the blood vessel may be divided by the same method as the motion contrast data. Further, the threshold value of the pixel when extracting the pixel as a blood vessel may be set to a value corresponding to the intensity image. The blood vessels obtained by estimating the second blood vessel connection relationship are inferior in the resolution of the blood vessels that can be extracted than the blood vessel connection relationship (first blood vessel connection relationship) obtained from the motion contrast data, but the information is derived from the blood vessel structure. is there.

Next, in step S653, which is a step of comparing the blood vessel connection state, the signal processing unit 144 executes a process of comparing the connection state of the blood vessels of the second blood vessel connection relationship and the first blood vessel connection relationship. To compare the state of blood vessel connection, the presence or absence of blood vessels may be compared with respect to the first and second blood vessel connection relationships at the corresponding positions.

Next, in step S654, which is a comparison result display step (display step), the control unit 143 displays at least a part of the result of comparing the connected state of the blood vessels with the display unit 146 as an intensity image and / or a motion contrast image. It is displayed overlaid on. For example, on the other hand, the part that is not presumed to be a blood vessel is displayed. Conversely, both may display presumed blood vessels. The comparison may be the entire image or only a specific area.

When the blood vessel image of the motion contrast image is superimposed on the intensity image, it corresponds to superimposing and displaying the information of the capillaries that cannot be obtained on the intensity image on the intensity image. On the contrary, when the blood vessel image of the intensity image is superimposed on the motion contrast image, it corresponds to displaying the region where the blood flow is stagnant. In the region where the blood flow is stagnant, the pixel value of the motion contrast is smaller than that in the region where the blood flow is present. When the pixel value is equal to or less than the threshold value, it is determined that there is no blood vessel. Examples of the region where blood flow is stagnant include a region where stagnation is caused by a hemangioma, a region where an inner diameter is narrow in the middle route, and a region where the blood flow is blocked.

By superimposing the information on the connection state of one blood vessel on the other as described above, it is possible to make it easier to compare both images.

Further, in the display of blood vessels, a specific blood vessel may be selected and emphasized. In this case, after step S654, which is a comparison result display step, a step of selecting pixels in the vicinity of a specific blood vessel is further provided. Specifically, for example, an arbitrary pixel is selected from a display image with a mouse or the like. In response to this selection operation, the signal processing unit 144 selects the blood vessel closest to the selected pixel. Here, the selected blood vessel has the above-mentioned information on the first and second blood vessel connection relationships. The result of comparing the intensity image and the blood vessel obtained by the motion contrast image with respect to the selected blood vessel is displayed. As a method of displaying the comparison result, for example, an image of a translucent blood vessel having a different display color may be superimposed and displayed. An example in which a specific blood vessel is selected and a motion contrast blood vessel image is superimposed on the intensity image is shown in FIG.

In addition, as a selection mode of blood vessels at the time of superimposition, it is also conceivable to have a mode having a plurality of modes. As a mode in this case, for example, for a blood vessel designated by the cursor 407, a mode in which the papilla to the end of the blood vessel is superimposed on the intensity image 400 can be considered. Further, for the blood vessel designated by the cursor 407, a mode in which the intensity image is superimposed from the designated position to the end portion may be included. Further, the region may be designated by the cursor 407, and a mode in which the blood vessels included in the region are superimposed on the intensity image 400 may be included. Further, the blood vessel diameter estimation described later may be performed, and a mode in which blood vessels estimated to be smaller than a predetermined blood vessel diameter at the time of superposition are not superposed may be included. Further, these styles may be combined.

With such a configuration, a comparison result focusing on a specific blood vessel can be obtained. For example, one of the retinal arteriovenous blood vessels can be selected, and the state of the blood vessels connected to the blood vessels can be extracted and displayed. Alternatively, instead of selecting a specific blood vessel, it may be color-coded in advance for each run from the optic disc. For example, color coding may be performed based on the direction in which the optic nerve head spreads from the center.

Further, a step of acquiring blood vessel information may be further provided for the blood vessels extracted from the motion contrast image. For example, at least the blood vessel diameter may be detected in the blood vessel region. For the blood vessel diameter, the axial direction and the radial direction when approximated to a cylinder are estimated with respect to the extracted blood vessel, and the blood vessel diameter is obtained. The blood vessel diameter data may be further added to the volume data of, for example, the first blood vessel connection-related data (three-dimensional distribution information of blood vessels and blood vessel connection information). With such a configuration, information on the morphology of blood vessels can be obtained from the motion contrast image.

Furthermore, blood vessels may be classified based on the calculated blood vessel diameter. First, blood vessels smaller than a predetermined diameter are extracted. The predetermined diameter may be, for example, a diameter corresponding to a capillary vessel. Further, in a predetermined region of the motion contrast image, the abundance ratio of the pixels extracted as the blood vessel (for example, capillaries) and the other pixels is calculated. Furthermore, the abundance ratio is displayed as a two-dimensional map. With such a mapping configuration, it is possible to quantitatively visualize the density of blood vessels. The predetermined region may be divided based on, for example, the macula (or the optic disc). FIG. 16 shows an example of mapping. In the example of FIG. 16, the map 408 divided into a plurality of regions centered on the macula is superimposed on the two-dimensional motion contrast image. By dividing into a plurality of regions, it becomes easy to qualitatively evaluate the density of capillaries. Further, the mapped information may be superimposed and displayed not only on the motion contrast image but also on the two-dimensional intensity image, or may be displayed on another screen.

With such a configuration, it is possible to visualize the information on the morphology of the blood vessel from the motion contrast image in association with the intensity image.

(Second embodiment)
Next, as a second embodiment of the present invention, changing the display range in the depth direction will be described.
By operating the slider 405 of FIG. 8, the display range in the depth direction in which the intensity image and the motion contrast image are displayed is changed. The slider 405 includes a slide portion 405a indicating the depth range to be displayed and a bar 405b indicating the adjustable range in the depth direction of the retina as described above. The slide portion 405a can change the width and position in the depth direction to be displayed as an image, and is used to select the display range in the depth direction. The intensity image 400 and the motion contrast image 401 to be displayed are updated at the same time according to the change of the display range by the operation of the slide unit 405a. According to this embodiment, since the depth direction is changed at the same time, it is possible to easily understand which position is being viewed even if the range of the display depth of the image is changed.

Although an example using a slider has been described here as a method of changing the depth range, other methods may be used. For example, the display depth may be substantially changed by displaying each layer structure of the eye to be inspected. The present embodiment includes a step (corresponding to step 300 in FIG. 5) of detecting the layer structure of the tomographic image of the eye to be inspected from the data of the three-dimensional tomographic image. The process of setting the display depth can be selected based on the layer structure information of the eye to be inspected instead of the slider. The layer structure of the human eye is known, and examples thereof include the following six layers. The breakdown of the 6 layers is (1) nerve fiber layer (NFL), (2) ganglion cell layer (GCL) + inner plexiform layer (IPL) combined layer, and (3) inner granular layer (INL) + outer plexiform. Layer (OPL) combined, (4) Outer plexiform layer (ONL) + Outer plexiform membrane (ELM) combined layer, (5) Ellipsoid Zone (EZ) + Interdigitation Zone (IZ) + Retinal pigment epithelium (RPE) And (6) Choroid.

The segmentation of each of these layers in the retina will be described. The map generation unit 148 creates an image by applying a median filter and a Sobel filter to the tomographic image to be processed extracted from the intensity image (hereinafter, also referred to as a median image and a Sobel image, respectively). Next, a profile is created for each A scan from the created median image and Sobel image. The median image has a brightness value profile, and the Sobel image has a brightness gradient profile. Then, the peak in the profile created from the Sobel image is detected. By referring to the profile of the median image corresponding to before and after the detected peak and between the peaks, the boundary of each region of the retinal layer is extracted. Pixels and layers are associated with each other based on the extracted boundary information.

FIG. 10 shows an example of a layer selection method to be displayed after segmentation. In this example, a selection button 410 corresponding to each layer in the tomographic image is provided. By selecting the selection button 410, the display range of the image or the layer to be displayed is selectively indicated. The selection button 410 may be layer-by-layer as illustrated in the figure, or may be a combination of a plurality of layers. For example, it may be a selection button divided into an upper retina layer and a lower retina layer. Alternatively, a button may be provided so that selection can be made in units of boundaries. Further, these may be combined to provide a radio button 411 for switching between layer-based selection and boundary-based selection as shown in FIG. Further, in order to make it easier to understand the corresponding correspondence, the tomographic image 412 may be displayed side by side with these selection buttons 410 and the like.

The intensity image and the motion contrast image are updated according to the selection process of the layer to be displayed. A slider 413 for adjusting the display threshold value of the motion contrast image may be provided for each layer, and the display threshold value may be readjusted when the image is updated. With the above configuration, the display layer is changed at the same time for the intensity image and the motion contrast image, so that it is easy to understand which layer is being viewed even if the display layer is changed. In addition, by changing the depth range based on the layer structure, it is possible to display information focusing on a specific layer.

When the image is displayed based on the layer structure of the eye to be inspected, the intensity image and the motion contrast image may be two-dimensional images. Here, the process of generating a two-dimensional image will be described. In the two-dimensional image generation step, the three-dimensional tomographic image data is projected and integrated in the depth direction within the corresponding range based on the specified depth or the selected layer to generate a two-dimensional intensity image. An example of the projection method will be described with reference to FIG.

FIG. 11A represents a tomographic image, and the area between Z1 and Z2 in the depth direction corresponds to the selected range, respectively. FIG. 11B shows the three-dimensional volume data 500 and the depth display bar 501. The volume area 502 designated as between the depths Z1 and Z2 in the depth display bar 501 is the selected range. A two-dimensional intensity image can be generated by projecting and integrating voxels (volume data of a three-dimensional tomographic image) in the volume region 502 in the depth direction. Similarly, for 3D motion contrast, each voxel in the same area is projected and integrated in the depth direction, or projected or integrated to generate a 2D motion contrast image.

In order to generate a two-dimensional image, it is possible not only to integrate the pixel values of the corresponding pixels, but also to extract and project a representative value such as the maximum value. The generated two-dimensional image is displayed as an intensity image and a motion contrast image. Further, these display processes may not be performed simultaneously, but may be sequentially performed so as to change the display threshold value of the motion contrast image after confirming the display state by forming a two-dimensional image. By creating a two-dimensional image and displaying it in parallel, it is possible to easily grasp the entire information focusing on a specific layer at once.

In the example shown in FIG. 11, the voxels are projected and integrated in a specified fixed depth range, but this may be performed in a depth range corresponding to the layer structure. That is, a depth range corresponding to the layer selected for each location may be selected, and voxels may be projected / integrated in this range. By doing so, it is possible to generate a two-dimensional image that more reflects the layered structure.

Further, when displaying the above-mentioned two-dimensional image, a tomographic image may be displayed in parallel as in FIG. In this case, the position of the cut surface of the tomographic image is specified by either the two-dimensional intensity image or the two-dimensional motion contrast image. Further, as the tomographic image to be displayed, a tomographic image at a designated position or a corresponding position is selected (or generated) from the 3D tomographic image data and / or the 3D motion contrast. The original image that specifies the position of the tomographic image to be displayed and the tomographic image to be displayed can be appropriately combined. Further, the tomographic image of the motion contrast image may be superimposed on the tomographic image of the intensity image. Further, as shown in FIG. 8, a display line 404 showing a cross section of the tomographic image may be provided on both images. By designating and displaying the position of the tomographic image from either the intensity image or the motion contrast image, the tomographic image at the desired position can be easily displayed.

Further, the above-mentioned tomographic image may further display information on the layer structure. In this case, the layer structure is superimposed on the tomographic image and displayed based on the information obtained by detecting the layer structure of the tomographic image of the eye to be inspected from the data of the three-dimensional tomographic image. The display method may be such that the layer structure can be identified. For example, a line may be displayed for each layer boundary, or a line may be colored for each layer. An example of displaying the tomographic image separately for each layer is shown in FIG. In the example of FIG. 12, the boundary line 422 (dotted line) based on the detected layer is superimposed on the tomographic image. The boundary line 422a of the selected layer is a thick dotted line, and the selected layer area 423 is gray. By superimposing the detected layer structure information, the display can be made easier to understand.

(Third embodiment)
Next, a third embodiment of the present invention will be described. In the present embodiment, the fundus image is acquired separately from the motion contrast image acquisition by OCT. Examples of the fundus image acquisition means include a fundus camera, a scanning laser ophthalmoscope (SLO), and a blood vessel image by fluorescence contrast. These configurations constitute a subject image acquisition means for acquiring a subject image in a first predetermined range of the eye 118 to be inspected, similar to the configuration for generating an intensity image composed of OCT in the first embodiment. Similar to the intensity images obtained by OCT, the third vascular connection relationship is calculated from the fundus images obtained by these. The calculated blood vessel connection relationship and the first blood vessel connection relationship based on the motion contrast image are aligned based on the feature amount of the blood vessel. A known alignment algorithm can be used for alignment.

After aligning, both images are displayed side by side. A display example is shown in FIG. In the example of FIG. 17, the motion contrast image (OCTA image) 401 and the fundus image 420 are displayed side by side. The radio switch unit 421 switches the fundus image to be displayed. For example, by switching operation, for example, a two-dimensional intensity image by integrating OCT, a fundus image by a fundus camera, or a fundus image by SLO is switched. Similar to the first embodiment, both images are provided with cursors 407 at corresponding positions.

Further, as in the first embodiment (display example shown in FIG. 8), both images may be enlarged or moved at the same time. With such a configuration, it is possible to easily compare an image obtained by another observation device with a motion contrast image. For example, by displaying it side by side with an image obtained from a color fundus camera, it is possible to easily compare a blood vessel photograph viewed in color with a motion contrast image.

(Fourth Embodiment)
Next, a fourth embodiment of the present invention will be described. In the present embodiment, the blood vessel connection relationship based on the measured motion contrast image and the blood vessel connection relationship based on other motion contrast images are displayed side by side. The other motion contrast image may be a previously captured image. A display example of the fourth embodiment is shown in FIG. In the example of FIG. 18, the motion contrast image 401 on the left side is compared with the motion contrast image 430 on the right side taken in the past. Alignment is performed between these images based on the connection relationship of each blood vessel as in the third embodiment. In addition, both images may be enlarged or moved at the same time. In this case, the operation executed in the step of acquiring the subject image in the above-described embodiment is a step of acquiring another motion contrast image at a time different from the motion contrast image in the signal acquisition step at the time of measurement. Is replaced by.

In the example of FIG. 18A, an image obtained when the signal of the blood vessel in the region 431 surrounded by the broken line of the motion contrast image 401 disappears is shown as a schematic diagram. The connected states of both displayed images are compared, and at least a part of the comparison result is displayed overlaid on one of the images. FIG. 18B shows an example in which the comparison result (blood vessel) of the region 431 is superimposed and displayed by a dotted line. With such a configuration, it is possible to easily grasp the change over time in the connection relationship of blood vessels.

The comparison process executed here may be a comparison with a pair of images or a comparison between a plurality of images. When comparing a plurality of images, the display colors may be separated. Alternatively, the comparison target may be sequentially switched.

As described above, according to each of the above-described embodiments, the motion contrast image obtained by OCTA and the intensity image obtained by OCT can be more intuitively understood by displaying the information obtained from one image on the other image. It is possible to provide an image that is easy to use. That is, it is possible to facilitate the correspondence between the motion contrast image obtained by OCTA and the intensity image obtained by OCT. Further, when displaying both images side by side, it is possible to provide an image that is easier to compare by displaying a cursor indicating the corresponding pixel.

Furthermore, by providing a process that can change the magnification, display position, and viewpoint position of both images at the same time, it is possible to provide an image that makes it easy to understand which position is being viewed even if the image display method is changed. ..

Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to the specific embodiments, and various modifications are made within the scope of the gist of the present invention described in the claims.・ Can be changed. For example, the positional relationship of image display and the shape of GUI can be changed. Further, the display may be stereoscopically displayed by a 3D display.

(Other embodiments)
The present invention is not limited to the above-described embodiment, and can be modified or modified in various ways within a range that does not deviate from the gist of the present invention. For example, although the case where the subject is an eye is described as an embodiment, the present invention can be applied to a subject such as skin or an organ other than the eye. In this case, the present invention has an aspect as an image forming method or device other than an ophthalmic device, which forms an image based on data obtained from a medical device such as an endoscope. Further, it is desirable that the OCT device described above is grasped as an inspection device exemplified by an ophthalmic device, and the eye to be inspected is grasped as an aspect of an object to be inspected.

Further, the present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device can program the program. It can also be realized by the process of reading and executing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100 Optical interference fault acquisition unit 143 Control unit 144 Signal processing unit 145 Signal acquisition control unit 146 Display unit 147 Image generation unit 148 Map generation unit 149 Display control unit

Claims (19)

Using each of the data of a plurality of 3-dimensional motion contrast data obtained by photographing a subject's eye at different times, information acquisition means for acquiring measurement information for at least one blood vessel,
A classification means for classifying at least one blood vessel using the acquired measurement information, and
It is a motion contrast Enface image generated by using at least a part range of each data of the plurality of three-dimensional motion contrast data in the depth direction of the eye to be inspected, and corresponds to the plurality of three-dimensional motion contrast data. A display control means for displaying a plurality of motion contrast Enface images side by side on a display means, and displaying display information indicating positions corresponding to each other in the plurality of motion contrast Enface images on the display means.
Information processing device equipped with. The classification means classifies blood vessels having a diameter smaller than a predetermined diameter as capillaries by using the acquired measurement information.
The information processing apparatus according to claim 1, wherein the information acquisition means further acquires information indicating the density of the classified capillaries by using the respective data of the plurality of three-dimensional motion contrast data. An image acquisition means for acquiring a frontal image of the fundus of the eye to be inspected, and
A display control means for displaying information indicating the density of the capillaries on the acquired front image and displaying it on the display means.
The information processing apparatus according to claim 2, further comprising. The display control means causes the display means to identifiable information indicating the density of the capillaries in a plurality of regions centered on a predetermined region of the eye to be inspected in the acquired front image. The information processing apparatus according to claim 3. The information acquisition means is any one of claims 1 to 4, wherein the information acquisition means acquires information on the distribution of the at least one blood vessel and information on the blood vessel diameter of the at least one blood vessel at each site of the fundus of the eye to be inspected. The information processing device described in. A claim further comprising a comparison means for comparing measurement information on blood vessels based on each of the plurality of three-dimensional motion contrast data and information on distribution of blood vessels extracted using the three-dimensional tomographic information of the eye to be inspected. Item 2. The information processing apparatus according to any one of Items 1 to 5. A smoothing means for smoothing each of the plurality of three-dimensional motion contrast data,
A pixel extraction means for extracting pixels having a pixel value equal to or higher than a predetermined threshold value in the smoothed three-dimensional motion contrast data as blood vessel candidate pixels.
An estimation means for estimating the connection state of the extracted pixels of the blood vessel candidate and estimating the pixels in which the number of connected pixels equal to or larger than a predetermined threshold value is used as the pixel indicating the blood vessel.
The information processing apparatus according to any one of claims 1 to 6, further comprising. The information acquisition means according to any one of claims 1 to 7 , wherein the diameter of at least one blood vessel extracted by using the respective data of the plurality of three-dimensional motion contrast data is acquired as the measurement information. Information processing equipment. Claims a range of at least part in response to an instruction from the examiner is configured to be changed, a plurality of motion contrast Enface image the displayed depending on the change in the scope of at least a part is updated in conjunction Item 2. The information processing apparatus according to any one of Items 1 to 8. In response to an instruction from the examiner Table示閾value is configured to be changed, any of claims 1 to 9 multiple motion contrast Enface image the display in accordance with a change in the display threshold is updated in conjunction The information processing apparatus according to item 1. An information acquisition means for acquiring measurement information on at least one blood vessel using the three-dimensional motion contrast data of the eye to be inspected, and
A display control means for causing the display means to display the acquired measurement information and a motion contrast Enface image generated by using at least a part of the range of the three-dimensional motion contrast data in the depth direction of the eye to be inspected. With
An information processing device that is configured so that the display threshold value of the motion contrast Enface image can be changed according to an instruction from an examiner, and the displayed motion contrast Enface image is updated according to the change of the display threshold value. The display control means superimposes the information indicating the density of blood vessels as the acquired measurement information on the motion contrast Enface image and displays it on the display means, and displays the information indicating the density of the blood vessels on the front image of the eye to be inspected. The information processing device according to claim 11, which is displayed on the display means in an overlapping manner. Interference signal sets for multiple frames acquired with the intention of having the same cross section in the second predetermined range included in the first predetermined range of the subject are acquired for different cross sections in order to form a three-dimensional tomographic image. Signal acquisition means and
A motion contrast image generation means for generating a motion contrast image using pixel data corresponding to each frame in an interference signal set for a plurality of frames acquired with the intention of the same cross section.
Information about a selected part of the subject image and one of the motion contrast images in the first predetermined range is superimposed on the area corresponding to the selected part of the other image. Display control means to be displayed on the display means and
Information processing device equipped with. The display control means is a part of the area selected in the one image in response to an instruction from the examiner, and the information about the selected part of the area is selected in the other image. The information processing apparatus according to claim 13 , wherein the display means displays the image on the area corresponding to a part of the area. The display control means is a part of the region selected in the one image in response to an instruction from the examiner, and the information about the blood vessel of the subject obtained from the selected part of the image is described. The information processing device according to claim 13 or 14 , which is displayed on the display means. Using each of the data of a plurality of 3-dimensional motion contrast data obtained by photographing a subject's eye at different times, a step of acquiring the measurement information for at least one blood vessel,
The step of classifying the at least one blood vessel using the acquired measurement information, and
It is a motion contrast Enface image generated by using at least a part range of each data of the plurality of three-dimensional motion contrast data in the depth direction of the eye to be inspected, and corresponds to the plurality of three-dimensional motion contrast data. A step of displaying a plurality of motion contrast Enface images side by side on a display means and displaying display information indicating positions corresponding to each other in the plurality of motion contrast Enface images on the display means.
Information processing methods including. The process of acquiring measurement information on at least one blood vessel using the 3D motion contrast data of the eye to be inspected, and
A step of displaying the acquired measurement information and a motion contrast Enface image generated by using at least a part of the range of the three-dimensional motion contrast data in the depth direction of the eye to be inspected on a display means. ,
An information processing method in which the display threshold value of the motion contrast Enface image can be changed according to an instruction from an examiner, and the displayed motion contrast Enface image is updated according to the change of the display threshold value. Interference signal sets for multiple frames that are intentionally acquired in the second predetermined range included in the first predetermined range of the subject with the same cross section are acquired for different cross sections in order to form a three-dimensional tomographic image. And the process to do
A step of generating a motion contrast image using the corresponding pixel data between each frame in the interference signal set for a plurality of frames acquired with the intention of the same cross section.
Information about a selected part of the subject image and one of the motion contrast images in the first predetermined range is superimposed on the area corresponding to the selected part of the other image. The process of displaying on the display means and
Information processing methods including. A program for causing a computer to execute each step of the information processing method according to any one of claims 16 to 18.

Images