JP7140512B2 - Image processing device, image processing method and program - Google Patents

Description

The disclosure of the present specification relates to an image processing device, an image processing method, and a program.

Patent Literature 1 discloses a method for determining whether a given blood vessel is an artery or a vein from a tomographic image.

Japanese Patent Application No. 2015-207572

However, no technology has been known for determining whether a given blood vessel is an artery or a vein from a motion contrast image obtained by OCT Angiography (OCTA).

The disclosure of the present specification has been made in view of the above problem, and one of the objects thereof is to support determination of whether a predetermined blood vessel is an artery or a vein based on a motion contrast image.

It should be noted that it is not limited to the above object, but it is a function and effect derived from each configuration shown in the mode for carrying out the invention described later, and it is also disclosed in the present specification that there is a function and effect that cannot be obtained by the conventional technology. It can be positioned as one of the other purposes.

The image processing apparatus disclosed in the present specification includes acquisition means for acquiring a motion contrast image of the fundus, and a representative motion contrast value of an evaluation region within a predetermined distance from a region corresponding to a blood vessel in the motion contrast image. and determining means for determining whether the blood vessel is an artery or a vein based on the representative value is an average value of motion contrast values in the evaluation area .

It is possible to support determination as to whether a given blood vessel is an artery or a vein based on the motion contrast image.

It is a figure which shows an example of an apparatus configuration. FIG. 4 is a diagram showing an example of scanning during OCTA imaging; It is a figure for demonstrating an example of a motion contrast image. FIG. 4 is a diagram showing an example of a procedure for obtaining a motion contrast image; FIG. 4 is a diagram showing an example of a procedure for superimposing motion contrast images; It is a figure for demonstrating an example of the determination method of an artery and a vein. It is a figure which shows an example of the determination procedure of an artery and a vein. It is a figure which shows an example of a FAZ detection procedure. FIG. 4 is a diagram for explaining an example of blood vessel classification; It is a figure for demonstrating an example of the determination method of an artery and a vein. It is a figure which shows an example of the determination procedure of an artery and a vein.

An apparatus configuration for acquiring an OCTA image will be described. Next, the overall method of acquiring the MC image from the OCT information will be described, and the detailed flow will be described sequentially.

[Hardware configuration]
FIG. 1(a) is a diagram showing an example of an apparatus configuration. A light source 001 is an SLD light source, and a coupler 002 splits light into measurement light and reference light at a desired splitting ratio. The measurement light passes through the coupler 002 and is emitted from the collimator 021 to the sample optical system 102 . The sample optical system 102 includes a focus lens 022, an angle-variable X galvanometric mirror 023 and a Y galvanometric mirror 024, and lenses 025 and 026 forming an objective lens system. A beam spot is formed by the measuring light on the fundus of the eye. Here, the beam spot guided onto the fundus is two-dimensionally scanned on the fundus by driving the X galvanometric mirror 023 and the Y galvanometric mirror 024 . The measurement light reflected and scattered by the fundus of the subject's eye 027 is guided to the coupler 002 after passing through the sample optical system 102 . The reference light is guided to the reference optical system 103, becomes collimated light by the collimator lens 031, passes through the ND filter 032, and is attenuated to a predetermined light amount. After that, the reference light is reflected by a mirror 033 that can move in the optical axis direction and correct the optical path length difference with the sample optical system 102 while maintaining the collimated state, and is returned to the same optical path. The folded reference light is guided to the coupler 002 after passing through the ND filter 032 and the collimator lens 031 .

The measurement light and reference light that have returned to the coupler 002 are combined by the coupler 002 and guided to the detection system (or spectroscope) 104 . The combined light is emitted by a collimator 042, dispersed by a diffraction grating 043, received by a line sensor 045 via a lens 044, and output as an interference signal. The line sensor 045 is arranged so that each pixel receives light corresponding to the wavelength component of the light split by the diffraction grating 043 .

FIG. 1(b) is a diagram showing an example of an apparatus configuration including an image processing apparatus. The OCT optical system includes focus drive means 061 for moving the focus lens 022, galvanometric mirror drive means 062 for driving the X galvanometric mirror 023 and Y galvanometric mirror 024, and the mirror 033 in the optical axis direction. A mirror driving means 063 for moving is provided for each. Each drive means, light source 001, line sensor 045, sampling section 051, memory 052, signal processing means 053, operation input means 056, monitor 055, etc. are connected to control means 054 to control the movement of the entire apparatus. Here, the image processing apparatus includes signal processing means 053 and control means 054 . At least one of the sampling section 051, the memory 052, the monitor 055 and the operation input means 056 may be included in the image processing apparatus. The signal processing means 053 and the control means 054 are realized by executing a program stored in a memory by a processor such as a CPU provided in the image processing apparatus. A processor such as a CPU may function as the sampling unit 051 by executing a program stored in a memory.

An output signal from the line sensor 045 is output as an interference signal by the sampling section 051 for any galvanometric mirror driving position driven by the galvanometric mirror driving means 062 . Subsequently, the galvanometric mirror driving position is offset by the galvanometric mirror driving means 062, and an interference signal at that position is output. After that, interference signals are generated one after another by repeating this process. The interference signal generated by the sampling section 051 is stored in the memory 052 together with the galvanometric mirror drive position. The interference signal stored in the memory 052 is frequency-analyzed by the signal processing means 053 , becomes a cross-sectional image of the fundus of the eye 027 to be examined, and is displayed on the monitor 055 . It is also possible to generate a three-dimensional fundus volume image and display it on the monitor 055 based on the galvanometric mirror drive position information.

The control means 054 acquires background data at arbitrary timing during photographing. The background data refers to a signal in a state where no measurement light is incident on the subject, that is, a signal of only the reference light. Background data is acquired by performing signal acquisition in a state in which the position of the measurement light is adjusted so that it does not return.

[Scan example]
Next, an example of the scan pattern of this embodiment will be described with reference to FIG. Note that the numerical values exemplified below are examples and can be changed to other numerical values. FIG. 2(a) is a diagram showing an arbitrary scan, and FIG. 2(b) is a diagram showing a scan pattern reflecting numerical values specifically executed in this embodiment. Since OCTA measures temporal changes in OCT interference signals due to blood flow, multiple measurements are required at the same location. In this embodiment, the OCT apparatus repeats B-scanning at the same location m times and performs scanning moving to n y-positions. A specific scan pattern is shown in FIG. B scan is repeated m times for n y positions from y1 to yn on the fundus plane. When m is large, the number of times of measurement at the same place increases, so blood flow detection accuracy improves. On the other hand, the scanning time becomes longer, and there arises a problem that motion artifacts are generated in the image due to eye movement (fixational eye movement) during scanning and a problem that the burden on the subject increases. In the present embodiment, m=4 (FIG. 2(b)) is set in consideration of the balance between the two.

Note that the repetition number m may be changed according to the A-scan speed of the OCT apparatus and the motion analysis of the fundus oculi surface image of the subject 027 . In FIG. 2, p indicates the sampling number of A-scans in one B-scan. That is, the plane image size is determined by p×n. If p×n is large, a wide range can be scanned with the same measurement pitch, but the scan time is lengthened, and the problems of motion artifacts and patient burden described above occur. In the present embodiment, n=p=300 is set in consideration of the balance between the two. Incidentally, the above n and p can be freely changed as appropriate. Δx in FIG. 2A is the interval (x pitch) between adjacent x positions, and Δy is the interval (y pitch) between adjacent y positions. In this embodiment, the x-pitch and y-pitch are determined as 1/2 of the beam spot diameter of the irradiation light on the fundus, and are set to 10 μm (FIG. 2(b)) in this embodiment. By setting the x-pitch and the y-pitch to 1/2 of the diameter of the beam spot on the fundus, it is possible to form a high-definition image. Even if the x-pitch and y-pitch are made smaller than 1/2 of the fundus beam spot diameter, the effect of further increasing the definition of the generated image is small. Conversely, if the x-pitch and y-pitch are made larger than 1/2 of the fundus beam spot diameter, the definition deteriorates, but a wide range of images can be acquired with a small data volume. The x-pitch and y-pitch may be freely changed according to clinical requirements. The scanning range of this embodiment is p×Δx=3 mm in the x direction and n×Δy=3 mm in the y direction (see FIG. 2B).

[OCTA image acquisition]
Next, a method for generating the OCTA image 301 will be described using FIG. In step S401, the signal processing means 053 extracts the repetitive B-scan interference signal (for m sheets) at the position _yk . At step S402, the signal processing means 053 extracts the j-th tomographic data. In step S403, the signal processing means 053 subtracts the acquired background data from the interference signal. In step S404, the signal processing means 053 transforms the interference signal from which the background has been subtracted into a wavenumber function, and performs Fourier transform. In this embodiment, a fast Fourier transform (FFT) is applied. Note that zero padding may be performed before the Fourier transform to increase the interference signal. By performing the zero padding process, the gradation after Fourier transform is increased, and the alignment accuracy can be improved in step 409 to be described later. At step S405, the signal processing means 053 calculates the absolute value of the complex signal obtained by the Fourier transform performed at step S404. This value is the intensity of the tomographic image of the scan. In step S406, the signal processing means 053 determines whether the index j has reached a predetermined number (m). That is, it is determined whether the intensity calculation of the tomographic image at the position _yk has been repeated m times. If the number is less than the predetermined number, the process returns to S402 to repeat the intensity calculation of the tomographic image at the same Y position. If the predetermined number is reached, proceed to the next step.

In step S407, the signal processing means 053 calculates the similarity of images in the same tomographic images of m frames at a certain _yk position. Specifically, the signal processing means 053 selects an arbitrary one of m frames of tomographic images as a template, and calculates the correlation value with the remaining m−1 frames of images. Note that another method may be used to calculate the correlation value. In step S408, the signal processing unit 053 selects images with high correlation values above a certain threshold among the correlation values calculated in step S407. The threshold can be arbitrarily set, and is set so as to exclude frames in which the correlation as an image has decreased due to blinking or involuntary eye movement of the subject. As described above, OCTA is a technique for distinguishing between tissue with flow (eg, blood) and tissue without flow among subject tissues based on the correlation value between images. That is, tissue with flow is extracted on the assumption that tissue without flow has high correlation between images. It is judged as if the whole is an organization with flow. In this step, in order to avoid such erroneous detection, tomographic images with low correlation are eliminated as images in advance, and only images with high correlation are selected. As a result of the image selection, the m-frame images acquired at the same position yk are appropriately selected to become the _q -frame images. Here, the possible values of q are 1≤q≤m.

In step S409, the signal processing unit 053 aligns the tomographic images of the q frames selected in step S408. The frames selected as templates for alignment may be correlated with each other in all combinations, the sum of correlation coefficients may be calculated for each frame, and the frame with the maximum sum may be selected. Next, the template is matched for each frame to obtain positional deviation amounts (δX, δY, δθ). Specifically, Normalized Cross-Correlation (NCC), which is an index representing the degree of similarity, is calculated while changing the position and angle of the template image, and the difference between the image positions when this value is maximized is obtained as the displacement amount. In the present invention, the index representing the degree of similarity can be changed in various ways as long as it is a scale representing the similarity between the features of the image in the template and the frame.例えばＳｕｍ ｏｆ Ａｂｓｏｌｕｔｅ Ｄｉｆｆｅｒｅｎｃｅ（ＳＡＤ）、Ｓｕｍ ｏｆ Ｓｑｕａｒｅｄ Ｄｉｆｆｅｒｅｎｃｅ（ＳＳＤ）、Ｚｅｒｏ－ｍｅａｎｓ Ｎｏｒｍａｌｉｚｅｄ Ｃｒｏｓｓ－Ｃｏｒｒｅｌａｔｉｏｎ（ＺＮＣＣ）、Ｐｈａｓｅ Ｏｎｌｙ Ｃｏｒｒｅｌａｔｉｏｎ（ＰＯＣ）、Ｒｏｔａｔｉｏｎ Ｉｎｖａｒｉａｎｔ Ｐｈａｓｅ Ｏｎｌｙ Ｃｏｒｒｅｌａｔｉｏｎ（ＲＩＰＯＣ）等を用いてもよい。

Next, the signal processing means 053 applies positional correction to (q−1) frames other than the template according to the amount of positional deviation (δX, δY, δθ) to align the frames. If q is 1, this step is not performed.

In step S410, the signal processing means 053 calculates MC. MC is, for example, a value indicating decorrelation of brightness between tomographic images. In the present embodiment, among q frames of intensity images selected in step S408 and aligned in step S409, a variance value is calculated for each pixel at the same position, and the variance value is defined as MC. There are various ways to obtain MC, and in the present invention, MC can be applied as long as it is an index representing changes in luminance values of pixels of a plurality of tomographic images at the same Y position. When q=1, that is, when the image correlation is low due to the influence of blinking and involuntary eye movement, and MC cannot be calculated at the same position _yk , different processing is performed. For example, the step may be terminated with the feature amount set to 0, or when MC in images of y _k −1 and y _k +1 before and after is obtained, values may be interpolated from variance values before and after. In this case, an abnormality may be notified by regarding the feature amount that could not be calculated correctly as a supplementary value. Alternatively, the Y position where the feature amount could not be calculated may be stored and rescanning may be performed automatically. Alternatively, a warning prompting remeasurement may be issued without performing automatic rescanning.

In step S411, the signal processing unit 053 averages the intensity images aligned in step S409 to generate an intensity-averaged image.

In step S412, the signal processing means 053 performs threshold processing on MC output in step S410. The threshold value is obtained by extracting an area where only random noise is displayed on the noise floor from the intensity-averaged image output by the signal processing means 053 in step S411, calculating the standard deviation σ, and calculating the average brightness of the noise floor (intensity ) value + 2σ. The signal processing means 053 sets 0 to the value of MC corresponding to the area where each intensity is equal to or less than the threshold value. This threshold processing can reduce noise by removing MC derived from random noise. The smaller the threshold value, the higher the MC detection sensitivity, but the noise component also increases. Also, the larger the value, the less the noise, but the lower the sensitivity of MC detection. In this embodiment, the threshold is set as the average intensity value of the noise floor+2σ, but the threshold is not limited to this.

In step S413, the signal processing means 053 determines whether the index k has reached a predetermined number (n). That is, it is determined whether image correlation degree calculation, image selection, alignment, intensity image averaging calculation, MC calculation, and threshold processing have been performed at all n Y positions. If the number is less than the predetermined number, the process returns to S401, and if the number has reached the predetermined number, the process proceeds to the next step S414. When step S413 is completed, an intensity average image and three-dimensional MC volume data (three-dimensional OCTA data) in tomographic images at all Y positions have been generated. The three-dimensional MC image is an example of MC three-dimensional volume data. In step S414, the signal processing unit 053 generates an MC front image (MC EnFace image) by integrating the generated three-dimensional OCTA data in the depth direction. In other words, the signal processing means 053 corresponds to an example of acquisition means for acquiring a motion contrast image of the fundus. FIG. 3A is an example of an MC front image. At this time, when generating the MC front image, the image depth range to be integrated may be set arbitrarily. For example, the signal processing unit 053 extracts the layer boundary of the fundus retina based on the intensity averaged image generated in step S411, and generates the MC front image so as to include the desired layer. After generating the MC front image, the signal processing means 053 terminates the signal processing flow. By using the apparatus configuration, imaging method, and signal processing procedure described above, it is possible to obtain an MC image (OCTA image) in a desired region. Note that the MC image includes a three-dimensional MC image and a two-dimensional MC image such as an EnFace image. In this example, an OCTA image is acquired under the condition of m=4.

For three-dimensional OCTA data obtained by imaging the macula, as shown in FIG. An MC frontal image of the superficial capillary (superficial capillary) such as 3(a) is obtained. The signal processing means 053 can extract the fundus blood vessel 301 at the center of the macula from the MC front image. Note that the depth range for generating the MC front image is not limited to the above example.

[MC image superimposed image acquisition]
Next, a method of acquiring a plurality of MC images will be described with reference to FIG. In step S501, the control means 054 causes the monitor 055 to display a GUI prompting the user to input the number F of MCs to be obtained, and in step S502 the user inputs F (F: an integer equal to or greater than 1). Note that the MC acquisition number F may be a predetermined value regardless of the user's input. In step S503, the control unit 054 controls the OCT optical system to obtain the MC image acquisition number i in order from 1, and as shown in steps S504, 505, and 506, acquires MC images up to F specified by the user. . In step S505, the processing of FIG. 4 is executed. Note that the generation of the MC image may be performed in step S505, may be performed after the scanning for the number F of MC acquisitions is completed, or may be performed in parallel with the scanning for the number F of MC acquisitions. may be

After acquiring the desired number of MC images, the signal processing means 053 stores data (data of a plurality of MC images) in the memory 052 in step S507. In step S508, the signal processing means 053 acquires a superimposed image of MC using the plurality of saved MC images, and stores it in the memory 052 as separate data (step S509). In this embodiment, 10 MC images are acquired. Through this processing, a plurality of MC superimposed images can be acquired. Note that the MC image to be superimposed may be a three-dimensional MC image or a two-dimensional MC image (EnFace image of MC). It should be noted that superimposition of MC images is not an essential process for arteriovenous determination, which will be described later, and arteriovenous determination may be performed from MC images that have not been superimposed.

(Example 1: First arteriovenous determination method)
In this embodiment, an example of arteriovenous separation using superimposed MC images obtained as described above will be described with reference to FIG.

First, the signal processing means 053 acquires from the memory 052 the MC superimposed image 601 shown in FIG. The signal processing means 053 obtains the MC image 602 shown in FIG. 6B from the image 601 by removing the inner limiting membrane and nerve fiber layer (RNFL). By removing the inner limiting membrane and the nerve fiber layer from the MC front image, the MC front image becomes easy to determine arteries and veins. Note that the MC image 062 may be generated by excluding the inner limiting membrane to the upper layer of the inner nuclear layer. That is, the MC image 602 is an MC front image of the layer of blood vessels located in the lower layer of the inner nuclear layer. Note that the MC image 602 may or may not include the layer closer to the choroid than the lower layer in the inner nuclear layer. Alternatively, an MC front image of only the upper layer of the inner nuclear layer may be generated. An MC front image of a portion of the inner nuclear layer is less likely to be affected by blood vessels in the superficial layer, so it is possible to improve the determination accuracy of arteries and veins. The signal processing means 053, for example, generates an MC image 602 from the three-dimensional MC image. Note that the MC front image generated for arteriovenous determination is not limited to the above example. For example, the MC front image generated for arteriovenous determination may be an image containing the ganglion cell layer without containing the inner limiting membrane and the nerve fiber layer, or an image containing the inner limiting membrane, the nerve fiber layer, and the ganglion cell. The image may include a portion of the inner nuclear layer without including the stratum corneum and the inner plexiform layer. Note that, for example, the MC front image generated for arteriovenous determination may not include a layer closer to the choroid than a part of the internal nuclear layer. Here, the part of the inner nuclear layer is, for example, either one of the cases where the inner nuclear layer is divided into upper and lower halves on the vitreous body side and the choroid side.

The signal processing means 053 binarizes the MC image 602 to obtain an image 603 shown in FIG. 6(c). The signal processing means 053 obtains an erosion image 604 by performing erosion on the image 603 (another method may be used as long as it is a noise removal method). The signal processing means 053 obtains the major blood vessels (606-610) from the image 604. FIG. Acquisition of the blood vessel may be performed based on the user's designation for the MC image 602, or may be performed automatically without depending on the user's designation. Also, acquisition of blood vessels may be performed based on a three-dimensional MC image, or may be performed based on a two-dimensional MC image. For example, when acquiring blood vessels based on a 3D MC image, it is possible to acquire blood vessels that intersect in the depth direction as separate objects because they are less susceptible to the crossing of blood vessels when viewed from the front. becomes. In the case of automatically acquiring blood vessels, blood vessels whose length is greater than or equal to a predetermined value may be extracted as main blood vessels.

The signal processing means 053 acquires an image 605 in which the extracted blood vessels are adapted to the original image 602 . Here, the signal processing means 053 identifies the positions of the main blood vessels 606 to 610 in the MC image 602 by, for example, associating the positional relationship between the MC image 602 and the image 605 . Here, since the blood vessel 609 is a branched blood vessel of the blood vessel 606, the signal processing means 053 treats the blood vessel 606 and the blood vessel 609 as the same blood vessel. Therefore, blood vessels 606, 607, 608, and 610 are independent in this process. The signal processing means 053 measures the luminance around the blood vessels (2 pixels: about 20 μm along the blood vessel wall normal) of the independent blood vessels (606, 607, 608, 610). The signal processing means 053 calculates the luminance per unit pixel around the blood vessel (Iave: luminance value per unit pixel) as a representative value of the motion contrast value. Here, the area where luminance is measured is an example of the evaluation area. This evaluation region may or may not be adjacent to a blood vessel. For example, the evaluation region may be a region located within 2 pixels from the blood vessel even if it is not adjacent to the blood vessel. The signal processing means 053 determines whether the blood vessel is an artery or a vein based on the evaluation area that satisfies such conditions. That is, the signal processing means 053 is an example of a determination means for determining whether a blood vessel is an artery or a vein based on the motion contrast value of the evaluation region, which is the region located within a predetermined distance from the region corresponding to the blood vessel in the motion contrast image. Equivalent to.

Also, the evaluation area is not limited to the area of 2 pixels from the blood vessel, and may be larger than 2 pixels or smaller than 2 pixels. For example, the evaluation region may be determined based on vessel diameter. For example, the shorter the blood vessel diameter, the closer the evaluation region to the blood vessel. That is, the predetermined distance from the region corresponding to the blood vessel, which is the distance defining the location where the evaluation region is located, may be determined based on the thickness of the region corresponding to the blood vessel. Specifically, this predetermined distance may be shorter as the thickness of the region corresponding to the blood vessel is smaller.

Alternatively, the width of the evaluation region in the direction intersecting the running direction of the blood vessel may be shortened as the blood vessel diameter becomes shorter. That is, the size of the evaluation region in the direction intersecting the running direction of the blood vessel may be determined based on the thickness of the region corresponding to the blood vessel.

The size of the evaluation region in the running direction of the blood vessel is one of the important parameters in arteriovenous determination. Therefore, the size of the evaluation region in the running direction of the blood vessel may be larger than a predetermined value. For example, the signal processing means 053 may set the evaluation region for the entire blood vessel. Also, the ratio of the evaluation region to the running distance of the blood vessel in the running direction of the blood vessel may be greater than a predetermined value. Note that the evaluation area may be set to a two-dimensional MC image (MC front image), or may be set to a three-dimensional MC image.

The signal processing means 053 determines that the blood vessel is a vein if the calculated average luminance value (Iave) of each blood vessel exceeds a certain threshold value (Is), and determines that the blood vessel is an artery if not. I judge. That is, the signal processing means 053 determines that the blood vessel is a vein when the representative value is greater than the threshold, and determines the blood vessel as an artery when the representative value is less than the threshold.

Actually, the signal processing means 053 can acquire data as shown in FIG. 6(f). Since the average luminance value Iave of the blood vessel 606 is lower than Is, the signal processing means 053 can determine that the blood vessel 606 is an artery. Also, since the luminance average value Iave of the blood vessels 607, 608, and 610 is higher than Is, the signal processing means 053 can determine the blood vessels 607, 608, and 610 as veins. Note that the threshold Is may be fixed or variable. The signal processing means 053 may change the threshold value Is according to the blood vessel diameter, for example. The signal processing means 053 may increase the threshold value Is as the blood vessel diameter decreases. That is, the threshold Is may be a constant value regardless of the blood vessel, or may be a value adaptively set for each blood vessel.

The above process will be described with reference to the flow chart of FIG.

In step S701, the signal processing means 053 reads from the memory 052 the superimposed MC image (three-dimensional MC image) obtained by the processing in FIG. At step S702, the signal processing means 053 acquires an EnFace image from the three-dimensional MC image. In this embodiment, the signal processing means 053 divides the GCL (ganglion cell layer)/IPL (inner plexiform layer) layer boundary and the IPL/INL (inner nuclear layer) (−15 μm: the negative sign indicates the choroid side) An EnFace image was acquired with a depth range of . In step S703, the signal processing means 053 applies image processing to the image obtained in step S702 to extract major blood vessels. In this embodiment, the signal processing means 053 performs erosion processing once after binarizing the MC image (see FIG. 6(d)). The signal processing means 053 extracts a set (line) of dots (pixels) that continue for a certain number or more in the image. In this example, the signal processing means 053 extracted lines of 80 pixels or more (approximately 1.0 mm or more) as major blood vessels. Note that the numerical value for blood vessel extraction is not limited to 80 pixels.

At step S704, the signal processing means 053 numbers the major blood vessels obtained at step S703. At step S705, the signal processing means 053 adapts the numbered blood vessels to the MC image acquired at S702 (see FIG. 6(e)). In step S706, the signal processing means 053 uses the image 605 obtained in step S705 to measure the luminance (motion contrast value) around each blood vessel. In this embodiment, the brightness around the blood vessel (2 pixels: approximately 20 μm area) is measured, and the brightness average value (Iave) is calculated for each blood vessel. The signal processing means 053 may use the median value of luminance instead of the average luminance. In step S707, the signal processing means 053 compares the average brightness (Iave) of each blood vessel obtained in step S706 with the threshold value (Is). In step S708, the signal processing means 053 determines an artery when Iave is lower than Is. In step S709, if Iave is higher than Is, it is determined as vein.

According to the above example, it is possible to determine arteries and veins from the motion contrast image. In place of Iave, the median value, standard deviation, maximum value, or the like may be used as the representative value of the evaluation area.

(Example 2: Second arteriovenous determination method)
In Example 1, arteriovenous determination was performed based on the brightness at a position 20 μm from the blood vessel wall perpendicular line, but in the present example, arteriovenous determination is performed by measuring the brightness profile from the center of the blood vessel as follows.

Specifically, as in the first embodiment, the signal processing means 053 extracts blood vessels from the MC image (FIG. 10(a)). As shown in FIG. 10(a), the signal processing means 053 defines areas 1011, 1012, 1013 of 500 μm perpendicularly to the running direction of the extracted blood vessels 1010, 1014. FIG. Areas 1011, 1012, and 1013 may be defined based on user input, or may be defined independently of user input. In this specification, the term “perpendicular” is a concept including completely perpendicular and substantially perpendicular. Also, the size of the areas 1011, 1012, and 1013 is not limited to 500 μm, and may be other values. For example, the above areas may be sized individually for each vessel. The size of the area may be changed according to the diameter of the blood vessel. For example, the smaller the diameter of the blood vessel, the smaller the size of the area. The signal processing means 053 measures the brightness profile (motion contrast value profile) for each of the areas 1011 , 1012 and 1013 . The luminance profile of the area 1011 was measured as shown in FIG. 10(b), similarly, the area 1012 was measured as shown in FIG. 10(c), and the area 1013 was measured as shown in FIG. 10(d). The signal processing means 053 determines arteries and veins from the luminance profile shown in FIG. 10(b). The signal processing means 053 sets, for example, the luminance value (motion contrast value) of a region in which blood vessels do not exist (the same level as the background noise) as Is (luminance threshold). The signal processing means 053 can divide the profile into a profile region 1001 with higher luminance than Is, a profile region 1002 with lower luminance than Is, a profile region 1003 with higher luminance in the profile, and a profile region 1004 with lower luminance than Is. . Comparing the area 1011 in FIG. 10A with the luminance profile in FIG. It can be confirmed that thick blood vessels and a non-vascular area exist in the profile area 1004 . Furthermore, the avascular regions of the profile region 1002 and the profile region 1004 show that the avascular region exists at a plurality of points (pixels), that is, regions of several tens of μm or more. From the vascular structure of the fundus, a blood vessel 1010 surrounded by an avascular region can be determined as an artery.

Similarly, the brightness profile of area 1012 will be described with reference to FIG. 10(c). . Similar to the area 1011, the signal processing means 053 processes the profile area 1005 with higher luminance than Is, the profile area 1006 with lower luminance than Is, the profile area 1007 with higher luminance in the profile, and the profile area 1007 with lower luminance than Is. The profile shown in FIG. 10( c ) can be divided into profile regions 1008 . Comparing the area 1012 with the luminance profile in FIG. thick), and the profile region 1008 shows the presence of an avascular region. Furthermore, it can be seen that the avascular regions of profile region 1006 and profile region 1008 have avascular regions at a plurality of points, that is, regions of several tens of μm or more. Based on the existence of the avascular region, the signal processing means 053 can determine that the region 1012 is an artery from the MC image because the region 1012 also has an avascular region around it. The blood vessel in the area 1012 is a natural result because the blood vessel 1010 branches off. In this embodiment, a plurality of areas 1011 and 1012 are set for one blood vessel 1010. Arteriovenous determination is performed based on the profile of one area, and the determination result is reflected in the connecting blood vessels. good too. Also, in order to improve the accuracy of determination, only when a plurality of areas are determined to be arteries (or veins), the determination target blood vessel may not be determined to be an artery.

Similarly, the brightness profile of region 1013 will be described with reference to FIG. 10(d). As with the region 1011, the signal processing means 053 can divide the profile into a profile region 1009 with a higher luminance than Is, a profile region 1015 with a lower luminance than Is, and a profile region 1015 with a higher luminance in the profile. Profile regions 1009 and 1016 with luminance higher than Is contain some kind of blood vessels such as capillaries. Furthermore, it can be seen that the low intensity profile region 1015 is a single point and is between blood vessels rather than an avascular region. The signal processing means 053 can determine that the blood vessel 1014 is a vein because there is no avascular region (or because the avascular region is below the threshold).

That is, the signal processing means 053 can determine the artery and vein from the size of the avascular region obtained from the motion contrast value profile.

The determination described above, that is, the procedure of the second arteriovenous determination method will be described with reference to FIG. In step S1101, the signal processing means 053 reads out the superimposed MC image obtained by the processing in FIG. In step S1102, the signal processing means 053 extracts blood vessels from the MC image. In step S1103, the signal processing means 053 sets a straight line of 200 μm in the direction perpendicular to the running direction of the blood vessel, and measures the luminance (motion contrast value) on the straight line. In step S1104, the signal processing means 053 reads the threshold value Is from the memory 052. FIG. In step S1105, the signal processing unit 053 measures the number (number of pixels) n of regions whose brightness is lower than the threshold value Is. The threshold Is corresponds to an example of a first threshold. In this embodiment, only the number of pixels is counted. In steps S1106 and 1107, if the number of pixels measured in step S1105 is n>4, the signal processing means 053 determines that the area is an artery because the number of avascular regions is greater than the threshold. In steps S1106 and 1109, if n≤4 , the signal processing means 053 determines that the avascular region is a vein because the avascular region is below the threshold. That is, it can be said that the signal processing means 053 determines whether the blood vessel is an artery or a vein based on the size of the region in which the motion contrast value is less than the first threshold in the evaluation region. Also, the above "4" corresponds to an example of the second threshold. That is, it can be said that the signal processing unit 053 determines that the blood vessel is an artery when the size of the region in which the motion contrast value is less than the first threshold in the value region is greater than the second threshold. Further, it can be said that the signal processing unit 053 determines that the blood vessel is a vein when the size of the region in which the motion contrast value is less than the first threshold in the evaluation region is smaller than the second threshold.

In step S1110, the control means 054 causes the monitor 055 to display the result. For example, the control means 054 causes the monitor 055 to display an MC image in which arteries and veins are shown in different colors.

As described above, the artery and vein can be determined by measuring the luminance around the blood vessel using the MC image. In this embodiment, the profile area is a linear area, but a two-dimensional area, a three-dimensional area, an area made up of a set of points, or the like may be used. Similarly, the size may be any size other than those of the examples. For the luminance measurement, luminance average, standard deviation, maximum value, etc. may be used as representative values of the evaluation area.

As for the threshold value Is, it is possible to make a highly accurate determination by changing it according to the installation environment, setting by the user, and race. As described in the flow of FIG. 11, the flow for extracting blood vessels from images may be manual or automatic.

Although MC images are superimposed in this embodiment, it is also possible to determine arteries and veins using MC images alone.

(Modification 1)
At step 702, when acquiring the MC image, if the MC front image is acquired based on the INL (inner nuclear layer), the arteriovenous determination becomes more stable. Specifically, the signal processing means 053, as shown in FIG. line 309 (below ganglion cell layer) to line 306 (INL middle), line 306 (INL middle) to line 310 (outer plexiform layer). Using this division information, the signal processing means 053 divides into four layers as follows, and when the above-described arteriovenous determination is performed in the second layer, the data is stabilized. A first layer is defined as the layer between line 307 and about 5 μm below line 308 (in the choroidal direction). A second layer is defined as the layer between more than 5 μm below line 308 and about 5 μm above line 309 . The third layer is defined as the layer between line 306 and about 5 μm above line 309 . A fourth layer is defined as the layer between lines 306 and 310 . By using the second layer as the MC image (MC front image), arteriovenous determination can be obtained with higher accuracy. MC images with the second layer correspond to images that do not include the inner limiting membrane and nerve fiber layer, but include the ganglion cell layer. Line 304 in FIG. 3C indicates the INL upper end, and line 305 indicates the INL lower end. Also, the third layer and the fourth layer may be used for arteriovenous determination. An MC image using the fourth layer corresponds to an image that does not include the inner limiting membrane, nerve fiber layer and ganglion cell layer, but includes only a portion of the entire inner nuclear layer.

Although the line 306 is the center of the INL, the line 306 is not limited to this, and the line 306 may be separated into two upper and lower layers based on the arrangement of blood vessels in the INL. Since the blood vessels in the INL are linearly aligned in the X direction (direction orthogonal to the depth direction) (for example, the blood vessels in the upper and lower two stages are respectively linearly aligned), the signal processing means 053 is arranged according to the blood vessel array. Based on this, the position for separating the INL into two layers can be obtained. Specifically, the signal processing means 053 obtains lines connecting the blood vessels in the INL in the X direction (or Y direction) for two stages of the blood vessels in the INL, and obtains the midpoint between the obtained lines in the depth direction. A line connecting a plurality of midpoints obtained in the X direction may be used as the line 306 . That is, the signal processing means 053 corresponds to an example of a determination means for determining a part of the inner nuclear layer (the upper layer or the lower layer of the inner nuclear layer) based on the arrangement of the blood vessels contained in the inner nuclear layer. . The signal processing means 053 uses the lines 306 to generate MC images of the third and fourth layers. That is, it corresponds to an example of generating means for generating a motion contrast image based on a part of the inner nuclear layer.

Also, an MC image of only a portion (upper or lower layer) of the inner nuclear layer defined by line 306 may be generated. This modified example can also be applied to the first, third, and fourth embodiments.

(Modification 2)
In this example, a clear MC image can be acquired by separating blood vessels, and an example in which the analyzes in Examples 1 and 2 are further improved will be described.

An image 901 in FIG. 9A is an image displaying only the MC signal in the XZ cross section. A black dot 902 in FIG. 9(a) is an area (point) having a temporal change in the OCT image. A black dot 902 is a portion having a signal with a motion contrast value greater than or equal to a predetermined value, that is, it corresponds to a blood vessel portion. The black spot 903 is also the same. Black dots 902 indicate blood vessels in the superficial layer of the retina, and black dots 903 indicate blood vessels in the choroidal area.

If the brightness (motion contrast value) is added in the x direction at the arrow positions of a, b, c, and d in the image of FIG. 9(a), it becomes as shown in FIG. 9(b). The positions of the arrows a, b, c, and d can be identified from the peak positions of the profile of the luminance tomographic image in the A-scan direction. Note that the A-scans for specifying the positions of the arrows a, b, c, and d may use the integrated values of a plurality of A-scans. It was found that the blood vessels existing in the superficial capillary and the deep capillary can be classified into four blood vessels, a to d, as shown in FIG. 9(a). In particular, the blood vessel layer a has a low brightness and is considered to belong to the first layer in the above example. Similarly, vascular layer b is thought to have belonged to the second layer, vascular layer c to the third layer, and vascular layer d to the fourth layer. For example, the signal processing means 053 changes the angle of the line for integration in the x-direction so that the integrated value of the brightness of the blood vessel layer b corresponding to the second layer is increased, and the integrated value of the brightness of the blood vessel layer b is maximized. An MC front image for arteriovenous determination may be generated with reference to a line with a different angle. The signal processing means 053 generates the MC front image using, for example, a region having a predetermined width in a direction perpendicular to the line centered on the line of the angle at which the integrated value of the brightness of the blood vessel layer b is maximum. Note that the predetermined width may be an arbitrary value determined in advance, or may use a variance value or a standard deviation value of the motion contrast values of the blood vessel layer. That is, the signal processing means 053 generates an MC front image based on the arrangement of blood vessels.

The position of the blood vessel layer b may be specified by the user, or may be determined based on the position of the peak obtained by sequentially integrating the brightness in the x direction in the z direction using the signal processing means 053. be. The signal processing means 053 can specify, for example, the second peak position from the choroid side as the blood vessel layer b.

By generating an MC front image using blood vessels as in this example, an MC image suitable for arteriovenous determination can be obtained. In addition, it is suitable not only for arteriovenous determination but also for diagnosis because an MC front image of a desired blood vessel is generated. This modified example can also be applied to the first, third, and fourth embodiments.

(Example 3: Display of information on arteriovenous determination)
In Examples 1 and 2, it was described that arteriovenous determination is automatically performed based on the MC image based on the motion contrast value or the size of the avascular region in the evaluation region. An example of displaying supporting information will be described.

In this embodiment, the control means 054 causes the monitor 055 to display at least one of Iave obtained in step S706 of FIG. 7 and the number n of pixels indicating the avascular region obtained in step S1105 of FIG. In this embodiment, automatic determination of whether a blood vessel is an artery or a vein is not performed. Iave or number n corresponds to an example of information indicating whether a blood vessel is an artery or a vein. Also, the control means 054 corresponds to an example of a display control means for causing the display unit to display information indicating whether it is an artery or a vein.

The control means 054 causes the monitor 055 to display at least one of Iave and the number n regarding the blood vessel selected by the user or automatically selected by the device. The control means 054 may further cause the monitor 055 to display at least one of Iave and the number n, which is a reference for arteriovenous determination. Also, the MC image may be further displayed on the monitor 055 . For example, when a blood vessel on an MC image is designated by the user, at least one of Iave and number n related to the blood vessel, and a threshold value of at least one of Iave and number n, which is a reference for arteriovenous determination, are displayed on the monitor 055. .

In this way, the user can grasp information necessary for arteriovenous determination.

(Example 4: FAZ detection)
A present Example describes the analysis apparatus which can provide FAZ analysis more stably.

FAZ is an avascular region in the fovea, which is the region 311 in FIG. 3(a). In conventional superficial capillary images using MC signals, FAZ measurements were not stable. The reason is that the dividing boundary between the superficial capillary and the deep capillary is ambiguous and depends on the layer segmentation definition.

The flow of this embodiment will be described with reference to FIG.

In step S801, the signal processing means 053 acquires the MC image after superimposition from the memory 052. FIG. In step S802, the image obtained in step S801 is separated into the first, second, third, and fourth layers described above. In step S803, the signal processing means 053 extracts the MC image of the third layer divided in step S802. For example, the signal processing unit 053 generates a third-layer MC image (MC front image) based on the superimposed three-dimensional MC image obtained in step S801. In step S804, the signal processing unit 053 measures FAZ using the third layer MC image acquired in step S803. In this example, the FAZ was measured by binarizing the MC image of the third layer.

When the FAZ is measured with the flow as described above, the repeatability is improved as compared with the measurement with the conventional superficial capillary. Moreover, in diseased eyes, the numerical value of FAZ was further stabilized by performing the same processing.

The processing of this embodiment is particularly effective in measuring changes over time because it is possible to accurately extract slight changes when acquiring information on changes over time in the same subject.

(Other embodiments)
Although the embodiments have been described in detail above, the technology disclosed herein can be embodied as, for example, a system, an apparatus, a method, a program, a recording medium (storage medium), or the like. Specifically, it may be applied to a system composed of a plurality of devices (for example, a host computer, an interface device, an imaging device, a web application, etc.), or may be applied to a device composed of a single device. good.

Moreover, it goes without saying that the object of the present invention is achieved by the following. That is, a recording medium (or storage medium) recording software program code (computer program) for realizing the functions of the above-described embodiments is supplied to the system or device. Such a storage medium is, of course, a computer-readable storage medium. Then, the computer (or CPU or MPU) of the system or device reads and executes the program code stored in the recording medium. In this case, the program code itself read from the recording medium implements the functions of the above-described embodiments, and the recording medium recording the program code constitutes the present invention.

Note that the above-described embodiments and modifications may be combined as appropriate.

053 signal processing means 054 control means 055 monitor

Claims (16)

acquisition means for acquiring a motion contrast image of the fundus;
determination means for determining whether the blood vessel is an artery or a vein based on a representative value of motion contrast values of an evaluation region, which is a region located within a predetermined distance from a region corresponding to the blood vessel in the motion contrast image ;
The image processing apparatus , wherein the representative value is an average value of motion contrast values in the evaluation area . 2. The determining means determines that the blood vessel is an artery if the representative value is less than a threshold, or determines that the blood vessel is a vein if the representative value is greater than the threshold. image processing device. acquisition means for acquiring a motion contrast image of the fundus;
It is determined whether the blood vessel is an artery or a vein based on the size of the region in which the motion contrast value of the evaluation region, which is the region located within a predetermined distance from the region corresponding to the blood vessel in the motion contrast image, is less than a first threshold. and a determination means,
The determining means determines that the blood vessel is an artery when the size of the area in the evaluation area where the motion contrast value is less than the first threshold is greater than the second threshold, or the motion contrast value in the evaluation area is determined to be an artery. An image processing apparatus, wherein the blood vessel is determined to be a vein when the size of the region smaller than the first threshold is smaller than the second threshold. 4. The image processing apparatus according to claim 1 , wherein the evaluation area is an area adjacent to the area corresponding to the blood vessel. 5. The image processing apparatus according to any one of claims 1 to 4 , wherein the size of the evaluation area in the running direction of the blood vessel is larger than a predetermined value. 5. The image processing apparatus according to any one of claims 1 to 4 , wherein a ratio of the evaluation area to the running distance of the blood vessel in the running direction of the blood vessel is larger than a predetermined value. 5. The method according to any one of claims 1 to 4 , wherein the size of the evaluation area in a direction intersecting the running direction of the blood vessel is determined based on the thickness of the area corresponding to the blood vessel. image processing device. 8. The image processing apparatus according to any one of claims 1 to 7 , wherein the predetermined distance is determined based on the thickness of the area corresponding to the blood vessel. 9. The image processing apparatus according to claim 8 , wherein the predetermined distance becomes shorter as the thickness of the region corresponding to the blood vessel becomes smaller. 10. The image processing apparatus according to claim 1 , wherein said motion contrast image is an EnFace image. The motion contrast image is an image that does not include the inner limiting membrane and the nerve fiber layer but includes the ganglion cell layer, or an image that does not include the inner limiting membrane, the nerve fiber layer and the ganglion cell layer and is the entire inner nuclear layer 11. The image processing apparatus according to claim 10 , wherein the image is an image including only a part. a determining means for determining a portion of the inner nuclear layer based on the arrangement of blood vessels contained in the inner nuclear layer;
12. The image processing apparatus according to claim 11 , further comprising generating means for generating the motion contrast image based on the portion of the inner nuclear layer determined by the determining means. 13. The image processing apparatus according to claim 12 , wherein the determining means determines the part of the inner nuclear layer based on the arrangement of blood vessels in a direction orthogonal to the depth direction. 14. The image processing apparatus according to claim 12 , wherein the motion contrast image does not include an image of a layer closer to the choroid than part of the inner nuclear layer. an acquisition step of acquiring a motion contrast image of the fundus;
a determination step of determining whether the blood vessel is an artery or a vein based on a representative value of the motion contrast values of the evaluation region, which is the region located within a predetermined distance from the region corresponding to the blood vessel in the motion contrast image ;
The image processing method , wherein the representative value is an average value of motion contrast values in the evaluation area . A program for causing a computer to execute the image processing method according to claim 15 .

Images