JP6720267B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

The present invention is a technique relating to improving the quality of a tomographic image in an eye part.

A tomographic image capturing device of an eye such as an optical coherence tomography (hereinafter referred to as “OCT”) is capable of three-dimensionally observing a state inside a retinal layer. This tomographic imaging apparatus has been attracting attention in recent years because it is useful for more accurate diagnosis of diseases.

In ophthalmological diagnosis, a volume image and a high-quality two-dimensional tomographic image for recognizing a layer that is not included in a low-quality tomographic image may be used in order to understand the state of the entire retinal layer. The volume image is a set of two-dimensional tomographic images.

The image quality of a tomographic image obtained by OCT depends on the intensity of near infrared light incident on the retina. For this reason, in order to improve the image quality of the tomographic image, it is necessary to increase the intensity of the light irradiating the retina, but from the viewpoint of safety, there is a limit to the intensity of the light irradiating the retina. For this reason, it is desired to generate a high-quality tomographic image while irradiating the near-infrared light in an intensity range where there is no problem for safety. In response to such a demand, a technique is disclosed in which captured two-dimensional tomographic image groups are superimposed on each other to generate a cross-sectional image with less noise (see Patent Document 1).

JP 2008-237238A

However, in Patent Document 1, all the plurality of tomographic images are simply averaged. Therefore, when the correlation between the added tomographic images is low, the reduction of the diagnostic information may be large. In particular, the eye may suffer from involuntary eye movements, so that not all areas of adjacent images are similar.

The present invention has been made in view of the above problems, and an object thereof is to improve the image quality of a tomographic image.

An image processing apparatus according to an aspect of the present invention for achieving the above object has the following configuration.

That is, the plurality of tomographic images of the fundus of the eye to be inspected, which are obtained based on the measurement light of OCT (Optical Coherence Tomography) controlled to scan the same position a plurality of times, are included in the plurality of tomographic images. First alignment means for aligning the tomographic images of
After the alignment by the first alignment means, the partial region which is a partial region of the tomographic image having a predetermined width in the direction orthogonal to the A-scan direction is used and the partial region for another tomographic image. of similarity acquired by performing positioning of the a-scan image which is narrower than the partial region with respect to the other tomographic image using the acquired result of the similarity of the partial region, with a-scan image unit Second alignment means for aligning;
Generating means for generating one tomographic image from the plurality of tomographic images after the alignment by the second aligning means.

According to the present invention, the image quality of a tomographic image can be improved.

It is a figure which shows the structure of an image processing system. It is a flow chart which shows the flow of the tomographic image photography processing in an image processing device. It is a figure for demonstrating an overlay image generation process. It is a figure for explaining a superposition field. It is a figure for demonstrating the determination process of superposition. It is a figure which shows the structure of an image processing system. It is a flow chart which shows the flow of the tomographic image photography processing in an image processing device. It is a figure for explaining a superposition field.

(Example 1)
Hereinafter, an image processing system including the image processing apparatus according to the present embodiment will be described with reference to the drawings.

FIG. 1 is a diagram showing a configuration of an image processing system 100 including an image processing apparatus 110 according to this embodiment. As shown in FIG. 1, the image processing system 100 is configured by connecting an image processing apparatus 110 to a tomographic image capturing apparatus 120 via an interface.

The tomographic image capturing apparatus 120 is an apparatus that captures a tomographic image of an eye portion, and includes, for example, a time domain OCT or a Fourier domain OCT. Since the tomographic image capturing apparatus 120 is a known apparatus, detailed description will be omitted.

The image processing device 110 includes an acquisition unit 111, a storage unit 112, an image processing unit 113, and a display control unit 114.

The acquisition unit 111 acquires the tomographic image captured by the tomographic image capturing apparatus 120 and stores the tomographic image in the storage unit 112. The image processing unit 113 generates a new two-dimensional tomographic image from the tomographic image stored in the storage unit 112. The display control unit 114 controls to display the processed image on a monitor (not shown).

Although a plurality of points may be sequentially scanned, as an example in which the image quality is particularly improved, in FIG. The schematic diagram of a three-dimensional tomographic image group is shown. Here, the direction in which the measurement light is scanned to capture a two-dimensional tomographic image is called the main scanning direction. The direction orthogonal to the main scanning direction will be referred to as the sub-scanning direction.

In general, the tomographic image capturing apparatus 120 captures an image in the main scanning direction while shifting the measurement light in the sub scanning direction. That is, the present invention can be applied to the case of shifting in the sub-scanning direction.
In FIG. 3A, x and z represent coordinate axes, and t represents a time axis. T _{1 to} T _n are two-dimensional tomographic images of the macula taken at different times. That is, the two-dimensional tomographic image group is formed by a set of two-dimensional tomographic images obtained by photographing almost the same portion.

Here, improving the image quality means improving the S/N ratio.

Further, improvement of image quality means improvement of S/N ratio.

Next, the processing procedure of the image processing apparatus 110 of the present embodiment will be described with reference to the flowcharts of FIGS. 2(a) and 2(b).

<Step S201>
In step S201, the tomographic imaging apparatus 120 is controlled by a control unit (not shown) in order to image the retinal layer. First, the positions of the depth direction (z direction in FIG. 3A), which is the direction in which the measurement light is irradiated, and the plane direction (x direction in FIG. 3A) orthogonal to the z direction are adjusted. Here, matching the position in the depth direction corresponds to matching the position of the coherent gate for obtaining the tomographic image.

<Step S202>
In step S201, the control unit (not shown) adjusts the position of the retinal layer to a position suitable for imaging, and then in step S202, the imaging instruction unit (not shown) instructs the start of imaging.

<Step S203>
In step S203, when the operator gives a photographing instruction, the control unit (not shown) repeatedly scans substantially the same portion to photograph a plurality of two-dimensional tomographic images.

The control unit (not shown) also has a function of adjusting the interval of movement in the sub-scanning direction.

<Step S204>
In step S204, the image processing unit 113 generates a new two-dimensional tomographic image using the two-dimensional tomographic image group stored in the storage unit 112. The process of generating a high-quality two-dimensional tomographic image will be described with reference to FIG.

<Step S210>
In step S210, the first alignment unit 115 aligns the two-dimensional tomographic images. As the alignment processing, for example, an evaluation function indicating the similarity between two two-dimensional tomographic images is defined in advance, and the tomographic image is deformed so that the value of this evaluation function becomes the best. Examples of the evaluation function include a method of evaluating with a pixel value (for example, a method of performing evaluation using the correlation coefficient of Expression (1)). Further, as the image transformation process, a process of performing translation or rotation by using an affine transformation or changing an enlargement ratio can be cited. As the positioning process, the positions may be matched on the basis of feature points. For example, a feature such as each retinal layer or lesion is extracted from the two-dimensional tomographic image. The inner limiting membrane, nerve fiber layer, photoreceptor inner segment/outer segment junction, and retinal pigment epithelium layer have high brightness values, and the layer boundaries have high contrast. Make a match.

Therefore, the position deformation parameter that maximizes the evaluation function is calculated while deforming the two-dimensional tomographic images, and the positions of the two-dimensional tomographic images are matched. When the number of two-dimensional tomographic images for superposition is N, the N-1 two-dimensional tomographic images are aligned with respect to the reference two-dimensional tomographic image.

<Step S211>
In step S211, the determination unit 117 determines an A-scan image to be superimposed for each corresponding A-scan image. This process will be described with reference to FIG. T _i ′ and T _i+1 ′ are two-dimensional tomographic images taken at different times, and are aligned in step S210. A _ij ′ and A _(i+1)j ′ represent the A scan images corresponding to T _i ′ and T _i+1 ′ after the alignment, respectively. Note that, in the A-scan image in the present invention, one pixel column parallel to the z-axis direction in FIG. 4 is referred to as an A-scan image.

The A-scan image is an image that matches the incident direction of the measurement light, and the individual A-scan images obtained from the same location have substantially the same image information. Therefore, even if the entire two-dimensional tomographic image has low similarity due to the eye's involuntary eye movement, there may be data having high similarities in the A-scan images of different two-dimensional tomographic images.

The same diagonally shaded areas R _ij ′ and R _{(i+1) j} ′ represent +-α areas in the x-axis direction around A _ij ′ and A _{(i+1) j} ′. The calculator 116 calculates the similarity between the A-scan images in the image regions R _ij ′ and R _(i+1)j ′ centered on the A-scan images. When R _ij ′ is the reference region for performing the overlay determination, the calculation unit 116 calculates the degree of similarity between all corresponding regions of R _1j ′ to R _nj ′. Here, the equation when the correlation coefficient is used as the evaluation function indicating the similarity between the A scan images is shown in Equation 1.

In the case of FIG. 4, the region R _ij ′ is f(x, y) and R _{(i+1) j} ′ is g (x, y).

Represents the average of the area f(x, y) and the area g(x, y), respectively.

The determination unit 117 selects a region to be used for superposition for each region. The processing of the determination unit will be described with reference to FIG. FIG. 5 shows an example of the result of calculation of the similarity between regions by the calculation unit 116.

The horizontal axis represents the numbers 1 to N of the two-dimensional tomographic images taken. The vertical axis represents the degree of similarity between the reference area and the area in another two-dimensional tomographic image. FIG. 5A shows an example in which the threshold value Th is set and a region having a similarity equal to or higher than a predetermined value is selected. FIG. 5B is a diagram in which FIG. 5A is sorted in order of similarity, and is an example in which the top M sheets of similarity are selected. In FIG. 5C, the sorting process is performed in the order of the similarity, and the change rate of the similarity after the sorting process is represented by a line graph. Here, an example is shown in which an image before the rate of change of the similarity is significantly reduced is selected. That is, the similarity at which the rate of change of the similarity after the sorting process shows a predetermined value is obtained, and a region having a value equal to or higher than the obtained similarity is selected.

Also, the rate of change between similarities is checked without sorting. The calculation unit 116 stops the calculation of the evaluation value when the similarity is lower than the predetermined value. An image whose evaluation value is equal to or greater than a predetermined value and before calculation is stopped is selected.

In the case of the threshold value of (a), those having a bad evaluation value are not used for superposition, and therefore only the regions having a good evaluation value can be superposed.

This is suitable when the moving distance per unit time in the sub-scanning direction is small. This is because there is little change in the tissue structure, but it has the effect of not selecting an image of a region in which the tissue structure caused by a specific eye movement or blinking greatly differs.

When M is selected as the fixed number in (b), there is no variation in the number of data for which the overlay averaging process is performed in image units. There is no variation in the number of data items for which the overlay averaging process is performed for each A-scan image unit. Since the degree of noise reduction can be made uniform, it is more suitable when the image quality is the same.

When looking at the change rate of (c), even if the image quality is poor overall and the degree of similarity is low due to illness or the like, it is possible to select similar regions.

The method (d) is also suitable when the movement distance per unit time in the sub-scanning direction becomes large. This is because if the displacement in the sub-scanning direction exceeds a predetermined value, the tissue structure of the retina changes, and it is possible to prevent the calculation unit 116 from unnecessarily calculating the evaluation value. In other words, by looking at the rate of change, changes in organizational structure can be understood.

In the above, in each area, the area to be superposed is selected based on the evaluation value. Therefore, if the retinal layer is deformed in the two-dimensional tomographic image due to involuntary eye movement, or if the image quality is partially degraded due to blinking, vignetting, etc., it is not used for superposition and is newly generated. The image quality is improved.

Further, the determination unit 117 performs a process of combining the above (a) to (d) according to the moving distance per unit time in the sub-scanning direction. For example, if the movement distance is less than the predetermined value, the process (a) or (b) is used, and if the distance exceeds the predetermined value, the process (c) or (d) is performed. With the combination of (a) and (d), speed-oriented processing can be performed, and with the combination of (b) and (d), image-quality-oriented processing can be performed.

<Step S212>
In step S212, the generation unit 118 performs superposition processing. Here, superposition in the case of two A-scan images will be described. FIG. 3B is a diagram for explaining a process of processing a two-dimensional tomographic image for each A scan image and generating one two-dimensional tomographic image as a composite image. That is, the averaging process of the A-scan images located on different two-dimensional tomographic images captured at different times will be described as an example. FIG. 3C is a high-quality two-dimensional tomographic image generated by performing the averaging process using M (two in this example) pixels for each pixel. That is, in FIG. 3C, A _ij ″ is a new A scan image calculated by performing the averaging process on the corresponding A scan image.

In FIG. 3B, T _i ′ and T _i+1 ′ are two-dimensional tomographic images obtained by imaging the same cross section at different times. A _ij ′ and A _(i+1)j ′ represent respective A scan images in the tomographic images T _i ′ and T _i+1 ′. The generation unit 118 calculates A _ij ″ in FIG. 3C by performing the averaging process on the A scan images A _ij ′ and A _{(i+1) j} ′. The high-quality two-dimensional tomographic image The generation process of the (composite image) is not limited to the averaging process, but may be a median value calculating process, a weighted averaging process, etc. These processes are performed by the A scan of the reference image T _i ′. This is performed for all the images (from A _i1 ′ to A _im ′) For example, in the weighted addition averaging process, the above-described similarity is used as the weight of addition.

In the present embodiment, the region for calculating the degree of similarity is set to R _ij ′, and the overlapping process is performed on A _ij ′ (A scan image unit), but the present invention is not limited to this. .. For example, the superimposing process may be performed in units of areas in which the degree of similarity is calculated. In addition, the superimposing process may be performed on the two-dimensional tomographic image. Further, α may be set to 0 (in the case of α=0, R _ij ′=A _ij ′), the similarity may be calculated for each A scan image unit to make a determination, and superimposition may be performed.

<Step S205>
In step S205, the display control unit 114 displays the high-quality two-dimensional tomographic image generated in step S204 on a display unit (not shown). Also, an example has been described in which the scanning range of the measurement light of the tomographic image capturing apparatus 120 is fixed and substantially the same retina region is scanned, but the present invention can be applied even if the entire retina is sequentially scanned, as described above. Is.

As is clear from the above description, in the present embodiment, in a plurality of aligned two-dimensional tomographic images, the similarity between the regions is calculated using the corresponding A scan image and the surrounding region, and each region is calculated. The area used for the overlaying process was determined in units. As a result, even if the position of the entire image is correct, even if the retinal layer in the two-dimensional tomographic image is deformed due to fixational eye movements, the region used for the overlay processing is determined in units of partial regions. Therefore, it is possible to generate a high-quality two-dimensional tomographic image.

According to the present embodiment, it is possible to obtain a high-quality two-dimensional tomographic image even when the retinal layer in the two-dimensional tomographic image is deformed due to a slight eye movement. Here, the high-quality image refers to an image having an improved S/N ratio as compared with one shot. Alternatively, it refers to an image in which the amount of information necessary for diagnosis is increasing.

(Example 2)
In the first embodiment, in a two-dimensional tomographic image that has been aligned, the similarity between the regions is calculated using the corresponding A-scan image and the surrounding region, and the region used for the overlay processing in each region unit. Was judged. The present embodiment is characterized in that, in a two-dimensional tomographic image that has been aligned, the A-scan image and the peripheral region are used to search for a region having a higher evaluation value in a neighboring region, and superimpose processing is performed. .. According to this embodiment, the overall feature is used for the alignment, and the local feature is used for the alignment, and then the overlay process is performed.

FIG. 6 is a diagram showing a configuration of an image processing system 600 including an image processing device 610 according to this embodiment. As shown in FIG. 6, the image processing device 610 includes an acquisition unit 111, a storage unit 112, an image processing unit 613, and a display control unit 114. Of these, the functions other than the image processing unit 613 have the same functions as those in the above-described first embodiment, and therefore description thereof is omitted here.

In the image processing unit 613, the second alignment unit 619 performs local alignment using the A scan image and the surrounding area.

Hereinafter, the processing procedure of the image processing apparatus 610 according to the present embodiment will be described with reference to FIGS. 7 and 8. Note that the steps other than step S704 are the same as steps S201 to S203 and step S205 of the first embodiment, and therefore description thereof will be omitted.

<Step S704>
In step S704, the image processing unit 613 performs global alignment and local alignment, and performs image overlay processing to generate a high-quality two-dimensional tomographic image. The high-quality two-dimensional tomographic image generation process will be described with reference to FIG.

<Step S710>
In step S710, the first alignment unit 115 aligns the two-dimensional tomographic images. This processing is the same as step S210 of the first embodiment, so description will be omitted.

<Step S711>
In the two-dimensional tomographic image that has been globally aligned in step S710, local alignment is performed by performing alignment in units of corresponding A scan images. This process will be described with reference to FIG. T _i ′ and T _i+1 ′ are images of the same cross section taken at different times, and are two-dimensional tomographic images that have been aligned in step S710. A _ij ′ represents the A scan image at T _i ′. A shaded area R _ij ″ represents a range of +−α in the x-axis direction around A _ij ′. R _{(i+1) j} ″ represents a rectangular area corresponding to R _ij ″ at T _i+1 ′. S _(i+1)j ′ represents a search range for moving the rectangular area of R _(i+1)j ″. When the reference A-scan image is A _ij ′ and the area for calculating the similarity is R _ij ″, the rectangular area of R _(i+1)j ″ is scanned and calculated within the search range S _(i+1)j ′. The unit 116 calculates the evaluation value.

The evaluation value is R _ij ″ and R _{(i+1) j} ″, and the calculation unit 116 calculates the correlation between the pixel values, and the determination unit 117 performs the evaluation. Alternatively, the layer thickness is detected by detecting the retinal layer boundary, and the calculation unit 116 calculates the evaluation value of the similarity using the layer thickness. FIG. 8B shows a case where the similarity is evaluated using the layer thickness. FIG. 8B shows the retinal layer boundaries (inner limiting membrane L ₁ and retinal pigment epithelium layer L ₂ ) within the search range S _(i+1)j ′ and the layer thicknesses 1 to 3 obtained from the retinal layer boundaries. There is. The layer thickness on each A-scan image is calculated in the rectangular region R _(i+1)j ″. Then, the evaluation value is calculated by the calculation unit 116 by comparing the layer thicknesses in R _ij ″ and R _(i+1)j ″. The layer used to calculate the layer thickness is not limited to the above, and the layer thickness may be compared using other layer boundaries such as nerve fiber layer and photoreceptor inner segment outer segment junction. ..

In the present embodiment, the process of the overlay determination by the determination unit 117 is omitted. However, after performing the alignment in units of A scan images, whether the overlay is performed based on the similarity as in the first embodiment. You may make it judge.

<Step S712>
In step S712, the generation unit 118 performs a process of overlapping the corresponding A-scan images. In step S711, the generation unit 118 performs a process of superimposing the A-scan image located at the center of the rectangular region R _(i+1)j ″ having the maximum evaluation value and the A-scan image A _ij ′ of the reference image. To do.

Note that in the present embodiment, R _ij ″ is set as a rectangular area within a range of +−α centered on the A scan image position. As this setting method, α is not a fixed value but is reflected in a two-dimensional tomographic image. 8(c) shows an example of region setting, for example, when the shape of the retinal layer is flat, α Is set wide (R _(i+1)j ″). If the two-dimensional tomographic image has a curved shape of the retinal layer and there are characteristic points (there are many vertical and horizontal edges), the range of α is set narrow (R _(i+1)k ″). The setting of the area range may be changed for each case from the image characteristics, or may be changed for each A-scan image in one two-dimensional tomographic image.

As is clear from the above description, in the present embodiment, in the aligned two-dimensional tomographic image, the A-scan image and the peripheral region are used to search for a region having a higher evaluation value in the neighboring region, and the overlapping region is searched. A matching process was performed. As a result, even if the position of the entire image is matched, even if the retinal layer in the two-dimensional tomographic image is deformed due to fixational eye movements, etc., the alignment processing is performed in local units, resulting in high image quality. It is possible to generate a three-dimensional tomographic image.

(Other embodiments)
Each of the above embodiments implements the present invention as an image processing apparatus. However, the embodiment of the present invention is not limited to the image processing apparatus. The present invention can also be realized as software that operates on a computer. The CPU of the image processing apparatus controls the entire computer using computer programs and data stored in the RAM and ROM. In addition, the execution of software corresponding to each unit of the image processing apparatus is controlled to realize the function of each unit.

110 image processing device 120 tomographic image capturing device 112 storage unit 113 image processing unit 114 display control unit 115 first alignment unit 116 calculation unit 117 determination unit 118 generation unit 619 second alignment unit

Claims (9)

Based on the layer boundaries included in each of the tomographic images of the fundus of the eye to be inspected obtained based on the measurement light of OCT (Optical Coherence Tomography) controlled to scan the same position multiple times, the plurality of tomographic images First alignment means for aligning images,
After the alignment by the first alignment means, the partial region which is a partial region of the tomographic image having a predetermined width in the direction orthogonal to the A-scan direction is used and the partial region for another tomographic image. of similarity acquired by performing positioning of the a-scan image which is narrower than the partial region with respect to the other tomographic image using the acquired result of the similarity of the partial region, with a-scan image unit Second alignment means for aligning;
Generating means for generating one tomographic image from the plurality of tomographic images after the alignment by the second aligning means,
An image processing apparatus comprising: The image processing apparatus according to claim 1, wherein the second alignment unit acquires the similarity based on a correlation of the partial region with the other tomographic image. The image processing apparatus according to claim 1, wherein a size of the partial region in a direction orthogonal to the A scan direction is variable. The size of the partial region in a direction orthogonal to the A scan direction is changed according to image characteristics of a retinal layer in the tomographic image. The image processing device described. Regarding the size of the partial region in the direction orthogonal to the A-scan direction, the characteristic portion in which the shape of the retinal layer in the tomographic image is curved more than other regions is larger than the other regions. The image processing apparatus according to any one of claims 1 to 4, wherein the size is set to be small. The partial region image processing apparatus according to any one of claims 1 to 5, characterized in that a region having a predetermined width around the A-scan image of the alignment target. An image processing apparatus according to any one of claims 1 to 6 ,
A tomographic imaging apparatus having control means for controlling the same position on the fundus so as to scan the measurement light of the OCT a plurality of times;
A system comprising: Based on a layer boundary included in each of a plurality of tomographic images of the fundus of the eye to be inspected, which is obtained based on measurement light of OCT (Optical Coherence Tomography) controlled to scan the same position a plurality of times by the processor, A first alignment step of aligning a plurality of tomographic images,
After the alignment in the first alignment step, the processor determines a partial region which is a partial region of the tomographic image having a predetermined width in a direction orthogonal to the A scan direction in each of the plurality of tomographic images. The similarity of the partial region with respect to another tomographic image is acquired by using the A-scan image having a width narrower than that of the partial region with respect to the other tomographic image , and the acquisition result of the similarity of the partial region is used. And a second alignment step for performing alignment in units of A scan images ,
A generation step of generating, by the processor, one tomographic image from the plurality of tomographic images after the alignment in the second alignment step;
An image processing method comprising: A program that causes a computer to execute the image processing method according to claim 8 .