JP2011087672A - Fundus image processor and fundus observation device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a fundus image processor and a fundus observation device, highly accurately performing comparative observation of a fundus. <P>SOLUTION: A positional deviation amount calculation unit 232 calculates a positional deviation amount between fundus photographed images H1 and H2 in a fundus surface direction on the basis of the fundus photographed images H1 and H2. A cross sectional position specifying unit 233 specifies a cross sectional position at the almost same positions on the fundus Ef represented by the respective fundus photographed images H1 and H2 on the basis of the positional deviation amount. On the basis of a first tomographic image at the cross-sectional position of a first three-dimensional image M1 corresponding to the cross-sectional position specified by the first fundus photographed image H1 and a second tomographic image at the cross-sectional position of a second three-dimensional image M2 corresponding to the cross-sectional position specified by the second fundus photographed image H2, an inclination deviation amount calculation unit 235 calculates an inclination deviation amount between the first tomographic image and the second tomographic image on a plane spread by a direction along the specified cross-sectional position and a fundus depth direction. Thus, the positional deviation amount and the inclination deviation amount between the first and second three-dimensional images M1 and M2 are obtained. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a fundus image processing apparatus that processes a fundus photographing image obtained by photographing the fundus, and a three-dimensional image of the fundus formed using optical coherence tomography (OCT), and The present invention relates to a fundus observation apparatus capable of forming a fundus photographed image and / or a three-dimensional image of the fundus.

  In recent years, OCT that forms an image representing the surface form or internal form of an object to be measured using a light beam from a laser light source or the like has attracted attention. Since OCT has no invasiveness to the human body like X-ray CT, it is expected to be applied particularly in the medical field and the biological field. For example, in the field of ophthalmology, an apparatus for forming an image of the fundus oculi, cornea, or the like has entered a practical stage.

  Patent Document 1 discloses an apparatus to which OCT is applied. In this device, the measuring arm scans an object with a rotary turning mirror (galvanomirror), a reference mirror is installed on the reference arm, and the intensity of the interference light of the light beam from the measuring arm and the reference arm is dispersed at the exit. An interferometer is provided for analysis by the instrument. Further, the reference arm is configured to change the phase of the reference light beam stepwise by a discontinuous value.

  The apparatus of Patent Document 1 uses a so-called “Fourier Domain OCT (Fourier Domain OCT)” technique. In other words, a low-coherence beam is irradiated onto the object to be measured, the reflected light and the reference light are superimposed to generate interference light, and the spectral intensity distribution of the interference light is acquired and subjected to Fourier transform. Thus, the form of the object to be measured in the depth direction (z direction) is imaged. Note that this type of technique is also called a spectral domain.

  Furthermore, the apparatus described in Patent Document 1 includes a galvanometer mirror that scans a light beam (signal light), thereby forming an image of a desired measurement target region of the object to be measured. Since this apparatus is configured to scan the light beam only in one direction (x direction) orthogonal to the z direction, the image formed by this apparatus is in the scanning direction (x direction) of the light beam. It becomes a two-dimensional tomogram in the depth direction (z direction) along.

  In Patent Document 2, a plurality of two-dimensional tomographic images in the horizontal direction are formed by scanning (scanning) the signal light in the horizontal direction (x direction) and the vertical direction (y direction), and based on the plurality of tomographic images. A technique for acquiring and imaging three-dimensional tomographic information of a measurement range is disclosed. Examples of the three-dimensional imaging include a method of displaying a plurality of tomographic images side by side in a vertical direction (referred to as stack data) and a method of rendering a plurality of tomographic images to form a three-dimensional image. Conceivable.

  Patent Documents 3 and 4 disclose other types of OCT apparatuses. Patent Document 3 scans the wavelength of light applied to an object to be measured, acquires a spectral intensity distribution based on interference light obtained by superimposing reflected light of each wavelength and reference light, On the other hand, an OCT apparatus for imaging the form of an object to be measured by performing Fourier transform on the object is described. Such an OCT apparatus is called a swept source type. The swept source type is a kind of Fourier domain type.

  In Patent Document 4, the traveling direction of light is obtained by irradiating the object to be measured with light having a predetermined beam diameter, and analyzing the component of interference light obtained by superimposing the reflected light and the reference light. An OCT apparatus for forming an image of an object to be measured in a cross-section orthogonal to is described. Such an OCT apparatus is called a full-field type or an en-face type.

  Patent Document 5 discloses a configuration in which OCT is applied to the ophthalmic field. Prior to the application of OCT, fundus cameras, slit lamps, and the like have been used as devices for observing the eye to be examined (see, for example, Patent Document 6 and Patent Document 7). A fundus camera is a device that shoots the fundus by illuminating the subject's eye with illumination light and receiving the fundus reflection light. A slit lamp is a device that acquires an image of a cross-section of the cornea by cutting off a light section of the cornea using slit light.

  An apparatus using OCT has an advantage over a fundus camera or the like in that a high-definition image can be acquired, and further, a tomographic image or a three-dimensional image can be acquired.

  As described above, an apparatus using OCT can be applied to observation of various parts of an eye to be examined, and can acquire high-definition images, and thus has been applied to diagnosis of various ophthalmic diseases.

JP 11-325849 A JP 2002-139421 A JP 2007-24677 A JP 2006-153838 A JP 2008-73099 A JP-A-9-276232 JP 2008-259544 A

  In various medical fields, for example, follow-up observations and pre- and post-operative observations, a diagnostic method that grasps changes over time in the state of an attention site (affected area, characteristic site, etc.) by comparing images acquired at different timings. (Referred to as “comparative observation”). In the ophthalmology field, for example, comparative observation is performed in the diagnosis of glaucoma and macular diseases. At that time, not only the state of the target region depicted in the image but also the physical quantity related to the target region may be quantitatively compared. Examples of physical quantities to be compared include the size of the affected area (radius, diameter, area, volume, etc.), the cup size of the optic nerve head, the disk size, and the rim size.

  When such a physical quantity is compared quantitatively, it is naturally important to obtain the physical quantity to be compared as accurately as possible. Further, in order to ensure the accuracy of comparison, it is necessary to acquire an image from which a physical quantity is calculated under the same conditions as much as possible.

  However, in OCT measurement in the ophthalmology field, conditions are inevitably different each time an image is acquired due to rotation of the eyeball, displacement of the fixation position, displacement of alignment, and the like. As a result, the accuracy of comparing physical quantities is reduced, and the operation of comparing the images themselves is difficult.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide a fundus image processing apparatus and a fundus oculi observation apparatus that can perform fundus comparative observation with high accuracy. .

In order to achieve the above object, the invention according to claim 1 includes a first fundus photographed image of the eye to be examined and a first three-dimensional image of the fundus of the eye to be examined, which are obtained at the first examination timing. Storage means for preliminarily storing a second fundus photographed image of the eye to be examined and a second three-dimensional image of the fundus acquired at a second examination timing different from the first examination timing; First calculating means for calculating a positional shift amount between the first fundus photographic image and the second fundus photographic image in the fundus surface direction based on the fundus photographic image and the second fundus photographic image; Designating means for designating a cross-sectional position at substantially the same position on the fundus described in each of the first fundus photographed image and the second fundus photographed image based on the calculated positional deviation amount; Corresponding to the cross-sectional position designated in the first fundus image. A first tomographic image at a cross-sectional position of the first three-dimensional image, and a second tomographic image at a cross-sectional position of the second three-dimensional image corresponding to the cross-sectional position designated by the second fundus image. Based on the above, a second amount of tilt deviation between the first tomographic image and the second tomographic image in a plane stretched by the direction along the designated cross-sectional position and the fundus depth direction is calculated. And a calculating means.
The invention according to claim 2 is the fundus image processing apparatus according to claim 1, wherein the specifying unit specifies a plurality of the cross-sectional positions, and the second calculating unit includes the plurality of cross-sectional positions. The cross-sectional positions of the first and second three-dimensional images corresponding to the cross-sectional positions are set to the designated cross-sectional positions based on the first tomographic image and the second tomographic image at the cross-sectional positions. The tilt displacement amount in a plane stretched between a direction along the depth direction and the fundus depth direction is calculated.
The invention according to claim 3 is the fundus image processing apparatus according to claim 2, wherein the designation means is a pair of linear cross-sectional positions intersecting at right angles as the plurality of cross-sectional positions. And the second calculation means calculates the amount of inclination deviation for the cross-sectional positions of the first and second three-dimensional images corresponding to the pair of cross-sectional positions, respectively. It is characterized in that the amount of tilt deviation in each of the planes is obtained.
The invention according to claim 4 is the fundus image processing apparatus according to claim 2, wherein the designation means has two or more straight lines that are arranged radially and intersect with each other as the plurality of cross-sectional positions. A second cross-sectional position is calculated, and the second calculating unit calculates the amount of inclination deviation for the cross-sectional positions of the first and second three-dimensional images corresponding to the two or more cross-sectional positions, The maximum value of the inclination deviation amounts corresponding to the two or more cross-sectional positions calculated by the second calculation means is selected, and the inclination deviation amount is set so as to cancel the selected inclination deviation amount. Correction means for correcting a deviation in inclination between the first three-dimensional image and the second three-dimensional image in a plane stretched by the direction along the corresponding cross-sectional position and the depth direction of the fundus. It is characterized by.
The invention according to claim 5 is the fundus image processing apparatus according to any one of claims 1 to 3, wherein the positional deviation amount calculated by the first calculation unit and the Alignment means for performing alignment between the first three-dimensional image and the second three-dimensional image so as to cancel out the tilt deviation amount calculated by the second calculation means; It is characterized by.
The invention according to claim 6 is the fundus image processing apparatus according to any one of claims 1 to 3, wherein the positional deviation amount calculated by the first calculation means and / or Based on the amount of tilt deviation calculated by the second calculation means, the first three-dimensional image is analyzed to calculate a first value of a predetermined physical quantity, and the second three-dimensional image And analyzing means for calculating the second value of the predetermined physical quantity.
The invention according to claim 7 is the fundus image processing apparatus according to any one of claims 1 to 6, wherein the first calculation means uses the fundus as the positional deviation amount. The parallel movement amount and the rotational movement amount in the surface direction are calculated.
The invention according to claim 8 is an imaging means for imaging the fundus of the subject's eye, and the low-coherence light is divided into signal light and reference light, and the signal light passing through the fundus and the reference light path are passed through. An optical system that generates and detects interference light by superimposing reference light, an image forming unit that forms a three-dimensional image of the fundus based on the detection result of the interference light, and the imaging at a first examination timing A first fundus photographed image of the eye to be examined photographed by means, a first three-dimensional image of the fundus formed by the image forming means, and a second examination timing different from the first examination timing. Storage means for storing the photographed second fundus photographed image of the subject eye and the formed second three-dimensional image of the fundus, the first fundus photographed image, and the second fundus photographed image Based on the fundus surface direction First calculation means for calculating a positional deviation amount between the first fundus photographic image and the second fundus photographic image, and the first fundus imaging based on the calculated positional deviation amount. Designating means for designating a cross-sectional position at substantially the same position on the fundus depicted in each of the image and the second fundus photographed image, and the first corresponding to the cross-sectional position designated in the first fundus photographed image A first tomographic image at a cross-sectional position of the first three-dimensional image, and a second tomographic image at a cross-sectional position of the second three-dimensional image corresponding to the cross-sectional position designated by the second fundus image. A second calculating unit configured to calculate an inclination shift amount between the first tomographic image and the second tomographic image on a plane stretched by the direction along the designated cross-sectional position and the fundus depth direction; And a fundus oculi observation device.
The invention according to claim 9 divides low-coherence light into signal light and reference light, and superimposes the signal light passing through the fundus of the eye to be examined and the reference light passing through the reference light path to interfere light. An optical system that generates and detects the image, an image forming unit that forms a three-dimensional image of the fundus oculi based on the detection result of the interference light, and a first fundus of the eye to be examined acquired at a first examination timing A photographed image and a second fundus photographed image of the eye to be examined acquired at a second examination timing different from the first examination timing are stored in advance, and at the first examination timing, by the image forming unit. Storage means for storing the formed first three-dimensional image of the fundus and the formed second three-dimensional image of the fundus at the second examination timing; the first fundus image; Based on the second fundus image A first calculating means for calculating a positional shift amount between the first fundus captured image and the second fundus captured image in the fundus surface direction, and based on the calculated positional shift amount, Designating means for designating a cross-sectional position at substantially the same position on the fundus depicted in each of the first fundus photographed image and the second fundus photographed image, and a cross-sectional position designated in the first fundus photographed image And a second tomographic image at a cross-sectional position of the second three-dimensional image corresponding to the cross-sectional position designated by the second fundus image. An inclination shift amount between the first tomographic image and the second tomographic image on a plane stretched by the direction along the designated cross-sectional position and the depth direction of the fundus is calculated based on the tomographic image of the first tomographic image. An eye comprising: a second calculating means; It is an observation device.
The invention described in claim 10 is different from the imaging means for imaging the fundus of the eye to be examined, the first three-dimensional image of the fundus acquired at the first examination timing, and the first examination timing. A second three-dimensional image of the fundus acquired at the second examination timing is stored in advance, and a first fundus photographed image of the eye to be examined photographed by the photographing means at the first examination timing Storage means for storing the second fundus photographed image of the subject eye photographed at the second examination timing, and the fundus based on the first fundus photographed image and the second fundus photographed image. First calculation means for calculating a positional deviation amount between the first fundus photographic image and the second fundus photographic image in the surface direction, and based on the calculated positional deviation amount, the first Fundus photographed image and second fundus photographed image Designating means for designating a cross-sectional position at substantially the same position on the fundus depicted in each of the first and three-dimensional images in the cross-sectional position of the first three-dimensional image corresponding to the cross-sectional position designated in the first fundus image Based on the first tomographic image and the second tomographic image at the sectional position of the second three-dimensional image corresponding to the sectional position designated in the second fundus photographing image, And a second calculating means for calculating an amount of inclination deviation between the first tomographic image and the second tomographic image on a plane stretched by the direction along the direction of the fundus depth. It is an observation device.

  According to the present invention, it is possible to obtain the positional deviation amount and the inclination deviation amount interposed between images to be compared in comparative observation of the fundus. Therefore, by performing comparative observation with reference to the obtained positional deviation amount and inclination deviation amount, it is possible to compare the images to be compared as those acquired under substantially the same conditions. This makes it possible to perform comparative observation of the fundus with high accuracy.

It is the schematic showing an example of composition of an embodiment of a fundus observation device concerning this invention. It is the schematic showing an example of composition of an embodiment of a fundus observation device concerning this invention. It is a schematic block diagram showing an example of composition of an embodiment of a fundus observation device concerning this invention. It is the schematic for demonstrating the state of the position shift between the two fundus photography images acquired in the embodiment of the fundus observation device concerning this invention. It is the schematic for demonstrating the cross-sectional position designated on the fundus image in the embodiment of the fundus oculi observation device according to the present invention. It is the schematic for demonstrating the tomogram formed from a three-dimensional image in embodiment of the fundus oculi observation device concerning this invention. It is the schematic for demonstrating the process which calculates the inclination shift amount in embodiment of the fundus oculi observation device concerning this invention. It is the schematic for demonstrating the process which calculates the inclination shift amount in embodiment of the fundus oculi observation device concerning this invention. It is the schematic showing an example of the position shift state between the 1st 3D image and the 2nd 3D image in embodiment of the fundus oculi observation device concerning this invention. It is the schematic showing an example of the result of the alignment process between the 1st three-dimensional image and the 2nd three-dimensional image in embodiment of the fundus oculi observation device concerning this invention. It is the schematic for demonstrating an example of the image analysis process in embodiment of the fundus oculi observation device concerning this invention. It is the schematic for demonstrating the modification of embodiment of the fundus observation apparatus concerning this invention.

  An example of an embodiment of a fundus image processing apparatus and a fundus observation apparatus according to the present invention will be described in detail with reference to the drawings. This fundus oculi observation device is equipped with this fundus image processing device. That is, this fundus oculi observation device acquires part or all of an image to be subjected to processing by the fundus image processing device. Hereinafter, such a fundus oculi observation device will be described in detail.

  The fundus oculi observation device according to the present invention forms a tomographic image or a three-dimensional image of the fundus using OCT. In this specification, images acquired by OCT may be collectively referred to as OCT images. In addition, a measurement operation for forming an OCT image may be referred to as OCT measurement.

  In the following embodiment, a configuration to which Fourier domain type OCT is applied will be described in detail. In particular, in this embodiment, a fundus oculi observation device capable of acquiring both a fundus OCT image and a fundus photographed image is taken up as in the device disclosed in Patent Document 5.

  Note that the fundus oculi observation device according to the present invention may be configured to acquire only one of an OCT image and a fundus photographic image. That is, the present invention also includes the following two types of devices: (1) A fundus observation device (OCT device) that can acquire only an OCT image by itself, and other devices (fundus camera, slit lamp microscope) (Slit lamp), configured to receive a fundus photographic image acquired by a scanning laser ophthalmoscope (SLO (Scanning Laser Ophthalmoscope), etc.) and memorize it; (2) Only the fundus photographic image itself A fundus observation device (fundus camera, slit lamp microscope, scanning laser ophthalmoscope, etc.) that can be acquired, and configured to receive an OCT image acquired by another device (OCT device) and store it. Things.

[Constitution]
As shown in FIGS. 1 and 2, the fundus oculi observation device 1 includes a fundus camera unit 2, an OCT unit 100, and an arithmetic control unit 200. The retinal camera unit 2 has almost the same optical system as a conventional retinal camera. The OCT unit 100 is provided with an optical system for acquiring an OCT image of the fundus. The arithmetic control unit 200 includes a computer that executes various arithmetic processes and control processes. The arithmetic control unit 200 is an example of the “fundus image processing apparatus” of the present invention.

[Fundus camera unit]
The fundus camera unit 2 shown in FIG. 1 is provided with an optical system for acquiring a two-dimensional image (fundus photographed image) representing the surface form of the fundus oculi Ef of the eye E to be examined. The fundus photographed image includes an observation image and a photographed image. The observation image is, for example, a monochrome moving image formed at a predetermined frame rate using near infrared light. The captured image is a color image obtained by flashing visible light, for example. The fundus camera unit 2 may be configured to be able to acquire images other than these, such as a fluorescein fluorescent image, an indocyanine green fluorescent image, a spontaneous fluorescent image, and the like. Note that the fundus photographed image used in the present invention is mainly a photographed image. The fundus camera unit 2 is an example of the “photographing means” of the present invention.

  Similar to the conventional fundus camera, the fundus camera unit 2 is provided with a chin rest and a forehead for supporting the subject's face so as not to move. Further, the fundus camera unit 2 is provided with an illumination optical system 10 and a photographing optical system 30 as in the conventional fundus camera. The illumination optical system 10 irradiates the fundus oculi Ef with illumination light. The photographing optical system 30 guides the fundus reflection light of the illumination light to the imaging device (CCD image sensors 35 and 38). The imaging optical system 30 guides the signal light from the OCT unit 100 to the fundus oculi Ef and guides the signal light passing through the fundus oculi Ef to the OCT unit 100.

  The observation light source 11 of the illumination optical system 10 is constituted by a halogen lamp, for example. The light (observation illumination light) output from the observation light source 11 is reflected by the reflection mirror 12 having a curved reflection surface, passes through the condensing lens 13, passes through the visible cut filter 14, and is converted into near infrared light. Become. Further, the observation illumination light is once converged in the vicinity of the photographing light source 15, reflected by the mirror 16, and passes through the relay lenses 17 and 18, the diaphragm 19 and the relay lens 20. Then, the observation illumination light is reflected by the peripheral part (region around the hole part) of the perforated mirror 21 and illuminates the fundus oculi Ef via the objective lens 22.

  The fundus reflection light of the observation illumination light is refracted by the objective lens 22, passes through a hole formed in the central region of the aperture mirror 21, passes through the dichroic mirror 55, passes through the focusing lens 31, and then goes through the dichroic mirror. 32 is reflected. Further, the fundus reflection light passes through the half mirror 40, is reflected by the dichroic mirror 33, and forms an image on the light receiving surface of the CCD image sensor 35 by the condenser lens 34. The CCD image sensor 35 detects fundus reflected light at a predetermined frame rate, for example. The display device 3 displays an image (observation image) K based on fundus reflected light detected by the CCD image sensor 35.

  The imaging light source 15 is constituted by, for example, a xenon lamp. The light (imaging illumination light) output from the imaging light source 15 is applied to the fundus oculi Ef through the same path as the observation illumination light. The fundus reflection light of the imaging illumination light is guided to the dichroic mirror 33 through the same path as that of the observation illumination light, passes through the dichroic mirror 33, is reflected by the mirror 36, and is reflected by the condenser lens 37 of the CCD image sensor 38. An image is formed on the light receiving surface. On the display device 3, an image (captured image) H based on fundus reflection light detected by the CCD image sensor 38 is displayed. The display device 3 that displays the observation image K and the display device 3 that displays the captured image H may be the same or different.

  An LCD (Liquid Crystal Display) 39 displays a fixation target and an eyesight measurement target. The fixation target is a target for fixing the eye E to be examined, and is used at the time of fundus photographing or OCT measurement.

  A part of the light output from the LCD 39 is reflected by the half mirror 40, reflected by the dichroic mirror 32, passes through the focusing lens 31 and the dichroic mirror 55, and passes through the hole of the perforated mirror 21. The light is refracted by the objective lens 22 and projected onto the fundus oculi Ef.

  By changing the display position of the fixation target on the screen of the LCD 39, the fixation position of the eye E can be changed. As the fixation position of the eye E, for example, a position for acquiring an image centered on the macular portion of the fundus oculi Ef, or a position for acquiring an image centered on the optic disc as in the case of a conventional fundus camera And a position for acquiring an image centered on the fundus center between the macula and the optic disc.

  Further, the fundus camera unit 2 is provided with an alignment optical system 50 and a focus optical system 60 as in the conventional fundus camera. The alignment optical system 50 generates a visual target (alignment visual target) for performing alignment (alignment) of the apparatus optical system with respect to the eye E. The focus optical system 60 generates a visual target (split visual target) for focusing on the fundus oculi Ef.

  Light (alignment light) output from an LED (Light Emitting Diode) 51 of the alignment optical system 50 is reflected by the dichroic mirror 55 via the apertures 52 and 53 and the relay lens 54, and passes through the hole portion of the perforated mirror 21. It passes through and is projected onto the cornea of the eye E by the objective lens 22.

  The corneal reflection light of the alignment light passes through the objective lens 22 and the hole, and a part thereof passes through the dichroic mirror 55, passes through the focusing lens 31, is reflected by the dichroic mirror 32, and passes through the half mirror 40. Then, it is reflected by the dichroic mirror 33 and projected onto the light receiving surface of the CCD image sensor 35 by the condenser lens 34. A light reception image (alignment target) by the CCD image sensor 35 is displayed on the display device 3 together with the observation image K. The user performs alignment by performing the same operation as that of a conventional fundus camera. Further, the arithmetic control unit 200 may perform alignment by analyzing the position of the alignment target and moving the optical system.

  When performing the focus adjustment, the reflecting surface of the reflecting rod 67 is obliquely provided on the optical path of the illumination optical system 10. The light (focus light) output from the LED 61 of the focus optical system 60 passes through the relay lens 62, is separated into two light beams by the split target plate 63, passes through the two-hole aperture 64, and is reflected by the mirror 65. The light is once focused on the reflecting surface of the reflecting bar 67 by the condenser lens 66 and reflected. Further, the focus light passes through the relay lens 20, is reflected by the perforated mirror 21, and forms an image on the fundus oculi Ef by the objective lens 22.

  The fundus reflection light of the focus light is detected by the CCD image sensor 35 through the same path as the cornea reflection light of the alignment light. A light reception image (split target) by the CCD image sensor 35 is displayed on the display device 3 together with the observation image. The arithmetic and control unit 200 analyzes the position of the split target and moves the focusing lens 31 and the focus optical system 60 to focus, as in the conventional case. Alternatively, focusing may be performed manually while visually checking the split target.

  An optical path including a mirror 41, a collimator lens 42, and galvanometer mirrors 43 and 44 is provided behind the dichroic mirror 32. This optical path is guided to the OCT unit 100.

  The galvanometer mirror 44 scans the signal light LS from the OCT unit 100 in the x direction. The galvanometer mirror 43 scans the signal light LS in the y direction. By these two galvanometer mirrors 43 and 44, the signal light LS can be scanned in an arbitrary direction on the xy plane.

[OCT unit]
The OCT unit 100 is provided with an optical system for acquiring an OCT image of the fundus oculi Ef (see FIG. 2). This optical system has the same configuration as a conventional Fourier domain type OCT apparatus. That is, this optical system divides low-coherence light into reference light and signal light, and generates interference light by causing interference between the signal light passing through the fundus oculi Ef and the reference light passing through the reference optical path. It is configured to detect spectral components. This detection result (detection signal) is sent to the arithmetic control unit 200.

  The light source unit 101 outputs a broadband low-coherence light L0. The low coherence light L0 includes, for example, a near-infrared wavelength band (about 800 nm to 900 nm) and has a temporal coherence length of about several tens of micrometers. Note that near-infrared light having a wavelength band that cannot be visually recognized by the human eye, for example, a center wavelength of about 1050 to 1060 nm, may be used as the low-coherence light L0.

  The light source unit 101 includes a light output device such as a super luminescent diode (SLD), an LED, or an SOA (Semiconductor Optical Amplifier).

  The low-coherence light L0 output from the light source unit 101 is guided to the fiber coupler 103 by the optical fiber 102 and split into the signal light LS and the reference light LR. The fiber coupler 103 functions as both a means for splitting light (splitter) and a means for combining light (coupler), but here it is conventionally referred to as a “fiber coupler”.

  The signal light LS is guided by the optical fiber 104 and becomes a parallel light beam by the collimator lens unit 105. Further, the signal light LS is reflected by the respective galvanometer mirrors 44 and 43, collected by the collimator lens 42, reflected by the mirror 41, transmitted through the dichroic mirror 32, and through the same path as the light from the LCD 39, the fundus oculi Ef. Is irradiated. The signal light LS is scattered and reflected on the fundus oculi Ef. The scattered light and reflected light may be collectively referred to as fundus reflected light of the signal light LS. The fundus reflection light of the signal light LS travels in the opposite direction on the same path and is guided to the fiber coupler 103.

  The reference light LR is guided by the optical fiber 106 and becomes a parallel light beam by the collimator lens unit 107. Further, the reference light LR is reflected by the mirrors 108, 109, 110, is attenuated by the ND (Neutral Density) filter 111, is reflected by the mirror 112, and forms an image on the reflection surface of the reference mirror 114 by the collimator lens 113. . The reference light LR reflected by the reference mirror 114 travels in the opposite direction on the same path and is guided to the fiber coupler 103. An optical element for dispersion compensation (such as a pair prism) and an optical element for polarization correction (such as a wavelength plate) may be provided in the optical path (reference optical path) of the reference light LR.

  The fiber coupler 103 combines the fundus reflection light of the signal light LS and the reference light LR reflected by the reference mirror 114. The interference light LC thus generated is guided by the optical fiber 115 and emitted from the emission end 116. Further, the interference light LC is converted into a parallel light beam by the collimator lens 117, dispersed (spectral decomposition) by the diffraction grating 118, condensed by the condenser lens 119, and projected onto the light receiving surface of the CCD image sensor 120. The diffraction grating 118 shown in FIG. 2 is a transmission type, but a reflection type diffraction grating may be used.

  The CCD image sensor 120 is, for example, a line sensor, and detects each spectral component of the split interference light LC and converts it into electric charges. The CCD image sensor 120 accumulates this electric charge and generates a detection signal. Further, the CCD image sensor 120 sends this detection signal to the arithmetic control unit 200.

  In this embodiment, a Michelson type interferometer is employed, but any type of interferometer such as a Mach-Zehnder type can be appropriately employed. Further, in place of the CCD image sensor, another form of image sensor such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor can be used.

[Calculation control unit]
The configuration of the arithmetic control unit 200 will be described. The arithmetic control unit 200 analyzes the detection signal input from the CCD image sensor 120 and forms an OCT image of the fundus oculi Ef. The arithmetic processing for this is the same as that of a conventional Fourier domain type OCT apparatus.

  The arithmetic control unit 200 controls each part of the fundus camera unit 2, the display device 3, and the OCT unit 100. For example, the arithmetic and control unit 200 displays an OCT image such as a tomographic image G (see FIG. 2) of the fundus oculi Ef on the display device 3.

  As the control of the fundus camera unit 2, the arithmetic control unit 200 controls the operation of the observation light source 11, the imaging light source 15 and the LEDs 51 and 61, the operation control of the LCD 39, the movement control of the focusing lens 31, and the movement control of the reflector 67. Further, movement control of the focus optical system 60, operation control of the galvanometer mirrors 43 and 44, and the like are performed.

  As control of the OCT unit 100, the arithmetic control unit 200 performs operation control of the light source unit 101, movement control of the reference mirror 114 and collimator lens 113, operation control of the CCD image sensor 120, and the like.

  The arithmetic control unit 200 includes, for example, a microprocessor, a RAM, a ROM, a hard disk drive, a communication interface, etc., as in a conventional computer. A computer program for controlling the fundus oculi observation device 1 is stored in a storage device such as a hard disk drive. The arithmetic control unit 200 may include a dedicated circuit board that forms an OCT image based on a detection signal from the CCD image sensor 120. The arithmetic control unit 200 may include an operation device (input device) such as a keyboard and a mouse, and a display device such as an LCD.

  The retinal camera unit 2, the display device 3, the OCT unit 100, and the arithmetic control unit 200 may be configured integrally (that is, in a single casing) or may be configured separately.

[Control system]
The configuration of the control system of the fundus oculi observation device 1 will be described with reference to FIG.

(Control part)
The control system of the fundus oculi observation device 1 is configured around the control unit 210 of the arithmetic control unit 200. The control unit 210 includes, for example, the aforementioned microprocessor, RAM, ROM, hard disk drive, communication interface, and the like. The control unit 210 is provided with a main control unit 211 and a storage unit 212.

(Main control unit)
The main control unit 211 performs the various controls described above. In particular, the main control unit 211 controls the scanning drive unit 70 and the focusing drive unit 80 of the fundus camera unit 2, and further the light source unit 101 and the reference drive unit 130 of the OCT unit 100.

  The scanning drive unit 70 includes, for example, a servo motor, and independently changes the directions of the galvanometer mirrors 43 and 44. The focusing drive unit 80 includes, for example, a pulse motor, and moves the focusing lens 31 in the optical axis direction. Thereby, the focus position of the light toward the fundus oculi Ef is changed. The reference driving unit 130 includes, for example, a pulse motor, and moves the collimator lens 113 and the reference mirror 114 integrally along the traveling direction of the reference light LR.

  Further, the main control unit 211 performs processing for writing data into the storage unit 212 and processing for reading data from the storage unit 212.

(Memory part)
The storage unit 212 stores various data. The data stored in the storage unit 212 includes, for example, image data of an OCT image, image data of a fundus image, and eye information to be examined. The eye information includes information about the subject such as patient ID and name, and information about the eye such as left / right eye identification information.

  In particular, the storage unit 212 is used as an example of the “storage unit” of the present invention, and stores an image used for processing according to this embodiment. As this image, there are a fundus image and a three-dimensional image of the fundus. Although details will be described later, the storage unit 212 stores the first fundus photographic image H1 and the first three-dimensional image M1 acquired at the first inspection timing, and acquires them at the second inspection timing. The second fundus photographed image H2 and the second three-dimensional image M2 are stored. The respective fundus images H1 and H2 are acquired by fundus imaging, and the three-dimensional images M1 and M2 are acquired by OCT measurement.

  Here, “timing” means the date and time of the inspection performed in a series of flows. The “date and time” may be the same day and the same time, the same day and a non-same time, a non-same day and the same time, a non-same day and a non-same time. That is, fundus imaging and OCT measurement according to the same timing are acquired by an examination performed as a series of flows including any one of these four options. In this embodiment, since the fundus oculi observation device 1 capable of performing both fundus imaging and OCT measurement is considered, a case where fundus imaging and a three-dimensional image are acquired on the same day and at approximately the same time will be described.

  When two separate devices (for example, a fundus camera and an OCT device) are used, these images are acquired on the same day and at the same time, at the same day and at the same time, or at the same day and at the same time. There is also a possibility. For example, it is assumed that the fundus is photographed in a medical institution that does not have an OCT apparatus and then moved to a medical institution having the fundus and OCT measurement is performed.

  However, it is desirable to perform fundus imaging and OCT measurement performed at the same timing within a predetermined period. This predetermined period is determined based on, for example, a disease progression rate or a cure rate. In other words, since the comparative observation is to compare images and analysis results between timings with different degrees of disease and healing, the fundus photographic image and the three-dimensional image at each timing have almost the same degree of disease and healing. There is a need. Then, the fundus photographed image and the three-dimensional image related to the same timing must be acquired in a state where the degree of disease and healing is almost the same. Therefore, the “predetermined period” is determined so as to secure this “substantially the same state”. In general, fundus imaging and OCT measurement are performed on the same day.

  In addition, it is desirable that the first examination timing and the second examination timing have a period in which the degree of disease or cure is different. That is, the comparative observation is performed for the purpose of grasping the progress of the disease and healing.

  The first fundus photographed image H1 and the first three-dimensional image M1 are stored in the storage unit 212 when they are acquired, for example, at the first examination timing. Similarly, the second fundus photographed image H2 and the second three-dimensional image M2 are stored in the storage unit 212 when they are acquired, for example, at the second examination timing. Note that instead of storing the fundus photographic image or the like immediately after being acquired, it is also possible to temporarily store them in another storage medium and later read them out and store them in the storage unit 212. Examples of the other storage medium include a portable storage medium such as a DVD-R, and a storage medium installed on a communication line such as NAS (Network Attached Storage).

(Image forming part)
The image forming unit 220 forms tomographic image data of the fundus oculi Ef based on the detection signal from the CCD image sensor 120. This process includes processes such as noise removal (noise reduction), filter processing, and FFT (Fast Fourier Transform) as in the conventional Fourier domain type optical coherence tomography.

  The image forming unit 220 includes, for example, the above-described circuit board and communication interface. In this specification, “image data” and “image” presented based on the “image data” may be identified with each other.

(Image processing unit)
The image processing unit 230 performs various types of image processing and analysis processing on the image formed by the image forming unit 220. For example, the image processing unit 230 executes various correction processes such as image brightness correction and dispersion correction.

(3D image forming unit)
The three-dimensional image forming unit 231 of the image processing unit 230 executes known image processing such as interpolation processing for interpolating pixels between tomographic images formed by the image forming unit 220, and performs a three-dimensional image of the fundus oculi Ef. Form image data. The image forming unit 220 and the three-dimensional image forming unit 231 are examples of the “image forming unit” of the present invention.

  Note that the image data of a three-dimensional image means image data in which pixel positions are defined by a three-dimensional coordinate system. As image data of a three-dimensional image, there is image data composed of voxels arranged three-dimensionally. This image data is called volume data or voxel data. When displaying an image based on volume data, the image processing unit 230 performs rendering processing (volume rendering, MIP (Maximum Intensity Projection), etc.) on the volume data, and views the image from a specific gaze direction. Image data of a pseudo three-dimensional image is formed. This pseudo three-dimensional image is displayed on a display device such as the display unit 240.

  It is also possible to form stack data of a plurality of tomographic images as image data of a three-dimensional image. The stack data is image data obtained by three-dimensionally arranging a plurality of tomographic images obtained along a plurality of scanning lines based on the positional relationship of the scanning lines. That is, stack data is image data obtained by expressing a plurality of tomographic images originally defined by individual two-dimensional coordinate systems by one three-dimensional coordinate system (that is, by embedding them in one three-dimensional space). is there.

  The image processing unit 230 is further provided with a positional deviation amount calculation unit 232, a cross-sectional position designation unit 233, an inclination deviation amount calculation unit 235, an alignment processing unit 236, an inclination deviation correction unit 237, and an image analysis unit 238. . Hereinafter, each of these parts 232 to 238 will be described.

(Position displacement calculation unit)
The positional deviation amount calculation unit 232 is based on the first fundus photographic image H1 and the second fundus photographic image H2, and between the first fundus photographic image H1 and the second fundus photographic image H2 in the fundus surface direction. Calculate the amount of displacement. The positional deviation amount calculation unit 232 is an example of the “first calculation means” in the present invention. This positional deviation amount is a vector amount.

  Here, “fundus surface direction” means a direction on the xy plane shown in FIG. 1, that is, a direction defined by the x coordinate and the y coordinate. In other words, the fundus surface direction is a direction orthogonal to the optical axis direction (z direction) of the apparatus optical system in a state where the apparatus optical system is aligned with the fundus oculi Ef. The optical axis direction at this time is the traveling direction of the signal light LS toward the fundus oculi Ef. In this specification, this optical axis direction is referred to as “fundus depth direction”.

  The process executed by the positional deviation amount calculation unit 232 will be described more specifically. Since the first fundus photographic image H1 and the second fundus photographic image H2 are obtained by photographing the same fundus oculi Ef at different timings, the eye E rotates, the fixation position shift, and the alignment shift. The position in the frame varies depending on the difference in conditions. For example, when the rotation state of the eye E to be examined is different, the first fundus photographed image H1 and the second fundus photographed image H2 are displaced in the rotational direction around the rotation point in the fundus surface direction. . In addition, when there is a shift in the fixation position or alignment, a displacement due to translation in the fundus surface direction is interposed between the first fundus photographed image H1 and the second fundus photographed image H2.

  In order to calculate such a positional deviation amount (parallel movement amount, rotational movement amount), the positional deviation amount calculation unit 232 first calculates the characteristic portion of the fundus oculi Ef depicted in each fundus oculi image H1, H2. Detect the position in the image. Examples of the characteristic part include an affected part, an optic disc, a macula, a blood vessel, and a branch point of the blood vessel. The feature part in the image is specified by, for example, analyzing the pixel value of each pixel of the image and specifying a pixel having a pixel value corresponding to the feature part to be detected. As an example, the optic disc is generally brighter than other parts of the fundus image (ie, with high brightness), so an image area corresponding to the optic disc is identified by searching for pixels with a brightness greater than or equal to a predetermined value. it can. On the contrary, the affected part and the macula are generally drawn darker than other parts, and therefore, it is only necessary to search for a pixel having a luminance of a predetermined value or less. Further, for blood vessels and their branch points, pixel values may be analyzed by known image processing to identify image regions corresponding to blood vessels (at this time, thinning processing or the like may be performed as necessary).

  The positional deviation amount calculation unit 232 calculates the displacement in the frame of the image area in each of the fundus photographic images H1 and H2 obtained in this manner, thereby obtaining the first fundus photographic image H1 and the second fundus photographic image. The amount of positional deviation (particularly the amount of translation) from H2 is calculated. In addition, when considering a characteristic part having an asymmetric shape such as a blood vessel, the amount of rotational movement can be easily calculated. In addition, in order to calculate the positional deviation amount (particularly, the rotational movement amount) with high accuracy and high accuracy, it is desirable to consider two or more characteristic parts.

  In addition, when considering a feature part whose shape hardly changes over time (for example, other than the affected part), an affine transformation is obtained so that the positions of the image regions of both feature parts most closely match, and the parallel movement in this affine transformation is obtained. It is possible to set the component as the target parallel movement amount and the rotational movement component as the target rotational movement amount. Note that it is also possible to obtain a positional deviation amount between the two fundus photographic images H1 and H2 by a method other than the above.

  An example of a positional shift state between the fundus photographed images H1 and H2 is shown in FIG. 4A shows the first fundus photographic image H1, and FIG. 4B shows the second fundus photographic image H2. FIG. 4C shows a state in which the first fundus photographic image H1 (solid line) and the second fundus photographic image H2 (broken line) are superimposed in order to show the positional relationship between both fundus photographic images H1 and H2. To express.

  The xy coordinate system shown in FIG. 4C corresponds to the xy coordinate system included in the xyz coordinate system shown in FIG. Further, “Δ” represents a positional shift amount of the second fundus photographic image H2 with respect to the first fundus photographic image H1. That is, Δx represents the parallel movement amount in the x direction, Δy represents the parallel movement amount in the y direction, and Δθ represents the rotational movement amount.

(Cross section position designation part)
The cross-section position specifying unit 233 is substantially on the fundus oculi Ef depicted in each of the first fundus photographic image H1 and the second fundus photographic image H2 based on the position shift amount calculated by the position shift amount calculation unit 232. Specify the cross-sectional position at the same position. The cross-section position designation unit 233 is an example of the “designation unit” of the present invention.

  A specific example of processing executed by the cross-section position specifying unit 233 will be described. Since the positional deviation amount between the two fundus photographic images H1 and H2 is obtained by the positional deviation amount calculation unit 232, the fundus photographic images H1 and H2 can be aligned in the fundus surface direction. This alignment can be performed by relatively moving the two fundus photographic images H1 and H2 so as to cancel out the obtained positional deviation amount. Instead of moving the image, association may be performed in consideration of the positional deviation amount between the pixel in the fundus photographic image H1 and the pixel in the fundus photographic image H2. In any case, the position in the fundus photographic image H1 and the position in the fundus photographic image H2 can be associated with each other by taking the positional deviation amount into consideration. Here, the amount of positional deviation is obtained as the positional deviation of the image area corresponding to the characteristic part of the fundus oculi Ef. Therefore, this alignment associates the position of the part in the first fundus photographic image H1 with the position of the part in the second fundus photographic image H2 for each part of the fundus oculi Ef.

  The cross-sectional position designating unit 233 sets the cross-sectional position at substantially the same position on the fundus oculi Ef depicted in each of the first fundus photographic image H1 and the second fundus photographic image H2 associated by such alignment. specify.

  The cross-sectional position specified by the cross-section position specifying unit 233 will be described. First, the number of designated cross-sectional positions may be one or plural. When a plurality of cross-sectional positions are designated, the subsequent processing is executed for each cross-sectional position. Examples of the plurality of cross-sectional positions include a cross-shaped cross-sectional position and a radial cross-sectional position.

  The cross-sectional position of the cross shape is composed of a pair of linear cross-sectional positions orthogonal to each other and intersecting each other. An example of the cross-sectional position of the cross shape is shown in FIG. The first cross-sectional position U1 shown in FIG. 5A is designated for the first fundus photographic image H1, and the cross-sectional position U2 shown in FIG. 5B is designated for the second fundus photographic image H2. Is done. FIG. 5C shows the first fundus photographic image H1 and the cross-sectional position U1 (solid line), the second fundus photographic image H2 and the cross-sectional position U2 (in order to show the positional relationship between the two cross-sectional positions U1 and U2. This represents a state in which a broken line) is overlaid. Both cross-sectional positions U1 and U2 are designated at substantially the same position on the fundus oculi Ef. The cross-shaped cross-sectional positions U1 and U2 have the same form as a scanning line for cross scanning described later.

  The radial cross-sectional position is composed of two or more linear cross-sectional positions arranged radially and intersecting each other (not shown). When the two cross-sectional positions are arranged so as to be orthogonal to each other in the radial cross-sectional position, the above-described cross-shaped cross-sectional position is obtained. The radial cross-sectional position has the same form as the scanning line of the radiation scan described later.

  The cross-sectional position specified by the cross-section position specifying unit 233 is not limited to the above example. For example, it is possible to designate a cross-sectional position in an arbitrary form such as a straight line shape, a circular shape, a concentric circle shape, and a spiral shape.

  In addition, it is desirable to specify the cross-sectional position so as to pass through a part of the fundus oculi Ef having a characteristic shape in the fundus depth direction (z direction). As a part having this characteristic shape, there is a part (uneven part) having irregularities in the fundus depth direction. Examples of such irregularities include the optic nerve head and the macula, and also the affected area with irregularities.

  In order to designate a cross-sectional position that passes through such an uneven part, the cross-sectional position designating unit 233 executes, for example, the following processing. When the image region specified by the positional deviation amount calculation unit 232 corresponds to the uneven portion, the cross-sectional position specifying unit 233 specifies the cross-sectional position so as to pass through the image region. The size (length, etc.) of the cross-sectional position may be set in advance, or may be set according to the size of the image area. On the other hand, when the image area specified by the positional deviation amount calculation unit 232 corresponds to a part other than the uneven part, the cross-sectional position specifying part 233 can be configured to specify the image area corresponding to the uneven part. This identification process is the same as the process performed by the positional deviation amount calculation unit 232.

  In this way, instead of specifying the uneven part from the fundus image, it is also possible to analyze the three-dimensional image and specify the uneven part. As an example, first, the cross-section position specifying unit 233 generates a two-dimensional image (integrated image) by integrating the three-dimensional images M1 and M2 in the fundus depth direction. In this process, pixel values (luminance values) of a plurality of pixels included in each depth direction image (A scan image) constituting each of the three-dimensional images M1 and M2 are added in the fundus depth direction. The dot-like image obtained by integrating the A-scan images has a luminance value obtained by adding the luminance value at each z position of the A-scan image in the depth direction (exceeds the maximum gradation value of the luminance value). If so, perform appropriate image processing). The accumulated image is described in detail in Japanese Patent Application No. 2005-337628 by the present applicant.

  The integrated image is an image depicting the shape of the surface of the fundus as if it were a fundus photographed image, but a high image quality such as a fundus photographed image cannot be obtained. The cross-sectional position specifying unit 233 specifies an image region corresponding to a characteristic part of the fundus oculi Ef in the three-dimensional image M1 based on the pixel value of the accumulated image based on the first three-dimensional image M1, and performs first fundus photographing. An image region corresponding to the characteristic part is specified in the image H1. Then, the cross-section position specifying unit 233 performs alignment between these characteristic portions in the same manner as the positional deviation amount calculation unit 232.

  Further, the cross-sectional position specifying unit 233 analyzes the first three-dimensional image M1 and specifies an image region corresponding to a predetermined uneven portion. Since the first three-dimensional image M1 depicts the three-dimensional form of the fundus oculi Ef, for example, by searching for unevenness in the z direction in the image region corresponding to the fundus surface, the uneven portion can be easily identified. . Subsequently, the cross-sectional position designating unit 233 specifies an image region in the first fundus photographic image H1 corresponding to the image region of the uneven portion based on the above-described alignment result of the characteristic portion, and this image region is determined. Specify the cross-sectional position to pass. The process of designating the cross-sectional position for the second fundus photographic image H2 can be executed in the same manner.

  In addition, it is also possible to perform the process of designating the cross-sectional position for each fundus photographed image H1, H2 based on each three-dimensional image M1, M2 without interposing an integrated image. For example, the cross-section position specifying unit 233 first specifies an image region corresponding to a characteristic part of the fundus oculi Ef in each of the first fundus photographic image H1 and the first three-dimensional image M1, and between these image regions. The amount of positional deviation in the xy direction is obtained. Since both images are defined using the x-coordinate and the y-coordinate, this process can be easily executed in the same manner as the positional deviation amount calculation unit 232. Next, the cross-section position specifying unit 233 specifies an image region corresponding to a predetermined uneven portion in the first three-dimensional image M1. Further, the cross-section position specifying unit 233 specifies an image area in the first fundus photographic image H1 corresponding to the image area of the uneven portion based on the position shift amount, and cross-sections so as to pass through the image area. Specify the position. The process of designating the cross-sectional position for the second fundus photographic image H2 can be executed in the same manner.

  Further, instead of the image region corresponding to the uneven portion as described above, a cross-sectional position passing through the image region corresponding to an arbitrary portion of the fundus oculi Ef can be set. Although details will be described later, the designated cross-sectional position is used to determine the amount of tilt deviation in the fundus depth direction. Therefore, if the cross-sectional position is specified so that the surface of the fundus oculi Ef or the deep layer of the fundus oculi Ef is clearly depicted in the tomographic image, the relative inclination of the surface or layer can be obtained. That is, on the condition that such a tomographic image is obtained, it is possible to designate a cross-sectional position that passes through an image region corresponding to an arbitrary part of the fundus oculi Ef.

  Note that the process of designating the cross-sectional position on each fundus photographed image H1, H2 is not limited to that described above. Any processing form can be applied as long as the cross-sectional position can be appropriately designated on each fundus photographic image H1, H2.

  The image processing unit 230 stores position information (cross-section position information) of the cross-sectional position designated on each fundus oculi image H1, H2. This cross-sectional position information is, for example, the coordinate value of the xy coordinate system in each fundus photographed image H1, H2. Since the optical axis (imaging optical axis) of the imaging optical system 30 is aligned with the eye E during fundus imaging, the frame centers of the respective fundus images H1 and H2 substantially coincide with the imaging optical axis. Considering this, it is also possible to store the displacement of the cross-sectional position with respect to the photographing optical axis as cross-sectional position information.

  The image processing unit 230 forms a tomographic image at the cross-sectional position specified by the cross-sectional position specifying unit 233 based on the three-dimensional images M1 and M2. Instead of actually forming the tomographic image, it is also possible to execute the subsequent processing with reference to the pixels at the cross-sectional positions of the three-dimensional images M1 and M2 corresponding to the designated cross-sectional positions.

  As an example of processing for forming a tomographic image, the image processing unit 230 first performs alignment between the first fundus photographic image H1 and the first three-dimensional image M1. This process can be performed, for example, through the above-described integrated image, or can be performed by aligning the image region of the characteristic part.

  Next, the image processing unit 230 specifies a position in the first three-dimensional image M1 corresponding to the cross-sectional position designated in the first fundus photographic image H1 based on the result of the above alignment. Then, the image processing unit 230 forms a tomographic image (first tomographic image) along the specific position (having the same form as the cross-sectional position) based on the first three-dimensional image M1. This processing is performed by specifying pixels (such as voxels) positioned along the specific position, and arranging the specified pixels two-dimensionally to form an image representing the form of the fundus oculi Ef in the cross section. . The same applies to the process of forming a tomographic image (second tomographic image) at the cross-sectional position designated in the second fundus photographic image H2.

  An example of a tomographic image formed by this processing is shown in FIG. FIG. 6A shows a cross-shaped cross-sectional position U1 designated in the first fundus photographic image H1. FIG. 6B shows a cross-shaped cross-sectional position U1 ′ set on the first three-dimensional image M1 in accordance with the cross-sectional position U1. Further, two tomographic images G1a and G1b along the cross-sectional position U1 ′ are shown in FIG. These tomographic images G1a and G1b are examples of the first tomographic image described above. The second tomogram is obtained in the same manner.

(Inclination deviation calculation unit)
The inclination shift amount calculation unit 235 has, for each cross-section position designated by the cross-section position designation unit 233, a surface (target surface) stretched by a direction along the cross-sectional position (cross-sectional direction) and a fundus depth direction (z direction). An inclination shift amount between the first tomographic image and the second tomographic image is calculated based on the first tomographic image and the second tomographic image. The tilt deviation amount calculation unit 235 is an example of the “second calculation unit” in the present invention.

  Here, since the first cross-sectional position and the second cross-sectional position are designated at substantially the same position on the fundus oculi Ef, the first tomographic image and the second tomographic image are images on substantially the same cross-section of the fundus oculi Ef. It is. Therefore, it can be said that the inclination shift amount calculation unit 235 compares the state at the first inspection timing and the state at the second inspection timing with respect to the cross section at substantially the same position.

  An example of processing executed by the tilt deviation amount calculation unit 235 will be described. The cross-sectional position designated by the cross-sectional position designation unit 233 is arranged in a direction along an arbitrary direction on the xy plane. For example, in a cross-shaped cross-sectional position, when one cross-sectional position is arranged in the direction along the x direction, the other cross-sectional position is arranged along the y direction. Accordingly, the target surfaces in this case are the xz plane and the yz plane.

  In addition, when the first cross-sectional positions are arranged in the x direction at the radial cross-sectional positions including four cross-sectional positions arranged at equal intervals, the second cross-sectional position is 45 degrees from the x direction toward the y direction. In the direction, the third cross-sectional position is arranged in the 90-degree direction, and the fourth cross-sectional position is arranged in the 135-degree direction. In this case, in addition to the xz plane and the yz plane, four target surfaces of two planes obtained by rotating the xz plane and the yz plane by 45 degrees about the intersecting line of these planes are obtained.

  When the cross-sectional position is linear, the cross-sectional direction is constant at each point on the cross-sectional position, and the surface stretched by the cross-sectional direction and the z direction is a plane. However, when the cross-sectional position is not linear, such as a curved line or a polygonal line, the cross-sectional direction changes according to a point on the cross-sectional position, and the obtained surface is not a flat surface. In this case, the target surface is a two-dimensional region obtained by extending the cross-sectional position in the depth direction. For example, when a circular cross-sectional position is used, a cylindrical target surface whose axis is the z direction is obtained.

  The inclination shift amount calculation unit 235 specifies an image region (comparison region) where a predetermined part of the fundus oculi Ef is depicted for each of the first tomographic image and the second tomographic image. The predetermined part is desirably a part that is clearly depicted in the tomographic image or a part that is depicted in a characteristic shape in the tomographic image. Furthermore, it is desirable that the predetermined portion is a portion whose shape does not substantially change with time. Examples of such a predetermined site include an optic disc, macula, retinal surface, and a predetermined layered tissue of the retina and choroid. The comparison region corresponding to the predetermined part is identified by well-known image processing such as analysis of the pixel value of the tomographic image and shape analysis of the image region.

  Next, the inclination shift amount calculation unit 235 compares the comparison area (first comparison area) in the first tomographic image with the comparison area (second comparison area) in the second image area. An inclination is calculated, and this is set as an inclination shift amount between the first tomographic image and the second tomographic image.

  An example of the above process will be described with reference to FIGS. FIG. 7A shows the first tomographic image G1a shown in FIG. The first tomographic image G1a is a tomographic image along a linear cross-sectional position (target cross-sectional position) substantially along the x direction at the cross-shaped cross-sectional position U1 shown in FIG. FIG. 7B shows a second tomographic image G2a along the linear cross-sectional position corresponding to the target cross-sectional position at the cross-sectional position U2 shown in FIG. 5B.

  The first and second tomographic images G1a and G2a depict the macular (corresponding image area) YS and its vicinity of the fundus oculi Ef, respectively. Symbols A1 and A2 in FIGS. 7A and 7B indicate comparison regions corresponding to the retina surface (the predetermined portion) of the fundus oculi Ef, respectively.

  FIG. 7C shows the tomographic images G1a and G2a in an overlapping manner in order to show that the first and second tomographic images G1a and G2a are relatively inclined. FIG. 8 is an enlarged view of only the comparison areas A1 and A2 shown in FIG.

  The tilt shift amount calculation unit 235 calculates the tilt shift amount between the first tomographic image G1a and the second tomographic image G2a by executing the following processing based on the comparison areas A1 and A2. To do.

  For this purpose, the tilt deviation amount calculation unit 235 first identifies the feature position in each of the comparison areas A1 and A2. This feature position may be one point in the comparison area or an area having a spread. In this processing example, the image positions P1 and P2 corresponding to the deepest part (fovea) of the macular YS in the comparison areas A1 and A2 are specified as characteristic parts, respectively.

  At this time, if necessary, the inclination shift amount calculation unit 235 obtains a curve that optimally approximates each of the comparison areas A1 and A2. As this approximate curve, an appropriate one such as a spline curve or a Bezier curve is used. Furthermore, the inclination shift amount calculation unit 235 analyzes the shape of the approximate curve, for example, by calculating the first-order differential coefficient and the second-order differential coefficient at each point of the approximate curve, so that the feature position (fovea) The image positions P1 and P2 corresponding to are specified.

  At this time, characteristics (position, shape, etc.) of the feature position in the comparison area can be used. For example, in this processing example, utilizing the fact that the fovea is the deepest part of the concave macular in the tomographic image, first, the concave part (macular) is identified based on the differential coefficient of each point, and this concave The deepest part is specified with reference to the differential coefficient of each point on the part.

  Further, the feature position corresponding to the fovea may be obtained as follows. First, a line segment connecting the edges of the concave portion is obtained. Next, the midpoint of the line segment is obtained. Further, a straight line orthogonal to the line segment and passing through the midpoint is obtained. Then, the intersection of the straight line and the concave part is obtained. This intersection is taken as the target feature position.

  When the feature position in the comparison area is specified, the inclination deviation amount calculation unit 235 determines the direction (or inclination) of the tomographic image based on the feature position. In this processing example, the tangent lines T1 and T2 of the comparison areas A1 and A2 at the image positions P1 and P2 corresponding to the fovea are obtained, and the tangent direction is set as the direction of each tomographic image G1a and G2a.

  In this processing example, the approximation curve is used to enable differentiation at each point on the comparison area, but there are cases where this is not the case. In that case, the direction of the tomographic image can be determined in the same manner as in this processing example as long as differentiation is possible at least at the feature position. Conversely, a differentiable point may be defined as a characteristic part. On the other hand, when the feature position is a non-differentiable point, for example, a differentiable point separated from the feature position by a predetermined distance (path) along the comparison region is obtained, and in this respect, the same processing as this processing example is obtained. It can be performed.

  In addition, mathematically speaking, “a curve that is continuous but cannot be differentiated” is extremely special, and it is unlikely that such a special curve will appear in the processing according to the present invention. . Therefore, it seems that substantially all cases can be handled by applying any of the above-described processes.

When the respective directions of the first and second tomographic images are determined, the inclination shift amount calculation unit 235 calculates an angle formed by these directions. In this processing example, the inclination shift amount calculation unit 235 calculates an angle Δθ _V formed by the tangent line T1 and the tangent line T2. This angle Δθ _V is employed as the amount of tilt deviation between the tomographic images G1a and G2a.

The tilt deviation amount is a vector amount. In this processing example, for example, the tangent line T1 in the first tomographic image G1a is used as a reference (that is, angle 0), and the counterclockwise direction is an angle with respect to the center position (intersection of the tangent line T1 and the tangent line T2) at the time of angle measurement. The positive direction. In the case shown in FIG. 8, the tangent T2 is displaced in the positive direction by [Delta] [theta] _V relative to the tangent T1. Note that the angle of the tangent line T1 with respect to the tangent line T2 may be obtained, or the angle of each tangent line T1, T2 with respect to a predetermined reference direction may be obtained. In any case, it is sufficient if the relative inclination shift amount between the tangent lines T1 and T2 can be expressed.

  When a plurality of cross-sectional positions are designated, the inclination shift amount calculation unit 235 performs the above-described processing for each cross-sectional position. Thereby, for each cross-sectional position, an inclination shift amount between the first tomographic image and the second tomographic image along the cross-sectional position is obtained.

  In this processing example, two cross-sectional positions forming a cross-shaped cross-sectional position, that is, a straight cross-sectional position extending in the horizontal direction (transverse cross-sectional position) and a linear cross-sectional position orthogonal to this (vertical cross-sectional position) The above-described processing is executed for each, and a pair of tilt deviation amounts are obtained.

  Moreover, when applying a radial cross-sectional position, said process is performed about each of several cross-sectional position which forms this. Thereby, a plurality of tilt shift amounts corresponding to a plurality of cross-sectional positions forming a radial cross-sectional position are obtained.

(Alignment processing unit)
The alignment processing unit 236 uses the first three-dimensional image M1 and the second two-dimensional image M1 based on the positional deviation amount calculated by the positional deviation amount calculation unit 232 and the inclination deviation amount calculated by the inclination deviation amount calculation unit 235. Is aligned with the three-dimensional image M2. The alignment processing unit 236 is an example of the “alignment unit” in the present invention.

  An example of processing executed by the alignment processing unit 236 will be described. FIG. 9 shows a positional shift state between the first three-dimensional image M1 and the second three-dimensional image M2. FIG. 9C illustrates a positional shift state between the first three-dimensional image M1 illustrated in FIG. 9A and the second three-dimensional image M2 illustrated in FIG. The three-dimensional images M1 and M2 are superimposed and presented.

As described above, the positional deviation amount calculated by the positional deviation amount calculation unit 232 includes the parallel movement amounts Δx and Δy and the rotational movement amount Δθ. The slope deviation amount calculated by the inclination shift amount calculating unit 235 is [Delta] [theta] _V. The cross-shaped cross-sectional positions U1 and U2 are formed by a horizontal cross-sectional position and a vertical cross-sectional position, respectively. This inclination deviation amount Δθ _V includes an inclination deviation amount Δθ _V 1 corresponding to the horizontal cross-sectional position and an inclination deviation amount Δθ _V 2 corresponding to the vertical cross-sectional position (not shown).

Positioning process unit 236, the values [Delta] x, [Delta] y, [Delta] [theta], so as to cancel the [Delta] [theta] _V, and aligned by relatively moving these 3-dimensional images M1, M2. For example, the alignment processing unit 236 translates the second three-dimensional image M2 by −Δx in the x direction and −Δy in the y direction, and rotates and moves it by −Δθ in the xy plane. Further, the alignment processing unit 236 applies the second three-dimensional image M2 to -Δθ in a plane stretched by the direction along the cross-sectional position on the second three-dimensional image M2 and the fundus depth direction (z direction). Rotate and move by _V1 . Similarly, the alignment processing unit 236 displays the second 3D image M2 in a plane stretched by the direction along the longitudinal cross-sectional position on the second 3D image M2 and the fundus depth direction (z direction). Rotate and move by Δθ _V 2.

Note that the alignment processing of the three-dimensional images M1 and M2 is not limited to the above. That is, the value [Delta] x, to counteract [Delta] y, [Delta] [theta], the [Delta] [theta] _V, so long as it is capable of moving one or both of the two three-dimensional images M1, M2, it is possible to apply any of the alignment process.

  According to such an alignment process, as shown in FIG. 10, the image of the fundus oculi Ef depicted in the first three-dimensional image M1, the image of the fundus oculi Ef depicted in the second three-dimensional image M2, and Are relatively parallel and rotated between the three-dimensional images M1 and M2.

(Inclination correction unit)
The inclination shift correction unit 237 operates when a radial cross-sectional position is specified by the cross-sectional position specifying unit 233. In this case, the tilt shift amount calculation unit 235 calculates the tilt shift amount for each cross-sectional position forming a radial cross-sectional position.

  The inclination deviation correction unit 237 first selects the maximum value of the inclination deviation amounts corresponding to these cross-sectional positions. This process is executed by selecting an inclination value (maximum inclination deviation amount) having the maximum absolute value from among the inclination deviation amounts (a vector amount, that is, expressed by an angle having a direction).

  Next, the inclination deviation correction unit 237 is arranged in a plane stretched by the direction along the cross-sectional position corresponding to the maximum inclination deviation amount and the fundus depth direction (z direction) so as to cancel the selected maximum inclination deviation amount. The deviation of the inclination between the first and second three-dimensional images M1 and M2 is corrected.

As an example of this process, consider a case where the amount of inclination deviation corresponding to the cross-sectional position along the x direction is selected as the maximum inclination deviation amount Δθ _Vmax . In this case, the inclination deviation correction unit 237 relatively rotates the first and second three-dimensional images M1 and M2 on the xz plane so as to cancel the maximum inclination deviation amount Δθ _Vmax . As a specific example, the inclination shift correction unit 237 rotates the second three-dimensional image M2 by −Δθ _Vmax on the xy plane. Thereby, the inclination (maximum inclination) in the plane direction of the image of the fundus oculi Ef depicted in each of the first and second three-dimensional images M1 and M2 is corrected.

  Such an inclination shift correcting unit 237 is an example of the “correcting unit” of the present invention.

(Image Analysis Department)
The image analysis unit 238 analyzes the first three-dimensional image M1 and calculates a first value of a predetermined physical quantity. Similarly, the image analysis unit 238 analyzes the second three-dimensional image M2 and calculates a second value of a predetermined physical quantity. Note that the three-dimensional images M1 and M2 analyzed by the image analysis unit 238 have not been subjected to the above-described alignment process or tilt shift correction process. Further, “analyze the three-dimensional images M1 and M2” includes a case where an OCT image (tomographic image or the like) based on the three-dimensional images M1 and M2 is analyzed. The image analysis unit 238 is an example of the “analysis unit” in the present invention.

  Processing executed by the image analysis unit 238 will be described. The “predetermined physical quantity” to be calculated is an arbitrary test result (obtained as a numerical value) referred to in diagnosis of fundus oculi disease. Specific examples include the size of the lesion (radius, diameter, area, volume, etc.), the depth of the lesion from the retina surface, the size of the optic disc (cup size, disk size, rim size, etc.), retinal pigment epithelium (Retinal Pigment). Epithelium (RPE) spacing, retinal thickness, etc.

  As described above, the images used for the processing of the image analysis unit 238 include the three-dimensional images M1 and M2 of the fundus oculi Ef and tomographic images obtained from the three-dimensional images M1 and M2. Note that the process of forming a tomographic image from the three-dimensional images M1 and M2 is executed in the same manner as described above. The process of forming a tomographic image at an arbitrary cross-sectional position based on a three-dimensional image is called multi-planar reconstruction (MPR). Alternatively, the fundus photographic images H1 and H2 may be analyzed to calculate a predetermined physical quantity.

  The case of analyzing the tomogram of the optic nerve head and its vicinity will be described with reference to FIG. The tomographic image J1 shown in FIG. 11A is obtained by specifying a linear cross-sectional position along the x direction through the image region OP corresponding to the optic nerve head with respect to the first three-dimensional image M1. It is done. Similarly, the tomographic image J2 shown in FIG. 11B is obtained by designating the same cross-sectional position for the second three-dimensional image M2. That is, the tomographic images J1 and J2 are two-dimensional images on the xz plane. Reference numerals B1 and B2 denote retinal surfaces (corresponding image areas).

  The image analysis unit 238 analyzes the tomographic image J1 and calculates the RPE interval d1. More specifically, the image analysis unit 238 first analyzes the pixel values of the pixels constituting the tomographic image J1, and specifies the image region OP1 corresponding to the optic nerve head and the image region C1 corresponding to RPE. . Here, the image region OP1 is specified by, for example, specifying the image region B1 corresponding to the retina surface and analyzing the shape of the image region B1 to specify the concave image region corresponding to the optic disc. it can. The image region C1 corresponding to the RPE is specified by, for example, specifying image regions of various layered tissues (boundaries) of the fundus oculi Ef depicted in the tomographic image J1, and determining the luminance of these image regions and the retinal surface B1. This can be done by selecting the equivalent of RPE based on depth. Since the RPE is located so as to surround the optic disc, the tomographic image J1 specifies the image region C1 on both the + x side and the −x side with respect to the image region OP1 corresponding to the optic disc. .

  Further, the image analysis unit 238 specifies a pixel (closest pixel) located closest to the image region OP1 among pixels in the image region C1 corresponding to the RPE located on the + x side with respect to the image region OP1. . In this process, for example, for each pixel in the image area C1, a distance to each pixel in the image area OP1 (for example, Euclidean distance in the xz coordinate system) is calculated, and the shortest of these distances is selected. The shortest of the shortest pixel distances in the image area C1 obtained in this way is selected. The pixel corresponding to the selected shortest distance becomes the target pixel. Further, for each pixel in the image area C1, the radius of a circle centered on the pixel is gradually enlarged until it touches (or intersects) the image area OP1. The radius at this time is the distance between the pixel and the image area OP1. The pixel with the shortest distance is the target pixel. The closest pixel is similarly specified for the image region C1 corresponding to the RPE located on the −x side with respect to the image region OP1.

  The image analysis unit 238 calculates a distance d1 between the nearest pixel located on the + x side and the nearest pixel located on the −x side with respect to the image region OP1. The distance d1 may be a Euclidean distance in the xz coordinate system, or may be a distance along the x direction (that is, an x component of the Euclidean distance). This distance d1 is the RPE interval based on the tomographic image J1.

  A tomographic image J2 shown in FIG. 11B is relatively inclined with respect to the tomographic image J1 shown in FIG. If processing similar to that of the tomographic image J1 is executed as in the prior art, this inclination is not reflected in the calculation of the RPE interval, so the analysis results of the tomographic images J1 and J2 acquired under different conditions are compared. Large errors may occur. For example, in the case shown in FIG. 11, for example, when comparing distances along the x direction, the distance d1 shown in FIG. 11A and the distance d2 ′ shown in FIG. 11B are obtained as the RPE intervals. As a result, the comparison accuracy of the RPE interval is greatly reduced.

  Strictly speaking, the tomographic images J1 and J2 are both inclined with respect to the true direction due to the movement of the eye E or the like, and it is extremely difficult to obtain the true value of the RPE interval. However, in the comparative observation, since it is necessary to compare both analysis results under (substantially) the same conditions, it is possible to improve accuracy by considering the relative inclination. However, it is possible to approach the true value by correcting the inclination of each tomographic image J1, J2.

  In order to improve the accuracy of comparative observation, the image analysis unit 238 performs the following processing to reflect the inclination in the analysis result. First, the image analysis unit 238 specifies an image region OP2 corresponding to the optic disc and an image region C2 corresponding to RPE in the same manner as the processing related to the tomographic image J1. Further, the image analysis unit 238 specifies the closest pixel for the image region C2 corresponding to the RPE located on the + x side and the −x side with respect to the image region OP2.

Further, the image analysis unit 238 indicates the measurement direction of the distance d1 when the RPE interval is obtained for the tomographic image J1 (that is, the direction of the double-sided arrow indicating the RPE interval in FIG. 11A), and the inclination deviation amount calculation unit 235. Is tilted by the tilt deviation amount Δθ _V calculated by the above. Then, a distance d2 between the nearest pixel on the + x side and the nearest pixel on the -x side is calculated along this inclination direction. This distance d2 is the RPE interval based on the tomographic image J2. Thereby, RPE intervals d1 and d2 measured in substantially the same direction with respect to the tomographic images J1 and J2 are obtained, and the RPE intervals can be compared with high accuracy.

In this processing example, in order to measure the RPE interval under almost the same conditions, only the inclination deviation amount Δθ _V obtained by the inclination deviation amount calculation unit 235 is considered, but the type of measurement target (predetermined physical quantity) is considered. Accordingly, it is possible to consider only the positional deviation amounts Δx, Δy, Δθ obtained by the positional deviation amount calculation unit 232, or to consider both the positional deviation amounts Δx, Δy, Δθ and the inclination deviation amount Δθ _V. It is.

  The image processing unit 230 that functions as described above includes, for example, the aforementioned microprocessor, RAM, ROM, hard disk drive, circuit board, and the like. In a storage device such as a hard disk drive, a computer program for causing the microprocessor to execute the above functions is stored in advance.

(Display section, operation section)
The display unit 240 includes the display device of the arithmetic control unit 200 described above. The operation unit 250 includes the operation device of the arithmetic control unit 200 described above. The operation unit 250 may include various buttons and keys provided on the housing of the fundus oculi observation device 1 or outside. For example, when the fundus camera unit 2 has a housing similar to that of a conventional fundus camera, the operation unit 250 may include a joystick, an operation panel, or the like provided on the housing. In addition, the display unit 240 may include various display devices such as a touch panel monitor provided on the housing of the fundus camera unit 2.

  The display unit 240 and the operation unit 250 do not need to be configured as individual devices. For example, a device in which a display function and an operation function are integrated, such as a touch panel monitor, can be used.

[Scanning signal light and OCT images]
Here, the scanning of the signal light LS and the OCT image will be described.

  Examples of scanning modes of the signal light LS by the fundus oculi observation device 1 include horizontal scanning, vertical scanning, cross scanning, radiation scanning, circular scanning, concentric scanning, and spiral (vortex) scanning. These scanning modes are selectively used as appropriate in consideration of the observation site of the fundus, the analysis target (such as retinal thickness), the time required for scanning, the precision of scanning, and the like.

  The horizontal scan scans the signal light LS in the horizontal direction (x direction). The horizontal scan also includes an aspect in which the signal light LS is scanned along a plurality of horizontal scanning lines arranged in the vertical direction (y direction). In this aspect, it is possible to arbitrarily set the scanning line interval. Further, the above-described three-dimensional image can be formed by sufficiently narrowing the interval between adjacent scanning lines (three-dimensional scanning). The same applies to the vertical scan.

  In the cross scan, the signal light LS is scanned along a cross-shaped trajectory composed of two linear trajectories (straight trajectories) orthogonal to each other. In the radiation scan, the signal light LS is scanned along a radial trajectory composed of a plurality of linear trajectories arranged at a predetermined angle. The cross scan is an example of a radiation scan.

  In the circle scan, the signal light LS is scanned along a circular locus. In the concentric scan, the signal light LS is scanned along a plurality of circular trajectories arranged concentrically around a predetermined center position. A circle scan is an example of a concentric scan. In the spiral scan, the signal light LS is scanned along a spiral (spiral) locus while the radius of rotation is gradually reduced (or increased).

  Since the galvanometer mirrors 43 and 44 are configured to scan the signal light LS in directions orthogonal to each other, the signal light LS can be scanned independently in the x and y directions, respectively. Furthermore, by simultaneously controlling the directions of the galvanometer mirrors 43 and 44, it is possible to scan the signal light LS along an arbitrary locus on the xy plane. Thereby, various scanning modes as described above can be realized.

  By scanning the signal light LS in the above-described manner, a tomographic image in the fundus depth direction (z direction) along the scanning line (scanning locus) can be formed. In particular, when the interval between scanning lines is narrow, the above-described three-dimensional image can be formed.

  A region on the fundus oculi Ef to be scanned with the signal light LS as described above, that is, a region on the fundus oculi Ef to be subjected to OCT measurement is referred to as a scanning region. The scanning area in the three-dimensional scan is a rectangular area in which a plurality of horizontal scans are arranged. The scanning area in the concentric scan is a disk-shaped area surrounded by the locus of the circular scan with the maximum diameter. In addition, the scanning area in the radial scan is a disk-shaped (or polygonal) area connecting both end positions of each scan line.

[Action / Effect]
The operation and effect of the fundus oculi observation device 1 as described above will be described.

  The fundus oculi observation device 1 includes the first fundus photographic image H1 and the first three-dimensional image M1 acquired at the first examination timing, and the second fundus photographic image H2 and the first fundus photographic image H2 obtained at the second examination timing. 2 three-dimensional images M2 are stored.

  Furthermore, the fundus oculi observing device 1 is based on the first fundus photographic image H1 and the second fundus photographic image H2, and the first fundus photographic image H1 and the second fundus photographic image H2 in the fundus surface direction (xy direction). Displacement amounts Δx, Δy, and Δθ between are calculated. Next, the fundus oculi observation device 1 designates a cross-sectional position at substantially the same position on the fundus oculi Ef depicted in each fundus oculi image H1, H2 based on the calculated positional deviation amounts Δx, Δy, Δθ.

  Subsequently, the fundus oculi observation device 1 forms a first tomographic image at the cross-sectional position designated in the first fundus photographic image H1 based on the first three-dimensional image M1, and also the second three-dimensional image. Based on M2, a second tomographic image is formed at the cross-sectional position designated in the second fundus photographic image H2. As described above, instead of actually forming a tomographic image, the subsequent processing is executed with reference to the pixels at the cross-sectional positions of the three-dimensional images M1 and M2 corresponding to the designated cross-sectional positions. May be.

Then, the fundus oculi observation device 1 uses the first tomographic image and the second tomographic image to generate a first tomographic image on a plane stretched in the direction along the designated cross-sectional position and the fundus depth direction (z direction). An inclination shift amount Δθ _V between the image and the second tomographic image is calculated.

According to the fundus oculi observation device 1 operating in this way, the positional deviation amounts Δx, Δy, Δθ and the inclination deviation amount Δθ _V interposed between images (particularly the three-dimensional images M1, M2) to be compared in the comparative observation are obtained. Can be sought. Therefore, by performing comparative observation with reference to these shift amounts, it is possible to compare images to be compared as those acquired under substantially the same conditions. This makes it possible to perform comparative observation of the fundus with high accuracy.

  In particular, the fundus oculi observation device 1 can specify a plurality of cross-sectional positions for the respective fundus photographic images H1 and H2. In that case, the fundus oculi observation device 1 forms a first tomographic image and a second tomographic image for each of a plurality of designated sectional positions, and the first tomographic image and the second tomographic image at each sectional position. Based on the above, the amount of tilt deviation in the surface stretched between the direction along the cross-sectional position and the fundus depth direction is calculated. Thereby, a plurality of pairs of tomographic images corresponding to a plurality of cross-sectional positions are obtained.

  As an example of designating a plurality of cross-sectional positions, the fundus oculi observation device 1 can designate a cross-shaped cross-sectional position, that is, a pair of linear cross-sectional positions that intersect at right angles to each other. In this case, the fundus oculi observation device 1 calculates a tilt shift amount by forming a first tomographic image and a second tomographic image for each designated cross-sectional position. Thereby, the amount of inclination shift in each of a pair of planes orthogonal to each other is obtained. By acquiring such a pair of tilt shift amounts, whatever the tilt shift interposed between the three-dimensional images M1 and M2, the tilt shift is calculated as a vector sum of the pair of tilt shift amounts. Can be expressed as

  As another example of designating a plurality of cross-sectional positions, a radial cross-sectional position, that is, two or more linear cross-sectional positions arranged radially and intersecting each other can be designated. In this case, the fundus oculi observation device 1 calculates a tilt shift amount by forming a first tomographic image and a second tomographic image for each designated cross-sectional position. Thereby, the same number of inclination deviation amounts as the number of cross-sectional positions (two or more) can be obtained. Further, the fundus oculi observation device 1 selects the maximum value (maximum inclination deviation amount) of these two or more inclination deviation amounts. Then, the fundus oculi observation device 1 has the first three-dimensional image M1 on the plane stretched by the direction along the cross-sectional position corresponding to the maximum tilt shift amount and the fundus depth direction so as to cancel the maximum tilt shift amount. The inclination deviation between the second three-dimensional image M2 is corrected. As a result, the inclination deviation can be corrected in the direction with the largest inclination, so that it is possible to correct the inclination deviation in the direction that has the greatest influence on the comparative observation while saving the resources of the arithmetic control unit 200. is there.

  In addition, instead of correcting only for the direction in which the tilt deviation amount is maximum, it is possible to increase the accuracy of the correction by configuring the correction so that each direction in which the tilt deviation amount is a predetermined value or more is performed. Further, correction may be performed for each direction up to a predetermined number from the one with the large amount of tilt deviation.

  Further, instead of automatically specifying the cross-sectional position as described above, the cross-sectional position may be manually specified. For example, the first and second fundus photographed images H1 and H2 can be displayed on the display unit 240, and the operation unit 250 can be operated to specify the cross-sectional position on these display images. In addition, one fundus photographed image (for example, the first fundus photographed image H1) is displayed and the cross-sectional position is manually designated, and the other fundus photographed image (for example, the second fundus photographed image H2) is manually selected. It is also possible to configure such that a position substantially the same as the designated cross-sectional position is automatically designated. That is, the designation of the cross-sectional position in the fundus photographic images H1 and H2 may be performed automatically, only a part may be performed automatically, or all may be performed manually. . When all are performed automatically, there is an advantage that it is possible to eliminate the time and effort required for designation. When only a part is automatically performed, there are advantages that it is possible to reduce the time and effort required for designation, and that a doctor can designate a desired cross-sectional position. When all are performed manually, the doctor can specify the desired cross-sectional position in both fundus photographic images H1 and H2, so even if the target region is difficult to extract by image processing, the doctor's knowledge and experience can be obtained. There is an advantage that the cross-sectional position can be specified by taking advantage of it. Which of these methods is adopted is arbitrary. It is also possible to configure such that these methods can be selectively used.

Further, the fundus oculi observation device 1, the calculated position deviation amount [Delta] x, [Delta] y, so as to cancel the [Delta] [theta] and inclination shift amount [Delta] [theta] _V, a position between the first three-dimensional image M1 M2 second three-dimensional image It is possible to combine. As a result, when the three-dimensional images M1 and M2 are visually compared, the common points and differences between the three-dimensional images M1 and M2 can be easily grasped, and thus the accuracy of comparative observation can be improved. Is possible.

Further, the fundus oculi observation device 1, the calculated position deviation amount [Delta] x, [Delta] y, based on one or both of [Delta] [theta] and the slope deviation amount [Delta] [theta] _V, the first analysis of a given physical quantity based on the first three-dimensional image M1 A result (first value) and a second analysis result (second value) of a predetermined physical quantity based on the second three-dimensional image M2 can be calculated. In the conventional comparative observation, these deviation amounts are not taken into consideration, and therefore there is a possibility that the comparison accuracy of the analysis result may be lowered according to the difference in conditions between the first inspection timing and the second inspection timing. It was. On the other hand, according to this embodiment, since the analysis result is obtained in consideration of the above-described deviation amount due to the difference in the conditions, it is possible to compare the analysis result with higher accuracy than before. .

[Modification]
The configuration described above is merely an example for favorably implementing the present invention. Therefore, arbitrary modifications within the scope of the present invention can be made as appropriate.

For example, in the above embodiment, the positional deviation amounts Δx, Δy, Δθ in the fundus surface direction (xy direction) and the inclining deviation amount Δθ _V in the plane including the fundus depth direction (z direction) are obtained. In addition to these, a positional shift amount in the z direction may be obtained.

  As a method for obtaining the amount of positional deviation in the z direction, for example, there is the following. First, z coordinate values in a frame of a predetermined part of the fundus oculi Ef in each of the first tomographic image and the second tomographic image are obtained. Then, a deviation of these z coordinate values (that is, a positional deviation amount in the z direction) is obtained. Here, the predetermined part of the fundus oculi Ef includes the retina surface, the layered tissue of the fundus oculi Ef, and the like. Note that three-dimensional images may be compared instead of comparing tomographic images.

  In the above embodiment, the amount of positional deviation and the amount of inclination deviation are expressed using the xyz coordinate system, and the image alignment and analysis processing are performed. However, these processes are performed using other coordinate systems. You may make it perform.

  For example, as shown in FIG. 12, in the fundus oculi image H1, the central position (papillary center) OPa of the optic disc (corresponding image area) OP and the central position (fovea of the macula) YS (corresponding image area) , Macular center) YSa. This process can be executed using a conventional fundus image analysis technique. Further, a two-dimensional polar coordinate system is defined in which the nipple center OPa is the origin and the direction connecting the nipple center OPa and the fovea YSa is the reference direction S. Here, the unit of the distance from the origin OPa can be set, for example, based on the distance between the nipple center OPa and the fovea YSa.

  According to this two-dimensional polar coordinate system, an arbitrary position P in the fundus photographic image H1 can be expressed by coordinate values (r, θ). Here, r is the distance between the origin OPa and the position P. Θ is an angle formed by a line segment connecting the origin OPa and the position P with respect to the reference direction S. Here, as usual, the direction of turning counterclockwise from the origin OPa when viewing the fovea YSa is defined as the positive direction of the angle. This two-dimensional polar coordinate system can be used instead of the above xy coordinate system.

  It is also possible to define a two-dimensional orthogonal coordinate system with the reference direction S shown in FIG. 12 as the first reference direction and the direction orthogonal to the reference direction S as the second reference direction. Similarly, a two-dimensional oblique coordinate system can be defined.

  In the above embodiment, the fundus photographed image is described as an image photographed by a fundus camera or the like, but the fundus photographed image is not limited to this. For example, an arbitrary image depicting a two-dimensional form of the fundus surface, such as an image obtained by photographing the fundus (for example, the above-described integrated image), can be used as a fundus image.

  In the above embodiment, the position of the reference mirror 114 is changed to change the optical path length difference between the optical path of the signal light LS and the optical path of the reference light LR, but the method of changing the optical path length difference is limited to this. Is not to be done. For example, the optical path length difference can be changed by moving the fundus camera unit 2 or the OCT unit 100 with respect to the eye E to change the optical path length of the signal light LS. It is also effective to change the optical path length difference by moving the measurement object in the depth direction (z direction), particularly when the measurement object is not a living body part.

  The computer program in the above embodiment can be stored in any recording medium readable by a computer. As this recording medium, for example, an optical disk, a magneto-optical disk (CD-ROM / DVD-RAM / DVD-ROM / MO, etc.), a magnetic storage medium (hard disk / floppy (registered trademark) disk / ZIP, etc.), etc. are used. Is possible. It can also be stored in a storage device such as a hard disk drive or memory.

  It is also possible to transmit / receive this program through a network such as the Internet or a LAN.

DESCRIPTION OF SYMBOLS 1 Fundus observation apparatus 2 Fundus camera unit 3 Display apparatus 10 Illumination optical system 30 Shooting optical system 43, 44 Galvano mirror 100 OCT unit 200 Arithmetic control unit 210 Control unit 211 Main control unit 212 Storage unit 220 Image forming unit 230 Image processing unit 231 Three-dimensional image forming unit 232 Position shift amount calculating unit 233 Cross section position specifying unit 235 Tilt shift amount calculating unit 236 Position processing unit 237 Tilt shift correcting unit 238 Image analyzing unit E Eye to be examined Ef Fundus H1, H2 Fundus photographed images M1, M2 Three-dimensional image U1, U2 Cross-sectional position G1a, G2a, J1, J2 Tomographic image

Claims (10)

The first fundus photographed image of the eye to be examined and the first three-dimensional image of the fundus of the subject eye obtained at the first examination timing, and the second examination timing different from the first examination timing. Storage means for storing in advance a second fundus photographed image of the eye to be examined and a second three-dimensional image of the fundus;
Based on the first fundus photographed image and the second fundus photographed image, a first amount of positional deviation between the first fundus photographed image and the second fundus photographed image in the fundus surface direction is calculated. Means for calculating
Designating means for designating a cross-sectional position at substantially the same position on the fundus depicted in each of the first fundus photographed image and the second fundus photographed image based on the calculated positional deviation amount;
Corresponding to the first tomographic image at the cross-sectional position of the first three-dimensional image corresponding to the cross-sectional position specified in the first fundus image and the cross-sectional position specified in the second fundus image. Based on the second tomographic image at the cross-sectional position of the second three-dimensional image, the first tomographic image and the second on the plane stretched by the direction along the designated cross-sectional position and the fundus depth direction. Second calculating means for calculating the amount of tilt deviation between the tomographic image of
A fundus image processing apparatus comprising: The designation means designates a plurality of the cross-sectional positions,
The second calculation unit is configured to determine the first tomographic image and the second tomographic image at the cross-sectional positions of the first and second three-dimensional images corresponding to the plurality of cross-sectional positions, respectively. On the basis of the direction of the specified cross-sectional position and the amount of tilt deviation in the surface stretched by the fundus depth direction,
The fundus image processing apparatus according to claim 1. The designation means designates a pair of linear cross-sectional positions that intersect at right angles as the plurality of cross-sectional positions,
The second calculating means calculates the amount of inclination shift for the cross-sectional positions of the first and second three-dimensional images corresponding to the pair of cross-sectional positions, respectively, so that each of the pair of planes orthogonal to each other is calculated. Find the amount of tilt deviation at
The fundus image processing apparatus according to claim 2. The designation means designates two or more linear cross-sectional positions that are arranged radially and intersect with each other as the plurality of cross-sectional positions;
The second calculation means calculates the amount of tilt deviation for the cross-sectional positions of the first and second three-dimensional images corresponding to the two or more cross-sectional positions,
The maximum value of the inclination deviation amounts corresponding to the two or more cross-sectional positions calculated by the second calculation means is selected, and the inclination deviation amount is set so as to cancel the selected inclination deviation amount. A correction unit that corrects a deviation in inclination between the first three-dimensional image and the second three-dimensional image in a plane stretched by the direction along the corresponding cross-sectional position and the fundus depth direction;
The fundus image processing apparatus according to claim 2. The first three-dimensional image and the second three-dimensional image so as to cancel out the positional deviation amount calculated by the first calculation means and the inclination deviation amount calculated by the second calculation means. An alignment means for performing alignment with
The fundus image processing apparatus according to claim 1. Based on the positional shift amount calculated by the first calculation means and / or the tilt shift amount calculated by the second calculation means, the first three-dimensional image is analyzed and a predetermined physical quantity of the predetermined physical quantity is calculated. Analyzing means for calculating a value of 1 and analyzing the second three-dimensional image to calculate a second value of the predetermined physical quantity;
The fundus image processing apparatus according to claim 1. The first calculation means calculates a parallel movement amount and a rotational movement amount in the fundus surface direction as the positional deviation amount.
The fundus image processing apparatus according to any one of claims 1 to 6, wherein: Photographing means for photographing the fundus of the eye to be examined;
An optical system that divides low-coherence light into signal light and reference light, and generates and detects interference light by superimposing the signal light passing through the fundus and the reference light passing through a reference optical path;
Image forming means for forming a three-dimensional image of the fundus based on the detection result of the interference light;
A first fundus photographed image of the eye to be examined photographed by the photographing means at a first examination timing, a first three-dimensional image of the fundus formed by the image forming means, and the first examination timing; Storage means for storing a second fundus photographed image of the subject eye photographed at a different second examination timing and a second three-dimensional image of the fundus formed.
Based on the first fundus photographed image and the second fundus photographed image, a first amount of positional deviation between the first fundus photographed image and the second fundus photographed image in the fundus surface direction is calculated. Means for calculating
Designating means for designating a cross-sectional position at substantially the same position on the fundus depicted in each of the first fundus photographed image and the second fundus photographed image based on the calculated positional deviation amount;
Corresponding to the first tomographic image at the cross-sectional position of the first three-dimensional image corresponding to the cross-sectional position specified in the first fundus image and the cross-sectional position specified in the second fundus image. Based on the second tomographic image at the cross-sectional position of the second three-dimensional image, the first tomographic image and the second on the plane stretched by the direction along the designated cross-sectional position and the fundus depth direction. Second calculating means for calculating the amount of tilt deviation between the tomographic image of
A fundus oculi observation device comprising: An optical system that divides low-coherence light into signal light and reference light, and generates and detects interference light by superimposing the signal light passing through the fundus of the eye to be examined and the reference light passing through the reference optical path;
Image forming means for forming a three-dimensional image of the fundus based on the detection result of the interference light;
A first fundus photographed image of the eye to be examined acquired at a first examination timing, and a second fundus photographed image of the eye to be examined obtained at a second examination timing different from the first examination timing. A first three-dimensional image of the fundus that is stored in advance and formed by the image forming unit at the first examination timing, and a second three-dimensional image of the fundus that is formed at the second examination timing Storage means for storing images;
Based on the first fundus photographed image and the second fundus photographed image, a first amount of positional deviation between the first fundus photographed image and the second fundus photographed image in the fundus surface direction is calculated. Means for calculating
Designating means for designating a cross-sectional position at substantially the same position on the fundus depicted in each of the first fundus photographed image and the second fundus photographed image based on the calculated positional deviation amount;
Corresponding to the first tomographic image at the cross-sectional position of the first three-dimensional image corresponding to the cross-sectional position specified in the first fundus image and the cross-sectional position specified in the second fundus image. Based on the second tomographic image at the cross-sectional position of the second three-dimensional image, the first tomographic image and the second on the plane stretched by the direction along the designated cross-sectional position and the fundus depth direction. Second calculating means for calculating the amount of tilt deviation between the tomographic image of
A fundus oculi observation device comprising: Photographing means for photographing the fundus of the eye to be examined;
A first three-dimensional image of the fundus obtained at the first examination timing and a second three-dimensional image of the fundus obtained at a second examination timing different from the first examination timing are stored in advance. In addition, a first fundus photographed image of the subject eye photographed by the photographing means at the first examination timing and a second fundus photographed image of the subject eye photographed at the second examination timing. Storage means for storing
Based on the first fundus photographed image and the second fundus photographed image, a first amount of positional deviation between the first fundus photographed image and the second fundus photographed image in the fundus surface direction is calculated. Means for calculating
Designating means for designating a cross-sectional position at substantially the same position on the fundus depicted in each of the first fundus photographed image and the second fundus photographed image based on the calculated positional deviation amount;
Corresponding to the first tomographic image at the cross-sectional position of the first three-dimensional image corresponding to the cross-sectional position specified in the first fundus image and the cross-sectional position specified in the second fundus image. Based on the second tomographic image at the cross-sectional position of the second three-dimensional image, the first tomographic image and the second on the plane stretched by the direction along the designated cross-sectional position and the fundus depth direction. Second calculating means for calculating the amount of tilt deviation between the tomographic image of
A fundus oculi observation device comprising: