WO2016006482A1 - Ophthalmic apparatus and control method therefor - Google Patents

Ophthalmic apparatus and control method therefor Download PDF

Info

Publication number
WO2016006482A1
WO2016006482A1 PCT/JP2015/068597 JP2015068597W WO2016006482A1 WO 2016006482 A1 WO2016006482 A1 WO 2016006482A1 JP 2015068597 W JP2015068597 W JP 2015068597W WO 2016006482 A1 WO2016006482 A1 WO 2016006482A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
boundary
photoreceptor cell
acquired
region
Prior art date
Application number
PCT/JP2015/068597
Other languages
French (fr)
Inventor
Keiko Yonezawa
Kazuhide Miyata
Original Assignee
Canon Kabushiki Kaisha
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Canon Kabushiki Kaisha filed Critical Canon Kabushiki Kaisha
Publication of WO2016006482A1 publication Critical patent/WO2016006482A1/en

Links

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/1025Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for confocal scanning
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/102Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for optical coherence tomography [OCT]
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/12Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for looking at the eye fundus, e.g. ophthalmoscopes
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/0059Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
    • A61B5/0062Arrangements for scanning
    • A61B5/0066Optical coherence imaging
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/12Edge-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10141Special mode during image acquisition
    • G06T2207/10144Varying exposure
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30041Eye; Retina; Ophthalmic

Definitions

  • he present invention relates to an ophthalmic
  • the present invention relates to an ophthalmic apparatus to be used in an ophthalmic diagnosis and treatment, which
  • a scanning laser ophthalmoscope which is an ophthalmic apparatus using a principle of a confocal laser microscope.
  • the SLO is an ophthalmic apparatus configured to perform a raster scanning on a fundus of an eye with laser light, which is measurement light, and acquire a planar image of the fundus of the eye from the
  • an adaptive optics SLO capable of acquiring a planar image with a high transverse resolution.
  • AO-SLO adaptive optics SLO
  • an aberration of an eye to be inspected is measured by a wavefront sensor in real time, and aberrations of measuring light and return right thereof generated in an eye to be inspected are corrected by a wavefront correction device of an adaptive optics system.
  • AO-SLO it is further attempted to diagnose a disease and evaluate a drug response by extracting a photoreceptor cell in a retina with the use of an acquired planar image of the retina, and analyzing a density and distribution of the
  • the focus is required to be set on a layer in which the photoreceptor cell exists, but a focus
  • a method disclosed in PTL 1 is proposed.
  • a determination process relating to a normal/abnormal region in visual performance is performed based on a tomographic image photographed by optical coherence tomography (OCT) .
  • the result of the determination process is superimposed on a front image of a fundus to be displayed, and a laser irradiation position for a laser treatment is acquired based on this display.
  • photoreceptor cell is not represented because the photoreceptor cell has a defect from a case where the photoreceptor cell is not represented because of a photographing reason such as a focus shift. If the photoreceptor cell is affected by a disease, the photographing by the AO-SLO is difficult to be
  • the tomographic image is acquired with the use of the OCT, to thereby obtain information relating to the photoreceptor cell from the image. Then, the information is used to determine a normal/abnormal region in visual performance.
  • the present invention has an object to provide an apparatus capable of more accurately representing a state of a
  • an ophthalmic apparatus includes: an image acquiring unit configured to acquire a planar image of a fundus of an eye to be inspected; a boundary acquiring unit
  • FIG. 1 is a block diagram for illustrating a functional configuration of an ophthalmic apparatus 20 according to a first embodiment of the present invention.
  • FIG. 2 is a flow chart for illustrating a processing procedure of the ophthalmic apparatus 20 according to the first embodiment of the present invention.
  • FIG. 3 is a schematic view for showing a state in which a plurality of AO-SLO images are displayed on a WF-SLO image .
  • FIG. 4 is a schematic view of a fixation target map for designating a fixation position.
  • FIG. 5 is a flow chart for illustrating the details of boundary determination in Step S250 illustrated in FIG. 2.
  • FIG. 6A is an example of a frequency conversion image acquired by subjecting the AO-SLO image to frequency conversion .
  • FIG. 6B is another example of the frequency conversion image acquired by subjecting the AO-SLO image to the frequency conversion.
  • FIG. 7A is a view for showing a state in which a polar coordinate system is set on the frequency conversion image.
  • FIG. 7B is a graph for showing a method of extracting an index for determining whether or not a photoreceptor cell exists.
  • FIG. 8 is an example of a photographing position in a case where a region in which the photoreceptor cell exists is entirely photographed.
  • FIG. 9 is a flow chart for illustrating a processing procedure of an ophthalmic apparatus 20 according to a second embodiment of the present invention.
  • FIG. 10 is an example of a photographing position in a case where a boundary of the region in which the photoreceptor cell exists is photographed.
  • FIG. 11A is an example for illustrating the existence of the photoreceptor cell for each of divided images.
  • FIG. 11B is another example for illustrating the existence of the photoreceptor cell for each of the divided images.
  • FIG. 12 is a flow chart for illustrating a processing procedure of an ophthalmic apparatus.20 according to a third embodiment of the present invention.
  • FIG. 13 is an example of a photographing position in a case where an AO-SLO image with a different resolution is used to photograph the boundary portion in detail.
  • FIG. 14 is an example of an image shown by
  • a first embodiment of the present invention when an image of a retina is to be acquired by an AO-SLO, photographing is started from a region in which a photoreceptor cell is likely to exist. A region first designated is photographed. Then, the photographing region is shifted in the side to side direction and the up and down direction while maintaining overlapped amount of the photographing regions, and the
  • photographing is performed at the shifted positions by the AO-SLO. Through the steps described above, a region in which the photoreceptor cell exists is photographed. Those processes are described in detail below .
  • the region in which the photoreceptor cell is likely to exist is first specified based on a type of disease and a progress in past photographing.
  • a peripheral portion of a fundus is the region in which the photoreceptor cell is likely to exist.
  • the fovea is the region in which the photoreceptor cell is likely to exist.
  • a position in a region in which the photoreceptor cell existed in the past photographing, which is far from a boundary of the region, is the region in which the photoreceptor cell is likely to exist .
  • the photographing region is shifted from side to side and up and down while maintaining overlapped amount of the photographing regions., and the photographing by the AO-SLO is continued. Moreover, if a region in which a photoreceptor cell defect occurs is found at this time, a position of the region is set to a boundary image position in which a boundary of the defect region exists, and the boundary search is performed again from a peripheral portion of the initial photographing position in different directions.
  • the AO-SLO image of the region in which the photoreceptor cell is likely to exist is first
  • the photoreceptor cell layer can be represented with high image quality when the photoreceptor cell layer exists, and with regard to the region in which the photoreceptor cell defect occurs, the boundary thereof can be correctly represented.
  • FIG. 3 is a schematic view for showing an arrangement on the fundus of each of the plurality of AO-SLO images acquired by the AO-SLO used in this embodiment and a wide-field SLO (WF-SLO) image.
  • WF-SLO wide-field SLO
  • FIG. 4 is a view of a fixation target map for operating a displaying position of the fixation target on the fixation lamp.
  • the fixation target is first displayed in a state in which the center of the fixation target map of FIG. 4 is selected. This position is hereinafter referred to as "reference position". At this time, the eye to be inspected staring at the displayed fixation target is photographed so that the AO-SLO image of the vicinity of a macula and the WF-image can be photographed.
  • the WF-SLO image is an image acquired by photographing a wide field of the retina with an image size of 8 mm>6 mm and a pixel number of 533x400, and an entire image of the retina is acquired as the WF-SLO image.
  • the AO- SLO images are associated with the WF-SLO image, and thus it is displayed that at which positions on the entire retina the AO-SLO images each having a narrow angle of field are photographed.
  • the AO-SLO images include three kinds of AO-SLO images having the same pixel number of 400*400 and sizes of the photographing regions of 1.7 mm* 1.7 mm, 0.82 mmxO.82 mm, and 0.34 mmxO.34 mm, respectively. That is, the photographing is performed with such a condition that allows the AO- SLO images to be acquired with three kinds of
  • the AO-SLO image having the photographing region of 1.7 mm* 1.7 mm is an L image
  • the AO-SLO image having the photographing region of 0.82 mmxO.82 mm is an M image
  • the AO-SLO image having the photographing region of 0.34 mmxO.34 mm is an S image.
  • a photographing time and frame rate of the AO-SLO image can be changed.
  • the frame rate is 32 frames per second and the photographing time is 2 seconds, and hence the AO-SLO image is formed by 64 images.
  • the AO-SLO image and the WF-SLO image are referred to. as "planar image".
  • FIG. 1 is a block diagram for illustrating a functional configuration of an ophthalmic apparatus 20 according to the first embodiment.
  • the ophthalmic apparatus 20 includes a processing apparatus 10, a planar image photographing apparatus 3, and a display part 4.
  • the processing apparatus 10 performs an image process and computes an aberration correction coefficient.
  • the planar image photographing apparatus 3 photographs the planar images of the AO-SLO image and the WF-SLO image as. an SLO that has a wide field and is not subjected to aberration correction.
  • the display part 4 displays the
  • the ophthalmic apparatus 20 is also connected to an external database (DB) 1.
  • the DB 1 stores an image and patient data acquired by other modalities, and a past image and data acquired by the planar image photographing apparatus 3. Moreover, the ophthalmic apparatus 20 can acquire those pieces of. data and the like as necessary.
  • the processing apparatus 10 includes an image acquiring part 100, an information acquiring part 110, a control part 120, a memory part 130, an image processing part 140, and an output part 150.
  • the image acquiring part 100 acquires the planar image acquired by the planar image photographing apparatus 3.
  • the acquired planar image is stored in the memory part 130 through the control part 120.
  • the image acquiring part 100 can also acquire an OCT tomographic image acquired by an OCT 2 via the database 1.
  • the image acquiring part 100 of this embodiment corresponds to an image acquiring unit configured to acquire the planar image of the fundus of the eye to be inspected.
  • the information acquiring part 110 acquires an input made by the user, measurement data for the aberration correction, various kinds of data stored in the database 1, and the like.
  • the image processing part 140 includes a position
  • the position alignment part 141 associates a region on the WF-SLO image and the AO-SLO image with each other, and aligns positions of those images.
  • the boundary acquiring part 142 further
  • segmentalizes a single AO-SLO image segmentalizes a single AO-SLO image, and determines whether or not the photoreceptor cell is included in each of those segmentalized regions. Further, it is determined whether or not the original AO-SLO image includes the boundary of the region in which the
  • a boundary acquiring unit configured to acquire, from the planar image, the boundary between an existence region in which the photoreceptor cell exists and a region in which the photoreceptor cell does not exist in the eye to be inspected .
  • the photographing position setting part 143 sets, based on whether or not the boundary exists, a photographing region in which the AO-SLO image is to be next acquired.
  • the connecting part 144 associates, for all of the photographed images including the boundary of the photoreceptor cell, the individual photographing regions with each other on the WF-SLO image based on the positon alignment result of each of the AO-SLO images.
  • the connecting part 144 of this embodiment corresponds, as described later, to a setting unit configured to connect the boundaries acquired from the plurality of AO-SLO images, to thereby define the existence region of the photoreceptor cell on the fundus .
  • the image processing part 140 may extract the photoreceptor cell layer or an RPE layer from the OCT tomographic image acquired from the OCT 2, to thereby specify a region in which the photoreceptor cell layer is normal. Moreover, in this case, if the photoreceptor cell is damaged, for example, has a defect, it is preferred to acquire a photoreceptor cell layer supposed to include the damaged photoreceptor cell. Further, a position on the WF-SLO image to which the OCT tomographic image corresponds is determined, and a region determined to include the normal
  • a control method for the AO-SLO may be determined from a state of the photoreceptor cell layer at a photographing position designated by the user.
  • the output part 150 outputs, to a monitor or the like, the region in which the normal photoreceptor cell layer exists, which is associated with the WF-SLO image, and designates to a modality a control method for
  • Step S210 the image acquiring part 100 acquires a planar image of a retina of an eye to be inspected photographed by the planar image photographing
  • the acquired planar image is stored in the memory part 130 through the control part 120.
  • the planar image is not limited to the WF-SLO image and may be an AO-SLO image photographed in a wide field.
  • Step S220 the information acquiring part 110 first performs the photographing by the AO-SLO, and acquires information on, as a photographing position to be used when an AO-SLO image is photographed, a position in which a photoreceptor cell is likely to exist.
  • This initial photographing position may be set by a user based on a past photographed image and information provided by other modalities. Alternatively, the initial photographing position may be set in advance for each disease, and the set position may be acquired. Specifically, in the case of retinitis pigmentosa, a fovea may be set to the initial photographing position. Then, the acquired photographing position is stored in the memory part 130 through the control part 120.
  • Step S230 the image acquiring part 100 transmits the information on the photographing position acquired in Step S220 to the planar image photographing
  • the planar image photographing apparatus 3 photographs an AO-SLO image on a fundus at the position. Then, the image acquiring part 100 acquires the AO-SLO image and stores the acquired AO- SLO image in the memory part 130 through the control part 120.
  • Step S240 the position alignment part 141 aligns positions of the WF-SLO image acquired in Step S210 and the AO-SLO image acquired in Step S230.
  • the result of aligning in this manner the position of the AO-SLO image and the position on the WF-SLO is stored in the memory part 130 through the control part 120.
  • Step S250 the image processing part 140 determines whether or not there exists a boundary between a region in which the photoreceptor cell exists and a region in which a photoreceptor cell defect occurs in the AO-SLO image stored in the memory part 130.
  • Step S510 the boundary acquiring part 142 divides the acquired AO-SLO image into small regions.
  • a plurality of methods are conceivable as a method of dividing the image into regions.
  • accuracy of frequency conversion in a later step may be
  • the boundary between the region in which the photoreceptor cell exists and the region in which the photoreceptor cell does not exist may be determined more accurately.
  • an AO-SLO image having 400x400 pixel is divided into 16 regions each having 100x100 pixel.
  • DFT discrete Fourier transform
  • the number of pixels in the. small region is required to be 2 to the n-th power. Therefore, in this embodiment, preprocessing such as padding is performed on the region having 100x100 pixel so as to have 128x128 pixel, and then the DFT is performed thereon.
  • the regions obtained by dividing the image in this manner are stored in the memory part 130 through the control part 120.
  • Step S520 the boundary acquiring part 142 performs the frequency conversion on each of the 16 regions obtained by the division in Step S510.
  • FIG. 6A and FIG. 6B are each an example for showing a case where the regions after the division are subjected to the frequency conversion.
  • FIG. 6A in a region in which the photoreceptor cell can be confirmed, there is a ring-like structure in a spatial frequency corresponding to a density of the photoreceptor cell.
  • FIG. 6B in a region in which the photoreceptor cell cannot be confirmed, the ring-like structure cannot be confirmed.
  • the frequency conversion image acquired in this manner is stored in the memory part 130 through the control part 120.
  • Step S530 the boundary acquiring part 142
  • Step S510 determines, for each of the regions obtained by the division in Step S510, whether or not the photoreceptor cell exists therein. Specifically, based on the
  • FIG. 7B is a graph for showing the function I (r) acquired for the image shown in FIG. 6A. That is, the boundary acquiring unit acquires a
  • the density of the photoreceptor cell is determined based on a relationship between a distance from the fovea and a generally known photoreceptor cell density.
  • a report based on anatomical data described in, for example, NPL 1 is given. Therefore, it is also possible to determine the distance from the fovea of the AO-SLO image based on the position
  • one method uses a value of a function I (r 0 ) that corresponds to a spatial frequency r 0 corresponding to the density of the photoreceptor cell.
  • another method takes great dependence of the value of I (ro) on a contrast of the AO-SLO image into account, and uses a ratio between the spatial frequency ro corresponding to the density of the photoreceptor cell and a ratio between the function I (ro) and a function I (2r 0 ) corresponding to a spatial frequency 2ro that is twice the value of the spatial frequency r 0 .
  • I(r 0 ) the index is not limited thereto.
  • hen I (r 0 ) is larger than a certain threshold I E , it is determined that the photoreceptor cell exists, whereas when I(r 0 ) is smaller than a certain threshold I L , it is determined that the photoreceptor cell does not exist.
  • a state having the value of I (r 0 ) that falls between the I E and the I L is determined to be undetermined.
  • Step S540 the boundary acquiring part 142
  • Step S530 determines whether or not the boundary exists based on the existence determination result of the photoreceptor cell for each of the regions determined in Step S530. Specifically, when the photoreceptor cells exist in all of the 16 regions, it is determined that the boundary does not exist. Moreover, when the 16 regions include both of the region in which the photoreceptor cell exists and the region in which the photoreceptor cell does not exist, it is determined that the boundary exists. In this case, when there exists the region that is located between the region in which the
  • the undetermined region is
  • the adjacent region in which the photoreceptor cell exists is determined as the boundary region. Further, when it is determined that no photoreceptor cell exists in all of the 16 regions, the photographing is determined as having failed.
  • Step S260 the photographing position setting part 143 sets a next photographing position based on the boundary determination result determined in Step S540.
  • FIG. 8 is a schematic view for showing a fundus in the case of retinitis pigmentosa. It is known that retinitis pigmentosa causes the photoreceptor cell defect from the peripheral portion.
  • a circular gray region shown in the vicinity of the fovea is the region in which the photoreceptor cell exists, and the peripheral portion of the circular gray region is the region in which the photoreceptor cell defect occurs.
  • Step S220 a position of the fovea is set as the initial photographing position.
  • An AO-SLO image photographed at the position of the fovea (image at an image position 1) is processed following Step S510 to Step S540.
  • the next photographing position is set to a position on the left of the current photographing position, which includes a region overlapping with the current photographing position (image position 2).
  • a dimension of the overlapped region is about 15% of a
  • photographing field angle is herein 400x400 pixel and hence the dimension is set to 60 pixels.
  • the dimension is not limited to this value and can be appropriately changed depending on a fixation state of the eye to be . inspected and whether or not a tracking system is provided.
  • the next photographing position of the AO-SLO image set in this manner is stored in the memory part 130 through the control part 120. That is, the image acquiring part 100 includes a module functioning as a
  • photographing position setting unit configured to set, depending on the position of the boundary acquired by the boundary acquiring unit, a photographing position on the fundus at which the planar image is to be photographed.
  • Step S240 to Step S250 are performed on an image photographed at the image position 2. In this case, when it is determined that the boundary does not exist in the image similarly to the image
  • photographing position is set to a position on the left of the image position 2, which is overlapped therewith by 60 pixels (image position 3) similarly to the above- mentioned case.
  • the similar processes are performed on an image photographed at the image position 3.
  • the next photographing position is set to a position on the right of the image 1, which is overlapped therewith by 60 pixels (image position 4).
  • the similar processes are performed on an image photographed at the image position 4.
  • the next photographing position is set to a position on the right of the image position 4 (image position 5) .
  • the similar processes are performed on an image photographed at the image position 6.
  • the next photographing position is set to an image position 7 above the image position 6, which is overlapped therewith by 60 pixels.
  • the photographing position is shifted from side to side and up and down until an image including the boundary of the region in which the photoreceptor cell exists is obtained.
  • the cruciform photographing region With respect to the cruciform photographing region
  • positions in a rectangular region covering the cross shape are set to the next photographing positions in turns.
  • Step S540 when it is determined that the photographing has failed in Step S540, there is displayed a
  • photographing position is set to each of positions that are adjacent to the image in the side to side direction and the up and down direction and have not been
  • Step S270 in a state in which the next photographing position is not set.
  • Step S270 the control part 143 acquires the next photographing position set by the photographing
  • Step S260 Information on a photographing position acquired in Step S260 is transmitted to the planar image
  • Step S240 the flow continues to Step S240.
  • Step S280 for all of the photographed images including the boundary of the photoreceptor cell, the connecting part 144 acquires, from the memory part 130, the information on the region in which the photographed images including the boundary of the photoreceptor cell.
  • the connecting part 144 described herein also forms a position alignment unit configured to associate with each other the acquired boundary and the
  • Step S210 to Step S250 and Step S270 are the same as those described with reference to FIG. 2, and hence the description thereof is omitted.
  • Step S1060 the photographing position setting part 143 sets a next photographing position based on the boundary determination result determined in Step S540.
  • FIG. 10 is a schematic view for showing a fundus in the case of retinitis pigmentosa. It is known that
  • retinitis pigmentosa causes the photoreceptor cell defect from the peripheral portion.
  • a circular gray region shown in the vicinity of the fovea is the region in which the photoreceptor cell exists, and the peripheral portion of the circular gray region is the region in which the photoreceptor cell defect occurs .
  • Step S220 a position of the fovea is set as the
  • Step S510 An AO-SLO image photographed at the position of the fovea (image at the image position 1) is processed following Step S510 to Step S540. There is described a process for proceeding in the case where it is determined that the boundary does not exist in the image position 1 as a result of the process above.
  • the next photographing position is set to a position on the left of the current photographing position, which includes a region overlapping with the current photographing position (image position 2).
  • a dimension of the overlapped region is about 15% of a
  • photographing field angle is herein 400x400 pixel and hence the dimension is set to 60 pixels.
  • the dimension is not limited to this value and can be appropriately changed depending on a fixation state of the eye to be inspected and whether or not a tracking system is provided.
  • next photographing position of the AO-SLO image set in this manner is stored in the memory part 130 through the control part 120.
  • Step S240 to Step S250 are performed on an AO-SLO image photographed at the image position 2
  • the next photographing position is set to a position on the left of the image position 2, which is overlapped therewith by 60 pixels (image position 3) similarly to the above-mentioned case.
  • the similar processes are performed on an AO-SLO image photographed at the image position 3.
  • the boundary search is started from the fovea, and the image acquired at the image position 3 is set to a search start image as an image first determined to include the boundary.
  • the next photographing position is set to an image position 4 adjacent to the upper portion of the image position 3, which is along the extending
  • the selection method for an adjacent position is herein determined based on the existence determination result of the photoreceptor cell determined for each of the regions in Step S250.
  • FIG. 11A is the illustration of the result of the
  • a dark color region represents a region determined to include the photoreceptor cell
  • a light color region represents a region determined to include no photoreceptor cell
  • a light color region represents a region in which the existence of the photoreceptor cell cannot be
  • the region representing the boundary extends from a lower right corner so as to reach substantially the center of an upper side, and hence it is expected that the boundary crosses the upper side of the image.
  • the next photographing position is set to a position above the image position 3, which is overlapped
  • the next photographing position is set to a position overlapped by 60 pixels in a direction of the corresponding side.
  • FIG. 11B is the similar illustration of the
  • a boundary expected position is an upper right corner position. In this case, the next photographing
  • the photographing position set in this manner is represented as an image position 5 of FIG. 10.
  • Step S540 when it is determined that the photographing has failed in Step S540, it is preferred to display a new photographing position moved from the set photographing position toward a region in which the photoreceptor cell is more likely to exist.
  • the method involving photographing all of the regions in which the photoreceptor cells exist while searching for the boundary of the photoreceptor cell defect there is described the method involving performing the photographing along the boundary between the region in which the photoreceptor cell defect occurs and the region in which the photoreceptor cell exists.
  • the region in which the photoreceptor cell exists is large, a large number of regions are to be photographed in the method of the first embodiment, which causes a burden on the eye to be inspected such as due to an increase in examination time.
  • the number of photographing regions is reduced compared to the first embodiment, but there is a risk that the boundary cannot be correctly retrieved when the shape of the boundary is irregular.
  • photoreceptor cells exist by a method similar to that of the first embodiment and further photographing, with a high resolution AO-SLO, only a target region in which the boundary is desired to be clearly obtained and the like.
  • Step S210 to Step S280 are the same as those described with reference to FIG. 2, and hence the description thereof is -omitted.
  • Step S220 is subjected to the aberration correction but is a low resolution image
  • an AO-SLO image acquired in Step S1220 is a high resolution image subjected to the aberration correction.
  • the low resolution AO-SLO image has a level of resolution with which it is difficult to individually detect the photoreceptor cells but the region in which the
  • the above-mentioned M image of the planar image corresponds to the low resolution AO-SLO image, for example.
  • the high resolution AO-SLO image is an image having a level of resolution with which the photoreceptor cells can be individually detected, and the above-mentioned S image of the planar image corresponds thereto.
  • the above-mentioned classification is not a limitation, and a resolution to be used can be changed depending on a dimension and range of a photographed disease, and a photoreceptor cell density desired to be resolved.
  • Step S1220 the information acquiring part 110 acquires, based on the information on the region in which the photoreceptor cell exists and the boundary thereof, which is displayed to the user in Step S280, a photographing position of a high resolution planar image that has been selected by the user.
  • the acquired photographing position is stored in the memory part 130 through the control part 120.
  • Step S1230 the image acquiring part 100 transmits the information on the photographing position acquired in Step S1220 to the planar image photographing
  • the planar image photographing apparatus 3 photographs a high resolution AO-SLO image at the position. Then, the image acquiring part 100 acquires the high resolution AO-SLO image from the planar image photographing apparatus 3.
  • Step S1220 corresponds to the region including the boundary acquired in Step S280
  • the photographing is performed at the same photographing position with focus positions set at a plurality of positions.
  • the photographing is performed at the focus position acquired by the aberration correction (focus reference image), and images are also acquired at focus positions moved back and forth by 10 ⁇ in 3 steps each. That is, the photographing is performed at the focus positions moved toward a vitreous body side by 10 ⁇ , 20 ⁇ , and 30 ⁇ and toward a choroid side by 10 ⁇ , 20 ⁇ , and 30 ⁇ .
  • 7 images are acquired as a total.
  • This operation is performed because, in the vicinity of the boundary, reflection of beacon light required for the aberration correction may be unstable depending on a state of the photoreceptor cell layer, and hence it may be difficult to perform the photographing at a correct focus position.
  • a represented state of the photoreceptor cell layer is significantly affected by a shift, of the focus position.
  • the images are acquired at the plurality of focus positions as
  • Step S220 when the photographing position acquired in Step S1220 is located outside of the boundary of the region in which the photoreceptor cell exists, which, is acquired in Step S280, that is, when the photographing position acquired in Step S1220 is located in the region determined to include no photoreceptor cell, an alert indicating the fact is displayed to the user. Then, there are displayed alternatives such as performing the normal
  • the acquired high resolution AO-SLO image group is stored in the memory part 130 through the control part 120.
  • Step S1240 the position alignment part 141 aligns positions of the WF-SLO image acquired in Step S210 and each of the images in the high resolution AO-SLO image group acquired in Step S1230. The result of
  • Step S1250 the image processing part 140 determines whether or not, in the AO-SLO image group stored in the memory part 130, there exists the boundary between the region in which the photoreceptor cell exists and the region in which the photoreceptor cell defect occurs. Note that, the details of the process performed in this step are the same as those of the processes of Step S510 to Step S540 of FIG. 5, and hence the description thereof is omitted. Note that, the process is
  • Step S1230 the existence determination of the photoreceptor cell is performed on the region of each of the images. Then, the regions in each of which the photoreceptor cells exist are connected to each other and a boundary in a case where a dimension of the region in which the photoreceptor cell exists is the maximum is determined as a true boundary.
  • FIG. 13 is a schematic view for showing an arrangement of each of the images in the high resolution AO-SLO image group on the WF-SLO image photographed in this embodiment.
  • a second planar image having a resolution different from that of the planar image acquired in the steps previous to this step is acquired.
  • the second planar image is acquired by the planar image photographing apparatus 3 that is increased in
  • the second image acquiring unit perform the photographing at the boundary of the photoreceptor cell
  • the boundary acquiring part 142 acquire again the boundary of the existence region of the photoreceptor cell from each of the second planar images photographed at the plurality of focus positions.
  • the acquired second planar image includes the photoreceptor cell, the second image acquiring unit perform the photographing at the plurality of focus positions for the photographing position at which the second planar image is acquired. Moreover, in this case, it is more preferred that the boundary acquiring part 142 select again, from among the second planar images photographed at the plurality of focus positions, the second planar image to be connected by the connecting part 144.
  • the higher resolution AO-SLO image based on the region in which the photoreceptor cell exists and the boundary thereof that are acquired by the method described in the first embodiment.
  • the photographing position of the high resolution AO-SLO image includes the boundary of the region in which the photoreceptor cell exists.
  • a detailed photoreceptor cell image may be acquired with high probability if the focus position is correctly set. Therefore, the AO-SLO images are photographed at the plurality of focus positions in advance, to thereby acquire a plurality of AO-SLO images.
  • the number of the photographed AO-SLO images may be 7 as described in the third embodiment or the AO-SLO images may be photographed in a wider field. Alternatively, based on a history of the past
  • photographed may be set in advance.
  • the image may be divided into the regions as described in Step S510 of the first embodiment, and the regions having large indices may be connected to each other, to thereby generate the image.
  • each of the first embodiment to the fourth embodiment there is described the method involving, with the use of only the AO-SLO image at the time of photographing, photographing the region in which the photoreceptor cell exists and the boundary thereof while searching for the boundary between the region in which the photoreceptor cell exists and the region in which the photoreceptor cell defect occurs.
  • information on the region in which the photoreceptor cell is likely to exist may be obtained in advance.
  • the boundary of the photoreceptor cell obtained from the AO-SLO image photographed in the past may be referred to. That is, in the case of a fifth embodiment of the present invention, the boundary may be defined by acquiring the boundary based on the OCT tomographic image of the eye to be inspected, acquiring the boundary from the planar image of the eye to be inspected acquired in the past, or selecting the boundary from among the plurality of boundaries
  • FIG. 14 is an example in which the information on the boundary of the photoreceptor cell obtained in the past photographing is superimposed on the WF-SLO image to be displayed. In FIG. 14, there are shown regions
  • the region in which the photoreceptor cell exists is decreased at higher speed on the right side than on the left side, and hence it can be predicted that the boundary may exist on the inner side of the boundary in the last photographing.
  • photoreceptor cell obtained in the past photographing can be extracted as a region, and the extracted region can be determined as an analysis target region.
  • the photoreceptor cell included in the difference region exists in the second last photographing but causes the defect in the last photographing.
  • the boundary acquiring part 142 extracts a difference among the. plurality of boundaries acquired in the past as the analysis target region.
  • the mode can be switched between a mode using the information from other modalities and the past progress information in the photographing and a mode not using those pieces of information in the photographing.
  • Embodiment ( s ) of the present invention can also be realized by a computer of a system or apparatus that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be referred to more fully as a 'non-transitory computer-readable storage medium') to perform the functions of one or more of the above- described embodiment ( s ) and/or that includes one or more circuits (e.g., application specific integrated circuit (ASIC) ) for performing the functions of one or more of the above-described embodiment ( s ) , and by a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage medium to perform the functions of one or more of the above-described embodiment ( s ) and/or controlling the one or more circuits to perform the functions of one or more of the above-described embodiment (s) .
  • ASIC application specific integrated circuit
  • computer may comprise one or more processors (e.g., central processing unit (CPU) , micro processing unit (MPU) ) and may include a network of separate computers or separate processors to read out and execute the computer executable instructions.
  • processors e.g., central processing unit (CPU) , micro processing unit (MPU)
  • CPU central processing unit
  • MPU micro processing unit
  • the storage medium may include, for example, one or more of a hard disk, a random-access memory (RAM) , a read only memory (ROM) , a storage of distributed computing systems, an optical disk (such as a compact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)TM) , a flash memory device, a memory card, and the like.
  • RAM random-access memory
  • ROM read only memory
  • BD Blu-ray Disc

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Medical Informatics (AREA)
  • Animal Behavior & Ethology (AREA)
  • Veterinary Medicine (AREA)
  • Public Health (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Biophysics (AREA)
  • Ophthalmology & Optometry (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Pathology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Eye Examination Apparatus (AREA)

Abstract

Provided is an ophthalmic apparatus capable of accurately representing a photoreceptor cell in AO-SLO imaging. In an ophthalmic apparatus capable of WF-SLO and AO-SLO imaging, the following units are arranged: an image acquiring unit configured to acquire a planar image of a fundus of an eye to be inspected; a boundary acquiring unit configured to acquire, from the planar image, a boundary between an existence region in which a photoreceptor cell of the eye to be inspected exists and a region in which the photoreceptor cell does not exist; and a setting unit configured to set the existence region of the photoreceptor cell on the fundus based on the boundaries acquired from a plurality of the planar images.

Description

DESCRIPTION
Title of Invention: OPHTHALMIC APPARATUS AND CONTROL METHOD THEREFOR
Technical Field
[0001] he present invention relates to an ophthalmic
apparatus to be used in particular for an ophthalmic diagnosis and treatment, and a control method for the ophthalmic apparatus. More specifically, the present invention relates to an ophthalmic apparatus to be used in an ophthalmic diagnosis and treatment, which
processes an acquired image to set an appropriate photographing position, and a control method for the ophthalmic apparatus.
Background Art
[0002] An examination of a fundus of an eye is widely
performed for the purpose of a diagnosis in the early stage of a disease that usually ranks high in adult disease or cause of blindness. For the examination of the fundus of the eye, for example, a scanning laser ophthalmoscope (SLO) , which is an ophthalmic apparatus using a principle of a confocal laser microscope, is used. The SLO is an ophthalmic apparatus configured to perform a raster scanning on a fundus of an eye with laser light, which is measurement light, and acquire a planar image of the fundus of the eye from the
intensity of return light with a high resolution at a high speed. Moreover, in recent years, an adaptive optics SLO (AO-SLO) capable of acquiring a planar image with a high transverse resolution has been developed. In the AO-SLO, an aberration of an eye to be inspected is measured by a wavefront sensor in real time, and aberrations of measuring light and return right thereof generated in an eye to be inspected are corrected by a wavefront correction device of an adaptive optics system. With such an AO-SLO, it is further attempted to diagnose a disease and evaluate a drug response by extracting a photoreceptor cell in a retina with the use of an acquired planar image of the retina, and analyzing a density and distribution of the
photoreceptor cell.
[0003] In a case where a change in photoreceptor cell due to the progress of a disease or a drug response is
evaluated, it is required to detect the photoreceptor cell from an AO-SLO image acquired with a high
resolution. In order to photograph the photoreceptor cell, the focus is required to be set on a layer in which the photoreceptor cell exists, but a focus
position cannot be correctly set in some cases if the photoreceptor cell layer is damaged. In order to cope with such a case, for example, a method disclosed in PTL 1 is proposed. In the method, a determination process relating to a normal/abnormal region in visual performance is performed based on a tomographic image photographed by optical coherence tomography (OCT) .
The result of the determination process is superimposed on a front image of a fundus to be displayed, and a laser irradiation position for a laser treatment is acquired based on this display.
Citation List
Patent Literature
[0004] PTL 1: Japanese Patent Application Laid-Open No. 2012- 135550
PTL 2: Japanese Patent Application Laid-Open No. 2012- 213513
Non Patent Literature
[0005] PL 1: Human Photoreceptor Topography
CHRISTINE A. CURCIO, KENNETH R. SLOAN, ROBERT E. KALINA, AND ANITA E. HENDRICKSON
THE JOURNAL OF COMPARATIVE NEUROLOGY 292: 497-523
(1990)
Summary of Invention Technical Problem
[0006] hen a photoreceptor cell is detected by the AO-SLO, it is required to discriminate a case where the
photoreceptor cell is not represented because the photoreceptor cell has a defect from a case where the photoreceptor cell is not represented because of a photographing reason such as a focus shift. If the photoreceptor cell is affected by a disease, the photographing by the AO-SLO is difficult to be
performed in many cases. That is, it is difficult to determine whether the photoreceptor cell has not been correctly photographed or the photoreceptor cell having a defect has been photographed.
[0007] In the method disclosed in PTL 1, the tomographic image is acquired with the use of the OCT, to thereby obtain information relating to the photoreceptor cell from the image. Then, the information is used to determine a normal/abnormal region in visual performance.
Therefore, it is essential to concurrently use the OCT.
[0008] In a method disclosed in PTL 2, a photographing
position of a narrow angle or field image is set along a structure such as a blood vessel represented as a wide field image, and then photographing is performed. However, in the method, no consideration is made on a case where it is difficult to set a photographing position with the wide field image, such as when the photographing position is set for an existing range of the photoreceptor cell. Therefore, it is not suggested how to determine whether or not the photoreceptor cell is accurately photographed.
[0009] In view of the above-mentioned problems, the present invention has an object to provide an apparatus capable of more accurately representing a state of a
photoreceptor cell layer, and a control method for the apparatus .
Solution to Problem [0010] In order to solve the above-mentioned problems, an ophthalmic apparatus according to one embodiment of the present invention includes: an image acquiring unit configured to acquire a planar image of a fundus of an eye to be inspected; a boundary acquiring unit
configured to acquire, from the planar image, a
boundary between an existence region in which a
photoreceptor cell of the eye to be inspected exists and a region in which the photoreceptor cell does not exist; and a setting unit configured to set the
existence region of the photoreceptor cell on the fundus based on the boundaries acquired from a
plurality of the planar images.
Advantageous Effects of Invention
[0011] According to the one embodiment of the present
invention, in a fundus inspection, it is possible to represent a more accurate state of the photoreceptor cell layer.
[0012] Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings. Brief Description of Drawings
[0013] FIG. 1 is a block diagram for illustrating a functional configuration of an ophthalmic apparatus 20 according to a first embodiment of the present invention.
FIG. 2 is a flow chart for illustrating a processing procedure of the ophthalmic apparatus 20 according to the first embodiment of the present invention.
FIG. 3 is a schematic view for showing a state in which a plurality of AO-SLO images are displayed on a WF-SLO image .
FIG. 4 is a schematic view of a fixation target map for designating a fixation position.
FIG. 5 is a flow chart for illustrating the details of boundary determination in Step S250 illustrated in FIG. 2. FIG. 6A is an example of a frequency conversion image acquired by subjecting the AO-SLO image to frequency conversion .
FIG. 6B is another example of the frequency conversion image acquired by subjecting the AO-SLO image to the frequency conversion.
FIG. 7A is a view for showing a state in which a polar coordinate system is set on the frequency conversion image.
FIG. 7B is a graph for showing a method of extracting an index for determining whether or not a photoreceptor cell exists.
FIG. 8 is an example of a photographing position in a case where a region in which the photoreceptor cell exists is entirely photographed.
FIG. 9 is a flow chart for illustrating a processing procedure of an ophthalmic apparatus 20 according to a second embodiment of the present invention.
FIG. 10 is an example of a photographing position in a case where a boundary of the region in which the photoreceptor cell exists is photographed.
FIG. 11A is an example for illustrating the existence of the photoreceptor cell for each of divided images.
FIG. 11B is another example for illustrating the existence of the photoreceptor cell for each of the divided images.
FIG. 12 is a flow chart for illustrating a processing procedure of an ophthalmic apparatus.20 according to a third embodiment of the present invention.
FIG. 13 is an example of a photographing position in a case where an AO-SLO image with a different resolution is used to photograph the boundary portion in detail. FIG. 14 is an example of an image shown by
superimposedly displaying regions acquired with time, in each of which the photoreceptor cell exists.
Description of Embodiments [0014 ] Preferred embodiments of the present invention will now be described in detail in accordance with the
accompanying drawings .
[0015] (First Embodiment)
In a first embodiment of the present invention, when an image of a retina is to be acquired by an AO-SLO, photographing is started from a region in which a photoreceptor cell is likely to exist. A region first designated is photographed. Then, the photographing region is shifted in the side to side direction and the up and down direction while maintaining overlapped amount of the photographing regions, and the
photographing is performed at the shifted positions by the AO-SLO. Through the steps described above, a region in which the photoreceptor cell exists is photographed. Those processes are described in detail below .
[0016] In actual steps, the region in which the photoreceptor cell is likely to exist is first specified based on a type of disease and a progress in past photographing. Specifically, in a case of a disease such as Stargardt disease known for causing a photoreceptor cell defect from a fovea, a peripheral portion of a fundus is the region in which the photoreceptor cell is likely to exist. Moreover, in a case of a disease such as retinitis pigmentosa known for causing a photoreceptor cell defect from the peripheral portion, the fovea is the region in which the photoreceptor cell is likely to exist. Moreover, if the retina was photographed by the AO-SLO in the past, a position in a region in which the photoreceptor cell existed in the past photographing, which is far from a boundary of the region, is the region in which the photoreceptor cell is likely to exist .
[0017] In this way, the photographing is started from the
region in which the photoreceptor cell is likely to exist to confirm that the photoreceptor cell exists in the region. Then, the photographing region is shifted from side to side and up and down while maintaining overlapped amount of the photographing regions., and the photographing by the AO-SLO is continued. Moreover, if a region in which a photoreceptor cell defect occurs is found at this time, a position of the region is set to a boundary image position in which a boundary of the defect region exists, and the boundary search is performed again from a peripheral portion of the initial photographing position in different directions.
[ 0018 ] Through the photographing of the region in which the photoreceptor cell exists in this manner, when a photoreceptor cell layer exists, the photoreceptor cell layer can be represented with high image quality, and with regard to the region in which the photoreceptor cell defect occurs, the boundary of that region can be correctly represented. That is, in the present
invention, the AO-SLO image of the region in which the photoreceptor cell is likely to exist is first
photographed based on the type of disease and the progress in the past photographing, and the AO-SLO images are then photographed while searching for the boundaries on the fundus in which the photoreceptor cell exists. With this, the photoreceptor cell layer can be represented with high image quality when the photoreceptor cell layer exists, and with regard to the region in which the photoreceptor cell defect occurs, the boundary thereof can be correctly represented.
[0019] <Planar Image>
FIG. 3 is a schematic view for showing an arrangement on the fundus of each of the plurality of AO-SLO images acquired by the AO-SLO used in this embodiment and a wide-field SLO (WF-SLO) image. In the AO-SLO, a position of a fixation target displayed on a fixation lamp is changed so that an eye to be inspected is photographed in a state of staring at different positions, to thereby photograph different positions on the retina. FIG. 4 is a view of a fixation target map for operating a displaying position of the fixation target on the fixation lamp.
[0020] The fixation target is first displayed in a state in which the center of the fixation target map of FIG. 4 is selected. This position is hereinafter referred to as "reference position". At this time, the eye to be inspected staring at the displayed fixation target is photographed so that the AO-SLO image of the vicinity of a macula and the WF-image can be photographed.
[0021] The WF-SLO image is an image acquired by photographing a wide field of the retina with an image size of 8 mm>6 mm and a pixel number of 533x400, and an entire image of the retina is acquired as the WF-SLO image. The AO- SLO images are associated with the WF-SLO image, and thus it is displayed that at which positions on the entire retina the AO-SLO images each having a narrow angle of field are photographed. The AO-SLO images include three kinds of AO-SLO images having the same pixel number of 400*400 and sizes of the photographing regions of 1.7 mm* 1.7 mm, 0.82 mmxO.82 mm, and 0.34 mmxO.34 mm, respectively. That is, the photographing is performed with such a condition that allows the AO- SLO images to be acquired with three kinds of
resolutions. In this case, the AO-SLO image having the photographing region of 1.7 mm* 1.7 mm is an L image, the AO-SLO image having the photographing region of 0.82 mmxO.82 mm is an M image, and the AO-SLO image having the photographing region of 0.34 mmxO.34 mm is an S image. A photographing time and frame rate of the AO-SLO image can be changed. In this case, the frame rate is 32 frames per second and the photographing time is 2 seconds, and hence the AO-SLO image is formed by 64 images. The AO-SLO image and the WF-SLO image are referred to. as "planar image".
[ 0022 ] <Configuration of Ophthalmic Apparatus>
FIG. 1 is a block diagram for illustrating a functional configuration of an ophthalmic apparatus 20 according to the first embodiment.
In FIG. 1, the ophthalmic apparatus 20 includes a processing apparatus 10, a planar image photographing apparatus 3, and a display part 4. The processing apparatus 10 performs an image process and computes an aberration correction coefficient. The planar image photographing apparatus 3 photographs the planar images of the AO-SLO image and the WF-SLO image as. an SLO that has a wide field and is not subjected to aberration correction. The display part 4 displays the
computation result of the processing apparatus 10 and the like to a user. In this embodiment, the ophthalmic apparatus 20 is also connected to an external database (DB) 1. The DB 1 stores an image and patient data acquired by other modalities, and a past image and data acquired by the planar image photographing apparatus 3. Moreover, the ophthalmic apparatus 20 can acquire those pieces of. data and the like as necessary.
[0023] The processing apparatus 10 includes an image acquiring part 100, an information acquiring part 110, a control part 120, a memory part 130, an image processing part 140, and an output part 150. The image acquiring part 100 acquires the planar image acquired by the planar image photographing apparatus 3. The acquired planar image is stored in the memory part 130 through the control part 120. Moreover, in this embodiment, the image acquiring part 100 can also acquire an OCT tomographic image acquired by an OCT 2 via the database 1. The image acquiring part 100 of this embodiment corresponds to an image acquiring unit configured to acquire the planar image of the fundus of the eye to be inspected. The information acquiring part 110 acquires an input made by the user, measurement data for the aberration correction, various kinds of data stored in the database 1, and the like.
[0024] The image processing part 140 includes a position
alignment part 141, a boundary acquiring part 142, a photographing position setting part 143, and a
connecting part 144. The position alignment part 141 associates a region on the WF-SLO image and the AO-SLO image with each other, and aligns positions of those images. The boundary acquiring part 142 further
segmentalizes a single AO-SLO image, and determines whether or not the photoreceptor cell is included in each of those segmentalized regions. Further, it is determined whether or not the original AO-SLO image includes the boundary of the region in which the
photoreceptor cell exists based on the determination result. The boundary acquiring part 142 of this
embodiment corresponds to a boundary acquiring unit configured to acquire, from the planar image, the boundary between an existence region in which the photoreceptor cell exists and a region in which the photoreceptor cell does not exist in the eye to be inspected .
[0025] The photographing position setting part 143 sets, based on whether or not the boundary exists, a photographing region in which the AO-SLO image is to be next acquired. Moreover, the connecting part 144 associates, for all of the photographed images including the boundary of the photoreceptor cell, the individual photographing regions with each other on the WF-SLO image based on the positon alignment result of each of the AO-SLO images. The connecting part 144 of this embodiment corresponds, as described later, to a setting unit configured to connect the boundaries acquired from the plurality of AO-SLO images, to thereby define the existence region of the photoreceptor cell on the fundus .
[0026] Note that, the image processing part 140 may extract the photoreceptor cell layer or an RPE layer from the OCT tomographic image acquired from the OCT 2, to thereby specify a region in which the photoreceptor cell layer is normal. Moreover, in this case, if the photoreceptor cell is damaged, for example, has a defect, it is preferred to acquire a photoreceptor cell layer supposed to include the damaged photoreceptor cell. Further, a position on the WF-SLO image to which the OCT tomographic image corresponds is determined, and a region determined to include the normal
photoreceptor cell is associated with the region on the WF-SLO image. Then, based on the above-mentioned layer analysis result, a control method for the AO-SLO may be determined from a state of the photoreceptor cell layer at a photographing position designated by the user.
[0027] The output part 150 outputs, to a monitor or the like, the region in which the normal photoreceptor cell layer exists, which is associated with the WF-SLO image, and designates to a modality a control method for
photographing an AO-SLO image of the photographing position designated by the user.
[ 0028 ] <Processing Procedure of Image Processing Apparatus> Next, a processing procedure of the ophthalmic
apparatus 20 according to this embodiment is described referring to a flowchart of FIG. 2.
[0029]<Step S210>
In Step S210, the image acquiring part 100 acquires a planar image of a retina of an eye to be inspected photographed by the planar image photographing
apparatus 3. Then, the acquired planar image is stored in the memory part 130 through the control part 120. Note that, in this embodiment, a case where the planar image is a WF-SLO image is described. However, the planar image is not limited to the WF-SLO image and may be an AO-SLO image photographed in a wide field.
[0030]<Step S220>
In Step S220, the information acquiring part 110 first performs the photographing by the AO-SLO, and acquires information on, as a photographing position to be used when an AO-SLO image is photographed, a position in which a photoreceptor cell is likely to exist. This initial photographing position may be set by a user based on a past photographed image and information provided by other modalities. Alternatively, the initial photographing position may be set in advance for each disease, and the set position may be acquired. Specifically, in the case of retinitis pigmentosa, a fovea may be set to the initial photographing position. Then, the acquired photographing position is stored in the memory part 130 through the control part 120.
[0031]<Step S230>
In Step S230, the image acquiring part 100 transmits the information on the photographing position acquired in Step S220 to the planar image photographing
apparatus 3 so that the planar image photographing apparatus 3 photographs an AO-SLO image on a fundus at the position. Then, the image acquiring part 100 acquires the AO-SLO image and stores the acquired AO- SLO image in the memory part 130 through the control part 120.
[0032]<Step S240>
In Step S240, the position alignment part 141 aligns positions of the WF-SLO image acquired in Step S210 and the AO-SLO image acquired in Step S230. The result of aligning in this manner the position of the AO-SLO image and the position on the WF-SLO is stored in the memory part 130 through the control part 120.
[0033]<Step S250>
In Step S250, the image processing part 140 determines whether or not there exists a boundary between a region in which the photoreceptor cell exists and a region in which a photoreceptor cell defect occurs in the AO-SLO image stored in the memory part 130.
[0034] Next, with reference to a flow chart of FIG. 5, a
procedure of the determination of the existence of the boundary in Step S250 is described.
[0035]<Step S510>
In Step S510, the boundary acquiring part 142 divides the acquired AO-SLO image into small regions. In this case, a plurality of methods are conceivable as a method of dividing the image into regions. However, when a dimension of one region is small, accuracy of frequency conversion in a later step may be
deteriorated. On the other hand, when the dimension of one region is small and the number of regions is thus increased, the boundary between the region in which the photoreceptor cell exists and the region in which the photoreceptor cell does not exist may be determined more accurately.
[0036] In the light of the foregoing, in this embodiment, an AO-SLO image having 400x400 pixel is divided into 16 regions each having 100x100 pixel. In this case, when a two-dimensional discrete Fourier transform (DFT) is performed on the image, the number of pixels in the. small region is required to be 2 to the n-th power. Therefore, in this embodiment, preprocessing such as padding is performed on the region having 100x100 pixel so as to have 128x128 pixel, and then the DFT is performed thereon.
The regions obtained by dividing the image in this manner are stored in the memory part 130 through the control part 120.
[0037]<Step S520>
In Step S520, the boundary acquiring part 142 performs the frequency conversion on each of the 16 regions obtained by the division in Step S510.
[0038] FIG. 6A and FIG. 6B are each an example for showing a case where the regions after the division are subjected to the frequency conversion. As shown in FIG. 6A, in a region in which the photoreceptor cell can be confirmed, there is a ring-like structure in a spatial frequency corresponding to a density of the photoreceptor cell. On the contrary, as shown in FIG. 6B, in a region in which the photoreceptor cell cannot be confirmed, the ring-like structure cannot be confirmed.
The frequency conversion image acquired in this manner is stored in the memory part 130 through the control part 120.
[0039]<Step S530>
In Step S530, the boundary acquiring part 142
determines, for each of the regions obtained by the division in Step S510, whether or not the photoreceptor cell exists therein. Specifically, based on the
frequency conversion result determined in Step S520, whether or not the photoreceptor cell exists is
determined with intensity of a signal of the spatial frequency corresponding to the density of the
photoreceptor cell.
[0040] Specifically, as shown in FIG. 7A, a polar coordinate (r, Θ) whose origin is the center of a frequency
conversion image of (N/2, N/2) when a size of the frequency conversion image is Ν χ Ν is considered.
Further, a function I (r) is calculated by integrating a value of each of the pixels of the frequency conversion image in the Θ direction. Note that, r=0, 1, 2, - · Ν/2 is established. FIG. 7B is a graph for showing the function I (r) acquired for the image shown in FIG. 6A. That is, the boundary acquiring unit acquires a
boundary that defines whether or not the photoreceptor cell exists as a region based on the intensity of the signal corresponding to the density of the photoreceptor cell, which is obtained by performing the frequency conversion on the planar image.
[0041] In this case, the density of the photoreceptor cell is determined based on a relationship between a distance from the fovea and a generally known photoreceptor cell density. As the generally known photoreceptor cell density used herein, a report based on anatomical data described in, for example, NPL 1 is given. Therefore, it is also possible to determine the distance from the fovea of the AO-SLO image based on the position
alignment result determined in Step S240, to thereby obtain the photoreceptor cell density at the
photographing position from this distance.
[0042] As an index for determining whether or not the
photoreceptor cell exists, one method uses a value of a function I (r0) that corresponds to a spatial frequency r0 corresponding to the density of the photoreceptor cell. Alternatively, another method takes great dependence of the value of I (ro) on a contrast of the AO-SLO image into account, and uses a ratio between the spatial frequency ro corresponding to the density of the photoreceptor cell and a ratio between the function I (ro) and a function I (2r0) corresponding to a spatial frequency 2ro that is twice the value of the spatial frequency r0. A description is made based on the method using the value of I(r0) below, but the index is not limited thereto.
[0043] hen I (r0) is larger than a certain threshold IE, it is determined that the photoreceptor cell exists, whereas when I(r0) is smaller than a certain threshold IL, it is determined that the photoreceptor cell does not exist. A state having the value of I (r0) that falls between the IE and the IL is determined to be undetermined.
Values of the thresholds are, for example, IE=2,000 and IL=1, 000.
[0044] The result of determining whether or not the photoreceptor cell exists in each of the regions, which is acquired in this manner, is stored in the memory part 130 through the control part 120.
[0045]<Step S540>
In Step S540, the boundary acquiring part 142
determines whether or not the boundary exists based on the existence determination result of the photoreceptor cell for each of the regions determined in Step S530. Specifically, when the photoreceptor cells exist in all of the 16 regions, it is determined that the boundary does not exist. Moreover, when the 16 regions include both of the region in which the photoreceptor cell exists and the region in which the photoreceptor cell does not exist, it is determined that the boundary exists. In this case, when there exists the region that is located between the region in which the
photoreceptor cell exists and the region in which the photoreceptor cell does not exist and is determined as being undetermined, the undetermined region is
determined as a boundary region. When the region in which the photoreceptor cell exists and the region in which the photoreceptor cell does not exist are
adjacent to each other, the adjacent region in which the photoreceptor cell exists is determined as the boundary region. Further, when it is determined that no photoreceptor cell exists in all of the 16 regions, the photographing is determined as having failed.
[0046] The result of determining whether or not there exists a boundary in the image, which is acquired in this manner, is stored in the memory part 130 through the control part 120.
[0047]<Step S260>
In Step S260, the photographing position setting part 143 sets a next photographing position based on the boundary determination result determined in Step S540.
[0048] FIG. 8 is a schematic view for showing a fundus in the case of retinitis pigmentosa. It is known that retinitis pigmentosa causes the photoreceptor cell defect from the peripheral portion. In FIG. 8, a circular gray region shown in the vicinity of the fovea is the region in which the photoreceptor cell exists, and the peripheral portion of the circular gray region is the region in which the photoreceptor cell defect occurs.
[0049] In Step S220, a position of the fovea is set as the initial photographing position. An AO-SLO image photographed at the position of the fovea (image at an image position 1) is processed following Step S510 to Step S540. As a result of the process, when it is determined that the boundary does not exist, the next photographing position is set to a position on the left of the current photographing position, which includes a region overlapping with the current photographing position (image position 2). In this case, a dimension of the overlapped region is about 15% of a
photographing field angle of the AO-SLO. The
photographing field angle is herein 400x400 pixel and hence the dimension is set to 60 pixels. However, the dimension is not limited to this value and can be appropriately changed depending on a fixation state of the eye to be. inspected and whether or not a tracking system is provided.
[0050] The next photographing position of the AO-SLO image set in this manner is stored in the memory part 130 through the control part 120. That is, the image acquiring part 100 includes a module functioning as a
photographing position setting unit configured to set, depending on the position of the boundary acquired by the boundary acquiring unit, a photographing position on the fundus at which the planar image is to be photographed.
[0051] Now., a method of setting a next photographing position in the case of the example of retinitis pigmentosa shown in FIG. 8 is described in detail.
[0052] The processes of Step S240 to Step S250 are performed on an image photographed at the image position 2. In this case, when it is determined that the boundary does not exist in the image similarly to the image
photographed at the image position 1, the next
photographing position is set to a position on the left of the image position 2, which is overlapped therewith by 60 pixels (image position 3) similarly to the above- mentioned case.
[0053] ext, the similar processes are performed on an image photographed at the image position 3. In this case, when it is determined that the boundary exists in the image photographed at the image position 3, the next photographing position is set to a position on the right of the image 1, which is overlapped therewith by 60 pixels (image position 4). Then, the similar processes are performed on an image photographed at the image position 4. As a result of the processes, when it is determined that the boundary does not exist in the image photographed at the image position 4, the next photographing position is set to a position on the right of the image position 4 (image position 5) .
[0054] ext, the similar processes are performed on an image photographed at the image position 5. In this case, when it is determined that the boundary exists in the image photographed at the image position 5 this time, the next photographing position is set to an image position 6 above the image position 1, which is
overlapped therewith by 60 pixels. Then, the similar processes are performed on an image photographed at the image position 6. As a result of the processes, when it is determined that the boundary does not exist in the image photographed at the image position 6, the next photographing position is set to an image position 7 above the image position 6, which is overlapped therewith by 60 pixels.
[0055] Next, the similar processes are performed on an image photographed at the image position 7. In this case, when it is determined that the boundary exists in the image photographed at the image position 7 this time, the next photographing position is set to an image position 8 below the image position 1, which is
overlapped therewith by 60 pixels.
[0056] In this way, from the position of the fovea as the
initial photographing position, the photographing position is shifted from side to side and up and down until an image including the boundary of the region in which the photoreceptor cell exists is obtained. With respect to the cruciform photographing region
determined in this manner, positions in a rectangular region covering the cross shape (image position 9 to image position 20 of FIG. 8) are set to the next photographing positions in turns.
[ 0057 ] Moreover, when it is determined that the photographing has failed in Step S540, there is displayed a
photographing position moved from the set photographing position toward a region in which the photoreceptor cell is more likely to exist. Specifically, in the case of retinitis pigmentosa shown in FIG. 8, a
position moved from a photographing position at which the photographing has failed toward the fovea by 60 pixels is displayed. This operation is performed in order to take the following case into account: the photoreceptor cell defect has already occurred in the vicinity of the center of the initially set
photographing position to cause the focus adjustment to be undetermined, and hence the photographing may have failed.
[0058 ] Finally, after the photographing of the above-mentioned image positions 1 to 20 is finished, it is further confirmed that the image, which is determined as the image in which only the photoreceptor cell exists and the boundary does not exist, is not located in the outermost peripheral portion of the photographing region. When such an image exists, the next
photographing position is set to each of positions that are adjacent to the image in the side to side direction and the up and down direction and have not been
photographed. When such an image does not exist, the flow proceeds to Step S270 in a state in which the next photographing position is not set.
[0059]<Step S270>
In Step S270, the control part 143 acquires the next photographing position set by the photographing
position setting part in Step S260, and determines whether or not all of the regions in which the
photoreceptor cells including the boundary portion exist are photographed. When the photographing of all of the regions has not been completed and the next photographing position is set, the flow returns to Step S230. Information on a photographing position acquired in Step S260 is transmitted to the planar image
photographing apparatus 3, and an AO-SLO image at the position is acquired. Then, the acquired AO-SLO image is stored in the memory part 130 through the control part 120, and the flow continues to Step S240.
[0060] When the next photographing position is not set, the completion of the photographing is transmitted to the planar image photographing apparatus 3 through the output part 150, and the flow proceeds to Step S280.
[0061]<Step S280>
In Step S280, for all of the photographed images including the boundary of the photoreceptor cell, the connecting part 144 acquires, from the memory part 130, the information on the region in which the
photoreceptor cell exists, the region in which the photoreceptor cell does not exist, and the boundary region, which are set on the planar image in Step S540. Then, based on the position alignment result of each of the planar images determined in Step S240, the above- mentioned regions are associated with each other on the WF-SLO image so that the images or the pieces of
information are connected to each other. The
connection result of the regions in which the
photoreceptor cells exist and the boundary regions acquired in this manner is stored in the memory part 130 through the control part 120. Simultaneously, the connection result is displayed on the display part 4 through the output part 150. At this time, the
positions of the boundaries are superimposedly
displayed on the WF-SLO image acquired in Step S210.
That is, the connecting part 144 described herein also forms a position alignment unit configured to associate with each other the acquired boundary and the
photographing position on the fundus from which the planar image is acquired.
[0062] With the configuration described above, when the image of the retina is acquired by the AO-SLO, all of the regions in which the photoreceptor cells exist can be photographed together with the boundaries of the
photoreceptor cells.
[0063] (Second Embodiment)
In the first embodiment, there is described the method involving starting the photographing from the region in which the photoreceptor cell is likely to exist,
searching for the boundary of the region in which the photoreceptor cell defect occurs while the existence of the photoreceptor cell is confirmed, and photographing all of the regions in which the photoreceptor cells exist. In a second embodiment of the present invention, there is described a case of photographing not all of the regions in which the photoreceptor cells exist, but only the boundary region serving as the boundary between the region in which the photoreceptor cell exists and the region in which the photoreceptor cell defect occurs.
[0064] The functional configuration of an ophthalmic apparatus used in this embodiment is the same as the
configuration of FIG. 1 used in the first embodiment, and hence the description thereof is omitted. Next, with reference to a flow chart of FIG. 9, a processing procedure of an ophthalmic apparatus 20 according to this embodiment is described. Note that, the details of the respective processes. of Step S210 to Step S250 and Step S270 are the same as those described with reference to FIG. 2, and hence the description thereof is omitted.
[0065]<Step S1060>
In Step S1060, the photographing position setting part 143 sets a next photographing position based on the boundary determination result determined in Step S540.
[0066] FIG. 10 is a schematic view for showing a fundus in the case of retinitis pigmentosa. It is known that
retinitis pigmentosa causes the photoreceptor cell defect from the peripheral portion. In FIG. 10, a circular gray region shown in the vicinity of the fovea is the region in which the photoreceptor cell exists, and the peripheral portion of the circular gray region is the region in which the photoreceptor cell defect occurs .
[0067] In Step S220, a position of the fovea is set as the
initial photographing position. An AO-SLO image photographed at the position of the fovea (image at the image position 1) is processed following Step S510 to Step S540. There is described a process for proceeding in the case where it is determined that the boundary does not exist in the image position 1 as a result of the process above. In this case, the next photographing position is set to a position on the left of the current photographing position, which includes a region overlapping with the current photographing position (image position 2). In this case, a dimension of the overlapped region is about 15% of a
photographing field angle of the AO-SLO. The
photographing field angle is herein 400x400 pixel and hence the dimension is set to 60 pixels. However, the dimension is not limited to this value and can be appropriately changed depending on a fixation state of the eye to be inspected and whether or not a tracking system is provided.
[0068] The next photographing position of the AO-SLO image set in this manner is stored in the memory part 130 through the control part 120.
[0069] ow, a method of setting a next photographing position in the case of the example of retinitis pigmentosa shown in FIG. 10 is described in detail.
[0070] The processes of Step S240 to Step S250 are performed on an AO-SLO image photographed at the image position 2 In this case, when it is determined that the boundary does not exist at the position similarly to the AO-SLO image at the image position 1, the next photographing position is set to a position on the left of the image position 2, which is overlapped therewith by 60 pixels (image position 3) similarly to the above-mentioned case.
[0071] Next, the similar processes are performed on an AO-SLO image photographed at the image position 3. In this case, when it is determined that the boundary exists in the image photographed at the image position 3, the boundary search is started from the fovea, and the image acquired at the image position 3 is set to a search start image as an image first determined to include the boundary. Moreover, it is confirmed at the image position 3 to which direction the boundary is extended. For example, when the boundary is in contact with an upper side of the field, it is determined that the boundary extends to an upper portion of the field. Then, the next photographing position is set to an image position 4 adjacent to the upper portion of the image position 3, which is along the extending
direction of the boundary. That is, the selection method for an adjacent position is herein determined based on the existence determination result of the photoreceptor cell determined for each of the regions in Step S250.
[0072] FIG. 11A is the illustration of the result of the
existence determination of the photoreceptor cell for each of the regions in the image acquired at the image position 3 of FIG. 10, the existence determination being acquired in Step S250. In FIG. 11A, a dark color region represents a region determined to include the photoreceptor cell, a light color region represents a region determined to include no photoreceptor cell, and a light color region represents a region in which the existence of the photoreceptor cell cannot be
determined. In this case, it is assumed that the boundary exists in the region located between the dark color region and the light color region, in which the existence of the photoreceptor cell cannot be
determined.
[0073] With reference to FIG. 11A, in the image, the region representing the boundary extends from a lower right corner so as to reach substantially the center of an upper side, and hence it is expected that the boundary crosses the upper side of the image. In such a case, the next photographing position is set to a position above the image position 3, which is overlapped
therewith by 60 pixels. That is, regardless of which sides (upper, lower, right, and left sides) of the . photographing field the boundary crosses, in the same manner, the next photographing position is set to a position overlapped by 60 pixels in a direction of the corresponding side.
[0074]Next, FIG. 11B is the similar illustration of the
result of the existence determination of the
photoreceptor cell for each of the regions in an image acquired at the image position 4 of FIG. 10. In the case of the image acquired at the image position 4, a boundary expected position is an upper right corner position. In this case, the next photographing
position is set so that the upper right corner is overlapped with the next photographing position by 60x2=120 pixels. The photographing position set in this manner is represented as an image position 5 of FIG. 10.
[0075] ote that, the boundary is retrieved in the clockwise direction in FIG. 10, but both of the clockwise
direction and the counterclockwise direction are acceptable. When a position at which an image being analyzed is acquired overlaps with the image position 3 at which the search start image is acquired, the next photographing position is not set.
[0076] Moreover, similarly to the first embodiment, when it is determined that the photographing has failed in Step S540, it is preferred to display a new photographing position moved from the set photographing position toward a region in which the photoreceptor cell is more likely to exist.
[0077] (Third Embodiment)
In the first embodiment, there is described the method involving photographing all of the regions in which the photoreceptor cells exist while searching for the boundary of the photoreceptor cell defect. Further, in the second embodiment, there is described the method involving performing the photographing along the boundary between the region in which the photoreceptor cell defect occurs and the region in which the photoreceptor cell exists. However, when the region in which the photoreceptor cell exists is large, a large number of regions are to be photographed in the method of the first embodiment, which causes a burden on the eye to be inspected such as due to an increase in examination time. In the second embodiment, the number of photographing regions is reduced compared to the first embodiment, but there is a risk that the boundary cannot be correctly retrieved when the shape of the boundary is irregular.
[0078] Therefore, in a third embodiment of the present
invention, there is described a method involving photographing, with an AO-SLO image having a lowered resolution, all of the regions in which the
photoreceptor cells exist by a method similar to that of the first embodiment and further photographing, with a high resolution AO-SLO, only a target region in which the boundary is desired to be clearly obtained and the like.
[0079] The functional configuration of an ophthalmic apparatus used in this embodiment is also the same as that of the ophthalmic apparatus of the first embodiment
illustrated in FIG. 1, and hence the description thereof is omitted. Next, with reference to a flow chart of FIG. 12, a processing procedure of an
ophthalmic apparatus 20 according to this embodiment is described. Note that, the details of the respective processes of Step S210 to Step S280 are the same as those described with reference to FIG. 2, and hence the description thereof is -omitted.
[0080] In this embodiment, an AO-SLO image acquired in Step
S220 is subjected to the aberration correction but is a low resolution image, and an AO-SLO image acquired in Step S1220 is a high resolution image subjected to the aberration correction. Specifically, the low resolution AO-SLO image has a level of resolution with which it is difficult to individually detect the photoreceptor cells but the region in which the
photoreceptor cell exists and the region in which the photoreceptor cell defect occurs can be grasped. The above-mentioned M image of the planar image corresponds to the low resolution AO-SLO image, for example. The high resolution AO-SLO image is an image having a level of resolution with which the photoreceptor cells can be individually detected, and the above-mentioned S image of the planar image corresponds thereto. However, the above-mentioned classification is not a limitation, and a resolution to be used can be changed depending on a dimension and range of a photographed disease, and a photoreceptor cell density desired to be resolved.
[0081]<Step S1220>
In Step S1220, the information acquiring part 110 acquires, based on the information on the region in which the photoreceptor cell exists and the boundary thereof, which is displayed to the user in Step S280, a photographing position of a high resolution planar image that has been selected by the user.
Then, the acquired photographing position is stored in the memory part 130 through the control part 120.
[0082]<Step S1230>
In Step S1230, the image acquiring part 100 transmits the information on the photographing position acquired in Step S1220 to the planar image photographing
apparatus 3 so that the planar image photographing apparatus 3 photographs a high resolution AO-SLO image at the position. Then, the image acquiring part 100 acquires the high resolution AO-SLO image from the planar image photographing apparatus 3.
[0083] In this case, when the photographing position acquired in Step S1220 corresponds to the region including the boundary acquired in Step S280, the photographing is performed at the same photographing position with focus positions set at a plurality of positions.
Specifically, the photographing is performed at the focus position acquired by the aberration correction (focus reference image), and images are also acquired at focus positions moved back and forth by 10 μπι in 3 steps each. That is, the photographing is performed at the focus positions moved toward a vitreous body side by 10 μπι, 20 μπι, and 30 μιη and toward a choroid side by 10 μπι, 20 μπι, and 30 μπι. Thus, 7 images are acquired as a total.
[0084] This operation is performed because, in the vicinity of the boundary, reflection of beacon light required for the aberration correction may be unstable depending on a state of the photoreceptor cell layer, and hence it may be difficult to perform the photographing at a correct focus position. A represented state of the photoreceptor cell layer is significantly affected by a shift, of the focus position. Thus, the images are acquired at the plurality of focus positions as
described above so that the boundary of the region in which the photoreceptor cell exists can be acquired more accurately.
[0085] Moreover, for the same reason, when the photographing position acquired in Step S1220 is located outside of the boundary of the region in which the photoreceptor cell exists, which, is acquired in Step S280, that is, when the photographing position acquired in Step S1220 is located in the region determined to include no photoreceptor cell, an alert indicating the fact is displayed to the user. Then, there are displayed alternatives such as performing the normal
photographing even though the reflection of beacon light may be unstable, manually setting a focus
position, and using a focus position that was used when a neighboring region in which the photoreceptor cell exists was photographed.
Then, the acquired high resolution AO-SLO image group is stored in the memory part 130 through the control part 120.
[0085]<Step S1240>
In Step S1240, the position alignment part 141 aligns positions of the WF-SLO image acquired in Step S210 and each of the images in the high resolution AO-SLO image group acquired in Step S1230. The result of
associating with each other each of the high resolution AO-SLO images and the position on the WF-SLO image in this manner is stored in the memory part 130 through the control part 120.
[0087]<Step S1250>
In Step S1250, the image processing part 140 determines whether or not, in the AO-SLO image group stored in the memory part 130, there exists the boundary between the region in which the photoreceptor cell exists and the region in which the photoreceptor cell defect occurs. Note that, the details of the process performed in this step are the same as those of the processes of Step S510 to Step S540 of FIG. 5, and hence the description thereof is omitted. Note that, the process is
performed on all of the 7 images acquired in Step S1230, and the existence determination of the photoreceptor cell is performed on the region of each of the images. Then, the regions in each of which the photoreceptor cells exist are connected to each other and a boundary in a case where a dimension of the region in which the photoreceptor cell exists is the maximum is determined as a true boundary.
[0088] FIG. 13 is a schematic view for showing an arrangement of each of the images in the high resolution AO-SLO image group on the WF-SLO image photographed in this embodiment. As described above, in this embodiment, based on the existence region of the photoreceptor cell acquired by Step S280, a second planar image having a resolution different from that of the planar image acquired in the steps previous to this step is acquired. The second planar image is acquired by the planar image photographing apparatus 3 that is increased in
resolution and thus functions as a second image
acquiring unit.
[0089] In this case, as described above, it is preferred that, when the acquired second planar image includes the boundary of the photoreceptor cell, the second image acquiring unit perform the photographing at the
plurality of focus positions for the photographing position at which the second planar image is acquired. Moreover, in this case, it is more preferred that the boundary acquiring part 142 acquire again the boundary of the existence region of the photoreceptor cell from each of the second planar images photographed at the plurality of focus positions. Alternatively, as
described above, it is preferred that, when the
acquired second planar image includes the photoreceptor cell, the second image acquiring unit perform the photographing at the plurality of focus positions for the photographing position at which the second planar image is acquired. Moreover, in this case, it is more preferred that the boundary acquiring part 142 select again, from among the second planar images photographed at the plurality of focus positions, the second planar image to be connected by the connecting part 144.
[0090] (Fourth Embodiment)
In the third embodiment, there is described the method involving photographing, with the use of the low
resolution AO-SLO image, the higher resolution AO-SLO image based on the region in which the photoreceptor cell exists and the boundary thereof that are acquired by the method described in the first embodiment.
Moreover, there is described the process performed when, In the photographing, the photographing position of the high resolution AO-SLO image includes the boundary of the region in which the photoreceptor cell exists.
Accordingly, in a fourth embodiment of the present invention, there is described a process performed when the photographing position of the high resolution AO- SLO image does not include the boundary but corresponds to the region in which the photoreceptor cell exists.
[0091] Specifically, when the photographing position
corresponds to the region in which the photoreceptor cell exists, a detailed photoreceptor cell image may be acquired with high probability if the focus position is correctly set. Therefore, the AO-SLO images are photographed at the plurality of focus positions in advance, to thereby acquire a plurality of AO-SLO images. The number of the photographed AO-SLO images may be 7 as described in the third embodiment or the AO-SLO images may be photographed in a wider field. Alternatively, based on a history of the past
photographing of the eye to be inspected and the photographing position, a focus range to be
photographed may be set in advance.
[0092] From among the plurality of AO-SLO images acquired at the different focus positions in this manner, an image in which the photoreceptor cell is represented is selected. Specifically, the frequency conversion image of the AO-SLO image described in Step S250 of the first embodiment is used, to thereby select an image having the maximum index.
[0093] Further, instead of selecting the image, the image may be divided into the regions as described in Step S510 of the first embodiment, and the regions having large indices may be connected to each other, to thereby generate the image.
[0094] (Fifth Embodiment)
In each of the first embodiment to the fourth embodiment, there is described the method involving, with the use of only the AO-SLO image at the time of photographing, photographing the region in which the photoreceptor cell exists and the boundary thereof while searching for the boundary between the region in which the photoreceptor cell exists and the region in which the photoreceptor cell defect occurs. However, information on the region in which the photoreceptor cell is likely to exist may be obtained in advance.
[0095] Specifically, information obtained from other
modalities, such as an acquired OCT tomographic image and fluorescein fundus angiographic image, may be referred to. Alternatively, for the same eye to be inspected, the boundary of the photoreceptor cell obtained from the AO-SLO image photographed in the past may be referred to. That is, in the case of a fifth embodiment of the present invention, the boundary may be defined by acquiring the boundary based on the OCT tomographic image of the eye to be inspected, acquiring the boundary from the planar image of the eye to be inspected acquired in the past, or selecting the boundary from among the plurality of boundaries
acquired in the past.
[0096] FIG. 14 is an example in which the information on the boundary of the photoreceptor cell obtained in the past photographing is superimposed on the WF-SLO image to be displayed. In FIG. 14, there are shown regions
including the photoreceptor cells, which were acquired when the progress was observed four times in the past, and a more recently acquired region is shown with a darker color. Displaying such information to the user allows the user to predict an existing position of the photoreceptor cell at the current stage. Specifically, in FIG. 14, the region in which the photoreceptor cell exists is decreased at higher speed on the right side than on the left side, and hence it can be predicted that the boundary may exist on the inner side of the boundary in the last photographing.
[0097 ] Moreover, a difference from the boundary of the
photoreceptor cell obtained in the past photographing can be extracted as a region, and the extracted region can be determined as an analysis target region.
Specifically, when a difference region between the boundary in the last photographing and the boundary in the second last photographing is determined, the photoreceptor cell included in the difference region exists in the second last photographing but causes the defect in the last photographing. With respect to the image in the second last photographing, the
photoreceptor cell included in the difference region and the photoreceptor cell included in the boundary in the last photographing are analyzed and compared to each other. In this way, a photoreceptor cell that is to cause a defect can be analyzed. That is, in such a case, the boundary acquiring part 142 extracts a difference among the. plurality of boundaries acquired in the past as the analysis target region.
[0098] When the information from other modalities and past progress information are available, the region in which the photoreceptor cell is likely to exist can be predicted from those pieces of information. Therefore, it is conceivable that the photographing with higher accuracy can be achieved by combining the method of this embodiment and the method involving analyzing the photographed AO-SLO image to search for the boundary as described in each of the first embodiment to the fourth embodiment .
[0099] Specifically, it is conceivable that, when it is
determined whether or not the boundary is included in the photographed image in Step S250 and the
determination result differs from the prediction, an alert is displayed to the user and an option such as shifting the focus position and performing the
photographing again at the same position is
simultaneously displayed to the user.
[0100] Moreover, whether or not the information from other
modalities and the past progress information are
available depends on the eye to be inspected, and hence the mode can be switched between a mode using the information from other modalities and the past progress information in the photographing and a mode not using those pieces of information in the photographing.
[0101] Other Embodiments
Embodiment ( s ) of the present invention can also be realized by a computer of a system or apparatus that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be referred to more fully as a 'non-transitory computer-readable storage medium') to perform the functions of one or more of the above- described embodiment ( s ) and/or that includes one or more circuits (e.g., application specific integrated circuit (ASIC) ) for performing the functions of one or more of the above-described embodiment ( s ) , and by a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage medium to perform the functions of one or more of the above-described embodiment ( s ) and/or controlling the one or more circuits to perform the functions of one or more of the above-described embodiment (s) . The
computer may comprise one or more processors (e.g., central processing unit (CPU) , micro processing unit (MPU) ) and may include a network of separate computers or separate processors to read out and execute the computer executable instructions. The computer
executable instructions may be provided to the computer, for example, from a network or the storage medium. The storage medium may include, for example, one or more of a hard disk, a random-access memory (RAM) , a read only memory (ROM) , a storage of distributed computing systems, an optical disk (such as a compact disc (CD), digital versatile disc (DVD), or Blu-ray Disc ( BD)™) , a flash memory device, a memory card, and the like.
[0102]While the present invention has been described with reference to exemplary embodiments, it is to be
understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest
interpretation so as to encompass all such
modifications and equivalent structures and functions.
[0103] This application claims the benefit of Japanese Patent Application No. 2014-140524, filed July 8, 2014, which is hereby incorporated by reference herein in its entirety .

Claims

[Claim 1] An ophthalmic apparatus, comprising:
an image acquiring unit configured to acquire a planar image of a fundus of an eye to be inspected; a boundary acquiring unit configured to acquire, from the planar image, a boundary between an existence region in which a photoreceptor cell of the eye to be inspected exists and a region in which the
photoreceptor cell does not exist; and
a setting unit configured to set the existence region of the photoreceptor cell on the fundus based on the boundaries acquired from a plurality of the planar images .
[Claim 2] An ophthalmic apparatus according to claim 1, wherein the boundary acquiring unit acquires the boundary based on intensity of a signal corresponding to a density of the photoreceptor cell, the intensity of the signal being obtained by performing frequency conversion on the planar image.
[Claim 3] An ophthalmic apparatus according to claim 1 or 2,
wherein the image acquiring unit comprises a
photographing position setting unit configured to set, depending on a position of the boundary acquired by the boundary acquiring unit, a photographing position on the fundus at which the planar image is to be photographed.
[Claim 4 ] An ophthalmic apparatus according to any one of claims
1 to 3, further comprising a position alignment unit configured to associate with each other the acquired boundary and a photographing position on the fundus at which the planar image is acquired.
[Claim 5] An ophthalmic apparatus according to any one of claims
1 to 4, further comprising a second image acquiring unit configured to acquire, based on the acquired existence region of the photoreceptor cell, a second planar image having a resolution different from a resolution of the planar image.
[Claim 6] An ophthalmic apparatus according to claim 5, wherein when the acquired second planar image includes the boundary of the photoreceptor cell, the second image acquiring unit performs photographing at a plurality of focus positions for a photographing position on the fundus at which the second planar image is acquired.
[Claim 7 ] An ophthalmic apparatus according to claim 6, wherein the boundary acquiring unit acquires the boundary of the existence region of the photoreceptor cell from each of the second planar images photographed at the plurality of focus positions.
[Claim 8 ] An ophthalmic apparatus according to claim 5, wherein when the acquired second planar image includes the photoreceptor cell, the second image acquiring unit performs photographing at a plurality of focus positions for a photographing position on the fundus at which the second planar image is acquired.
[Claim 9] An ophthalmic apparatus according to claim 8, wherein the boundary acquiring unit selects the second planar image to be used in the setting by the setting unit from among the second planar images photographed at the plurality of focus positions.
[Claim 1 0]An ophthalmic apparatus, comprising:
a boundary acquiring unit configured to acquire, on an fundus of an eye to be inspected, a boundary between an existence region in which a photoreceptor cell exists and a region in which the photoreceptor cell does not exist ;
a position alignment unit configured to associate with each other the acquired boundary and a photographing position on the fundus of the eye to be inspected at which a planar image is photographed; and
an image acquiring unit, configured to acquire a second planar image having a resolution different from a resolution of the planar image based on the acquired boundary of the existence region of the photoreceptor cell .
[Claim ll]An ophthalmic apparatus according to claim 10, wherein the boundary is acquired based on an OCT tomographic image of the eye to be inspected.
[Claim 12] An ophthalmic apparatus according to claim 10, wherein the boundary is acquired from the planar image of the eye to be inspected acquired in the past.
[Claim 13] An ophthalmic apparatus according to claim 10, wherein the boundary comprises a plurality of boundaries acquired in the past.
[Claim 14] An ophthalmic apparatus according to claim 13, wherein the boundary acquiring unit extracts a difference among the plurality of boundaries acquired in the past as an analysis target region.
[Claim 15]A program for causing a computer to function as each of the units of the ophthalmic apparatus of any one of claims 1 to 14.
[Claim 16] A storage medium having stored thereon the program of claim 15.
[Claim 17]A control method for an ophthalmic apparatus,
comprising:
acquiring a planar image of a fundus of an eye to be inspected;
acquiring, from the planar image, a boundary between an existence region in which a photoreceptor cell of the eye to be inspected exists and a region in which the photoreceptor cell does not exist; and
setting the existence region of the photoreceptor cell on the fundus based on the boundaries acquired from a plurality of the planar images.
[Claim 18] A program for causing a computer to execute each of the steps of the control method for an ophthalmic apparatus of claim 17.
PCT/JP2015/068597 2014-07-08 2015-06-23 Ophthalmic apparatus and control method therefor WO2016006482A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2014-140524 2014-07-08
JP2014140524A JP6410498B2 (en) 2014-07-08 2014-07-08 Ophthalmic apparatus and control method thereof

Publications (1)

Publication Number Publication Date
WO2016006482A1 true WO2016006482A1 (en) 2016-01-14

Family

ID=53541874

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2015/068597 WO2016006482A1 (en) 2014-07-08 2015-06-23 Ophthalmic apparatus and control method therefor

Country Status (2)

Country Link
JP (1) JP6410498B2 (en)
WO (1) WO2016006482A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6836212B2 (en) * 2016-07-04 2021-02-24 株式会社ニデック Ophthalmologic imaging equipment

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120070059A1 (en) * 2009-06-02 2012-03-22 Canon Kabushiki Kaisha Image processing apparatus, image processing method, and computer program
US20120130270A1 (en) * 2010-11-19 2012-05-24 Canon Kabushiki Kaisha Image processing apparatus and image processing method
US20130058553A1 (en) * 2011-09-06 2013-03-07 Canon Kabushiki Kaisha Image processing apparatus and image processing method
US20130188135A1 (en) * 2012-01-20 2013-07-25 Canon Kabushiki Kaisha Image processing apparatus and image processing method
WO2013125187A1 (en) * 2012-02-20 2013-08-29 Canon Kabushiki Kaisha Image processing apparatus, ophthalmologic imaging system, and image processing method
EP2749205A1 (en) * 2012-12-28 2014-07-02 Canon Kabushiki Kaisha Ophthalmological apparatus, alignment method, and program

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2895414B2 (en) * 1995-05-08 1999-05-24 アロカ株式会社 Ultrasonic volume calculator
JP4832794B2 (en) * 2005-04-27 2011-12-07 オリンパスメディカルシステムズ株式会社 Image processing apparatus and image processing program
JP5148928B2 (en) * 2007-05-31 2013-02-20 株式会社トプコン Fundus examination apparatus and program
JP4819851B2 (en) * 2008-07-31 2011-11-24 キヤノン株式会社 Diagnosis support apparatus and method, program, and recording medium
JP5822485B2 (en) * 2011-02-25 2015-11-24 キヤノン株式会社 Image processing apparatus, image processing method, image processing system, SLO apparatus, and program
JP5843542B2 (en) * 2011-09-20 2016-01-13 キヤノン株式会社 Image processing apparatus, ophthalmologic photographing apparatus, image processing method, and program
JP2014097191A (en) * 2012-11-14 2014-05-29 Canon Inc Imaging apparatus, imaging method and program
EP3000382A1 (en) * 2012-12-28 2016-03-30 Canon Kabushiki Kaisha Image processing apparatus and image processing method

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120070059A1 (en) * 2009-06-02 2012-03-22 Canon Kabushiki Kaisha Image processing apparatus, image processing method, and computer program
US20120130270A1 (en) * 2010-11-19 2012-05-24 Canon Kabushiki Kaisha Image processing apparatus and image processing method
US20130058553A1 (en) * 2011-09-06 2013-03-07 Canon Kabushiki Kaisha Image processing apparatus and image processing method
US20130188135A1 (en) * 2012-01-20 2013-07-25 Canon Kabushiki Kaisha Image processing apparatus and image processing method
WO2013125187A1 (en) * 2012-02-20 2013-08-29 Canon Kabushiki Kaisha Image processing apparatus, ophthalmologic imaging system, and image processing method
EP2749205A1 (en) * 2012-12-28 2014-07-02 Canon Kabushiki Kaisha Ophthalmological apparatus, alignment method, and program

Also Published As

Publication number Publication date
JP6410498B2 (en) 2018-10-24
JP2016016103A (en) 2016-02-01

Similar Documents

Publication Publication Date Title
AU2021232682B2 (en) Detection of pathologies in ocular images
US9098742B2 (en) Image processing apparatus and image processing method
US10176614B2 (en) Image processing device, image processing method, and program
US20240249122A1 (en) Diagnosis support device, learning device, diagnosis support method, learning method, and program
EP2772184B1 (en) Image processing apparatus and image processing method
EP2772186B1 (en) Image processing apparatus and image processing method
US20190069776A1 (en) Ophthalmic imaging system with automatic retinal feature detection
US20120140179A1 (en) Tomography apparatus, control method for the same, program, and storage medium
EP2360641B1 (en) Image processing method and image processing device
US20140185889A1 (en) Image Processing Apparatus and Image Processing Method
CN109215039B (en) Method for processing fundus picture based on neural network
US20230245772A1 (en) A Machine Learning System and Method for Predicting Alzheimer&#39;s Disease Based on Retinal Fundus Images
CN116491893B (en) Method and device for evaluating change of ocular fundus of high myopia, electronic equipment and storage medium
JP2006263127A (en) Ocular fundus diagnostic imaging support system and ocular fundus diagnostic imaging support program
JP6147001B2 (en) Image processing apparatus and image processing method
CN110598652B (en) Fundus data prediction method and device
WO2016006482A1 (en) Ophthalmic apparatus and control method therefor
JP2014128448A5 (en)
WO2016031151A1 (en) Ophthalmic apparatus and control method therefor
JP7440665B2 (en) retinal image processing
JP6576035B2 (en) Ophthalmic apparatus, photoreceptor cell detection method and program
WO2024089642A1 (en) System and method for detecting glaucoma
JP2017023482A (en) Picture processing device, picture processing method, and program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15736663

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15736663

Country of ref document: EP

Kind code of ref document: A1