JP2017127580A - Examination apparatus and examination method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce positional deviations of moving images captured with a tracking function enabled and distortions in frame surfaces of such moving images.SOLUTION: Provided is an examination apparatus, comprising: image acquisition means for acquiring an image, based on return light from an object to be examined scanned with illumination light; detection means for detecting a positional deviation of the object to be examined when acquiring one frame of image of the object to be examined; tracking means for tracking the scanning position of the illumination light to correct the detected positional deviation; image splitting means for splitting the one frame of image into plural split images according to the time the positional deviation was detected; and image generation means for aligning each of the plural split images to generate a new image.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an inspection apparatus for obtaining an image to be used for inspection of an inspection object, an inspection method, and a program for causing a computer to execute the inspection method.

  In recent years, SLO (Scanning Laser Ophthalmoscope) and OCT (Optical Coherence Tomography) have been used as an imaging apparatus for ophthalmology. The SLO is an apparatus that scans a laser beam irradiated to the fundus two-dimensionally and receives reflected light to form an image. OCT is an imaging device that uses interference of low-coherence light, and is used particularly for the purpose of obtaining a tomographic image of the fundus or its vicinity.

  In recent years, such an ophthalmic imaging apparatus has been improved in resolution by increasing the NA of an irradiation laser. Here, when an image of the fundus is captured, the image must be captured through the optical tissue of the eye such as the cornea or the crystalline lens. For this reason, as the resolution increases, the influence on the image quality of the captured image due to the aberration of these optical tissues has become remarkable.

  Therefore, research on AO-SLO and AO-OCT in which an adaptive optics (AO) function for measuring aberration of the eye and correcting the aberration is incorporated in the optical system is underway. For example, Non-Patent Document 1 shows an example of AO-OCT. In the AO-OCT, a wavefront that expresses eye aberration is measured by a Shack-Hartmann wavefront sensor, and the wavefront is corrected by a wavefront correction device such as a deformable mirror. By performing imaging of the fundus through such an adaptive optical system, AO-SLO and AO-OCT enable high-resolution imaging with reduced effects of eye aberrations.

  Further, in these ophthalmic imaging devices, an index called a fixation target is presented to the subject, and the eye movement during imaging is suppressed by gazing at the fixation target, and an image at a desired imaging position is obtained. Trying to get. However, the eye to be examined normally performs minute eye movements called fixation microtremors that are involuntarily repeated in order to maintain vision. Moreover, it is difficult for the subject to keep a close eye on the fixation target due to problems such as maintenance of concentration and fatigue. Furthermore, when the subject has a disease in the eye, there is a reduction in visual acuity, narrowing of the visual field, and the movement of the eye becomes large.

  As a countermeasure for such fixation fixation, there is a fundus tracking technique that measures the movement of the eye to be examined and changes the irradiation position of the laser light on the fundus in real time so as to track the movement (also referred to as tracking). . For example, in the tracking technique disclosed in Patent Document 1, an image serving as a reference for tracking is set as a reference image. Then, a relative positional shift between the target image obtained from the fundus every moment by imaging and the reference image is calculated. The scanning position of the laser beam on the fundus is moved by a scanner. By correcting the scanning position in consideration of the calculated positional deviation, an image in which the influence of the movement of the eye to be examined is suppressed is obtained.

US2015 / 0077710A1

Y. Zhang et al, Optics Express, Vol. 14, no. 10, 15 May 2006

  The above-described calculation of the relative positional deviation and the correction of the laser beam scanning position based on the result are not performed every time one scanning line of the laser beam is executed, but a two-dimensional image composed of a plurality of scanning lines ( This is performed at the stage when a flat image is obtained. For this reason, the shift of the laser beam scanning position due to fixation fine movement or the like is accumulated while obtaining a plurality of scanning lines. Therefore, in the case of the subject's eye with poor fixation, there may be a difference between the actual eye movement at the timing of correcting and changing the laser light irradiation position by tracking the fundus and the corrected positional deviation. . Such a difference causes a positional shift or distortion in the frame plane in the captured image or moving image.

  The present invention has been made in view of the above-described situation, and an object thereof is to reduce a positional deviation or distortion in a frame plane of an image captured by operating a tracking function.

In order to solve the above-described problem, an inspection apparatus according to an embodiment of the present invention includes:
Image acquisition means for acquiring an image based on return light from the inspection object scanned with illumination light;
Detecting means for detecting a positional deviation of the inspection object when acquiring an image of the inspection object;
Tracking means for tracking the scanning position of the illumination light so as to correct the detected displacement;
Image dividing means for dividing the one image into a plurality of divided images according to the time of tracking;
Image generating means for aligning each of the plurality of divided images and generating a new image.

  According to the present invention, it is possible to reduce the displacement of the moving image captured by operating the tracking function and the distortion in the frame plane.

It is a schematic diagram which shows the outline of a structure of the fundus imaging apparatus (SLO) according to the first embodiment of the present invention. It is a figure explaining the WF-SLO image used for execution of tracking operation in the 1st example. It is a flowchart explaining the tracking operation | movement in a 1st Example. It is a flowchart explaining the image processing operation in a 1st Example. It is explanatory drawing of the process which divides | segments an AO-SLO image into a frame and aligns it. It is a flowchart explaining the image processing operation | movement in the 2nd Example of this invention.

  Embodiments of the present invention will be described below with reference to the drawings. The following examples do not limit the present invention related to the scope of claims, and all combinations of features described in the following examples are essential to the solution means of the present invention. Not exclusively.

  In the following embodiments, a configuration having an AO-SLO and a WF-SLO as an ophthalmic imaging apparatus is described. However, the application target of the present invention is not limited to this configuration, and the eye to be examined is illuminated with illumination light, such as the above-described OCT, AO-OCT, AO-SLO single device, or a combination device combining these. Applicable to general ophthalmic imaging apparatus for scanning.

[First embodiment]
The SLO according to the first embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a schematic diagram showing an outline of the configuration of the fundus imaging apparatus (SLO) according to the first embodiment of the present invention.

  In the fundus imaging apparatus according to this embodiment illustrated in FIG. 1, there are two imaging configurations for obtaining a fundus image. One is a scanning laser ophthalmoscope (AO-SLO) to which an adaptive optical system for obtaining a high-definition image is applied. The other is a scanning laser ophthalmoscope (WF-SLO) that does not apply adaptive optics. WF-SLO is inferior in resolution in an image obtained than AO-SLO, but has a characteristic that the imaging angle of view is wider than that of AO-SLO. These constitute image acquisition means for acquiring an image based on return light from the eye to be scanned scanned with illumination light. AO-SLO is the first image acquisition means in this embodiment, and WF-SLO is the second image acquisition means. An image acquisition unit is configured.

Here, the AO-SLO used in this embodiment will be described with reference to FIG.
In this AO-SLO, an SLD light source (Super Luminescent Diode) having a wavelength of 840 nm is used as the light source 101 that emits illumination light. Although the wavelength of the light source 101 is not particularly limited, light having a wavelength of about 800 to 1500 nm is preferably used for fundus imaging in order to reduce the glare of the subject and maintain resolution. In this embodiment, the SLD light source is used, but other light sources can also be used.

  The light emitted from the light source 101 enters the collimator lens 103 through the single mode optical fiber 102, and is irradiated onto the illumination optical path as a parallel light beam (illumination light 105) by the collimator lens 103. The irradiated illumination light 105 is transmitted through the light splitting unit 104 formed of a beam splitter and guided to the compensation optical system.

  The compensation optical system includes a light dividing unit 106, a wavefront measuring device 115, a wavefront correcting device 108, and reflection mirrors 107-1 to 107-4 for guiding illumination light and the like to them. Here, the reflection mirrors 107-1 to 107-4 are arranged so that at least the pupil of the eye 111 and the wavefront measuring device 115 and the pupil of the eye 111 and the wavefront correcting device 108 are optically conjugate. The wavefront measuring device 115 constitutes wavefront detecting means for detecting the wavefront of the return light in this embodiment. Further, as the light splitting unit 106, a beam splitter is used in this embodiment. The illumination light 105 transmitted through the light splitting unit 106 is reflected by the reflection mirrors 107-1 and 107-2 and enters the wavefront correction device 108. The illumination light 105 reflected by the wavefront correction device 108 is emitted to the reflection mirror 107-3.

  In this embodiment, a reflective spatial phase modulator using a liquid crystal element is used as the wavefront correction device 108 constituting wavefront correction means for correcting the wavefront of the return light. However, the wavefront correction means is not limited to this, and may be a deformable mirror (also referred to as a deformable mirror) whose mirror shape is variable. That is, as long as the wavefront can be corrected, the configuration to be used is not limited to these.

  In FIG. 1, the illumination light 105 reflected by the reflection mirrors 107-3 and 4 is scanned one-dimensionally or two-dimensionally by the scanning optical systems 109-1 and 109-2. In this embodiment, the resonance scanner 109-1 is used for main scanning of illumination light (for scanning in the horizontal direction of the fundus) for the purpose of capturing a two-dimensional fundus image, and sub-scanning (for scanning in the vertical direction of the fundus). ) Was used as a tip tilt mirror 109-2. The chip tilt mirror 109-2 also serves as means for correcting the imaging position by changing the scanning position of the illumination light for tracking (both horizontal and vertical directions of the fundus). The tip tilt mirror 109-2 is normally maintained at a standby angle inclined at a predetermined angle with respect to the optical axis on which the illumination light 105 is incident, and the illumination light 105 is emitted while the eye to be examined is aligned with the fundus imaging apparatus. The direction of reflection is directed to the scanning center when the illumination light 105 is scanned. In scanning the fundus with the illumination light 105, the reflection direction of the illumination light 105 is changed by reciprocating within a predetermined angle range from the standby angle. In actual tracking, by changing the direction of the scanning center, the scanning direction of the illumination light is tracked in accordance with the movement of the eye 111 so as to correct a detected positional deviation described later. The tip tilt mirror 109-2 is connected to the tracking control unit 121, and the operation of changing the direction of the scanning center is executed based on an instruction from the tracking control unit 121.

  In this embodiment, a combination of a resonant scanner and a chip tilt mirror is used as the scanning means. However, the configuration of the scanning means is not limited to this, and other scanning optical mirrors such as a galvanometer mirror may be used in combination. As a specific example, it is conceivable to secure the same function as the scanning means of this embodiment by using one resonance scanner and two or more galvanometer mirrors. Furthermore, in these configurations used as scanning optical systems, in order to make each mirror that reflects and scans illumination light into an optically conjugate state with the fundus etc., an optical element such as a mirror or lens is used between each scanner. Is also possible.

  The illumination light 105 deflected by the scanning optical systems 109-1 and 109-2 is irradiated to the eye 111 through the eyepiece lenses 110-1 and 110-2. The illumination light 105 irradiated to the eye 111 is reflected or scattered by the fundus. By adjusting the positions of the eyepieces 110-1 and 110-2, it is possible to optimally irradiate the fundus with illumination light in accordance with the diopter of the eye 111. In this embodiment, a lens is used for the eyepiece, but the eyepiece may be constituted by a spherical mirror or the like instead of these lenses.

  The reflected light reflected or scattered from the retina of the fundus of the eye 111 travels in the reverse direction along the path of the illumination light 105 when it is incident. A part of the reflected light is further reflected by the light splitting unit 106 to the wavefront measuring device 115. The light beam composed of a part of the reflected light guided to the wavefront measuring device 115 is used to measure the wavefront distribution of the light beam.

  In this embodiment, a Shack-Hartmann sensor is used as the wavefront measuring device 115 constituting the wavefront measuring means. In this embodiment, the Shack-Hartmann sensor is used as the wavefront measuring means, but the method for measuring the wavefront is not limited to the use of the wavefront measuring means. For example, a method of obtaining a wavefront by inverse calculation from an imaged point image using other wavefront measuring means such as a curvature sensor may be used.

  In FIG. 1, a part of the reflected light transmitted through the light dividing unit 106 is reflected by the light dividing unit 104, and the reflected light is guided to the light intensity sensor 114 through the collimator 112 and the optical fiber 113. The light that is the return light from the eye 111 scanned with the illumination light is converted into an electrical signal by the light intensity sensor 114, and an image in which the light intensity is arranged is configured by the control unit 117. In this embodiment, the image, which is a planar image, is further stored in a memory (not shown) by the control unit 117 and is displayed as a fundus image on the display 118, which is a display unit, after performing image processing described later.

  The wavefront measuring device 115 is connected to the adaptive optics control unit 116 and transmits the wavefront distribution measured by the wavefront measuring device 115 to the adaptive optics control unit 116. The wavefront correction device 108 is also connected to the adaptive optics controller 116, and performs spatial phase modulation of the wavefront distribution in accordance with an instruction from the adaptive optics controller 116 so as to compensate the transmitted wavefront distribution. More specifically, the adaptive optics control unit 116 calculates a correction amount for correcting to a wavefront distribution having no aberration based on the wavefront distribution measured by the wavefront measuring device 115, and the wavefront correcting device 108 receives the correction amount. Command to create a correction state corresponding to the calculation result.

  As described above, in the fundus imaging apparatus, since the eye to be examined is included in a part of the optical system, the optical system becomes indeterminate. For this reason, it is generally difficult to reach a wavefront distribution with small aberrations by measuring and correcting the wavefront distribution once. Therefore, the measurement and correction of the wavefront distribution are repeated, and correction is executed while converging the wavefront distribution to the aberration that can be imaged. In this embodiment, by performing such an operation, it is possible to reduce the aberration of the optical path until the illumination light reaches the fundus, and to reduce the size of the spot condensed on the fundus. As a result, an image with high spatial resolution is obtained.

  Here, the WF-SLO 120 used in this embodiment will be described. The configuration of WF-SLO is basically the same as the configuration of AO-SLO described so far. The WF-SLO 120 is different from the above-described AO-SLO in that the wavefront measurement device, the wavefront correction device, and the adaptive optics control unit are not provided. As described above, in this embodiment, the angle of view of the image captured by the WF-SLO 120 is wider than that of the AO-SLO. Further, the resolution of the image captured by the WF-SLO 120 is inferior to the resolution of the AO-SLO. That is, in this embodiment, the WF-SLO 120 can obtain a low-resolution and wide-angle image, and the AO-SLO can obtain a high-resolution and narrow-angle image.

  The wavelength of the light source that emits the illumination light used in the WF-SLO 120 is 900 nm. A dichroic mirror 119 is disposed in the optical path of the illumination light in the AO-SLO. The illumination light from the WF-SLO 120 is reflected by the dichroic mirror 119 and combined with the illumination light of the AO-SLO. The combined light is applied to the eye 111 through the eyepieces 110-1 and 110-2. The light irradiated to the eye 111 is condensed on the fundus by a crystalline lens that is an eye lens, reflected by the fundus, and returned to the fundus imaging apparatus. The light returned from the eye 111 is separated into light directed to the WF-SLO 120 and light directed to the optical path of the AO-SLO by the dichroic mirror 119 according to the wavelength band, and is guided to each optical system.

  In WF-SLO120, the intensity | strength of the obtained reflected light is arranged, a fundus image is comprised and acquired. An example of an image (referred to as a WF image) acquired by the WF-SLO 120 is shown in FIG. As shown in the figure, in the obtained WF image, the optic disc 200, the blood vessel 201, and the region 202 where the fovea exists are simultaneously imaged. The WF-SLO 120 is connected to the tracking control unit 121. The scanning position of the illumination light 105 is deviated from the planned position due to the eye movement such as the above-mentioned fixation fine movement, and the image of the area deviated from the area according to the eye movement is not an image of a desired area on the fundus. Will be obtained. The tracking control unit 121 obtains the magnitude and direction of this shift from the WF image as described below, and instructs the scanning unit to change the scanning position of the illumination light so that an image of a desired region can be obtained. Here, the operation for obtaining the image of the desired region described above is referred to as an operation for tracking the scanning position of the illumination light.

  Although not shown in the drawing, in the fundus imaging apparatus, a fixation target that prompts the subject to fixate his eyes to reduce eye movement is presented to the subject. The means for presenting the fixation target to the subject is configured using, for example, an organic EL display. The presentation position of the fixation target can be changed, and the direction of the eyes is changed by changing the presentation position. By changing the direction of the eye in this way, the location where the illumination light of the WF-SLO 120 and the AO-SLO is condensed on the fundus can be changed, and the position to be imaged can be changed. In this embodiment, an organic EL display is used for presenting the fixation target, but the present invention is not limited to this, and a liquid crystal display, a fluorescent display tube, or the like may be used.

  Here, the tracking function for correcting the imaging position by detecting the change of the illumination light irradiation position due to the eye movement and changing the scanning position of the illumination light scanner so as to correct the influence of the eye movement is described. To do. The tracking function in the present embodiment will be described with reference to the flowchart of FIG. 3 showing the tracking operation for executing the tracking function. Note that the processing flow for this tracking function is controlled by the tracking control unit 121 shown in FIG. 1 as described above. In the present embodiment, the tracking control unit 121, the chip tilt mirror 109-2, and the like constitute tracking means for tracking the scanning position of the illumination light so as to correct the detected positional deviation. The tracking control unit 121 preferably uses a GPU (Graphic Processor Unit) or an FPGA (Field Programmable Gate Array) that can perform high-speed processing operations in order to realize the real-time performance of the tracking function.

  When the tracking function is started, in step S301, the tracking control unit 121 uses the WF-SLO 120 to acquire the WF-SLO reference image 203 obtained by imaging the entire WF image shown in FIG. Here, the X direction in FIG. 2A corresponds to the main scanning direction of illumination light on the fundus, and the Y direction corresponds to the sub scanning direction. When acquiring the WF image, every time the illumination light 105 is scanned in the main scanning direction to obtain intensity information of one scan line, scanning is performed to shift the irradiation position of the illumination light 105 in the sub-scanning direction. In step S301, an image obtained by performing one scan of shifting the irradiation position in the sub-scanning direction is displayed as a reference image.

  Next, in step S302, the tracking control unit 121 uses the WF-SLO 120 again to acquire the detected image 204-1 illustrated in FIG. In this embodiment, the detected image 204-1 is a partial area of the WF image, and this image is called a strip image. In this embodiment, the size (length) in the Y direction of one strip image is set to 1/5 of the size (length) in the Y direction of the WF-SLO reference image 203. Note that the size of the strip image in the Y direction is arbitrary, and it is preferable that the strip image is appropriately changed according to the scanning speed, the number of required tracking control, and the like.

  In step S303, the tracking control unit 121 compares the WF-SLO reference image 203 and the detected image 204-1 and detects the direction and amount of relative displacement between these images. As a detection algorithm, a general image matching method may be used, and here, a phase only correlation method is used. The relative positional deviation is detected as a detection value ΔX in the X direction (horizontal direction of the fundus) in FIGS. 2A and 2B and a detection value ΔY in the Y direction (vertical direction of the fundus).

  The detection values ΔX and ΔY were in a state in which the eye to be examined moved by the direction and amount corresponding to these detection values at the timing when the detection image 204-1 was obtained, compared to when the WF-SLO reference image 203 was obtained. It is shown that. Accordingly, the tracking control unit 121 performs an operation for canceling the eye movement corresponding to the detected ΔX and ΔY and acquiring an image by AO-SLO in step S304. That is, the position where these detection values are canceled is commanded to the AO-SLO chip tilt mirror 109-2 as the scanning start position of the illumination light. The command is issued as a change amount of the scanning position of the illumination light by the tip tilt mirror 109-2 (a shake angle amount from the normal angle of the rotation center angle in the rotation operation of the mirror in a predetermined angle range).

  More specifically, for the command value (signal) commanding driving in the sub-scanning direction in the AO-SLO chip tilt mirror 109-2, the detected drive signals for ΔX and ΔY are added, and the mirror is added to the mirror. On the other hand, the scanning start position is changed. By changing the command value, the scanner scanning position for performing sub-scanning is changed (the scanning position is tracked by eye movement), and the acquisition position of the AO-SLO image is corrected. As a result, the imaging position of the AO-SLO is tracked in the X direction (horizontal direction of the fundus) and the Y direction (vertical direction of the fundus) with respect to the eye movement.

  When step S304 ends, it is confirmed in step S305 whether image acquisition in AO-SLO has ended. When the acquisition of the AO-SLO image is continued, the tracking control unit 121 moves the flow to step S302 and repeats the tracking operation of the image acquisition position from step S302 to step S304. Further, during this repetition, the WF-SLO 120 moves on the fundus of the detected image as shown in the detected images 204-2, 204-3, 204-4, and 204-5 shown in FIG. The acquisition position of is sequentially changed. For convenience, each detected image in FIG. 2B is drawn in the same manner as an image at a corresponding position on the WF-SLO reference image. However, in actuality, only irradiation and scanning of the illumination light 105 are performed in order to obtain this image on the fundus, and the irradiation and scanning positions deviate from the desired positions on the fundus according to the movement of the eyes. This image has not been obtained. For example, in the case of the detected image 204-1, a picture in the illustrated image was not obtained for the end part, and a picture of another part of the fundus not shown was obtained in the opposite end part. A strip image or the like is obtained. After acquiring the detected image 204-5, the tracking control unit 121 instructs the scanner (not shown) of the WF-SLO 120 to scan the illumination light so that the next acquired image is the detected image 204-1. Do. In this embodiment, the repeated operation of changing the acquisition position of the strip image is repeated so as to obtain the detected images 204-1 to 204-5 in this order.

  Assuming that the frame update period of the WF image (the image of the frame corresponding to the WF-SLO reference image 203 in this embodiment) is T, in this embodiment, the position detection and the change of the command value correspond to the number of times the detected image is acquired. Then, it can be performed five times during one update period T. By making the detected image 204 a strip image that is a part of the WF image, it is possible to increase the frequency of execution of position detection per unit time and the frequency of transmission of changed command values per unit time. As a result, the time delay when tracking the eye movement can be shortened, and the tracking residual with respect to the eye movement can be reduced. Further, in order to shorten the interval between frames of the WF image and the interval at the time of acquiring the detected image before and after this interval, it is also possible to acquire images in both the forward and backward passes in the sub-scanning direction. Specifically, the image acquired next to the detected image 204-5 may be the detected image 204-5 again instead of the detected image 204-1. In this case, it is also preferable to acquire the detected images 204-4, 204-3, 204-2, and 204-1 one after another in this order. By adopting such an acquisition order, it is possible to omit the movement of the scanner that returns the irradiation position of the illumination light in order to acquire the next detected image from the detected image 204-5. Therefore, the interval between frames can be shortened, that is, the time delay when tracking the eye movement can be further reduced.

  In parallel with the repetition of the flow from step S302 to step S304 performed by the tracking control unit 121, acquisition of an AO-SLO image by AO-SLO is performed. At that time, in step S304, a tracking function for shifting (changing) the scanning position of the illumination light corresponding to the movement of the eye operates. That is, when acquiring one AO-SLO image, a positional deviation of the region on the fundus scanned by the illumination light 105 is detected a plurality of times, and a tracking operation for correcting the detected positional deviation is executed. . Therefore, since the acquisition position of the AO-SLO image is corrected following the movement of the eye, a good AO-SLO image with little positional deviation can be obtained as the entire frame of the AO-SLO image. Providing AO-SLO moving images by continuously displaying images of a plurality of frames (one frame corresponds to one AO-SLO image) continuously acquired and with little mutual positional deviation in this way. Is possible. According to the AO-SLO moving image, it is possible to know blood flow in the fundus.

  In step S305, when the acquisition of the AO-SLO image is completed and it is not necessary to perform the tracking operation, the tracking control unit 121 advances the flow and ends the fundus tracking operation. In the present embodiment, the WF-SLO 120 and the tracking control unit 121 constitute detection means for detecting the fundus position shift when acquiring one AO-SLO image. As described above, the detection means detects a relative displacement between the WF-SLO reference image 203 and the detected images 204-1 to 20-5 acquired at different times from the WF-SLO reference image 203. To do. The detected positional deviation is used as positional deviation information used for tracking. In addition, it is preferable that the to-be-detected images 204-1 to 20-5 are acquired as a scheduled imaging range in a range included in a planned acquisition range on the fundus (on the inspection object) of the WF-SLO reference image 203.

  Here, when the fixation movement of the eye to be examined is relatively large, the AO-SLO image captured by operating the tracking function may be displaced or distorted in the frame plane. In the tracking program described above, the tracking control unit 121 gives a command value changed to correct the eye movement to the tip tilt mirror 109-2 so as to cancel the eye movement every time the strip image is acquired. Yes. However, for example, there is an eye to be examined that continues to move so that the amount of change in fixation is large and the direction of change often changes. With such an eye to be examined, the eye to be examined continues to move even during the acquisition of the strip image, and the amount of deviation between ΔX and ΔY obtained at the time of strip image acquisition is obtained from the result of integration of this movement. . In other words, the image acquisition area in the eye to be examined immediately before obtaining the deviation amount may not correspond exactly to the area from which the image obtained from the obtained deviation amount is obtained. In addition, when the eye movement is large, the influence of the eye movement immediately before and after the shift amount is obtained, and there is a further positional shift with respect to the position where the image is to be acquired. There is a possibility that the generated image will be generated.

  That is, in the case of such an eye to be examined, at the timing when the command value is issued to the tip tilt mirror 109-2, the eye has an image acquisition range that is different from the image acquisition range to be tracked with the immediately preceding strip image. There is a case of moving. As a result, in the captured AO-SLO image, misalignment occurs like a tomography or image distortion occurs at the scan position at the timing when a command value for correcting the misalignment is transmitted to the tip tilt mirror 109-2. May occur.

  In order to avoid the occurrence of such a phenomenon, the image width in the sub-scanning direction of the strip image is narrowed to increase the frequency of execution of position detection per unit time and the frequency of transmission of changed command values per unit time. It is also possible. However, if the image width is too narrow, for example, the detection accuracy of the relative displacement between the images to be detected for tracking performed in step S303 in FIG. 3 is lowered. For this reason, there is a limit to the correspondence method for narrowing the image width.

  For this reason, in this embodiment, the AO-SLO image captured by operating the tracking function is subjected to image processing to correct the positional deviation of the AO-SLO image and the distortion in the frame plane. That is, the tracking control unit 121 grasps the irradiation position information on the AO-SLO image corresponding to the timing at which the command value for correcting the irradiation position of the illumination light is transmitted to the tip tilt mirror 109-2, and the control unit 117. In consideration of the irradiation position information, image processing is performed on the AO-SLO image. As a result, a discontinuous portion in the AO-SLO image does not occur, and an AO-SLO moving image that does not give a sense of incongruity to the examiner even when the image is continuously displayed can be provided.

  Hereinafter, an image processing method in this embodiment will be described. FIG. 4 is a flowchart for explaining the image processing function in this embodiment. The processing function is controlled and executed by the control unit 117.

  When the image processing function is started, in step S401, the control unit 117 acquires an AO-SLO reference image for aligning the captured AO-SLO image. This reference image is used as a base image for alignment. As a method for selecting the reference image, an arbitrary frame among frames continuously displayed on the display 118 may be selected, or a user may designate a frame. Alternatively, the control unit 117 may automatically select a frame. When selecting automatically, for example, the frame having the best image quality may be selected from each frame of the image. Alternatively, an image having the largest common area (overlap) between the images may be selected. Furthermore, a frame may be selected by combining them. Further, when selecting by image quality, any image quality evaluation method may be used, for example, evaluation may be performed by contrast, SN ratio, or the like. When selecting by overlap, it is possible to perform alignment between frames as pre-processing and evaluate the overlap region. In this embodiment, a part of modules of the control unit 117 selects a reference image of a new image from each frame that is acquired by AO-SLO and becomes an original image when generating a new image in the eye 111. The selection means is configured.

In step S <b> 402, the control unit 117 performs a calculation for detecting misregistration for all of the frames continuously obtained as an AO-SLO image. Specifically, each frame is aligned with the reference image selected in step S401 by image matching processing. Here, an arbitrary method may be used for the image matching processing. In this embodiment, the phase correlation limiting method is used, and the deviation amounts (offset amounts) of all frames are obtained as Δ _A X and Δ _A Y. .

  In step S403, the control unit 117 determines the scan position in the AO-SLO image corresponding to the timing at which the command value changed for correcting the irradiation position is transmitted to the tip tilt mirror 109-2 in each frame. get. Specifically, the tracking control unit 121 stores the timing at which the command value is transmitted as the center of the scanning position by the AO-SLO tip tilt mirror 109-2 in order to cancel the eye movement in step S304 in FIG. deep. Then, the control unit 117 acquires the scan position of the AO-SLO image at the timing synchronized with the transmission of the command value. Alternatively, when the control unit 117 stores the AO-SLO image at the time of imaging, it may be stored with the scan position information embedded in the tag information of the image and referred to.

  Hereinafter, the scan position information will be described with reference to FIG. FIG. 5 is an explanatory diagram of a process for dividing and aligning frames in an AO-SLO image. In the figure, a frame 500 shows one frame of the AO-SLO image, and the X direction in FIG. The Y direction of the frame 500 is the sub-scanning direction of the illumination light, and corresponds to the driving direction of the illumination light by the chip tilt mirror 109-2. Further, when acquiring one frame of the AO-SLO image, the changed command value is transmitted to the tip tilt mirror 109-2 for tracking a plurality of times. The timing at which the tracking control unit 121 transmits this command value corresponds to the time immediately before scanning the scan line, which is indicated as times t1, t2, t3, t4, and t5 in the image shown in FIG.

  In step S404, the control unit 117 performs detailed alignment processing by image matching processing for each frame of the continuously acquired AO-SLO images with respect to the reference image selected in step S401. Note that the division of a single AO-SLO image into a plurality of images according to the time at which the misregistration is detected is executed by a module that functions as an image dividing unit in the control unit 117. The alignment process is performed for each of the divided images obtained by dividing the frame 500 based on the scan position acquired in step S403. The divided images obtained by dividing at times t1, t2, t3, t4, and t5 are shown as divided images 501-1, 501-2, 501-3, 501-4, and 501-5 in FIG.

Specific alignment processing will be described below. First, the frame 500 across the alignment target, the offset amount obtained by calculating for each frame delta _{A X,} the target frame using the delta _{A Y} is parallel moved in step S402, the AO-SLO reference image Perform position alignment. Next, alignment processing is performed on each of the divided images 501-1 to 50 after parallel movement on the AO-SLO reference image. Arbitrary image alignment processing may be used for the alignment processing, and here, a position shift is detected for each divided image using a phase only correlation method. Further, after detecting the positional deviation, the vector quantity of each positional deviation is distributed to each pixel of the image by using the bicubic interpolation method, and the non-rigid body is aligned.

  Note that misalignment detection and image complement processing for non-rigid body alignment may be performed using any method. Further, after detecting the displacement, rigid body alignment processing may be performed in which alignment is performed for each divided image. In this case, the method is not limited as long as alignment is performed for each divided image. In this embodiment, the alignment processing is performed for each rectangular divided image. However, in order to perform non-rigid registration with a finer divided image, each of the divided images 501-1 to 50-1 is divided into smaller images. May be processed. Further, when it is determined that the detection accuracy is low at the stage of detecting the displacement, a process may be performed in which the vector amount at the detected position is not used. In this case, the detection accuracy may be determined using an arbitrary method. Here, it is preferable to perform processing for determining that the accuracy is low when the peak calculated by the phase-only correlation method falls below a certain threshold value. Note that the above-described series of alignment processing is performed for each frame of the captured AO-SLO image.

  When the image processing for each frame of the AO-SLO image is completed, the control unit 117 advances the flow from step S404, and the image processing program is terminated. Each image (AO-SLO moving image) of the AO-SLO image that has undergone image processing such as alignment is stored in the control unit 117 and displayed on the display 118 as a fundus image as necessary. A part of the module of the control unit 117 constitutes an image generation unit that aligns each of the plurality of divided images and generates a new image. Further, the image generation means includes a first alignment means for performing alignment between a plurality of frames acquired based on the detected positional deviation, and a space between each divided image based on each image of the divided images. And second alignment means for performing alignment at.

  As described above, by obtaining an image using the tracking program and the image processing program according to the present embodiment, a moving image in which the positional deviation of the image captured by tracking and the distortion in the frame plane is accurately corrected is acquired. be able to.

[Second Example]
In the second embodiment, in addition to the image processing in the first embodiment described above, the result of detecting the shift in the imaging position due to the fixation eye movement of the eye to be imaged is also referred to, and the position of the captured image A method for performing the alignment will be described. That is, in this embodiment, a new image is generated based on the positional deviation detected from the WF-SLO image from the AO-SLO image acquired in a plurality of (multiple frames) as in the first embodiment. The AO-SLO reference image to be used is selected. In addition, since the structure of the fundus imaging apparatus in the present embodiment is the same as that in the first embodiment, description thereof is omitted here. The same applies to the tracking operation to be executed.

  In the following, the details of the image processing operation in which the second embodiment is different from the first embodiment will be described. FIG. 6 is a flowchart for explaining the image processing operation in the second embodiment. As in the first embodiment, the processing operation is controlled and executed by the control unit 117.

  When the image processing operation is started, the control unit 117 acquires information regarding eye movement during tracking in step S410. Specifically, the information regarding the positional deviation detected for each detected image (strip image) 204 of each WF image acquired in step S303 is regarded as eye movement. For example, the displacement amounts detected during the time required to capture one frame of the AO-SLO image are respectively (X1, Y1), (X2, Y2), ..., (Xn, Yn). ) Is extracted from the positional deviation amount held by the tracking control unit 121.

  Next, in step S411, based on the tracking information, the control unit 117 selects an AO-SLO reference image. Here, unlike step S401 in the first embodiment, the reference image is selected by evaluating the information about the positional deviation detected during the acquisition of one frame of the AO-SLO image acquired in step S410. Here, in order to improve the accuracy of image alignment, it is effective to select an image with as little image distortion and displacement as possible as a reference image. In the present embodiment, the position shift detected in the WF-SLO 120 corresponds to the eye movement acquired for each frame in step S410 on the XY coordinates (X1, Y1), (X2, Y2),. .., (Xn, Yn) are expanded in the direction and size, and each is evaluated. Then, a frame presumed to be acquired in a state where the eye movement is relatively small and the fixation is stable is selected as the AO-SLO reference image based on the overall position shift.

  Any method can be used as the eye movement evaluation method. In the present embodiment, the difference between adjacent detection points is calculated as the movement distance of the eye that moved while capturing the strip image of each WF image, and this value is used for evaluation. Specifically, (ΔX1, ΔY1) = (X1-X2, Y1-Y2),..., (ΔXn-1, ΔYn-1) = (Xn-1-Xn, Yn-1-Yn) are calculated. To do. These are based on information obtained from the WF-SLO 120, but are considered to indicate the transition of the movement distance of the eye during acquisition of one frame of the AO-SLO image. When the square root of each (ΔX, ΔY) or ΔX, ΔY is small, it is considered that there was no steep eye movement. Note that even if the average moving distance is small, there may be a case where a large movement is included instantaneously, and the AO-SLO image obtained in such a state is not suitable as a reference image. Therefore, in this embodiment, a frame that does not have a moving distance larger than a threshold value arbitrarily set in one frame and has the smallest moving distance is set as a reference image. Note that the evaluation method for one frame is not limited to this, and a frame with a relatively small amount of change in eye movement may be selected. Further, the method described in step S410 and the reference image selection method described in step S401 described above may be used in combination.

  Since the processing in steps S402 and S403 executed after the AO-SLO reference image is selected is the same as the processing in the first embodiment, the same reference numerals are added and description thereof is omitted here.

  In step S414, the control unit 117 performs inter-frame alignment in consideration of the tracking information. The alignment method here is basically the same as the method described in step S404, but different processing is performed depending on the displacement value acquired in step S410. First, the position shift values obtained at the timings t1, t2,..., Tn when the position shift correction is performed based on the information from the WF-SLO 120 are respectively (X1, Y1), (X2,. Y2), ..., (Xn, Yn) are associated. This operation is performed for each of the continuously acquired AO-SLO images.

  Then, the position shift value detected at time tn is evaluated, and processing different from normal alignment is performed. For example, when the detected positional deviation (Xn, Yn) is larger than the allowable operation range of the tip tilt mirror 109-2, it is predicted that the tracking has failed. Therefore, the image data when the corresponding amount of misregistration is detected is not used for alignment or a mask process is performed so as not to show to the user. In addition, the detected positional deviation (Xn, Yn) may be a value that overlaps all existing divided images in the same frame when the position of the image is shifted based on the amount of deviation. In such a case, since the process is duplicated, the alignment process is not performed, or the data is discarded and not used.

  Further, as calculated in step S411, when the eye movement distance (ΔX, ΔY) obtained from the difference between adjacent detection points of the detection position or the square root of ΔX, ΔY is greater than a certain threshold value. It is considered that there was a steep eye movement. In this case, the image corresponding to the sharp movement of the eye to be examined is not used for alignment, or is processed so as not to be shown to the user by performing mask processing. Eye movement has a steep movement called a saccade, and when this movement occurs, the image is distorted like a shooting star, causing image quality degradation. The previous processing corresponds to the case where such eye movement occurs.

  If it is determined from the detected misregistration (Xn, Yn) that the above-described condition is not an inappropriate image for alignment or the like, alignment processing is performed as in step S404 in the first embodiment. I do. In step S411, as described above, the eye movement is evaluated according to the value of the detected position when tracking is performed, and different processing is performed.

  When the image processing for each frame of the AO-SLO image is completed, the control unit 117 advances the flow from step S414, and the image processing operation is ended. Each image (AO-SLO moving image) of the AO-SLO image that has undergone image processing such as alignment is stored in the control unit 117 and displayed on the display 118 as a fundus image as necessary.

  As described above, by obtaining an image using the tracking operation and the image processing operation described in the present embodiment, a moving image in which the positional deviation of the image captured by tracking and the distortion in the frame plane is accurately corrected is acquired. It becomes possible to do.

  In this embodiment, unlike the image processing operation in the first embodiment, the selection of the reference image in consideration of the tracking information in step S411 and the alignment between frames in consideration of the tracking information in step S414 are performed. ing. However, even if only one of these processes is performed in the corresponding step in the first embodiment, the effect can be obtained.

[Other Examples]
In the first embodiment and the second embodiment, a chip tilt mirror capable of handling both main scanning (horizontal direction of the fundus) and sub-scanning (vertical direction of the fundus) as a scanner operated by the tracking function. 109-2 is used. However, even if a galvano mirror is used for sub-scanning and the tracking function is operated only in the sub-scanning direction, the effect of tracking is reduced by making the other configurations the same as those of these embodiments, but similar effects. Can be obtained.

  In addition, the scanning timing of illumination light in each frame of WF-SLO and AO-SLO may be synchronized. In this case, the timing for instructing the displacement correction to the tip tilt mirror 109-2, that is, the position at which the AO-SLO image is divided into the divided images for alignment is fixed in each frame of the AO-SLO image. . As a result, it is possible to fix the extraction position of the divided image used for inter-frame alignment.

  In the first and second embodiments, the WF-SLO 120 is used to acquire an image used for the tracking operation. However, it is also possible to construct a tracking system using a single AO-SLO optical system without using WF-SLO. In this case, both the reference image for tracking and the detected image are acquired by AO-SLO. In this configuration, the tracking operation and the image processing operation are performed using the same AO-SLO image. Even in such a case, the same effects as those of the above-described embodiments can be obtained by making other configurations similar to those of these embodiments.

  Further, the case where AO-SLO is used as an apparatus for acquiring an image subjected to image processing has been described as an example. However, an ophthalmologic apparatus that operates the eye to be examined with illumination light or measurement light, for example, OCT that can acquire an optical coherence tomographic image, is provided with the tracking function of the present invention by providing a WF-SLO. The same effects as those of the embodiment described above can be obtained.

  The present invention is not limited to the above-described embodiments, and various modifications and changes can be made without departing from the spirit of the present invention. For example, in the above-described embodiment, a fundus imaging apparatus that examines the fundus of the eye to be examined is described. It is also possible to apply the present invention to an inspection apparatus. In this case, the subject to be inspected does not cause aberrations, so that the compensation optical system is not necessary, but tracking and image processing are necessary, and the present invention has an aspect as a medical device other than the fundus imaging apparatus, for example, an endoscope. Therefore, it is desirable that the present invention is grasped as an inspection apparatus exemplified by the fundus imaging apparatus, and the eye to be examined is grasped as one aspect of the inspection object.

  Furthermore, the present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus execute the program. It can also be realized by a process of reading and executing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

101: light source 108: wavefront correction device 109: scanning optical system 111: eye 114: light intensity sensor 115: wavefront measuring device 116: adaptive optics control unit 117: control unit 118: display 119: dichroic mirror 120: WF-SLO
121: Tracking control unit

Claims (12)

Image acquisition means for acquiring an image based on return light from the inspection object scanned with illumination light;
Detecting means for detecting a positional deviation of the inspection object when acquiring an image of the inspection object;
Tracking means for tracking the scanning position of the illumination light so as to correct the detected displacement;
Image dividing means for dividing the one image into a plurality of divided images according to the time of tracking;
An inspection apparatus comprising: an image generation unit configured to generate a new image by aligning each of the plurality of divided images.   The detection means detects, as the positional deviation, a relative positional deviation between a reference image and a detected image in a region included in the reference image acquired at a time different from the reference image. The inspection apparatus according to claim 1.   The apparatus according to claim 1, further comprising a selection unit configured to select a reference image of the new image from an original image acquired when the new image of the inspection object is acquired by the image acquisition unit. The inspection apparatus according to 1 or 2.   The inspection according to claim 3, wherein the selection unit selects a reference image of the new image from the plurality of images of the inspection object acquired based on the detected displacement. apparatus. The image generating means includes
First alignment means for aligning a plurality of images acquired by the image acquisition means based on the detected displacement;
The inspection apparatus according to claim 1, further comprising: a second alignment unit that performs alignment between each of the divided images based on the images of the divided images. .   The inspection apparatus according to claim 1, wherein the image acquisition unit includes a wavefront detection unit that detects a wavefront of the return light, and a wavefront correction unit that corrects the wavefront. . The image acquisition means includes
First image acquisition means for acquiring an original image when generating the new image;
A second image acquisition means different from the first image acquisition means for acquiring an image for detecting the displacement;
When the first image acquisition unit acquires the one image, the detection unit detects the first image based on a plurality of images of the inspection object acquired by the second image acquisition unit. The inspection apparatus according to any one of claims 1 to 6, wherein a position shift of a region where the illumination light is scanned in the acquisition unit is detected.   The inspection apparatus according to claim 7, wherein the first image acquisition unit acquires a planar image of the inspection object generated based on the intensity of the return light.   The inspection apparatus according to claim 7, wherein the second image acquisition unit acquires a planar image of the inspection object generated based on the intensity of the return light.   The inspection apparatus according to claim 1, wherein the image generation unit generates a moving image of the inspection object based on the generated image. Acquiring an image based on the return light from the object scanned with the illumination light; and
A step of detecting a position shift of the inspection object when acquiring one image of the inspection object;
Tracking the scanning position of the illumination light so as to correct the detected displacement;
Dividing the one image into a plurality of divided images according to the tracked time;
And a step of aligning each of the plurality of divided images to generate a new image.   A program for causing a computer to execute each step of the inspection method according to claim 11.

Images