JP7318272B2 - OPHTHALMIC PHOTOGRAPHY AND OPHTHALMIC IMAGE PROCESSING PROGRAM - Google Patents

Description

The present disclosure relates to an ophthalmic imaging apparatus that processes an image of an eye to be examined, and an ophthalmic image processing program.

Conventionally, in the field of ophthalmology, there have been known various techniques for improving image quality such as S/N ratio by synthesizing a plurality of ophthalmologic images photographed at the same location by image processing. Such a method is widely used, for example, in front images such as SLO images and tomographic images such as OCT images (see Patent Documents 1 and 2, for example).

In recent years, the use and application of scanning ophthalmologic imaging apparatuses that take images of an eye to be inspected with a wider angle of view have been rapidly expanding (see, for example, Patent Document 3).

Further, there is a demand for higher resolution in scanning-type fundus imaging devices.

JP 2010-110392 A JP 2013-179972 A JP 2016-123467 A

However, in the scanning type photographing apparatus, the larger the angle of view or the higher the resolution, the longer the time required for photographing one image. As the imaging time increases, it becomes unavoidable that eye movements occur during the imaging (during scanning) of each image. That is, it becomes difficult to avoid distortion in each image. In such a case, it is difficult to deal with the method disclosed in Patent Document 1.

The present disclosure has been made in view of at least one of the problems of the prior art, and aims to provide an ophthalmic photographing apparatus and an ophthalmic image processing program capable of obtaining an image of an eye to be examined with little distortion. This is a technical issue.

An ophthalmologic imaging apparatus according to a first aspect of the present disclosure includes an imaging optical system including scanning means for scanning a tissue of an eye to be inspected with light from a light source, and a light receiving element for receiving return light from the tissue; a photographing control means for photographing a plurality of images of the eye to be inspected based on a signal from the light receiving element; as a template, corresponding points or corresponding regions between the image of the eye to be examined and the template are set at a plurality of positions between the image of the eye to be examined and the template, and for each corresponding point or each corresponding region Local matching for calculating a movement amount, and distortion correction processing for correcting two-dimensional distortion of the image of the eye to be examined with respect to the template based on the movement amount for each corresponding point or each corresponding region ; The image of the subject's eye after distortion correction is compared with the template, and the corresponding points or the corresponding regions are set more densely according to the comparison result, and the local matching and the distortion correction processing are performed. Start over .
An ophthalmologic imaging apparatus according to a second aspect of the present disclosure includes an imaging optical system including: scanning means for scanning a tissue of an eye to be inspected with light from a light source; and a light receiving element for receiving light returned from the tissue; a photographing control means for photographing a plurality of images of the eye to be inspected based on a signal from the light receiving element; as a template, corresponding points or corresponding regions between the image of the eye to be examined and the template are set at a plurality of positions between the image of the eye to be examined and the template, and for each corresponding point or each corresponding region local matching for calculating a movement amount, distortion correction processing for correcting the two-dimensional distortion of the image of the eye to be examined with respect to the template based on the movement amount for each corresponding point or each corresponding region, after distortion correction For each image of the eye to be inspected, a smaller weighting factor is set for each area as the distortion remaining with respect to the template is larger, and based on the weighting factor for each area in each image, a plurality of distortion-corrected a synthesizing process for obtaining a synthesized image by synthesizing the images of the eye to be inspected for each region.

An ophthalmic image processing program according to a third aspect of the present disclosure is executed by a processor of a computer to acquire a plurality of images of an eye to be inspected captured by a scanning optical system; Using any one of the images of the eye to be examined as a template, corresponding points or corresponding regions between the image of the eye to be examined and the template are set at a plurality of positions between the image of the eye to be examined and the template, local matching for calculating a movement amount for each corresponding point or each corresponding area; and distortion for correcting distortion of an image of the subject's eye with respect to the template based on the movement amount for each corresponding point or each corresponding area. and performing a correction process on the computer, the computer comparing the image of the subject's eye after distortion correction with the template, and further determining the corresponding points or the corresponding regions according to the comparison result. The local matching and the distortion correction processing are redone by setting a high density .
An ophthalmic image processing program according to a fourth aspect of the present disclosure is executed by a processor of a computer to acquire a plurality of images of an eye to be inspected captured by a scanning optical system; Using any one of the images of the eye to be examined as a template, corresponding points or corresponding regions between the image of the eye to be examined and the template are set at a plurality of positions between the image of the eye to be examined and the template, local matching for calculating a movement amount for each corresponding point or each corresponding area; and distortion for correcting distortion of an image of the subject's eye with respect to the template based on the movement amount for each corresponding point or each corresponding area. For each image of the eye to be inspected after correction processing and distortion correction, a smaller weighting factor is set for each region as the distortion remaining with respect to the template is larger, and the weighting coefficient for each region in each image is set to a synthesizing process for obtaining a synthesized image by synthesizing the plurality of images of the subject's eye after distortion correction for each region based on the computer.

According to the present disclosure, an image of the eye to be examined can be obtained with less distortion.

It is a figure which shows schematic structure of the apparatus which concerns on embodiment. It is a figure which shows schematic structure of an imaging optical system. 4 is a flowchart showing the flow of operations according to the embodiment; 9 is a flowchart for explaining template acquisition processing; FIG. 10 is a diagram illustrating a reference area (reference point) of a comparison image; FIG. 10 is a diagram showing reference points of a comparative image and corresponding points specified on a template by matching of reference areas; FIG. 4 is a diagram for explaining affine transformation between each corresponding point on a template and each reference point on a comparison image; FIG. 4 is a diagram for explaining the presence or absence of distortion based on corresponding points and affine mapping; 4 is a flowchart for explaining imaging processing; 9 is a flowchart for explaining first distortion correction/synthesis processing; An example of dividing a template is shown. This is an example of dividing a template, showing a more detailed division than in FIG. 8A. This is an example of dividing a template, showing a more detailed division than in FIG. 8B. FIG. 10 is a diagram for explaining the process of determining whether re-deformation is necessary in the image of the subject's eye after distortion correction; FIG. 10 is a diagram for explaining a method of calculating a weighting coefficient for addition based on similarity; FIG. 11 is a flowchart for explaining a second distortion correction/synthesis process; FIG. FIG. 10 is a diagram showing an example of how control points (corresponding points) are arranged in the second distortion correction/synthesis processing; FIG. 10 is a diagram showing an outline of a fitting curve of a movement amount; FIG. 10 is a diagram showing an example of a division mode when distortion correction/combination processing is applied to a photographed image with a wide angle of view. 14A shows a more detailed division than FIG. 14A. 14B shows a more detailed division than FIG. 14B.

Hereinafter, embodiments according to the present disclosure will be described with reference to the drawings. For the sake of convenience, hereinafter, unless otherwise specified, the processing content of the "ophthalmic image processing program" according to the embodiment will be described as being executed by the "ophthalmic photographing apparatus".

An ophthalmologic imaging apparatus captures a front image of the fundus (hereinafter referred to as a fundus image) as an image of an eye to be examined (ophthalmologic image). However, the "ophthalmic photographing apparatus" is not necessarily limited to photographing the fundus, and may photograph other parts such as the anterior segment of the subject's eye.

<Device configuration>
An ophthalmologic imaging apparatus 1 (see FIG. 1) according to the embodiment includes at least imaging optical systems 10 and 20 (see FIG. 2) and an image processor (image processor) 80 . Hereinafter, the ophthalmologic imaging apparatus 1 will be abbreviated as the present apparatus 1 . By having the image processor 80, the apparatus 1 becomes a computer that executes various image processing. The image processor 80 may be shared by a processor that controls the operation of the entire device. The image processor 80 may be separate from the processor that controls the operation of the entire device. A memory accessible from the processor of the image processor 80 may store an ophthalmic image processing program according to the embodiment.

For example, as shown in FIG. 1, this apparatus 1 is roughly divided into an optical unit 1a and a control unit 1b. 1b, respectively. The control unit 1b has various memories 75 in addition to a processor (CPU) 70 . The ophthalmic image processing program may be stored in the memory 75 . Further, an operation section 85 (user interface) may be connected to the control unit 1b. The operation unit 85 may be a pointing device such as a mouse and a touch panel, or may be another user interface. For example, a PC may be used as the control unit 1b.

The device 1 may also have a monitor 90 . For example, a captured fundus image is displayed on the monitor 90 . In addition, various GUIs may be displayed on the monitor 90 .

<Outline of the imaging optical system>
The imaging optical systems 10 and 20 capture fundus images. In this embodiment, the front image of the fundus is captured as the fundus image. However, it is not necessarily limited to this. The imaging optical systems 10 and 20 are roughly divided into an irradiation optical system 10 and a light receiving optical system 20 (see FIG. 2).

The imaging optical systems 10 and 20 are scanning optical systems. The imaging optical systems 10 and 20 illustrated in FIG. 2 have a scanning unit 16 as an example of scanning means. The imaging optical systems 10 and 20 also have three light receiving elements 25, 27 and 29 as light receiving elements. In FIG. 2, the imaging optical systems 10 and 20 are confocal optical systems.

Unless otherwise specified, the imaging optical systems 10 and 20 are assumed to be two-dimensional scan type optical systems in the following description. In the two-dimensional scan type, illumination light is formed in a spot shape on the fundus and is scanned two-dimensionally on the fundus. However, it is not necessarily limited to this, and a line scan type (or slit scan type) optical system may be applied. In this case, the illumination light is formed in a line shape or a slit shape on the fundus, and the illumination light is scanned in a direction intersecting the cross-sectional longitudinal direction of the luminous flux of the illumination light.

<Irradiation optical system>
First, the irradiation optical system 10 will be described. The irradiation optical system 10 has a scanning unit 16 that condenses the illumination light into a point on the fundus and causes the scanning unit 16 to scan the illumination light on the fundus.

The scanning unit 16 (an example of scanning means) scans the illumination light in two different directions. For convenience of explanation, it is assumed that the illumination light is scanned by raster scanning in the following explanation. In this case, as shown in FIG. 2, the scanning section 16 may include a first optical scanner 16a for main scanning and a second optical scanner 16b for sub-scanning. However, it is not necessarily limited to this, and the scanning unit 16 may be a two-degree-of-freedom device (for example, MEMS or the like). Any one of a galvanomirror, a polygon mirror, a resonant scanner, a MEMS, an acoustooptic device (AOM), and the like may be appropriately selected as the optical scanner.

The irradiation optical system 10 irradiates the fundus with illumination light from the light source 11 . The irradiation optical system 10 may be capable of simultaneously irradiating light in a plurality of wavelength ranges, or selectively switching and irradiating light.

Unless otherwise specified, the irradiation optical system 10 is assumed to irradiate visible light and infrared light in the following description.

The irradiation optical system 10 also has a beam splitter 13 , a lens 14 and an objective lens 17 .

The beam splitter 13 couples and separates the optical paths of the irradiation optical system 10 and the light receiving optical system 20 . The beam splitter 13 may be a perforated mirror or other member.

Lens 14 is a focusing lens and is displaced by a drive (not shown). The lens 14 and the drive section are used as a focus adjustment section 40 (visibility correction section) in this embodiment. In this embodiment, the position of lens 14 is controlled by processor 70 .

The objective lens 17 is an objective optical system of the device 1 . The objective lens 17 is used to guide the laser light scanned by the scanning unit 16 to the fundus Er. The objective lens 17 forms a pivot point P at the position of the exit pupil. At the turning point P, the illumination light that has passed through the scanning unit 16 is turned. The objective optical system may be a reflecting system using a mirror instead of a refractive system like the objective lens 17 .

The illumination light that has passed through the scanning unit 16 passes through the objective lens 17, passes through the turning point P, and is irradiated to the fundus Er. By turning the illumination light around the turning point P, the illumination light is scanned on the fundus Er. The illumination light applied to the fundus Er is reflected at a condensing position (for example, the surface of the retina). The fundus reflected light of the illumination light is emitted from the pupil as parallel light.

<Light receiving optical system>
Next, the light receiving optical system 20 will be described. The light receiving optical system 20 may have one or more light receiving elements. The light-receiving optical system 20 guides the return light from the fundus Er to at least one of the light-receiving elements 25, 27, and 29 to receive the light.

As shown in FIG. 2 , the light receiving optical system 20 in this embodiment may share the members arranged from the objective optical system 17 to the beam splitter 13 with the irradiation optical system 10 . In this case, the light from the fundus Er traces the optical path of the irradiation optical system 10 and is guided to the beam splitter 13 . The beam splitter 13 guides the light from the fundus Er to the independent optical path of the light receiving optical system 20 .

Further, the light receiving optical system 20 shown in FIG. 1 has a lens 21, a pinhole plate 23, and a light separation section (light separation unit) 30 on the reflected light path of the beam splitter 13. FIG.

The pinhole plate 23 is an example of a harmful light removing section. Since the pinhole plate 23 is arranged on the conjugate plane of the fundus, it removes light from positions other than the condensing point (or focal plane) of the fundus Er and receives (light from the condensing point) as a light receiving element. Leads to at least one of 25, 27, 29.

The light separation unit 30 separates light from the fundus Er. In this embodiment, the light separation unit 30 wavelength-selectively separates the light from the fundus Er. Further, the light separating section 30 may also serve as a light branching section that branches the optical path of the light receiving optical system 20 . For example, as shown in FIG. 2, the light separation section 30 may include two dichroic mirrors (dichroic filters) 31 and 32 having different light separation characteristics (wavelength separation characteristics). The optical path of the light receiving optical system 20 is branched into three by two dichroic mirrors 31 and 32 . Also, one of the light receiving elements 25, 27 and 29 is arranged at the end of each branched optical path.

For example, the light separation unit 30 separates the wavelengths of light from the fundus Er, and causes the three light receiving elements 25, 27, and 29 to receive light in different wavelength ranges. For example, light of three colors of blue, green, and red may be received by the light receiving elements 25, 27, and 29 one by one. In this case, a color image may be generated based on the light receiving results of the light receiving elements 25 , 27 and 29 .

In addition, the light separation unit 30 may allow different light receiving elements to receive the fluorescence from the fundus and the infrared light that is the fundus reflected light of the observation light. Thereby, it may be possible to capture an infrared image simultaneously with a fluorescence image. The fluorescence image may be an autofluorescence image or a contrast-enhanced fluorescence image.

The imaging optical systems 10 and 20 described above have a plurality of light receiving elements, so that each light receiving element can simultaneously receive return light having a wavelength corresponding to each light receiving element. However, it is not necessarily limited to this. For example, as in "Japanese Patent Laid-Open No. 2008-228781" by the present applicant, returning light in different wavelength ranges is alternately (sequentially) received by one light receiving element, so that each frame (or even each line) However, the present disclosure can also be applied to devices that acquire fundus images based on different wavelength ranges.

<Switching the angle of view>
The imaging optical systems 10 and 20 may have an angle-of-view switching section for changing the angle of view. For example, the angle-of-view switching unit may change the angle of view by switching the optical configuration of the objective optical system in the imaging optical systems 10 and 20 . Specifically, the angle-of-view switching section may include an insertion/removal mechanism for inserting/removing the optical element with respect to the objective optical system. Lenses, mirrors, prisms, and the like can be used as optical elements. Also, the angle-of-view switching unit may be a zoom mechanism that changes the refraction state by changing the positional relationship of two or more lenses along the optical path. Also, a refractive power variable lens such as a liquid crystal lens may be provided as the view angle switching section. Furthermore, the angle-of-view switching unit may include the scanning unit 16, and the angle of view may be switched by changing the swing angles of the optical scanners 16a and 16b. Note that the imaging optical systems 10 and 20 in the present embodiment may be capable of capturing a fundus image at an angle of view of 80° or more (with reference to the exit pupil). The angle-of-view switching unit may switch between a first angle of view less than 80° and a second angle of view greater than or equal to 80°.

<Acquisition (capture) of fundus image>
The processor 70 forms a fundus image, for example, based on the received light signals output from the light receiving elements 25 , 27 and 29 . More specifically, the processor 70 forms a fundus image in synchronization with scanning (here, raster scanning) by the scanning unit 16 .

In this embodiment, the light receiving elements 25, 27 and 29 form three readout channels. The processor 70 generates up to three types of images corresponding to three readout channels (hereinafter simply referred to as “channels”) for each raster scan. In this embodiment, various fundus images are generated by the image processor 80 .

The image processor 80 may acquire, as observation images, a plurality of frames of fundus images sequentially formed based on the operation of the apparatus as described above. The observation images of each frame may be displayed on the monitor 90 in chronological order. The observation image is a moving image made up of fundus images acquired substantially in real time.

“Acquisition” in this embodiment means that the fundus image is placed in a state that can be processed by the image processor 80 . For example, the fundus image is acquired by storing the fundus image in a predetermined memory area accessible by the image processor 80 .

In addition, part of the plurality of fundus images that are sequentially formed is taken in (captured) as a photographed image (capture image). At that time, the captured image is stored in a storage medium (for example, memory 75). A storage medium in which captured images are stored may be a non-volatile storage medium (eg, hard disk, flash memory, etc.). In this embodiment, for example, a fundus image formed at a predetermined timing (or period) after a trigger signal (for example, a release operation signal, etc.) is output is captured.

<Setting the shooting mode>
The processor 70 may change the shooting mode (shooting method). The processor 70 may change at least one of the wavelength of the illumination light and the channel for reading the received light signal according to the shooting mode. The type of fundus image acquired as a captured image is changed according to the capturing mode. For example, a color image may be obtained as the photographed image in the first photographing mode, and a fluorescence image may be obtained as the photographed image in the second photographing mode.

Also, the insertion/removal of a filter (not shown) may be changed according to the shooting mode.

<Change in the time required to acquire one fundus image>
The time required to acquire one fundus image may be changeable. In this case, the processor 70 controls the scanning speed in the scanning section 16 to change the required time. As the required time is changed, the number of pixels per fundus image (in other words, resolution) is changed. At this time, the scanning speed of both the main scanning direction and the sub-scanning direction may be controlled, or only one of the scanning speeds may be controlled.

As an example, the fundus image may be formed with three different numbers of pixels: 4096×4096 (approximately 0.6 sec), 1024×1024 (approximately 0.14 sec), and 512×512 (approximately 0.07 sec). In addition, the numerical value in parenthesis () is an example of required time.

<Description of operation>
Next, the operation of taking a fundus image with the present apparatus 1 will be described with reference to FIGS. 3 to 12. FIG.

As shown in FIG. 3, the processor 70 first starts turning on the fixation lamp and acquiring and displaying an observation image (S1, S2). As the lighting control of the fixation lamp, the imaging optical systems 10 and 20 may be controlled so as to temporarily turn on the visible light at the timing when the light is scanned to a predetermined presentation position. In this manner, an internal fixation lamp is formed and may be presented to the subject's eye. The observation image is preferably displayed at a higher frame rate. Therefore, the number of pixels in the observed image is set to a relatively small value (here, 512×512).

Also, various adjustments and settings are performed (S3). For example, the positional relationship between the subject's eye and the device may be adjusted, the focus state may be adjusted, or other adjustments may be made. Adjustment may be manual or automatic. If manually adjusted, the examiner may refer to the observed image. Moreover, the processor 70 may set at least one condition of the imaging mode and the number of pixels in the captured image based on the operation input of the examiner. The number of pixels of the captured image may be set to a value larger than that of the observed image.

Thereafter, template acquisition processing is performed (S4). A template that serves as a reference image for alignment of captured images is obtained by the template obtaining process. The template may be captured under preset conditions in the process of S3. The template may be an image captured under the same conditions as the captured image, or may be an image captured under conditions different from the captured image. Details of the template acquisition process will be described later with reference to FIG.

The shooting process (S5) is executed with the template acquired. The shooting process is executed, for example, based on a trigger signal (eg, release operation signal, etc.). Details of the shooting process will be described later with reference to FIG. After the photographing process (S5) is completed, the photographing result may be saved and displayed on the monitor 90 (S6). When the fundus image is captured properly, the captured image may be displayed as the captured image. Further, if an error occurs during photographing, the fact may be notified via the monitor 90 .

<Template acquisition processing>
Next, the template acquisition process (S5) according to the embodiment will be described in detail with reference to FIG. In the template acquisition process, first, at least one fundus image is captured as a template (S11). The template is preferably a fundus image of the same type as the photographed image obtained by the photographing process. Therefore, capture may be performed as a template based on a trigger signal that initiates imaging.

In this embodiment, the comparison image (second template) is obtained at a timing different from that of the template (S12). The comparison image is a fundus image captured under conditions set to reduce distortion compared to the template. The comparison image is compared with the template by image processor 80 to detect distortions in the template. Furthermore, in this embodiment, the image processor 80 corrects distortions in the template.

An example of the above condition is that the time required to acquire the comparison image is shorter than the time required to acquire the image that serves as the template. In this case, the number of pixels of the comparative image may be smaller than that of the template (and captured image). For example, the comparison image may be captured with 512×512 pixels, whereas the template may be captured with 1024×1024 or 4096×4096 pixels.

In addition, as another example of the above conditions, the illumination light used to acquire the comparison image is less likely to be dazzling to the subject than the illumination light used to acquire the template. mentioned. In other words, the comparison image may be captured under conditions set so that reflexive eye movements due to glare are less likely to occur than when the template is captured. As a specific example, the comparison image may be an infrared fundus image, and the template may be a visible fundus image. Note that an observation image, which is one type of fundus image using infrared light, may be used as the comparison image. The shooting of the comparison image itself may be performed before the input of the trigger signal. For example, an observation image (or an added image of a plurality of observation images) stored in memory before the trigger signal is input may be used as the comparison image.

Further, the process of S12 may include the process of taking a comparative image. In this case, the processor 70 changes any of the above conditions from the template and takes a fundus image as a comparison image. Some or all of the time required for acquisition, the wavelength range of the illumination light, and the channels may differ between the comparison image and the template. Moreover, it is preferable that the shooting range of the comparative image is the same as that of the template and the shot image, or includes the shooting range of the template and the shot image.

<Specific Examples of Distortion Detection Processing and Determination Processing>
A specific example of the distortion detection process (S13) and the determination process (S14) will be described with reference to FIGS. 5A to 5D. In a specific example, the image processor 80 obtains a plurality of corresponding points between the template and the comparison image, and detects the difference in displacement between the respective corresponding points as distortion.

First, as shown in FIG. 5A, a plurality of reference areas are set in the comparison image. In FIG. 5A, regions h1 to h5 with different positions are used as reference regions. Preferably, four or more reference areas are set.

In FIG. 5B, reference points i1 to i5 (corresponding points on the comparison image side) shown on the comparison image are center points of regions h1 to h5, respectively. The image processor 80 obtains corresponding points j1 to j5 corresponding to the reference points i1 to i5 on the template. The reference point and the corresponding point corresponding to the reference point are considered to indicate substantially the same position on the fundus. For example, the image processor 80 performs template matching for each of the regions h1-h5. In matching, the degree of similarity between each area of the template and each of the areas h1 to h5 is obtained. On the template, the region with the highest degree of similarity with each of the regions h1 to h5 is the region corresponding to the regions h1 to h5. Corresponding points j1-j5 are identified as the center of each corresponding region. For matching, various processes such as the phase-only correlation method can be applied.

Note that "matching" is image processing for obtaining a position where the degree of similarity between two images is maximized (equal to or greater than a threshold value) (by obtaining a degree of similarity map). In the various types of matching in this embodiment, not only Phase Only Correlation (POC) but also various degrees of similarity may be used. For example, SSD (Sum of Squared Difference), SAD (Sum of Absolute Difference), and normalized cross-correlation (NCC: Normalized Cross-Correlation, ZNCC: Zero-mean Normalized Cross-Correlation), etc. good.

Next, the image processor 80 obtains a transformation matrix M for affine transformation between the reference points i1 to i5 and the corresponding points j1 to j5, and converts each of the reference points i1 to i5 on the template with the transformation matrix M. (see FIG. 5C). The transformation matrix M can be obtained by having four or more reference points i1 to i5 and corresponding points j1 to j5 in each image. If the eye moves (specifically, if the movement changes) during imaging, two-dimensional distortion will occur in the template. It is considered that the deviation between the mapping of the reference points i1 to i5 by the transformation matrix M and the corresponding points j1 to j5 becomes large in the distorted region.

In FIG. 5C, five circles k1-k5 are centered on the mapped points, and the size of the circles indicates the allowable range of distortion. The image processor 80 determines that the distortion is within the allowable range when all the corresponding points j1 to j5 are included inside the circles k1 to k5 (S14: YES). On the other hand, since there are corresponding points j1 to j5 that are not included inside the circles k1 to k5, it is determined that the distortion is outside the allowable range (S14: NO). For example, FIG. 5D shows a template with unacceptable distortion around corresponding points j3 and j4.

When it is determined that the distortion exceeds the allowable range (S14: NO), in the example of FIG. 4, the template is reacquired (S11). The reacquired template is subjected to distortion evaluation again according to the flow described above. As a result, it is possible to obtain a template in which the distortion with respect to the comparison image is within an allowable range. In other words, it is possible to select an image that is closer to a comparative image taken under conditions set to reduce distortion compared to the template as a template. As a result, various kinds of processing of the photographed image based on the template are likely to be performed satisfactorily.

However, if the distortion exceeds the allowable range, the condition may not be suitable for the eye to be inspected, such as the eye being difficult to fix. In other words, it is conceivable that an image of an eye to be examined whose fixation is difficult to stabilize is captured. Even if a new fundus image such as a template is taken for such an eye under the same conditions as the previous time, there is a possibility that the distortion of the new image will still exceed the allowable range.

Therefore, when it is determined that the distortion exceeds the allowable range (S14: NO), the processor 70 performs the fundus image under the condition (the third condition in this embodiment) set to reduce the distortion. (Third image in this embodiment) may be captured. In other words, the third image is captured under the third condition in which distortion is less likely to occur than the pre-captured template (one of the first images in this embodiment). The third image may be a template (third template) obtained by recapturing. Moreover, not only a template but also a fundus image synthesized based on the template may be used. Also, when using the third image as a template, a second distortion detection process may be performed by the image processor 80 . The comparative image used in the second distortion detection process may be a fundus image captured under conditions set so that the distortion is less than that of the third image.

Further, when it is determined that the distortion has exceeded the allowable range (S14: NO), the following processing may be performed instead of re-imaging or along with re-imaging. For example, shooting may be interrupted. For example, when imaging with visible light, the burden on the subject can be reduced by immediately interrupting imaging. Further, in this case, in the process of S6, a message to that effect may be displayed on the monitor 90 as the photographing result.

However, even if it is determined that the distortion exceeds the allowable range (S14: No), it is not always necessary to interrupt the photographing process. In this case, for example, the processor 70 may cause the monitor 90 to display a confirmation screen for asking the examiner whether or not to adopt the template whose distortion has been determined to exceed the allowable range. With the confirmation screen displayed, the image processor 80 may delete the template by accepting an input operation for rejecting the use of the template. After deletion, the template may be recaptured manually or automatically.

For example, a synthesized image of the template and the comparison image may be displayed on the monitor 90 as the confirmation screen. The composite image may be an image obtained by superimposing the comparison image and the template. As a specific example, the comparison image and the template may be images expressed in different color channels. Alternatively, the image may be an image in a checkered pattern display format in which the comparison image and the template are alternately and repeatedly arranged (see, for example, Japanese Unexamined Patent Application Publication No. 2013-15421 filed by the present applicant).

In this embodiment, the template distortion correction process (S15) may be performed based on the comparison image. The distortion correction process may be, for example, non-rigid registration or rigid registration. For non-rigid body registration, for example, the technique described later can be used. By performing the distortion correction process, it is possible to obtain an image in which distortion with respect to the comparison image is further reduced as a template for the captured image.

Also, the template may be an added image of a plurality of fundus images. If the template and the comparison image have different channels (or different wavelength ranges of illumination light), different features may be drawn between the template and the comparison image. . In this case, if the distortion correction of a plurality of templates is performed based on the comparison image, the distortion of some of the plurality of templates may not be properly corrected. In this case, as a result, the difference in channels can cause noise in the composite image based on the template after distortion correction. Therefore, for example, one of the templates for which distortion correction has been completed for the comparison image may be used as a fourth template, and distortion correction for the remaining photographed images may be performed based on the fourth template. Since the fourth template and other captured images have the same channel, the above noise is easily suppressed.

<Detailed description of shooting process>
In the example of FIG. 3, after the template is obtained as described above, the photographing process (S5) is executed.

Here, the photographing process (S5) according to the embodiment will be described in detail with reference to FIG. In the photographing process, as described above, a photographed image is captured (S21). In the distortion correction/synthesis processing (S22) of the captured image, at least the two-dimensional distortion caused by eye movement during photographing is corrected with reference to the template.

At this time, in the present embodiment, by local matching, corresponding points or corresponding regions between the image of the eye to be examined and the template are set at a plurality of positions between the image of the eye to be examined and the template, and each corresponding point or corresponding region The amount of movement (also referred to as the amount of positional deviation) for each position is calculated. Then, based on the amount of movement for each corresponding point or each corresponding region acquired by local matching, the entire captured image is transformed. At this time, the captured image may be transformed into a non-rigid body (non-rigid registration). This corrects distortion (two-dimensional distortion) in the captured image. In this embodiment, the distortion-corrected photographed image is combined with the template. At this time, for example, the processes of S21 and S22 may be repeated until a predetermined number of shot images are synthesized. Alternatively, the image quality of the synthesized image may be evaluated each time a new shot image is synthesized, and the shooting process (S5) may be terminated when the image quality evaluation value exceeds the threshold. Through such photographing processing, a composite image of a plurality of photographed images is obtained as a photographing result. The composite image may be, for example, an added image of a plurality of photographed images. The composite image may be stored in the memory 75 or displayed on the monitor 90 as a photographing result.

<First Distortion Correction/Synthesis Processing>
First, the flow of the first distortion correction/synthesis processing applicable in this embodiment will be described based on FIG. In the first distortion correction/synthesis processing, the template image and the captured image are divided into a plurality of blocks (also referred to as divided regions), and the amount of movement between corresponding blocks between the two images is calculated. (local matching). Then, the entire captured image is non-rigidly deformed based on the amount of movement of each block. Note that the blocks referred to here are examples of corresponding regions set at a plurality of positions in the template image and the captured image.

In FIG. 7, it is determined based on global matching whether or not the photographed image captured immediately before (by the processing of S21) is an image suitable for distortion correction (S31). Global matching in this embodiment is processing for roughly aligning a template image and a photographed image. For example, matching between the entire images may be performed, or matching using only a partial area (for example, central area) of the image may be performed. If the maximum value of similarity (for example, correlation) in global matching is equal to or greater than a predetermined threshold, the captured image may be determined to be suitable for correction (S41: GOOD). If the degree of similarity is equal to or less than the threshold, the captured image may be determined as unsuitable for correction (S41: BAD). In the latter case, the distortion correction/synthesis processing may be terminated without adding the photographed image captured immediately before to the template.

Next, local matching is performed by the image processor 80 (S32). The template image and the captured image are each divided into a plurality of blocks (also called divided regions and corresponding regions), and matching is performed between corresponding blocks. As an example, as shown in FIG. 8A, it may be divided into 12 blocks, which are divided horizontally into 3 equal parts and vertically into 4 equal parts. For example, matching may be performed for each block, and the maximum value of similarity (for example, correlation by POC) and the amount of movement when the similarity is maximized may be obtained for each block.

The amount of movement of each block reflects the distortion of the image. Therefore, the finer the block is divided, the more accurately the distortion can be detected. On the other hand, the finer the block, the greater the processing load. Therefore, in this embodiment, the result of matching is evaluated after division, and the number of divisions is increased stepwise (recursively) according to the result of matching. This makes it possible to achieve both accuracy and reduction in processing load. As an example, based on the degree of similarity for each block, it is determined whether or not the number of divisions is appropriate (S33). For example, when there are a predetermined number or more of blocks whose maximum value of similarity exceeds the threshold, it may be determined that the number of divisions is appropriate (sufficient) (S33: YES). On the other hand, when the maximum similarity value is less than the threshold, the number of divisions may be determined to be inappropriate (insufficient) (S33: NO). In this case, in this embodiment, the image is divided into smaller blocks (S32), and matching is performed again (S33). For example, FIG. 8A (first stage: vertical 4 divisions, horizontal 3 divisions) ⇒ FIG. 8B (second stage: vertical 8 divisions, horizontal 3 divisions) ⇒ FIG. 8C (third stage: vertical 12 divisions, horizontal 3 divisions) ⇒ FIG. . . , the number of divisions may be increased step by step.

In the present apparatus 1, the scanning speed in the horizontal direction (main scanning direction) is much faster than that in the vertical direction (sub-scanning direction), so distortion is less likely to occur in each main scanning line. Therefore, it is preferable that the number of divisions in the sub-scanning direction is larger than the number of divisions in the horizontal direction (main scanning direction). Also, each block may be elongated in the horizontal direction (in other words, the horizontal direction is the longitudinal direction). 8A to 8C, each block is formed as a rectangle whose longitudinal direction is the horizontal direction. It is preferable to divide into two or more blocks in the horizontal direction in order to detect the distortion due to the rotation component. In addition, in FIGS. 8A to 8C, the number of divisions in the horizontal direction is not increased stepwise like the number of divisions in the vertical direction. However, the number of divisions in the horizontal direction is not necessarily limited to this, and the number of divisions in the horizontal direction may be increased stepwise.

Next, the captured image is non-rigidly (non-linearly) deformed based on the amount of movement of each block in the final divided state (S34). Various techniques such as the thin plate spline method and the B-spline method are known as non-rigid deformation algorithms. In this embodiment, any one of various methods can be applied as appropriate. For example, when the thin plate spline method is applied, the center of each block is moved based on the amount of movement, and other points are moved by thin plate spline interpolation. As a result, the entire captured image may be deformed.

Next, it is determined whether re-deformation is necessary (S35). For example, the image processor 80 may obtain the degree of similarity between the template and the captured image after distortion correction, and determine whether or not re-deformation is necessary based on the degree of similarity. Similarity can be evaluated, for example, by mutual information between two images.

Here, if it is determined that re-deformation is necessary (S35: YES), the number of divisions of the photographed image is increased, and then the processes of S32 to S35 are repeated. For convenience, the number of repetitions is counted as the first loop, the second loop, the third loop, and so on. An upper limit may be set for the number of times the process is repeated. In the example shown in FIG. 9, the processing can be repeated up to the third loop.

For example, the determination may be made based on a comparison between a value indicating similarity and a predetermined threshold. If the value indicating the degree of similarity is greater than or equal to the threshold, it may be determined that re-transformation is unnecessary.

On the other hand, if the value indicating the degree of similarity is below the threshold, it may be determined that retransformation is necessary. In this case, for example, the processing may be repeated up to the third loop, and the image having the maximum value indicating the degree of similarity may be adopted as the object of the synthesis processing with other captured images. However, since it is not always the case that distortion correction has been performed correctly in an image with the maximum similarity, it is compared with a second threshold that defines the lower limit of the similarity, and if the similarity exceeds the second threshold, image with the maximum similarity may be adopted. In the example of FIG. 9, the degree of similarity between the captured image before deformation and the image after distortion correction in the first loop is used for the second threshold. In FIG. 9, a similarity of 1.18 is used as the second threshold. If the maximum value of the degree of similarity when the processing is repeated up to the third loop does not exceed the second threshold, it is considered that the distortion correction has failed. In this case, none of the images is adopted as a target of synthesis processing with other captured images.

Weighted addition calculation is performed on the photographed image after distortion correction adopted as the target of the synthesis process (S36). A weighting calculation is performed for each region of the image. For example, the template and the captured image after distortion correction are divided vertically and horizontally into a plurality of blocks (in FIG. 10, 16 blocks are divided vertically and horizontally), and the similarity of the corresponding blocks between the images is obtained. good too. The position of each block is then matched in each image. The similarity of corresponding blocks may be evaluated, for example, by normalized cross-correlation (ZNCC) or other similarity. A weighting factor for each block may be calculated according to the similarity of each block.

The captured image after distortion correction is added to another image after applying the calculated weighting factor to each region (S37). A smaller weighting factor is added to an area where more distortion remains with respect to the template. As a result, it is possible to suppress the influence of a region in which a large amount of distortion remains in the synthesized image.

The first distortion correction/synthesis processing may be performed as described above.

<Second Distortion Correction/Synthesis Processing>
Next, the flow of the second distortion correction/synthesis processing applicable in this embodiment will be described with reference to FIG.

In the second distortion correction/synthesis processing, a plurality of control points (reference points) are set on a plurality of scanning lines in the template. Although some details are different, local matching of a small region containing each control point can be used to determine the amount of movement (shift) of the control points (that is, the corresponding points) between the template and the captured image. In the second distortion correction/synthesis processing, the method of weighted addition is significantly different from that in the second distortion correction/synthesis processing. In the second distortion correction/synthesis processing, in each line, the amount of movement of the control point is fitted with a predetermined function, and the fitting error of each control point and the change in the amount of movement for adjacent lines are taken into consideration. weighting coefficients are derived. Details will be described below.

In FIG. 11, first, it is determined whether or not the captured image is suitable for distortion correction (S41). Since this process may be the same as the process of S31 in the first distortion correction/synthesis process, detailed description thereof will be omitted.

Next, the image processor 80 sets a plurality of control points on the template (S42). A plurality of control points are set for each of a plurality of lines (main scanning lines).

The line on which the control points are set may be predetermined. As an example, in FIG. 12, control points (indicated by circles in FIG. 12) are set on eight scanning lines arranged at regular intervals.

After setting the control points, local matching is performed. Specifically, the amount of movement of each control point between the template and the captured image is calculated (S43). For example, based on local matching by a small area containing the control point, the position of the point corresponding to the control point in the template (that is, the control point on the captured image side, indicated by Δ in FIG. 12) is searched, and both may be calculated. A small region may be a region of a predetermined size centered on a control point.

In the processing of S43, the movement amount may be calculated for each line. As shown in FIG. 13, for example, the movement amounts of a plurality of control points in one line may be fitted with a function. Any one of various methods such as least-squares approximation may be selected as the fitting method. In this case, as shown in FIG. 13, it may be considered that there are corresponding points on the fitting curve with each point on the line containing each control point. The function has, for example, a position x in the main scanning direction and a position y in the sub-scanning direction as variables.

Next, the captured image is non-linearly (non-rigidly) deformed based on the amount of movement calculated for each control point or for each line on which control points are set (S44). For example, the thin plate spline method may be used to transform the captured image, or another technique may be used to transform the captured image.

Next, it is determined whether re-deformation is necessary (S45). Since this process may be the same as the process of S35 in the first distortion correction/synthesis process, detailed description thereof will be omitted. If it is determined that re-deformation is necessary, the number of control points is increased and the processing of S42 to S45 is repeated (looped). In this case, it is preferable to set more control points by increasing the number of lines on which control points are set. The greater the distortion, the easier it is for the captured image to be deformed based on a greater number of control points, and thus the distortion is more likely to be favorably corrected.

Next, weighted calculation of addition is performed (S46), and addition processing is performed with the calculated coefficient (S47). The weighting factor may be calculated based on the degree of similarity for each region between the template and the captured image after distortion correction, as in the first distortion correction/synthesis processing.

Further, in the second distortion correction/synthesis processing, a weight A weighting factor may be calculated. A line with a large error is considered to contain a large distortion with respect to the template, or to contain an artifact due to vignetting or the like. Therefore, for example, when the sum of the errors of each control point on a certain line is equal to or greater than a threshold, the line and its surroundings are given weighting coefficients that are not used for addition with other captured images. good too. For convenience, the weighting coefficient based on the error between the fitting curve and each control point is called the degree of abnormality.

Further, a weighting coefficient may be calculated based on the magnitude of change (difference) in the amount of movement between adjacent lines of a plurality of lines each having a control point set thereon. If the change in the amount of movement is large, it is considered that the movement of the eye has changed while the adjacent lines are being scanned, resulting in distortion. Therefore, for example, the larger the change (difference) in the movement amount, the smaller the coefficient may be given. A weighting factor based on the magnitude of change (difference) in the amount of movement between adjacent lines is referred to as reliability for convenience.

Coefficients based on similarity, unusualness, and confidence can be multiplied. That is, two or more of these coefficients can be applied simultaneously to each region of the captured image after distortion correction. By performing addition processing using two or more coefficients, it becomes easier to obtain a good addition image in which the influence of distortion is further suppressed.

<Distortion Correction in Multiple Channels Imaged Simultaneously>
When distortion correction is performed on each channel image of a plurality of channel images of the subject's eye that are simultaneously photographed, the above process does not necessarily have to be performed for each channel. For example, the above processing may be performed for any one channel, and the distortion correction amount and weighting coefficient calculated for one channel may be applied to images of other channels captured at the same time. good.

<Set the corresponding area or corresponding point avoiding the internal reflection image>
For example, it is conceivable that internal reflection images of the imaging optical systems 10 and 20 may be reflected in the imaging range. An internal reflection image is generated by reflection of illumination light by optical members included in the imaging optical systems 10 and 20 . The internal reflection image is, for example, an artifact due to reflection on an optical member included in the imaging optical system. The internal reflection image appears at a fixed position on the image regardless of eye movement. Therefore, the image processor 80 may arrange a plurality of blocks (corresponding regions) or control points (corresponding points) while avoiding internal reflected images in the first or second distortion correction/synthesis processing. As a result, the corresponding areas or corresponding points between the template and the captured image are set by excluding elements that remain stationary regardless of the movement of the eye. improves.

<Corresponding points and adjustment of corresponding points according to angle of view and resolution>
Further, for example, the parameters related to the blocks (corresponding regions) in the first distortion correction/synthesis processing and the parameters related to the control points (corresponding points) in the second distortion correction/synthesis processing are the angle of view and the number of pixels of the captured image. Or it may be adjusted by the image processor 80 depending on both. Parameters that can be adjusted for blocks (divided regions) may be, for example, at least one of the number, size, arrangement, and shape of blocks. Also, the adjustable parameter for the control points may be at least one of the number (density) and arrangement.

For example, the greater the number of pixels, the longer the imaging time, and the more likely the captured image is to be affected by distortion. On the other hand, blocks or control points may be increased as the number of pixels increases. That is, the number of corresponding points between the template and the captured image in local matching may be increased as the number of pixels of the captured image increases. This makes it easier to perform correction processing with an appropriate processing load for the distortion of the captured image.

Further, for example, in FIGS. 13A to 13C, as a result of the angle of view of the captured image being increased by the angle-of-view switching unit, the edge of the objective optical system (edge 201 of the lens in FIGS. 13A to 13C) appears as an internally reflected image. It is conceivable that the image may be reflected on the image. In this case, in the area inside the edge, the tissue of the eye to be examined is delineated and distortion due to eye movement can occur in this area. On the other hand, the edge and the area outside it cannot be distorted. Therefore, if the imaging range includes the edge of the optical system, the image processor 80 may arrange a plurality of blocks or control points only in the area inside the edge. In other words, the image processor 80 changes the plurality of blocks or control points according to the angle of view so that the plurality of blocks or control points are arranged to avoid internal reflection images that occur at a specific angle of view. good too.

<transformation examples, etc.>
Although the above has been described based on the embodiments, the contents of the embodiments can be changed as appropriate in carrying out the present disclosure.

<Matching with reduced image>
For example, at least one of the various matching and distortion correction/synthesis processes in the above embodiments may use a reduced image whose image size is reduced. In this case, a reduced image is generated by reducing the fundus image that is the target of image processing based on the template. Additionally, a reduced image of the template may be generated. By performing image processing with a reduced size, the processing load on the image processor 80 is reduced. The correspondence relationship between each pixel in the reduced image and each pixel in the original image is uniquely determined by the magnification at the time of reduction. The image processor 80 applies the parameter/deformation matrix obtained by matching at the reduced size to the original size fundus image at the above magnification. As a result, various types of matching, distortion correction, and synthesis processing can be executed with a lower processing load (in other words, at a higher speed).

<Application of various image processing in scenes other than capture>
In the above-described embodiment, various types of processing by the image processor 80 are executed when capturing a photographed image. However, it is not necessarily limited to this, and various image processing may be performed by the image processor 80 on the fundus image captured in advance. For example, the execution timing of all or part of <template acquisition processing>, <distortion detection processing and determination processing>, <first and second distortion correction/synthesis processing>, etc. is not at the time of shooting, but at the time of browsing. may be

When various types of image processing are performed on a fundus image that has been captured in advance, at least one of various templates and the fundus image synthesized with the template is processed in advance based on operation input from the examiner. It may be possible to select from among a plurality of photographed fundus images.

In this case, a selection screen may be displayed on the monitor 90 . At least two of the plurality of fundus images are preferably displayed side by side on the selection screen. Also, the displayed image may optionally be replaceable with another image. This allows the examiner to favorably compare a plurality of fundus images. Then, a well-captured image is likely to be selected as a target for image processing.

<Manual setting of corresponding points or corresponding areas>
In the above embodiment, a case has been described in which corresponding points or corresponding regions (parameters relating to corresponding points or corresponding regions) are automatically set as appropriate for the template and the captured image. However, it is not necessarily limited to this, and may be set manually based on an operation input by the examiner. The setting here includes not only zero-based setting, but also correction of automatically set corresponding points and corresponding areas (parameters related to corresponding points or corresponding areas) based on operation input.

In this case, at least one parameter of the number (density) and arrangement of the corresponding points may be settable based on the operation input. Also, with respect to the corresponding regions, at least one of parameters of the number, size, arrangement, and shape may be settable based on the operation input. When the corresponding points or corresponding regions are manually set, the image processor 80 executes various processes such as local matching, as in the case of automatic setting. The parameters may be arbitrarily set by the examiner, or may be selected from among a plurality of predetermined patterns in accordance with an operation input, or both may be used together. may

When manually setting the corresponding points or the corresponding areas, it may be possible to confirm the setting state of the corresponding points or the corresponding areas by graphical display. For example, a fundus image (for example, a template) superimposed with indices indicating corresponding points or corresponding regions may be displayed on the monitor 90 for confirmation.

<About application to OCT>
In the above embodiment, the ophthalmologic imaging apparatus is an apparatus that captures a front image (two-dimensional image) of the fundus using a scanning optical system, but is not necessarily limited to this. For example, the technique of the present disclosure can also be applied to an optical coherence tomography (hereinafter referred to as OCT) that captures a tomographic image of an eye to be examined. In the case of OCT, the imaging optical system may include an OCT optical system (not shown) that acquires OCT data of the fundus based on the spectral interference signal of the return light and the reference light. Alternatively, the imaging optical system in the ophthalmologic imaging apparatus may include both the front imaging optical system (for example, the imaging optical systems 10 and 20 in the above embodiments) and the OCT optical system.

<Regarding partial implementation of the above embodiment>
<Template Acquisition Processing> and <Photographing Processing> in the present embodiment can be independent, and they do not necessarily need to be performed integrally. For example, the template used in the <imaging process> does not necessarily have to be subjected to distortion detection or distortion correction using a comparison image. Alternatively, only the <template acquisition process> may be implemented as an ophthalmic photographing apparatus and an ophthalmic image processing program.

1 ophthalmic photographing apparatus 1b control unit 10 irradiation optical system 20 light receiving optical system 70 processor 80 image processor

Claims (5)

an imaging optical system including scanning means for scanning a tissue of an eye to be inspected with light from a light source, and a light receiving element for receiving light returned from the tissue;
a photographing control means for photographing a plurality of images of the subject's eye based on the signal from the light receiving element;
and an image processing means,
The image processing means is
Using one of the plurality of images of the eye to be examined as a template, corresponding points or corresponding regions between the image of the eye to be examined and the template are set at a plurality of positions between the image of the eye to be examined and the template. together with local matching for calculating the amount of movement for each corresponding point or each corresponding area;
executing a distortion correction process for correcting two-dimensional distortion of the image of the eye to be inspected with respect to the template based on the movement amount for each corresponding point or each corresponding region;
The image of the subject's eye after distortion correction is compared with the template, and the corresponding points or the corresponding regions are set more densely according to the comparison result, and the local matching and the distortion correction processing are performed. Redo, ophthalmic imaging equipment. The image processing means performs the local matching and the distortion correction process based on the degree of similarity between the image of the subject's eye and the template obtained for each corresponding point or each corresponding region in the local matching. 2. The ophthalmic photographing apparatus according to claim 1 , wherein it is determined whether or not to redo. an imaging optical system including scanning means for scanning a tissue of an eye to be inspected with light from a light source, and a light receiving element for receiving light returned from the tissue;
a photographing control means for photographing a plurality of images of the subject's eye based on the signal from the light receiving element;
and an image processing means,
The image processing means is
Using one of the plurality of images of the eye to be examined as a template, corresponding points or corresponding regions between the image of the eye to be examined and the template are set at a plurality of positions between the image of the eye to be examined and the template. together with local matching for calculating the amount of movement for each corresponding point or each corresponding area;
Distortion correction processing for correcting two-dimensional distortion of the image of the subject's eye with respect to the template based on the movement amount for each corresponding point or each corresponding region;
For each image of the eye to be inspected after distortion correction, a weighting coefficient is set for each region as the distortion remaining with respect to the template becomes larger, and based on the weighting coefficient for each region in each image, the distortion is an ophthalmologic imaging apparatus for executing a synthesizing process of obtaining a synthesized image by synthesizing a plurality of corrected images of the subject's eye for each region. by being executed by the processor of the computer,
an acquisition step of acquiring a plurality of images of the subject's eye captured by a scanning optical system;
Using one of the plurality of images of the eye to be examined as a template, corresponding points or corresponding regions between the image of the eye to be examined and the template are set at a plurality of positions between the image of the eye to be examined and the template. and local matching for calculating a movement amount for each corresponding point or each corresponding region;
causing the computer to perform a distortion correction process for correcting the distortion of the image of the eye to be inspected with respect to the template based on the movement amount for each corresponding point or each corresponding region ;
Further, the computer compares the image of the subject's eye after distortion correction with the template, and according to the comparison result, sets the corresponding points or the corresponding regions more densely, and performs the local matching, and , an ophthalmic image processing program for redoing the distortion correction processing . by being executed by the processor of the computer,
an acquisition step of acquiring a plurality of images of the subject's eye captured by a scanning optical system;
Using one of the plurality of images of the eye to be examined as a template, corresponding points or corresponding regions between the image of the eye to be examined and the template are set at a plurality of positions between the image of the eye to be examined and the template. and local matching for calculating a movement amount for each corresponding point or each corresponding region;
Distortion correction processing for correcting distortion of the image of the subject's eye with respect to the template based on the movement amount for each corresponding point or each corresponding region;
For each image of the eye to be inspected after distortion correction, a weighting coefficient is set for each region as the distortion remaining with respect to the template becomes larger, and based on the weighting coefficient for each region in each image, the distortion is A synthesizing process for obtaining a synthesized image by synthesizing the plurality of corrected images of the eye to be inspected for each region;
An ophthalmic image processing program that causes the computer to execute.