JP2016101235A - Image processing device for ophthalmology and image processing program for ophthalmology - Google Patents

Abstract

PROBLEM TO BE SOLVED: To excellently form a panoramic image.SOLUTION: An imaging device 1 obtains a panoramic image by joining a plurality of slices of cell images forming the fundus oculi of a subject eye E. The imaging device 1 acquires a pair of slices from the plurality of slices (S12), and acquires the coincidence for an overlapped portion while matching a pair for the overlapped portions of two still images included in the pair and formed in a case where the still images are arranged on a plane in a physical relation at prescribed sites (S13). The imaging device 1 generates the panoramic image by performing image joining processing for joining the slices being the pair to the plurality of pairs acquired by pair acquisition means in the order of priority being the order of priority for performing the image joining processing in the plurality of pairs and according to the coincidence of each pair (S16).SELECTED DRAWING: Figure 8

Description

  The present disclosure relates to an ophthalmic image processing apparatus and an ophthalmic image processing program.

  Conventionally, an apparatus for acquiring a still image of a local part of an eye to be examined is known. Further, there is known a method for forming a wide range image (so-called panoramic image) of an eye to be examined by aligning and joining still images of local parts obtained by such an apparatus (patented). Reference 1). The examiner can easily obtain information useful for diagnosis and examination, which is difficult to obtain from a local still image before joining, from a wide range of panoramic images.

  In Non-Patent Document 1, the examiner manually adjusts the initial position of the still image so that each still image has a positional relationship at an approximate acquisition position, and then a portion where the still images overlap is described. A method for forming a panoramic image to be matched in detail as a template is disclosed.

JP 2014-110825 A

Thevenaz P1, Unser M, “User-friendly semiautomated assembly of accurate image mosaics in microscopy” Microscopy Research and Technique 2007 Feb; VOL70, Issue 2, P135-146

  By the way, the image quality of a panoramic image is affected by the combination of still images to be joined and the order of joining when forming a panoramic image. Further, when the examiner manually adjusts the initial position of the still image, the burden on the examiner may be large.

  The present disclosure has been made in view of at least one of the problems of the prior art, and an object thereof is to provide an ophthalmic image processing apparatus and an ophthalmic image processing program capable of forming a panoramic image satisfactorily. To do.

  The ophthalmic image processing apparatus according to the first aspect of the present disclosure is an ophthalmic image processing apparatus that obtains a panoramic image by connecting a plurality of still images of cell images that form a predetermined portion of an eye to be examined. A pair acquisition means for acquiring a pair of the still images from the still images, and an overlapping portion of the two still images included in the pair, wherein each of the still images is a plane in a positional relationship at the predetermined portion. Matching means for acquiring the degree of coincidence regarding the overlapping part while matching the pair with respect to the overlapping part formed when arranged on top, and a pair for the plurality of pairs acquired by the pair acquisition means The image joining process for joining still images is a priority for performing the image joining process in the plurality of pairs, and each pair obtained by the matching means An image joining means for generating the panoramic image by performing in priority order according to the matching degree comprises.

  An ophthalmic image processing apparatus according to a second aspect of the present disclosure is an ophthalmic image processing apparatus that obtains a panoramic image by connecting a plurality of still images of cell images that form a predetermined part of an eye to be examined, and stores in advance Reading out a plurality of still images stored in the unit, and acquiring a pair of the still images from the plurality of still images; and the predetermined of the plurality of still images stored in the storage unit in advance Using the information related to the positional relationship at the part, the matching degree acquisition means for acquiring the matching degree related to the overlapping portion formed when the plurality of still images are arranged on the plane in the positional relation at the predetermined part; and An image joining process for joining two still images to be paired is performed in the order in which the image joining process is performed in the plurality of pairs, and in the order determined based on the matching degree. And an image joining means for generating the panoramic image by performing relative pair.

  An ophthalmic image processing program according to the third aspect of the present disclosure is an ophthalmic image processing program for generating a panoramic image by connecting a plurality of still images of cell images forming a predetermined part of an eye to be examined. A pair acquisition step of acquiring a pair of the still images from the plurality of still images by being executed by a processor of the ophthalmic image processing apparatus; and an overlapping portion of the two still images included in the pair. A matching step of matching the pair with respect to the overlapping portion formed when the still images are arranged on a plane in a positional relationship at the predetermined portion, and obtaining a matching degree regarding the overlapping portion; For the plurality of pairs acquired by the acquisition step, an image joining process for joining the still images to be paired is performed on the plurality of pairs. The image joining step of generating the panoramic image by performing the order of priority for performing the image joining process in accordance with the priority order according to the matching degree of each pair obtained by the matching means. Let the device run.

  An ophthalmic image processing program according to a fourth aspect of the present disclosure is an ophthalmic image processing program for generating a panoramic image by connecting a plurality of still images of cell images forming a predetermined part of an eye to be examined. A pair acquisition step of reading a plurality of the still images stored in the storage unit in advance by being executed by the processor of the ophthalmic image processing apparatus and acquiring the pair of the still images from the plurality of still images; Formed when the plurality of still images are arranged on a plane according to the positional relationship at the predetermined portion using information on the positional relationship at the predetermined portion of the plurality of still images stored in advance in the storage unit A collation degree acquisition step for obtaining a collation degree related to the overlapping portion to be performed, and an image joining process for joining the two still images to be paired, An image joining step of generating the panoramic image by performing the image joining process on a plurality of the pairs in an order in which the image joining process is performed based on the matching degree; To run.

  According to the ophthalmic image processing apparatus and the ophthalmic image processing program of the present disclosure, there is an effect that a panoramic image can be favorably formed.

It is the figure which showed the external view of the imaging device 1 of this embodiment. 2 is a schematic diagram illustrating an optical system of the photographing apparatus 1. FIG. It is the block diagram which showed the control system of the imaging device 1 of this embodiment. It is the figure which showed an example of the reference | standard image. 5 is a flowchart illustrating main processing executed by a control unit 800. It is the figure which showed the position made into a target when acquiring a slice on a 2nd fundus image as an example. FIG. 7 is a diagram showing, as an example, a plurality of slices obtained when slices are acquired with the respective rectangles shown in FIG. 6 as targets. It is the flowchart which showed the panorama image generation process. It is the figure which illustrated the positional relationship of the slice in initial arrangement. It is the figure which showed typically the procedure of the image processing for obtaining correlation map _{TA B} which removed noise from correlation map TB _{→ A} and correlation map TA _{→ B.} (A) is the figure which showed distribution of the correlation in 1 line which passes luminescent spot P in correlation map TB _{→ A} , (b) is 1 line which passes luminescent spot P in correlation map TA _{→ B.} (C) is a diagram showing the correlation distribution in one line passing through the bright spot P in the correlation map _TA⇔B . It is the figure which showed an example of the coincidence degree list. It is a schematic diagram for demonstrating the procedure which joins each slice. It is the figure which showed the panoramic image produced | generated from the slice shown in FIG. 7 as a result of a panoramic image generation process.

  In the following, exemplary embodiments will be described with reference to the drawings. The imaging device 1 is a fundus imaging device with wavefront aberration compensation (AO-SLO) that acquires a fundus image of the eye to be examined while correcting the wavefront aberration of the eye to be examined. In the present embodiment, the imaging apparatus 1 connects a plurality of still images of cell images that form the fundus of the eye to be examined (“predetermined part” in the present embodiment) to connect a panoramic image to a panoramic image (also referred to as a collage image). ) Is also used. Note that the cell image is an image in which cells at a predetermined site are photographed.

  First, a schematic configuration of the photographing apparatus 1 will be described with reference to FIG. The photographing apparatus 1 includes a base 510, a face support unit 600, and a photographing unit 500. The face support unit 600 is attached to the base 510. The imaging unit 500 houses an optical system described later, and is provided on the base 510. The face support unit 600 is provided with a chin rest 610. The chin rest 610 is moved in the left / right direction (X direction), the up / down direction (Y direction), and the front / rear direction (Z direction) with respect to the base of the face indicating unit 600 by operating a chin rest driving means (not shown).

  Next, the optical system of the photographing apparatus 1 will be described with reference to FIG. The imaging apparatus 1 of the present embodiment includes a fundus imaging optical system 100, a wavefront aberration detection optical system (hereinafter referred to as an aberration detection optical system) 110, aberration compensation units 20, 72, and a second imaging unit 200. The tracking unit 300 and the anterior ocular segment observation unit 700 are provided.

  The fundus imaging optical system 100 projects laser light (illumination light) on the eye E and receives reflected light from the fundus of the laser light to capture a fundus image of the eye E. The fundus of the eye E is photographed by the fundus imaging optical system 100 with high resolution (high resolution) and high magnification. As described below, the fundus imaging optical system 100 may have a configuration of a scanning laser ophthalmoscope using a confocal optical system, for example. The fundus imaging optical system 100 includes a first illumination optical system 100a and a first imaging optical system 100b. In the present embodiment, the aberration compensation units 20 and 72 are arranged in the fundus imaging optical system 100 in order to correct the aberration of the eye to be examined. The aberration compensation unit includes a diopter correction unit 20 for correcting low-order aberrations (diopter: for example, spherical power) of the eye to be examined and high-order aberration compensation for correcting high-order aberrations of the eye to be examined. Part (wavefront compensation device) 72.

  The first illumination optical system 100a illuminates the fundus two-dimensionally by irradiating the eye E with laser light and scanning the laser light on the fundus. The first illumination optical system 100a includes a light source 11, a lens 12, a polarization beam splitter (PBS) 14, a beam splitter (BS) 71, a concave mirror 16, in an optical path from the light source 11 (first light source) to the fundus. Concave mirror 17, flat mirror 18, aberration compensation unit 72 (wavefront compensation device 72), beam splitter (BS) 75, concave mirror 21, concave mirror 22, scanning unit 25, concave mirror 26, concave mirror 27, flat mirror 31, Lens 32, plane mirror 33, aberration compensation unit 20 (diopter correction unit 20), plane mirror 35, concave mirror 36, deflection unit 400, dichroic mirror 90, concave mirror 41, plane mirror 42, plane mirror 43, and concave surface A mirror 45.

  The light source 11 emits laser light. In the present embodiment, the laser light has a near-infrared wavelength that is difficult to be visually recognized by the eye to be examined. For example, in the present embodiment, the light source 11 is an SLD (Super Luminescent Diode) having a wavelength of 840 nm. The light source 11 may be any light source that emits spot light having a high convergence property, and may be, for example, a semiconductor laser.

  The laser light emitted from the light source 11 is converted into parallel light by the lens 12 and then enters the wavefront compensation device 72 via the PBS 14, BS 71, concave mirrors 16 and 17, and the flat mirror 18. In the present embodiment, the laser light passes through the PBS 14 and becomes a light beam having only an S-polarized component. The wavefront compensation device 72 corrects higher-order aberrations of the eye to be examined by controlling the wavefront of the incident light. The detailed configuration of the wavefront compensation device 72 will be described later. In the present embodiment, the laser beam is guided from the wavefront compensation device 72 to the BS 75, reflected by the concave mirror 21 and the concave mirror 22, and travels toward the scanning unit 25.

  In the present embodiment, the scanning unit 25 is used together with the deflecting unit 400 in order to scan laser light two-dimensionally on the fundus. The scanning unit 25 is a resonant mirror used for main scanning of laser light. The laser light is scanned in the X direction on the fundus by the scanning unit 25.

  The light that has passed through the scanning unit 25 enters the diopter correction unit 20 via the concave mirrors 26 and 27, the plane mirror 31, the lens 32, and the plane mirror 33.

  The diopter correction unit 20 is a unit for performing diopter correction. The diopter correction unit 20 includes a pair of lenses and plane mirrors in addition to the drive unit 20a. The optical path length is adjusted by moving the plane mirror and lens of the diopter correction unit 20 in a predetermined direction by the drive unit 20a. As a result, the diopter error of the eye E is corrected.

  The illumination light guided from the diopter correction unit 20 to the plane mirror 35 is reflected by the concave mirror 36 and travels toward the deflection unit 400.

  The deflection unit 400 scans the laser light emitted from the light source 11 in the vertical direction (Y direction) on the fundus. Furthermore, the deflecting unit 400 is also used to move the scanning range of the laser light on the fundus. For example, in the present embodiment, the deflecting unit 400 may include two optical scanners (specifically, an X galvanometer mirror and a Y galvanometer mirror) having different directions of deflecting laser light.

  The light that has passed through the deflecting unit 400 is guided into the pupil of the eye E through the dichroic mirror 90, the concave mirror 41, the plane mirrors 42 and 43, and the concave mirror 45. The laser light is collected on the fundus of the eye E. As described above, the laser beam is two-dimensionally scanned on the fundus by the operations of the scanning unit 25 and the deflecting unit 400.

  Further, the dichroic mirror 90 has a characteristic of transmitting light beams from the second photographing unit 200 and tracking unit 300 described later and reflecting light beams from the light source 11 and the light source 76 described later. Note that the emission ends of the light sources 11 and 76 and the fundus of the eye E are conjugate. In this way, the first illumination optical system 100a is formed.

  Next, the first photographing optical system 100b will be described. The first imaging optical system 100 b receives the reflected light of the laser light irradiated on the fundus by the light receiving element 56. The imaging apparatus 1 acquires a first fundus image (in this embodiment, an AO-SLO image) based on a signal from the light receiving element 56. The first imaging optical system 100b shares the optical path from the eye E to the BS 71 with the first illumination optical system 100a. The first imaging optical system 100b includes elements arranged in the reflection side optical path of the BS 71, that is, a plane mirror 51, a PBS 52, a lens 53, a pinhole plate 54, a lens 55, and a light receiving element 56. . In this embodiment, the light receiving element 56 is an APD (avalanche photodiode). Further, the pinhole plate 54 is placed at a position conjugate with the fundus.

  The fundus reflection light of the laser light from the light source 11 traces the first illumination optical system 100 a in the reverse direction, is reflected by the BS 71 and the flat mirror 51, and only the S-polarized light is transmitted by the PBS 52. This transmitted light is focused on the pinhole of the pinhole plate 54 via the lens 53. The reflected light focused at the pinhole is received by the light receiving element 56 through the lens 55. A part of the illumination light is reflected on the cornea, but most of the illumination light is removed by the pinhole plate 54. Therefore, the light receiving element 56 can receive the reflected light from the fundus while suppressing the influence of corneal reflection.

  A light reception signal of the light receiving element 56 is processed by an image processing unit (for example, the control unit 800, the image processing unit 803, etc., see FIG. 3), and thereby a first fundus image is acquired. In this embodiment, a fundus image of one frame is formed by main scanning of the scanning unit 25 and sub-scanning of a Y-scan galvanometer mirror provided in the deflection unit 400. Note that the deflection angle (swing angle) of the mirror in the scanning unit 25 and the deflection unit 400 is determined so that the angle of view of the fundus image (fundus image) acquired by the first imaging unit 100 becomes a predetermined angle. Here, in order to observe and photograph a predetermined range of the fundus at a high magnification (here, observation at a cell level or the like), the angle of view is set to about 1 to 5 degrees. In this embodiment, the angle is 1.5 degrees. Although depending on the diopter of the eye to be examined, the imaging range of the first fundus image is about 500 μm square.

  Further, when the reflection angle of the X-scanning galvanometer mirror and the Y-scanning galvanometer mirror provided in the deflection unit 400 is moved larger than the imaging field angle of the first fundus image, the first fundus image is captured on the fundus. The position (that is, the scanning range of the laser beam) is changed.

  The second photographing unit 200 is a unit for obtaining a fundus image (second fundus image) having a wider angle of view than the angle of view of the first photographing unit 100. The second fundus image is used, for example, as an image for position designation and position confirmation for obtaining the first fundus image. The second imaging unit 200 of the present embodiment preferably has a configuration capable of acquiring and observing a fundus image of the eye E to be examined in real time with a wide angle of view (for example, about 20 to 60 degrees). For example, as the second imaging unit 200, an observation / imaging optical system of an existing fundus camera and an optical system and a control system of a scanning laser ophthalmoscope (SLO) may be used.

  The tracking unit 300 detects a temporal change in positional deviation due to fixation eye movement of the eye E and obtains movement position information. In the tracking unit 300, the light reception result obtained at the start of tracking is sent as reference information to the control unit 800 (see FIG. 3), and thereafter, the light reception result (light reception information) obtained for each scan is sequentially transmitted to the control unit. To 800. The control unit 800 compares the received light information obtained thereafter with the reference information, and obtains the movement position information by calculation so that the same received light information as the reference information is obtained. The control unit 800 drives the deflection unit 400 based on the obtained movement position information. By performing such tracking, the deflection unit 400 is driven so that even if the subject eye E moves slightly, the movement of the deflection unit 400 is performed, and thus the movement of the fundus image displayed on the monitor 850 is suppressed. It becomes. The dichroic mirror 91 has a characteristic of transmitting the light beam from the second photographing unit 200 and reflecting the light beam from the tracking unit 300.

  The anterior segment observation unit 700 is a unit that illuminates the anterior segment of the eye E with visible light and captures a front image of the anterior segment. An image captured by the anterior segment observation unit 700 is output to the monitor 850. The anterior ocular segment image acquired by the anterior ocular segment observation unit 700 is used for alignment between the imaging unit 500 and the eye E to be examined. The dichroic mirror 95 has a characteristic of transmitting the light flux from the second imaging unit 200 and the tracking unit 300 and reflecting the light flux from the anterior ocular segment observation unit 700.

  Next, the aberration detection optical system 110 will be described. The aberration detection optical system 110 includes a wavefront sensor 73. The aberration detection optical system 110 projects measurement light onto the fundus of the eye E, and receives (detects) the fundus reflection light of the measurement light as an index pattern image with the wavefront sensor 73. The aberration detection optical system 110 has a part of optical elements on the optical path of the first illumination optical system 100a and the first photographing optical system 100b (in the present embodiment, on the common optical path), and the optical paths of the optical systems 100a and 100b. Some are shared. That is, the aberration detection optical system 110 of the present embodiment shares the BS 71 to the concave mirror 45 arranged on the optical path of the optical systems 100a and 100b with the optical systems 100a and 100b. Further, the aberration detection optical system 110 includes a light source 76, a lens 77, PBS 78, BS 75, BS 71, a dichroic mirror 86, a PBS 85, a lens 84, a plane mirror 83, and a lens 82.

  The light source 76 is used for detecting the aberration of the eye E. In the present embodiment, the light source 76 emits light having a wavelength different from that of the light source 11. As an example, in this embodiment, a laser diode that emits laser light having a wavelength of 780 nm is used as the light source 76 as measurement light. The measurement light emitted from the light source 76 is converted into a parallel light beam by the lens 77 and then incident on the PBS 78.

  The PBS 78 is an example of first polarization means provided in the wavefront compensation unit. The PBS 78 polarizes the light emitted from the light source 76 in a predetermined direction. More specifically, the PBS 78 is polarized in a direction (ie, P-polarized light) orthogonal to the polarization direction (ie, S-polarized light) of the PBS 14. The light that has passed through the PBS 78 is reflected by the BS 75 and guided to the optical path of the first illumination optical system 100a. As a result, the measurement light is condensed on the fundus of the eye E through the optical path of the first illumination optical system 100a.

  The measurement light is reflected at the condensing position of the fundus (for example, the retina surface). The fundus reflection light of the measurement light follows the optical path of the first illumination optical system 100a (that is, the optical path of the first photographing optical system 100b) in the opposite direction to that during projection. On the way, the measurement light is reflected by the wavefront compensation device 72. Thereafter, the measurement light is reflected by the BS 71, thereby deviating from the optical path of the first illumination optical system 100a. Thereafter, the measurement light is reflected by the dichroic mirror 86 and guided to the wavefront sensor 73 through the PBS 85, the lens 84, the plane mirror 83, and the lens 82.

  The PBS 85 is a second polarization unit provided in the wavefront compensation unit. The PBS 85 is used to guide light to the wavefront sensor 73 by transmitting light polarized in one direction (here, S-polarized light) out of the light irradiated to the eye E from the light source 76. The The PBS 85 blocks a component polarized in a direction orthogonal to the transmitted component (P-polarized light). The dichroic mirror 86 has a characteristic of transmitting light (840 nm) having the wavelength of the light source 11 and reflecting light (780 nm) having the wavelength of the light source 76 for detecting aberration. Therefore, the wavefront sensor 73 detects light having an S-polarized component from the fundus reflection light of the measurement light. In this way, light reflected by the cornea and the optical element is suppressed from being detected by the wavefront sensor 73.

  The wavefront sensor 73 receives fundus reflection light of measurement light for aberration measurement in order to detect wavefront aberration of the eye E. As the wavefront sensor 73, an element that can detect wavefront aberration including low-order aberration and high-order aberration (more specifically, a Hartmann Shack detector, a wavefront curvature sensor that detects a change in light intensity, and the like) is used. May be. In the present embodiment, the wavefront sensor 73 includes, for example, a microlens array composed of a number of microlenses, a two-dimensional imaging element 73a (or a two-dimensional light receiving element) for receiving a light beam that has passed through the microlens array, Have The microlens array of the wavefront sensor 73 is disposed at a position substantially conjugate with the pupil of the eye E to be examined. Further, the imaging surface (light receiving surface) of the two-dimensional imaging element 73a is disposed at a position substantially conjugate with the fundus of the eye E to be examined.

  An index pattern image 61 (Hartmann image in this embodiment) is formed on the imaging surface of the two-dimensional imaging element 73a by the light beam that has passed through the microlens array (not shown). Therefore, the fundus reflection light passes through the microlens array and is received by the two-dimensional imaging element 73a, thereby being imaged as a Hartmann image (dot pattern image). In the present embodiment, the aberration information of the eye to be examined is acquired from the Hartmann image, and the wavefront compensation device 72 is controlled based on the aberration information. Details of the Hartmann image will be described later.

The wavefront compensation device 72 is disposed in the optical path of the fundus imaging optical system 100, and compensates the wavefront aberration of the eye E by controlling the wavefront of the incident light. In the present embodiment, a liquid crystal spatial light modulator may be used for the wavefront compensation device 72. As an example, a description will be given below assuming that a reflective LCOS (Liquid Crystal On Silicon) or the like is used. In this case, the wavefront compensation device 72 has predetermined laser light (S-polarized light) from the light source 11, fundus reflected light (S-polarized light) of the laser light, reflected light (S-polarized component) of wavefront aberration detection light, and the like. Are arranged in a direction capable of correcting the aberration with respect to the linearly polarized light (S-polarized light). As a result, the wavefront compensation device 72 can modulate the S-polarized component of the incident light. In the present embodiment, the reflection surface of the wavefront compensation device 72 is arranged at a position that is substantially conjugate with the pupil of the eye to be examined.

  In the wavefront compensation device 72 of the present embodiment, the alignment direction of the liquid crystal molecules in the liquid crystal layer is substantially parallel to the polarization plane of incident reflected light. The predetermined plane on which the liquid crystal molecules rotate according to the change in the voltage applied to the liquid crystal layer includes the incident optical axis and the reflected optical axis of the fundus reflection light with respect to the wavefront compensation device 72, and the mirror layer of the wavefront compensation device 72. And a plane that includes the normal line.

  In this embodiment, the wavefront compensation device 72 is a liquid crystal modulation element, and in particular, a reflective LCOS or the like is used, but the present invention is not limited to this. Other reflective wavefront compensation devices may be used. For example, a deformable mirror that is a form of MEMS (Micro Electro Mechanical Systems) may be used. Further, a transmissive wavefront compensation device may be used instead of the reflective wavefront compensation device. In the transmission type device, the wavefront aberration is compensated by transmitting the reflected light from the fundus.

  In the above description, a light source that emits illumination light having a wavelength different from that of the first light source is used as the aberration detection light source. However, the first light source may also serve as the aberration detection light source.

  In the present embodiment described above, the wavefront sensor and the wavefront compensation device are the pupil conjugate of the eye to be examined. However, the position may be a position that is substantially conjugate with a predetermined part of the anterior eye portion of the eye to be examined. There may be.

  Next, with reference to FIG. 3, a control system of the photographing apparatus 1 in the present embodiment will be described. The photographing apparatus 1 has a control unit 800. The control unit 800 is a processor (for example, a CPU) that controls the entire apparatus of the photographing apparatus 1. In the present embodiment, a storage unit 801, an operation input unit 802, an image processing unit 803, and a monitor 850 are electrically connected to the control unit 800. The control unit 800 includes a light source 1, a driving unit 10a, a scanning unit 15, a light receiving element 56, a wavefront compensation device 72, a wavefront sensor 73, a light source 76, a second imaging unit 200, a tracking unit 300, a deflection unit 400, Are electrically connected.

  The storage unit 801 stores various control programs and fixed data. Further, the storage unit 801 may store images, temporary data, and the like acquired by the photographing apparatus 1.

  The control unit 800 controls each member described above such as the first imaging unit 100 based on the operation signal output from the operation unit 802. The operation input unit 802 includes an operation member operated by an examiner. A keyboard and a mouse may be used as the operation member.

  Based on the signals received by the light receiving element 56 and the second imaging unit 200, the image processing unit 803 forms the fundus images to be examined having different angles of view (that is, the first fundus image and the second fundus image). Then, the formed first fundus image and second fundus image are displayed on the monitor 850 (see FIG. 4).

  The monitor 850 may be a display monitor mounted on the photographing apparatus 1 or a general-purpose display monitor that is separate from the photographing apparatus 1. Moreover, the structure in which these were used together may be sufficient. The monitor 850 can display fundus images (first fundus image and second fundus image) captured by the imaging apparatus 1 as a moving image and a still image, respectively.

  The control of the wavefront compensation device 72 by the controller 800 is executed based on the wavefront aberration detected by the wavefront sensor 73. In the present embodiment, feedback control of the wavefront compensation device 72 based on the detection signal from the wavefront sensor 73 is performed. By controlling the wavefront compensation device 72, the high-order aberrations of the laser light emitted from the light source 1 and the fundus reflection light of the laser light are removed together with the S-polarized component of the reflected light of the light source 76. In this way, the aberrations of the laser light emitted from the light source 1 and the fundus reflection light of the laser light are removed. As a result, the imaging apparatus 1 acquires a high-resolution first fundus image (that is, a wavefront compensated image) from which high-order aberrations of the eye E have been removed (wavefront compensated). At this time, low-order aberrations may be corrected by the diopter correction unit 10.

  The operation of the present apparatus having the above configuration will be described with reference to the flowchart shown in FIG. The imaging apparatus 1 operates according to a control program stored in the storage unit 801 after the power is turned on. In the following operation, the imaging device 1 acquires still images of the first fundus image (AO-SLO image) at a plurality of acquisition positions. In addition, the imaging device 1 performs image processing for joining still images of the first fundus image acquired at different positions on the fundus to form a panoramic image of the first fundus image. As a result, a still image for confirming the fundus tissue in a wide range and at a high magnification is obtained.

  First, the imaging unit 500 is aligned with the eye E (S1). For example, the control unit 800 acquires a real-time anterior segment image using the anterior segment observation unit 700 and causes the monitor 850 to display the anterior segment image. While observing the anterior ocular segment image on the monitor 850, the examiner manually or automatically adjusts the position of the chin rest 610 to perform rough alignment. In this case, the examiner instructs the subject to fixate a fixation target (not shown).

  In the present embodiment, after the alignment is completed, when a start switch provided as one of the operation input units 802 is operated by the examiner, the fundus image using the first imaging unit 100 and the second imaging unit 200 is displayed. Acquisition starts (S2). In this case, the diopter correction of each of the photographing units 100 and 200 may be performed by the control unit 800. For example, diopter correction in the first photographing unit 100 is performed using the diopter correction unit 10.

  Next, the control unit 800 acquires (captures) still images (hereinafter referred to as slices) of the first fundus image at a plurality of acquisition positions on the fundus and stores them in the storage unit 801 (S3). In the present embodiment, a slice used for forming a panoramic image is acquired in S3. In addition, the control unit 800 stores the acquired position information related to the positional relationship of each slice on the fundus in the storage unit 801 in association with each slice (S4). Although details will be described later, the acquisition position information is used to specify the positional relationship between a plurality of slices forming a panoramic image.

  In the process of S3, for example, acquisition positions to be targets of a plurality of slices may be determined in advance by target position information. In this case, the control unit 800 determines the driving range (that is, the scanning range) of the optical scanner (scanning unit 25 and deflection unit 400) based on the target position information. Then, the image processing unit 803 forms a still image based on the optical scanning in the determined driving range, whereby a slice is acquired by the imaging device 1 (S3). Here, one slice may be formed based on optical scanning for one frame by an optical scanner, or may be an averaged image composed of images of a plurality of frames. The target position information may be, for example, position information on the second fundus image, or information on the scanning angle of the optical scanner. The target position information may be stored in advance in the storage unit 801, for example.

  In the present embodiment, as a result of the process of S3, a plurality of slices are acquired at acquisition positions where each slice at least partially overlaps another slice. Here, as an example, as illustrated in FIG. 6, a plurality of slices are acquired while the target acquisition position is shifted in the horizontal direction. However, the arrangement of the target acquisition positions is not limited to the example shown in FIG. In FIG. 6, a plurality of rectangles formed by alternate long and short dash lines are shown. Each rectangle indicates a target acquisition range of each slice. In FIG. 6, 13 rectangles are arranged (that is, 13 target acquisition positions are set), and each rectangle has at least one of the left and right edges overlapped with another rectangle. Yes. FIG. 7 exemplarily shows slices acquired with the position indicated by each rectangle as a target. In FIG. 7, for convenience, numbers “0” to “12” are associated with 13 slices one by one (the number on the upper left of the image is the corresponding number). The image of “0” is an image acquired with the leftmost rectangular position in FIG. 6 as a target, and the larger the number, the more the target position is on the right side.

  The acquired position information acquired by the process of S4 is information related to the positional relationship of each slice on the fundus. In this embodiment, the rough acquisition position of the slice in the fundus is indicated by the acquisition position information. As shown in this embodiment, the acquisition position information may be information different from the target position information, and may indicate the actual acquisition position on the fundus of the slice acquired by the process of S3, for example. . Since the eyes are constantly moving due to eye movement (for example, fine fixation), the actual acquisition position of the slice may deviate from the acquisition position predetermined as a target. In the present embodiment, acquisition position information indicating the acquisition position of the slice in the fundus is stored in the storage unit 801 in association with the slice. Thereby, the positional relationship between a plurality of slices forming a panoramic image is specified with high accuracy. The actual acquisition position can be specified using, for example, the second fundus image acquired simultaneously with the slice. Therefore, the acquired position information may be acquired as position information on the second fundus image (for example, deviation information from a tissue or a feature point serving as a reference in the second fundus image).

  Next, the control unit 800 generates a panoramic image from a plurality of slices acquired in advance (S5). Here, the processing content of the panoramic image generation processing (S5) in the present embodiment will be described in detail with reference to the flowchart of FIG.

  First, in the panoramic image generation process (S5), a plurality of slices acquired in advance in the process of S3 (that is, stored in advance in the storage unit 801 in this embodiment) are initially arranged (S11). . In the present embodiment, the initial arrangement of slices is automatically performed by the control unit 800. In this case, each slice is arranged on a plane (for example, on a two-dimensional coordinate system for performing image processing) in a positional relationship on the fundus based on the acquired position information. In the present embodiment, the acquisition position information is information that roughly indicates the actual acquisition position of the slice. Therefore, each slice is arranged so as to be shifted from the arrangement at the target acquisition position.

  Next, the control unit 800 acquires a pair of slices from the plurality of initially arranged slices (S12). In the present embodiment, when the images are arranged on a plane in a positional relationship in the initial arrangement, a pair is formed in which an overlapping portion where at least a part of the image overlaps is formed.

For example, the control unit 800 determines whether or not overlapping portions are formed for all combinations of selecting two slices from a plurality of slices that are initially arranged. Then, the control unit 800 may acquire a pair from a plurality of slices based on the determination result. In this case, for example, when N slices are initially arranged, a pair may be acquired from a combination of _{two N} C slices. For example, the acquisition position information of two slices may be used for acquiring a pair (that is, determining whether to form a pair).

  Next, for each pair acquired in S12, the control unit 800 performs a matching process, which is an image process for matching a paired slice with respect to an overlapping portion (S13). In the matching process, the correlation of each slice regarding the overlapped portion is obtained as the degree of coincidence with the matching of the still images. As a result of this image processing, in this embodiment, the degree of coincidence of overlapping portions between slices (also referred to as similarity degree) when arranged in the positional relationship in the initial arrangement may be a coincidence ratio or a similarity ratio. (For example, a numerical value) indicating the degree of coincidence. In the present embodiment, as described below, the degree of coincidence is obtained based on the correlation between two slices forming a pair (more specifically, mainly the correlation between the one area and the corresponding area described above). . Various processes can be adopted for the image processing for obtaining the degree of coincidence. In the present embodiment, the matching degree and the positional deviation amount in a pair are acquired together by image processing for matching two slices that form a pair.

  Here, with reference to FIG. 9 to FIG. 11, one specific example of the matching degree acquisition method will be shown.

  First, a region Ω (rectangular region Ω in the example of FIG. 9) of the paired slices A and B is cut out from one slice. Then, the cut out region Ω is matched (positioned) with respect to the other slice as an image matching template. At that time, phase images of the template (region Ω) and a reference image that is a slice with which the template is matched are generated. Then, for example, a correlation map between the phase image of the template image and the phase image of the reference image is obtained. The correlation map is a map showing the correlation for each positional relationship in the two phase images. In the correlation map, a correlation peak is obtained at a position where the two phase images overlap well. For example, the correlation map can be formed by obtaining the correlation between two phase images for each shifted position while shifting the two phase images.

Here, in the image matching of the two slices A and B,
(1) Matching using the region Ω cut out from the slice A as a template and the slice B as a reference image.
(2) Matching using the region Ω cut out from the slice B as a template and the slice A as a reference image.
There are two possible ways. In this specific example, both matchings (1) and (2) are performed on a pair of slices. Hereinafter, the correlation map obtained by the above matching (1) is referred to as TB _{→ A,} and the correlation map obtained by the matching (2) is referred to as TA _{→ B} (see FIG. 10). In FIG. 10, P shown above each of the correlation maps T _{B → A} and T _{A → B} indicates the peak position of the correlation. For example, in a map image that shows the high correlation as the high luminance, the peak is shown as a bright spot. The peak P in the correlation map T _{B → A and} the peak P in the correlation map T _{A → B} are shown at positions that are mirror-symmetrical. However, since the correlation map T _{B → A} and the correlation map T _{A → B} are the results of collating different templates, even if one of the maps is inverted, they become different maps (FIG. 11 (a), (See (b)). 11A and 11B show the correlation distribution on one line including the peak P in the correlation maps T _{B → A} and T _{A → B.} Also, in the graphs of FIGS. 11A and 11B, the distribution actually obtained by performing the matching with the slice “5” shown in FIG. 7 as the slice A and the slice “6” as the slice B is shown. Show. When such correlation maps TB _{→ A} and TA _{→ B} are compared, as shown in FIG. 11A, in the correlation map TB _{→ A} , the peak P is buried in the noise appearing at the center of the distribution. It is difficult to detect the peak P well. On the other hand, as shown in FIG. 11B, in the correlation map TA _{→ B} , since the peak P is not buried in noise, the position of the peak P is easily detected accurately. However, not only the correlation map T _{B → A} but also the correlation map T _{A → B} is affected by noise. Therefore, in order to obtain a coincidence with high accuracy, it is necessary to suppress the influence of noise.

Therefore, in this specific example, an inverted map is obtained which is a map obtained by inverting one of the correlation maps T _{B → A} and T _{A → B} so that the peak P overlaps the other map. In the example of FIG. 10, the inversion map T ^T _{A → B} is generated by inverting the correlation map T _{A → B.} Then, the product for each element of the correlation map T _{B → A} that is not inverted and the inverted map T ^T _{A → B} is taken. A map (integration result map) obtained as a result of integration is denoted as _TAＴB . In the example of FIG. 10, the integration result for each element of the correlation map T _{B → A} and the inversion map T ^T _{A → B} is shown as an integration result map T A _⇔B . The integration result map _TA⇔B can be generally obtained by the following equation.

As shown in FIG. 11C, in the integration result map T _A⇔B , the noise level is suppressed and the peak P is emphasized compared to the correlation maps T _{B → A} and T _{A → B.} Therefore, the degree of coincidence between the slices A and B can be favorably obtained from the integration result map _TA⇔B .

It can be considered that the slices A and B are better matched as the peak P of the integration result map _TA⇔B is sharper. Therefore, in order to quantify the sharpness of the peak P, the peak of the integration result map _TA⇔B is fitted with, for example, a Gaussian function. Then, the width Q (for example, half width, δ, etc.) of the fitted Gaussian function is obtained. In this specific example, when the value of the peak width Q is small, the correlation (matching degree) between slices is high. The method for evaluating the sharpness of the peak P is not limited to the above method. For example, the correlation map may evaluate the sharpness of the peak P by comparing the degree of similarity (the degree of coincidence) with the model using the sinc function.

  Further, in the present embodiment, a deviation amount between the two slices A and B that are matched based on the correlation map (and each map of the inverted map and the integration result map) is obtained. That is, the position of the peak P in the above map indicates the amount of deviation between the two matched slices A and B. In the present embodiment, by detecting the position of the peak P, a deviation amount between the two slices A and B is obtained. That is, the control unit 800 obtains the amount of deviation together with the degree of coincidence between the two slices A and B.

  Returning to the flowchart of FIG. In the present embodiment, the control unit 800 creates a list (hereinafter referred to as a coincidence degree list) indicating the degree of coincidence and the amount of deviation in each pair (S14). Then, the matching degree list is sorted in order of matching degree (S15). FIG. 12 exemplifies a matching degree list in a sorted state. In the example of FIG. 12, the information of each pair is sorted so that the matching level of the pair becomes higher in the upper stage. In the matching degree list, a pair (slice forming a pair) and a matching degree of two slices in the pair are associated with each pair. Also, as shown in FIG. 12, the amount of deviation between two slices may be associated with a pair together with the degree of coincidence.

  Next, the control unit 800 sequentially joins each slice in pairs, and as a result, creates a panoramic image (S16). More specifically, the control unit 800 performs image processing (image joining processing) for joining still images that form pairs with each pair in the matching degree list. The priority order in which this image joining process is performed in each pair is determined according to the matching degree of each pair obtained as a result of matching. The image joining process in the present embodiment is an image process for joining two slices to be paired in an arrangement according to the positional deviation amount of the pair. And the control part 800 produces | generates a panoramic image as a result of the image joining process with respect to several pairs.

  In the present embodiment, in the matching list of FIG. 12, the control unit 800 selects pairs in descending order of matching (that is, in order from the higher correlation pair), and performs image joining processing on the selected pairs. As a result, a partial panoramic image (hereinafter referred to as a partial panoramic image) is formed.

  In the example of FIG. 12, partial panoramas are performed in order from the top pair in the matching list (that is, in the order of “4” and “5” pairs → “10” and “11” pairs →...). Form an image. In joining, blending processing such as addition averaging may be performed on overlapping portions of slices so that the joints between slices are not noticeable. In addition, when processing such as addition averaging processing is performed on an overlapping portion between slices, the overlapping portion may become dark. In such a case, an overlapping portion with any other slice may be removed from any one of the slices.

  In the present embodiment, when at least one of the two slices forming a pair already forms a partial panoramic image together with another slice, the other slice is joined to the partial panoramic image. For example, in the example of FIG. 12, after the partial panorama image C1 is formed by the pair of “7” and “8” in the fourth row from the top of the list, “8” in the fifth row of the list. No. and “9” pair are joined. In this case, as shown in FIG. 13A, the slice “9” is joined to the portion “8” in the partial panorama image C1 composed of the pair “7” and “8”. . As a result, the “7”, “8”, and “9” slices are joined together as a partial panorama image C2.

  In addition, when a pair of “9” and “11” in the sixth row from the top of the list is joined, a partial panorama image C3 (matching) consisting of “10” and “11” slices in advance And a partial panorama image C2 (refer to the fourth and fifth levels of the matching degree list) composed of slices “7” to “9”. In such a case, the joints of “9” and “11” are joints of the partial panorama images (see FIG. 13B).

  By the way, in the coincidence list composed of three or more pairs, there is a high match for a certain pair (hereinafter referred to as the first pair) and the first coincidence that is the coincidence of the first pair. A pair group composed of two or more pairs each indicating a degree and having a different combination of slices from the first pair may be included as a pair used to form a panoramic image. For example, in FIG. 12, when a pair at the nth stage from the top is the first pair, a plurality of pairs at the n-1th stage from the first stage can be considered as a pair group.

  In this case, since the image joining process is performed on two or more pairs included in the pair group, the slices included in the first pair are not subjected to the image joining process on the first pair. , (Via one or more other slices). In such a case, it is preferable that the control unit 800 of the present embodiment does not perform the image joining process for the first pair. However, on the other hand, if the first pair is not joined together even if the image joining process is performed on each pair of the higher priority pair group, the control unit 800 does not apply to the first pair. Perform image joining processing. Here, a specific example when the bonding process is not performed on the first pair is shown. For example, when the pair of numbers “9” and “10” in the ninth row of the list of FIG. 12 is the first pair, the image joining process is performed on a pair group consisting of pairs with higher matching degrees. Are joined together. That is, the image joining process is performed on the “10” and “11” pairs in the second row of the list and the “9” and “11” pairs in the sixth row of the list, respectively. The first pair “9” and “10” is joined together via the “11” slice. In this case, the control unit 800 does not perform the image joining process for directly joining the “9” slice and the “10” slice that are the first pair.

A bonding pattern that can form a better bonding portion when a plurality of bonding patterns (combination of pairs for which image bonding processing is performed) for bonding two slices integrally are assumed by such processing. Is uniquely selected by the control unit 800. As a result, it is easy to obtain good image quality at the joined portion of the slice. Further, if two slices already joined together are rejoined by image joining processing, the image may be unnecessarily distorted at the joined portion. However, in this embodiment, since such rejoining is not performed, image distortion can be reduced. Moreover, in the above-mentioned algorithm, since the number of joint locations is suppressed, as a result, the time for forming a panoramic image is suppressed.
In addition, although this embodiment demonstrates the case where said rejoining is not performed, it is not necessarily restricted to this. The joining of a pair of matching degrees indicating a better matching state should be given priority over the joining of a pair of matching degrees indicating a worse matching state, as long as it is within an acceptable accuracy range in a panoramic image. The rejoining may be allowed.

  In the present embodiment, as a result of the panoramic image generation process as described above, the joining in a pair having a high degree of matching (correlation) of slices is given priority over the joining in a pair having a low matching degree. As a result, a panoramic image is obtained in which the images are joined well (see FIG. 14). In particular, in the present embodiment, a panoramic image in which each slice is connected to a slice having the highest matching degree in each slice is formed by joining the slices in order from the pair having the higher matching degree. Therefore, it is easy to obtain a panoramic image with good image quality.

  Here, the degree of coincidence is not only a difference in positional deviation and image distortion in the XY directions, but also a shift in defocus amount in slices (a shift from an appropriate defocus amount for each slice, a defocus amount between slices). Are included). Therefore, in this embodiment, a good panoramic image in which the influence of these elements is suppressed can be formed.

  In the panoramic image generation method according to the present embodiment, the joining order is uniquely determined without selecting and joining the pairs once joined together. Therefore, the processing described above can achieve both the suppression of the time required for generating the panoramic image and the generation of a good panoramic image.

  In this embodiment, the initial placement of each slice image is performed based on acquisition position information indicating the actual acquisition position of each slice. Therefore, since the slice images are arranged in advance with high accuracy before the matching of each pair using the portion where the slices overlap with each other as a template, the matching accuracy can be improved. As a result, in the present embodiment, the degree of coincidence in each pair is derived more accurately, and the order in which the pairs are combined is determined more appropriately. As a result, it is easy to obtain a panoramic image with good image quality.

In the present embodiment, the control unit 800 automatically executes the initial arrangement of each slice image based on the acquired position information. Therefore, the burden on the examiner required for creating the panoramic image is suppressed.
<Transformation example>
As described above, the description has been given based on the embodiment, but the present disclosure can be variously modified.

  For example, in the above-described embodiment, a case has been described in which slices used for generating a panoramic image are automatically set. However, the present invention is not necessarily limited thereto. For example, the structure which an examiner selects manually may be sufficient. In addition, when a plurality of slices having the same acquisition position on the fundus are acquired, a slice to be used for generating a panoramic image is manually or automatically selected from the plurality of slices. A configuration to be selected may be provided in the photographing apparatus 1.

  For example, in the above embodiment, the case where the control unit 800 initially arranges a plurality of slices has been described. In this case, manual adjustment means may be provided for the examiner to adjust the position of each slice while visually checking the positional relationship of each slice. For example, after the initial arrangement is completed, the control unit 800 displays a plurality of slices side by side in the initial arrangement positional relationship on the monitor 850, and a signal for adjusting at least one positional relationship of the initially arranged slices. You may make it accept from the operation input part 802. FIG. Then, the control unit 800 may adjust the positional relationship of the initially arranged slices according to the signal from the operation input unit 802. For example, in an eye with poor fixation (for example, an eye lacking the central visual field), the accurate acquisition position of the slice cannot be specified, and it may be difficult to accurately perform the initial arrangement based on the acquisition position information. In such a case, it is possible to manually adjust the position of the slice in the initial arrangement and improve the arrangement accuracy, so that a good panoramic image can be easily obtained.

  Further, for example, in order to join N slices together, it is necessary to perform at least N-1 times of image joining processing. However, for example, when the matching degree of the pair having the priority of image joining up to the (N-1) th is lower than the threshold, the imaging apparatus 1 informs the examiner that the matching has not been performed well. For example, manual readjustment of the initial placement of the slices and reselection of the slices to be used for the panoramic image may be prompted.

Moreover, although the said embodiment demonstrated the case where joining of each slice was performed in order from a pair with higher matching degree (namely, order of matching degree), it is not necessarily restricted to this. For example, the configuration may be such that each pair is grouped according to the matching degree of the pair, and the image joining process is performed preferentially from the pairs included in the group having a higher matching degree. In this case, the priority of the image joining process in the group may be set according to the degree of coincidence or may be at random. Note that, by grouping, for example, two groups may be set, or three or more groups may be set (for example, the first group composed of a pair having a high degree of coincidence and the first group (You may divide into the 2nd group comprised by a pair whose matching degree is lower than this group, and the 3rd group constituted by a pair whose matching degree is lower than the 2nd group.)
Further, for example, the order of coupling of each pair may be determined in consideration of not only the matching degree of each pair but also the area of overlap between slices in that pair. When the area of the region where the two slices overlap is greater than a certain level, the image quality at the joining point of the two slices is stabilized. Therefore, when the panorama image is formed, the control unit 800 sets the priority for performing the image joining process for the pair whose overlapping area is smaller than the threshold in the paired slices based on the matching degree only. (For example, the priority order determined in the order of coincidence) may be lowered. When the priority is lowered, as a result, the image joining process (and matching) for the pair may not be performed, and the order of the image joining process is slower than the order determined by the matching degree order. May be. For pairs whose overlap area is less than the threshold, by setting the priority for performing image joining processing to a low level, pairs that are stably combined are preferentially combined, so a good panoramic image can be obtained. It becomes easy.

  In the above-described embodiment, the case of matching slices using the phase-only correlation method has been described. However, the same information as that in the above-described correlation map may be obtained, and the present invention is not necessarily limited thereto. As a matching method, for example, known template matching, matching by image feature points, or the like may be used. Further, the phase only correlation method (POC) in the above embodiment may be a method of matching by parallel movement or may be a rotation invariant phase only correlation method (RIPOC).

  In the above embodiment, the control unit 800 uses one of the paired slices (one phase image) as a template and the other (the other phase image) as a reference image to be matched with the template in the matching. The correlation map obtained in this case is replaced by the relationship between the template and the reference image in the paired still images, and two types are acquired. Further, one of the two types is inverted by mirror-symmetrically inverting one of the correlation maps. A case has been described in which the degree of coincidence is obtained from the integration result of the map and the other correlation map. However, the following calculation method has been proposed and can be used. In this calculation method, two phase images are converted into a frequency space by Fourier transformation, a product of both after the transformation is obtained, and the product is subjected to inverse Fourier transformation. At this time, the peak position accuracy of the correlation map can be improved by performing inverse transformation while limiting the frequency space to the contour information of the image.

  Further, in the above-described embodiment, a case has been described in which the target acquisition position is automatically set by the control unit 800 when the imaging apparatus 1 acquires a slice. It may be set manually by the examiner. In addition, when the examiner manually designates the target slice acquisition position, the acquisition position information corresponds to, for example, an operation signal when the operation input unit 802 is operated to instruct the still image acquisition position. The information which shows the acquisition position to perform may be sufficient.

  In the above embodiment, the acquisition position information of each slice has been described based on a wide-angle fundus image obtained by the second imaging unit 200. However, the present invention is not necessarily limited thereto. For example, the acquisition position information may be obtained based on the anterior ocular segment image obtained via the anterior ocular segment observation unit 700 at each slate acquisition timing. Further, if the ophthalmologic apparatus 1 has a plurality of fixation targets having different fixation directions of the eye to be examined, lighting information of the fixation target used for photographing each slice can also be used as acquisition position information. is there.

  Further, for example, in the configuration in which the initial placement of slices is automatically performed by the control unit 800, the order of joining a plurality of pairs may be set according to, for example, the matching degree of two slices forming a pair. In this case, the matching degree may be a parameter different from the matching degree in the pair described above. In the present disclosure, the matching degree is a parameter that affects the matching accuracy (in other words, the matching accuracy), and may include, for example, the above-described matching degree and the area of an area where two slices overlap. It is a concept. Even in such a configuration, the effect of improving the image quality of the panoramic image by the accurate initial arrangement of each slice can be enjoyed in the same manner as in the above embodiment. Further, the effect of suppressing the burden on the examiner in the panoramic image creation process can be enjoyed in the same way as in the above embodiment.

  Moreover, although the case where a panoramic image is created in the control system of the photographing apparatus 1 has been described in the above embodiment, the present invention is not necessarily limited thereto. For example, a panoramic image may be created in an image processing device separate from the image capturing device 1. The image processing apparatus may be a processing apparatus included in an ophthalmic apparatus separate from the photographing apparatus 1 or a general-purpose computer (for example, a personal computer). For example, the image processing apparatus acquires a plurality of slices of the first fundus image based on the light reception signal from the light receiving element 56 included in the imaging element 1 and having different acquisition positions in the eye to be examined. A panoramic image can be formed by stitching together.

  In this case, the processor of the image processing apparatus executes the panoramic image generation process (S5) executed by the photographing apparatus 1 of the above-described embodiment on a hard disk or the like that stores an image processing program executed by the processor of the image processing apparatus. An image processing program to be executed may be prepared. In this case as well, a panoramic image can be generated in the same manner as the imaging device 1 of the above embodiment.

  In the above embodiment, the case where the wavefront sensor 73 is provided in the optical system of the ophthalmologic photographing apparatus 1 in order to measure the wavefront aberration of the eye E is not limited to this. What is necessary is just to have the structure which measures the wavefront aberration of an optometry based on the reflected light from a fundus. For example, a case where the wavefront aberration is measured using the Phase Diversity method is conceivable. This method performs image processing using an image obtained when a measurement light having a known wavefront aberration called Phase Diversity is intentionally applied to the target optical system (here, the eye to be examined). The wavefront aberration in the wavefront aberration of the optical system is estimated. Measurement light having Phase Diversity can be generated by controlling the wavefront compensation device 73 to a predetermined state, for example. Of course, it may be generated by other methods.

  In the above-described embodiment, the case where the panoramic image is formed from the still image of the cell image forming the fundus has been described. However, in the case where the panoramic image is generated from the cell image (still image) of the part other than the fundus in the eye to be examined. Alternatively, the techniques of this disclosure may be used. For example, a panoramic image of a corneal endothelial cell image may be generated using the technique of the present disclosure in some cases.

1 photographing apparatus 800 control unit 801 storage unit 803 image processing IC
TB _{→ A} , TA _{→ B} correlation map T ^TA _{→ B} inversion map

Claims (10)

An ophthalmic image processing apparatus that obtains a panoramic image by joining a plurality of still images of cell images that form a predetermined part of an eye to be examined,
Pair acquisition means for acquiring a pair of the still images from the plurality of still images;
The overlapping portion of the two still images included in the pair is matched with the pair with respect to the overlapping portion formed when the still images are arranged on a plane in a positional relationship at the predetermined portion. While matching means for obtaining the degree of coincidence regarding the overlapping portion,
For the plurality of pairs acquired by the pair acquisition means, image joining processing for joining the still images to be paired is a priority order for performing the image joining processing in the plurality of pairs, and the matching means Image joining means for generating the panoramic image by performing in priority order according to the degree of matching of each pair obtained by
An ophthalmic image processing apparatus. The image joining means includes
Each of the first pair and the first pair, which is the first matching degree that is the matching degree of the first pair, shows a high degree of coincidence, and the first pair has a different combination of the still images 2 When generating the panoramic image using a pair group consisting of two or more pairs,
When the two still images included in the first pair are joined together by the image joining process for each pair of the pair group, the image joining process is performed on the first pair. Not
When the two images as the first pair are not integrally joined by the image joining process for each pair of the pair group, the image joining process is performed on the first pair. The ophthalmic image processing apparatus according to claim 1. The matching means includes
The correlation map obtained when one of the still images in the pair is a template and the other is a reference image to which the template is matched is replaced with the still image in the pair. Get two types,
3. The ophthalmic use according to claim 1, wherein the degree of coincidence is obtained from an integration result of an inverted map obtained by inverting one of the two correlation maps in mirror symmetry and the other correlation map. 4. Image processing device.   4. The image joining unit according to claim 1, wherein, in the plurality of pairs acquired by the pair acquisition unit, the image joining process is performed in order from the pair having a higher matching degree. 5. Ophthalmic image processing apparatus.   The image joining unit divides the plurality of pairs acquired by the pair obtaining unit into a plurality of groups each including one or more pairs according to the degree of matching degree of the pairs, and the image joining process is more highly matched. The ophthalmic image processing apparatus according to claim 1, wherein the ophthalmic image processing apparatus performs the processing in order from a group indicating the degree.   The image joining means is compared with the first priority order according to the degree of coincidence only for the pair of the plurality of pairs acquired by the pair acquisition means, where the area of the overlapping portion is smaller than the threshold. The ophthalmic image processing apparatus according to claim 1, wherein the image joining process is performed with a low second priority.   The matching unit reads information on the positional relationship of the plurality of still images stored in advance in the storage unit, and the plurality of still images are arranged on the plane based on the acquired position information. The ophthalmic image processing apparatus according to claim 1, wherein the matching is performed with respect to the overlapping portion formed in a case. An ophthalmic image processing apparatus that obtains a panoramic image by joining a plurality of still images of cell images that form a predetermined part of an eye to be examined,
A pair acquisition unit that reads a plurality of the still images stored in the storage unit in advance, and acquires a pair of the still images from the plurality of still images;
Formed when the plurality of still images are arranged on a plane according to the positional relationship at the predetermined portion using information on the positional relationship at the predetermined portion of the plurality of still images stored in advance in the storage unit A matching degree acquisition means for acquiring a matching degree related to the overlapped portion,
The image joining process for joining the two still images to be the pair is an order in which the image joining process is performed in the plurality of pairs, and the plurality of pairs in the order determined based on the matching degree Image joining means for generating the panoramic image by performing
An ophthalmic image processing apparatus. An ophthalmic image processing program for generating a panoramic image by joining a plurality of still images of cell images forming a predetermined part of an eye to be examined,
By being executed by the processor of the ophthalmic image processing apparatus,
A pair acquisition step of acquiring a pair of the still images from the plurality of still images;
The overlapping portion of the two still images included in the pair is matched with the pair with respect to the overlapping portion formed when the still images are arranged on a plane in a positional relationship at the predetermined portion. A matching step for obtaining a degree of coincidence regarding the overlapping portion,
An image joining process for joining the still images to be paired with the plurality of pairs obtained by the pair obtaining step is a priority order for performing the image joining process in the plurality of pairs, and the matching unit An image joining step for generating the panoramic image by performing in a priority order according to the matching degree of each pair obtained by:
Is executed by the ophthalmic image processing apparatus. An ophthalmic image processing program for generating a panoramic image by joining a plurality of still images of cell images forming a predetermined part of an eye to be examined,
By being executed by the processor of the ophthalmic image processing apparatus,
A pair acquisition step of reading a plurality of the still images stored in the storage unit in advance, and acquiring a pair of the still images from the plurality of still images;
Formed when the plurality of still images are arranged on a plane according to the positional relationship at the predetermined portion using information on the positional relationship at the predetermined portion of the plurality of still images stored in advance in the storage unit A matching level acquisition step for acquiring a matching level for the overlapping portion to be performed;
The image joining process for joining the two still images to be the pair is an order in which the image joining process is performed in the plurality of pairs, and the plurality of pairs in the order determined based on the matching degree An image joining step for generating the panoramic image by performing
Is executed by the ophthalmic image processing apparatus.