JP6853690B2 - Ophthalmologic imaging equipment - Google Patents

Description

This invention also relates to the ophthalmic imaging equipment.

Diagnostic imaging occupies an important position in the field of ophthalmology. Typical ophthalmic modality includes a modality that can acquire a frontal image of an eye to be inspected and a modality that can acquire a cross-sectional image or a three-dimensional image. Modality that can acquire a frontal image includes a scanning laser ophthalmoscope (SLO), a fundus camera, a slit lamp microscope, a surgical microscope, and a specular microscope. In addition, modality that can acquire a cross-sectional image or the like includes an optical coherence tomography (OCT device), an ultrasonic diagnostic device, and the like.

Generally, a modality capable of acquiring a front image has a high lateral resolution and a low axial resolution (longitudinal resolution). On the other hand, the modality that can acquire a cross-sectional image or the like has a low lateral resolution and a high axial resolution. For example, the lateral resolution of the SLO is about several micrometers, and the axial resolution is about several hundred micrometers. On the other hand, the lateral resolution of the optical interference tomographic meter is about several tens of micrometers, and the axial resolution is about 5 to 10 micrometers.

Japanese Unexamined Patent Publication No. 2016-214768

An object of the present invention is to provide a new method for ophthalmic diagnostic imaging.

The first aspect of the embodiment is a front image acquisition unit that photographs the eye to be inspected to acquire a first front image, a three-dimensional image acquisition unit that scans the eye to be inspected to acquire a three-dimensional image, and the above three. It is an ophthalmic imaging apparatus including a front image creating unit that creates a second front image from a three-dimensional image, and an evaluation unit that evaluates the degree of agreement between the first front image and the second front image.

A second aspect of the embodiment is an image processing unit that processes the second front image based on the first front image when the evaluation value of the degree of agreement obtained by the evaluation unit satisfies a predetermined condition. including.

In the third aspect of the embodiment, the front image creating unit creates a plurality of second front images having different depth regions from the three-dimensional image, and the evaluation unit creates the plurality of second front images of the plurality of second front images. The degree of agreement between each of the first front image and the first front image is evaluated, and the second front image corresponding to the maximum value among the plurality of evaluation values obtained by the evaluation unit is specified for the plurality of second front images. The image processing unit processes the second front image specified by the image specifying unit based on the first front image.

A fourth aspect of the embodiment includes a determination unit for determining whether the evaluation value obtained by the evaluation unit is equal to or greater than a predetermined threshold value, and the determination unit determines that the evaluation value is equal to or greater than the predetermined threshold value. At that time, the image processing unit processes the second front image based on the first front image.

In the fifth aspect of the embodiment, the image processing unit creates a composite image of the first front image and the second front image.

In the sixth aspect of the embodiment, the image processing unit embeds a part of the first front image in the second front image.

In the seventh aspect of the embodiment, the image processing unit changes the pixel value of the second front image based on the first front image.

In the eighth aspect of the embodiment, the front image creating unit creates a plurality of second front images having different depth regions from the three-dimensional image, and the evaluation unit creates the plurality of second front images of the plurality of second front images. The degree of agreement between each and the first front image is evaluated.

A ninth aspect of the embodiment is a first selection in which one of the plurality of second front images is selected based on a plurality of evaluation values obtained by the evaluation unit for the plurality of second front images. Including part.

In a tenth aspect of the embodiment, the front image creating unit creates a second front image by projecting a segmentation unit that specifies a layer region by performing segmentation on the three-dimensional image and a layer region projected in a predetermined direction. Includes a projection processing unit.

The eleventh aspect of the embodiment includes a first registration unit that aligns the first front image and the second front image, and the evaluation unit includes the first alignment unit. The degree of agreement between the front image and the second front image is evaluated.

In the twelfth aspect of the embodiment, the first registration unit analyzes the first front image to detect the first feature point group, and analyzes the second front image to analyze the second feature point. It includes a feature point detection unit that detects a group, and aligns the first front image and the second front image based on the first feature point group and the second feature point group.

In the thirteenth aspect of the embodiment, the first registration unit performs gamma curve correction, sigmoid curve correction, on at least one of the first front image and the second front image input to the feature point detection unit. The feature point detection section includes a pretreatment section that performs pretreatment including at least one of a brightness correction, a Gaussian filter, and a median filter, and the feature point detection section performs edge detection on the first front image to perform the first feature point. The group is detected, and the second front image is subjected to edge detection to detect the second feature point group.

A fourteenth aspect of the embodiment includes a second registration unit that aligns the first front image with the three-dimensional image, and the evaluation unit is based on the result of the alignment. The degree of agreement between the first front image and the second front image is evaluated.

In the fifteenth aspect of the embodiment, the front image acquisition unit acquires two or more first front images corresponding to different wavelength bands, and the evaluation unit acquires each of the two or more first front images and the said. The degree of agreement with the second front image is evaluated.

In the sixteenth aspect of the embodiment, the front image acquisition unit acquires two or more first front images corresponding to different wavelength bands, and the front image creation unit has a depth region from the three-dimensional image. A plurality of different second front images are created, and the evaluation unit evaluates the degree of agreement between each of the two or more first front images with at least a part of the plurality of second front images. .. Further, the 16th aspect of the embodiment includes a second selection unit that selects a second front image corresponding to each of the two or more first front images based on the evaluation value obtained by the evaluation unit. ..

A seventeenth aspect of the embodiment is a front image acquisition unit that captures an eye to be inspected to acquire a first front image, a three-dimensional image receiving unit that receives a three-dimensional image of the eye to be inspected, and a third-dimensional image to be obtained. 2 This is an ophthalmic imaging apparatus including a front image creating unit for creating a front image and an evaluation unit for evaluating the degree of agreement between the first front image and the second front image.

An eighteenth aspect of the embodiment is a front image receiving unit that receives a first front image of the eye to be inspected, a three-dimensional image acquisition unit that scans the eye to be inspected to acquire a three-dimensional image, and a third from the three-dimensional image. 2 This is an ophthalmic imaging apparatus including a front image creating unit for creating a front image and an evaluation unit for evaluating the degree of agreement between the first front image and the second front image.

A nineteenth aspect of the embodiment is an image receiving unit that receives a first front image and a three-dimensional image of the eye to be inspected, a front image creating unit that creates a second front image from the three-dimensional image, and the first front image. It is an ophthalmic image processing apparatus including an evaluation unit for evaluating the degree of agreement between the image and the second front image.

According to the embodiment, a new method for ophthalmologic image diagnosis can be provided.

It is the schematic which shows an example of the structure of the ophthalmologic imaging apparatus which concerns on embodiment. It is the schematic which shows an example of the structure of the ophthalmologic imaging apparatus which concerns on embodiment. It is the schematic which shows an example of the structure of the ophthalmologic imaging apparatus which concerns on embodiment. It is the schematic which shows an example of the structure of the ophthalmologic imaging apparatus which concerns on embodiment. It is the schematic which shows an example of the structure of the ophthalmologic imaging apparatus which concerns on embodiment. It is a schematic diagram for demonstrating an example of the process executed by the ophthalmologic imaging apparatus which concerns on embodiment. It is a schematic diagram for demonstrating an example of the process executed by the ophthalmologic imaging apparatus which concerns on embodiment. It is a schematic diagram for demonstrating an example of the process executed by the ophthalmologic imaging apparatus which concerns on embodiment. It is the schematic which shows an example of the structure of the ophthalmologic imaging apparatus which concerns on embodiment. It is the schematic which shows an example of the structure of the ophthalmologic imaging apparatus which concerns on embodiment. It is the schematic which shows an example of the structure of the ophthalmologic imaging apparatus which concerns on embodiment. It is the schematic which shows an example of the structure of the ophthalmologic imaging apparatus which concerns on embodiment. It is the schematic which shows an example of the structure of the ophthalmologic imaging apparatus which concerns on embodiment. It is the schematic which shows an example of the structure of the ophthalmologic imaging apparatus which concerns on embodiment. It is a flowchart which shows an example of the operation of the ophthalmologic imaging apparatus which concerns on embodiment.

An exemplary embodiment will be described below. The matters described in the cited document and any known technique can be incorporated into the embodiment.

The apparatus according to the embodiment includes a function of acquiring a front image of the front image of the eye to be inspected and a function of acquiring a three-dimensional image of the eye to be inspected.

The front image acquisition function is realized, for example, by a front image acquisition unit that captures an eye to be inspected and acquires a front image, or a front image reception unit that receives a front image of the eye to be inspected from the outside. The front image acquisition unit may be, for example, any of an SLO, a fundus camera, a slit lamp microscope, and a surgical microscope. The front image receiving unit may be, for example, either a communication interface for receiving the front image via the communication line or an interface for reading the front image recorded on the recording medium.

The three-dimensional image acquisition function is realized, for example, by a three-dimensional image acquisition unit that scans the eye to be inspected to acquire a three-dimensional image, or a three-dimensional image reception unit that receives a three-dimensional image of the eye to be inspected from the outside. The three-dimensional image acquisition unit may be, for example, an optical interference tomogram or an ultrasonic diagnostic apparatus. The 3D image receiving unit may be, for example, either a communication interface for receiving a 3D image via a communication line or an interface for reading a 3D image recorded on a recording medium.

When one or both of the front image acquisition unit and the three-dimensional image acquisition unit are provided, the apparatus of the embodiment is configured as an ophthalmologic imaging apparatus. On the other hand, when both the front image receiving unit and the three-dimensional image receiving unit are provided, the apparatus of the embodiment is configured as an ophthalmic image processing apparatus.

Hereinafter, unless otherwise specified, the left-right direction is the X direction, the up-down direction is the Y direction, and the front-back direction (depth direction) is the Z direction when viewed from the subject. The Z direction is a direction along the optical axis of the optical system. The X, Y, and Z directions define a three-dimensional Cartesian coordinate system. The resolution in the Z direction is the axial resolution. Further, the resolution on the XY plane orthogonal to the Z direction is the lateral resolution.

<Optical system 100>
Examples of the optical system of the ophthalmologic imaging apparatus according to the embodiment are shown in FIGS. 1 to 3. In this embodiment, an ophthalmologic imaging apparatus in which an SLO and an optical interference tomography device are combined will be described, but the embodiment is not limited to this as described above.

The ophthalmologic imaging apparatus can operate in a plurality of imaging modes. For example, there are a wide-angle shooting mode and a high-magnification shooting mode as operation modes related to the size of the scanning range (angle of view, magnification). The switching of the angle of view is realized, for example, by selectively using two or more objective lenses having different refractive powers. Alternatively, the angle of view may be changed by changing the deflection angle of the light beam by the light deflector (optical scanner). Further, the angle of view may be changed by using a zoom optical system capable of moving or selectively using one or more zoom lenses. The method and configuration for changing the angle of view are not limited to these.

FIG. 1 shows an example of an optical system in the wide-angle shooting mode. FIG. 2 shows an example of an objective lens system for switching the angle of view. FIG. 3 shows an example of the optical system of the ophthalmologic imaging apparatus in the high magnification imaging mode. Reference numeral P in FIGS. 1 and 3 indicates a position optically conjugate with the fundus Ef (fundus conjugate position), and reference numeral Q indicates a position optically conjugate with the pupil of the eye E to be inspected (pupil conjugate position). ..

The optical system 100 scans the fundus Ef with a light beam and collects data. The optical system 100 includes a scanning system that scans the fundus Ef with a light beam projected on the eye E to be inspected via the objective lens system 110, and a detection system that detects the return light of the projected light beam via the objective lens system 110. Including the system. An image of the fundus Ef is constructed based on the output from the detection system (that is, the data collected by the optical system 100). The optical system 100 includes an SLO optical system 130 and an OCT optical system 140. The SLO optical system 130 includes an SLO scanning system and an SLO detecting system. The OCT optical system 140 includes an OCT scanning system and an OCT detection system.

The ophthalmologic imaging apparatus is provided with an anterior segment imaging system 120 for observing and photographing the anterior segment. The optical system 100, the objective lens system 110, and the anterior segment imaging system 120 can move in the X direction, the Y direction, and the Z direction. The anterior segment image obtained by the anterior segment imaging system 120 is used for alignment and tracking of the optical system 100.

<Objective lens system 110>
In an exemplary embodiment, an objective lens (unit) is prepared for each shooting mode, and an objective lens unit corresponding to the selected shooting mode is selectively used. In this embodiment, as shown in FIG. 2, the objective lens unit 110A for a wide-angle shooting mode (for example, an angle of view of 100 degrees) and the objective lens unit 110B for a high-magnification shooting mode (for example, an angle of view of 50 degrees) are , Selectively arranged in the optical path of the optical system 100.

The objective lens system 110 includes an angle of view changing mechanism 115 in addition to the objective lens units 110A and 110B. The angle of view changing mechanism 115 includes, for example, a known rotation mechanism or slide mechanism, and selectively (exclusively) arranges the objective lens units 110A and 110B in the optical path. The angle of view changing mechanism 115 arranges the objective lens unit 110A (110B) in the optical path so that the optical axis of the objective lens unit 110A (110B) substantially coincides with the optical axis O of the optical system 100.

The angle of view changing mechanism 115 may include a configuration for manually moving the objective lens units 110A and 110B. In this case, it is possible to provide a type detection unit for detecting the type of the objective lens unit arranged in the optical path, specify the shooting mode from the detection result, and execute the control according to the specific result. The angle of view changing mechanism 115 may have a configuration for electrically (and automatically) moving the objective lens units 110A and 110B. In this case, the control unit 200, which will be described later, sends a control signal for arranging the objective lens unit corresponding to the selected shooting mode in the optical path to the angle of view changing mechanism 115.

The objective lens unit 110A for the wide-angle shooting mode includes lenses 111A and 112A, a dichroic mirror DM1A, and a concave lens 113A. The dichroic mirror DM1A combines the optical path of the optical system 100 and the optical path of the anterior segment imaging system 120. The dichroic mirror DM1A transmits the light guided by the optical system 100 and reflects the light for photographing the anterior segment of the eye. A fundus conjugate position P is arranged between the dichroic mirror DM1A and the concave lens 113A.

The objective lens unit 110B for the high magnification shooting mode includes a lens 111B and a dichroic mirror DM1B. The dichroic mirror DM1B has the same function as the dichroic mirror DM1A.

The dichroic mirror DM1A and the dichroic mirror DM1B are arranged at (almost) the same position in the optical path of the optical system 100. As a result, it is not necessary to adjust the position and orientation of the anterior segment imaging system 120 when the imaging mode is switched.

In an exemplary embodiment, a single dichroic mirror can be configured to be shared by a plurality of objective lens units. For example, in the example shown in FIG. 2, the dichroic mirrors DM1A and DM1B may be the same member. That is, a configuration can be applied in which the objective lens unit 110A including only the lenses 111A and 112A and the concave lens 113A and the objective lens unit 110B including only the lens 111B are selectively used.

The objective lens system 110 can be moved along the optical axis O. That is, the objective lens system 110 can be moved in the Z direction with respect to the optical system 100. As a result, the focal position of the SLO optical system 130 and the focal position of the OCT optical system 140 are changed.

In an exemplary embodiment, three or more objective lens units can be selectively used. For example, an objective lens unit for a high magnification shooting mode, a medium magnification shooting mode, and a low magnification shooting mode, and an angle of view changing mechanism for selectively arranging these in the optical path may be provided.

Hereinafter, a state in which the objective lens unit 110A is arranged in the optical path will be mainly described. Unless otherwise specified, the same or similar matters in the state where the objective lens unit 110B is arranged will be omitted.

<Anterior segment imaging system 120>
The anterior segment imaging system 120 includes an anterior segment illumination light source 121, a lens 122, an anterior segment imaging camera 123, an imaging lens 124, and a beam splitter BS1. The beam splitter BS1 combines the optical path of the anterior segment illumination light and the optical path of its return light.

The anterior segment illumination light source 121 includes an infrared light source such as an infrared LED. The anterior segment illumination light emitted by the anterior segment illumination light source 121 is refracted by the lens 122, reflected by the beam splitter BS1 toward the dichroic mirror DM1A, and reflected by the dichroic mirror DM1A toward the eye E to be inspected. The return light of the anterior segment illumination light from the eye E to be examined is reflected by the dichroic mirror DM1A, transmitted through the beam splitter BS1, and focused on the anterior segment imaging camera 123 (detection surface of the imaging element) by the imaging lens 124. Will be done. The detection surface of the image sensor is arranged at the pupil conjugate position Q (or its vicinity). The image sensor is, for example, a CCD image sensor or a CMOS image sensor.

<SLO optical system 130 and OCT optical system 140>
The optical path of the SLO optical system 130 and the optical path of the OCT optical system 140 are coupled by a dichroic mirror DM2. In an exemplary embodiment, at least part of the SLO optical system 130 is a telecentric optical system, at least a part of the OCT optical system 140 is a telecentric optical system, and the optical paths of these telecentric optical systems are coupled by a dichroic mirror DM2. The optical path. According to this example, even if the objective lens system 110 is moved to change the focal position of the optical system 100, the aberration of the pupil (for example, the exit pupil by the objective lens system 110) does not increase, so that the focus adjustment is facilitated. be able to.

<SLO optical system 130>
The SLO optical system 130 includes an SLO light source 131, a collimating lens 132, a beam splitter BS2, a condenser lens 133, a confocal diaphragm 134, a detector 135, an optical scanner 136, and a lens 137. The beam splitter BS2 combines the optical path of the SLO light projected on the eye E to be examined and the optical path of the return light thereof.

The SLO light source 131 outputs light having a wavelength that can be used for the SLO. The SLO light source 131 includes, for example, a laser diode, a super luminescent diode, a laser driven light source, and the like. The SLO light source 131 is arranged at the fundus conjugate position P (or its vicinity). The SLO light source 131 includes a visible light source and / or an infrared light source. The visible light output from the visible light source may be used as the fixation light for projecting the fixation target onto the fundus Ef.

The SLO light source 131 may be configured to selectively output light in different wavelength bands. Further, the SLO light source 131 may be configured so that two or more lights having different wavelength bands can be output in parallel. At least one of visible light and infrared light is used for SLO imaging.

The optical scanner 136 includes an optical scanner 136X that deflects light in the X direction and an optical scanner 136Y that deflects light in the Y direction. One of the optical scanners 136X and 136Y is a low-speed scanner (galvano mirror, etc.), and the other is a high-speed scanner (resonant mirror, polygon mirror, MEMS mirror, etc.). The reflective surface of the optical scanner 136Y is arranged at the pupil conjugate position Q (or its vicinity).

The opening formed in the confocal diaphragm 134 is arranged at the fundus conjugate position P (or its vicinity). The detector 135 includes, for example, an avalanche photodiode or a photomultiplier tube.

The light beam (SLO light) output from the SLO light source 131 is converted into a parallel light beam by the collimating lens 132, is transmitted through the beam splitter BS2, is deflected by the optical scanner 136, is refracted by the lens 137, and is transmitted through the dichroic mirror DM2. , Is projected onto the fundus Ef via the objective lens system 110. The return light of the SLO light projected on the fundus Ef travels in the same optical path in the opposite direction, is guided to the beam splitter BS2, is reflected by the beam splitter BS2, is focused by the condenser lens 133, and is focused on the confocal aperture 134. It passes through the opening and is detected by the detector 135.

<OCT optical system 140>
The OCT optical system 140 includes a focusing lens 141, an optical scanner 142, a collimating lens 143, and an interference optical system 150.

The focusing lens 141 is moved along the optical axis of the OCT optical system 140. As a result, the focal position of the OCT optical system 140 is changed independently of the SLO optical system 130. After the focusing state of the SLO optical system 130 and the OCT optical system 140 is adjusted by moving the objective lens system 110, the focusing state of the OCT optical system 140 can be finely adjusted by moving the focusing lens 141.

The optical scanner 142 includes an optical scanner 142X that deflects light in the X direction and an optical scanner 142Y that deflects light in the Y direction. Each of the optical scanners 142X and 142Y is, for example, a galvano mirror. The intermediate position between the two optical scanners 142X and 142Y corresponds to the pupil conjugate position Q (or its vicinity).

The collimating lens 143 guides the OCT light (measurement light) emitted from the fiber end c3 of the optical fiber f4 to the optical scanner 142 as a parallel luminous flux, and collects the return light of the measurement light from the fundus Ef toward the fiber end c3. It glows.

The interferometric optical system 150 includes an OCT light source 151, fiber couplers 152 and 153, a reference prism 154, and a detector 155. The interferometric optics 150 comprises, for example, a configuration for performing a swept source OCT or a spectral domain OCT. In swept source OCT, a tunable light source is used as the OCT light source 151, and a balanced photodiode is used as the detector 155. In the spectral domain OCT, a low coherence light source (broadband light source) is used as the OCT light source 151, and a spectroscope is used as the detector 155.

The OCT light source 151 emits, for example, a light beam L0 that emits light having a center wavelength of 1050 nm. The light L0 is guided by the fiber coupler 152 through the optical fiber f1 and is divided into the measurement light LS and the reference light LR.

The reference light LR is emitted from the fiber exit end c1 through the optical fiber f2, is converted into a parallel light flux by the collimating lens 156, is folded back by the reference prism 154, is converted into a focused light flux by the collimating lens 157, and is incident on the fiber incident end c2. It is guided to the fiber coupler 153 through the optical fiber f3. The reference prism 154 is moved to change the optical path length of the reference light LR as in the conventional case. Further, a polarization controller, an attenuator, an optical path length correction member, and a dispersion compensation member may be provided in the optical path of the reference light.

On the other hand, the measurement light LS generated by the fiber coupler 152 is emitted from the fiber end c3 through the optical fiber f4, is converted into a parallel light beam by the collimated lens 143, passes through the optical scanner 142 and the focusing lens 141, and is transmitted by the dichroic mirror DM2. It is reflected, refracted by the objective lens system 110, and projected onto the fundus Ef. The measurement light LS is reflected and scattered at various depth positions of the fundus Ef. The return light of the measurement light LS including the backscattered light travels in the opposite direction along the same path, is guided to the fiber coupler 152, and reaches the fiber coupler 153 through the optical fiber f5.

The fiber coupler 153 superimposes the measurement light LS incident through the optical fiber f5 and the reference light LR incident through the optical fiber f3 to generate interference light. FIG. 1 and the like show the case of swept source OCT. The fiber coupler 153 branches the interference light at a predetermined branching ratio (for example, 1: 1) to generate a pair of interference light LCs. The pair of interference light LCs are detected by a detector 155 (balanced photodiode). In the case of spectral domain OCT, the detector 155 (spectrometer) decomposes the interference light generated by the fiber coupler 153 into a plurality of wavelength components and detects them.

The detector 155 sends the result (detection signal) of detecting the pair of interference light LCs to a data acquisition system (Data Acquisition System, or DAQ) (not shown). A clock is supplied to the data acquisition system from the OCT light source 151. This clock is generated in synchronization with the output timing of each wavelength swept within a predetermined wavelength range by a tunable light source. The data acquisition system samples the detection signal based on this clock. The sampling result is sent to the processor for forming the OCT image.

<Processing system>
Some configuration examples of the processing system of the ophthalmologic imaging apparatus according to the embodiment will be described. The processing system illustrated in FIG. 4 includes a processor for executing various data processing (signal processing, image processing, calculation, control, storage, etc.).

The "processor" includes, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), a programmable logic device (for example, a SPLD (Simple It means a circuit such as Device), FPGA (Field Programmable Gate Array)). The processor realizes the function according to the embodiment by reading and executing a program stored in a storage circuit or a storage device, for example.

<Control unit 200>
The control unit 200 includes a configuration for controlling each unit of the ophthalmologic imaging apparatus. The control unit 200 includes a main control unit 201 and a storage unit 202. The function of the main control unit 201 is realized by a processor or the like. Various data, various information, and various computer programs are stored in the storage unit 202. The storage unit 202 includes a semiconductor memory and a magnetic storage device.

The processing executed by the ophthalmologic imaging apparatus is realized by the cooperation of hardware resources (processor etc.) and software (computer program etc.). Further, an actuator is provided in at least a part of various mechanisms provided in the ophthalmologic imaging apparatus, and the main control unit 201 sends a control signal to the actuator.

<Main control unit 201>
The main control unit 201 controls each unit of the ophthalmologic imaging apparatus. For example, as an example of control regarding the objective lens system 110, the main control unit 201 controls the angle of view changing mechanism 115 for arranging one of the objective lens units 110A and 110B in the optical path, and sets the objective lens system 110 on the optical axis O. It is possible to control a movement mechanism (not shown) for moving along the.

As an example of control relating to the SLO optical system 130, the main control unit 201 can execute control of the SLO light source 131, control of the optical scanner 136, and control of the detector 135. The control of the SLO light source 131 includes turning on, turning off, selecting a wavelength band, adjusting the amount of light, adjusting the aperture, and the like. The control of the optical scanner 136 includes a control of a scanning position, a control of a scanning range, a control of a scanning pattern, a control of a scanning speed, and the like. Control of the detector 135 includes adjustment of the exposure of the detection element, adjustment of the gain, adjustment of the detection rate, and the like.

As an example of control relating to the OCT optical system 140, the main control unit 201 executes control of the OCT light source 151, control of the optical scanner 142, movement control of the focusing lens 141, movement control of the reference prism 154, and control of the detector 155. can do. The control of the OCT light source 151 includes turning on, turning off, selecting a wavelength band, adjusting the amount of light, adjusting the aperture, and the like. The control of the optical scanner 142 includes a control of a scanning position, a control of a scanning range, a control of a scanning pattern, a control of a scanning speed, and the like. The control of the detector 155 includes adjustment of the exposure of the detection element, adjustment of the gain, adjustment of the detection rate, and the like.

As an example of control relating to the anterior segment imaging system 120, the main control unit 201 can execute control of the anterior segment illumination light source 121, control of the anterior segment imaging camera 123, and the like. The control of the anterior segment illumination light source 121 includes lighting, extinguishing, adjusting the amount of light, adjusting the aperture, and the like. The control of the anterior segment imaging camera 123 includes adjustment of the exposure of the image sensor, adjustment of the gain, adjustment of the imaging rate, and the like.

An example of control relating to the optical system 100 is control of the optical system moving mechanism 100A for moving the optical system 100 in the X direction, the Y direction, and the Z direction. The optical system moving mechanism 100A includes, for example, a mechanism for moving the optical system in the X direction, a mechanism for moving the optical system in the Y direction, and a mechanism for moving the optical system in the Z direction. Each of these mechanisms includes, for example, an actuator that generates a driving force. Instead of or in addition to such an electric mechanism, the optical system moving mechanism 100A may include a mechanical mechanism.

When performing SLO photographing, for example, the main control unit 201 controls the optical scanner 136 according to a predetermined scanning pattern (for example, raster scan) while lighting (blinking) the SLO light source 131 at a predetermined timing. In another example, the main control unit 201 controls the optical scanner 136 according to a predetermined scanning pattern while continuously lighting the SLO light source 131.

When shooting with visible light, the main control unit 201 performs the shooting control as described above, and only at the timing corresponding to the predetermined fixation position (fixation timing), the visible light different from the visible light for shooting (the visible light (fixation timing)). The SLO light source 131 is controlled so as to output (visible light for fixation). At the fixation timing, the output of visible light for photographing may be stopped and the visible light for fixation may be output. Alternatively, both visible light for photographing and visible light for fixation may be output at the fixation timing.

When shooting with invisible light (infrared light), the main control unit 201 controls the SLO light source 131 so as to output visible light only at a predetermined fixation timing while performing the shooting control as described above.

When performing OCT, the main control unit 201 controls the optical scanner 142 according to a predetermined scanning pattern while lighting (blinking) the OCT light source 151 at a predetermined timing. In another example, the main control unit 201 controls the optical scanner 142 according to a predetermined scanning pattern while continuously lighting the OCT light source 151.

<Memory unit 202>
The storage unit 202 stores information, data, programs, and the like used by the ophthalmologic imaging apparatus. Further, the storage unit 202 stores data (SLO image, OCT image, anterior ocular region image, etc.) acquired by the ophthalmologic imaging apparatus.

<Image forming unit 210>
The image forming unit 210 forms an image of the fundus Ef based on the data collected by the optical system 100. Since the ophthalmologic imaging apparatus of this embodiment can perform both SLO and OCT, the image forming unit 210 includes an SLO image forming unit 211 and an OCT image forming unit 212.

In other embodiments, the ophthalmologic imaging apparatus may be capable of performing only one of SLO and OCT. In this case, the image forming unit 210 may include only one of the SLO image forming unit 211 and the OCT image forming unit 212.

The SLO image forming unit 211 forms an SLO image based on the data collected by the SLO optical system 130. More specifically, the SLO image forming unit 211 forms an SLO image based on the detection signal input from the detector 135 and the pixel position signal input from the control unit 200, similarly to the conventional SLO. To do.

The OCT image forming unit 212 forms an OCT image based on the data collected by the OCT optical system 140. More specifically, the OCT image forming unit 212 forms an OCT image based on the detection signal input from the detector 155 and the pixel position signal input from the control unit 200. The OCT image forming unit 212 generates a spectral distribution based on the output from the detector 155 for each series of wavelength scans (for each A line) as in the conventional case, and performs signal processing such as Fourier transform on this spectral distribution. Give. Thereby, the reflection intensity profile in each A line is obtained. Further, the OCT image forming unit 212 forms a cross-sectional image (B scan image) by imaging the reflection intensity profile of each A line.

The OCT image forming unit 212 can form a three-dimensional image based on a plurality of B-scan images. This three-dimensional image is a three-dimensional data set such as stack data and volume data. Further, the OCT image forming unit 212 can form a display image by rendering a three-dimensional data set.

The image forming unit 210 can form an anterior segment image based on the output from the anterior segment photographing camera 123. Various images (image data) formed by the image forming unit 210 are stored in, for example, a storage unit 202.

<Data processing unit 220>
The data processing unit 220 executes various data processing. As an example of data processing, there is processing on image data formed by the image forming unit 210 or another device. Examples of this processing include various image processing, analysis processing for images, and diagnostic support processing such as image evaluation based on image data.

The data processing unit 220 will be described with reference to FIGS. 5 to 12. Any two or more of the plurality of examples described below can be combined.

The data processing unit 220 matches the function of creating a front image from the three-dimensional image of the fundus Ef acquired by using OCT (front image creation unit) and the front image and the front image acquired by SLO. It has a function (evaluation unit) to evaluate the degree. Hereinafter, some examples having at least these functions will be described.

<Data processing unit 220A>
The data processing unit 220A shown in FIG. 5 includes a front image creating unit 221, a registration unit 222, an evaluation unit 223, and an image processing unit 224.

The front image creating unit 221 forms a front image from a three-dimensional image of the fundus Ef formed by the OCT image forming unit 212 (or another OCT device). Such a front image will be referred to as an OCT front image. Typical examples of OCT frontal images are projection images and shadowgrams.

A projection image is created by projecting a three-dimensional image in a predetermined direction. Similar to a typical SLO image, the OCT front image created from the three-dimensional image of the fundus Ef expresses the morphology of the surface of the fundus Ef. The direction in which the three-dimensional image is projected is typically the Z direction (depth direction, axial direction, A scan direction).

A shadow gram is created by projecting a part of a three-dimensional image (for example, partial data corresponding to a specific layer) in a predetermined direction. In the shadow gram, the morphology of the tissue represented in the selected partial data (typically three-dimensional data) is represented as a projection image. For example, a shadowgram created from partial data corresponding to the internal limiting membrane (ILM) of the fundus Ef represents the morphology of the internal limiting membrane (the surface of the fundus Ef). In addition, the morphology of the inner plexiform layer and the inner nuclear layer is expressed in the shadow gram created from the partial data corresponding to the composite layer including the inner plexiform layer (IPL) and the inner nuclear layer.

The front image creating unit 221 of this example includes a segmentation unit 2211 and a projection processing unit 2212. In the embodiment in which only the projection image can be created, it is not necessary to provide the segmentation unit 2211.

The segmentation unit 2211 analyzes the three-dimensional image acquired by using OCT to identify at least one segment. The segment identified is typically an image of the layered tissue of the eye or an image of the layer boundary (layer area). Such processing is called segmentation, and is executed based on, for example, a change in the pixel value (luminance value) in the Z direction.

Examples of segments identified from the three-dimensional image of the fundus Ef include the image of the internal limiting membrane, the image of the nerve fiber layer, the image of the ganglion cell layer, the image of the inner nuclear layer, the image of the inner nuclear layer, and the outer nuclear layer. There are images, images of the outer nuclear layer, images of the outer limiting membrane, images of retinal pigment epithelium, images of Bruch's membrane, images of choroid, images of choroidal sclera, images of sclera, and the like.

In one example, the segmentation unit 2211 is configured to execute a process of specifying a plurality of segments depicted in a three-dimensional image and a process of selecting one or more segments of interest from the specified plurality of segments. Will be done. In this case, the data of one or more selected segments or the information indicating the selected one or more segments is sent to the projection processing unit 2212.

In another example, the segmentation unit 2211 executes a process of extracting the partial data (three-dimensional data) of interest from the three-dimensional image and a process of specifying one or more segments drawn in the extracted partial data. It is configured as follows. In this case, the data of one or more specified segments or the information indicating the specified one or more segments is sent to the projection processing unit 2212.

The projection processing unit 2212 creates a shadow gram by projecting data of one or more segments sent from the segmentation unit 2211 in a predetermined direction. This projection process includes, for example, a process of adding the luminance values of the pixel groups arranged in a row in the Z direction.

When creating a projection image, the projection processing unit 2212 projects a three-dimensional image acquired by using OCT in a predetermined direction.

In any case, the projection direction is not limited to the Z direction and may be any direction. For example, the projection direction can be determined based on the orientation of the segment of interest or the orientation of other tissues. When considering the orientation of the segment of interest, for example, the normal of the front surface (the surface on the anterior segment side) or the normal of the posterior surface of the segment of interest can be obtained, and the projection direction can be determined based on the orientation of this normal. When considering the orientation of other tissues, for example, the orientation of the surface normal of the fundus Ef, the orientation of the normal of a specific layer inside the fundus Ef, the dent direction of the optic nerve head, the dent direction of the yellow spot, the direction of the axis The projection direction can be determined based on the orientation, the orientation of the visual axis, and the like.

The registration unit 222 performs alignment (registration) between the SLO image of the fundus Ef and the OCT front image formed by the projection processing unit 2212. Registration includes, for example, a first process of detecting a feature region from each image and a second process of aligning both images with reference to the feature regions detected from both.

The characteristic region detected in the first treatment is, for example, any of a region corresponding to the optic nerve head, a region corresponding to the macula, a region corresponding to a blood vessel, a region corresponding to a lesion, and a region corresponding to a laser treatment scar. It may be there. In the first process, the registration unit 222 can detect the feature region by referring to the pixel value and the pixel arrangement.

In the second process, the registration unit 222, for example, sets the relative position of the SLO image and the OCT front image so that the feature area detected from the SLO image and the feature area detected from the OCT front image match as much as possible. adjust. At this time, the registration unit 222 may specify the contour and representative points (center point, center of gravity point, etc.) of the feature region, and perform registration so as to match them.

The registration unit 222 of this example includes a pretreatment unit 2221, a feature point detection unit 2222, and an alignment unit 2223.

The pre-processing unit 2221 executes a process (pre-processing) for facilitating the detection of feature points in the image. As will be described later, in this example, edge detection is performed to detect feature points. The accuracy of edge detection improves as the feature points in the image become clearer. The preprocessing of this example may include processing for sharpening feature points in an image (noise reduction, noise removal, contrast adjustment, etc.).

The pretreatment of this example may include at least one of a gamma curve correction, a sigmoid curve correction, a brightness correction, a Gaussian filter, and a median filter. Gamma correction is a known nonlinear luminance conversion for a gamma value that indicates the response characteristics of the gradation of an image. The sigmoid curve correction is a known nonlinear luminance conversion using a sigmoid function (sigmoid curve). Luminance correction is a known luminance conversion that extends this luminance distribution over a wider range when the luminance values of an image are unevenly distributed in a certain range. The Gaussian filter is a known smoothing process (noise reduction process) using a Gaussian function. The median filter is a known smoothing process (noise reduction process) that converts the value of a specific pixel into the median value of the brightness values of the surrounding pixel groups.

The preprocessing unit 2221 applies, for example, the following types of processing to each of the SLO image and the OCT front image in the following order: gamma correction or sigmoid curve correction; brightness correction; Gaussian filter; median filter. The type and order of processing executed by the preprocessing unit 2221 is not limited to this. Further, the preprocessing unit 2221 may apply the preprocessing to only one of the SLO image and the OCT front image. Further, the pre-processing unit 2221 may be configured to be able to determine whether or not to apply the pre-processing, determine the type of processing to be applied, and determine the order of processing.

The feature point detection unit 2222 analyzes the SLO image and detects a plurality of feature points (first feature point group). Further, the feature point detection unit 2222 analyzes the OCT front image to detect a plurality of feature points (second feature point group). Here, at least one of the SLO image and the OCT front image is a preprocessed image. Typically, both the SLO image and the OCT front image are preprocessed images.

By applying edge detection to the SLO image, the feature point detection unit 2222 of this example detects a first feature point group representing an edge of a feature portion such as a blood vessel depicted in the SLO image. Similarly, the feature point detection unit 2222 of this example detects the second feature point group representing the edge of the feature portion such as a blood vessel drawn in the OCT front image by applying the edge detection to the OCT front image. The method applied to the detection of feature points is not limited to edge detection.

If the number of feature points is too large, the processing time for image alignment in the subsequent stage increases. Therefore, the feature point detection unit 2222 can generate the first feature point cloud by thinning out the number of feature points detected from the SLO image. Similarly, the feature point detection unit 2222 can generate a second feature point group by thinning out the number of feature points detected from the OCT front image.

The alignment unit 2223 aligns the SLO image and the OCT front image based on the first feature point group detected from the SLO image and the second feature point group detected from the OCT front image.

In the image alignment of this example, for example, the SLO image is set as the reference image and the OCT front image is set as the comparison image. In this case, the position of the OCT front image is adjusted with respect to the SLO image. This position adjustment may include at least one of adjusting the orientation of the image and adjusting the size of the image.

When a plurality of SLO images are present, the alignment unit 2223 can select a reference image from these SLO images. This selection includes, for example, a process of evaluating the contrast of each of the plurality of SLO images and a process of identifying the SLO image having the maximum evaluation value. The SLO image specified by this can be set as a reference image.

When the displacement between the first feature point cloud in the SLO image and the second feature point cloud in the OCT front image is large, the search range for alignment becomes wide and the processing time may increase. Therefore, in this example, the movement-limited correlation method is used in which the difference between the processing time when the displacement is large and the processing time when the displacement is small is small. The method applied to the alignment is not limited to the phase-limited correlation method.

In this example using the phase-limited correlation method, the alignment unit 2223 applies a Fourier transform to the SLO image (reference image) to obtain the amplitude characteristic and the phase characteristic. Similarly, the alignment unit 2223 applies a Fourier transform to the OCT front image (comparative image) to obtain the amplitude characteristic and the phase characteristic. Next, the alignment unit 2223 obtains the difference between the phase characteristic of the SLO image and the phase characteristic of the OCT front image. Next, the alignment unit 2223 obtains the moving direction and the moving amount of the OCT front image so as to cancel the displacement of the OCT front image with respect to the SLO image by applying the inverse Fourier transform to the obtained difference. Finally, the alignment unit 2223 aligns the SLO image and the OCT front image by moving the OCT front image based on the obtained movement direction and movement amount.

The evaluation unit 223 evaluates the degree of agreement between the SLO image and the OCT front image. In this example, the evaluation unit 223 calculates a value (evaluation value) indicating the degree of agreement between the SLO image and the OCT front image whose relative positions are adjusted by the registration unit 222.

The degree of matching evaluated by the evaluation unit 223 includes, for example, not only the degree of matching of the position between the SLO image and the OCT front image, but also the tissue depicted in the SLO image and the tissue depicted in the OCT front image. The degree of agreement with is also reflected. For example, even if registration is performed with high accuracy and high accuracy, if the depth region of the fundus Ef depicted in the SLO image and the depth region of the fundus Ef depicted in the OCT front image are different. , The degree of agreement between the SLO image and the OCT front image is low.

The method of evaluating the degree of agreement between the SLO image and the OCT front image is arbitrary. For example, the evaluation unit 223 of this example creates a difference image between the SLO image and the OCT front image after registration. This process includes, for example, a process of associating the pixels of the SLO image with the pixels of the OCT front image and a process of calculating the difference in the luminance value between the associated pixels.

Further, the evaluation unit 223 integrates the obtained plurality of differences. This integral value becomes smaller as the degree of agreement is higher, and conversely becomes larger as the degree of agreement is lower. This integrated value is used as an evaluation value indicating the degree of agreement between the SLO image and the OCT front image. The evaluation value is not limited to this, and the evaluation value can be obtained by any method.

The image processing unit 224 processes the OCT front image based on the SLO image when the evaluation value of the degree of matching obtained by the evaluation unit 223 satisfies a predetermined condition. Some examples of this condition will be described later (see FIGS. 6 and 7).

An example of the processing executed by the image processing unit 224 will be described. In the first example, the image processing unit 224 can create a composite image of the SLO image and the OCT front image. This composite image is, for example, a process of calculating the statistical value of the brightness value of the SLO pixel and the brightness value of the OCT pixel based on the correspondence between the pixel of the SLO image and the pixel of the OCT front image by the evaluation unit 223. including. This statistical operation includes, for example, arithmetic mean or weighted mean.

In the second example, the image processing unit 224 embeds a part of the SLO image in the OCT front image. Generally, the lateral resolution of the SLO image is higher than the lateral resolution of the OCT front image. Therefore, there may be a portion that is drawn in the SLO image but not in the OCT front image. In addition, there may be a portion that is clearly depicted in the SLO image but not clearly depicted in the OCT front image.

The processes that can be executed in this example will be described. The image processing unit 224 acquires a difference image between the SLO image and the OCT front image. The difference image may be the above-mentioned difference image created by the evaluation unit 223 or the difference image created by the image processing unit 224. The difference image is created, for example, by subtracting the pixel value of the OCT front image from the pixel value of the SLO image.

For example, the image processing unit 224 creates a new OCT front image by adding the pixel value of the difference image to the pixel value of the OCT front image. As a result, a part of the SLO image is embedded in the OCT front image.

Alternatively, the image processing unit 224 creates a new OCT front image by specifying an image area in the SLO image corresponding to the difference image and adding the pixel value of the specified image area to the pixel value of the OCT front image. To do. As a result, a part of the SLO image is embedded in the OCT front image.

In another example, the main control unit 201 causes the user interface unit 230 to display the SLO image (and the OCT front image). The user uses the user interface unit 230 to specify a desired image area in the SLO image. The image processing unit 224 identifies an image area in the OCT front image corresponding to the designated image area. Further, the image processing unit 224 replaces the image area in the OCT front image with the image area in the SLO image. As a result, a part of the SLO image is embedded in the OCT front image.

In yet another example, the main control unit 201 causes the user interface unit 230 to display an OCT front image (and an SLO image). The user uses the user interface unit 230 to specify a desired image area in the OCT front image. The image processing unit 224 identifies an image area in the SLO image corresponding to the designated image area. Further, the image processing unit 224 replaces the image area in the OCT front image with the image area in the SLO image. As a result, a part of the SLO image is embedded in the OCT front image.

A third example of the process executed by the image processing unit 224 will be described. In this example, the image processing unit 224 adjusts the pixel value of the OCT front image based on the SLO image. As described above, there may be a portion that is clearly depicted in the SLO image but not clearly depicted in the OCT front image. This example can be used to sharpen the OCT front image. Hereinafter, FIGS. 6A to 6B will be referred to.

Image G1 shown in FIG. 6A is (a part of) an OCT front image. The grid shown in the OCT front image G1 represents the arrangement of pixels. The blood vessels of the fundus Ef are depicted in the OCT frontal image G1. FIG. 6B shows the positional relationship between the blood vessel V depicted in the SLO image (not shown) and the OCT front image G1. The contours (boundaries) of blood vessels V are indicated by the symbols V1 and V2. As can be seen from the positional relationship between the OCT front image G1 and the contours V1 and V2 of the blood vessel V, the contour of the blood vessel V drawn in the OCT front image G1 is unclear.

The image processing unit 224 adjusts the pixel value of the OCT front image G1 so that the contours V1 and V2 of the blood vessel V are drawn more clearly. For example, as shown in FIG. 6C, the image processing unit 224 creates a new OCT front image G2 by reducing the luminance values of the pixels p1, p2, and p4 and increasing the luminance values of the pixels p3. .. In the OCT front image G2 after processing, the contours V1 and V2 of the blood vessel V are more clearly depicted as compared with the OCT front image G1 before processing.

<Data processing unit 220B>
The data processing unit 220B shown in FIG. 7 includes a front image creation unit 221, a registration unit 222, an evaluation unit 223, an image identification unit 225, and an image processing unit 224. The front image creation unit 221, the registration unit 222, the evaluation unit 223, and the image processing unit 224 may be configured in the same manner as those of the data processing unit 220A.

The front image creation unit 221 of this example creates a plurality of OCT front images having different depth regions from the three-dimensional images acquired by using OCT. The depth region is set, for example, automatically or manually. The different depth regions may be exclusive to each other or there may be an overlap between them.

For example, the front image creating unit 221 creates an OCT front image corresponding to the surface layer of the retina, an OCT front image corresponding to the middle layer of the retina, an OCT front image corresponding to the deep layer of the retina, and an OCT front image corresponding to the choroid. be able to.

The registration unit 222 of this example performs registration between each of the plurality of OCT front images created by the front image creation unit 221 and the SLO image.

When the depth regions drawn in the SLO image and the depth regions drawn in the OCT front image do not overlap each other, or when the depth regions are significantly different from each other, the SLO image and the OCT front image are used. Registration may not be possible. Further, when the overlap between the depth regions is small, the accuracy and accuracy of registration may be low. Such an OCT front image can be excluded from the target of subsequent processing.

The evaluation unit 223 of this example evaluates the degree of agreement between each of the plurality of OCT front images created by the front image creation unit 221 and the SLO image. As described above, any one of the plurality of OCT front images can be excluded from the evaluation process.

The image specifying unit 225 identifies the OCT front image corresponding to the maximum value among the plurality of evaluation values obtained by the evaluation unit 223 for the plurality of OCT front images.

For example, the SLO image depicts the surface layer of the retina, and the evaluation value "a1" of the OCT front image corresponding to the surface layer of the retina, the evaluation value "a2" of the front image of the OCT corresponding to the middle layer of the retina, and the deep layer of the retina. It is assumed that the evaluation value "a3" of the corresponding OCT front image and the evaluation value "a4" of the OCT front image corresponding to the choroid are obtained. In this case, the image specifying unit 225 compares these evaluation values "a1" to "a4" and specifies the maximum value among them. In this example, the evaluation value "a1" is specified as the maximum value.

The image specifying unit 225 identifies the OCT front image (in this example, the OCT front image corresponding to the surface layer of the retina) corresponding to the maximum value.

The image processing unit 224 of this example processes the OCT front image specified by the image specifying unit 225 based on the SLO image.

The image specifying unit 225 may specify two or more OCT front images. In this case, the image processing unit 224 can process each of the two or more specified OCT front images based on the SLO image. Alternatively, the image processing unit 224 can create a new OCT front image (for example, a composite image) from two or more specified OCT front images, and process the new OCT front image based on the SLO image.

In one example, the image specifying unit 225 identifies an OCT front image having the largest evaluation value (first OCT front image) and an OCT front image having the second largest evaluation value (second OCT front image). The image processing unit 224 processes the first OCT front image based on the SLO image, and processes the second OCT front image based on the SLO image. Alternatively, the image processing unit 224 creates a composite image of the first OCT front image and the second OCT front image, and processes this composite image based on the SLO image.

<Data processing unit 220C>
The data processing unit 220C shown in FIG. 8 includes a front image creating unit 221, a registration unit 222, an evaluation unit 223, a determination unit 226, and an image processing unit 224. The front image creation unit 221, the registration unit 222, the evaluation unit 223, and the image processing unit 224 may be configured in the same manner as those of the data processing unit 220A.

The determination unit 226 determines whether the evaluation value obtained by the evaluation unit 224 is equal to or greater than a predetermined threshold value. The threshold value may be either a default value, an automatically set value, or a manually set value.

When the front image creating unit 221 creates a plurality of OCT front images, the determination unit 226 can make a determination for each evaluation value of these OCT front images.

When the determination unit 226 determines that the evaluation value is equal to or higher than the threshold value, the image processing unit 224 processes the OCT front image based on the SLO image. On the other hand, when the determination unit 226 determines that the evaluation value is less than the threshold value, the image processing unit 224 does not process the OCT front image.

<Data processing unit 220D>
The data processing unit 220D shown in FIG. 9 includes a front image creating unit 221, a registration unit 222, an evaluation unit 223, a selection unit 227, and an image processing unit 224. The front image creation unit 221, the registration unit 222, the evaluation unit 223, and the image processing unit 224 may be configured in the same manner as those of the data processing unit 220A.

The front image creation unit 221 of this example creates a plurality of OCT front images having different depth regions from the three-dimensional images acquired by using OCT.

The evaluation unit 223 of this example evaluates the degree of agreement between each of the plurality of OCT front images created by the front image creation unit 221 and the SLO image. As described above, any one of the plurality of OCT front images can be excluded from the evaluation process.

The selection unit 227 selects one of the OCT front images based on the plurality of evaluation values obtained by the evaluation unit 223 for the plurality of OCT front images. This selection may be, for example, comparison of the magnitude of the evaluation value, comparison with the threshold value, or the like.

The image processing unit 224 processes each OCT front image selected by the selection unit 227 based on the SLO image.

<Data processing unit 220E>
The data processing unit 220E shown in FIG. 10 includes a registration unit 228, a front image creation unit 221, an evaluation unit 223, and an image processing unit 224. The front image creation unit 221, the evaluation unit 223, and the image processing unit 224 may be configured in the same manner as those of the data processing unit 220A. Further, the registration unit 228 may be configured in the same manner as the registration unit 222 of the data processing unit 220A.

In another example, the OCT front image created from the 3D image and the SLO image are registered, whereas in this example, the 3D image acquired by using the OCT and the SLO image are registered. ..

The timing of registration between the 3D image and the SLO image and the timing of creating the OCT front image from the 3D image are arbitrary. For example, an OCT frontal image can be created after registration. Alternatively, registration can be performed after the OCT front image is created. Alternatively, registration and creation of an OCT front image can be performed in parallel.

The registration unit 228 aligns the three-dimensional image acquired by using OCT and the SLO image. The alignment between the 3D image and the SLO image includes, for example, a process of creating a shadow gram (or projection image) from the 3D image, a process of aligning the created shadow gram and the SLO image, and the process of aligning the created shadow gram and the SLO image. It includes a process of reflecting the result of alignment on a three-dimensional image.

The alignment between the 3D image and the SLO image is, for example, a process of detecting a feature area from the SLO image and a process of detecting a feature area in the 3D image, and the SLO image and 3 based on both feature areas. It includes a process of aligning with a three-dimensional image.

The evaluation unit 223 of this example has a degree of agreement between the OCT front image created from the three-dimensional image by the front image creation unit 221 and the SLO image based on the result of the alignment executed by the registration unit 228. To evaluate.

For example, the evaluation unit 223 aligns the OCT front image and the SLO image using the result of the alignment executed by the registration unit 228, and between the aligned OCT front image and the SLO image. Evaluate the degree of agreement.

For example, when the evaluation value obtained by the evaluation unit 223 is equal to or greater than the threshold value, the image processing unit 224 processes the OCT front image based on the SLO image. In another example, the image processing unit 224 processes the OCT front image selected by comparing a plurality of evaluation values obtained by the evaluation unit 223 based on the SLO image.

<Data processing unit 220F>
For example, when the ophthalmologic imaging apparatus can acquire two or more SLO images having different wavelength bands, as in the case where the optical system shown in FIG. 11 is applied, the data processing unit 220F shown in FIG. 12 can be applied. ..

First, the optical system shown in FIG. 11 will be described. When this optical system is applied, the ophthalmologic imaging apparatus can output light beams in a plurality of wavelength bands.

The optical system shown in FIG. 11 represents an SLO light source unit 131A applied in place of the SLO light source 131 and the collimating lens 132 of FIGS. 1 and 3. Other parts may be the same as in the above embodiment.

The SLO light source unit 131A can output both infrared light and visible light. The SLO light source unit 131A includes an infrared light source 131a, a red light source 131r, a green light source 131g, and a blue light source 131b. The infrared light source 131a outputs a light beam in the (near) infrared band. The red light source 131r outputs a light beam in the red band. The green light source 131 g outputs a light beam in the green band. The blue light source 131b outputs a light beam in the blue band. Each of these light sources 131a, 131r, 131g and 131b is, for example, a semiconductor laser.

The infrared light output from the infrared light source 131a is converted into a parallel luminous flux by the collimating lens 132a. The red light output from the red light source 131r is converted into a parallel luminous flux by the collimating lens 132r. The green light output from the green light source 131 g is converted into a parallel luminous flux by the collimating lens 132 g. The blue light output from the blue light source 131b is converted into a parallel luminous flux by the collimating lens 132b.

The beam splitter BSr synthesizes red light converted into a parallel luminous flux by the collimating lens 132r into an optical path of infrared light converted into a parallel luminous flux by the collimating lens 132a. The beam splitter BSg synthesizes green light converted into a parallel luminous flux by the collimating lens 132g into an optical path of infrared light converted into a parallel luminous flux by the collimating lens 132a. The beam splitter BSb synthesizes blue light converted into a parallel luminous flux by the collimating lens 132b into an optical path of infrared light converted into a parallel luminous flux by the collimating lens 132a. The three beam splitters BSr, BSg and BSb synthesize the optical paths of four light beams having different wavelength bands. This combined optical path is led to the beam splitter BS2.

The main control unit 201 controls each of the infrared light source 131a, the red light source 131r, the green light source 131g, and the blue light source 131b. The main control unit 201 may include a synchronization circuit for synchronously controlling the light sources 131a, 131r, 131g and 131b. This synchronization circuit may be configured to synchronize the control of the light sources 131a, 131r, 131g and 131b with the control of the optical scanner 136.

The data processing unit 220F shown in FIG. 12 can evaluate the degree of coincidence between each of two or more SLO images having different wavelength bands and the OCT front image. When a plurality of OCT front images are created from a three-dimensional image, the data processing unit 220F selects an OCT front image corresponding to each of two or more SLO images having different wavelength bands from the plurality of OCT front images. Can be done.

The data processing unit 220F includes a front image creating unit 221, a registration unit 222, an evaluation unit 223, a selection unit 229, and an image processing unit 224. The front image creation unit 221, the registration unit 222, the evaluation unit 223, and the image processing unit 224 may be configured in the same manner as those of the data processing unit 220A.

The SLO optical system 130 of this example acquires two or more SLO images corresponding to different wavelength bands. The OCT optical system 140 acquires a three-dimensional image in the same manner as in the above embodiment. These images are input to the data processing unit 220F.

The front image creation unit 221 creates a plurality of OCT front images having different depth regions from the three-dimensional images acquired by using OCT.

The registration unit 222 executes registration for any combination of two or more SLO images and a plurality of OCT front images. For example, the registration unit 222 aligns each of the two or more SLO images with each of the plurality of OCT front images. When the depth region drawn by the SLO image and the depth region drawn by the OCT front image can be associated, the registration unit 222 corresponds to the same depth region (or similar depth region). It may be configured to align the SLO image and the OCT front image. Images that have not been registered can be excluded from subsequent processing.

The evaluation unit 223 evaluates the degree of agreement between each of the two or more SLO images and at least a part of the plurality of OCT front images. For example, for each of the two or more SLO images, the evaluation unit 223 evaluates the degree of agreement between the SLO image and each OCT front image. In another example, the evaluation unit 223 calculates an evaluation value for each combination of the registered SLO image and the OCT front image.

The selection unit 229 selects the OCT front image corresponding to each of the two or more SLO images based on the evaluation value obtained by the evaluation unit 223. For example, for each of the two or more SLO images, the selection unit 229 specifies the maximum value among the evaluation values (1 or more) calculated based on the SLO image, and sets the OCT front image corresponding to this maximum value. It is selected as the OCT image corresponding to the SLO image.

In another example, for each of the two or more SLO images, the selection unit 229 compares the evaluation value (1 or more) calculated based on the SLO image with a predetermined threshold value, and OCT corresponding to the evaluation value equal to or more than the threshold value. The front image is selected as the OCT image corresponding to the SLO image.

The selection unit 229 may select two or more OCT front images for one SLO image. In this case, these OCT front images can be ranked based on, for example, the magnitude of the evaluation value.

The image processing unit 224 processes the OCT front image selected by the selection unit 227 based on the corresponding SLO image.

<User interface unit 230>
The user interface (UI) unit 230 has a function for exchanging information between the user and the ophthalmologic imaging apparatus. The UI unit 230 includes a display device and an operation device (input device). Display devices include, for example, a liquid crystal display (LCD). Operating devices include various hardware keys and / or software keys.

The control unit 200 receives the operation content for the operation device and outputs a control signal corresponding to the operation content to each unit. It is possible to integrally configure at least a part of the operating device and at least a part of the display device. The touch panel display is an example.

<Operation>
The operation of the ophthalmologic imaging apparatus according to the embodiment will be described. A typical example of the operation is shown in FIG.

(Execute S1: 3D-OCT)
First, preparatory processing such as alignment of the ophthalmologic imaging device with respect to the eye E to be inspected, focus adjustment with respect to the fundus Ef, and optimization of OCT imaging conditions is executed. These are performed by known techniques. In addition, the projection of the fixation light onto the eye E to be inspected is started.

After the preparatory processing is completed, the main control unit 201 controls the OCT optical system 140, the OCT image forming unit 212, and the like in order to perform a three-dimensional OCT scan of the fundus Ef. As a result, a three-dimensional image of the fundus Ef can be obtained. This three-dimensional image is, for example, volume data.

(S2: Execute multi-color SLO)
The ophthalmologic imaging apparatus of this example shall include the optical system shown in FIG. The main control unit 201 controls the SLO optical system 130, the SLO image forming unit 211, and the like in order to perform multicolor SLO imaging of the fundus Ef.

As a result, an infrared SLO image based on the infrared light from the infrared light source 131a, a red SLO image based on the red light from the red light source 131r, a green SLO image based on the green light from the green light source 131g, and a blue light source. A blue SLO image based on the blue light from 131b is obtained.

In this example, since SLO imaging is performed with visible light, SLO imaging is performed after OCT in consideration of the influence of the miosis of the eye E to be inspected. However, it is also possible to perform SLO imaging before OCT, or to perform OCT and SLO imaging in parallel.

(S3: Save the image)
The main control unit 201 stores the three-dimensional image acquired in step S1 and the infrared SLO image, red SLO image, green SLO image, and blue SLO image acquired in step S2 in the storage unit 202 (or other). Store in the storage device). At this time, information such as subject identification information (patient ID and the like), serial number, shooting date and time, type of acquired image, and the like are attached to the image.

(S4: Image selection)
For example, in response to an instruction from the user, the main control unit 201 causes the user interface unit 230 to display a predetermined screen. A list of subjects is presented on this screen. The user uses the user interface unit 230 to select a desired subject from this list of subjects.

The main control unit 201 displays a list of shooting information (serial number, shooting date / time, image type, etc.) regarding the selected subject. The user selects a desired serial number from this shooting information list.

The main control unit 201 acquires an image corresponding to the selected serial number from the storage unit 202 (or another storage device). For example, in response to an instruction from the user, the main control unit 201 causes the user interface unit 230 to display the acquired image.

In this example, the three-dimensional image acquired in step S1 and the infrared SLO image, red SLO image, green SLO image, and blue SLO image acquired in step S2 are stored in the storage unit 202 (or another storage unit 202). Obtained from the device). The main control unit 201 can display the infrared SLO image, the red SLO image, the green SLO image, and the blue SLO image selectively or in parallel. Further, a rendered image of a three-dimensional image may be displayed.

(S5: Designation of SLO image as reference image)
The user uses the user interface unit 230 to specify a reference image from the SLO images selected in step S4. In this example, it is assumed that an infrared SLO image, a red SLO image, a green SLO image, and a blue SLO image are specified.

(S6: Preprocessing is applied to SLO images)
The preprocessing unit 2221 applies a predetermined preprocessing to each of the infrared SLO image, the red SLO image, the green SLO image, and the blue SLO image.

(S7: Apply edge detection to SLO image)
The feature point detection unit 2222 applies edge detection to each of the infrared SLO image, the red SLO image, the green SLO image, and the blue SLO image to which the preprocessing was applied in step S6. As a result, the feature point group in the infrared SLO image, the feature point group in the red SLO image, the feature point group in the green SLO image, and the feature point group in the blue SLO image are acquired, respectively.

(S8: Apply segmentation to 3D-OCT images)
The segmentation unit 2211 analyzes the three-dimensional image selected in step S4 to identify a plurality of segments. In this example, for example, one or more segments contained in the surface layer of the retina, one or more segments contained in the middle layer of the retina, one or more segments contained in the deep layer of the retina, and one or more segments contained in the choroid are specified. Suppose it was done.

(S9: Create OCT front image)
The projection processing unit 2212 creates a shadow gram by projecting each segment identified in step S8 in the Z direction. In this example, one or more OCT frontal images corresponding to the surface layer of the retina, one or more OCT frontal images corresponding to the middle layer of the retina, one or more OCT frontal images corresponding to the deep layer of the retina, and one or more OCT frontal images corresponding to the choroid. An OCT front image is created.

(S10: Preprocessing is applied to the OCT front image)
The preprocessing unit 2221 applies a predetermined preprocessing to each of the OCT front images created in step S9.

(S11: Apply edge detection to OCT front image)
The feature point detection unit 2222 applies edge detection to each of the OCT front images to which the preprocessing has been applied in step S10. As a result, the feature point group in each OCT front image is acquired.

(S12: Registration of SLO image and OCT front image is executed)
The alignment unit 2223 sets a combination of one or more OCT front images for each reference image (SLO image).

At this time, the alignment unit 2223 can combine all of the considered OCT front images with each reference image, for example. In this example, the alignment unit 2223 associates each of the plurality of OCT front images created in step 9 with the infrared SLO image, and corresponds to each of the plurality of OCT front images with respect to the red SLO image. Attached, each of the plurality of OCT front images can be associated with the green SLO image, and each of the plurality of OCT front images can be associated with the blue SLO image. Assuming that the number of OCT front images created in step S9 is "N", the total number of combinations in this case is "4" for the number of SLO images selected in step S5 and "N" for the number of OCT front images. It becomes "4 × N" which is the product of.

Another example will be described. When the wavelength band of SLO and the depth region of OCT are associated in advance, or when the user associates the wavelength band of SLO with the depth region of OCT, the alignment unit 2223 is used. One or more OCT front images in a depth region corresponding to the wavelength band of the reference image can be associated with one reference image (SLO image). In this example, the infrared wavelength band is associated with the choroid, the red wavelength band is associated with the deep retinal layer, the green wavelength band is associated with the middle layer of the retina, and the blue wavelength band is associated with the surface layer of the retina. Is associated with. In this case, the alignment unit 2223 associates each of one or more OCT front images corresponding to the choroid with the infrared SLO image, and one or more OCT front images corresponding to the deep retina with respect to the red SLO image. Each is associated, the green SLO image is associated with each of one or more OCT front images corresponding to the middle layer of the retina, and the blue SLO image is associated with each of one or more OCT front images corresponding to the surface layer of the retina. Can be done. The number of OCT front images corresponding to the choroid is "Na", the number of OCT front images corresponding to the deep retina is "Nd", the number of OCT front images corresponding to the middle layer of the retina is "Nm", and the number of OCT corresponding to the surface layer of the retina is "Nm". Assuming that the number of front images is "Ns", the total number of combinations in this case is "Na + Nd + Nm + Ns" which is the sum of these.

For each of these combinations, the alignment unit 2223 is based on the feature point cloud in the SLO image detected in step S7 and the feature point cloud in the OCT front image detected in step 11. The SLO image and the OCT front image are aligned.

(S13: Evaluate the degree of agreement)
For each of the combinations registered in step S12, the evaluation unit 223 evaluates the degree of agreement between the SLO image and the OCT front image. As a result, for each reference image (SLO image), one or more evaluation values corresponding to one or more OCT front images can be obtained.

(S14: Select the OCT front image corresponding to the SLO image)
For each reference image (SLO image), the data processing unit 220 selects an OCT front image corresponding to the SLO image based on one or more evaluation values acquired in step S13. This process is executed by using, for example, the image identification unit 225, the determination unit 226, the selection unit 227, or the selection unit 229.

In this example, for example, the OCT front image corresponding to the choroid is selected for the infrared SLO image, the OCT front image corresponding to the deep layer of the retina is selected for the red SLO image, and the OCT front image corresponding to the middle layer of the retina is selected for the green SLO image. The OCT front image is selected, and for the blue SLO image, the OCT front image corresponding to the surface layer of the retina is selected.

(S15: OCT front image processed)
For each reference image (SLO image), the image processing unit 224 processes the OCT front image selected in step S14 based on the SLO image.

The main control unit 201 can display the processed OCT front image on the user interface unit 230. At this time, the OCT front image before processing and the corresponding SLO image can be displayed together with the OCT front image after processing. The processed OCT front image is used for observation, diagnosis, analysis, and the like.

When SLO images of two or more visible components (red component, green component, blue component) are obtained as in this example, two or more OCT images corresponding to the SLO images of two or more visible components are combined. Can be done.

For example, when a "red" OCT front image corresponding to a red SLO image, a "green" OCT front image corresponding to a green SLO image, and a "blue" OCT front image corresponding to a blue SLO image are obtained, these By synthesizing the three OCT front images, it is possible to create a "color" OCT front image corresponding to a color SLO image which is a composite image of the red SLO image, the green SLO image, and the blue SLO image. Further, a "red-free" OCT front image can be created by synthesizing the "green" OCT front image and the "blue" OCT front image.

<Action / effect>
Some examples of actions and effects that can be provided by the ophthalmologic imaging apparatus according to the embodiment will be described below.

The ophthalmologic imaging apparatus according to the embodiment includes a front image acquisition unit, a three-dimensional image acquisition unit, a front image creation unit, and an evaluation unit.

The front image acquisition unit is configured to take an image of the eye to be inspected and acquire a first front image. In the above embodiment, the front image acquisition unit includes the objective lens system 110, the SLO optical system 130, and the SLO image forming unit 211, and scans the fundus Ef of the eye E to be inspected with SLO light to form an SLO image. In other embodiments, the frontal image acquisition unit may be of any modality, such as a fundus camera, slit lamp microscope, surgical microscope, and the like.

The three-dimensional image acquisition unit is configured to scan the eye to be inspected and acquire a three-dimensional image. In the above embodiment, the three-dimensional image acquisition unit includes the objective lens system 110, the OCT optical system 140, and the OCT image forming unit 212, and scans with the OCT light (measurement light LS) of the fundus Ef of the eye E to be inspected. Form a three-dimensional image. In other embodiments, the 3D image acquisition unit may be of any modality, such as an ultrasonic diagnostic device.

The front image creating unit is configured to create a second front image from the three-dimensional image acquired by the three-dimensional image acquisition unit. In the above embodiment, the front image creating unit includes the front image creating unit 221 and forms the OCT front image from the three-dimensional image acquired by using OCT.

The evaluation unit is configured to evaluate the degree of agreement between the first front image acquired by the front image acquisition unit and the second front image created by the front image creation unit. In the above embodiment, the evaluation unit includes the evaluation unit 223 and evaluates the degree of agreement between the SLO image and the OCT front image.

According to such an embodiment, between a first front image having a relatively high lateral resolution (eg, an SLO image) and a second front image having a relatively low lateral resolution (eg, an OCT front image). The degree of agreement can be evaluated. As a result, it is possible to grasp the relationship between the first front image and the second front image, to associate them with each other, and to perform image processing and analysis processing using both of them. ..

In an embodiment, the ophthalmologic imaging apparatus may further include an image processing unit. The image processing unit is configured to process the second front image based on the first front image when the evaluation value of the degree of agreement obtained by the evaluation unit satisfies a predetermined condition. In the above embodiment, the image processing unit includes the image processing unit 224, and processes the OCT front image based on the SLO image when the evaluation value of the degree of matching obtained by the evaluation unit 223 satisfies a predetermined condition. ..

In the embodiment, the image processing unit is configured to perform, for example, any of the following processes. In the first example, the image processing unit creates a composite image of the first front image and the second front image. In the second example, the image processing unit embeds a part of the first front image in the second front image. In the third example, the image processing unit changes the pixel value of the second front image based on the first front image.

According to such an embodiment, a first front image (eg, SLO image) having a relatively high lateral resolution is used to obtain a second front image (eg, an OCT front image) having a relatively low lateral resolution. By processing, a new second front image can be acquired. The second front image after processing is an image in which the eye to be inspected is depicted in more detail than the second front image before processing.

In the embodiment, the front image creating unit can create a plurality of second front images having different depth regions from the three-dimensional image acquired by the three-dimensional image acquisition unit. In this case, the evaluation unit can evaluate the degree of agreement between each of the plurality of second front images created by the front image creation unit and the first front image acquired by the front image acquisition unit.

Further, the ophthalmologic imaging apparatus of this embodiment may further include an image identification unit. The image specifying unit identifies the second front image corresponding to the maximum value among the plurality of evaluation values obtained by the evaluation unit for the plurality of second front images created by the front image creating unit. In the above embodiment, the image identification unit includes the image identification unit 225, and identifies the OCT front image corresponding to the maximum value among the plurality of evaluation values obtained by the evaluation unit 223 for the plurality of OCT front images. The image specifying unit functions to specify a second front image such that the evaluation value of the degree of agreement satisfies a predetermined condition.

The image processing unit can process the second front image specified by the image specifying unit based on the first front image.

According to such an embodiment, the second front image having a high degree of coincidence with the first front image is specified, and further, the first front image (eg, SLO image) having a relatively high lateral resolution is used. It is possible to process a second front image (eg, OCT front image) having a relatively low lateral resolution.

In an embodiment, the ophthalmologic imaging apparatus may further include a determination unit. The determination unit determines whether the evaluation value obtained by the evaluation unit is equal to or greater than a predetermined threshold value. In the above embodiment, the determination unit includes the determination unit 226 and determines whether the evaluation value obtained by the evaluation unit 223 is equal to or greater than a predetermined threshold value. The determination unit functions to determine whether or not the evaluation value of the degree of agreement satisfies a predetermined condition.

When the determination unit determines that the evaluation value is equal to or higher than a predetermined threshold value, the image processing unit can process the second front image based on the first front image.

According to such an embodiment, the second front image having a high degree of coincidence with the first front image is specified, and further, the first front image (eg, SLO image) having a relatively high lateral resolution is used. It is possible to process a second front image (eg, OCT front image) having a relatively low lateral resolution.

In the embodiment, the front image creating unit can create a plurality of second front images having different depth regions from the three-dimensional image acquired by the three-dimensional image acquisition unit. In this case, the evaluation unit can evaluate the degree of agreement between each of the plurality of second front images created by the front image creation unit and the first front image acquired by the front image acquisition unit.

According to such an embodiment, it is possible to grasp the degree of agreement between each of the plurality of second front images representing various depth regions and the first front image.

Further, the ophthalmologic imaging apparatus according to the embodiment may include a first selection unit. The first selection unit is one of a plurality of second front images based on a plurality of evaluation values obtained by the evaluation unit for the plurality of second front images created from the three-dimensional images by the front image creation unit. Is configured to select. In the above embodiment, the first selection unit includes the selection unit 227, and a plurality of OCT front images created by the front image creation unit 221 based on a plurality of evaluation values obtained by the evaluation unit 223. Select one of the OCT front images.

According to such an embodiment, it is possible to automatically select a second front image having a high degree of agreement with the first front image from a plurality of second front images representing various depth regions. it can.

In the embodiment, the front image creating unit may include a segmentation unit and a projection processing unit.

The segmentation unit performs segmentation on the three-dimensional image acquired by the three-dimensional image acquisition unit to specify the layer region. In the above embodiment, the segmentation section includes the segmentation section 2211, and the three-dimensional image formed by the OCT image forming section 212 is segmented to specify the layer region of the fundus Ef.

The projection processing unit creates a second front image by projecting the layer region specified by the segmentation unit in a predetermined direction. In the above embodiment, the projection processing unit includes the projection processing unit 2212 and creates an OCT front image by projecting the layer region specified by the segmentation unit 2211 in a predetermined direction (eg, Z direction).

According to such an embodiment, it is possible to obtain a second front image representing a desired layer region (desired structure).

In embodiments, the ophthalmologic imaging apparatus may further include a first registration section. The first registration unit aligns between the first front image and the second front image. In the above embodiment, the first registration unit includes the registration unit 222, the SLO image formed by the SLO image forming unit 211, and the OCT front image created from the three-dimensional image by the front image creating unit 221. Align between.

The evaluation unit can evaluate the degree of agreement between the first front image and the second front image aligned by the first registration unit.

According to such an embodiment, the degree of coincidence between the aligned first front image and the second front image can be evaluated, so that the positional deviation between the first front image and the second front image can be evaluated. Can reduce the effect of on the evaluation of the degree of agreement. Thereby, the degree of agreement between the tissue depicted in the first front image and the tissue depicted in the second front image can be evaluated with higher accuracy and higher accuracy.

In the embodiment, the first registration unit may include a feature point detection unit. The feature point detection unit analyzes the first front image to detect the first feature point group, and analyzes the second front image to detect the second feature point group. In the above embodiment, the feature point detection unit includes the feature point detection unit 2222, analyzes the SLO image to detect the first feature point group, and analyzes the OCT front image to obtain the second feature point group. To detect.

Further, the first registration unit aligns the first front image and the second front image based on the first feature point group and the second feature point group detected by the feature point detection unit. Can be done.

According to such an embodiment, by using the feature points in the image, the positioning of the first front image and the second front image can be performed with high accuracy and high accuracy.

In the embodiment, the first registration unit may include a pretreatment unit. The preprocessing unit has at least one of a gamma curve correction, a sigmoid curve correction, a luminance correction, a Gaussian filter, and a median filter on at least one of the first front image and the second front image input to the feature point detection unit. Perform pretreatment including. In the above embodiment, the preprocessing unit includes the preprocessing unit 2221, and the Gamma curve correction, the sigmoid curve correction, the brightness correction, and the Gaussian are applied to at least one of the SLO image and the OCT front image input to the feature point detection unit 2222. Pretreatment is performed including at least one of a filter and a median filter.

The feature point detection unit can perform edge detection on the first front image to detect the first feature point group, and perform edge detection on the second front image to detect the second feature point group.

According to such an embodiment, feature points can be detected through edge detection and pretreatment for preferably performing this. As a result, the alignment between the first front image and the second front image can be performed with higher accuracy and higher accuracy.

In embodiments, the ophthalmologic imaging apparatus may further include a second registration section. The second registration unit is configured to align the first front image acquired by the front image acquisition unit with the three-dimensional image acquired by the three-dimensional image acquisition unit. In the above embodiment, the second registration unit includes the registration unit 228, and is between the first front image formed by the SLO image forming unit 211 and the three-dimensional image formed by the OCT image forming unit 212. Align.

The evaluation unit can evaluate the degree of agreement between the first front image and the second front image based on the result of the alignment performed by the second registration unit.

According to such an embodiment, the position of the first front image and the second front image created from the three-dimensional image is used by utilizing the result of the alignment between the first front image and the three-dimensional image. You can make adjustments. Thereby, the degree of matching between the aligned first front image and the second front image can be evaluated. Therefore, it is possible to reduce the influence of the positional deviation between the first front image and the second front image on the evaluation of the degree of agreement. Therefore, the degree of coincidence between the tissue depicted in the first front image and the tissue depicted in the second front image can be evaluated with higher accuracy and higher accuracy.

In the embodiment, the front image acquisition unit can acquire two or more first front images corresponding to different wavelength bands. In this case, the evaluation unit can evaluate the degree of agreement between each of the two or more first front images and the second front image.

According to such an embodiment, when the first front image corresponding to various wavelength bands is acquired, the degree of matching with the second front image can be evaluated for each of the first front images. it can. Thereby, the degree of agreement between the first front image (eg, SLO image) corresponding to various wavelength bands and the second front image (eg, OCT front image) can be evaluated. As a result, it is possible to grasp the relationship between the first front image and the second front image, to associate them with each other, and to perform image processing and analysis processing using both of them. ..

In the embodiment, the front image acquisition unit can acquire two or more first front images corresponding to different wavelength bands. In this case, the front image creating unit can create a plurality of second front images having different depth regions from the three-dimensional image acquired by the three-dimensional image acquisition unit. Further, the evaluation unit determines the degree of agreement between each of the two or more first front images corresponding to different wavelength bands with at least a part of the plurality of second front images created by the front image creation unit. Can be evaluated.

In this case, the ophthalmologic imaging apparatus may include a second selection unit. The second selection unit is configured to select a second front image corresponding to each of the two or more first front images corresponding to different wavelength bands based on the evaluation value obtained by the evaluation unit. In the above embodiment, the second selection unit includes the selection unit 229, and the OCT front image corresponding to each of the two or more SLO images corresponding to different wavelength bands based on the evaluation value obtained by the evaluation unit 223. Select.

According to such an embodiment, when the first front image corresponding to various wavelength bands is acquired, for each of these first front images, the depth region corresponding to the wavelength band of the first front image is obtained. The second front image of the above can be specified. Thereby, it is possible to perform diagnosis, image processing, and analysis processing using the first front image (example: SLO image) and the second front image (example: OCT front image) that depict the same tissue. It becomes.

<Other Embodiments / Modifications>
The embodiments shown above are merely examples for carrying out the present invention. A person who intends to carry out the present invention can make arbitrary modifications, omissions, additions, etc. within the scope of the gist of the present invention. Hereinafter, other embodiments and modifications will be described. It should be noted that any configuration or function included in the above embodiment can be combined with the examples described below.

The ophthalmologic imaging apparatus according to the above embodiment has both a front image acquisition unit that photographs the eye to be inspected and acquires a first front image, and a three-dimensional image acquisition unit that scans the eye to be inspected and acquires a three-dimensional image. Includes. However, the embodiment is not limited to this.

For example, the ophthalmologic imaging apparatus according to another embodiment includes a front image acquisition unit, a three-dimensional image reception unit, a front image creation unit, and an evaluation unit.

The front image creating unit is configured to take an image of the eye to be inspected and acquire a first front image, as in the above embodiment.

Unlike the 3D image acquisition unit of the above embodiment, the 3D image receiving unit is configured to receive a 3D image of the eye to be inspected from the outside. A typical example of the 3D image receiving unit is a communication interface for receiving a 3D image via a communication line, or an interface for reading a 3D image recorded on a recording medium.

The front image creating unit is configured to create a second front image from the three-dimensional image received by the three-dimensional image receiving unit. The front image creating unit may be configured in the same manner as in the above embodiment.

The evaluation unit is configured to evaluate the degree of agreement between the first front image and the second front image, as in the above embodiment.

The ophthalmologic imaging apparatus according to such an embodiment typically removes the OCT optical system 140 and the OCT image forming unit 212 from the components of the above embodiment, and has a three-dimensional image receiving unit (the above-mentioned). An interface, etc.) is added. The ophthalmologic imaging apparatus according to this embodiment is, for example, an ophthalmologic imaging apparatus that does not have an OCT function or an ultrasonic imaging function, and is typically a modality such as an SLO, a fundus camera, a slit lamp microscope, or a surgical microscope. ..

According to such an embodiment, it is possible to evaluate the degree of matching between the first front image having a relatively high lateral resolution and the second front image having a relatively low lateral resolution. As a result, it is possible to grasp the relationship between the first front image and the second front image, to associate them with each other, and to perform image processing and analysis processing using both of them. ..

The ophthalmologic imaging apparatus according to still another embodiment includes a front image receiving unit, a three-dimensional image acquisition unit, a front image creating unit, and an evaluation unit.

Unlike the front image acquisition unit of the above embodiment, the front image receiving unit is configured to receive the first front image of the eye to be inspected from the outside. A typical example of the front image receiving unit is a communication interface for receiving a three-dimensional image via a communication line, or an interface for reading a three-dimensional image recorded on a recording medium.

The three-dimensional image creating unit is configured to scan the eye to be inspected and acquire a three-dimensional image, as in the above embodiment.

The front image creating unit is configured to create a second front image from the three-dimensional image acquired by the three-dimensional image acquisition unit, as in the above embodiment.

The evaluation unit is configured to evaluate the degree of agreement between the first front image and the second front image, as in the above embodiment.

The ophthalmologic imaging apparatus according to such an embodiment typically removes the SLO optical system 130 and the SLO image forming unit 211 from the components of the above embodiment, and also removes the front image receiving unit (the above interface). Etc.) are added. The ophthalmologic imaging apparatus according to this embodiment is, for example, an ophthalmologic imaging apparatus having no function as an SLO, a fundus camera, a slit lamp microscope, a surgical microscope, or the like, and is typically a light capable of performing a three-dimensional scan. It is an interference tomography.

According to such an embodiment, it is possible to evaluate the degree of matching between the first front image having a relatively high lateral resolution and the second front image having a relatively low lateral resolution. As a result, it is possible to grasp the relationship between the first front image and the second front image, to associate them with each other, and to perform image processing and analysis processing using both of them. ..

The embodiment is not limited to the ophthalmologic imaging apparatus, and may be an information processing apparatus (ophthalmic image processing apparatus) that does not include both the front image acquisition unit and the three-dimensional image acquisition unit. The ophthalmic image processing apparatus according to the embodiment includes an image receiving unit, a front image creating unit, and an evaluation unit.

The image receiving unit receives the first front image and the three-dimensional image of the eye to be inspected. A typical example of an image receiving unit is a communication interface for receiving a first front image and a three-dimensional image via a communication line, or for reading a first front image and a three-dimensional image recorded on a recording medium. Interface.

The front image creating unit is configured to create a second front image from the three-dimensional image, as in the above embodiment.

The evaluation unit is configured to evaluate the degree of agreement between the first front image and the second front image, as in the above embodiment.

The ophthalmologic image processing apparatus according to such an embodiment typically removes the optical system 100, the optical system moving mechanism 100A, and the image forming unit 210 from the components of the above-described embodiment, and also removes the image receiving unit 210. (The above interface, etc.) is added. The ophthalmic image processing device according to this embodiment may be, for example, an information processing device (server, computer terminal, etc.) installed in a medical institution, a cloud server, a mobile computer, or the like.

According to such an embodiment, it is possible to evaluate the degree of matching between the first front image having a relatively high lateral resolution and the second front image having a relatively low lateral resolution. As a result, it is possible to grasp the relationship between the first front image and the second front image, to associate them with each other, and to perform image processing and analysis processing using both of them. ..

100 Optical system 130 SLO optical system 140 OCT optical system 211 SLO image forming unit 212 OCT image forming unit 220 Data processing unit 221 Front image creating unit 2211 Segmentation unit 2212 Projection processing unit 222 Registration unit 2221 Preprocessing unit 2222 Feature point detection unit 2223 Alignment unit 223 Evaluation unit 224 Image processing unit 225 Image identification unit 226 Judgment unit 227 Selection unit 228 Registration unit 229 Selection unit

Claims (2)

A front image acquisition unit that captures the eye to be inspected and acquires the first front image,
A three-dimensional image acquisition unit that scans the eye to be inspected and acquires a three-dimensional image,
A front image creation unit that creates a second front image from the three-dimensional image,
An evaluation unit that evaluates the degree of agreement between the first front image and the second front image ,
When the evaluation value of the degree of matching calculated by the evaluation unit satisfies a predetermined condition, see contains an image processing unit for processing the second front image based on the first front image,
The image processing unit embeds a part of the first front image in the second front image.
An ophthalmologic imaging device characterized by this. A front image acquisition unit that captures the eye to be inspected and acquires the first front image,
A three-dimensional image acquisition unit that scans the eye to be inspected and acquires a three-dimensional image,
A front image creation unit that creates a second front image from the three-dimensional image,
An evaluation unit that evaluates the degree of agreement between the first front image and the second front image ,
When the evaluation value of the degree of matching calculated by the evaluation unit satisfies a predetermined condition, see contains an image processing unit for processing the second front image based on the first front image,
The image processing unit changes the pixel value of the second front image based on the first front image.
An ophthalmologic imaging device characterized by this.

Images