JP2022074224A - Ophthalmologic information processing device, ophthalmologic device, ophthalmologic information processing method and program - Google Patents

Abstract

To provide a new technique for accurately detecting an optic disk hemorrhagic area with high reproductivity.SOLUTION: An ophthalmologic information processing device comprises: a determiner and a detector. The determiner uses an optic disk hemorrhagic determination model which is obtained by machine learning with first teacher data being front images of a plurality of eyegrounds each attached with a label indicating presence/absence of optic disk hemorrhagic, to determine presence/absence of optic disk hemorrhagic with respect to a front image of the eyeground of an eye to be examined. The detector uses an optic disk hemorrhagic area detection model which is obtained by machine learning with second teacher data being a plurality of pairs of image groups, each pair of which is formed of the front image of the eyeground and an optic disk hemorrhagic area image indicating an optic disk hemorrhagic area drawn in the front image, to detect an optic disk hemorrhagic area drawn in the front image determined to include optic disk hemorrhagic by the determiner.SELECTED DRAWING: Figure 5

Description

The present invention relates to an ophthalmic information processing apparatus, an ophthalmic apparatus, an ophthalmic information processing method, and a program.

Glaucoma is a leading cause of blindness. Optic nerve disorders and visual field disorders caused by glaucoma are basically progressive and irreversible, and gradually progress without the patient's awareness. Therefore, early detection of glaucoma and prevention or suppression of the progression of the disorder by early treatment are very important.

One of the risk factors for the progression of such glaucoma is papillary bleeding. For example, a doctor observes a fundus image acquired using a fundus camera to determine the presence or absence of papillary bleeding. Such doctors are required to utilize sufficient experience and knowledge to accurately determine the presence or absence of papillary bleeding.

For example, Patent Document 1 discloses a method for extracting a fundus blood vessel, a bleeding portion, and an optic disc region by utilizing a difference in pixel values in a color fundus image.

Japanese Unexamined Patent Publication No. 2008-73280

In the fundus image, the overall color tone, the site, and the pixel value of the blood vessel differ depending on the eye to be inspected. Therefore, it is considered that it is often difficult to extract a bleeding portion or the like with high accuracy even if only the difference in pixel values is used as in the conventional method. Moreover, it will be even more difficult to pinpoint the location of the bleeding site with high accuracy, such as whether or not the location of the bleeding site is within the optic disc region. If the bleeding part cannot be detected with high accuracy, the doctor cannot determine the presence or absence of papillary bleeding with high reproducibility and high accuracy.

The present invention has been made in view of such circumstances, and one of the objects thereof is to provide a new technique for detecting a papillary bleeding region with high reproducibility and high accuracy.

The first aspect of the embodiment uses a papillary bleeding determination model obtained by machine learning using a plurality of frontal images of the fundus labeled with a label indicating the presence or absence of papillary bleeding as the first teacher data, and the fundus of the eye to be examined. A group of a plurality of pairs of a determination device for determining the presence or absence of papillary bleeding with respect to the frontal image of the eye, and a papillary bleeding region image representing the papillary bleeding region depicted in the frontal image of the fundus. 2. Using the papillary bleeding area detection model obtained by machine learning as teacher data, a detector that detects the papillary bleeding area visualized in the front image where the papillary bleeding is determined by the determination device, and a detector. It is an ophthalmic information processing device including.

In the second aspect of the embodiment, in the first aspect, the second teacher data includes a front image included in the first teacher data.

A third aspect of the embodiment includes, in the first or second aspect, a first learning unit that generates the papillary bleeding determination model by performing supervised machine learning using the first teacher data.

A fourth aspect of the embodiment is a second learning unit that generates the papillary bleeding region detection model by performing supervised machine learning using the second supervised data in any of the first to third aspects. including.

In the fifth aspect of the embodiment, in any one of the first to fourth aspects, the frontal image of the fundus is a color frontal image.

In the sixth aspect of the embodiment, in any one of the first to fifth aspects, the position of the papillary bleeding region in the frontal image of the fundus of the eye to be inspected is determined by analyzing the frontal image of the fundus of the eye to be inspected. It includes an analysis unit that generates at least one of the position information to be represented, the shape information representing the shape of the papillary bleeding region, and the occurrence information representing the occurrence status of the papillary bleeding region.

In the seventh aspect of the embodiment, in the sixth aspect, the position information includes information indicating the direction of the representative position of the papillary bleeding region with respect to the reference position.

In the eighth aspect of the embodiment, in the sixth or seventh aspect, the position information includes information indicating whether the representative position of the papillary bleeding region is inside the optic disc region or outside the optic disc region.

In the ninth aspect of the embodiment, in the eighth aspect, the position information includes information indicating whether the representative position is inside the optic disc margin or outside the optic disc margin.

In the tenth aspect of the embodiment, in any one of the sixth to ninth aspects, the shape information includes at least one of the ellipticity of the papillary bleeding region and the area of the papillary bleeding region.

In the eleventh aspect of the embodiment, in any one of the sixth to tenth aspects, the occurrence information includes at least one of the occurrence frequency of the papillary bleeding region and the occurrence interval of the papillary bleeding region.

The twelfth aspect of the embodiment is, in any one of the sixth to eleventh aspects, the OCT data of the eye to be inspected and the frontal image of the inspected eye in which the papillary bleeding region detected by the detector is visualized. The analysis unit includes the alignment unit for aligning, and the analysis unit uses the OCT data aligned with the front image by the alignment unit to obtain the position information, the shape information, and the generation information. Generate at least one.

The thirteenth aspect of the embodiment is an ophthalmologic apparatus including an imaging unit for photographing the fundus of the eye to be inspected and an ophthalmologic information processing apparatus according to any one of the first to twelfth aspects.

A fourteenth aspect of the embodiment is described in the twelfth aspect, the imaging unit for photographing the fundus of the eye to be inspected, the OCT unit for acquiring the OCT data by performing optical coherence tomography on the eye to be inspected, and the twelfth aspect. An ophthalmic information device, including an ophthalmic information device.

In the fifteenth aspect of the embodiment, a papillary bleeding determination model obtained by machine learning using a plurality of frontal images of the fundus labeled with a label indicating the presence or absence of papillary bleeding as first teacher data is used to determine the fundus of the eye to be examined. A plurality of pairs of image groups consisting of a determination step for determining the presence or absence of papillary bleeding with respect to the frontal image of the above and a papillary bleeding region image representing the papillary bleeding region depicted in the frontal image of the fundus are the first. 2. Using the papillary bleeding region detection model obtained by machine learning as teacher data, a detection step for detecting the papillary bleeding region visualized in the front image determined to have papillary bleeding in the determination step, and a detection step. It is an ophthalmic information processing method including.

In the 16th aspect of the embodiment, in the 15th aspect, the second teacher data includes a front image included in the first teacher data.

The 17th aspect of the embodiment includes, in the 15th or 16th aspect, a first learning step of generating the papillary bleeding determination model by performing supervised machine learning using the first supervised data.

The eighteenth aspect of the embodiment is a second learning step of generating the papillary bleeding region detection model by performing supervised machine learning using the second supervised data in any of the fifteenth to seventeenth aspects. including.

In the 19th aspect of the embodiment, in any one of the 15th to 18th aspects, the frontal image of the fundus is a color frontal image.

In the 20th aspect of the embodiment, in any of the 15th to 19th aspects, the position of the papillary bleeding region in the frontal image of the fundus of the eye to be inspected is determined by analyzing the frontal image of the fundus of the eye to be inspected. The analysis step includes generating at least one of the position information to be represented, the shape information to represent the shape of the papillary bleeding region, and the outbreak information to represent the occurrence status of the papillary bleeding region.

In the 21st aspect of the embodiment, in the 20th aspect, the position information includes information indicating the direction of the representative position of the papillary bleeding region with respect to the reference position.

In the 22nd aspect of the embodiment, in the 20th or 21st aspect, the position information includes information indicating whether the representative position of the papillary bleeding region is inside the optic disc region or outside the optic disc region.

In the 23rd aspect of the embodiment, in the 22nd aspect, the position information includes information indicating whether the representative position is inside the optic disc margin or outside the optic disc margin.

In the 24th aspect of the embodiment, in any of the 20th to 23rd aspects, the shape information includes at least one of the ellipticity of the papillary bleeding region and the area of the papillary bleeding region.

In the 25th aspect of the embodiment, in any of the 20th to 24th aspects, the occurrence information includes at least one of the occurrence frequency of the papillary bleeding region and the occurrence interval of the papillary bleeding region.

The 26th aspect of the embodiment is, in any one of the 20th to the 25th aspects, the OCT data of the eye to be inspected and the frontal image of the inspected eye in which the papillary bleeding region detected in the detection step is visualized. The analysis step includes the alignment step for aligning, and the analysis step uses the OCT data aligned with the front image in the alignment step to obtain the position information, the shape information, and the generation information. Generate at least one.

The 27th aspect of the embodiment is a program for causing a computer to execute each step of the ophthalmologic information processing method according to any one of the 15th to 26th aspects.

It is possible to arbitrarily combine the configurations according to the above-mentioned plurality of embodiments.

INDUSTRIAL APPLICABILITY According to the present invention, it is possible to provide a new technique for detecting a papillary bleeding region with high reproducibility and high accuracy.

It is a schematic diagram which shows an example of the structure of the ophthalmic apparatus which concerns on 1st Embodiment. It is a schematic diagram which shows an example of the structure of the ophthalmic apparatus which concerns on 1st Embodiment. It is a schematic diagram which shows an example of the structure of the ophthalmic apparatus which concerns on 1st Embodiment. It is a schematic diagram which shows an example of the structure of the ophthalmic apparatus which concerns on 1st Embodiment. It is a schematic diagram for demonstrating an example of the operation of the ophthalmologic apparatus which concerns on 1st Embodiment. It is a schematic diagram which shows an example of the structure of the ophthalmic apparatus which concerns on 1st Embodiment. It is a schematic diagram for demonstrating the operation of the ophthalmic apparatus which concerns on 1st Embodiment. It is a schematic diagram which shows an example of the structure of the ophthalmic apparatus which concerns on 1st Embodiment. It is a schematic diagram for demonstrating the operation of the ophthalmic apparatus which concerns on 1st Embodiment. It is a schematic diagram for demonstrating the operation of the ophthalmic apparatus which concerns on 1st Embodiment. It is a schematic diagram for demonstrating the operation of the ophthalmic apparatus which concerns on 1st Embodiment. It is a flowchart which shows the operation example of the ophthalmic apparatus which concerns on 1st Embodiment. It is a schematic diagram for demonstrating the operation of the ophthalmic apparatus which concerns on 1st Embodiment. It is a schematic diagram which shows the 1st structural example of the ophthalmic system which concerns on 2nd Embodiment. It is a schematic diagram which shows the 2nd structural example of the ophthalmic system which concerns on 2nd Embodiment. It is a schematic diagram which shows an example of the structure of the ophthalmic information processing apparatus which concerns on 2nd Embodiment. It is a schematic diagram which shows an example of the structure of the ophthalmic system which concerns on the 1st modification of 2nd Embodiment. It is a schematic diagram which shows an example of the structure of the ophthalmic system which concerns on the 2nd modification of 2nd Embodiment.

An example of an ophthalmic information processing apparatus, an ophthalmic apparatus, an ophthalmic information processing method, and an embodiment of a program according to the present invention will be described in detail with reference to the drawings. In addition, the description contents of the document cited in this specification and arbitrary known techniques can be incorporated into the following embodiments.

The ophthalmologic information processing apparatus according to the embodiment determines the presence or absence of papillary bleeding on the fundus image of the eye to be inspected using a trained model, and the fundus determined to have papillary bleeding using another trained model. Detect the papillary bleeding area depicted in the image. The trained model used for determining the presence or absence of papillary bleeding is a papillary bleeding determination model obtained by machine learning using a plurality of fundus images labeled indicating the presence or absence of papillary bleeding as first teacher data. The trained model used for detecting the papillary bleeding region is a machine in which a plurality of pairs of images, which are a pair of a fundus image and a papillary bleeding region image representing the papillary bleeding region drawn on the fundus image, are used as the second teacher data. It is a papillary bleeding area detection model obtained by learning. The fundus image is preferably a color frontal image of the fundus obtained by photographing the fundus with a fundus camera or the like.

As a result, by generating a papillary bleeding region detection model by performing machine learning using only the fundus image in which the papillary bleeding region exists, the learning parameters of the papillary bleeding region detection model are specialized for the detection of the papillary bleeding region. Will be updated. As a result, the detection accuracy of the papillary bleeding region can be improved. Therefore, it is possible to detect the papillary bleeding region in the fundus image with higher reproducibility and accuracy as compared with the case of detecting the papillary bleeding region in the fundus image of the test eye using one trained model. ..

In some embodiments, the ophthalmic information processing apparatus analyzes the fundus image to generate an analysis result of the papillary bleeding region detected with high reproducibility and high accuracy. The analysis result includes at least one of the position information indicating the position of the papillary bleeding region, the shape information representing the shape of the papillary bleeding region, and the occurrence information indicating the occurrence status of the papillary bleeding region.

In some embodiments, the ophthalmologic information processing apparatus defines the OCT coordinate system by aligning the fundus image in which the papillary bleeding region is detected with the OCT (Optical Coherence Tomography) data of the subject eye. Position information and shape information are generated using the position information of. This makes it possible to obtain highly accurate position information and shape information of the papillary bleeding region from the fundus image mapped to the OCT coordinate system that defines the highly reproducible OCT data.

In some embodiments, there is a function of a fundus camera for acquiring a fundus image of the eye to be inspected and a function of an OCT device (optical coherence tomography) capable of performing OCT measurement for acquiring OCT data of the eye to be inspected. The ophthalmologic apparatus provided with the above has the function of the ophthalmologic information processing apparatus according to the embodiment.

In some embodiments, the ophthalmologic apparatus having the function of a fundus camera has the function of the ophthalmologic processing apparatus according to the embodiment. In this case, the ophthalmic appliance acquires OCT data from an externally provided OCT apparatus.

In some embodiments, the ophthalmic appliance with the functionality of the OCT apparatus has the functionality of the ophthalmic processing apparatus according to the embodiment. In this case, the ophthalmologic apparatus acquires a fundus image from an external fundus camera.

In some embodiments, the ophthalmologic processing apparatus acquires a fundus image from an externally provided fundus camera and obtains OCT data from an externally provided OCT apparatus.

In some embodiments, the ophthalmologic processing apparatus acquires fundus images and OCT data from an ophthalmologic apparatus having the function of a fundus camera and the function of an OCT device.

In some embodiments, the fundus image is a C-scan image, a projection image, or an en-face image formed by OCT data, a scanning Laser Ophthalmoscope (SLO), a slit lamp ophthalmoscope, or an operation. It is an image acquired by an ophthalmologic device such as an ophthalmoscope. When the fundus image is a C-scan image, a projection image, or an en-face image, it is not necessary to align the fundus image with the OCT data.

The ophthalmic information processing method according to the embodiment includes one or more steps for realizing a process executed by a processor (computer) in the ophthalmic information processing apparatus according to the embodiment. The program according to the embodiment causes a processor to execute each step of the ophthalmic information processing method according to the embodiment. The recording medium according to the embodiment is a computer-readable non-temporary recording medium (storage medium) on which the program according to the embodiment is recorded.

In the present specification, the "processor" is, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), a programmable logic device (for example, a SPLD (SimpleProgram)). It means a circuit such as a Program Control Device (Programmable Logical Device), an FPGA (Field Program Cable Gate Array)) and the like. The processor realizes the function according to the embodiment by reading and executing a program stored in a storage circuit or a storage device, for example.

In this specification, images acquired by OCT may be collectively referred to as OCT images. Further, the measurement operation for forming an OCT image may be referred to as OCT measurement. In addition, papillary bleeding may be simply referred to as DH (Disc Hemorrhage).

[First Embodiment]
The ophthalmologic apparatus according to the first embodiment includes a function of a fundus camera, a function of an OCT apparatus, and a function of an ophthalmology processing apparatus according to an embodiment. The ophthalmic appliance may further comprise at least one function of an ophthalmic measuring device and an ophthalmic treatment device. In some embodiments, the ophthalmologic measuring device comprises at least one of an ophthalmic refraction tester, a tonometer, a specular microscope, a wavefront analyzer, a perimeter, and a microperimeter. In some embodiments, the ophthalmic treatment device comprises at least one of a laser treatment device, a surgical device, and a surgical microscope.

<Structure>
〔Optical system〕
FIG. 1 shows a configuration example of an ophthalmic apparatus according to the first embodiment.

The ophthalmic apparatus 1 according to the first embodiment includes a fundus camera unit 2, an OCT unit 100, and an arithmetic control unit 200. The fundus camera unit 2 is provided with an optical system and a mechanism for acquiring a front image of the eye E to be inspected. The OCT unit 100 is provided with a part of an optical system and a mechanism for performing OCT. The other part of the optical system and mechanism for performing OCT is provided in the fundus camera unit 2. The arithmetic and control unit 200 includes one or more processors that perform various arithmetic and control. In addition to these, any element such as a member for supporting the subject's face (chin rest, forehead pad, etc.) or a lens unit for switching the target part of OCT (for example, an attachment for anterior ocular segment OCT). Or unit may be provided in the ophthalmic apparatus 1. In some embodiments, the lens unit is manually configured to be inserted and removed between the eye E to be inspected and the objective lens 22 described below. In some embodiments, the lens unit is configured to be automatically inserted and removed between the eye E to be inspected and the objective lens 22 described later under the control of the control unit 210 described later.

[Fundus camera unit]
The fundus camera unit 2 is provided with an optical system for photographing the fundus Ef of the eye to be inspected E. The acquired image of the fundus Ef (called a fundus image, a fundus photograph, etc.) is a front image such as an observation image, a captured image, or the like. The observed image is obtained by taking a moving image using near-infrared light. The captured image is a still image (for example, a color image) using flash light. Further, the fundus camera unit 2 can take an anterior eye portion Ea of the eye to be inspected E and acquire a frontal image (anterior eye portion image).

The fundus camera unit 2 includes an illumination optical system 10 and a photographing optical system 30. The illumination optical system 10 irradiates the eye E to be inspected with illumination light. The photographing optical system 30 detects the return light of the illumination light from the eye E to be inspected. The measurement light from the OCT unit 100 is guided to the eye E to be inspected through the optical path in the fundus camera unit 2, and the return light is guided to the OCT unit 100 through the same optical path.

The light output from the observation light source 11 of the illumination optical system 10 (observation illumination light) is reflected by the reflection mirror 12 having a curved reflecting surface, passes through the condenser lens 13, and passes through the visible cut filter 14. It becomes near-infrared light. Further, the observation illumination light is once focused in the vicinity of the photographing light source 15, reflected by the mirror 16, and passes through the relay lenses 17, 18, the diaphragm 19, and the relay lens 20. Then, the observation illumination light is reflected at the peripheral portion of the perforated mirror 21 (the region around the perforated portion), passes through the dichroic mirror 46, is refracted by the objective lens 22, and is the eye to be inspected E (fundibular Ef or anterior eye). Illuminate part Ea). The return light of the observation illumination light from the eye E to be inspected is refracted by the objective lens 22, passes through the dichroic mirror 46, passes through the hole formed in the central region of the perforated mirror 21, and passes through the dichroic mirror 55. .. The return light transmitted through the dichroic mirror 55 passes through the photographing focusing lens 31 and is reflected by the mirror 32. Further, this return light passes through the half mirror 33A, is reflected by the dichroic mirror 33, and is imaged on the light receiving surface of the image sensor 35 by the condenser lens 34. The image sensor 35 detects the return light at a predetermined frame rate. The focus of the photographing optical system 30 is adjusted so as to match the fundus Ef or the anterior eye portion Ea.

The light output from the photographing light source 15 (photographing illumination light) is applied to the fundus Ef through the same path as the observation illumination light. The return light of the photographing illumination light from the eye E to be examined is guided to the dichroic mirror 33 through the same path as the return light of the observation illumination light, passes through the dichroic mirror 33, is reflected by the mirror 36, and is reflected by the condenser lens 37. An image is formed on the light receiving surface of the image sensor 38.

The LCD (Liquid Crystal Display) 39 displays a fixative and a visual acuity measurement target. A part of the light flux output from the LCD 39 is reflected by the half mirror 33A, reflected by the mirror 32, passes through the photographing focusing lens 31 and the dichroic mirror 55, and passes through the hole portion of the perforated mirror 21. The luminous flux that has passed through the hole portion of the perforated mirror 21 passes through the dichroic mirror 46, is refracted by the objective lens 22, and is projected onto the fundus Ef.

By changing the display position of the fixative on the screen of the LCD 39, the fixative position of the eye E to be inspected can be changed. Examples of fixative positions include the fixative position for acquiring an image centered on the macula, the fixative position for acquiring an image centered on the optic nerve head, and the center of the fundus between the macula and the optic nerve head. There is a fixative position for acquiring an image centered on the macula, and a fixative position for acquiring an image of a portion far away from the macula (peripheral part of the macula). The ophthalmic appliance 1 according to some embodiments includes a GUI (Graphical User Interface) and the like for designating at least one such fixative position. The ophthalmologic apparatus 1 according to some embodiments includes a GUI or the like for manually moving the fixative position (display position of the fixative target).

The configuration for presenting the movable fixative to the eye E is not limited to a display device such as an LCD. For example, a movable fixative can be generated by selectively lighting a plurality of light sources in a light source array (light emitting diode (LED) array or the like). In addition, a movable fixative can be generated by one or more movable light sources.

Further, the ophthalmic appliance 1 may be provided with one or more external fixative light sources. One of one or more external fixative light sources is capable of projecting fixative light onto the companion eye of the eye E to be inspected. The projection position of the fixative light in the companion eye can be changed. By changing the projection position of the fixative light with respect to the companion eye, the fixative position of the eye E to be inspected can be changed. The fixative position by the external fixative light source may be the same as the fixative position of the eye E to be inspected using the LCD 39. For example, a movable fixative can be generated by selectively turning on a plurality of external fixative light sources. In addition, a movable fixative can be generated by one or more movable external fixative light sources.

The alignment optical system 50 generates an alignment optotype used for alignment of the optical system with respect to the eye E to be inspected. The alignment light output from the LED 51 passes through the diaphragms 52 and 53 and the relay lens 54, is reflected by the dichroic mirror 55, and passes through the hole portion of the perforated mirror 21. The light that has passed through the hole of the perforated mirror 21 passes through the dichroic mirror 46 and is projected onto the eye E to be inspected by the objective lens 22. The corneal reflected light of the alignment light is guided to the image sensor 35 through the same path as the return light of the observation illumination light. Manual alignment and auto alignment can be executed based on the received light image (alignment optotype image).

The focus optical system 60 generates a split optotype used for focus adjustment with respect to the eye E to be inspected. The focus optical system 60 is moved along the optical path (illumination optical path) of the illumination optical system 10 in conjunction with the movement of the photographing focusing lens 31 along the optical path (photographing optical path) of the photographing optical system 30. The reflection rod 67 is removable with respect to the illumination optical path. When adjusting the focus, the reflecting surface of the reflecting rod 67 is inclined and arranged in the illumination optical path. The focus light output from the LED 61 passes through the relay lens 62, is separated into two light fluxes by the split optotype 63, passes through the two-hole diaphragm 64, is reflected by the mirror 65, and is reflected by the condenser lens 66. It is once imaged on the reflecting surface of 67 and reflected. Further, the focus light passes through the relay lens 20, is reflected by the perforated mirror 21, passes through the dichroic mirror 46, is refracted by the objective lens 22, and is projected onto the fundus Ef. The fundus reflected light of the focus light is guided to the image sensor 35 through the same path as the corneal reflected light of the alignment light. Manual focus and autofocus can be executed based on the received light image (split optotype image).

The dichroic mirror 46 synthesizes an optical path for photographing and an optical path for OCT (optical path of an interference optical system). The dichroic mirror 46 couples both optical systems so that the optical axis of the photographing optical path (photographing optical system 30) and the optical axis of the OCT optical path (interference optical system) are substantially coaxial with each other. The dichroic mirror 46 reflects light in the wavelength band used for OCT and transmits light for fundus photography. In the optical path for OCT (optical path of measurement light), the collimator lens unit 40, the optical path length changing unit 41, the optical scanner 42, the OCT focusing lens 43, the mirror 44, in this order from the OCT unit 100 side toward the dichroic mirror 46 side. And a relay lens 45 is provided.

The optical path length changing unit 41 is movable in the direction of the arrow shown in FIG. 1, and changes the length of the optical path for OCT. This change in the optical path length is used for correcting the optical path length according to the axial length and adjusting the interference state. The optical path length changing unit 41 includes a corner cube and a mechanism for moving the corner cube.

The optical scanner 42 is arranged at a position optically conjugate with the pupil of the eye E to be inspected. The optical scanner 42 deflects the measurement light LS passing through the optical path for OCT. The optical scanner 42 can deflect the measurement light LS one-dimensionally or two-dimensionally.

When one-dimensionally polarized, the optical scanner 42 includes a galvano scanner that deflects the measured light LS in a predetermined deflection angle range in a predetermined deflection direction. In the case of two-dimensional deflection, the optical scanner 42 includes a first galvano scanner and a second galvano scanner. The first galvano scanner deflects the measurement light LS so as to scan the imaging region (fundibular Ef or anterior segment Ea) in the horizontal direction orthogonal to the optical axis of the interference optical system (OCT optical system). The second galvano scanner deflects the measurement light LS deflected by the first galvano scanner so as to scan the imaging region in the direction perpendicular to the optical axis of the interference optical system. Examples of the scanning mode of the measured optical LS by the optical scanner 42 include horizontal scan, vertical scan, cross scan, radial scan, circular scan, concentric circular scan, and spiral scan.

The OCT focusing lens 43 is moved along the optical path of the measurement light LS in order to adjust the focus of the optical system for OCT. The OCT focusing lens 43 is used to make the first lens position for arranging the focal position of the measurement light LS at or near the fundus Ef of the eye E to be inspected and the measurement light LS irradiated to the eye E to be parallel light flux. It is possible to move within the range of movement including the position of the second lens. The movement of the photographing focusing lens 31, the movement of the focus optical system 60, and the movement of the OCT focusing lens 43 can be controlled in a coordinated manner.

[OCT unit]
An example of the configuration of the OCT unit 100 is shown in FIG. The OCT unit 100 is provided with an optical system for acquiring an OCT image of the eye E to be inspected. This optical system divides the light from the wavelength sweep type (wavelength scanning type) light source into the measurement light and the reference light, and causes the return light of the measurement light from the subject E to interfere with the reference light passing through the reference optical path. This is an interference optical system that generates interference light and detects this interference light. The detection result (detection signal) of the interference light by the interference optical system is an interference signal showing the spectrum of the interference light, and is sent to the arithmetic control unit 200.

The light source unit 101 is configured to include a wavelength sweep type (wavelength scanning type) light source capable of sweeping (scanning) the wavelength of emitted light, similar to a general swept source type ophthalmic apparatus. The wavelength sweep type light source is configured to include a laser light source including a resonator. The light source unit 101 changes the output wavelength with time in a near-infrared wavelength band that cannot be visually recognized by the human eye.

The light L0 output from the light source unit 101 is guided to the polarization controller 103 by the optical fiber 102, and its polarization state is adjusted. The polarization controller 103 adjusts the polarization state of the light L0 guided in the optical fiber 102 by, for example, applying stress from the outside to the looped optical fiber 102.

The light L0 whose polarization state is adjusted by the polarization controller 103 is guided by the optical fiber 104 to the fiber coupler 105 and divided into the measurement light LS and the reference light LR.

The reference optical LR is guided to the collimator 111 by the optical fiber 110 and converted into a parallel light flux, and is guided to the optical path length changing unit 114 via the optical path length correction member 112 and the dispersion compensation member 113. The optical path length correction member 112 acts to match the optical path length of the reference light LR with the optical path length of the measured light LS. The dispersion compensating member 113 acts to match the dispersion characteristics between the reference light LR and the measurement light LS. The optical path length changing unit 114 includes, for example, a corner cube and a moving mechanism for moving the corner cube, and the corner cube by the moving mechanism can be moved in the incident direction of the reference light LR. As a result, the optical path length of the reference light LR is changed.

The reference optical LR that has passed through the optical path length changing unit 114 is converted from a parallel luminous flux to a focused luminous flux by a collimator 116 via a dispersion compensating member 113 and an optical path length correcting member 112, and is incident on the optical fiber 117. The reference optical LR incident on the optical fiber 117 is guided by the polarization controller 118 to adjust its polarization state, is guided to the attenuator 120 by the optical fiber 119 to adjust the amount of light, and is guided to the fiber coupler 122 by the optical fiber 121. Be guided.

On the other hand, the measurement optical LS generated by the fiber coupler 105 is guided by the optical fiber 127 and converted into a parallel light flux by the collimator lens unit 40, and the optical path length changing unit 41, the optical scanner 42, the OCT focusing lens 43, and the mirror 44. And via the relay lens 45. The measurement light LS that has passed through the relay lens 45 is reflected by the dichroic mirror 46, refracted by the objective lens 22, and incident on the eye E to be inspected. The measurement light LS is scattered and reflected at various depth positions of the eye E to be inspected. The return light of the measurement light LS from the eye E to be inspected travels in the same direction as the outward path in the opposite direction, is guided to the fiber coupler 105, and reaches the fiber coupler 122 via the optical fiber 128.

The fiber coupler 122 combines (interferes with) the measured light LS incident via the optical fiber 128 and the reference light LR incident via the optical fiber 121 to generate interference light. The fiber coupler 122 generates a pair of interference light LCs by branching the interference light at a predetermined branching ratio (for example, 1: 1). The pair of interference light LCs are guided to the detector 125 through the optical fibers 123 and 124, respectively.

The detector 125 is, for example, a balanced photodiode (Balanced Photo Diode) that has a pair of photodetectors that detect each pair of interference light LCs and outputs the difference between the detection results. The detector 125 sends the detection result (interference signal) to the DAT (Data Acquisition System) 130. A clock KC is supplied to the DAQ 130 from the light source unit 101. The clock KC is generated in the light source unit 101 in synchronization with the output timing of each wavelength swept (scanned) within a predetermined wavelength range by the wavelength sweep type light source. The light source unit 101 optically delays one of the two branched lights obtained by branching the light L0 of each output wavelength, and then sets the clock KC based on the result of detecting these combined lights. Generate. The DAQ 130 samples the detection result of the detector 125 based on the clock KC. The DAQ 130 sends the detection result of the sampled detector 125 to the arithmetic control unit 200. The arithmetic control unit 200 obtains a reflection intensity profile in each A line by, for example, performing a Fourier transform or the like on the spectral distribution based on the detection result obtained by the detector 125 for each series of wavelength scans (for each A line). Form. Further, the arithmetic control unit 200 forms image data by imaging the reflection intensity profile of each A line.

In the configurations shown in FIGS. 1 and 2, the optical path length changing unit 41 for changing the length of the optical path (measurement optical path, measurement arm) of the measurement light LS and the optical path of the reference light LR (reference optical path, reference). Both of the optical path length changing portions 114 for changing the length of the arm) are provided. However, only one of the optical path length changing portions 41 and 114 may be provided. Further, it is also possible to change the difference between the reference optical path length and the measured optical path length by using an optical member other than these.

[Operation control unit]
The arithmetic control unit 200 analyzes the detection signal input from the DAQ 130 to form an OCT image of the eye E to be inspected. The arithmetic processing for that purpose is the same as that of the conventional swept source type OCT apparatus.

Further, the arithmetic control unit 200 controls each part of the fundus camera unit 2, the display device 3, and the OCT unit 100.

As the control of the fundus camera unit 2, the arithmetic control unit 200 controls the operation of the observation light source 11, the photographing light source 15, and the LEDs 51 and 61, the operation control of the LCD 39, the movement control of the photographing focusing lens 31, and the OCT focusing lens 43. It performs movement control, movement control of the reflection rod 67, movement control of the focus optical system 60, movement control of the optical path length changing unit 41, operation control of the optical scanner 42, and the like.

As a control of the display device 3, the arithmetic control unit 200 causes the display device 3 to display the OCT image of the eye to be inspected E.

As the control of the OCT unit 100, the arithmetic control unit 200 controls the operation of the light source unit 101, the movement control of the optical path length changing unit 114, the operation control of the attenuator 120, the operation control of the polarization controllers 103 and 118, and the operation of the detector 125. Control, operation control of DAQ130, etc. are performed.

The arithmetic control unit 200 includes, for example, a processor, RAM, ROM, a hard disk drive, a communication interface, and the like, as in a conventional computer. A computer program for controlling the ophthalmologic device 1 is stored in a storage device such as a hard disk drive. The arithmetic control unit 200 may include various circuit boards, for example, circuit boards for forming OCT images. Further, the arithmetic control unit 200 may include an operation device (input device) such as a keyboard and a mouse, and a display device such as an LCD.

The fundus camera unit 2, the display device 3, the OCT unit 100, and the arithmetic control unit 200 may be integrally configured (that is, in a single housing), or may be separately configured in two or more housings. It may have been done.

[Control system]
FIG. 3 shows a configuration example of the control system of the ophthalmic appliance 1. In FIG. 3, some of the components included in the ophthalmic appliance 1 are omitted.

The arithmetic control unit 200 includes a control unit 210 and controls each unit of the fundus camera unit 2, the display device 3, and the OCT unit 100.

(Control unit)
The control unit 210 executes various controls. The control unit 210 includes a main control unit 211 and a storage unit 212.

(Main control unit)
The main control unit 211 includes a processor and controls each unit of the ophthalmic apparatus 1. For example, the main control unit 211 includes focusing drive units 31A and 43A of the fundus camera unit 2, image sensors 35 and 38, LCD 39, an optical path length changing unit 41, an optical scanner 42, and movement of an optical system (movement mechanism 150). To control. Further, the main control unit 211 controls the light source unit 101 of the OCT unit 100, the optical path length changing unit 114, the attenuator 120, the polarization controllers 103 and 118, the detector 125, the DAT 130, and the like.

For example, the main control unit 211 displays the fixative at a position on the screen of the LCD 39 corresponding to the fixative position manually or automatically set. In addition, the main control unit 211 can change the display position of the fixative displayed on the LCD 39 (continuously or stepwise). Thereby, the fixative can be moved (that is, the fixative position can be changed). The display position and movement mode of the fixative are set manually or automatically. Manual settings are made using, for example, a GUI. The automatic setting is performed by, for example, the data processing unit 300.

The focusing drive unit 31A moves the photographing focusing lens 31 in the optical axis direction of the photographing optical system 30 and moves the focus optical system 60 in the optical axis direction of the illumination optical system 10. As a result, the focusing position of the photographing optical system 30 is changed. In some embodiments, the focusing drive unit 31A individually has a mechanism for moving the photographing focusing lens 31 and a mechanism for moving the focus optical system 60. The focusing drive unit 31A is controlled when adjusting the focus or the like.

The focusing drive unit 43A moves the OCT focusing lens 43 in the optical axis direction of the measurement optical path. As a result, the focusing position of the measurement light LS is changed. For example, by moving the OCT focusing lens 43 to the first lens position, the focusing position of the measurement light LS can be arranged at or near the fundus Ef. For example, by moving the OCT focusing lens 43 to the second lens position, the focusing position of the measurement light LS can be arranged at a far point position and the measurement light LS can be made into a parallel luminous flux. The in-focus position of the measurement light LS corresponds to the depth position (z position) of the beam waist of the measurement light LS.

The moving mechanism 150 moves, for example, at least the fundus camera unit 2 (optical system) three-dimensionally. In a typical example, the moving mechanism 150 includes at least a mechanism for moving the fundus camera unit 2 in the x direction (horizontal direction), a mechanism for moving the fundus camera unit 2 in the y direction (vertical direction), and a mechanism for moving the fundus camera unit 2 in the z direction (depth direction). , Includes a mechanism for moving in the front-back direction). The mechanism for moving in the x direction includes, for example, an x stage that can move in the x direction and an x movement mechanism that moves the x stage. The mechanism for moving in the y direction includes, for example, an y stage that can move in the y direction and a y movement mechanism that moves the y stage. The mechanism for moving in the z direction includes, for example, a z stage that can move in the z direction and a z movement mechanism that moves the z stage. Each moving mechanism includes an actuator such as a pulse motor and operates under the control of the main control unit 211.

Control over the moving mechanism 150 is used in alignment and tracking. Tracking is to move the device optical system according to the eye movement of the eye E to be inspected. Alignment and focus adjustment are performed in advance when tracking is performed. Tracking is a function of maintaining a suitable positional relationship in which alignment and focus are achieved by making the position of the device optical system follow the eye movement. In some embodiments, the moving mechanism 150 is configured to control the optical path length of the reference light (hence, the optical path length difference between the optical path of the measurement light and the optical path of the reference light).

In the case of manual alignment, the user operates the user interface 240 described later so that the displacement of the eye E to be inspected with respect to the optical system is canceled, so that the optical system and the eye E to be inspected are relatively moved. For example, the main control unit 211 controls the movement mechanism 150 by outputting a control signal corresponding to the operation content for the user interface 240 to the movement mechanism 150 to relatively move the optical system and the eye E to be inspected.

In the case of auto-alignment, the main control unit 211 controls the movement mechanism 150 so that the displacement of the eye E to be inspected with respect to the optical system is canceled, so that the optical system and the eye E to be inspected are relatively moved. In some embodiments, the main control unit 211 controls the optical system so that the optical axis of the optical system substantially coincides with the axis of the eye E to be inspected and the distance of the optical system to the eye E to be inspected is a predetermined operating distance. Is output to the moving mechanism 150 to control the moving mechanism 150 to move the optical system and the eye E to be inspected relative to each other. Here, the working distance is a default value also called the working distance of the objective lens 22, and corresponds to the distance between the eye E to be inspected and the optical system at the time of measurement using the optical system (at the time of photographing).

The main control unit 211 controls fundus photography and anterior eye photography by controlling the fundus camera unit 2 and the like. Further, the main control unit 211 controls the OCT measurement by controlling the fundus camera unit 2, the OCT unit 100, and the like. The main control unit 211 can perform a plurality of preliminary operations before performing the OCT measurement. Preliminary operations include alignment, coarse focus adjustment, polarization adjustment, and fine focus adjustment. The plurality of preliminary operations are performed in a predetermined order. In some embodiments, the plurality of preliminary operations are performed in the above order.

The type and order of preliminary operations are not limited to this, and are arbitrary. For example, a preliminary operation (small pupil determination) for determining whether or not the eye to be inspected E is a small pupil eye can be added to the preliminary operation. The small pupil determination is performed, for example, between the coarse focus adjustment and the optical path length difference adjustment. In some embodiments, the small pupil determination comprises the following sequence of processes: the process of acquiring a frontal image (anterior eye image) of the eye E to be examined; the process of identifying an image region corresponding to the pupil; identified. Processing to determine the size (diameter, circumference, etc.) of the pupil region; processing to determine whether or not the eye is a small pupil based on the determined size (threshold processing); The process to control. Some embodiments further include a circular or elliptical approximation of the pupil region to determine the pupil size.

The coarse focus adjustment is a focus adjustment using a split optotype. By determining the position of the photographing focusing lens 31 based on the information relating the eye refraction force acquired in advance and the position of the photographing focusing lens 31 and the measured value of the refraction force of the eye E to be inspected. You can also make rough focus adjustments.

The focus fine adjustment is performed based on the interference sensitivity of the OCT measurement. For example, by monitoring the interference intensity (interference sensitivity) of the interference signal acquired by the OCT measurement of the eye E to be inspected, the position of the OCT focusing lens 43 that maximizes the interference intensity is obtained, and the OCT alignment is performed at that position. By moving the focusing lens 43, it is possible to perform fine focus adjustment.

In the optical path length difference adjustment, a predetermined position in the eye E to be inspected is controlled so as to be a reference position in the measurement range in the depth direction. This control is performed on at least one of the optical path length changing units 41 and 114. Thereby, the optical path length difference between the measurement optical path and the reference optical path is adjusted. By setting the reference position by adjusting the optical path length difference, it becomes possible to accurately perform OCT measurement for a desired measurement range in the depth direction only by changing the wavelength sweep speed.

In the polarization adjustment, the polarization state of the reference light LR is adjusted in order to optimize the interference efficiency between the measurement light LS and the reference light LR.

(Memory)
The storage unit 212 stores various data. The data stored in the storage unit 212 includes, for example, image data of an OCT image, image data of a fundus image, image data of an anterior eye portion image, eye information to be inspected, and the like. The eye test information includes information about the subject such as the patient ID and name, and information about the test eye such as left eye / right eye identification information.

Further, various programs and data for operating the ophthalmic apparatus 1 are stored in the storage unit 212.

The control unit 210 can control the image forming unit 230 and the data processing unit 300.

(Image forming part)
The image forming unit 230 forms an OCT image (image data) of the eye E to be inspected based on the sampling data obtained by sampling the detection signal from the detector 125 with the DAQ 130. The OCT image formed by the image forming unit 230 includes an A scan (A mode) image, a B scan (B mode) image (tomographic image), a C scan (C mode) image, and the like. This process includes processing such as noise removal (noise reduction), filter processing, dispersion compensation, and FFT (Fast Fourier Transform), as in the case of the conventional swept source type OCT. For other types of OCT devices, the image forming unit 230 performs known processing according to the type.

The image forming unit 230 includes, for example, the circuit board described above. In this specification, "image data" and "image" based on the "image data" may be equated with each other.

(Data processing unit)
The data processing unit 300 processes the data acquired by photographing the eye E to be inspected and measuring the OCT. For example, the data processing unit 300 performs various image processing and analysis processing on the image formed by the image forming unit 230. Specifically, the data processing unit 300 executes various correction processes such as image brightness correction. In addition, the data processing unit 300 can perform various image processing and analysis processing on the image (the fundus image, the anterior eye portion image, etc.) obtained by the fundus camera unit 2.

The data processing unit 300 executes known image processing such as interpolation processing for interpolating pixels between tomographic images to form image data of a three-dimensional image of the fundus Ef or the anterior segment Ea. The image data of the three-dimensional image means the image data in which the positions of the pixels are defined by the three-dimensional coordinate system. As the image data of the three-dimensional image, there is image data composed of voxels arranged three-dimensionally. This image data is called volume data, voxel data, or the like. When displaying an image based on volume data, the data processing unit 300 performs rendering processing (volume rendering, MIP (Maximum Integrity Projection: maximum value projection), etc.) on the volume data to see from a specific line-of-sight direction. The image data of the pseudo three-dimensional image at the time is formed. This pseudo three-dimensional image is displayed on a display device such as the display unit 240A.

It is also possible to form stack data of a plurality of tomographic images as image data of a three-dimensional image. The stack data is image data obtained by three-dimensionally arranging a plurality of tomographic images obtained along a plurality of scan lines based on the positional relationship of the scan lines. That is, the stack data is image data obtained by expressing a plurality of tomographic images originally defined by individual two-dimensional coordinate systems by one three-dimensional coordinate system (that is, embedding them in one three-dimensional space). be.

The data processing unit 300 performs various renderings on the acquired three-dimensional data set (volume data, stack data, etc.) to obtain a B-mode image (vertical cross-sectional image, axial cross-sectional image) in an arbitrary cross section, and an arbitrary cross section. C-mode images (horizontal cross-sectional images, horizontal cross-sectional images), projection images, shadowgrams, and the like can be formed. An image of an arbitrary cross section, such as a B-mode image or a C-mode image, is formed by selecting pixels (pixels, voxels) on a designated cross section from a three-dimensional data set. The projection image is formed by projecting a three-dimensional data set in a predetermined direction (z direction, depth direction, axial direction). The shadow gram is formed by projecting a part of a three-dimensional data set (for example, partial data corresponding to a specific layer) in a predetermined direction. An image with the front side of the eye to be inspected as a viewpoint, such as a C-mode image, a projection image, and a shadow gram, is called a front image (en-face image).

The data processing unit 300 is a B-mode image or a frontal image (blood vessel-enhanced image, angiogram) in which retinal blood vessels and choroidal blood vessels are emphasized based on data collected in time series by OCT (for example, B-scan image data). Can be built. For example, time-series OCT data can be collected by repeatedly scanning substantially the same site of the eye E to be inspected.

In some embodiments, the data processing unit 300 compares the time-series B-scan images obtained by the B-scan on substantially the same portion, and sets the pixel value of the signal strength change portion to the pixel value corresponding to the change. By converting, an enhanced image in which the changed part is emphasized is constructed. Further, the data processing unit 300 forms an OCTA image by extracting information having a predetermined thickness at a desired portion from the constructed plurality of emphasized images and constructing the image as an en-face image.

Images generated by the data processing unit 300 (eg, three-dimensional images, B-mode images, C-mode images, projection images, shadowgrams, OCTA images) are also included in the OCT images.

Further, the data processing unit 300 analyzes the detection result of the interference light obtained by the OCT measurement and determines the focus state of the measurement light LS in the focus fine adjustment control. For example, the main control unit 211 performs repetitive OCT measurement while controlling the focusing drive unit 43A according to a predetermined algorithm. The data processing unit 300 calculates a predetermined evaluation value regarding the image quality of the OCT image by analyzing the detection result of the interference light LC repeatedly acquired by the OCT measurement. The data processing unit 300 determines whether or not the calculated evaluation value is equal to or less than the threshold value. In some embodiments, focus fine-tuning is continued until the calculated evaluation value is below the threshold. That is, when the evaluation value is equal to or less than the threshold value, it is determined that the focus state of the measurement light LS is appropriate, and the focus fine adjustment is continued until it is determined that the focus state of the measurement light LS is appropriate.

In some embodiments, the main control unit 211 monitors the strength (interference intensity, interference sensitivity) of the sequentially acquired interference signals while acquiring the interference signal by performing the repetitive OCT measurement as described above. do. Further, by moving the OCT focusing lens 43 while performing this monitoring process, the position of the OCT focusing lens 43 that maximizes the interference intensity is searched for. By such fine adjustment of the focus, the OCT focusing lens 43 can be guided to a position where the interference intensity is optimized.

Further, the data processing unit 300 analyzes the detection result of the interference light obtained by the OCT measurement, and determines the polarization state of at least one of the measurement light LS and the reference light LR. For example, the main control unit 211 performs repetitive OCT measurement while controlling at least one of the polarization controllers 103 and 118 according to a predetermined algorithm. In some embodiments, the main control unit 211 controls the attenuator 120 to change the attenuation of the reference light LR. The data processing unit 300 calculates a predetermined evaluation value regarding the image quality of the OCT image by analyzing the detection result of the interference light LC repeatedly acquired by the OCT measurement. The data processing unit 300 determines whether or not the calculated evaluation value is equal to or less than the threshold value. This threshold is preset. The polarization adjustment is continued until the calculated evaluation value becomes equal to or less than the threshold value. That is, when the evaluation value is equal to or less than the threshold value, it is determined that the polarization state of the measurement light LS is appropriate, and the polarization adjustment is continued until it is determined that the polarization state of the measurement light LS is appropriate.

In some embodiments, the main control unit 211 can also monitor the interference strength in the polarization adjustment.

The data processing unit 300 according to the embodiment determines whether or not the image formed by the image forming unit 230 or the data processing unit 300 is an analysis error image including a predetermined analysis error factor. Thereby, it is possible to automatically determine the necessity of re-imaging (re-acquisition, re-measurement) based on a certain criterion.

4 to 11 show explanatory views of the data processing unit 300 according to the embodiment. FIG. 4 shows a block diagram of a configuration example of the data processing unit 300 according to the embodiment. FIG. 5 shows a schematic diagram for explaining the operation of the data processing unit 300 of FIG. FIG. 6 shows a configuration example of the papillary bleeding (DH) determination model 311 of FIG. FIG. 7 shows a schematic diagram for explaining machine learning for the DH determination model 311 by the first learning unit 410 of FIG. FIG. 8 shows a configuration example of the papillary bleeding (DH) region detection model 321 of FIG. FIG. 9 shows a schematic diagram for explaining machine learning for the DH region detection model 321 by the second learning unit 420 of FIG. 10 and 11 show schematic views for explaining the operation of the analysis unit 360 of FIG.

In addition to the above data processing, the data processing unit 300 detects and detects the papillary bleeding (DH) region in the fundus image (for example, the above-mentioned captured image) of the eye E to be inspected acquired by using the fundus camera unit 2. It is possible to generate analysis information for the DH region. Such a data processing unit 300 includes a determination unit 310, a detector 320, a projection image forming unit 340, an alignment unit 350, an analysis unit 360, and a learning unit 400. The data processing unit 300 does not have to include the learning unit 400.

As shown in FIG. 5, the determination device 310 uses the DH determination model 311 obtained by machine learning to determine the presence or absence of DH with respect to the fundus image IMG0 acquired by the fundus camera unit 2. The detector 320 detects the DH region in the fundus image IMG0 determined by the determination device 310 to have DH by using the DH region detection model 321 obtained by machine learning. The projection image forming unit 340 forms a projection image based on the OCT data obtained by the OCT measurement performed by using the OCT unit 100 for the eye E to be inspected from which the fundus image IMG0 is acquired. The alignment unit 350 aligns the fundus image IMG0 in which the DH region is detected with the projection image as OCT data.

The analysis unit 360 analyzes the fundus image that has been aligned with the projection image (that is, the OCT data defined by the OCT coordinate system) by the alignment unit 350, so that the position information, the shape information, and the shape information of the DH region can be analyzed. Generate occurrence information. Such an analysis unit 360 includes a position information generation unit 361, a shape information generation unit 362, and a generation information generation unit 363. The position information generation unit 361 generates the position information of the DH region. The shape information generation unit 362 generates shape information in the DH region. The generation information generation unit 363 generates generation information in the DH region.

The learning unit 400 generates the DH determination model 311 and the DH region detection model 321 by executing machine learning. The learning unit 400 includes a first learning unit 410 and a second learning unit 420. The first learning unit 410 generates the DH determination model 311 by executing machine learning. The second learning unit 420 generates the DH region detection model 321 by executing machine learning.

(Judgment device)
The determination device 310 determines the presence or absence of DH with respect to the fundus image IMG0 by using the DH determination model 311 which is a learned model obtained in advance by machine learning.

The trained model according to the embodiment is used in a computer (processor) including a CPU and a memory. The function of the determination device 310 is realized by, for example, a convolutional neural network (hereinafter referred to as CNN). That is, the CPU has learned the weighting coefficient and the response function in the CNN with respect to the pixel value of the fundus image input to the convolutional layer 314 of the feature amount extraction unit 312 described later according to the command from the trained model stored in the memory. It operates so as to perform an operation based on the above and output a determination result (classification result) from the classifier 313 described later. The determination device 310 having such a configuration can extract a local correlation pattern while gradually reducing the resolution of the fundus image, and output a determination result based on the extracted correlation pattern.

As shown in FIG. 6, the DH determination model 311 (determiner 310) includes a feature amount extraction unit 312 and a classifier 313. The feature amount extraction unit 312 repeatedly extracts the feature amount and downsampling (filtering) for each predetermined image region with respect to the input fundus image IMG0 to extract the feature amount of the fundus image. The classifier 313 generates output information indicating the presence or absence of DH based on the feature amount extracted by the feature amount extraction unit 312, and indicates whether or not the fundus image IMG0 has DH based on the generated output information. Information Out1 (with / without DH) is output.

The feature amount extraction unit 312 includes a plurality of units in which units including a convolution layer and a pooling layer are connected in multiple stages. In each unit, the input of the pooling layer is connected to the output of the convolutional layer. The pixel values of the corresponding pixels in the fundus image are input to the input of the convolution layer in the first stage. The input of the convolutional layer in the subsequent stage is connected to the output of the pooling layer in the previous stage.

In FIG. 6, the feature amount extraction unit 312 includes two units connected in two stages. That is, in the feature amount extraction unit 312, the unit including the convolution layer 316 and the pooling layer 317 is connected to the subsequent stage of the unit including the convolution layer 314 and the pooling layer 315. The output of the pooling layer 315 is connected to the input of the convolution layer 316.

The classifier 313 includes a Fully Connected Layer 318, 319, and the output of the fully connected layer 318 is connected to the input of the fully connected layer 319.

In the feature amount extraction unit 312 and the classifier 313, a learned weighting coefficient (learning parameter) is assigned between the neurons in the two connected layers. The weighting factor is updated by performing machine learning. Each neuron performs an operation using a response function on the operation result including the weighting coefficient from one or more input neurons, and outputs the obtained operation result to the neuron in the next stage.

As shown in FIG. 7, the first learning unit 410 performs known machine learning on the existing initial model using the fundus image LIMG labeled with the presence / absence of DH as teacher data, thereby performing the initial model. A DH determination model 311 with an updated weighting coefficient is generated. The existing weighting coefficient is updated by machine learning using the fundus image LIMG with a label indicating the presence or absence of DH as teacher data. The teacher data may be a plurality of pairs of data in which the fundus image and the label information indicating whether or not the DH is drawn on the fundus image are paired. The presence or absence of DH is determined in advance for each fundus image based on the judgment result by a doctor, and is attached as a label of the fundus image. The first learning unit 410 can further learn the DH determination model 311 by executing machine learning using another teacher data on the generated DH determination model 311.

That is, whether or not DH is visualized on the fundus image of the eye E to be inspected by the first learning unit 410 by performing known supervised learning (machine learning) using a plurality of fundus image LiMGs as training data. A DH determination model 311 which is a trained model for determining (classifying) is generated. Machine learning may be unsupervised learning or reinforcement learning. In some embodiments, transfer learning is used to update the weighting factors.

The DH determination model 311 (feature amount extraction unit 312) may have a known layer structure such as VGG16, VGG19, InsertionV3, ResNet18, ResNet50, Xception, DenseNet201 and the like. The classifier 313 may have a known configuration such as a Random Forest or a support vector machine (SVM). For example, the DH determination model 311 according to the first embodiment is constructed by DenseNet201.

(Detector)
The detector 320 detects the DH region in the fundus image IMG0 determined by the determination device 310 to have DH by using the DH region detection model 321 which is a learned model obtained in advance by machine learning.

The trained model according to the embodiment is used in a computer (processor) including a CPU and a memory. The function of the detector 320 is realized by, for example, CNN. That is, the CPU has learned the weighting coefficient and the response function in the CNN with respect to the pixel value of the fundus image input to the convolutional layer 324 of the feature amount extraction unit 322 described later according to the command from the trained model stored in the memory. It operates so as to output an image in which the detected DH region is drawn from the restoration unit 323, which will be described later, by performing an operation based on the above. The detector 320 having such a configuration extracts a local correlation pattern while gradually reducing the resolution of the fundus image by the convolution operation, and the detection image corresponding to the fundus image by the deconvolution operation from the extracted correlation pattern. Can be output.

As shown in FIG. 8, the DH region detection model 321 (detector 320) includes a feature amount extraction unit 322 and a restoration unit 323. The feature amount extraction unit 322 extracts the feature amount of the fundus image by repeating the extraction and downsampling (filtering) of the feature amount for each predetermined image region with respect to the input fundus image IMG0. The restoration unit 323 repeatedly restores the image corresponding to the feature amount for each predetermined image area with respect to the feature amount extracted by the feature amount extraction unit 322, and the detection image Out2 corresponding to the feature amount (FIG. 5). See) is output. The detected DH region is visualized in the detected image Out2. In FIG. 8, the detection image Out2 is an output image (DH region detection image) DIMG in which the DH region DH0 is drawn.

Similar to the feature amount extraction unit 312, the feature amount extraction unit 322 includes a plurality of units in which units including a convolution layer and a pooling layer are connected in multiple stages. In each unit, the input of the pooling layer is connected to the output of the convolutional layer. The pixel values of the corresponding pixels in the fundus image are input to the input of the convolution layer in the first stage. The input of the convolutional layer in the subsequent stage is connected to the output of the pooling layer in the previous stage.

In FIG. 8, the feature amount extraction unit 322 includes two units connected in two stages. That is, in the feature amount extraction unit 322, the unit including the convolution layer 326 and the pooling layer 327 is connected to the subsequent stage of the unit including the convolution layer 324 and the pooling layer 325. The output of the pooling layer 325 is connected to the input of the convolutional layer 326.

The restoration unit 323 includes one or more units in which units including the deconvolution layer and the pooling layer are connected in one or more stages. In each unit, the input of the pooling layer is connected to the output of the deconvolution layer. The output of the pooling layer of the feature amount extraction unit 322 is connected to the input of the deconvolution layer in the first stage. The input of the deconvolution layer in the subsequent stage is connected to the output of the pooling layer in the previous stage.

In FIG. 8, the restoration unit 323 includes a deconvolution layer 328 corresponding to the pooling layer 327 of the feature amount extraction unit 322, and one unit. That is, in the restoration unit 323, a unit including the pooling layer 329 and the deconvolution layer 330 is connected to the rear stage of the deconvolution layer 328 corresponding to the pooling layer 327. The output of the pooling layer 329 is connected to the input of the deconvolution layer 330.

Similar to the DH determination model 311, in the feature amount extraction unit 322 and the restoration unit 323, a learned weighting coefficient (learning parameter) is assigned between the neurons in the two connected layers. Each neuron performs an operation using a response function on the operation result including the weighting coefficient from one or more input neurons, and outputs the obtained operation result to the neuron in the next stage.

As shown in FIG. 9, the second learning unit 420 has a plurality of pairs of the fundus image IMG1 and the DH region image TG1 representing the DH region drawn on the fundus image IMG1 as a pair with respect to the existing initial model. By performing known machine learning with the image group as teacher data, a DH region detection model 321 in which the weighting coefficient is updated with respect to the initial model is generated. The existing weighting coefficient is updated by machine learning using a plurality of pairs of image groups as teacher data. The DH region image is generated for each fundus image based on the judgment result by the doctor in advance. For example, the DH region image TG1 is generated by the doctor designating the boundary of the DH region in the fundus image IMG1 using the operation unit 240B. The second learning unit 420 can further learn the DH region detection model 321 by executing machine learning using still another teacher data for the generated DH region detection model 321.

That is, the second learning unit 420 detects the DH region in the fundus image by executing known supervised learning (machine learning) using a plurality of pairs of image groups (fund image IMG1, DH region image TG1) as training data. DH region detection model 321 which is a trained model for the purpose is generated. Machine learning may be unsupervised learning or reinforcement learning. In some embodiments, transfer learning updates the weighting factors.

Here, the teacher data (second teacher data) used by the second learning unit 420 for learning the DH region detection model 321 is the teacher data (first teacher data) used by the first learning unit 410 for learning the DH determination model 311. Includes at least part of. That is, the teacher data used by the second learning unit 420 for learning the DH region detection model 321 includes the fundus image included in the teacher data used by the first learning unit 410 for learning the DH determination model 311. As a result, it is possible to improve the detection accuracy of the DH region as described later without increasing the amount of teacher data as compared with the case of detecting the DH region using one trained model.

The DH region detection model 321 (detector 320) may have known layered structures such as VGG16, VGG19, InceptionV3, ResNet18, ResNet50, Xception, U-Net, ResUnet, ResUnet ++ and the like. For example, the DH region detection model 321 according to the first embodiment is described by ResUnet ++ (Debesh Jha etc., “ResUnet ++: An Advanced Architecture for Multimedia Segmentation”, 21st IEEE Construction). ..

(Projection image forming part)
The projection image forming unit 340 forms a projection image of the fundus Ef of the eye E to be inspected as described above. For example, the projection image forming unit 340 forms a projection image of the fundus Ef by projecting the volume data of the fundus Ef of the eye to be inspected E in the z direction.

(Alignment part)
The alignment unit 350 aligns the fundus image in which the DH region is detected with the projection image formed by the projection image forming unit 340. The fundus image in which the alignment is performed by the alignment unit 350 is a fundus image in which the DH region detected by the detector 320 is depicted. This makes it possible to associate the position of each pixel in the fundus image aligned by the alignment unit 350 with the position of the OCT coordinate system.

In some embodiments, the alignment unit 350 corresponds to a position corresponding to the optical axis of the OCT unit 100 (interfering optical system) in the projection image and the optical axis of the fundus camera unit 2 (photographing optical system) in the fundus image. Align the position so that it matches the position.

In some embodiments, the alignment unit 350 identifies a feature region in a projection image and a feature region in a fundus image, and aligns the feature regions so that the positions of the feature regions in both images match. In this case, for example, the region corresponding to the optic disc in the fundus image may be automatically or manually corrected so as to match the region corresponding to the optic disc in the projection image. Since the OCT data makes it possible to identify the characteristic site from the tomographic structure of the fundus with high accuracy, it becomes possible to detect the boundary of the region corresponding to the optic nerve head, which is difficult to distinguish from the fundus image alone, with high accuracy. ..

In some embodiments, the alignment unit 350 affine transforms at least one of the fundus image and the projection image when aligning the fundus image with the projection image. For example, the projection image is affine-transformed along with the coordinate system.

The OCT data obtained by the OCT measurement is highly reproducible and can specify the position and shape of the optic disc with high accuracy. As a result, the analysis result of the DH region by the analysis unit 360 described later can also be acquired with high reproducibility and high accuracy.

(Analysis department)
The analysis unit 360 generates position information, shape information, and generation information of the DH region by analyzing the fundus image that has been aligned with the projection image by the alignment unit 350. Specifically, the analysis unit 360 uses the position information of the coordinate system defined in the projection image to generate the position information of the above-mentioned DH region detected in the fundus image to be analyzed. Here, the analysis target may be a fundus image in which a region corresponding to the optic nerve head or the like is corrected by alignment with the projection image.

The analysis unit 360 can generate statistical information of position information, shape information, and generation information of the DH region. Statistical information includes statistical information targeting a plurality of test eyes, statistical information targeting the same test eye (or subject), and the like. Examples of statistical information include histograms.

Further, the analysis unit 360 manages the position information, the shape information and the occurrence information of the DH region, or the statistical information thereof in chronological order in relation to the detection date of the DH region or the occurrence date of the DH region estimated by the doctor. It is possible.

The position information generation unit 361 generates position information indicating the position or direction of the DH region with reference to the reference position in the fundus image. For example, the reference position is identified by analyzing a fundus image or a projection image. Examples of reference positions include the center of the optic disc and the center of gravity of the optic disc.

In some embodiments, the location information includes information indicating a representative position of the detected DH region relative to the center of the optic disc (information indicating the relative position of the DH region relative to the center of the optic disc). .. Examples of representative positions of the DH region include the center position of the DH region, the position of the center of gravity, the position in the DH region closest to the reference position, and the position in the DH region farthest from the reference position.

In some embodiments, the positional information includes information indicating the direction of the representative position of the detected DH region (angle with respect to the reference direction through the center of the optic disc) with respect to the center of the optic disc.

In some embodiments, the location information includes information indicating whether the representative location of the detected DH region is within or outside the optic disc region. In some embodiments, the location information includes information indicating whether the representative location of the detected DH region is within the optic disc margin (rim) or outside the optic disc margin.

The position information generation unit 361 can generate at least one of the above-mentioned plurality of position information.

The shape information generation unit 362 generates shape information representing the shape of the DH region detected in the fundus image. At this time, the shape information generation unit 362 quantifies the shape information of the DH region in the fundus image by using the shape of the DH region in the OCT coordinate system and the size information per pixel (pixel spacing) in the OCT coordinate system. It is possible to generate the coordinates.

In some embodiments, the shape information includes information indicating the ellipticity, area, etc. of the DH region. For example, the ellipticity can be obtained by approximating the DH region to an ellipse and calculating the ratio between the minor axis and the major axis of the specified ellipse. For example, the area of the DH region can be quantitatively obtained by specifying the boundary of the DH region, counting the number of pixels in the specified boundary, and using the size information per pixel in the OCT coordinate system.

The shape information generation unit 362 can generate at least one of the above-mentioned plurality of shape information.

The generation information generation unit 363 generates generation information of the detected DH region. For example, the generation information generation unit 363 (analysis unit 360) DH is based on the detection date of the DH region that has occurred in the past for a plurality of subjects to be inspected or the eye to be analyzed or the occurrence date of the DH region estimated by the doctor. Generate area generation information.

In some embodiments, the occurrence information includes at least one of the frequency of occurrence of the DH region and the interval of occurrence of the DH region.

The frequency of occurrence includes the frequency of occurrence of the DH region in a plurality of subjects or the same subject, the frequency of occurrence of the DH region in a plurality of subjects or the same subject, and the like. The frequency for each occurrence position includes the frequency for each block radially divided based on the reference position. In this case, the generation information is obtained for each generation position in the DH region based on the above position information, or for each block radially divided with reference to the reference position specified based on the above position information.

The interval of occurrence is the date of detection of the current DH region or the date of occurrence of the current DH region estimated by the doctor, and the date of detection of the DH region recorded most recently in the past or the date of occurrence of the DH region estimated by the doctor. It is obtained by the interval of.

FIG. 10 schematically shows an example of the position information generated by the position information generation unit 361 according to the first embodiment. FIG. 10 shows a histogram of the positions of the DH regions in a plurality of subjects to be inspected. In FIG. 10, the direction of T (ear side) is 0 degrees and the direction of N (nasal side) is 180 degrees with respect to the optic nerve head.

As shown in FIG. 10, it can be understood that the DH region is likely to occur in the T (ear side) direction with respect to the center of the optic nerve head. Although FIG. 10 shows a histogram of the position of the DH region with respect to a plurality of eyes to be inspected, the position information generation unit 361 may generate a histogram of the position of the DH region with respect to the same eye to be inspected as shown in FIG.

FIG. 11 schematically shows an example of shape information generated by the shape information generation unit 362 according to the first embodiment. FIG. 11 shows a histogram of the area of the DH region in a plurality of subjects to be inspected. In FIG. 11, the vertical axis represents the frequency and the horizontal axis represents the area [mm ² ].

As shown in FIG. 11, the distribution of the area of the DH region can be grasped. Although FIG. 11 shows a histogram of the area of the DH region for a plurality of subjects to be inspected, the shape information generation unit 362 may generate a histogram of the area of the DH region for the same eye to be inspected as shown in FIG.

In some embodiments, the analysis unit 360 aligns the fundus image in which the DH region is detected by the detector 320 with the layer thickness map of the predetermined layer region of the fundus Ef obtained by a known method by OCT measurement. To generate a composite image in which the fundus image and the layer thickness map are superimposed. The predetermined layer region includes a ganglion cell layer (GCL), a retinal nerve fiber layer (RNFL), a peri-optic disc retinal nerve fiber layer (cpRNFL), GCC (= NFL + GCL + IPL (inner plexiform layer)) and the like. In this case, the control unit 210 (main control unit 211) can display the generated composite image on the display unit 240A described later.

In some embodiments, the analysis unit 360 estimates the time (or date) of occurrence of the DH region based on the color of the DH region detected by the detector 320. For example, the analysis unit 360 estimates the occurrence time of the DH region based on the detected color of the DH region based on the reference color of the DH region. The color of the DH region is mainly specified by extracting the red component in the fundus image. In some embodiments, the analysis unit 360 estimates the generation interval of the DH region based on the difference in the color (for example, the pixel value of the red component) of the DH region detected last time and this time. In some embodiments, the analysis unit 360 estimates the time of occurrence of the DH region based on the color of the DH region detected on the previous inspection day and the interval of occurrence of the DH region.

The data processing unit 300 that functions as described above includes, for example, the above-mentioned processor, RAM, ROM, hard disk drive, circuit board, and the like. A computer program that causes a processor to execute the above functions is stored in a storage device such as a hard disk drive in advance.

(User interface)
As shown in FIG. 3, the user interface 240 includes a display unit 240A and an operation unit 240B. The display unit 240A includes the display device and the display device 3 of the arithmetic control unit 200 described above. The operation unit 240B includes the operation device of the arithmetic control unit 200 described above. The operation unit 240B may include various buttons and keys provided on the housing of the ophthalmologic device 1 or on the outside. For example, when the fundus camera unit 2 has a housing similar to that of a conventional fundus camera, the operation unit 240B may include a joystick, an operation panel, or the like provided on the housing. Further, the display unit 240A may include various display devices such as a touch panel provided on the housing of the fundus camera unit 2.

The display unit 240A and the operation unit 240B do not need to be configured as separate devices. For example, it is also possible to use a device such as a touch panel in which a display function and an operation function are integrated. In that case, the operation unit 240B includes the touch panel and a computer program. The operation content for the operation unit 240B is input to the control unit 210 as an electric signal. Further, the graphical user interface (GUI) displayed on the display unit 240A and the operation unit 240B may be used to perform operations and information input.

The data processing unit 300 is an example of the "ophthalmology information processing apparatus" according to the embodiment. The projection image is an example of "OCT data" according to the embodiment. The fundus camera unit 2 (shooting optical system 30) is an example of the “shooting unit” according to the embodiment. The OCT unit 100 and the image forming unit 230 (and / or the data processing unit 300) are examples of the "OCT unit" according to the embodiment.

[motion]
The operation of the ophthalmic apparatus 1 according to the first embodiment will be described.

FIG. 12 shows an operation example of the ophthalmic apparatus 1 according to the first embodiment. FIG. 12 shows a flowchart of an operation example of the ophthalmic appliance 1 according to the first embodiment. A computer program for realizing the process shown in FIG. 12 is stored in the storage unit 212. The main control unit 211 executes the process shown in FIG. 12 by operating according to this computer program.

In the following, it is assumed that the alignment between the eye E to be inspected and the optical system has already been completed. Further, it is assumed that the DH determination model 311 has already been machine-learned by the first learning unit 410, and the DH region detection model 321 has already been machine-learned by the second learning unit 420.

(S1: Acquire a color fundus image)
First, the main control unit 211 (control unit 210) controls the fundus camera unit 2 (photographing optical system 30) to image the fundus Ef of the eye E to be inspected. As a result, the main control unit 211 can acquire a color fundus image of the eye E to be inspected.

(S2: Determine the presence or absence of DH)
Next, the main control unit 211 controls the determination device 310 to determine whether or not the color fundus image acquired in step S1 has DH.

As described above, the determination device 310 determines (classifies) whether or not there is DH in the color fundus image using the DH determination model 311.

(S3: Is there DH?)
When it is determined in step S2 that the color fundus image has DH (S3: Y), the operation of the ophthalmologic apparatus 1 shifts to step S4.

When it is determined in step S2 that there is no DH in the color fundus image (S3: N), the operation of the ophthalmologic apparatus 1 is terminated (end).

(S4: DH area detected)
When it is determined in step S2 that the color fundus image has DH (S3: Y), the main control unit 211 controls the detector 320 and DH in the color fundus image determined to have DH in step S2. Have the area detected.

As described above, the detector 320 detects the DH region in the color fundus image determined to have DH in step S2 by using the DH region detection model 321.

(S5: Alignment)
Subsequently, the main control unit 211 controls the alignment unit 350 to perform OCT measurement on the fundus image in which the DH region is detected in step S4 and the eye E to be inspected, based on the OCT data obtained. Alignment with the projection image formed in the above is executed.

In some embodiments, after the DH region is detected in the fundus image in step S4, the main control unit 211 controls the OCT unit 100 to execute the OCT measurement, and controls the image forming unit 230 to control the OCT image. Is formed, and the projection image forming unit 340 is controlled to form a projection image.

In some embodiments, OCT measurement is performed on the eye E to be inspected in advance, and a projection image of the eye E to be inspected is formed by the projection image forming unit 340.

(S6: Generate location information)
Subsequently, the main control unit 211 controls the position information generation unit 361 to generate the position information of the DH region in the fundus image aligned in step S5.

For example, the position information generation unit 361 obtains a relative position between the center of the optic nerve head and the representative position of the DH region. For example, the position information generation unit 361 obtains the direction of the representative position of the DH region with reference to the center of the optic nerve head, and generates the position information as shown in FIG.

(S7: Generate shape information)
Subsequently, the main control unit 211 controls the shape information generation unit 362 to generate shape information of the DH region in the fundus image aligned in step S5.

For example, the shape information generation unit 362 obtains the area of the DH region as described above, and generates the shape information as shown in FIG.

(S8: Generate generation information)
Subsequently, the main control unit 211 controls the generation information generation unit 363 to generate generation information of the DH region detected in step S4.

For example, the generation information generation unit 363 generates generation information including the occurrence frequency of the DH region and the generation interval of the DH region as described above.

This completes the operation of the ophthalmic appliance 1.

Here, the detection accuracy of the DH region by the ophthalmic appliance 1 according to the first embodiment will be described.

FIG. 13 shows an explanatory diagram of the detection accuracy of the DH region by the ophthalmic appliance 1 according to the first embodiment. FIG. 13 schematically shows the detection accuracy of the DH region by the ophthalmic appliance 1 according to the first embodiment by comparing with the comparative example of the first embodiment. In the comparative example of the first embodiment, one trained model (ResUnit ++) is generated by machine learning, and the DH region is directly detected from the fundus image using the generated trained model.

In FIG. 13, mIOU (main Intersection Over Union) is used as an index showing the detection accuracy of the DH region. The mIOU corresponds to the average value of the IOU calculated for each image. Here, as represented by the equation (1), the IOU is the number of pixels (P) in the positive region and the true region in the fundus image as the input image and the detection image (DH region detection image) as the output image. It represents the ratio of the number of pixels (TP) in the true positive area to the sum of the number of pixels (T). That is, IOU is an index indicating that the closer it is to 1, the higher the detection accuracy.

In this embodiment, the positive region corresponds to a region detected as a DH region in the detected image (output image). The true region corresponds to the true DH region in the fundus image. The true positive region corresponds to a region (correct region) in which the DH region detected in the detected image and the true DH region in the fundus image coincide with each other.

FIG. 13 shows the mIOU (average value of 146 × 2 IOUs) according to the configuration according to the comparative example and the mIOU according to the configuration according to the first embodiment for 146 images with DH and 146 images without DH. show. That is, in the comparative example, when the DH region is detected using one trained model as described above, mIOU = 0.573. On the other hand, when the DH region is detected using the DH determination model 311 and the DH region detection model 321 in the first embodiment, mIOU = 0.611.

Therefore, FIG. 13 shows that the DH region can be detected with high accuracy according to the configuration according to the first embodiment as compared with the comparative example.

As described above, according to the first embodiment, it is possible to detect the DH region with high reproducibility and high accuracy as compared with the case where the DH region is detected from the fundus image using one trained model. Become. In addition, the position information, shape information, and generation information of the DH region are generated by aligning the fundus image in which the DH region is detected with the projection image and using the OCT data. Therefore, the detected DH region is generated. Can be evaluated quantitatively and with high accuracy.

[Second Embodiment]
In the first embodiment, the case where the DH region in the fundus image is detected by the ophthalmologic apparatus to which the ophthalmologic information processing apparatus according to the embodiment is applied has been described, but the configuration according to the embodiment is not limited to this.

In the second embodiment, the ophthalmologic information processing apparatus according to the embodiment executes the above-mentioned DH region detection process on the fundus image or OCT data (projection image) acquired by one or more externally provided devices. can do. Hereinafter, the second embodiment will be described with a focus on the differences from the first embodiment.

FIG. 14 shows a block diagram of a first configuration example of the ophthalmic system according to the second embodiment.

The ophthalmology system 500 according to the first configuration example includes an ophthalmology apparatus 510 ₁ to 510 _N (N is an integer of 2 or more) and an ophthalmology information processing apparatus 520. The ophthalmic information processing device 520 is connected to the ophthalmic devices 510 ₁ to 510 _N via the network 530. The network 530 may be a wired or wireless network (LAN, WAN).

The ophthalmic information processing device 520 can communicate with any of the ophthalmic devices 510 ₁ to 510 _N.

In some embodiments, any of the ophthalmic appliances 510 ₁ to 510 _N sends a request to the ophthalmic information processing apparatus 520, and the ophthalmic appliance to which the request is approved is directed to the ophthalmic information processing apparatus 520. Send the data. The transmitted data includes the image data of the fundus image of the eye E to be inspected and the above-mentioned OCT data or the image data of the projection image of the eye E to be inspected.

In some embodiments, the ophthalmic information processing device 520 sends a request to any of the ophthalmic devices 510 ₁ to 510 _N and receives data from the ophthalmic device that approves the request. The received data includes the image data of the fundus image of the eye E to be inspected and the above-mentioned OCT data or the image data of the projection image of the eye E to be inspected.

FIG. 15 shows a block diagram of a second configuration example of the ophthalmic system according to the second embodiment.

The ophthalmology system 500 according to the second configuration example includes an ophthalmology apparatus 510 and an ophthalmology information processing apparatus 520. The ophthalmic information processing device 520 is connected to the ophthalmic device 510 via a predetermined communication path. In some embodiments, the ophthalmic information processing device 520 is peer-to-peer connected to the ophthalmic device 510 via a network.

The ophthalmic information processing device 520 can communicate with the ophthalmic device 510.

In some embodiments, the ophthalmic appliance 510 sends a request to the ophthalmic information processing apparatus 520, and the ophthalmic appliance 510 for which the request is approved transmits data to the ophthalmic information processing apparatus 520. The transmitted data includes the image data of the fundus image of the eye E to be inspected and the above-mentioned OCT data or the image data of the projection image of the eye E to be inspected.

In some embodiments, the ophthalmic information processing device 520 sends a request to the ophthalmic device 510 and receives data from the ophthalmic device 510 that approves the request. The received data includes the image data of the fundus image of the eye E to be inspected and the above-mentioned OCT data or the image data of the projection image of the eye E to be inspected.

The ophthalmic appliances 510 ₁ to 510 _N and the ophthalmic appliance 510 have substantially the same configurations as the ophthalmic appliance 1 shown in FIGS. 1 to 3. In the second embodiment, among the functions of the ophthalmic apparatus 1 shown in FIGS. 1 to 3, a part of the functions of the data processing unit 300 is realized by the ophthalmic information processing apparatus 520.

FIG. 16 shows a block diagram of a configuration example of the ophthalmic information processing apparatus 520 according to the second embodiment. In FIG. 16, the same parts as those in FIG. 4 are designated by the same reference numerals, and the description thereof will be omitted as appropriate.

The ophthalmic information processing apparatus 520 includes a communication unit 521, a determination device 310, a detector 320, a projection image forming unit 340, an analysis unit 360, a learning unit 400, and a control unit 522. The ophthalmic information processing apparatus 520 may not include the learning unit 400.

The communication unit 521 performs communication interface processing with the ophthalmic apparatus 510 ₁ to 510 _N or the ophthalmic apparatus 510. That is, the communication unit 521 communicates with the ophthalmologic devices 510 ₁ to 510 _N or the ophthalmology device 510 according to a predetermined communication protocol, and receives image data of the fundus image from the ophthalmology devices 510 ₁ to 510 _N or the ophthalmology device 510. The communication data including the above-mentioned OCT data of the eye to be inspected E or the image data of the projection image is received. The communication unit 521 acquires the fundus image and OCT data of the eye to be inspected obtained by the ophthalmologic apparatus by receiving the image data included in the received communication data.

Like the control unit 210, the control unit 522 includes a processor and controls each unit of the ophthalmic information processing apparatus 520. The control unit 522 includes a main control unit and a storage unit, similarly to the control unit 210.

Further, the control unit 522 can control the display of the display unit 540 connected to the outside of the ophthalmic information processing apparatus 520 as the display control unit. The display unit 540 has the same function as the display unit 240A.

Since the operation of the ophthalmic information processing apparatus 520 is the same as the flow shown in FIG. 12, except that the fundus image and the projection image (OCT data) are acquired from the outside, detailed description thereof will be omitted. That is, the ophthalmology information processing apparatus 520 determines the presence or absence of DH with respect to the fundus image acquired from the ophthalmology apparatus 510 ₁ to 510 _N or the ophthalmology apparatus 510, and determines that there is DH. The DH region is detected on the fundus image. Further, the ophthalmology information processing apparatus 520 aligns with the fundus image using the OCT data acquired from the ophthalmology apparatus 510 ₁ to 510 _N or the ophthalmology apparatus 510, and obtains the analysis result of the above-mentioned DH region position information and the like. ..

The configuration of the ophthalmic system 500 according to the second embodiment is not limited to the configuration shown in FIGS. 14 and 15.

FIG. 17 shows a configuration example of an ophthalmic system according to a first modification of the second embodiment.

The ophthalmology system 500a according to the first modification of the second embodiment includes an ophthalmology information processing apparatus 520, a fundus camera 550, and an OCT apparatus 560. The ophthalmic information processing device 520 can communicate with each of the fundus camera 550 and the OCT device 560.

The fundus camera 550 has a function of the fundus camera unit 2 of the ophthalmologic apparatus 1 according to the first embodiment (that is, a function of acquiring a fundus image). The OCT device 560 has some functions (that is, a function of acquiring OCT data and forming an OCT image) of the OCT unit 100, the image forming unit 230, and the data processing unit 300 of the ophthalmic apparatus 1 according to the first embodiment. Have.

The ophthalmology information processing apparatus 520 acquires image data of the fundus image of the eye to be inspected E from the fundus camera 550, and acquires OCT data (or image data of the projection image) of the eye to be inspected E from the OCT device 560. The ophthalmic information processing apparatus 520 determines the presence or absence of DH in the fundus image acquired from the fundus camera 550, and detects the DH region in the fundus image determined to have DH, as in the first embodiment. do. Further, the ophthalmology information processing apparatus 520 aligns with the fundus image using the OCT data acquired from the OCT apparatus 560, and obtains the analysis result such as the position information of the DH region.

FIG. 18 shows a configuration example of an ophthalmic system according to a second modification of the second embodiment.

The ophthalmology system 500b according to the second modification of the second embodiment includes a fundus camera 550 and an OCT device 560, and the OCT device 560 includes an ophthalmology information processing unit 561. The OCT device 560 can communicate with the fundus camera 550.

The fundus camera 550 has a function of the fundus camera unit 2 of the ophthalmologic apparatus 1 according to the first embodiment (that is, a function of acquiring a fundus image). The OCT device 560 has some functions (that is, a function of acquiring OCT data and forming an OCT image) of the OCT unit 100, the image forming unit 230, and the data processing unit 300 of the ophthalmic apparatus 1 according to the first embodiment. Have. The ophthalmology information processing unit 561 has a function of the ophthalmology information processing device 520.

The ophthalmology information processing unit 561 acquires the image data of the fundus image of the eye to be inspected E from the fundus camera 550, and uses the OCT data (or the image data of the projection image) of the eye to be inspected E acquired by the OCT apparatus 560 to describe the above. The analysis process by the analysis unit 360 is executed. The ophthalmology information processing unit 561 determines the presence or absence of DH in the fundus image acquired from the fundus camera 550, and detects the DH region in the fundus image determined to have DH, as in the first embodiment. do. Further, the ophthalmology information processing unit 561 performs positioning with the fundus image using the OCT data acquired by the OCT device 560, and obtains the analysis result such as the position information of the DH region.

[Action]
The ophthalmology information processing apparatus, the ophthalmology apparatus, the ophthalmology information processing method, and the program according to the embodiment will be described.

The ophthalmic information processing apparatus (data processing unit 300, ophthalmic information processing apparatus 520, ophthalmic information processing unit 561) according to the embodiment includes a determination device (310) and a detector (320). The determination device uses a papillary bleeding determination model (311) obtained by machine learning using a plurality of frontal images (LIMG) of the fundus labeled with a label indicating the presence or absence of papillary bleeding as the first teacher data, and the eye to be inspected. The presence or absence of papillary bleeding is determined with respect to the front image (fundus image IMG0) of the fundus (Ef) of (E). The detector is a machine learning using a plurality of pairs of images as a second teacher data, which is a pair of a frontal image (IMG1) of the fundus and a papillary bleeding area image (TG1) representing the papillary bleeding area drawn on the frontal image. Using the papillary bleeding region detection model (322) obtained in the above, the papillary bleeding region visualized on the front image determined to have papillary bleeding by the determining device is detected.

According to such an aspect, the presence or absence of papillary bleeding in the frontal image of the fundus of the eye to be inspected is determined by the determining device, and the papillary bleeding in the frontal image is determined with respect to the frontal image determined by the detector to have papillary bleeding. The area is detected. At this time, in the papillary bleeding region detection model in the detector, the learning parameters are updated specifically for the detection of the papillary bleeding region by machine learning using only the fundus image in which the papillary bleeding region exists. As a result, the detection accuracy of the papillary bleeding region can be improved. Therefore, it is possible to detect the papillary bleeding region in the fundus image with higher reproducibility and accuracy as compared with the case of detecting the papillary bleeding region in the fundus image of the test eye using one trained model. ..

In the ophthalmic information processing apparatus according to the embodiment, the second teacher data includes a front image included in the first teacher data.

According to such an embodiment, since the papillary bleeding determination model of the determination device and the papillary bleeding area detection model of the detector are machine-learned using the same front image, the amount of teacher data is not increased. It becomes possible to detect the papillary bleeding region in the fundus image with higher reproducibility and higher accuracy.

The ophthalmic information processing apparatus according to the embodiment includes a first learning unit (410) that generates a papillary bleeding determination model by performing supervised machine learning using the first supervised data.

According to such an aspect, since the papillary bleeding determination model is trained by the first learning unit, it is possible to provide an ophthalmologic information processing apparatus capable of improving the determination accuracy of the presence or absence of papillary bleeding. become.

The ophthalmic information processing apparatus according to the embodiment includes a second learning unit (420) that generates a papillary bleeding region detection model by performing supervised machine learning using the second supervised data.

According to such an aspect, since the papillary bleeding region detection model is trained by the second learning unit, it is possible to provide an ophthalmic information processing apparatus capable of improving the detection accuracy of the papillary bleeding region. become.

In the ophthalmic information processing apparatus according to the embodiment, the front image of the fundus is a color front image.

According to such an aspect, by using a front image having a larger amount of information, it becomes possible to further improve the determination accuracy by the determination device and the detection accuracy by the detector.

The ophthalmic information processing apparatus according to the embodiment analyzes the frontal image of the fundus of the eye to be inspected, thereby indicating the position of the papillary bleeding region in the frontal image of the fundus of the eye to be inspected and the shape information representing the shape of the papillary bleeding region. , And an analysis unit (360) that generates at least one of the developmental information representing the developmental status of the papillary bleeding region.

According to such an embodiment, the papillary bleeding region detected with higher reproducibility and high accuracy is analyzed. Therefore, the papillary bleeding region in the fundus image of the eye to be inspected can be analyzed with high reproducibility and high accuracy. You will be able to get it.

In the ophthalmic information processing apparatus according to the embodiment, the position information includes information indicating the direction of the representative position of the papillary bleeding region with respect to the reference position.

According to such an aspect, it becomes easy to grasp the direction of the occurrence position of the papillary bleeding region with respect to the reference position in the fundus. As a result, it becomes possible to contribute to the identification of the symptoms of the future disease estimated according to the direction of occurrence of the papillary bleeding region with respect to the reference position and the determination of the future treatment policy.

In the ophthalmic information processing apparatus according to the embodiment, the position information includes information indicating whether the representative position of the optic disc bleeding region is inside the optic disc region or outside the optic disc region.

According to such an aspect, it becomes easy to grasp whether the papillary bleeding region is inside the optic disc region or outside the optic disc region. Thereby, it becomes possible to contribute to the identification of the symptom of the future disease estimated according to whether or not the papillary bleeding region is within the optic disc region and the determination of the future treatment policy.

In the ophthalmic information processing apparatus according to the embodiment, the position information includes information indicating whether the representative position is inside the optic disc margin or outside the optic disc margin.

According to such an aspect, it becomes easy to grasp whether the papillary bleeding region is inside the optic disc margin or outside the optic disc margin. Thereby, it becomes possible to contribute to the identification of the symptom of the future disease estimated according to whether or not the papillary bleeding region is within the optic disc margin and the determination of the future treatment policy.

In the ophthalmic information processing apparatus according to the embodiment, the shape information includes at least one of the ellipticity of the papillary bleeding region and the area of the papillary bleeding region.

According to such an aspect, it becomes easy to grasp the shape and size of the papillary bleeding region. As a result, it becomes possible to contribute to the identification of the symptoms of future diseases estimated according to the shape and size of the papillary bleeding area and the determination of future treatment policies.

In the ophthalmic information processing apparatus according to the embodiment, the occurrence information includes at least one of the occurrence frequency of the papillary bleeding region and the occurrence interval of the papillary bleeding region.

According to such an aspect, it becomes easy to grasp the occurrence state of the papillary bleeding region. As a result, it will be possible to contribute to the identification of the symptoms of future diseases estimated according to the occurrence of the papillary bleeding area and the decision of the future treatment policy.

The ophthalmic information processing apparatus according to the embodiment includes an alignment unit (350) for aligning the OCT data of the eye to be inspected with the front image of the eye to be inspected in which the papillary bleeding region detected by the detector is visualized, and is analyzed. The unit generates at least one of position information, shape information, and generation information using the OCT data aligned with the front image by the alignment unit.

According to such an aspect, since the papillary bleeding region in the front image is aligned with the OCT data, the papillary bleeding region is defined in the OCT coordinate system that defines the OCT data acquired with high reproducibility and high accuracy. It will be possible to identify. As a result, it becomes possible to quantitatively obtain the analysis result of the papillary bleeding region with high reproducibility and high accuracy.

The ophthalmic apparatus (1) according to the embodiment includes an imaging unit (imaging optical system 30) for photographing the fundus of the eye to be inspected, and an ophthalmic information processing apparatus according to any one of the above.

According to such an aspect, it is possible to provide an ophthalmic apparatus capable of acquiring a frontal image of an eye to be inspected and detecting a papillary bleeding region in the acquired frontal image with higher reproducibility and high accuracy. become.

The ophthalmic apparatus (1) according to the embodiment includes an imaging unit (imaging optical system 30) that photographs the fundus of the eye to be inspected, and an OCT unit (OCT unit) that acquires OCT data by performing optical coherence tomography on the eye to be inspected. The OCT unit 100, the image forming unit 230, a part of the data processing unit 300), and the ophthalmologic information processing apparatus described above are included.

According to such an aspect, an ophthalmic apparatus capable of acquiring a frontal image of an eye to be inspected and OCT data and detecting a papillary bleeding region in the acquired frontal image with higher reproducibility and high accuracy is provided. You will be able to. In addition, using the acquired OCT data, it becomes possible to quantitatively obtain the analysis result of the detected papillary bleeding region with high reproducibility and high accuracy.

The ophthalmic information processing method according to the embodiment includes a determination step and a detection step. The determination step uses a papilla bleeding determination model (311) obtained by machine learning using a plurality of frontal images (LIMGs) of the fundus labeled with a label indicating the presence or absence of papilla bleeding as the first teacher data. The presence or absence of papillary bleeding is determined with respect to the front image of the fundus (Ef) of (E). The detection step is machine learning using a plurality of pairs of images as a second teacher data, which is a pair of a frontal image of the fundus (IMG1) and a papillary bleeding area image (TG1) representing the papillary bleeding area drawn on the frontal image. Using the papillary bleeding region detection model (322) obtained in the above, the papillary bleeding region visualized on the front image determined to have papillary bleeding in the determination step is detected.

According to such an aspect, the presence or absence of papillary bleeding in the frontal image of the fundus of the eye to be inspected is determined in the determination step, and the papillary bleeding in the frontal image with respect to the frontal image determined to have papillary bleeding in the detection step. The area is detected. At this time, in the papillary bleeding region detection model in the detection step, the learning parameters are updated specifically for the detection of the papillary bleeding region by machine learning using only the fundus image in which the papillary bleeding region exists. As a result, the detection accuracy of the papillary bleeding region can be improved. Therefore, it is possible to detect the papillary bleeding region in the fundus image with higher reproducibility and accuracy as compared with the case of detecting the papillary bleeding region in the fundus image of the test eye using one trained model. ..

In the ophthalmic information processing method according to the embodiment, the second teacher data includes a front image included in the first teacher data.

According to such an embodiment, the papillary bleeding determination model in the determination step and the papillary bleeding region detection model in the detection step are machine-learned using the same front image, so that the amount of teacher data is not increased. It becomes possible to detect the papillary bleeding region in the fundus image with higher reproducibility and higher accuracy.

The ophthalmic information processing method according to the embodiment includes a first learning step of generating a papillary bleeding determination model by performing supervised machine learning using the first teacher data.

According to such an aspect, since the papillary bleeding determination model is trained in the first learning step, it is possible to provide an ophthalmologic information processing method capable of improving the determination accuracy of the presence or absence of papillary bleeding. become.

The ophthalmic information processing method according to the embodiment includes a second learning step of generating a papillary bleeding region detection model by performing supervised machine learning using the second supervised data.

According to such an aspect, since the papillary bleeding region detection model is trained in the second learning step, it is possible to provide an ophthalmologic information processing method capable of improving the detection accuracy of the papillary bleeding region. become.

In the ophthalmic information processing method according to the embodiment, the frontal image of the fundus is a color frontal image.

According to such an aspect, by using a front image having a larger amount of information, it becomes possible to further improve the determination accuracy in the determination step and the detection accuracy in the detection step.

The ophthalmologic information processing method according to the embodiment analyzes the frontal image of the fundus of the eye to be inspected, thereby indicating the position of the papillary bleeding region in the frontal image of the fundus of the eye to be inspected and the shape information representing the shape of the papillary bleeding region. , And an analysis step to generate at least one of the developmental information representing the developmental status of the papillary bleeding area.

According to such an embodiment, the papillary bleeding region detected with higher reproducibility and high accuracy is analyzed. Therefore, the papillary bleeding region in the fundus image of the eye to be inspected can be analyzed with high reproducibility and high accuracy. You will be able to get it.

In the ophthalmic information processing method according to the embodiment, the position information includes information indicating the direction of the representative position of the papillary bleeding region with respect to the reference position.

According to such an aspect, it becomes easy to grasp the direction of the occurrence position of the papillary bleeding region with respect to the reference position in the fundus. As a result, it becomes possible to contribute to the identification of the symptoms of the future disease estimated according to the direction of occurrence of the papillary bleeding region with respect to the reference position and the determination of the future treatment policy.

In the ophthalmic information processing method according to the embodiment, the position information includes information indicating whether the representative position of the papillary bleeding region is inside the optic disc region or outside the optic disc region.

According to such an aspect, it becomes easy to grasp whether the papillary bleeding region is inside the optic disc region or outside the optic disc region. Thereby, it becomes possible to contribute to the identification of the symptom of the future disease estimated according to whether or not the papilla bleeding region is within the optic disc region and the determination of the future treatment policy.

In the ophthalmic information processing method according to the embodiment, the position information includes information indicating whether the representative position is inside the optic disc margin or outside the optic disc margin.

According to such an aspect, it becomes easy to grasp whether the papillary bleeding region is inside the optic disc margin or outside the optic disc margin. Thereby, it becomes possible to contribute to the identification of the symptom of the future disease estimated according to whether or not the papillary bleeding region is within the optic disc margin and the decision of the future treatment policy.

In the ophthalmic information processing method according to the embodiment, the shape information includes at least one of the ellipticity of the papillary bleeding region and the area of the papillary bleeding region.

According to such an aspect, it becomes easy to grasp the shape and size of the papillary bleeding region. As a result, it becomes possible to contribute to the identification of the symptoms of future diseases estimated according to the shape and size of the papillary bleeding area and the determination of future treatment policies.

In the ophthalmic information processing method according to the embodiment, the occurrence information includes at least one of the occurrence frequency of the papillary bleeding region and the occurrence interval of the papillary bleeding region.

According to such an aspect, it becomes easy to grasp the occurrence state of the papillary bleeding region. As a result, it will be possible to contribute to the identification of the symptoms of future diseases estimated according to the occurrence of the papillary bleeding area and the decision of the future treatment policy.

The ophthalmologic information processing according to the embodiment includes an alignment step of aligning the OCT data of the eye to be examined with the front image of the eye to be examined in which the papillary bleeding region detected in the detection step is visualized, and the analysis step is a position. At least one of position information, shape information, and generation information is generated using the OCT data aligned with the front image in the alignment step.

According to such an aspect, since the papillary bleeding region in the front image is aligned with the OCT data, the papillary bleeding region is defined in the OCT coordinate system that defines the OCT data acquired with high reproducibility and high accuracy. It will be possible to identify. As a result, it becomes possible to quantitatively obtain the analysis result of the papillary bleeding region with high reproducibility and high accuracy.

The program according to the embodiment causes a computer to execute each step of the ophthalmologic information processing method according to any one of the above.

According to such an aspect, the presence or absence of papillary bleeding in the frontal image of the fundus of the eye to be inspected is determined in the determination step, and the papillary bleeding in the frontal image with respect to the frontal image determined to have papillary bleeding in the detection step. The area is detected. At this time, in the papillary bleeding region detection model in the detection step, the learning parameters are updated specifically for the detection of the papillary bleeding region by machine learning using only the fundus image in which the papillary bleeding region exists. As a result, the detection accuracy of the papillary bleeding region can be improved. Therefore, it is possible to detect the papillary bleeding region in the fundus image with higher reproducibility and accuracy as compared with the case of detecting the papillary bleeding region in the fundus image of the test eye using one trained model. ..

<Others>
The embodiments shown above are merely examples for carrying out the present invention. A person who intends to carry out the present invention can make arbitrary modifications, omissions, additions, etc. within the scope of the gist of the present invention.

In some embodiments, a program is provided for causing a computer to execute the control method of the ophthalmologic device described above. Such programs can be stored on any non-temporary recording medium that can be read by a computer. Examples of the recording medium include a semiconductor memory, an optical disk, a magneto-optical disk (CD-ROM / DVD-RAM / DVD-ROM / MO, etc.), a magnetic storage medium (hard disk / floppy (registered trademark) disk / ZIP, etc.), and the like. Can be used. It is also possible to send and receive this program through a network such as the Internet or LAN.

1, 510, 510 ₁ to 510 _N Ophthalmology equipment 100 OCT unit 210 Control unit 211 Main control unit 230 Image formation unit 300 Data processing unit 310 Judgment device 320 Detector 340 Projection image formation unit 350 Alignment unit 360 Analysis unit 361 Position information Generation unit 362 Shape information generation unit 363 Generation information generation unit 400 Learning unit 410 First learning unit 420 Second learning unit 500, 500a, 500b Ophthalmology system 520 Ophthalmology information processing device 561 Ophthalmology information processing unit E Eye to be inspected Ef Sole

Claims (27)

Using a papillary bleeding determination model obtained by machine learning using multiple frontal images of the fundus labeled with a label indicating the presence or absence of papillary bleeding as the first teacher data, papillary bleeding with respect to the frontal image of the fundus of the eye to be inspected. A judgment device that determines the presence or absence of
A papillary bleeding region detection model obtained by machine learning using a plurality of pairs of images, which are a pair of a frontal image of the fundus and a papillary bleeding region image representing the papillary bleeding region drawn on the frontal image, as second teacher data. Using the detector, a detector for detecting the papillary bleeding region visualized in the front image, which is determined to have the papillary bleeding by the determination device,
Including ophthalmic information processing equipment. The ophthalmic information processing apparatus according to claim 1, wherein the second teacher data includes a front image included in the first teacher data. The ophthalmic information processing apparatus according to claim 1 or 2, comprising a first learning unit that generates the papillary bleeding determination model by performing supervised machine learning using the first supervised data. .. The invention according to any one of claims 1 to 3, further comprising a second learning unit that generates the papillary bleeding region detection model by performing supervised machine learning using the second supervised data. The ophthalmic information processing device described. The ophthalmologic information processing apparatus according to any one of claims 1 to 4, wherein the front image of the fundus is a color front image. By analyzing the frontal image of the fundus of the eye to be inspected, the position information indicating the position of the papillary bleeding region in the frontal image of the fundus of the eye to be inspected, the shape information representing the shape of the papillary bleeding region, and the papillary bleeding region. The ophthalmologic information processing apparatus according to any one of claims 1 to 5, wherein the ophthalmologic information processing apparatus includes an analysis unit that generates at least one of the occurrence information indicating the occurrence status of the above. The ophthalmologic information processing apparatus according to claim 6, wherein the position information includes information indicating the direction of a representative position of the papillary bleeding region with respect to a reference position. The ophthalmologic information processing apparatus according to claim 6, wherein the position information includes information indicating whether the representative position of the optic disc bleeding region is inside the optic disc region or outside the optic disc region. The ophthalmologic information processing apparatus according to claim 8, wherein the position information includes information indicating whether the representative position is inside the optic disc margin or outside the optic disc margin. The ophthalmic information processing apparatus according to any one of claims 6 to 9, wherein the shape information includes at least one of the ellipticity of the papillary bleeding region and the area of the papillary bleeding region. .. The ophthalmologic information processing according to any one of claims 6 to 10, wherein the occurrence information includes at least one of the occurrence frequency of the papillary bleeding region and the occurrence interval of the papillary bleeding region. Device. Includes an alignment section that aligns the OCT data of the eye to be inspected with the frontal image of the inspected eye in which the papillary bleeding region detected by the detector is visualized.
The analysis unit is characterized in that at least one of the position information, the shape information, and the generation information is generated by using the OCT data aligned with the front image by the alignment unit. The ophthalmic information processing apparatus according to any one of claims 6 to 11. An imaging unit that photographs the fundus of the eye to be inspected,
The ophthalmic information processing apparatus according to any one of claims 1 to 12.
Including ophthalmic equipment. An imaging unit that photographs the fundus of the eye to be inspected,
The OCT unit that acquires the OCT data by performing optical coherence tomography on the eye to be inspected, and
The ophthalmic information processing apparatus according to claim 12,
Including ophthalmic equipment. Using a papillary bleeding determination model obtained by machine learning using multiple frontal images of the fundus labeled with a label indicating the presence or absence of papillary bleeding as the first teacher data, papillary bleeding with respect to the frontal image of the fundus of the eye to be inspected. Judgment step to determine the presence or absence of
A papillary bleeding region detection model obtained by machine learning using a plurality of pairs of images, which are a pair of a frontal image of the fundus and a papillary bleeding region image representing the papillary bleeding region drawn on the frontal image, as second teacher data. Using the detection step of detecting the papillary bleeding region visualized in the front image where it is determined that the papillary bleeding is present in the determination step,
Ophthalmic information processing methods, including. The ophthalmic information processing method according to claim 15, wherein the second teacher data includes a front image included in the first teacher data. The ophthalmic information processing method according to claim 15, further comprising a first learning step of generating the papillary bleeding determination model by performing supervised machine learning using the first supervised data. .. 15. The described ophthalmic information processing method. The ophthalmologic information processing method according to any one of claims 15 to 18, wherein the frontal image of the fundus is a color frontal image. By analyzing the frontal image of the fundus of the eye to be inspected, the position information indicating the position of the papillary bleeding region in the frontal image of the fundus of the eye to be inspected, the shape information representing the shape of the papillary bleeding region, and the papillary bleeding region. The ophthalmologic information processing method according to any one of claims 15 to 19, further comprising an analysis step for generating at least one of the occurrence information representing the occurrence status of the above. The ophthalmologic information processing method according to claim 20, wherein the position information includes information indicating the direction of a representative position of the papillary bleeding region with respect to a reference position. The ophthalmologic information processing method according to claim 20, wherein the position information includes information indicating whether the representative position of the optic disc bleeding region is inside the optic disc region or outside the optic disc region. 22. The ophthalmologic information processing method according to claim 22, wherein the position information includes information indicating whether the representative position is inside the optic disc margin or outside the optic disc margin. The ophthalmic information processing method according to any one of claims 20 to 23, wherein the shape information includes at least one of the ellipticity of the papillary bleeding region and the area of the papillary bleeding region. .. The ophthalmologic information processing according to any one of claims 20 to 24, wherein the occurrence information includes at least one of the occurrence frequency of the papillary bleeding region and the occurrence interval of the papillary bleeding region. Method. Includes an alignment step that aligns the OCT data of the inspected eye with the frontal image of the inspected eye in which the papillary bleeding region detected in the detection step is visualized.
The analysis step is characterized in that at least one of the position information, the shape information, and the generation information is generated by using the OCT data aligned with the front image in the alignment step. The ophthalmic information processing method according to any one of claims 20 to 25. A program comprising causing a computer to execute each step of the ophthalmic information processing method according to any one of claims 15 to 26.

Images