JP2023076025A - Ophthalmologic apparatus, control method of ophthalmologic apparatus, and program - Google Patents

Abstract

To provide an ophthalmologic apparatus capable of reducing a load of a manual operation by an examiner.SOLUTION: An ophthalmologic apparatus includes: a head part for imaging an eye to be examined; a support part for supporting the face of a subject; a moving part capable of moving at least one of the head part and the support part; an observation part for executing imaging from a head part side to a support part side; an estimation part for estimating at least one of the information as to whether or not the subject is placing his or her face on the support part and the information on the position of the eye to be examined using a learned model that has executed learning with an image group including an image of the eye to be examined including the pupil and an image not including the pupil as inputs; and a control part for controlling the moving part on the basis of a result estimated by the estimation part using an image captured by the observation part as an input to the learned model.SELECTED DRAWING: Figure 7

Description

The present invention relates to an ophthalmologic apparatus, an ophthalmologic apparatus control method, and a program.

In recent years, an apparatus (OCT apparatus) using optical coherence tomography (OCT), which acquires a tomographic image using interference of low-coherence light, has been put into practical use. Since this OCT apparatus can obtain a tomographic image with a resolution approximately equal to the wavelength of light incident on an object to be inspected, it is possible to obtain a tomographic image of the object to be inspected with high resolution.

The OCT apparatus is particularly useful as an ophthalmic apparatus for obtaining a tomographic image of the retina located on the fundus. Furthermore, as other ophthalmologic devices, a fundus camera (a device for taking a two-dimensional image of the fundus) and a scanning laser ophthalmoscope (SLO) are also useful devices. Refractometers (eye refractive power measuring devices), tonometers (tonometers), and the like are also useful as ophthalmic devices. These composite devices are also useful devices.

In such an ophthalmologic apparatus, it is important to accurately align (align) the imaging unit (mainly the measurement optical system) of the apparatus with the eye to be examined in order to photograph the eye to be examined. In Patent Document 1, an alignment index is projected onto the cornea of an eye to be inspected, the reflected light is divided, an image is captured by an imaging element through an observation optical system, and the position of the divided alignment index image is used to determine the relative position of the apparatus and the eye to be inspected. A position-sensing and automatic alignment ophthalmic device is described. In order to achieve the required alignment accuracy in such a technique, an observation optical system capable of observing the subject's eye with high definition is required. Since it is necessary to reduce the field of view per pixel in order to observe the subject's eye with high definition, the observation range of the observation optical system in these devices is often limited to the subject's eye and its surroundings.

JP 2010-162424 A

However, since there are individual differences in the size of the face, the subject's eyes may fall out of the observation range of the observation optical system when the subject puts his/her face on the face support portion of the apparatus. . In such a case, the examiner needs to move the face support part and the measurement optical system of the apparatus to bring the subject's eye into the observation range, and then start the automatic alignment. Further, in the conventional automatic alignment technique, it is assumed that the examinee puts his/her face on the face support portion of the apparatus at the start of the automatic alignment. Therefore, it is necessary for the examiner to start the automatic alignment after confirming that the face of the subject is placed on the face support portion. Such manual operations before automatic alignment may become a burden on the examiner.

SUMMARY OF THE INVENTION One object of an embodiment of the present invention is to provide an ophthalmologic apparatus, an ophthalmologic apparatus control method, and a program that can reduce the burden of manual operations by an examiner.

An ophthalmologic apparatus according to an embodiment of the present invention includes a head section for photographing an eye to be examined, a support section for supporting the face of a subject, and a moving section capable of moving at least one of the head section and the support section. and an observation unit that photographs from the head unit side toward the support unit side, and an image group that includes an image of the subject's eye including the pupil and an image that does not include the pupil. an estimating unit for estimating at least one of information regarding whether or not the subject rests his/her face on the supporting unit and information regarding the position of the subject's eye; and a control unit that controls the moving unit based on the results estimated by the estimating unit that are used as inputs to the model.

According to one embodiment of the present invention, it is possible to reduce the burden of manual operation by the examiner.

1 shows an example of a schematic configuration of an OCT apparatus according to Example 1. FIG. 1 shows an example of a schematic optical configuration of an imaging unit according to Example 1. FIG. An example of display of an anterior eye observation image, a fundus front image, and a tomographic image is shown. 4 is a block diagram showing an example of a control unit according to Embodiment 1; FIG. 4 shows a plurality of examples of images for anterior eye observation. 1 is a schematic diagram of a neural network model; FIG. 4 is a flow chart showing a series of inspection processes. An example of a prism-equipped lens having an image split prism and an example of an anterior eye observation image obtained using the prism-equipped lens are shown. 10 is a flowchart showing a series of inspection processes according to Modification 1. FIG.

Exemplary embodiments for carrying out the invention will now be described in detail with reference to the drawings. However, the dimensions, materials, shapes, relative positions of components, etc. described in the following examples are arbitrary and can be changed according to the configuration of the device to which the present invention is applied or various conditions. Also, the same reference numbers are used in the drawings to indicate identical or functionally similar elements.

In the following, an image related to the fundus refers to an image containing information of the fundus. For example, images related to the fundus include tomographic images and three-dimensional tomographic images that are cross-sectional images of the fundus, fundus frontal images that are two-dimensional images of the fundus, images including ring images, and En-Face images and OCTA frontal images described later. images, OCTA tomographic images, and the like. Further, the image for anterior eye observation refers to an image acquired by the infrared CCD 112 for anterior eye observation, which will be described later.

Furthermore, a machine learning model refers to a learning model by a machine learning algorithm. Specific algorithms of machine learning include nearest neighbor method, naive Bayes method, decision tree, support vector machine, and the like. Another example is deep learning, in which neural networks are used to generate feature quantities and connection weighting coefficients for learning. As appropriate, any of the above algorithms can be used and applied to the following embodiments and modifications. Also, teacher data refers to learning data, and is composed of a pair of input data and output data. Further, correct data means output data of learning data (teaching data).

Note that a trained model is a model obtained (learned) by training a machine learning model based on an arbitrary machine learning algorithm using appropriate learning data in advance. However, although a trained model is obtained in advance using appropriate learning data, it is possible to perform additional learning instead of not performing further learning. Additional learning can also take place after the device has been installed at the point of use.

(Example 1)
An ophthalmologic imaging apparatus and a control method thereof will be described below as an example of an ophthalmologic apparatus and a control method thereof according to Embodiment 1 of the present invention. The ophthalmologic imaging apparatus according to the present embodiment is an ophthalmologic imaging apparatus for imaging the fundus of an eye to be examined, and particularly relates to an ophthalmologic imaging apparatus used to obtain a front fundus image and a tomographic image, which are two-dimensional images of the fundus of the eye to be examined. . In this embodiment, as an example of an ophthalmologic imaging apparatus, an optical coherence tomography apparatus (OCT apparatus) that aligns the apparatus with respect to the subject's eye based on an anterior eye observation image using a trained model will be described.

(Schematic configuration of the device)
A schematic configuration of an OCT apparatus according to this embodiment will be described with reference to FIG. FIG. 1 is a side view showing an example of a schematic configuration of an OCT apparatus 1. FIG. The OCT apparatus 1 is provided with an imaging unit 10 , a control unit 20 , a display unit 40 and an input unit 50 . The photographing unit 10 is provided with an optical head unit 100 , a stage unit 150 , a base unit 190 containing a spectroscope, which will be described later, and a face receiving unit 160 .

The optical head unit 100 includes a measurement optical system for irradiating the eye to be inspected with light, detecting the return light from the eye to be inspected, and photographing an anterior eye observation image, a fundus front image, and a tomographic image. The stage section 150 is an example of a moving section capable of moving the optical head section 100 in the xyz directions in the figure using a motor (not shown). The face receiving part 160 is a chin rest that can help fix the subject's eyes (eyes to be examined) by fixing the subject's chin and forehead. Further, the face receiving portion 160 includes a driving member such as a motor (not shown), and the driving member functions as an example of a moving portion capable of moving the face receiving portion 160 in the y direction in the figure.

The control unit 20 is connected to the imaging unit 10, the display unit 40, and the input unit 50, and can control them. The control unit 20 can, for example, control the movement of the stage unit 150 and align the optical head unit 100 with respect to the subject's eye. The control unit 20 can also generate an anterior eye observation image, a fundus front image, a tomographic image, etc. based on the data acquired by the imaging unit 10 .

The control unit 20 can be configured using a general computer including a processor and memory, but may be configured as a dedicated computer for the OCT apparatus 1 . Note that the control unit 20 may not only be a separate (external) computer to which the imaging unit 10 is communicably connected, but may also be a built-in (inside) computer of the imaging unit 10 . Also, the control unit 20 may be, for example, a personal computer, and a desktop PC, a notebook PC, a tablet PC (portable information terminal), or the like may be used.

The display unit 40 is configured by an arbitrary monitor, and displays various information such as subject information, various images, and a mouse cursor according to the operation of the input unit 50 under the control of the control unit 20 . The input unit 50 is an input device for giving instructions to the control unit 20, and specifically includes a keyboard and a mouse. Note that the display unit 40 may be a touch panel display, and in this case, the display unit 40 is also used as the input unit 50 .

In this embodiment, the imaging unit 10, the control unit 20, the display unit 40, and the input unit 50 are configured separately, but they may be configured partially or wholly as a single unit. Also, the control unit 20 may be connected to other imaging devices, storage devices, and the like (not shown).

(Configuration of measurement optical system and spectrometer)
Next, with reference to FIG. 2, a configuration example of the measurement optical system and the spectroscope of this embodiment will be described. FIG. 2 shows an example of a schematic optical configuration of the imaging unit 10. As shown in FIG. First, the internal configuration of the optical head section 100 will be described.

In the optical head unit 100, an objective lens 101-1 is arranged facing the eye E to be inspected, and a first dichroic mirror 102 is arranged as first optical path separating means on the optical axis thereof. The optical path from the objective lens 101-1 is divided by the first dichroic mirror 102 into the measurement optical path L1 of the OCT optical system, the optical path L2 for fundus observation and fixation lamp, and the observation optical path L3 for the anterior eye. It branches for each wavelength band. A second dichroic mirror 103 is arranged as a second optical path separating means in the reflection direction of the first dichroic mirror 102 . The optical path in the reflection direction of the first dichroic mirror 102 is split by the second dichroic mirror 103 into a measurement optical path L1 for the OCT optical system and an optical path L2 for fundus observation and fixation lamp for each wavelength band of light.

In the configuration according to the present embodiment, the optical path L3 for observation of the anterior eye is in the transmission direction of the first dichroic mirror 102, the measurement optical path L1 for the OCT optical system and the optical path L2 for fundus observation and fixation lamp are in the reflection direction. is placed. In addition, the measurement optical path L1 of the OCT optical system is arranged in the transmission direction of the second dichroic mirror 103, and the optical path L2 for fundus observation and fixation lamp is arranged in the reflection direction. However, the optical paths provided in the transmission direction and the reflection direction of each dichroic mirror may be opposite to each other.

Lenses 101-2, 108, 107 and a third dichroic mirror 104 are arranged in order from the second dichroic mirror 103 on the optical path L2 for fundus observation and internal fixation lamp. The optical path L2 for fundus observation and internal fixation lamp is divided into the optical path to the fundus observation CCD 105 and the optical path to the fixation lamp 106 by the third dichroic mirror 104, which is a third optical path separating means, for each wavelength band of light rays. is branched to In this embodiment, a fixation lamp 106 is arranged in the transmission direction of the third dichroic mirror 104, and a CCD 105 for fundus observation is arranged in the reflection direction. Note that the CCD 105 may be arranged in the transmitting direction of the third dichroic mirror 104 and the fixation lamp 106 may be arranged in the reflecting direction.

A lens 107 is a focus lens and is used for focus adjustment of the fixation lamp and the optical path L2 for observing the fundus. The lens 107 can be driven in the optical axis direction by a motor (not shown) or the like controlled by the control unit 20 .

The CCD 105 has sensitivity to the wavelength of light emitted from a fundus observation illumination light source (not shown), specifically around 780 nm. On the other hand, the fixation lamp 106 emits visible light to prompt the subject to fixate. The control unit 20 can generate a fundus front image, which is a two-dimensional image of the fundus of the subject's eye E, based on the signal output from the CCD 105 .

Lenses 109, 110, 111 and an infrared CCD 112 for anterior eye observation are provided in order from the first dichroic mirror 102 on the anterior eye observation optical path L3. Also, two anterior eye observation light sources 122 are provided so as to sandwich the objective lens 101-1 in the x direction of the figure.

The infrared CCD 112 is an example of an observation unit that captures an image of the front of the optical head unit 100, in other words, the direction in which the subject is arranged, via lenses 109, 110, and 111. FIG. The infrared CCD 112 photographs the anterior segment of the subject's eye, particularly when the subject's eye is positioned appropriately with respect to the optical head unit 100 . The infrared CCD 112 has sensitivity to the wavelength of the anterior eye observation light source 122, specifically around 970 nm. The control unit 20 generates an image in front of the optical head unit 100 including an anterior eye observation image based on the signal output from the infrared CCD 112 . Note that the anterior eye observation light source 122 can project an index onto the subject's eye so that a corneal bright spot appears in the anterior eye observation image.

Optical members constituting the OCT optical system are arranged in the measurement optical path L1 of the OCT optical system, and the OCT optical system has a configuration for taking a tomographic image of the fundus of the eye E to be examined. More specifically, the OCT optical system is used to obtain interference signals for forming tomographic images.

The measurement optical path L1 includes, in order from the second dichroic mirror 103, a lens 101-3, a mirror 113, an X scanner 114-1 and a Y scanner 114-2 for scanning the fundus of the eye E to be examined, and a lens 115. , 116 are arranged.

The X scanner 114-1 and the Y scanner 114-2 are an example of scanning means for OCT measurement light, and are arranged to scan the fundus of the eye E to be examined with the measurement light. The X scanner 114-1 is used to scan the measurement light in the x direction, and the Y scanner 114-2 is used to scan the measurement light in the y direction. In this embodiment, the X scanner 114-1 and Y scanner 114-2 are configured by galvanometer mirrors, but the X scanner 114-1 and Y scanner 114-2 may use any deflection means according to the desired configuration. may be configured Further, the scanning means for measuring light may be constituted by deflection means capable of deflecting light in two-dimensional directions with a single mirror such as a MEMS mirror.

The lens 115 is a focus lens and is used to focus the light from the light source 118 emitted from the optical fiber 117-2 connected to the optical coupler 117 onto the fundus. The lens 115 can be driven in the optical axis direction by a motor (not shown) or the like controlled by the controller 20 . Due to this focusing adjustment, the light returning from the fundus is formed into a spot on the tip of the optical fiber 117-2 and is incident thereon.

Next, the configuration of the optical path from the light source 118, the reference optical system, and the spectroscope will be described. A light source 118 is an SLD (Super Luminescent Diode), which is a typical low coherent light source. In this embodiment, an SLD with a central wavelength of 855 nm and a wavelength bandwidth of about 100 nm is used as the light source 118 . As the light source 118, SLD is selected in this embodiment, but ASE (Amplified Spontaneous Emission) or the like may be used as long as it can emit low coherent light.

Light source 118 is connected to optical coupler 117 via optical fiber 117-1. The optical coupler 117 is integrated with single-mode optical fibers 117-1 to 117-4. Light emitted from the light source 118 is split into measurement light and reference light by the optical coupler 117. The measurement light passes through the optical fiber 117-2 to the measurement light path L1, and the reference light passes through the optical fiber 117-3 to the reference optical system. led to. The measurement light passes through the measurement optical path L1, is irradiated to the fundus of the subject's eye E, and reaches the optical coupler 117 through the same optical path due to reflection and scattering by the retina.

On the other hand, the reference light split by the optical coupler 117 and guided by the optical fiber 117-3 is emitted to the reference optical system. The reference optical system includes a lens 121, a dispersion compensating glass 120, and a mirror 119 (reference mirror) in this order from the output end of the optical fiber 117-3.

The reference light emitted from the output end of the optical fiber 117-3 reaches the mirror 119 via the lens 121 and the dispersion compensating glass 120 and is reflected. A dispersion compensating glass 120 is inserted in the optical path to match the dispersion of the measurement light and the reference light. The reference light reflected by mirror 119 returns along the same optical path and reaches optical coupler 117 . The mirror 119 can be driven in the optical axis direction indicated by the arrow in the drawing by a motor (not shown) controlled by the controller 20 or the like.

The return light of the measurement light returned from the subject's eye E and the reference light reflected from the mirror 119 are combined by the optical coupler 117 to form interference light. Here, interference occurs when the optical path length of the measurement light and the optical path length of the reference light are substantially the same. Therefore, by moving the mirror 119 in the optical axis direction by the motor or the like described above and matching the optical path length of the reference light with the optical path length of the measurement light that varies depending on the eye E to be examined, interference light between the measurement light and the reference light can be eliminated. can be generated. The interference light is guided to spectroscope 180 via optical fiber 117-4.

The spectroscope 180 is an example of a detector that detects interference light. The spectroscope 180 is provided with lenses 181 and 183 , a diffraction grating 182 and a line sensor 184 . The interference light emitted from the optical fiber 117-4 becomes substantially parallel light through the lens 181, is dispersed by the diffraction grating 182, and is imaged on the line sensor 184 by the lens 183. FIG.

Information about the luminance distribution in the signal acquired by the line sensor 184, which is a detector, is output to the control unit 20 as an interference signal. The controller 20 can generate a tomographic image based on the interference signal output from the line sensor 184 .

Acquiring tomographic information from one point on the subject's eye E is called an A scan. Further, the luminance distribution on the line sensor 184 obtained at a certain position in the x direction is subjected to Fast Fourier Transform (FFT), and the obtained linear luminance distribution is converted into density or color information, and then A-scan is performed. called an image. Furthermore, a two-dimensional image in which a plurality of A-scan images are arranged is called a B-scan image (tomographic image).

Further, after taking a plurality of A-scan images for constructing one B-scan image, by moving the scanning position in the y-direction and scanning in the x-direction again, it is possible to obtain a plurality of B-scan images. can. A three-dimensional tomographic image can be constructed by arranging a plurality of B-scan images in the y direction. The control unit 20 causes the display unit 40 to display a plurality of B-scan images or a three-dimensional tomographic image constructed from a plurality of B-scan images, so that the examiner can use these images for diagnosis of the eye E to be examined. can.

In this embodiment, a Michelson interferometer is configured by a measurement optical system, a reference optical system, and a spectroscope optical system, which are composed of members arranged on the measurement optical path L1. Alternatively, a Mach-Zehnder interferometer may be used as the interferometer.

FIG. 3 shows an example of an anterior eye observation image 300, a fundus front image 301, and a B-scan image 302, which is a tomographic image, displayed on the display unit 40. FIG. The anterior eye observation image 300 is an image generated by processing the output of the infrared CCD 112, the fundus front image 301 is an image generated by processing the output of the CCD 105, and the B-scan image 302 is the line sensor 184. is an image generated by processing the output of , as described above.

(Configuration of control unit)
Next, the configuration of the control unit 20 will be described with reference to FIG. FIG. 4 is a block diagram showing an example of the configuration of the control section 20. As shown in FIG. The control unit 20 is provided with an acquisition unit 21 , a drive control unit 22 , an image generation unit 23 , a positional deviation estimation unit 24 and a storage unit 25 .

Acquisition unit 21 can acquire various signals output from CCD 105 , infrared CCD 112 , and line sensor 184 . The acquisition unit 21 can also acquire instructions from the operator through the input unit 50 . Furthermore, the acquisition unit 21 can acquire various information such as subject information stored in the storage unit 25, various images such as tomographic images, and various images generated by the image generation unit 23.

The drive control unit 22 controls driving of various components in the imaging unit 10 . Specifically, the drive control unit 22 controls the driving of, for example, the stage unit 150, the face receiving unit 160, the lens 107, the X scanner 114-1, the Y scanner 114-2, the lens 115, the mirror 119, and the like. is an example of a movement control means capable of The drive control unit 22 can also control the lighting and extinguishing of the anterior eye observation light source 122 , the fundus illumination light source (not shown), the fixation lamp 106 , and the light source 118 . Further, the drive control unit 22 controls the moving unit for alignment of the optical head unit 100 with respect to the subject's eye E in the xyz directions based on the estimation result by the positional deviation estimation unit 24, which will be described later. Control of the moving unit for alignment by the drive control unit 22 will be described later.

The image generation unit 23 generates a fundus front image based on the output signal of the CCD 105 acquired by the acquisition unit 21 , generates an anterior eye observation image based on the output signal of the infrared CCD 112 , and outputs the image from the line sensor 184 . A tomographic image can be generated based on the interference signal. Any known method may be used for generating an ophthalmologist image such as a tomographic image or a fundus front image from ophthalmologic information such as tomographic data or fundus data of the eye E to be examined.

Based on the image generated by the image generating unit 23 and a learned model described later, the positional deviation estimating unit 24 obtains information regarding whether or not the subject puts his/her face on the face receiving unit 160 and information regarding the position of the eye E to be examined. It is an example of an estimation unit that estimates at least one of information. In this embodiment, the positional deviation estimating unit 24 uses the anterior eye observation image as an input to the trained model to estimate the presence probability of the subject and the positional deviation of the pupil center between the optical head unit 100 and the eye E to be inspected. presume. Details of the estimation processing by the positional deviation estimation unit 24 will be described later.

The storage unit 25 can store various types of information and various images that have been generated. In addition, the storage unit 25 can store identification information and the like of the subject. Furthermore, the storage unit 25 can store a program for photographing and the like.

Each component other than the memory|storage part 25 of the control part 20 may be comprised by the software module performed by processors, such as CPU(Central Processing Unit) and MPU(Micro Processing Unit). Note that the processor may be, for example, a GPU (Graphical Processing Unit), an FPGA (Field-Programmable Gate Array), or the like. Also, each component may be configured by a circuit or the like that performs a specific function, such as an ASIC. The storage unit 25 may be configured by an arbitrary storage medium such as an optical disk such as a hard disk or a memory, for example.

(Image for anterior eye observation)
Next, the anterior eye observation image will be described with reference to FIGS. 5(a) to 5(g).

FIG. 5A shows an example of an anterior eye observation image in which the positional relationship between the optical head unit 100 and the center of the pupil of the subject's eye E is aligned. In this example, the pupil of the subject's eye E is present substantially in the center of the image. In addition, corneal bright points, which are regular reflections of light from the anterior eye observation light source 122 by the cornea, are present on both sides of the pupil.

FIG. 5B shows an example when the optical head unit 100 and the subject's eye E are separated in the z direction. In this example, since the subject's eye E is away from the focal plane of the infrared CCD 112, the image is blurred. In the image shown in FIG. 5(b), the eye E to be examined is far from the optical head unit 100, so the eye E to be examined appears smaller than the eye E to be examined in the image shown in FIG. 5(a). In addition, in the image shown in FIG. 5(b), since the angle of specular reflection by the cornea of the light from the anterior eye observation light source 122 is different, the interval between the corneal bright spots is It is narrower than the interval between corneal bright spots.

FIG. 5C shows an example in which the subject's eye E is shifted downward in the y direction when viewed from the optical head unit 100 . In this example, eyebrows are shown in addition to the pupil of the eye E to be examined. FIG. 5(d) shows an example in which the subject's eye E is further shifted downward in the y direction. In this example, the pupil of the subject's eye E does not exist in the image. In addition, bangs appear in the image. Note that the image shown in FIG. 5(d) is only an example, and it is assumed that the bangs may not appear because the hairstyle varies greatly from subject to subject.

FIG. 5E shows an example in which the subject's eye E is shifted to the upper right in the xy direction when viewed from the optical head unit 100 . In this example, the pupil of the subject's eye E exists only partially at the edge of the image, and the subject's ear appears in the image. FIG. 5(f) is an example in which the subject's eye E is further shifted to the upper right in the xy direction when viewed from the optical head unit 100. FIG. In this example, the pupil of the subject's eye E does not exist in the image, and the subject's ear appears closer to the center of the image.

FIG. 5G shows an example in which the subject does not put his or her face on the face receiving section 160 . In this example, the state of the room in which the OCT apparatus 1 is installed is shown. Also, when the subject does not put his or her face on the face receiving section 160, the distance from the optical head section 100 to the subject is long, so the subject's face is smaller and A blurred image is also assumed.

As described above, different characteristics are seen in the anterior eye observation image depending on the positional relationship between the optical head unit 100 and the subject. These features are important features for accurately constructing a trained model, which will be described later. Note that the diagrams shown here mainly show examples of images relating to the subject's right eye, but in the images relating to the left eye, features that are bilaterally symmetrical with those of the right eye can be seen.

(Configuration of learned model)
Here, a detailed description will be given of a trained model used in the alignment process, which will be described later. In the alignment processing according to the present embodiment, the recognition processing of the anterior eye observation image is implemented using a neural network learned by a machine learning process based on deep learning. FIGS. 6A and 6B show an example of a schematic diagram of a neural network model according to this embodiment. The trained model of this embodiment includes the following two neural network models.

FIG. 6( a ) shows an input of the two-dimensional pixel string data of the image for anterior eye observation, and the presence probability P of the subject, which is a value representing the probability that the subject's face is placed on the face receiving section 160 . An example of an output neural network model is shown. The existence probability P is a value between 0 and 1. The learned model, which outputs the existence probability P of the subject according to the present embodiment, is obtained from various anterior eye observation models obtained when the subject puts the face on the face receiving unit 160 and when the subject does not put the face on the face receiving unit 160. Images are used as training data for learning.

For the input data of the learning data without the face on it, the anterior eye observation data acquired while changing the lighting intensity, the orientation of the OCT device, the arrangement of the fixtures, etc. in various examination rooms where the OCT device can be installed. Contains images. Furthermore, the input data for the learning data may include an image in which the subject is shown but the face is not placed on the face receiving section 160 . For example, an image showing parts other than the face of the subject and an image showing the subject farther than the vicinity of the face receiving part 160 can be included. Existence probability P=0 is given as correct data (output data of learning data) for these images.

The input data of the learning data in which the face is placed includes the optical head unit 100 arranged in various positional relationships for subjects with various conditions such as age, gender, race, hairstyle, and wearing or not wearing a mask. An image for anterior eye observation acquired in the state of This includes an image in which the subject's eye E and the optical head unit 100 are out of position so that the subject's eye does not exist in the image, and only a part of the face other than the eye is shown. can be done. Existence probability P=1 is given as correct data for these images.

FIG. 6(b) shows a neural network model in which two-dimensional pixel row data of an anterior eye observation image is input and the amount of deviation of the subject's eye E from an appropriate alignment position in the coordinate system of the optical head unit 100 is output. Here is an example. Information (Δx, Δy, Δz) on the three-dimensional position is output as the displacement amount. The shift amounts Δx, Δy, and Δz are the relative distances between the center position of the pupil of the subject's eye E in the anterior eye observation image and the optimum position where the optical head unit 100 can most suitably capture an OCT tomographic image. The optimum position can be the relative position of the center of the pupil of the subject's eye E with respect to the optical head unit 100 determined in advance by design.

As the input data for the learning data of the model shown in FIG. 6(b), among the images used as the input data for the learning data of the model shown in FIG. Images can be used. When preparing such an image, the pupil of the subject's eye E and the position of the optical head section 100 should be accurately recorded when acquiring the image while shifting the optical head section 100 with respect to the subject's eye. , the deviation amounts Δx, Δy, and Δz, which are correct data, can be given with high accuracy.

A neural network according to the above basic specifications is a convolutional neural network (CNN) that can recognize patterns flexibly by configuring an intermediate layer after the input layer, for example, by combining a convolutional layer and a pooling layer. Network). Also, for example, the immediate vicinity of the output layer can be configured with a fully connected layer (not shown in particular) suitable for optimum value calculation.

The learning of these models uses the learning data described above to adjust the weighting coefficients of the edges that connect each node so that the relationship between the input layer and the output layer of each neural network is established. It can be performed by a back propagation method (backpropagation method).

Specifically, first, the error between the output data output from the output layer of the neural network according to the input data input to the input layer of the learning model and the correct data is obtained. A loss function may be used to calculate the error between the output data from the neural network and the correct data. Next, based on the obtained error, the error backpropagation method is used to update the connection weighting coefficients between the nodes of the neural network and the like so that the error becomes smaller. The error backpropagation method is a method of adjusting the connection weighting coefficients and the like between nodes of each neural network so as to reduce the above error. In addition to the error backpropagation method, various known learning methods such as so-called layered autoencoder, dropout, noise addition, and sparse regularization may be used together to improve the processing accuracy.

When data is input to a trained model that has been trained using such learning data, data that is highly likely to correspond to the input data according to the design of the trained model and the tendency of training using the learning data. is output. In this embodiment, when an image for anterior eye observation is input to the trained model trained using the learning data described above, the probability that the subject puts his/her face on the face receiving unit 160 (existence probability P ) and the relative distance (shift amounts Δx, Δy, Δz) between the optical head unit 100 and the center of the pupil of the subject's eye E are output.

Although a trained model including two models has been described here, the trained model according to this embodiment does not necessarily have to be divided into two models. When one model is used for estimation, there is a method in which the output of the model is set to four parameters, ie, the existence probability P of the subject and the deviation amounts Δx, Δy, and Δz. At this time, since the deviation amount cannot be defined for an image without the subject, for example, a method of setting the correct data of the deviation amount to an extremely large value is conceivable. In such a case, the positional deviation estimating section 24 ignores the deviation amounts Δx, Δy, and Δz, and determines that the subject does not put the face on the face receiving section 160 based on the presence probability P. good.

Further, without preparing a trained model with the existence probability P of the subject as an output parameter, based on the output from the trained model with the deviation amounts Δx, Δy, and Δz as parameters, the subject is resting his/her face on the face receiving portion 160 or not. At this time, since the deviation amount cannot be defined for an image without the subject, for example, a method of setting the correct data of the deviation amount to an extremely large value is conceivable. In this case, the positional deviation estimating section 24 detects that the subject puts his or her face on the face receiving section 160 when the deviation amounts Δx, Δy, and Δz output from the learned model are larger than a certain value (threshold value). It can be determined that no

Here, the model that outputs the relative distance of the subject's eye to the optical head unit 100 as the shift amounts Δx, Δy, and Δz has been described. However, a model that outputs the direction in which the subject's eye is positioned with respect to the optical head unit 100 instead of the relative distance may be applied to this embodiment as the amount of deviation.

In particular, when the positional relationship between the optical head unit 100 and the subject's eye is so deviated that the subject's eye does not exist in the image, it is difficult to accurately estimate the relative distance between them. Further, in this embodiment, when the positional relationship between the optical head unit 100 and the subject's eye is deviated, the amount of deviation is estimated again after bringing the optical head unit 100 closer to the subject's eye E as described later. It is not necessary to accurately estimate the relative distance when the positional relationship between the two is greatly deviated. Therefore, a model that outputs the direction of the pupil center of the subject's eye with respect to the optical head unit 100 may be used. In this case, a method of outputting three parameters in the x, y, and z directions and learning +1 or -1 for each parameter according to the direction of the pupil as correct data is conceivable. The input data for learning data may be the same data as described above.

In addition, six parameters indicating respective priorities of up, down, left, right, front and rear can be output from the model. In this case, the correct answer data should be 1 only in the direction in which the optical head unit 100 should be most preferentially moved and 0 in the rest. By using such a model, the optical head section 100 can be moved in the direction estimated to have the highest priority.

Furthermore, seven parameters, including a parameter indicating the state of completion of alignment, may be output from the model. In this case, the correct data indicating the state in which the alignment is completed may indicate the state in which the alignment is completed as 1, and the state in which the alignment has not been completed as 0. Furthermore, as described above, when the estimation is performed using one model, the subject absence flag indicating that the subject does not put the face on the face receiving section 160 may be used as the eighth parameter. In this case, the correct data indicating the subject absence flag is, for example, a binary value that is 0 when the subject existence probability P is equal to or greater than a certain value and 1 when it is less than a certain value. It can be information. In this example, cases where the parameter is 1 are exclusive, so there are no undefined parameters when there is no subject.

Also, the output of the model and the corresponding correct data may be control information for moving the optical head unit 100 and the face receiving unit 160 instead of the information regarding the position of the pupil. In this case, for example, the optimum control order and speed in consideration of the noise caused by driving the optical head unit 100 and the subject's fear of movement of the optical head unit 100 may be output.

In addition, parameters of model outputs and correct data may be added. For example, a parameter indicating whether the user is wearing spectacles or sunglasses may be output as an additional parameter. In this case, the drive control unit 22 moves, for example, the optical head unit 100 and the face receiving unit 160 with respect to the subject wearing spectacles or sunglasses, according to the output of the positional deviation estimation unit 24 based on the parameter. It is possible to control the moving part such as the stage part 150 so as not to perform the Also, the control unit 20 can cause the display unit 40 to display a warning message to remove the spectacles or sunglasses before photographing. In this case, an image of the user wearing eyeglasses or sunglasses and information on whether or not the eyeglasses or sunglasses are worn may be given as input data and output data of the learning data during learning.

Similarly, the model may output a parameter indicating whether the eyes are open as an additional parameter. The drive control unit 22 controls the stage so as not to move the optical head unit 100 and the face receiving unit 160 for the subject whose eyes are closed according to the output of the positional deviation estimation unit 24 based on the parameters. A moving part, such as part 150, can be controlled. In addition, the control unit 20 can cause a speaker (not shown) or the like to output a sound prompting the user to open their eyes. In this case, similarly, an image with the eyes closed and information on the open/closed state of the eyes may be given as input data and output data of the learning data.

Further, the model may output both the estimated right eye pupil position and the estimated left eye pupil position separately. With such a configuration, even when the optical head unit 100 is positioned on the right eye side, control can be performed to move the optical head unit 100 to the pupil of the left eye at the shortest distance. In this case, during learning, the estimated pupillary position of the right eye and the estimated pupillary position of the left eye may be given as output data of learning data. Note that the model may output parameters other than the parameters described above.

Also, here, the anterior eye observation image is used as input data for the model without special processing, but the result of some processing or analysis may be used as the input data. For example, an image with adjusted brightness or contrast, an image with detected edges, or an image with some feature points extracted by rule-based image processing (including data corresponding to two-dimensional coordinates) can be used as input data. good. Here, rule-based processing refers to processing that uses known regularity, such as the regularity of the shape of the retina.

Furthermore, information other than image-related information may be added as input data. For example, by adding the current moving direction and moving speed of the optical head unit 100 to the input data, the amount of deviation of the pupil at the timing that is actually reflected in the movement of the optical head unit 100 after the image acquisition time is predicted. be able to.

In addition, whether or not a mydriatic agent is applied to the subject's eye may be added to the input data. Since the pupil of the subject's eye to which the mydriatic agent is instilled is enlarged, an image having different characteristics can be obtained as an anterior eye observation image when such a subject's eye is photographed. Therefore, if information on whether or not a mydriatic agent is applied is added as an input when constructing a trained model, it may be possible to construct a more accurate model. Also, other parameters not listed here may be added as input data.

As for the processing algorithm, in addition to deep learning using the illustrated neural network, other processing algorithms such as support vector machines and Bayesian networks may be applied. In addition, since the anterior eye observation images are moving images that are assumed to be acquired continuously, an RNN (Recurrent Neural Network) suitable for handling time-series data may be used. Since the information of the previous image frame is used for the input of the RNN, the use of the RNN makes it possible to perform estimation based on changes over time.

In this case, for example, it is possible to construct a model that outputs a higher presence probability P of the subject when a plurality of frames of images in which the subject's face is placed are input in succession. In addition, it is possible to construct a model in which the values of the estimated relative position of the pupil (deviation amounts Δx, Δy, Δz) do not change abruptly and greatly, and it is possible to prevent erroneous estimation of the position of the pupil. There is

(Tomographic image capturing flow)
Next, with reference to FIG. 7, the flow of image capturing for inspection according to the present embodiment will be described in order of steps. FIG. 7 is a flow chart showing a series of inspection processes according to the present embodiment.

First, in step S<b>701 , the examiner inputs subject information to the control unit 20 through the input unit 50 . Here, the configuration is such that the inspection is started by the operation.

Next, in step S702, the drive control section 22 moves the face receiving section 160 and the optical head section 100 to predetermined positions. The predetermined position may be, for example, a position pre-designed for a subject with a most standard face size.

Next, in step S703, the image generator 23 generates an anterior eye observation image. More specifically, the drive control unit 22 turns on the anterior eye observation light source 122 , and the image generation unit 23 generates the anterior eye observation image based on the signal output from the infrared CCD 112 .

Next, in step S704, the positional deviation estimator 24 uses the learned model to estimate the existence probability P of the subject and the pupil information including the deviation amounts Δx, Δy, and Δz of the pupil of the subject's eye with respect to the optical head unit 100. do. Specifically, the positional deviation estimating unit 24 inputs the anterior eye observation image acquired in step S703 to the above-described trained model, and estimates these pieces of information using the obtained output.

Next, in step S<b>705 , the positional deviation estimator 24 determines (estimates) whether or not the face of the subject is placed on the face receiving section 160 . Specifically, if the value of the existence probability P of the subject estimated in step S704 is equal to or greater than a certain value, the positional deviation estimating unit 24 determines that the subject is resting his or her face on it. If the positional deviation estimating unit 24 determines in step S705 that the subject does not put the face on it, it determines that the alignment cannot be started yet, and the process returns to step S703. On the other hand, if the positional deviation estimating unit 24 determines in step S705 that the subject puts his/her face on it, the process proceeds to step S706.

In step S706, the positional deviation estimator 24 determines the alignment state of the apparatus. Specifically, the positional deviation estimation unit 24 determines the alignment information of the apparatus based on the deviation amounts Δx, Δy, and Δz estimated in step S704. The positional deviation estimator 24 makes the determination, for example, by comparing the deviation amounts Δx, Δy, Δz with the thresholds Tx, Ty, Tz. The threshold can be a value predetermined by design so that an OCT tomographic image can be suitably captured.

The positional deviation estimator 24 determines (estimates) that the alignment state is proper when the alignment completion conditions |Δx|<Tx, |Δy|<Ty, and |Δz|<Tz are satisfied. On the other hand, when the alignment completion condition is not satisfied, the positional deviation estimation unit 24 determines (estimates) that the alignment state is inappropriate. Note that the positional deviation estimating unit 24 determines that the alignment state is proper after the positional relationship between the eye E to be inspected and the optical head unit 100 is stabilized. It may be determined that the alignment state is correct only when the

If it is determined in step S706 that the alignment state is inappropriate, the process proceeds to step S707. In step S707, the drive control section 22 controls the driving member of the face receiving section 160 and the stage section 150 to move at least one of the face receiving section 160 and the optical head section 100 based on the estimated pupil position.

Specifically, the positional deviation estimator 24 sends the information on the relative positions of the pupils (deviation amounts Δx, Δy, Δz) estimated in step S<b>704 to the drive controller 22 . The drive control section 22 controls the moving section such as the stage section 150 to move the face receiving section 160 when |Δy| Here, when the face receiving portion 160 is to be moved, the drive control portion 22 sets the moving direction of the face receiving portion 160 to the direction in which |Δy| The member can be controlled to move the face receiving portion 160 a distance corresponding to Δy. As for the deviation in the xz direction, the drive control section 22 controls the stage section 150 to move the optical head section 100 . Also, the moving distance of the optical head unit 100 can be a distance corresponding to the deviation amounts Δx, Δy, and Δz.

Here, since it takes time to move the face receiving part 160 and the optical head part 100, it is conceivable that the subject's eye E may move during the movement. Therefore, immediately after starting the movement of the face receiving unit 160 and the optical head unit 100 in step S707, the control unit 20 returns the processing to step S703 to obtain a new image, and determines in step S706 that the alignment state is appropriate. The above control is repeated until determination is made.

If it is determined in step S706 that the alignment state is proper, the process proceeds to step S708. In step S708, the control unit 20 performs OCT imaging. Specifically, the drive control unit 22 controls the driving of the light source 118, the X scanner 114-1, the Y scanner 114-2, the lens 115, etc., and the image generation unit 23 responds to the interference signal output from the line sensor 184. Based on this, an OCT tomographic image is generated. Thereby, the control unit 20 can obtain an OCT tomographic image at an appropriate position. In step S708, the control unit 20 may perform OCT imaging after performing more detailed alignment using a fundus observation image or the like acquired using the CCD 105 for fundus observation. When the OCT imaging process in step S708 ends, the examination is completed in step S709.

Although the examination flow for only one eye has been described here, the examination flow may be one in which OCT imaging is performed for both eyes in succession. In that case, the above flow may be executed for each of the right eye and the left eye.

In this embodiment, the input of subject information in step S701 also serves to indicate the start of examination. However, the method of starting the inspection is not limited to this. For example, an operation such as turning on the power of the OCT apparatus 1 may be used as the start of the examination. Alternatively, the examination may be started by providing a dedicated input indicating the examination start, which is different from the subject information, through the input unit 50 . Also, the end of the previous inspection may be regarded as the start of the next inspection.

Also, the movement to the initial position in step S702 may be simplified. That is, the process may proceed to step S703 without waiting for arrival at the initial position. When proceeding to step S703 without waiting for arrival at the initial position, the following effects can be obtained. When the subject puts the face on the face receiving section 160 from the beginning, the relative position of the pupil of the subject's eye E estimated before the optical head section 100 reaches the initial position (amount of deviation Δx, Δy, Δz) The inspection time can be shortened. In particular, in an examination flow in which OCT imaging is performed continuously for both eyes, this control can greatly reduce the examination time. That is, after testing one eye, while moving to the initial position for testing the other eye, it is possible to estimate the pupil position of the other eye and head toward the other eye at the shortest distance.

Furthermore, the information estimated using the learned model in step S704 need not be all of the subject's existence probability P and the pupil deviation amounts Δx, Δy, and Δz, but may be some of them. In this case, it is conceivable that information that is not estimated using the learned model is estimated using other means such as a rule base, or that only predetermined control is performed.

For example, when only the existence probability P of the subject is estimated by the trained model, the amount of pupil deviation can be estimated by rule-based image analysis. As rule-based image analysis, for example, an anterior eye observation image is binarized, facial features such as pupils, eyebrows, and nose are recognized, and processing is performed to estimate pupil deviation amounts Δx and Δy. can be done. Furthermore, two corneal bright spots can be detected from the image, and the displacement amount Δz of the pupil can be estimated from the interval between them. It should be noted that even in the case of a configuration in which only the position information about some of the three axes related to the displacement amounts Δx, Δy, and Δz of the pupil is estimated using the learned model, the position information about the remaining axes is obtained using the same image processing. can be asked for.

Further, when estimating only the amount of deviation of the pupil by the trained model, the existence probability P of the subject is not determined, and control is performed on the premise that the subject always puts the face on the face receiving section 160. good. If the time from the start of the examination until the subject puts his/her face on the face receiving section 160 is short or zero, control can be performed on the assumption that the subject always puts his or her face on the face receiving section 160 . Alternatively, the control may be such that a sensor is provided in the face receiving section 160 and the sensor detects that the face of the subject is placed thereon.

Further, instead of estimating the position of the pupil of the subject's eye E in step S704 as described above, only the direction in which the pupil is positioned may be estimated. In step S707, the optical head unit 100 is moved in the estimated direction by a predetermined amount, and by repeating this, the optical head unit 100 can be brought closer to the center of the pupil step by step. In this case, when judging whether the alignment state is proper in step S706, the positional deviation estimation unit 24 can use, for example, the rule-based image analysis described above. Alternatively, the positional deviation estimating section 24 may determine that the optical head section 100 has become sufficiently close to the center of the pupil when the moving direction is switched at a certain frequency or more.

Further, if it is determined in step S705 that the face of the subject is not placed on the face receiving section 160, the flow returns to step S703. On the other hand, if it is determined in step S705 that the face of the subject is not placed on the face receiving section 160, the process returns to step S702, and the drive control section 22 causes the face receiving section 160 and the optical head section 100 to move. It may be moved to a predetermined position. Alternatively, control may be such that the process is normally returned to step S703, and the process is returned to step S702 only when it is determined that the subject does not put the subject's face on it for a given number of times. For example, when the face of the subject is not placed on the face receiving section 160, the face receiving section 160 or the optical head section 100 may be moved by erroneously judging that the subject is temporarily placing the face on the face receiving section 160. Suppose that With the above control, even in such a case, the face receiving section 160 and the optical head section 100 can be returned to the positions at the start of the examination when it is correctly determined that the face of the subject is not placed thereon. .

Alternatively, when it is determined that the face of the subject is not placed on the face receiving section 160, the drive control section 22 stops the movement of the optical head section 100 and the face receiving section 160 at that time (stand still). processing may be performed. Further, when it is determined that the face of the subject is not placed on the face receiving section 160 when the optical head section 100 and the face receiving section 160 are not moving, the drive control section 22 controls the optical head section 100 and the face receiving section 160. A process of not moving the face receiving portion 160 may be performed.

In addition, in step S705, the positional deviation estimating unit 24 determines that the subject's face is placed on the face receiving unit 160 if the existence probability P of the subject is greater than or equal to a certain value. A method may be used to determine whether or not the subject has his/her face on it. For example, immediately after the subject puts the face on the face receiving section 160, the position of the face is often not stable due to movement such as correcting the posture, so it is determined that the position of the face has stabilized to some extent. Therefore, it may be determined that the subject has put his/her face on it.

Specifically, the positional deviation estimating unit 24 determines that the subject's face is placed on the face receiving unit 160 only when the subject's presence probability P is equal to or greater than a certain value for a plurality of consecutive times. You can judge. Furthermore, the positional deviation estimating unit 24 determines that the existence probability P of the subject is equal to or greater than a certain value, and that the estimated relative positions of the pupils (the amounts of deviation Δx, Δy, and Δz) change more than once. It may be determined that the subject's face is placed on the face receiving section 160 only when it becomes smaller. Alternatively, the positional deviation estimating unit 24 may determine that the subject is placing his or her face when the existence probability P of the subject is equal to or greater than a certain value and the amount of change in the image is smaller than a certain value. good. The image change amount can be obtained, for example, by calculating the difference from the image of the previous frame.

Also, error processing may be added to step S706. Specifically, when the alignment state does not become proper even after a certain period of time has passed, the control unit 20 may add control to display a message on the display unit 40, or the drive control unit 22 may cause the optical head unit 100 to display a message. and control to return the face receiving portion 160 to the initial position.

Furthermore, although both the face receiving section 160 and the optical head section 100 are included in the control targets in step S707, only one of them may be included. Further, the drive control section 22 may move the face receiving section 160 and the optical head section 100 at the same time.

In addition, since the face receiving part 160 is a portion that comes into direct contact with the subject, special control may be performed when the face receiving part 160 is moved. For example, when the control unit 20 determines that the face receiving unit 160 is to be moved, a process of once displaying a warning message on the display unit 40, a process of waiting for the examiner's input, or the like may be added. As a result, it is possible to reduce the possibility that the subject will be surprised by the sudden movement of the face receiving section 160 .

Also, in this embodiment, the optical head unit 100 is not moved during the OCT imaging in step S708, but the present invention is not limited to this. Even if it is once determined that the alignment state is appropriate, there is a possibility that the subject's eye E will move during OCT imaging, making it impossible to acquire a suitable OCT tomographic image. Therefore, the alignment control from step S703 to step S707 may be continued even during OCT imaging.

Further, when the alignment process is performed using the moving image of the anterior eye observation image displayed on the preview screen, these processes may be performed for each frame of the moving image, or may be performed for each frame of the moving image. may be performed every number of frames.

In the OCT imaging in step S708, an image such as an En-Face image or an OCTA front image may be captured together with or instead of the tomographic image or front fundus image. Here, the En-Face image is at least a partial depth range of volume data (three-dimensional tomographic image) obtained using optical interference, and is defined based on two reference planes. A front image generated by projecting or integrating corresponding data onto a two-dimensional plane.

An OCTA front image is a motion contrast front image generated by projecting or accumulating data corresponding to the depth range described above on a two-dimensional plane for motion contrast data among a plurality of volume data. Here, the motion contrast data is data indicating changes between a plurality of volume data obtained by controlling the measurement light to scan the same region (same position) of the subject's eye a plurality of times.

Here, on the preview screen described above, instead of the fundus front image, an En-Face image or an OCTA front image corresponding to a predetermined depth range may be displayed as a moving image. Also, on the preview screen, an OCTA tomographic image may be displayed as a moving image instead of the tomographic image.

As described above, the OCT apparatus 1 according to the present embodiment includes the optical head section 100, the face receiving section 160, the driving members for the stage section 150 and the face receiving section 160, the infrared CCD 112, and the positional deviation estimating section 24. Prepare. The optical head unit 100 functions as an example of a head unit that photographs an eye to be examined. The face receiving part 160 functions as an example of a supporting part that supports the subject's face. The driving members of the stage section 150 and the face receiving section 160 function as an example of a moving section capable of moving at least one of the optical head section 100 and the face receiving section 160 . The infrared CCD 112 functions as an example of an observation section that captures images from the optical head section 100 side toward the face receiving section 160 side. In this embodiment, the infrared CCD 112 captures an anterior eye observation image in front of the optical head unit 100, in other words, in the direction in which the subject is arranged.

The position deviation estimating unit 24 is an estimating unit that estimates at least one of information regarding whether or not the subject places his/her face on the face receiving unit 160 and information regarding the position of the subject's eye using the learned model. Serves as an example. The trained model used in the present embodiment is a trained model that has been trained by inputting an image group including an image of the subject's eye that includes the pupil and an image that does not include the pupil. Information about the position of the eye to be inspected estimated by the positional deviation estimating unit 24 includes the relative position of the pupil of the eye to be inspected with respect to the optical head unit 100 and the direction of the position of the pupil of the eye to be inspected with respect to the position of the optical head unit 100. and at least one of

The drive control unit 22 functions as an example of a control unit that controls the moving unit based on the result estimated by using the image captured by the infrared CCD 112 as an input to the trained model. In this embodiment, the drive control unit 22 controls the moving unit to move the optical head unit 100 or the face receiving unit 160 based on at least one of the relative position and direction estimated by the positional deviation estimating unit 24. Control. More specifically, when the position deviation estimating unit 24 estimates information indicating that the subject puts his/her face on the face receiving unit 160, the drive control unit 22 determines the above-described relative position and direction. The moving unit is controlled to move at least one of the optical head unit 100 and the face receiving unit 160 based on at least one of the above.

Further, the drive control unit 22 determines whether the optical head unit 100 or the face receiving unit 160 is positioned when the positional deviation estimating unit 24 estimates information indicating that the subject does not put the face on the face receiving unit 160 . The moving parts are controlled so that at least one of them remains stationary or does not move. It should be noted that the drive control unit 22 controls the position of the optical head unit 100 and the face receiving unit 160 when the information indicating that the subject does not put the face on the face receiving unit 160 is estimated by the misalignment estimating unit 24 . The moving part may be controlled to move at least one of them to a predetermined position.

According to such a configuration, the examination is automatically started when the examiner puts the face on the face receiving section 160, and the alignment is automatically performed regardless of the size of the face of the examinee. Therefore, it is not necessary to confirm that the subject puts his/her face on the face receiving section 160 before starting the examination, and it is necessary to manually control the device until the pupil of the subject enters the observation range. There is no Therefore, by using the OCT apparatus 1 according to the present embodiment, it is possible to reduce the burden of manual operation by the examiner.

Note that, in this embodiment, the infrared CCD 112 is used as an example of an anterior ocular observation unit that captures an image of the anterior ocular segment of the subject's eye. Therefore, it is not necessary to provide a separate camera or the like for alignment, so the cost can be reduced and the size of the apparatus can be reduced.

Further, the drive control unit 22 determines that the above-described relative position is determined when the position deviation estimation unit 24 estimates the information indicating that the face of the subject is placed on the face receiving unit 160 continues for a certain period of time. and direction, the moving unit may be controlled to move at least one of the optical head unit 100 and the face receiving unit 160 . In this case, it can be determined that the subject's face has been put on it after it is determined that the position of the face has stabilized to some extent.

In the present embodiment, the trained model includes a trained model that outputs information regarding whether or not the subject puts his/her face on the support part, and a trained model that outputs information regarding the position of the subject's eye. including finished models. Therefore, the positional deviation estimating unit 24 uses the output from each trained model to estimate information regarding whether or not the subject puts his/her face on the face receiving unit 160 and information regarding the position of the subject's eye. .

On the other hand, the positional deviation estimating unit 24 uses only the learned model that has learned by outputting information about the position of the eye to be inspected including the relative position of the pupil of the eye to be inspected with respect to the optical head unit 100. Information regarding whether or not and information regarding the position of the subject's eye may be estimated. In this case, the positional deviation estimating section 24 estimates information regarding whether or not the subject rests his/her face on the face receiving section 160 based on the information regarding the position of the subject's eye output from the learned model. be able to. For example, the positional deviation estimating section 24 may estimate that the subject does not put the face on the face receiving section 160 when the deviation amounts Δx, Δy, and Δz output from the learned model are larger than the threshold values. can.

Also, the trained model may be one trained model that has learned by outputting information regarding whether or not the subject puts his/her face on the support portion and information regarding the position of the subject's eye. In this case, the position deviation estimating unit 24 uses the output from the one trained model to obtain information about whether the subject puts his/her face on the face receiving unit 160 and information about the position of the subject's eye. can be estimated.

Note that the OCT apparatus 1 can further include an input unit 50 that receives input from the examiner. In this case, the positional deviation estimating unit 24 may start the estimation process when the input unit 50 receives input from the examiner. According to such a configuration, the examiner can instruct the start of the above-described automatic alignment after confirming that the face of the subject is placed on the face receiving section 160 or the like.

In addition, the OCT apparatus 1 can further include an anterior eye observation light source 122 that functions as an example of a projection unit that projects an index onto the subject's eye. In this case, the infrared CCD 112 can capture an image including an index image projected onto the subject's eye by the anterior eye observation light source 122 . The positional deviation estimation unit 24 can calculate the positional relationship between the subject's eye and the optical head unit 100 by rule-based processing based on the index appearing in the anterior eye observation image.

Furthermore, the group of images learned by the trained model may include an image of the subject wearing eyeglasses and an image of the subject not wearing eyeglasses. In this case, the positional deviation estimating unit 24 can further estimate information regarding whether or not the subject is wearing spectacles using the image captured by the infrared CCD 112 as an input to the trained model. can. In this case, when information indicating that the subject is wearing spectacles is estimated by the positional deviation estimation unit 24, at least one of the optical head unit 100 and the face receiving unit 160 should not be moved. The drive control section 22 can be configured to control the moving section. According to such a configuration, it is possible to assist in calling attention to the subject to remove the spectacles before imaging.

Also, the group of images learned by the trained model may include an image in which the subject's eyes are open and an image in which the subject's eyes are closed. In this case, the positional deviation estimating unit 24 can further estimate information regarding whether or not the subject has his/her eyes open by using the image captured by the infrared CCD 112 as an input to the trained model. . In this case, when the information indicating that the subject's eyes are closed is estimated by the positional deviation estimation unit 24, at least one of the optical head unit 100 and the face receiving unit 160 is moved so as not to be moved. The drive control section 22 can be configured to control the section. According to such a configuration, it is possible to prevent malfunction of alignment based on an image acquired when the subject closes his or her eyes.

Note that this embodiment can also be applied to an ophthalmologic apparatus other than an OCT apparatus. For example, the alignment processing according to this embodiment can be applied to various ophthalmologic apparatuses such as a fundus camera, an SLO apparatus, an eye refractive power measuring apparatus, and a tonometer. In that case, step S708 in FIG. 7 becomes imaging and measurement according to the ophthalmologic equipment.

In addition, the alignment processing according to the present embodiment can also be applied to an apparatus that captures an anterior eye front image and an anterior segment OCT. In particular, in an apparatus capable of photographing both the fundus and the anterior segment, the working distance (distance in the z direction between the subject's eye E and the optical head unit 100) often differs between when photographing the fundus and when photographing the anterior segment. In that case, an examination flow may be considered in which different z-direction offset distances are added to the fundus imaging and the anterior segment imaging with respect to a common trained model. Alternatively, different models may be trained and prepared for each type of imaging so that the optimal z-direction position is aligned for each of fundus imaging and anterior segment imaging.

(Modification 1)
In this modified example, an example in which the OCT apparatus described in Example 1 is improved to improve alignment accuracy will be described. Specifically, two types of images of the anterior eye, wide-area and narrow-area, are acquired, and from the wide-area image, a trained model is used to estimate the approximate deviation of the pupil, and from the narrow-area image, a rule-based image is obtained. An example of estimating a highly accurate amount of deviation of the pupil by analysis will be described.

First, the reason why the alignment accuracy is required to be improved will be explained. In the OCT apparatus, if the alignment is off, part of the OCT measurement light flux is blocked by the iris, resulting in a dark tomographic image. Further, if the alignment is off, the reflection of the fundus observation light from the cornea or the lens reaches the imaging device, and there is a problem that unnecessary light is reflected in the observation image. Although these problems depend on the pupil diameter and corneal shape of the subject's eye, the effects can be reduced by aligning the optical head unit at a position closer to the ideal. Therefore, improving the alignment accuracy is effective in reducing these problems.

The OCT apparatus described in Example 1 obtains an image including the periphery of the subject's eye, such as the anterior eye observation image shown in FIG. This is because an image in which a wide range is captured contains more features, and thus a highly accurate trained model can be obtained when constructing the trained model. On the other hand, in order to improve alignment accuracy, it is necessary to perform more detailed adjustments using a high-resolution image, that is, an image with a narrow field of view per pixel. This means that an image with a narrow shooting range is used for alignment instead of an image with a wide range such as an image for anterior eye observation.

Further, for example, when obtaining the displacement amount using the anterior eye observation image shown in FIG. 5, it is not easy to accurately estimate the displacement amount (Δz) in the z direction. The image for anterior eye observation shown in FIG. 5 is an image containing information on the amount of deviation in the z direction, such as the amount of blurring of the image and the interval between corneal bright spots. However, there is a problem that the deviation direction (forward or backward) cannot be determined from the amount of blurring, and there is a problem that the distance between the corneal bright spots is inaccurate because it depends on the corneal curvature. Therefore, in this modified example, a high-resolution image for anterior eye observation is used, in which the z-direction deviation amount (Δz) can be easily estimated.

Since the schematic configuration and optical layout of the OCT apparatus according to this modified example are mostly the same as those in Example 1, the configurations shown in FIGS. explanation is omitted. The OCT apparatus according to this modified example will be described below, focusing on the differences from the OCT apparatus according to the first embodiment.

As a difference from the OCT apparatus according to the first embodiment, first, in this modified example, the magnification of the lenses 109, 110, and 111 is different from that of the first embodiment, and the photographing range of the anterior eye observation image obtained by the infrared CCD 112 is narrow and the resolution is small. is high. Furthermore, in this modified example, there is a difference that a prism is attached to the lens 110 . The prism-equipped lens 110 and its effect will be described below with reference to FIGS. 8(a) and 8(b).

The prism-equipped lens 110 is provided with Fresnel prisms 110-1 and 110-2, as shown in FIG. 8(a). The prism-equipped lens 110 is arranged at a position where the Fresnel prisms 110-1 and 110-2 are conjugate with the anterior segment of the eye E to be examined. In this modified example, due to the prism effect of the Fresnel prisms 110-1 and 110-2, the image formed by the luminous flux passing through the lens 110 with the prism is split vertically according to the distance between the optical head unit 100 and the subject's eye E. The prism-equipped lens 110 is arranged as shown. However, the direction in which the image is split by the prism effect may be arbitrarily set according to a desired configuration, and the prism-equipped lens 110 may be arranged in an arbitrary orientation accordingly.

Here, the positional relationship between the optical head unit 100 and the subject's eye E, that is, the case where the alignment position is at the ideal position will be described. In this case, the luminous flux of the reflected/scattered light from the anterior segment of the eye to be examined E forms an image once on the Fresnel prisms 110-1 and 110-2 of the prism-equipped lens 110, and the Fresnel prisms 110-1 and 110 The -2 prism effect splits the image (image split). However, since the photographing surface of the infrared CCD 112 is also conjugate with the Fresnel prisms 110-1 and 110-2, the anterior eye observation image photographed using the infrared CCD 112 is different from the anterior eye observation image 801 in FIG. become. In the anterior eye observation image 801, the anterior eye image is in the center of the image, and the upper half and lower half of the pupil image match.

On the other hand, when the alignment position is not at the ideal position in the xy direction, the anterior eye observation image becomes like the anterior eye observation image 802 in FIG. 8B. In the anterior eye observation image 802, the anterior eye image is at a position shifted from the center of the image, but since the position in the z direction is at an ideal position, the upper half and lower half of the pupil image match.

Next, when the alignment position is at the ideal position in the xy direction and is farther than the ideal position in the z direction (the distance between the optical head unit 100 and the subject's eye E is longer than the ideal distance), the anterior eye observation image is , an anterior eye observation image 803 in FIG. 8B. In the anterior eye observation image 803, the anterior eye image is in the center of the image, but the upper and lower halves of the pupil image are shifted.

Similarly, when the alignment position is not at the ideal position in the xyz direction (or xz direction or yz direction), the anterior eye image in the anterior eye observation image captured using the infrared CCD 112 is shifted from the center of the image, The upper and lower halves of the pupil image are also shifted. For example, when the alignment position is not at the ideal position in the x direction and is closer than the ideal position in the z direction (the distance between the optical head unit 100 and the subject's eye E is shorter than the ideal distance), the anterior eye observation image becomes like the anterior eye observation image 804 in FIG. 8B. In the anterior eye observation image 804 , in the anterior eye observation image 802 , the anterior eye image is positioned laterally displaced from the center of the image, and the upper and lower halves of the pupil image are displaced. In the anterior eye observation image 804, the direction of displacement between the upper half and the lower half of the pupil image is opposite to the direction of displacement between the upper and lower halves of the pupil image in the anterior eye observation image 803. It can be seen that the shift directions in the z direction of the alignment positions corresponding to the respective images are reversed.

When such an image for anterior eye observation is used, the positional deviation estimator 24 can obtain the deviation amounts Δx, Δy, and Δz of the pupil by rule-based image analysis if the pupil is shown in the image. Specifically, the positional deviation estimation unit 24 first detects only the pupil region by binarizing the anterior eye observation image with an appropriate threshold value. The positional deviation estimator 24 can obtain the pupil deviation amounts Δx and Δy by multiplying the detected relative distance between the center of gravity of the pupil region and the center of the image by a predetermined coefficient. The specified coefficient is a design value that corresponds to the distance in the anterior segment per pixel and is determined according to the design. Furthermore, the positional deviation estimating unit 24 can divide the detected pupil region into upper and lower parts at the boundary of the prism, calculate the center of gravity for each of the upper and lower parts, and obtain the amount of image deviation due to the prism from the difference in the horizontal direction between the centers of gravity. . The positional deviation estimator 24 can obtain the amount of deviation Δz of the pupil by multiplying the determined amount of deviation of the image by the prism by a predetermined coefficient. The prescribed factor is a design value corresponding to the z-direction distance per pixel for image splitting.

It should be noted that the method of obtaining (detecting method) the displacement amounts Δx, Δy, and Δz of the pupil is not limited to this. For example, the positional deviation estimating unit 24 may detect the pupil by applying various methods such as edge detection and circle approximation instead of binarization to the anterior eye observation image. Also, the positional deviation estimator 24 may detect the corneal bright spot instead of the pupil. Further, another sensor different from the infrared CCD 112, such as a distance sensor, may be provided, and the positional deviation estimator 24 may detect the pupil deviation amounts Δx, Δy, and Δz based on the output from the other sensor. good. Further, the amount of shift in the z direction may be detected using corneal bright spots or the amount of image blur without using image splitting.

By the way, such an image analysis algorithm is effective only for images in which the pupil is shown. If the image does not show the pupil, it is conceivable to use a trained model similar to that of the first embodiment. However, as described above, the magnification of the lenses 109, 110, and 111 according to this modified example is different from that of the first embodiment, and the imaging range of the anterior eye observation image obtained by the infrared CCD 112 is narrow and the resolution is high. Therefore, the image for anterior eye observation according to this modification as shown in FIG. When used for volume estimation processing, it may be difficult to estimate the pupil position. Therefore, the image for anterior eye observation according to this modified example may not be suitable for constructing a trained model. In addition, the anterior eye observation image according to the present modified example has a problem that learning is difficult because even if the same part of the subject is present, the images are completely different in each z direction due to image splitting.

Therefore, in this modified example, the anterior eye observation image obtained by the infrared CCD 112 is not used for building a learned model for the process of estimating the deviation amount. Instead, images captured by the CCD 105 for observing the fundus (frontal images of the anterior eye) are used to construct the learned model. Since the CCD 105 is for observing the fundus of the subject's eye, a fundus image can be obtained by observation at a predetermined working distance. On the other hand, by separating the optical head unit 100 from the subject in the z-direction and setting the working distance to be larger than the predetermined working distance, the CCD 105 can obtain an image around the anterior eye of the subject. Such an image has a wider shooting range than the image obtained by the infrared CCD 112 and does not cause an image split, and is therefore more suitable for learning. Specifically, images similar to the images shown in FIGS. 5A to 5G described in the first embodiment are obtained. Note that the learning data according to this modification may be created in the same manner as in Example 1, except that the anterior eye front image captured by the CCD 105 as described above is used instead of the anterior eye observed image.

An inspection flow according to this modified example will be described below with reference to FIG. 9 . First, the examiner starts examination in step S901. The method of starting the inspection is the same as in the first embodiment. Next, in step S902, the control section 20 moves the optical head section 100 and the face receiving section 160 to predetermined positions. This predetermined position is the same position in the xy direction as in the first embodiment, and in the z direction is a position away from the subject so that the face of the subject can be photographed using the CCD 105 .

Next, in step S903, the control unit 20 performs rough alignment using the learned model. The rough alignment is mostly the same as the alignment processing (steps S703 to S707) according to the first embodiment, but has the following differences from the alignment processing according to the first embodiment. First, the input to the trained model is the anterior eye front image captured by the CCD 105 . Also, since the optical head unit 100 is located away from the subject, alignment is performed only in the xy direction. Also, whether or not the alignment state is proper is determined only in the xy direction. Further, the thresholds Tx and Ty for judging that the alignment state is proper are relatively large values, and precise alignment (fine alignment) is performed after the rough alignment. Also, processing (step S904) is performed to adjust the position in the z direction before performing precise alignment.

In step S904, the drive control unit 22 moves the optical head unit 100 forward in the z direction, that is, in a direction to approach the subject. The z-direction position for moving the optical head unit 100 may be the same as the z-direction position for moving the optical head unit 100 in step S702 of the first embodiment. At this stage, since the rough alignment in the xy direction has already been completed, the image for anterior eye observation obtained using the infrared CCD 112 is an image in which the pupil of the subject's eye E is present near the center of the image. .

Next, in step S905, the control unit 20 performs fine alignment using the rule-based image analysis described above. As described above, since the anterior eye observation image acquired using the infrared CCD 112 according to this modification is an image with high resolution, fine alignment is performed using the anterior eye observation image. Alignment can be performed with higher accuracy than in the conventional method. Therefore, the alignment completion condition in step S905 can be set to a stricter value than the alignment completion condition in the first embodiment, for example, threshold values Tx, Ty, and Tz can be set to smaller values.

More specifically, first, the acquisition unit 21 acquires an anterior eye observation image captured using the infrared CCD 112 . The positional deviation estimating unit 24 obtains the pupil deviation amounts Δx, Δy, and Δz by performing the above-described rule-based image analysis on the anterior eye observation image. The positional deviation estimator 24 determines the alignment state of the apparatus by comparing the deviation amounts Δx, Δy, Δz with the thresholds Tx, Ty, Tz, as in step S706 of the first embodiment. If it is determined that the alignment state is inappropriate, the drive control section 22 controls the moving section such as the stage section 150 to move at least one of the face receiving section 160 and the optical head section 100, as in step S707. Move based on the estimated position of the pupil. After that, the anterior eye observation image is acquired again, and the above control is repeated until it is determined that the alignment state is appropriate. If it is determined that the alignment state is proper, the process proceeds to step S906.

In step S906, as in the first embodiment, the control unit 20 performs OCT imaging. After that, the inspection is completed in step S907.

Although the examination flow for only one eye has been described here, the examination flow may be one in which OCT imaging is performed for both eyes in succession. In that case, the above flow may be executed for each of the right eye and the left eye. Also, in order to shorten the examination time, the above flow may be performed only for the first examination, and rough alignment, that is, steps S903 and S904 may be skipped in the examination of the opposite eye. At this time, since the position of the eye on the opposite side can be roughly estimated from the position of the first examination, if the optical head unit 100 is moved by a predetermined distance from the position of the first examination, fine alignment of the eye on the opposite side can be performed directly. likely to migrate. According to such processing, inspection time can be shortened.

Before moving the optical head unit 100 to a position away from the subject's eye in step S902, fine alignment similar to step S905 may be tried once. That is, if the pupil of the eye to be inspected is already near the center of the image, only fine alignment equivalent to step S905 is executed as it is, and only when the pupil is not near the center of the image and fine alignment fails, the processing from step S902 is performed. good too. This method eliminates the need to once move the optical head unit 100 away from the subject's eye when the subject's eye is already in a suitable position, thereby shortening the examination time.

As described above, the positional deviation estimator 24 according to the present modification functions as an example of a pupil detector that detects the position of the pupil of the subject's eye. The drive control section 22 controls the moving section such as the stage section 150 based on the result of estimation by the positional deviation estimation section 24 and the result of detection of the position of the pupil. Particularly in this modified example, the observation unit includes the CCD 105 that functions as an example of a fundus observation unit that captures an image of the fundus of the subject's eye. The observation unit captures an anterior eye observation image (first image) in a first range using the infrared CCD 112, and captures the first image using the CCD 105 while the subject's eye E is separated from the optical head unit 100. An anterior eye front image (second image) of a wider range than the range is photographed. The positional deviation estimation unit 24 detects the position of the pupil of the subject's eye based on the anterior eye observation image, and uses the anterior eye front image as an input to the trained model to estimate information regarding the position of the subject's eye.

According to such a configuration, the examination is automatically started when the examiner places the face on the face receiving section 160, and the alignment can be automatically performed regardless of the size of the face of the examinee. It is possible to reduce the burden of manual operation by the examiner. In addition, since the control unit 20 can perform alignment with high accuracy using an image with a narrow imaging range and high resolution, a suitable OCT tomographic image can be obtained by using the OCT apparatus 1 according to the present modification. .

Also, in this modification, the CCD 105 and the infrared CCD 112 are used as an example of an observation section. Therefore, it is not necessary to provide a separate camera or the like for alignment, so the cost can be reduced and the size of the apparatus can be reduced.

Note that, in this modified example, fine alignment using the anterior segment observation image is performed after performing rough alignment using the learned model. In response to this, first, the control unit 20 detects the position of the pupil detected by the above rule-based image analysis or the position of the pupil estimated using the learned model based on the result estimated by the misalignment estimation unit 24. Alignment may be controlled according to .

In this case, if the position of the pupil estimated using the learned model is close to the ideal alignment position, the control unit 20 performs alignment using the position of the pupil detected by the rule-based image analysis described above. conduct. On the other hand, if the pupil position estimated using the trained model is far from the ideal position, the control unit 20 performs alignment using the pupil position estimated using the trained model. In such a case, if the position of the pupil estimated using the trained model is close to the ideal alignment position, the rough alignment process can be omitted, thereby shortening the inspection time. It should be noted that the determination as to whether or not the alignment position is close to the ideal alignment position may be made using the same conditions as the alignment completion conditions for the rough alignment described above.

Further, the control unit 20 may determine whether to perform rough alignment or fine alignment depending on whether or not the positions of the pupils are detected by the positional deviation estimating unit 24 . With such a configuration, when the position of the pupil is detected by the positional deviation estimating section 24, the control section 20 controls fine alignment based on the detected position of the pupil. On the other hand, if the position of the pupil is not detected, the control unit 20 controls rough alignment based on the result of estimation using the learned model. In such a case, it is not necessary to once move the optical head unit 100 away from the eye to be inspected when the eye to be inspected is already in a suitable position, so the inspection time can be shortened.

In this modified example, estimation using a learned model is performed on a wide-area image, and the pupil position is detected by rule-based image analysis on a narrow-area image. However, images used for these processes are not limited to this. For example, in contrast to this modified example, detection of the pupil position may be performed using a rule-based image analysis using a wide-area image, and estimation using a trained model may be performed using a narrow-area image. Alternatively, both images may be estimated by a trained model.

Furthermore, for an image using one image sensor, estimation by a trained model, estimation by rule-based image analysis, and estimation by combining them may be switched depending on the situation. In particular, estimation by the trained model is effective when the optical head unit 100 is far from the pupil, and estimation by rule-based image analysis is effective when the optical head unit 100 is close to the pupil. Therefore, it is preferable to perform estimation using a trained model first, and then perform rule-based estimation when it is determined that the optical head unit 100 is close to the pupil. Alternatively, rule-based estimation may be performed first, and then estimation using a trained model may be performed when the estimation result is an abnormal value or when it is determined that the pupil cannot be found. Alternatively, both estimations may always be performed, and the average of both estimation results may be used for alignment. Furthermore, the likelihood of both estimated values may be calculated using a predetermined criterion, and the estimated value with the higher likelihood may be used for alignment.

Also, in this modified example, the position of the pupil is detected by rule-based image analysis. However, the method of detecting the position of the pupil is not limited to rule-based image analysis. For example, the positional deviation estimator 24 may obtain the amount of deviation using predetermined information from a distance sensor or the like. Even in such a case, the present invention is applicable as long as the estimation by the trained model is at least partially performed. Moreover, this modification can also be applied to an ophthalmologic apparatus other than the OCT apparatus as in the first embodiment.

It should be noted that in the above-described embodiment and modified example, an image obtained using an imaging member used for inspection, such as the infrared CCD 112 or the CCD 105, is used as an input to the trained model. On the other hand, an image obtained using an alignment camera separately provided in the optical head unit 100 so as to capture an image in the imaging direction of the optical head unit 100 may be used as an input to the trained model. .

Also, the configuration of the measurement optical system in the ophthalmologic apparatus according to the above-described embodiments and modifications is an example, and may be arbitrarily changed according to a desired configuration. Furthermore, the learning data of various trained models is not limited to data obtained using the ophthalmologic apparatus itself that actually performs imaging. Data or the like obtained using an ophthalmologic apparatus may be used.

In addition, the trained models according to the above-described embodiments and modifications are characterized by the magnitude of luminance values of the anterior eye observation image and the anterior eye front image, the order and inclination of the bright and dark portions, the position, distribution, continuity, etc. It can be considered that it is extracted as part of the quantity and used for estimation processing. In addition, since trained models using RNN etc. perform learning using time-series data, the slope between input continuous time-series data values is also extracted as part of the feature value and estimated. It is thought that it is used for processing. Therefore, such a trained model is expected to be able to perform accurate estimation by using the influence of temporal changes in specific numerical values in the estimation process.

Various learned models according to the above-described embodiments and modifications can be provided in the control unit 20 . A trained model may be configured by, for example, a software module or the like executed by a processor such as a CPU, MPU, GPU, or FPGA, or may be configured by a circuit or the like that performs a specific function such as an ASIC. Also, the learned model may be provided in another server device or the like connected to the control unit 20 . In this case, the control unit 20 can use the trained model by connecting to a server or the like having the trained model via an arbitrary network such as the Internet. Here, the server provided with the learned model may be, for example, a cloud server, a fog server, an edge server, or the like. In addition, when configuring a network within a facility, within a site that includes a facility, within an area that includes multiple facilities, etc. to enable wireless communication, for example, Reliability of the network may be improved by configuring to use radio waves of a dedicated wavelength band. Alternatively, the network may be configured by wireless communication capable of high speed, large capacity, low delay, and multiple simultaneous connections.

(Other examples)
The present invention provides a program that implements one or more functions of the above-described embodiments and modifications to a system or device via a network or a storage medium, and the computer of the system or device reads and executes the program. It is feasible. A computer has one or more processors or circuits and may include separate computers or a network of separate processors or circuits for reading and executing computer-executable instructions.

A processor or circuit may include a central processing unit (CPU), a microprocessing unit (MPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), or a field programmable gateway (FPGA). Also, the processor or circuitry may include a digital signal processor (DSP), data flow processor (DFP), or neural processing unit (NPU).

Although the present invention has been described with reference to the embodiments and modifications, the present invention is not limited to the above embodiments and modifications. Inventions modified within the scope of the present invention and inventions equivalent to the present invention are also included in the present invention. Moreover, the above-described embodiments and modifications can be appropriately combined within the scope of the present invention.

1: OCT apparatus (ophthalmic apparatus), 22: drive control unit (control unit), 24: misalignment estimation unit (estimation unit), 100: optical head unit (head unit), 112: infrared CCD (observation unit), 150 : stage portion (moving portion), 160: face receiving portion (supporting portion, moving portion)

Claims (24)

a head for photographing an eye to be examined;
a support for supporting the subject's face;
a moving part capable of moving at least one of the head part and the support part;
an observation unit that captures images from the head unit side toward the support unit side;
Using a trained model trained by inputting an image group including an image of the eye to be examined including the pupil and an image not including the pupil, information on whether or not the subject rests his/her face on the supporting part and the image of the eye to be examined are obtained. an estimating unit that estimates at least one of positional information;
a control unit that controls the movement unit based on the result estimated by the estimation unit using an image captured by the observation unit as an input to the trained model;
An ophthalmic device comprising: The information about the position of the eye to be examined estimated by the estimating unit includes at least one of the relative position of the pupil of the eye to be examined with respect to the head and the direction of the position of the pupil of the eye to be examined with respect to the position of the head. The ophthalmic device according to claim 1. 3. The ophthalmologic apparatus according to claim 2, wherein the control section controls the moving section to move at least one of the head section and the support section based on at least one of the relative position and the direction. . When the information indicating that the subject is resting the face on the support is estimated, the controller controls the head and the support based on at least one of the relative position and the direction. 4. The ophthalmologic apparatus according to claim 3, wherein said moving part is controlled to move at least one of The control unit controls at least one of the relative position and the direction when the estimation result in which the estimation unit estimates information indicating that the face of the subject is placed on the support unit continues for a certain period of time. 5. The ophthalmologic apparatus according to claim 4, wherein the moving section is controlled to move at least one of the head section and the support section based on. The control unit causes at least one of the head unit and the support unit to stop or not to move when the estimation unit estimates information indicating that the subject does not put the face on the support unit. 6. The ophthalmic apparatus according to any one of claims 1 to 5, wherein the moving part is controlled so as to. The control unit moves at least one of the head unit and the support unit to a predetermined position when the estimation unit estimates that the subject does not put the face on the support unit. 6. The ophthalmologic apparatus according to any one of claims 1 to 5, wherein the moving part is controlled to allow the moving part to move. The learned model includes a learned model learned by outputting information regarding whether or not the subject puts the face on the support part, and a learned model learned by outputting information regarding the position of the subject's eye, An ophthalmic device according to any one of claims 1 to 7. The learned model is a learned model that has learned by outputting information regarding the position of the eye to be examined, including the relative position of the pupil of the eye to be examined with respect to the head,
8. The estimating unit according to any one of claims 1 to 7, wherein the estimating unit estimates information regarding whether or not the subject rests his or her face on the support based on information regarding the position of the subject's eye output from the learned model. or the ophthalmic device according to claim 1. 8. The learned model according to any one of claims 1 to 7, wherein the learned model includes a learned model obtained by outputting information regarding whether or not the subject rests his/her face on the support portion and information regarding the position of the subject's eye. 10. An ophthalmic device according to claim 1. further comprising an input unit for receiving input from the examiner,
The ophthalmologic apparatus according to any one of claims 1 to 10, wherein the estimation unit starts the estimation when the input unit receives an input from the examiner. further comprising a projection unit for projecting the index onto the subject's eye,
The ophthalmologic apparatus according to any one of claims 1 to 11, wherein the observation unit captures an image including the index image projected onto the subject's eye by the projection unit. The ophthalmologic apparatus according to any one of claims 1 to 12, wherein the observation unit includes an anterior eye observation unit that captures an image of an anterior segment of the subject's eye. The ophthalmologic apparatus according to any one of claims 1 to 13, wherein the observation unit captures an image including an image divided according to a distance between the head unit and the eye. 15. The observation unit according to any one of claims 1 to 14, wherein the observation unit captures a plurality of images including a first image of a first range and a second image of a range wider than the first range. ophthalmic equipment. The observation unit includes a fundus observation unit that captures an image of the fundus of the subject's eye,
16. The ophthalmologic apparatus according to claim 15, wherein the second image is an image captured by the fundus observation unit while the subject's eye is separated from the head unit. further comprising a pupil detection unit that detects the position of the pupil of the subject's eye,
The ophthalmologic apparatus according to any one of claims 1 to 16, wherein said control section controls said moving section based on a result of estimation by said estimation section and a result of detection by said pupil detection section. The control unit
when the position of the pupil is detected by the pupil detection unit, controlling the movement unit based on the detected position of the pupil;
18. The ophthalmologic apparatus according to claim 17, wherein when the position of the pupil is not detected by the pupil detection section, the movement section is controlled based on the estimation result of the estimation section. The control unit controls the moving unit according to information regarding the position of the pupil detected by the pupil detection unit or the position of the subject's eye estimated by the estimation unit, based on the result estimated by the estimation unit. 18. The ophthalmic device of claim 17. The observation unit captures a plurality of images including a first image of a first range and a second image of a range wider than the first range,
The pupil detection unit detects the position of the pupil based on the first image,
The ophthalmologic apparatus according to any one of claims 17 to 19, wherein the estimation unit uses the second image as an input to the trained model to estimate information about the position of the subject's eye. The group of images includes an image in which the subject wears the spectacles and an image in which the subject does not wear the spectacles,
The estimating unit uses an image captured by the observing unit as an input to the trained model to further estimate information about whether the subject wears eyeglasses,
When the estimation unit estimates that the subject is wearing spectacles, the control unit controls the movement unit so as not to move at least one of the head unit and the support unit. 21. The ophthalmic device of any one of claims 1-20, for controlling. The image group includes an image in which the subject's eyes are open and an image in which the subject's eyes are closed,
The estimating unit uses an image captured by the observing unit as an input to the trained model to further estimate information about whether the subject has his/her eyes open;
The control unit controls the movement unit so as not to move at least one of the head unit and the support unit when the estimation unit estimates that the subject has closed eyes. 22. An ophthalmic device according to any one of claims 1-21. a head section for photographing an eye to be examined; a support section for supporting the face of the subject; a moving section capable of moving at least one of the head section and the support section; A control method for an ophthalmologic apparatus comprising: an observation unit that photographs sideways, the method comprising:
Using a trained model trained by inputting an image group including an image of the eye to be examined including the pupil and an image not including the pupil, information on whether or not the subject rests his/her face on the supporting part and the image of the eye to be examined are obtained. estimating at least one of position information;
controlling the moving unit based on a result estimated using an image captured by the observation unit as an input to the trained model;
A method of controlling an ophthalmic device, comprising: A program that, when executed by a computer, causes the computer to perform each step of the ophthalmologic apparatus control method according to claim 23.

Images