JP2023180265A - Ophthalmologic apparatus, method for controlling ophthalmologic apparatus, program and recording medium - Google Patents

Abstract

To improve the image quality of an ophthalmologic apparatus in an imaging system using optical scan.SOLUTION: An ophthalmologic apparatus according to one embodiment comprises: an illumination system; an imaging system; a movement mechanism; and a control unit. The illumination system projects illumination light to a subject eye. The imaging system images the subject eye. The imaging system includes an imaging element. The illumination system and the imaging system are configured to satisfy a condition of shine proof. The movement mechanism moves the illumination system and the imaging system. The control unit performs control of the illumination system, imaging system, and movement mechanism to cause the imaging system to collect a series of images. The control unit controls at least one of the illumination system and the imaging system such that the projection time of the illumination light to the subject eye becomes shorter than the exposure time of the imaging element and at least a portion of the projection period of the illumination light to the subject eye and at least a portion of the exposure period of the imaging element overlap each other in the control for causing the imaging system to collect a series of images of the subject eye.SELECTED DRAWING: Figure 2

Description

The present invention relates to an ophthalmologic apparatus, a method for controlling an ophthalmologic apparatus, a program, and a recording medium.

Image diagnosis occupies an important position in the field of ophthalmology. Various ophthalmological devices are used in ophthalmological image diagnosis. Ophthalmological equipment includes slit lamp microscopes, fundus cameras, scanning laser ophthalmoscopes (SLO), and optical coherence tomography (OCT). In addition, various inspection and measurement devices such as refractometers, keratometers, tonometers, specular microscopes, wavefront analyzers, and microperimeters are also equipped with functions for photographing the anterior segment and fundus of the eye.

One of the most widely and frequently used of these various ophthalmic devices is the slit lamp microscope, also called the stethoscope for ophthalmologists. A slit lamp microscope is an ophthalmological device that illuminates a subject's eye with slit light and observes or photographs the illuminated cross section from the side with a microscope (see, for example, Patent Documents 1 and 2). Additionally, a slit lamp microscope is known that can scan a three-dimensional area of the eye to be examined at high speed by using an optical system configured to satisfy Scheimpflug conditions (for example, see Patent Document 3). reference). In addition to the slit lamp microscope, a rolling shutter camera is also known as an imaging method that scans an object with slit light.

JP 2016-159073 Publication Japanese Patent Application Publication No. 2016-179004 JP2019-213733A

One object of the present invention is to improve the image quality of an ophthalmological apparatus using an imaging method using optical scanning.

An ophthalmological apparatus according to an exemplary embodiment includes an illumination system, an imaging system, a movement mechanism, and a control unit. The illumination system projects illumination light onto the eye to be examined. The imaging system photographs the eye to be examined. The photographing system includes an image sensor. The illumination system and photographing system are configured to satisfy Scheimpflug conditions. The moving mechanism moves the illumination system and the imaging system. The control unit controls the illumination system, the imaging system, and the movement mechanism to cause the imaging system to collect a series of images. In controlling the imaging system to collect a series of images of the eye to be examined, the control unit controls the projection time of the illumination light to the eye to be examined to be shorter than the exposure time of the image sensor, and At least one of the illumination system and the photographing system is controlled so that at least part of the projection period and at least part of the exposure period of the image sensor overlap.

According to the exemplary embodiment, it is possible to improve the image quality of an ophthalmological apparatus that uses optical scanning for imaging.

FIG. 2 is a diagram for explaining the background of the embodiment. FIG. 2 is a diagram for explaining an overview of an embodiment. FIG. 1 is a schematic diagram illustrating the configuration of an ophthalmological device according to an exemplary embodiment. FIG. 1 is a schematic diagram illustrating the configuration of an ophthalmological device according to an exemplary embodiment. FIG. 1 is a schematic diagram illustrating the configuration of an ophthalmological device according to an exemplary embodiment. FIG. 1 is a schematic diagram illustrating the configuration of an ophthalmological device according to an exemplary embodiment. FIG. 1 is a schematic diagram illustrating the configuration of an ophthalmological device according to an exemplary embodiment. FIG. 1 is a schematic diagram illustrating the configuration of an ophthalmological device according to an exemplary embodiment. FIG. 1 is a schematic diagram illustrating the configuration of an ophthalmological device according to an exemplary embodiment. FIG. 1 is a schematic diagram illustrating the configuration of an ophthalmological device according to an exemplary embodiment. FIG. 1 is a schematic diagram illustrating the configuration of an ophthalmological device according to an exemplary embodiment. FIG. 1 is a schematic diagram illustrating the configuration of an ophthalmological device according to an exemplary embodiment. FIG. 1 is a schematic diagram illustrating the configuration of an ophthalmological device according to an exemplary embodiment. FIG. 1 is a schematic diagram illustrating the configuration of an ophthalmological device according to an exemplary embodiment. FIG. 1 is a schematic diagram illustrating the configuration of an ophthalmological device according to an exemplary embodiment. FIG. 1 is a schematic diagram illustrating the configuration of an ophthalmological device according to an exemplary embodiment. FIG. 2 is a flow diagram illustrating processing performed by an ophthalmologic apparatus according to an exemplary embodiment. FIG. 2 is a flow diagram illustrating processing performed by an ophthalmologic apparatus according to an exemplary embodiment. It is a schematic diagram for explaining processing performed by an ophthalmologic apparatus according to an exemplary embodiment. It is a schematic diagram for explaining processing performed by an ophthalmologic apparatus according to an exemplary embodiment. FIG. 1 is a schematic diagram illustrating the configuration of an ophthalmological device according to an exemplary embodiment. FIG. 1 is a schematic diagram illustrating the configuration of an ophthalmological device according to an exemplary embodiment.

Some exemplary aspects of embodiments will be described in detail with reference to the drawings.

Any known technology can be combined with any aspect of the present disclosure. For example, any matter disclosed in the documents cited herein can be combined with any aspect of the present disclosure. Furthermore, any known technology in the technical field related to the present disclosure can be combined with any aspect of the present disclosure.

All contents disclosed in Patent Document 3 (Japanese Unexamined Patent Publication No. 2019-213733) are incorporated into the present disclosure by reference. Furthermore, any technical matter disclosed by the applicant of the present application regarding technology related to the present disclosure (matters disclosed in patent applications, papers, etc.) can be combined with any aspect of the present disclosure.

It is possible to at least partially combine any two or more of the various aspects of the present disclosure.

At least some of the functionality of the elements described in this disclosure is implemented using circuitry or processing circuitry. The circuitry or processing circuitry may include a general purpose processor, a special purpose processor, an integrated circuit, a central processing unit (CPU), a graphics processing unit (GPU) configured and/or programmed to perform at least some of the disclosed functions. ), ASIC (Application Specific Integrated Circuit), programmable logic device (for example, SPLD (Simple Programmable Logic Device), CPLD (Complex Programmable Log) IC Device), FPGA (Field Programmable Gate Array), conventional circuit configurations, and any of these A processor is considered to be a processing circuitry or circuitry that includes transistors and/or other circuitry. In this disclosure, the term circuitry, unit, means, or similar terminology refers to the disclosure Hardware that performs at least some of the functions disclosed herein, or hardware that is programmed to perform at least some of the functions disclosed. or may be known hardware programmed and/or configured to perform at least some of the functions described.A processor where the hardware may be considered a type of circuitry. , the term circuitry, unit, means, or similar terms is a combination of hardware and software, the software being used to configure the hardware and/or the processor.

<Background of the embodiment>
The embodiment aims to improve ophthalmological imaging in which images are collected while moving an optical system that satisfies Scheimpflug conditions. In the conventional technology, image acquisition is performed in the operation mode shown in FIG. The operation mode shown in Figure 1 is a combination of light emission from the light source (output of illumination light, projection of illumination light onto the subject's eye), exposure (photography) of the camera, and scan position (position of the optical system being moved). This shows the control (synchronous control). More specifically, the operation mode shown in FIG. 1 includes continuous emission of illumination light, repeated photographing (exposure) by a camera, and continuous movement of the optical system from the scan start position to the scan end position. It is a combination of. Repetitive camera exposure is achieved by alternating exposure and charge transfer (and exposure standby).

In such conventional ophthalmological imaging techniques, a blurred image may be obtained due to movement of the scanning position of the camera during exposure or eyeball movement of the subject's eye. Image blurring not only degrades image quality but also adversely affects the quality of image analysis such as corneal detection and cell detection.

One idea to solve this problem is to shorten the exposure time of the camera, but since the illumination light is continuously projected onto the patient's eye even when the exposure is not in progress, it puts a heavy burden on the patient. . It is also possible to synchronize scan position movement control and camera exposure control to move the scan position in a stepwise manner, that is, to move the optical system intermittently. Repeated starting and sudden stopping causes vibration, which adversely affects the quality of the image.

<Overview of embodiment>
The embodiments of the present disclosure have been created based on at least such a background, and control is performed so that the projection time of illumination light to the eye to be examined is shorter than the exposure time of the camera (imaging device). configured. This control includes control of an optical system (illumination system) that projects illumination light onto the eye to be examined and/or control of an optical system (imaging system) that photographs the eye to be examined using an image sensor.

Control of the lighting system may be any type of control, for example, electrical control such as light source control (on/off) or electronic shutter control, or mechanical shutter control or rotary shutter control. It may be mechanical control such as control, or it may be a combination of electrical control and mechanical control. Note that these shutters are provided in the illumination system, and are designed to switch between passing and blocking the illumination light output from the light source (that is, switching between projecting and not projecting the illumination light to the subject's eye). It is configured.

Control of the illumination system is not limited to control for switching between a state in which illumination light is projected onto the eye to be examined (projection state) and a state in which it is not projected (non-projection state); The control may be for modulating the intensity (light amount) of the illumination light. Note that switching between the projection state and the non-projection state corresponds to switching the intensity of the illumination light projected onto the eye to be examined between a positive value and zero. On the other hand, intensity modulation corresponds to switching the intensity of illumination light projected onto the eye to be examined between a first value and a second value that are different from each other. Here, both the first value and the second value are non-negative values, and one or both of the first value and the second value is a positive value. Therefore, switching between the projection state and the non-projection state corresponds to an example of intensity modulation.

The control of the photographing system may be any type of control, for example, electrical control such as image sensor control or electronic shutter control, or mechanical control such as mechanical shutter control or rotary shutter control. It may be electrical control or a combination of electrical control and mechanical control.

The embodiment relates to a technique for generating a series of digital images (referred to as Scheimpflug images) by photographing the eye to be examined multiple times while moving an illumination system and a photographing system that satisfy Scheimpflug conditions. In order to collect a series of Scheimpflug images, the system is configured to control at least one of the illumination system and the imaging system so that the projection time of the illumination light onto the eye to be examined is shorter than the exposure time of the imaging system (imaging device). ing.

If both the illumination light projection time and the image sensor exposure time are variable, embodiments may be configured to make the projection time shorter than a fixed or preset exposure time by controlling only the illumination system. Alternatively, the exposure time may be made longer than a fixed or preset projection time by controlling only the imaging system, or the projection time may be made longer than the exposure time by controlling both the illumination system and the imaging system. (In other words, the exposure time may be configured to be longer than the projection time).

If the projection time of the illumination light is variable and the exposure time of the imager is fixed or preset, embodiments are configured to make the projection time shorter than the exposure time by controlling the illumination system. Conversely, when the projection time of the illumination light is fixed or preset and the exposure time of the image sensor is variable, the embodiment is configured to make the exposure time longer than the projection time by controlling the imaging system. Ru.

In the embodiment, the length of the period during which illumination light is projected onto the subject's eye (projection period) is the length of the period during which the image sensor can receive light (exposure period) (exposure time). In addition to performing control so that the projection period is shorter than , it is configured to perform control so that at least a portion of the projection period and at least a portion of the exposure period overlap.

FIG. 2 shows an example of an operation mode based on such control. The operation mode shown in FIG. 2 is similar to the conventional operation mode shown in FIG. Unlike the operation mode shown in FIG. 1 in which the illumination light is emitted continuously, the illumination light is outputted intermittently (pulsed light emission). The illumination light emission (projection) sequence (multiple times of light emission arranged in chronological order) and the exposure sequence of the camera (multiple exposures arranged in chronological order) shown in FIG. 2 are synchronized with each other.

The period of each light emission in the sequence of illumination light corresponds to a projection period, and its length corresponds to a projection time. Similarly, each exposure period in the exposure sequence corresponds to an exposure period, and its length corresponds to an exposure time. The projection time may be constant or non-constant. The exposure time may be constant or non-constant.

In the operation mode shown in FIG. 2, for each exposure period in the exposure sequence, a portion of the exposure period coincides with one projection period. That is, for each exposure period in the exposure sequence, the length of the projection period (projection time) is shorter than the length of the exposure period (exposure time), and a part of this exposure period and this projection period overlaps with the whole.

By adopting the operation mode shown in FIG. 2, in each exposure period in the exposure sequence, exposure (light reception by the image sensor, charge accumulation) is actually performed only during the projection period that is shorter than the exposure period. , image blur caused by movement of the scan position and eye movement during exposure can be reduced compared to conventional ophthalmological imaging techniques. Therefore, it becomes possible to provide images of higher quality than conventional ophthalmological imaging techniques. Furthermore, it becomes possible to perform image analysis such as corneal detection and cell detection with higher quality than conventional ophthalmological image analysis technology. Moreover, since the time during which the illumination light is projected onto the eye to be examined can be shortened, there is no need to place a large burden on the examinee. Furthermore, since the optical system does not repeatedly start and stop suddenly, vibrations caused by such vibrations do not occur during scanning and do not adversely affect the quality of photographing.

Some exemplary aspects of the embodiments outlined above will now be described. Exemplary aspects described below include aspects of the ophthalmological apparatus, aspects of the method for controlling the ophthalmological apparatus, aspects of the program, aspects of the recording medium, etc., but the aspects of the embodiments are not limited to these. .

<Ophthalmological equipment>
Some exemplary aspects of ophthalmic devices according to embodiments are provided.

FIG. 3 shows the configuration of an ophthalmologic apparatus according to one aspect of the embodiment. The ophthalmologic apparatus 1000 of this example includes an image collection section 1010 and a control section 1020.

The image collection unit 1010 collects Scheimpflug images of the eye to be examined. The image acquisition unit 1010 includes an optical system that satisfies Scheimpflug conditions for projecting illumination light onto the subject's eye and photographing it with an image sensor, and performs a series of Scheimpflug tests while changing the illumination light projection position and photographing position. Collect images. That is, the image collection unit 1010 collects a series of Scheimpflug images while moving the scan position (the illumination light projection position and the photographing position). In other words, the image collecting unit 1010 collects a series of Scheimpflug images by scanning the subject's eye while maintaining the state in which the optical system satisfies Scheimpflug conditions. Several configuration examples of such an image collection unit 1010 are shown in FIGS. 4A to 4C.

The image collection unit 1010A shown in FIG. 4A will be described. The image collection unit 1010A includes an illumination system 1011 and an imaging system 1012. The illumination system 1011 is configured to project illumination light onto the eye to be examined. The illumination light may be slit light. The slit light is, for example, light whose beam cross-sectional shape is defined by a mechanism that forms a slit (slit) of a predetermined shape or another mechanism. The photographing system 1012 is configured to photograph the eye to be examined, and includes an image sensor 1013 and an optical system (not shown) that guides light from the eye to the image sensor 1013.

The illumination system 1011 and the photographing system 1012 are configured to satisfy Scheimpflug conditions and function as a Scheimpflug camera. More specifically, the plane passing through the optical axis of the illumination system 1011 (the plane including the object plane), the principal plane of the imaging system 1012, and the imaging plane of the image sensor 1013 intersect on the same straight line. An illumination system 1011 and an imaging system 1012 are configured. By using the illumination system 1011 and photographing system 1012 that satisfy the Scheimpflug condition, the photographing system 1012 can focus on all positions in the object plane (all positions along the optical axis of the illumination system 1011). You can take pictures in this condition.

In some exemplary embodiments, the image collection unit 1010A shown in FIG. 4A is configured to scan a three-dimensional region of the subject's eye with illumination light (e.g., slit light) to collect a series of Scheimpflug images. There is. The image collecting unit 1010A of this example is configured to collect a series of Scheimpflug images by repeatedly performing imaging while moving the projection position of the slit light onto a three-dimensional region of the eye to be examined.

In some exemplary embodiments, the image collection unit 1010A may be configured to scan a three-dimensional region of the eye to be examined by translating the slit light in a direction perpendicular to the longitudinal direction of the slit light. Such a scanning mode is different from that of a conventional anterior eye imaging device that scans the anterior eye segment by rotating a slit light.

The longitudinal direction of the slit light is the longitudinal direction of the beam cross section of the slit light at the projection position to the subject's eye, in other words, the longitudinal direction of the image of the slit light formed on the subject's eye, and is aligned with the body axis of the subject. It may be substantially coincident with the direction along the body axis (body axis direction). The dimension of the slit light in the longitudinal direction may be arbitrary. In some exemplary embodiments, the dimension of the slit light in the longitudinal direction may be greater than or equal to the corneal diameter in the body axis direction of the subject, and the distance of parallel movement of the slit light may be greater than or equal to the corneal diameter in the body axis direction of the subject. The corneal diameter may be greater than or equal to the corneal diameter in the direction orthogonal to the corneal diameter. Note that it may be possible to change the conditions (length, width, etc.) of the slit light depending on the applied photographing mode.

Although the series of Scheimpflug images collected by the image collection unit 1010A of this example are a group of images (a group of frames) collected temporally continuously (along a time series), they are a three-dimensional area of the eye to be examined. Since the images are a group of images collected sequentially from a plurality of different positions in the image, they are spatially distributed, unlike general moving images.

In the image collection unit 1010A of this example, the illumination system 1011 projects the slit light onto a three-dimensional region of the eye to be examined, and the imaging system 1012 projects the slit light from the illumination system 1011 onto the three-dimensional region of the eye to be examined. to photograph. Furthermore, the image acquisition unit 1010A of this example scans the eye to be examined while maintaining the state in which the illumination system 1011 and the imaging system 1012 satisfy the Scheimpflug condition, thereby obtaining a series of images representing a three-dimensional region of the eye to be examined. Collect Scheimpflug images.

In order to move the scan position with respect to the eye to be examined in the scan executed by the image collecting unit 1010, the image collecting unit 1010A of this example includes, for example, a mechanism (moving mechanism) for moving the illumination system 1011 and the imaging system 1012; Or, it may include other mechanisms. Mechanisms other than the moving mechanism include, for example, mechanisms (illumination scanners, movable illumination mirrors) that move the illumination position by deflecting illumination light (slit light), and mechanisms that deflect light from the subject's eye toward the image sensor to capture images. There are mechanisms that move the position (imaging scanner, movable imaging mirror), etc.

The control unit 1020 of the ophthalmological apparatus 1000 to which the image collection unit 1010A of the present example is applied controls the projection time of the slit light onto the eye to be examined so that it is shorter than the exposure time of the image sensor 1013, and the slit light to the eye to be examined. The image collecting unit 1010A is controlled so that at least part of the projection period and at least part of the exposure period of the image sensor 1013 overlap. The control of the image collection unit 1010A includes either or both of the control of the illumination system 1011 and the control of the photographing system 1012.

Furthermore, the control unit 1020 of the ophthalmological apparatus 1000 to which the image collection unit 1010A of this example is applied controls the exposure of the image sensor 1013 in controlling the image collection unit 1010A (imaging system 1012) to collect a series of Scheimpflug images. It may be configured to control the imaging system 1012 to repeat the steps. Each exposure in this repeated exposure corresponds to one Scheimpflug image.

In the operational mode of FIG. 2 described above, an example of repetitive exposure is shown ("camera exposure"). With such repeated exposures, a series of Scheimpflug images are collected. In the operation mode of FIG. 2, a plurality of pulsed lights are used as the illumination light, but the illumination light is not limited to this.

For example, repeated exposure can be achieved by using continuous light as illumination light, providing a shutter in the imaging system 1012, and switching the opening and closing of the shutter at high speed by the control unit 1020. The advantages of this example include low cost and easy control. Furthermore, when the purpose is to image floating objects present in the eye, a light source with a relatively large amount of light is used, but it is difficult to perform pulse control (pulse driving) of a large amount of light source with high precision. This example is considered advantageous. However, since continuous light is used as the illumination light, there is a disadvantage that the burden on the subject increases. A person who intends to apply this example to a specific patient should consider whether or not to apply this example by comparing and contrasting the advantages and disadvantages of this example and taking into account the necessity of testing. considered desirable.

The control unit 1020 may be configured to control the illumination system 1011 to modulate the intensity of illumination light in parallel with the control of the imaging system 1012 to repeat the exposure of the image sensor 1013.

In some examples, the control unit 1020 repeats on/off (projection/non-projection) of the illumination light (that is, the illumination light is intermittently projected onto the subject's eye), as in the operation mode of FIG. control of the illumination system 1011 so as to

In some other examples, the control unit 1020 controls the illumination system 1011 to switch the intensity of illumination light between two different values. Both of these two values are positive values. In some other examples, the control unit 1020 may control the illumination system 1011 to switch the intensity of the illumination light to three or more different values.

The control unit 1020 may be configured to perform synchronous control of the illumination system 1011 and the imaging system 1012 in controlling the image collection unit 1010 to collect a series of Scheimpflug images. That is, the control unit 1020 can cause the image collection unit 1010 to collect a series of Scheimpflug images by synchronously controlling the illumination system 1011 and the imaging system 1012. This synchronous control may be performed by synchronizing the control of the photographing system 1012 with the control of the illumination system 1011, by synchronizing the control of the illumination system 1011 with the control of the photographing system 1012, or by synchronizing the control of the illumination system 1011 with the control of the photographing system 1012, or by synchronizing the control of the illumination system 1011 with the control of the photographing system 1012. The control of the illumination system 1011 and the control of the imaging system 1012 may be synchronized with respect to the above. This specific operation may be, for example, an operation of a moving mechanism that moves the illumination system 1011 and the imaging system 1012.

Such synchronous control makes it possible to perform control for collecting a series of Scheimpflug images with high precision. In other words, through such synchronous control, at least part of the period during which the illumination light is projected onto the eye to be examined is controlled while controlling to make the projection time of the illumination light onto the eye to be examined shorter than the exposure time of the image sensor 1013 with high precision. It becomes possible to perform control to overlap with at least a part of the exposure period of the image sensor 1013 with high precision. For example, by such synchronous control, a scan like the operation mode shown in FIG. 2 can be performed with high accuracy.

When performing synchronized control between the illumination system 1011 and the photographing system 1012, the control unit 1020 controls the illumination light projection time so that the illumination light projection time is shorter than the exposure time of the image sensor 1013, and The illumination system 1011 and the imaging system 1012 may be synchronously controlled so that the total projection time and the total exposure time and non-exposure time of the image sensor 1013 are equal.

Here, the projection time of the illumination light is the length of the period during which the illumination light is projected onto the subject's eye (projection period), for example, the length of the period during which the pulse-controlled illumination light source is turned on (1 or the length of a period in which a shutter is open that allows continuous light from an illumination source to pass intermittently (the length of a single shutter open period). . In addition, the non-projection time of illumination light is the length of the period during which the illumination light is not projected onto the subject's eye (non-projection period), for example, the length of the period during which the pulse-controlled illumination light source is turned off ( The length of a period in which a shutter is closed, which intermittently passes continuous light from an illumination light source (the length of a period in which the shutter is closed). good.

The exposure time of the image sensor 1013 is the length of the period during which the image sensor 1013 can receive light (exposure period), and the non-exposure time of the image sensor 1013 is the length of the period during which the image sensor 1013 can receive light. This is the length of the non-exposure period (non-exposure period). During the non-exposure period, the image sensor 1013 performs operations such as charge transfer and exposure standby.

In the operation mode of FIG. 2, one upper side (top side) of the rectangular pulse train representing the light emission operation mode of the light source represents the projection period, and the length thereof represents the projection time. Furthermore, one lower side (base) of the rectangular pulse train representing the operation mode of light emission of the light source represents a non-projection period, and its length represents the non-projection time. Further, one upper side (top side) of the rectangular pulse train representing the exposure operation mode of the camera represents the exposure period, and the length represents the exposure time. Furthermore, one lower side (bottom side) of the rectangular pulse train representing the exposure operation mode of the camera represents a non-exposure period, and its length represents the non-exposure time.

The illumination system 1011 and the photographing system 1012 are synchronously controlled so that the projection time is shorter than the exposure time, and the total time of the projection time and non-projection time is equal to the total time of the exposure time and non-exposure time. By executing the above, an operation mode as illustrated in FIG. 2 (that is, a scan in which the projection time is shorter than the exposure time and at least part of the projection period overlaps with at least part of the exposure period) is obtained. can be performed with high precision.

The control unit 1020 may be configured to control the illumination system 1011 to change the projection time of illumination light to the eye to be examined. This projection time change control is, for example, a light source control that changes the frequency of a control pulse signal applied to a pulse-controlled illumination light source, or a change in the opening/closing frequency of a shutter that intermittently passes continuous light from an illumination light source. Includes shutter control. By employing projection time change control, it becomes possible to set the projection time of illumination light according to various conditions regarding the eye to be examined, the type of examination, the camera (imaging device 1013), and the like.

The control unit 1020 may be configured to perform projection time change control based on the speed of scanning performed by the image acquisition unit 1010. That is, the control unit 1020 may be configured to execute projection time change control based on the movement speed of the scan position with respect to the eye to be examined in the scan executed by the image acquisition unit 1010.

In some exemplary aspects, the control unit 1020 controls the illumination system 1011 to change the projection time of the illumination light based on the speed of the illumination system 1011 and the imaging system 1012 that are moved by the aforementioned moving mechanism. may be configured. In other words, the control unit 1020 in some exemplary embodiments may be configured to perform projection time change control depending on the content of control over the moving mechanism.

Further, in some exemplary embodiments, projection time change control is performed in accordance with the content of control over mechanisms other than the moving mechanism (the above-mentioned illumination scanner, movable illumination mirror, photographing scanner, movable photographing mirror, etc.). It may be configured as follows.

By employing projection time change control based on scan speed, it becomes possible to set the illumination light projection time that matches the scan speed and execute Scheimpflug image collection.

The control unit 1020 may be configured to execute projection time change control according to the applied shooting mode. For example, the control unit 1020 is configured to determine the moving speed of the scanning position with respect to the eye to be examined (for example, the speed of the illumination system 1011 and the imaging system 1012 moved by the moving mechanism) based on the applied imaging mode. You can leave it there.

The imaging mode is a predetermined set of operation details for each type of examination (imaging, scanning). Although the number of photographing modes to be prepared is arbitrary (one or more), in this example, it is assumed that two or more photographing modes are prepared. The user or the control unit 1020 selects a shooting mode to be applied to the eye to be examined from among the two or more prepared shooting modes. The number of selected shooting modes may be arbitrary (one or more). When two or more imaging modes are selected, for example, two or more scans corresponding to the selected two or more imaging modes are sequentially applied to the eye to be examined.

The ophthalmological apparatus 1000 of this example holds photographing mode information in which a plurality of photographing modes are recorded in, for example, a storage device within the control unit 1020. Alternatively, the ophthalmological apparatus 1000 of this example can refer to imaging mode information stored in an external device.

An example of photographing mode information is shown in FIG. The photographing mode information 1200 in this example includes an anterior segment shape photographing mode, an inflammatory cell photographing mode, an intraocular lens (IOL) photographing mode, and an intraocular contact lens (ICL) photographing mode as photographing modes. There is. Note that the types of shooting modes are not limited to these.

Further, the photographing mode information 1200 of this example includes illumination control pulse height, illumination control pulse width, and scan range as types of photographing operation, that is, types of parameters related to scanning. The illumination control pulse height is the height of each pulse in a control pulse signal applied to a pulse-controlled illumination light source. The illumination control pulse width is the width of each pulse in a control pulse signal applied to a pulse-controlled illumination light source. The scan range is the distance that illumination light moves during scanning. The shooting mode information 1200 in this example is applied when a pulse-controlled illumination light source (an illumination light source that emits pulsed light) is used, but similar parameters may be provided in other cases as well. Those skilled in the art will understand.

The anterior segment shape imaging mode is a imaging mode used to observe and analyze the shape of anterior segment tissues such as the cornea, angle, iris, and crystalline lens. In the anterior segment shape imaging mode, the scan range is set relatively large (20 mm) in order to collect images over a wide range of the anterior segment. In addition, considering the large light reflection on the cornea and the fast scanning speed, the illumination control pulse height is set relatively large (0.1 milliwatts) and the illumination control pulse width is set relatively small. (10 ms).

The inflammatory cell imaging mode is an imaging mode used to observe and analyze inflammatory cells (anterior chamber cells) present in the anterior chamber. In the inflammatory cell imaging mode, the illumination control pulse height is set relatively large (1 milliwatts), and the illumination control pulse width is set sufficiently small (5 milliseconds). The scan range is set relatively small (2 mm).

The IOL photography mode is a photography mode used to observe and analyze an intraocular lens implanted in an eye to be examined. In the IOL imaging mode, the illumination control pulse height is relatively small, taking into account that it is required to accurately determine the position of the intraocular lens with respect to the pupil and cornea, and that light reflection by the intraocular lens is relatively small. (0.1 milliwatts) and the lighting control pulse width was set to medium (20 milliseconds). Furthermore, in the IOL photography mode, the intraocular lens located behind the iris is photographed through the pupil, so the scan range is set relatively large (15 mm) to sufficiently cover the pupil area.

The ICL photography mode is a photography mode used to observe and analyze an intraocular contact lens implanted in an eye to be examined. In the ICL imaging mode, it is required to image the hole formed in the center of the intraocular contact lens, a narrow scan interval is required to detect its position with high accuracy, and the intraocular contact lens Considering that the lighting control pulse itself is not a moving object, the lighting control pulse width is set large (30 milliseconds), and the lighting control pulse height is accordingly set relatively small (0.1 milliwatts). ). The scanning range was set to medium (10 mm).

By referring to such shooting mode information 1200, the control unit 1020 specifies parameter values (lighting control pulse height value, lighting control pulse width value, scan range value) corresponding to the selected shooting mode. can do.

The control unit 1020 may be configured to determine the scan speed according to the shooting mode. For example, the control unit 1020 may be configured to determine the scan speed based on a shooting mode selected from two or more preset shooting modes. In some example aspects, the controller 1020 may be configured to determine the scan speed based on parameter values identified with reference to the imaging mode information 1200.

An example of processing for determining scan speed will be explained. The premise is that there is a limit to how long humans can endure blinking (how long they can keep their eyes open). This time limit is defined as "T0". The limit time T0 may be a predetermined value or may be a value set for each subject (eye to be examined). Further, the time taken for scanning (scan time), that is, the time from the start of scanning to the end of scanning, is defined as "ST". The scan time ST is set to the limit time T0 or less (ST≦T0). In the photographing mode information 1200, a scan range for each photographing mode is set. The scan range corresponding to any shooting mode is defined as "SD". The scan speed corresponding to this shooting mode is defined as "SS". At this time, the scan speed SS can be determined by the following formula: SS=SD/ST.

Based on these assumptions and calculations, it is possible to determine the scan speed corresponding to each imaging mode. The photographing mode information may include the scan speed value determined in this manner. Photographing mode information 1210 shown in FIG. 6 is obtained by adding scan speed to photographing mode information 1200 shown in FIG. 5. The scan speed corresponding to each photographing mode in the photographing mode information 1210 is calculated on the assumption that "scan time ST=2 seconds".

The premises and calculation method for determining the scan speed value are not limited to the examples described above. Furthermore, the types of parameters whose values are determined by the control unit 1020 are not limited to scan speed.

In some exemplary embodiments, the number of Scheimpflug images collected in a scan (image count) can be a prerequisite. In other words, predetermined parameters can be set so that a predetermined number of Scheimpflug images are collected. The parameters set based on the number of images may be, for example, one or more of the illumination control pulse width, pulse interval (interval between two adjacent pulses), scan range, scan speed, and other parameters. .

The control unit 1020 may be configured to control the imaging system 1012 to change the exposure time of the image sensor 1013. This exposure time change control is, for example, an image sensor control that changes the frequency of a control pulse signal applied to the image sensor 1013, or a shutter that changes the opening/closing frequency of a shutter that intermittently allows light guided to the image sensor 1013 to pass through. Contains control. By employing exposure time change control, it becomes possible to set the exposure time of the image sensor 1013 according to various conditions regarding the eye to be examined, the type of examination, illumination light, and the like.

The control unit 1020 may be configured to perform exposure time change control based on the speed of the scan performed by the image acquisition unit 1010. That is, the control unit 1020 may be configured to perform exposure time change control based on the movement speed of the scan position with respect to the eye to be examined in the scan performed by the image collection unit 1010.

In some exemplary aspects, the control unit 1020 controls the imaging system 1012 to change the exposure time of the image sensor 1013 based on the speed of the illumination system 1011 and the imaging system 1012 that are moved by the above-described moving mechanism. It may be configured as follows. In other words, the control unit 1020 in some exemplary embodiments may be configured to perform exposure time change control depending on the content of control over the moving mechanism.

Further, in some exemplary embodiments, the exposure time change control is performed in accordance with the content of control over mechanisms other than the moving mechanism (the above-mentioned illumination scanner, movable illumination mirror, photographing scanner, movable photographing mirror, etc.). It may be configured as follows.

By employing exposure time change control based on the scan speed, it becomes possible to set the exposure time of the image sensor 1013 that matches the scan speed and execute Scheimpflug image collection.

The control unit 1020 may be configured to perform exposure time change control according to the applied shooting mode. For example, the control unit 1020 is configured to determine the moving speed of the scanning position with respect to the eye to be examined (for example, the speed of the illumination system 1011 and the imaging system 1012 moved by the moving mechanism) based on the applied imaging mode. You can leave it there.

Similar to the shooting mode related to projection time change control, the shooting mode related to exposure time change control is a mode in which the operation contents for each type of examination (photography, scanning) are set in advance, and one or more arbitrary number of images can be taken. mode is prepared. When two or more imaging modes are prepared, the user or the control unit 1020 selects the imaging mode applied to the eye to be examined from among these imaging modes. The number of photographing modes to be selected may be one or more. When two or more imaging modes are selected, for example, two or more scans corresponding to the selected two or more imaging modes are sequentially applied to the eye to be examined.

As in the case of projection time change control, the ophthalmological apparatus 1000 capable of executing exposure time change control holds photographing mode information in which a plurality of photographing modes are recorded, for example, in a storage device within the control unit 1020. Alternatively, it is possible to refer to shooting mode information stored in an external device.

As in the case of projection time change control, the control unit 1020 of the ophthalmological apparatus 1000 that can execute exposure time change control refers to the shooting mode information 1200 in FIG. 5 and/or the shooting mode information 1210 in FIG. It may be configured to specify parameter values (value of illumination control pulse height, value of illumination control pulse width, value of scan range, value of scan speed) corresponding to the selected imaging mode.

As in the case of the projection time change control, the control unit 1020 of the ophthalmological apparatus 1000 that can execute the exposure time change control may be configured to calculate the scan speed based on the imaging mode. The method for calculating the scan speed may be the same as in the case of projection time change control.

As in the case of projection time change control, the control unit 1020 of the ophthalmological apparatus 1000 that can execute exposure time change control is configured to execute exposure time change control using an arbitrary parameter such as the number of images as a precondition. It's okay.

The projection time change control and the exposure time change control may be executed separately or may be executed at least partially in cooperation (at least partially in conjunction). As an example of the latter, it is possible to execute exposure time change control using arbitrary information (final generated information, intermediate generated information) obtained in projection time change control; conversely, in exposure time change control, It is also possible to execute projection time change control using the obtained arbitrary information (final generation information, intermediate generation information). Furthermore, it is also possible to execute the projection time change control and the exposure time change control at least partially in parallel.

In some example aspects, the imaging system 1012 of the image acquisition unit 1010A of FIG. 4A may include more than one imaging system. An example is shown in FIG. 4B. Hereinafter, unless otherwise specified, any items regarding the image collection unit 1010A in FIG. 4A can be combined with the image collection unit 1010B of this example.

The imaging system 1012A of the image collection unit 1010B includes a first imaging system 1014 and a second imaging system 1015. The first imaging system 1014 and the second imaging system 1015 are arranged to photograph the eye to be examined from mutually different directions.

The image collecting unit 1010B of this example scans a three-dimensional region of the eye to be examined from two different directions in a scan using slit light (slit scan) for collecting a series of Scheimpflug images. The second imaging system 1015 may be configured to take an image. For example, images are collected so that the longitudinal direction of the beam cross section of the slit light at the point of incidence on the subject's eye is the vertical direction (Y direction), and the moving direction of the slit light is the horizontal direction (horizontal direction, X direction). 1010B, the first imaging system 1014 and the second imaging system 1015 are configured such that one of them photographs the eye to be examined from the left oblique direction, and the other one photographs the eye to be examined from the right oblique direction. It may be placed.

A series of Scheimpflug images collected by the first imaging system 1014 is referred to as a first Scheimpflug image group, and a series of Scheimpflug images collected by the second imaging system 1015 is referred to as a second Scheimpflug image group. The series of Scheimpflug images collected by such an image collection unit 1010B includes a first Scheimpflug image group and a second Scheimpflug image group.

Note that without performing a slit scan, one Scheimpflug image (first Scheimpflug image) is acquired by the first imaging system 1014, and one Scheimpflug image (second Scheimpflug image) is acquired by the second imaging system 1015. ), for convenience of terminology, one Scheimpflug image acquired by the first imaging system 1014 may be referred to as a first Scheimpflug image group, and one Scheimpflug image acquired by the second imaging system 1015 may be referred to as a first Scheimpflug image group. The Scheimpflug images are sometimes referred to as a second Scheimpflug image group. Thus, in this disclosure, the term "group" may be used not only when multiple elements are included, but also when only one element is included.

When the image collection unit 1010B of this example executes a slit scan, the first imaging system 1014 and the second imaging system 1015 perform the imaging of the eye to be examined in parallel. That is, the image collection unit 1010B performs imaging by the first imaging system 1014 and imaging by the second imaging system 1015 in parallel while moving the projection position of the slit light with respect to the three-dimensional region of the eye to be examined.

Furthermore, the image collection unit 1010B of this example is configured to perform the imaging by the first imaging system 1014 (collection of Scheimpflug images) and the imaging by the second imaging system 1015 (collection of Scheimpflug images) in synchronization with each other. It may be configured. By referring to this synchronization relationship, the first Scheimpflug image group and the second Scheimpflug image group can be easily associated without using image processing or the like. This association is performed, for example, by associating Scheimpflug images with a small difference in their acquisition times.

In such an aspect, the ophthalmological apparatus 1000 refers to the mutual synchronization relationship between the imaging by the first imaging system 1014 and the imaging by the second imaging system 1015, and converts the series of Scheimpflug images corresponding to the slit scan into the first It can be reconstructed from the Scheimpflug image group and the second Scheimpflug image group.

FIG. 4C shows a configuration example of an ophthalmological apparatus 1000 to which the image collection unit 1010B shown in FIG. 4B is applied. In the image collection unit 1010B of this example, the optical axis of the first imaging system 1014 and the optical axis of the second imaging system 1015 are arranged to be inclined in opposite directions with respect to the optical axis of the illumination system 1011. .

A series of Scheimpflug images collected by the imaging system 1012A of this example are directly or indirectly sent to the data processing device 1100. The data processing device 1100 is configured to be able to execute data processing such as image processing, and may be an element of the ophthalmological device 1000 of this example, and can perform data communication with the ophthalmological device 1000 of this example. It may be installed in a computer or system.

Data processing device 1100 includes an image selection section 1110. The image selection unit 1110 is configured to select an image from the first Scheimpflug image group acquired by the first imaging system 1014 and the second Scheimpflug image group acquired by the second imaging system 1015. . For example, the image selection unit 1110 may be configured to select one of the first Scheimpflug image acquired by the first imaging system 1014 and the second Scheimpflug image acquired by the second imaging system 1015.

The image selection unit 1110 determines the correspondence between the first Scheimpflug image group and the second Scheimpflug image group respectively collected based on the synchronization of the imaging by the first imaging system 1014 and the imaging by the second imaging system 1015. a new series of Scheimpflug images corresponding to the slit scan that acquired the first Scheimpflug image group and the second Scheimpflug image group based on the first Scheimpflug image group and the second Scheimpflug image group; It is configured. That is, the image selection unit 1110 is configured to reconstruct a series of Scheimpflug images from the first Scheimpflug image group and the second Scheimpflug image group collected by the first imaging system 1014 and the second imaging system 1015, respectively. has been done.

The method of image selection processing performed by the image selection unit 1110 may be arbitrary. For example, the method of image selection processing is determined and determined based on predetermined conditions and predetermined parameters, such as the configuration and/or arrangement of the first imaging system 1014 and the second imaging system 1015, and the purpose and/or use of image selection. /or can be selected.

The image collection unit 1010B performs imaging by the first imaging system 1014 and imaging by the second imaging system 1015 in mutual synchronization. Further, as described above, the optical axis of the first imaging system 1014 and the optical axis of the second imaging system 1015 are arranged to be inclined in opposite directions with respect to the optical axis of the illumination system 1011. For example, the optical axis of the first imaging system 1014 is arranged to be inclined to the left with respect to the optical axis of the illumination system 1011, and the optical axis of the second imaging system 1015 is arranged to be inclined to the left with respect to the optical axis of the illumination system 1011. It is placed tilted to the right. The first imaging system 1014 and the second imaging system 1015 arranged in this manner may be referred to as a left imaging system and a right imaging system, respectively.

The inclination angle of the optical axis of the first imaging system 1014 with respect to the optical axis of the illumination system 1011 and the inclination angle of the optical axis of the second imaging system 1015 with respect to the optical axis of the illumination system 1011 may be equal to each other or may be different from each other. You may. Further, these inclination angles may be fixed or variable.

In this example, for example, the illumination system 1011 is configured and arranged to project slit light whose cross-sectional longitudinal direction is oriented in the Y direction toward the subject's eye from the front direction, and the image collecting unit 1010B By integrally moving the illumination system 1011, first imaging system 1014, and second imaging system 1015 in the X direction, a slit scan is applied to a three-dimensional region of the anterior segment of the subject's eye.

The image selection unit 1110 of this example selects artifacts based on the correspondence between the first Scheimpflug image group collected by the first imaging system 1014 and the second Scheimpflug image group collected by the second imaging system 1015. By selecting a plurality of Scheimpflug images that do not include from the first Scheimpflug image group and the second Scheimpflug image group, a new Scheimpflug image corresponding to the slit scan that collected the first Scheimpflug image group and the second Scheimpflug image group A selection of a series of Scheimpflug images is performed. This artifact may be any type of artifact. When performing an anterior segment scan as in this example, this artifact may be an artifact caused by corneal reflection (referred to as a corneal reflection artifact). Some examples of processing executed by the image selection unit 1110 will be described below.

The projection position (Scheimpflug image, frame) of the slit light where the corneal reflection artifact occurs is different between the left imaging system and the right imaging system. For example, as in this example, the illumination system 1011, the first imaging system 1014, and the second imaging system 1015 project the slit light whose cross-sectional longitudinal direction is oriented in the Y direction from the front direction onto the subject's eye. When performing a slit scan by integrally moving the cornea in the X direction, the corneal reflected light of the slit light is likely to enter the left imaging system when the slit light is projected to a position to the left of the corneal vertex; When the slit light is projected to a position to the right of the corneal vertex, it is likely to enter the right imaging system.

Considering these circumstances, the image selection unit 1110 in some exemplary embodiments first selects the corneal vertex from among the first Scheimpflug image group collected by the first imaging system 1014 as the left imaging system. The corresponding Scheimpflug image (first corneal apex image) is specified, and the Scheimpflug image (first corneal apex image) corresponding to the corneal vertex is identified from the second Scheimpflug image group collected by the second imaging system 1015 as the right imaging system. 2 corneal vertex images).

In some exemplary aspects, the process of identifying the corneal apex image includes detecting an image corresponding to the corneal surface from each Scheimpflug image included in the first Scheimpflug image group, and detecting the pixels of these detected images. The process may include a process of specifying the pixel closest to the ophthalmologic apparatus 1000 of this example based on the Z coordinate of the image, and a process of setting the Scheimpflug image including the specified pixel as the first corneal vertex image. Setting of the second corneal vertex image may be performed in the same manner.

Next, the image selection unit 1110 selects a Scheimpflug image group located to the right of the first corneal vertex image from the first Scheimpflug image group, and selects a Scheimpflug image group located to the right of the first Scheimpflug image from the second Scheimpflug image group. A Scheimpflug image group located to the left of the corneal apex image is selected, and a series of Scheimpflug images consisting of the two selected Scheimpflug image groups (as well as the first corneal apex image and/or the second corneal apex image) is generated. form an image. This provides a series of Scheimpflug images that span the three-dimensional region of the anterior segment to which the slit scan has been applied and that do not (and are likely to) contain corneal reflection artifacts.

Another example of the process of identifying a corneal vertex image will be described. The image selection unit 1110 of some example aspects selects which of two images acquired substantially simultaneously by the first imaging system 1014 (e.g., the left imaging system) and the second imaging system 1015 (e.g., the right imaging system). Determine whether a corneal reflex artifact is included in the image. This corneal reflection artifact determination process includes a predetermined image analysis, and includes, for example, threshold processing regarding brightness information assigned to pixels. Note that the process of determining two images acquired substantially simultaneously can be executed based on the synchronous relationship between the imaging by the first imaging system 1014 and the imaging by the second imaging system 1015.

The threshold processing used in the artifact determination processing is performed, for example, to identify pixels to which a luminance value exceeding a preset threshold is assigned. Typically, the threshold value may be set higher than the brightness value of the slit light image (projection area of the slit light) in the image. Thereby, the image selection unit 1110 is configured to determine an image brighter than the slit light image as an artifact, without determining the slit light image as an artifact. Considering that an image brighter than a slit light image in a Scheimpflug image is likely to be an image resulting from regular reflection on the cornea, the artifacts detected by the image selection unit 1110 configured in this way are as follows: It can be considered that there is a high possibility that this is a corneal reflection artifact.

The image selection unit 1110 may perform any image analysis other than threshold processing, such as pattern recognition, segmentation, and edge detection, for artifact determination. Generally, any information processing technology, such as image analysis, image processing, machine learning, artificial intelligence, and cognitive computing, can be applied to artifact determination.

As a result of the artifact determination, when it is determined that one of the two images acquired substantially simultaneously by the first imaging system 1014 and the second imaging system 1015 includes an artifact, the image selection unit 1110 selects the artifact from the other image. Select an image. In other words, the image selection unit 1110 selects one of the two images acquired substantially simultaneously by the first imaging system 1014 and the second imaging system 1015, which is not the one image determined to contain the artifact. select.

Assuming a case where it is determined that both images contain an artifact, the image selection unit 1110 performs, for example, a process of evaluating the magnitude of the adverse effect that the artifact has on observation and diagnosis, and selects the image that has less adverse effect. The configuration may also be configured to execute the processing. This evaluation process may be performed based on any one or more of the dimensions, strength, shape, and position of the artifact, for example. Typically, artifacts with large dimensions, artifacts with high intensity, and artifacts located in or near a region of interest such as a slit light image are evaluated to have a large negative impact.

Note that in cases where both images include artifacts, the artifact removal disclosed in Patent Document 3 (Japanese Unexamined Patent Publication No. 2019-213733) may be applied.

By providing the image selection unit 1110 as described above, it is possible to provide an image of a three-dimensional region of the eye to be examined that does not include artifacts that interfere with observation, analysis, and diagnosis. Furthermore, it becomes possible to provide an image of a three-dimensional region of the eye to be examined that does not include artifacts to subsequent processing. For example, it becomes possible to construct a three-dimensional image or rendered image of the eye to be examined based on a group of images that do not include artifacts.

Even if the images are obtained by photographing substantially the same position, the dimensions of the depicted predetermined region are different between the image obtained with the left imaging system and the image obtained with the right imaging system. There is. For example, in the left and right images obtained by photographing substantially the same position with the left and right imaging systems, the thickness of the cornea and the dimensions of the inflammatory cells depicted may differ from each other. Even in such a case, the size of the object can be adjusted by using the image selection unit 1110.

FIG. 7 shows the configuration of an ophthalmologic apparatus according to one aspect of the embodiment. The ophthalmological apparatus 1500 of this example includes an evaluation processing section 1030 in addition to an image collection section 1010 and a control section 1020. The image collection unit 1010 of this example may be configured, at least in part, similarly to the image collection unit 1010 of the ophthalmologic apparatus 1000 in FIG. 3 . Further, the control unit 1020 of this example may be configured at least partially in the same manner as the control unit 1020 of the ophthalmologic apparatus 1000 in FIG. 3 .

The evaluation processing unit 1030 generates evaluation information for the eye to be examined based on the series of Scheimpflug images collected by the image collection unit 1010. The evaluation information includes information generated by evaluating data obtained in a medical examination (clinical examination) of the subject's eye. The medical test may be any ophthalmological test that utilizes images of the eye.

In some exemplary embodiments, the medical test may be a test for floaters present within the subject's eye. Examples of floating substances include inflammatory cells (anterior chamber cells), proteins (anterior chamber flare), and other floating substances within the anterior chamber, as well as vitreous fibers, detached retinal cells, and other floating substances within the vitreous body. There is.

Because intraocular floaters involve movement (movement), images of intraocular floaters obtained by conventional imaging are often blurred, and the images are evaluated with high quality (accuracy, precision, reproducibility, etc.) That was difficult. On the other hand, according to the imaging according to this aspect, as described above, the blurring of the image can be reduced, so it is possible to improve the quality of the examination regarding intraocular floaters.

Note that, according to the photographing according to this aspect, it is also possible to reduce the adverse effects that eye movement, body movement, pulsation, etc. have on the image. Therefore, it is also possible to improve the quality of evaluation regarding objects other than intraocular floaters. For example, the present embodiment can be applied to evaluation of any part of the eyeball, evaluation of artificial objects (intraocular lenses, intraocular contact lenses, minimally invasive glaucoma surgery (MIGS) devices, etc.).

The evaluation processing unit 1030 executes a process of processing a series of Scheimpflug images collected by the image collection unit 1010 to generate processed image data, and a process of generating evaluation information based on the generated processed image data. It may be configured to do so. For example, the evaluation processing unit 1030 performs a process of constructing a three-dimensional image (an example of processed image data) from a plurality of Scheimpflug images included in a series of Scheimpflug images, and a process of constructing evaluation information based on this three-dimensional image. It may be configured to execute the process of generating the data. The evaluation processing unit 1030 also performs processing to construct a three-dimensional image from a plurality of Scheimpflug images included in a series of Scheimpflug images, and generates a rendered image (an example of processed image data) from this three-dimensional image. and a process of generating evaluation information based on this rendered image. Furthermore, the evaluation processing unit 1030 may perform arbitrary digital image processing such as correction, editing, and emphasis as processing of the Scheimpflug image.

The evaluation information generated in this example may include any information (inflammatory state information) related to the inflammatory state of the subject's eye. For example, inflammatory status information includes information regarding inflammatory cells (anterior chamber cells) present in the anterior chamber, information regarding proteins (anterior chamber flare) present within the anterior chamber, information regarding clouding of the crystalline lens, and information regarding the onset and course of the disease. and information regarding disease activity. Further, the inflammatory state information may include comprehensive information based on two or more pieces of information.

Information regarding inflammatory cells includes information on arbitrary parameters such as density (concentration), number, position, and distribution of inflammatory cells, and evaluation information based on information on predetermined parameters. Evaluation information regarding inflammatory cells may be referred to as cell evaluation information. Information regarding anterior chamber flare includes information on arbitrary parameters such as the concentration, number, position, and distribution of flares, and evaluation information based on information on predetermined parameters. Information regarding lens opacity includes information on arbitrary parameters such as concentration, number, position, and distribution of opacity, and evaluation information based on information on predetermined parameters. Information regarding the onset and progress of a disease includes information on arbitrary parameters such as the presence or absence of onset, the state of onset, the period of disease, and the state of progress, and evaluation information based on information on predetermined parameters. Information regarding disease activity includes information on arbitrary parameters such as the state of disease activity, evaluation information based on information on predetermined parameters, and the like.

In some exemplary aspects, the evaluation processor 1030 includes information generated by the ophthalmic device 1500 as well as information input externally to the ophthalmic device 1500 (e.g., information acquired by another ophthalmic device, The evaluation information may also be generated based on the information inputted by the user.

The information referenced to generate the inflammatory status information exemplified above may be arbitrary, and may include, for example, the classification criteria for uveitis diseases proposed by The Standardization of Uveitis Nomenclature (SUN) Working Group. Moyoi (“Standardization of uveitis nomenclature for reporting clinical data. Results of the First International Workshop”, Ame rican Journal of Ophthalmology, Volume 140, Issue 3, September 2005, Pages 509-516).

Note that the inflammatory state information is not limited to the above example. Furthermore, the information referred to in order to generate the inflammatory state information is not limited to the above example.

In this example, the projection time of the illumination light to the subject's eye is set to be shorter than the exposure time of the image sensor, and at least part of the illumination light projection period to the subject's eye and at least part of the exposure period of the image sensor are set. A series of Scheimpflug images are acquired by controlling the image acquisition unit 1010 so that the images overlap. The region of the eye to be examined to which this slit scan is applied includes at least a portion or the entire region to which a slit scan is applied for acquiring data used to generate inflammatory status information. For example, when generating inflammatory status information including information regarding inflammatory cells and/or information regarding anterior chamber flare, a slit scan is applied to a region including at least a portion of the anterior chamber. An example of this is the inflammatory cell imaging mode described above. When generating inflammatory status information including information regarding lens opacity, a slit scan is applied to a region including at least a portion of the lens. The same applies when two or more Scheimpflug images are acquired by a method other than slit scanning, or when only one Scheimpflug image is acquired.

Generally, the area (site) of the eye to be examined to which a slit scan is applied is at least a portion of the anterior segment (e.g. cornea, iris, anterior chamber, angle, ciliary body, zonules, crystalline lens, nerves, blood vessels). lesions; treatment scars; artificial objects such as intraocular lenses, intraocular contact lenses, and MIGS devices); and/or at least a portion of the posterior segment of the eye (e.g., vitreous body, retina, choroid, sclera). , tissues such as the optic disc, lamina cribrosa, macula, nerves, and blood vessels; lesions; treatment scars; and artificial objects such as artificial retinas). In some exemplary embodiments, a slit scan may be applied to at least a portion of the near-ocular tissue, such as the eyelids or meibomian glands. In some exemplary embodiments, the three-dimensional region includes any two or all of at least a portion of the anterior segment, at least a portion of the posterior segment, and at least a portion of the near-ocular tissue. Slit scanning may be applied to a three-dimensional region.

Embodiments capable of generating inflammatory status information that includes cellular assessment information may be configured to perform at least one of the following three processes: (1) in the anterior chamber of the subject's eye; Segmentation (referred to as first segmentation or anterior chamber segmentation) to identify the corresponding image region (referred to as the anterior chamber region); (2) Segmentation to identify the image region (referred to as the cell region) corresponding to inflammatory cells; Segmentation (referred to as second segmentation or cell segmentation); (3) Processing for generating cell evaluation information (referred to as cell evaluation information generation processing). Some exemplary aspects of such embodiments are described below.

In the first exemplary aspect, the evaluation processing unit 1030 performs a first segmentation for identifying an anterior chamber region from a Scheimpflug image, and a first segmentation for identifying a cell region from the anterior chamber region identified by the first segmentation. The present invention is configured to perform at least one of two segmentations and a cell evaluation information generation process for generating cell evaluation information from the cell region specified by the second segmentation. The first segmentation in this aspect may be performed using a neural network trained by machine learning, but is not limited thereto. The second segmentation in this aspect may be performed using a neural network trained by machine learning, but is not limited thereto. The cell evaluation information generation process of this aspect may be executed using a neural network trained by machine learning, but is not limited thereto. Details of this aspect will be described later.

In a second exemplary aspect, the evaluation processing unit 1030 performs a second segmentation for identifying a cellular region from a Scheimpflug image and a second segmentation for identifying a cellular region from a Scheimpflug image without performing a first segmentation for identifying an anterior chamber region. 2. The cell evaluation information generating process is configured to execute at least one process of generating cell evaluation information from the cell area specified by segmentation. The second segmentation in this aspect may be performed using a neural network trained by machine learning, but is not limited thereto. The cell evaluation information generation process of this aspect may be executed using a neural network trained by machine learning, but is not limited thereto. Details of this aspect will be described later.

In a third exemplary aspect, the evaluation processing unit 1030 can obtain cell evaluation information from the Scheimpflug image without performing the first segmentation for identifying the anterior chamber region and the second segmentation for identifying the cell region. The cell evaluation information generation process is configured to generate cell evaluation information. The cell evaluation information generation process of this aspect may be executed using a neural network trained by machine learning, but is not limited thereto. Details of this aspect will be described later.

In a fourth exemplary aspect, the evaluation processing unit 1030 performs the first segmentation for identifying the anterior chamber region from the Scheimpflug image and the first segmentation for identifying the anterior chamber region from the Scheimpflug image without performing the second segmentation for identifying the cell region. The present invention is configured to execute at least one of cell evaluation information generation processing for generating cell evaluation information from the anterior chamber region specified by one segmentation. The first segmentation in this aspect may be performed using a neural network trained by machine learning, but is not limited thereto. The cell evaluation information generation process of this aspect may be executed using a neural network trained by machine learning, but is not limited thereto. Details of this aspect will be described later.

In a fifth exemplary aspect, the evaluation processor 1030 is performed without using a neural network trained by machine learning. This aspect includes a first segmentation for analyzing a Scheimpflug image to identify an anterior chamber region, and a second segmentation for analyzing an anterior chamber region identified by this first segmentation to identify a cell region. , and a cell evaluation information generation process for generating cell evaluation information based on the cell region specified by the second segmentation. Details of this aspect will be described later.

In a sixth exemplary aspect, the evaluation processing unit 1030 performs any one or more of at least a portion of the first segmentation, at least a portion of the second segmentation, and at least a portion of the cell evaluation information generation process. is configured to be executed by a machine learning-based configuration, and processes other than those executed by the machine learning-based configuration are executed by a non-machine learning-based configuration. This aspect is realized, for example, by partially combining any of the above-described first to fourth exemplary aspects and the fifth exemplary aspect, so please refer to the detailed description thereof. is omitted.

As mentioned above, the first segmentation may be a machine learning-based process, a non-machine learning-based process, or a combination of machine learning-based processes and non-machine learning-based processes. The second segmentation may also be a machine learning-based process, a non-machine learning-based process, or a combination of a machine learning-based process and a non-machine learning-based process. Furthermore, the cell evaluation information generation process may be a machine learning-based process or a non-machine learning-based process, or may be a combination of a machine learning-based process and a non-machine learning-based process.

The type of data input into the first segmentation, the type of data input into the second segmentation, and the type of data input into the cell evaluation information generation process may all be arbitrary. Hereinafter, some examples of possible combinations of a plurality of processes including the first segmentation, the second segmentation, and the cell evaluation information generation process will be described.

FIG. 8 shows an example of the configuration of the evaluation processing section 1030. The evaluation processing section 1030A of this example includes a first segmentation section 1031, a second segmentation section 1032, and a cell evaluation information generation processing section 1033.

The first segmentation unit 1031 includes a processor that performs first segmentation to identify the anterior chamber region, and is configured to identify the anterior chamber region from the Scheimpflug image acquired by the image acquisition unit 1010.

When performing the first segmentation using machine learning, the first segmentation unit 1031 is configured to perform the first segmentation using a previously constructed inference model (first inference model). The first inference model is trained to include at least an eye image (e.g., a Scheimpflug image of the eye, an eye image acquired by another modality, or a Scheimpflug image of the eye and an eye image acquired by another modality). It includes a neural network (first neural network) constructed by machine learning using data.

The data input to the first neural network is the Scheimpflug image, and the data output from the first neural network is the anterior chamber region. That is, the first segmentation unit 1031 selects the Scheimpflug images acquired by the image collecting unit 1010 (for example, one or more Scheimpflug images, one or more processed image data, or one or more Scheimpflug images and one Scheimpflug image). This Scheimpflug image is input to the first neural network of the first inference model, and the output data from the first neural network (anterior chamber region in the input Scheimpflug image) is obtained. is configured to do so.

The device for constructing the first inference model (inference model construction device) may be provided in the ophthalmological device 1500, in a peripheral device (such as a computer) of the ophthalmological device 1500, or in another device. It may also be a computer. The inference model construction device includes a learning processing section and a neural network.

The neural network provided in the inference model construction device typically includes a convolutional neural network (CNN). This convolutional neural network has a known structure and may include an input layer, a convolution layer, a pooling layer, a fully connected layer, an output layer, etc.

An image is input to the input layer. After the input layer, multiple pairs of convolution layers and pooling layers are arranged. The number of pairs of convolutional layers and pooling layers may be arbitrary.

The convolution layer performs convolution operations to understand features (such as contours) from the image. The convolution operation is a product-sum operation of a filter function (weighting coefficient, filter kernel) of the same dimension as the input image. The convolution layer applies convolution operations to multiple parts of the input image. More specifically, the convolution layer calculates the product by multiplying the value of each pixel of the partial image to which the filter function has been applied by the value (weight) of the filter function corresponding to that pixel, and calculates the product of this partial image. Find the sum of products over multiple pixels. The product-sum value thus obtained is substituted into the corresponding pixel in the output image. By performing a product-sum operation while moving the location (partial image) to which the filter function is applied, a convolution result for the entire input image can be obtained. According to such a convolution operation, a large number of images in which various features are extracted using a large number of weighting coefficients can be obtained. In other words, a large number of filtered images such as smoothed images and edge images are obtained. The multiple images generated by the convolutional layers are called feature maps.

In the pooling layer, the feature map generated by the immediately preceding convolutional layer is compressed (data thinned out, etc.). More specifically, the pooling layer calculates statistical values at predetermined neighboring pixels of the pixel of interest in the feature map at predetermined pixel intervals, and outputs an image smaller in size than the input feature map. Note that the statistical value applied to the pooling calculation is, for example, a maximum value (max pooling) or an average value (average pooling). Further, the pixel interval applied to the pooling operation is called a stride.

A convolutional neural network can extract many features from an input image by performing processing using multiple pairs of convolutional layers and pooling layers.

After the last pair of convolution and pooling layers is a fully connected layer. The number of fully connected layers may be arbitrary. The fully connected layer performs processes such as image classification, image segmentation, and regression using features compressed by a combination of convolution and pooling. After the last fully connected layer there is an output layer that provides the output results.

In some example aspects, a convolutional neural network may not include fully connected layers (e.g., a full-layer convolutional network (FCN)), a support vector machine, a recurrent neural network (RNN), etc. It's okay to stay. Furthermore, the machine learning that the inference model construction device applies to the neural network may include transfer learning. That is, this neural network may include a neural network in which learning using other training data (training images) has already been performed and parameters have been adjusted. Further, the inference model construction device may be configured to be able to apply fine tuning to a trained neural network. Further, the neural network to which machine learning is applied by the inference model construction device may be constructed using a known open source neural network architecture.

The inference model construction device applies machine learning using training data to a neural network. When the neural network includes a convolutional neural network, the parameters of the neural network adjusted by the inference model construction device include, for example, filter coefficients of the convolutional layer and connection weights and offsets of the fully connected layer.

As mentioned above, the training data may include one or more Scheimpflug images acquired for one or more eyes. Since the Scheimpflug image of the eye is the same type of image as the image input to the first neural network, the first neural network It is possible to improve the quality (accuracy, precision, etc.) of the network output.

The types of images included in the training data are not limited to Scheimpflug images; for example, the training data may include images acquired by other ophthalmological modalities (fundus camera, OCT device, SLO, surgical microscope, etc.) or arbitrary images. Generated by processing images acquired by the medical department's image diagnostic modalities (ultrasound diagnostic equipment, X-ray diagnostic equipment, X-ray CT equipment, magnetic resonance imaging (MRI) equipment, etc.) or actual eye images. It may also include images (processed image data), pseudo images, and the like. Furthermore, the number of images, etc. included in the training data may be increased using techniques such as data expansion and data augmentation.

The training method (machine learning method) for constructing the first neural network may be arbitrary, for example, any one of supervised learning, unsupervised learning, and reinforcement learning, or a combination of two or more of them. It may be.

In some example aspects, supervised learning is performed using training data generated by annotations that label input images. In this annotation, for example, for each image included in the training data, the anterior chamber region in the image is identified and labeled. Identification of the anterior chamber region is performed, for example, by at least one of a physician, a computer, and other inference models. The inference model construction device can construct the first neural network by applying supervised learning using such training data to the neural network.

The first inference model including the first neural network constructed in this manner takes as input a Scheimpflug image (for example, a Scheimpflug image acquired by the image collection unit 1010 and its processed image data), and This is a trained model whose output is the anterior chamber region (for example, information indicating the range or position of the anterior chamber region) in the Scheimpflug image.

In order to avoid concentrating processing on specific units of the first neural network, the inference model construction device randomly selects and disables some units of the neural network and performs learning using the remaining units. Good (dropout).

The method used to construct an inference model is not limited to the example shown here. For example, any method such as support vector machine, Bayesian classifier, boosting, k-means, kernel density estimation, principal component analysis, independent component analysis, self-organizing mapping, random forest, generative adversarial network (GAN), etc. It can be used to build inference models.

The first segmentation unit 1031 of this example executes a process of identifying the anterior chamber region from the Scheimpflug image of the eye to be examined by using such a first inference model (first neural network).

The second segmentation unit 1032 includes a processor that performs second segmentation to identify a cell region, and is configured to identify a cell region from the anterior chamber region identified by the first segmentation unit 1031.

When performing the second segmentation of this example using machine learning, the second segmentation unit 1032 is configured to perform the second segmentation using a pre-built inference model (second inference model). . The second inference model is trained to include at least an eye image (e.g., a Scheimpflug image of the eye, an eye image acquired by another modality, or a Scheimpflug image of the eye and an eye image acquired by another modality). It includes a neural network (second neural network) constructed by machine learning using data.

The eye images included in the training data of this example include an image corresponding to at least a portion of the anterior chamber of the eye (referred to as an anterior chamber image). The eye images included in the training data of this example may include the results of manual segmentation or automatic segmentation on anterior segment images (e.g., Scheimpflug images, images acquired with other modalities), such as The information may be information indicating the range or position of the anterior chamber image extracted from the anterior segment image, or the anterior chamber image in the anterior segment image.

The data input to the second neural network is the anterior chamber region identified from the Scheimpflug image by the first segmentation unit 1031 (or the identified and extracted anterior chamber region (the same applies hereinafter)), and the data is input to the second neural network. The data output from is the cell area. That is, the second segmentation unit 1032 receives the anterior chamber region specified by the first segmentation unit 1031, inputs this anterior chamber region to the second neural network of the second inference model, and outputs the output data from the second neural network. (cell area in the input anterior chamber area).

Construction of the second inference model (second neural network) may be performed in the same manner as construction of the first inference model (first neural network). For example, construction of the second inference model (second neural network) is executed by the above-mentioned inference model construction device. Unless otherwise mentioned, the inference model construction device (learning processing unit and neural network) of this example may be the same as the inference model construction device in constructing the first inference model (first neural network).

The training data used to construct the second neural network includes one or more Scheimpflug images acquired for one or more eyes (e.g., an anterior segment image in which an anterior chamber region is identified, an anterior chamber image). good. The types of images included in the training data are not limited to Scheimpflug images; for example, the training data may include images acquired by other ophthalmology modalities, images acquired by an image diagnostic modality in any clinical department, or images acquired in actual practice. The image may include an image generated by processing an eye image, a pseudo image, or the like.

The training method (machine learning method) for constructing the second neural network may be arbitrary, for example, any one of supervised learning, unsupervised learning, and reinforcement learning, or a combination of two or more of them. It may be.

In some example aspects, supervised learning is performed using training data generated by annotations that label input images. In this annotation, for example, for each image included in the training data, a cell region in the image is identified and labeled. Identification of the cellular region is performed, for example, by at least one of a physician, a computer, and other inference models. The inference model construction device can construct the second neural network by applying supervised learning using such training data to the neural network.

The second inference model including the second neural network constructed in this way takes as input the anterior chamber region specified by the first segmentation unit 1031, and uses the cell region in the input anterior chamber region (for example, This is a trained model whose output is information indicating the range or position of the cell area.

The second segmentation unit 1032 of this example uses such a second inference model (second neural network) to execute a process of identifying a cell region from the anterior chamber region in the Scheimpflug image of the eye to be examined.

Cell evaluation information generation processing section 1033 includes a processor that executes cell evaluation information generation processing for generating cell evaluation information, and generates cell evaluation information from the cell region specified by second segmentation section 1032.

When executing the cell evaluation information generation process of this example using machine learning, the cell evaluation information generation processing unit 1033 executes the cell evaluation information generation process using a previously constructed inference model (third inference model). It is configured as follows. The third inference model is trained to include at least an eye image (e.g., a Scheimpflug image of the eye, an eye image acquired by another modality, or a Scheimpflug image of the eye and an eye image acquired by another modality). It includes a neural network (third neural network) constructed by machine learning using data.

The eye images included in the training data of this example include at least an anterior chamber image in which an image of an inflammatory cell is depicted, and may further include an anterior chamber image in which an image of an inflammatory cell is not depicted. The eye images included in the training data of this example may include the results of manual segmentation or automatic segmentation on anterior segment images (e.g., Scheimpflug images, images acquired with other modalities), such as The information may be a cell image extracted from an anterior chamber image in a partial image, or information indicating the range or position of a cell region in an anterior chamber image.

The data input to the third neural network is the output from the second segmentation unit 1032 or data generated based thereon (for example, data indicating the range, position, distribution, etc. of the cell region, or data indicating the identification result of the cell region). The data output from the third neural network is cell evaluation information. That is, the cell evaluation information generation processing unit 1033 receives the cell region identification results from the second segmentation unit 1032 or the data generated based thereon, and applies the cell region identification results or the data generated based thereon to the third inference model. It is configured to input into the third neural network and obtain output data (cell evaluation information) from the third neural network. Note that, as described above, the cell evaluation information is evaluation information regarding predetermined parameters regarding inflammatory cells (for example, the density, number, position, distribution, etc. of inflammatory cells).

Construction of the third inference model (third neural network) may be performed in the same manner as construction of the first inference model (first neural network). For example, construction of the third inference model (third neural network) is executed by the above-mentioned inference model construction device. Unless otherwise mentioned, the inference model construction device (learning processing unit and neural network) of this example may be the same as the inference model construction device in constructing the first inference model (first neural network).

The training data used to construct the third neural network includes one or more Scheimpflug images acquired for one or more eyes (e.g., an anterior segment image including an anterior chamber region in which a cell region has been identified, an anterior segment image including an anterior chamber region in which a cell region has been identified, (anterior chamber image). The types of images included in the training data are not limited to Scheimpflug images; for example, the training data may include images acquired by other ophthalmology modalities, images acquired by an image diagnostic modality in any clinical department, or images acquired in actual practice. The image may include an image generated by processing an eye image, a pseudo image, or the like.

The training method (machine learning method) for constructing the third neural network may be arbitrary, for example, any one of supervised learning, unsupervised learning, and reinforcement learning, or a combination of two or more of them. It may be.

In some example aspects, supervised learning is performed using training data generated by annotations that label input images. In this annotation, for example, cell evaluation information generated from the image is attached as a label to each image (in which a cell region is specified) included in the training data. Generation of cell evaluation information from images is performed, for example, by at least one of a physician, a computer, and other inference models. The inference model construction device can construct the third neural network by applying supervised learning using such training data to the neural network.

The third inference model including the third neural network constructed in this way takes as input the cell region identification result by the second segmentation unit 1032 or the data generated based thereon, and specifies the input cell region. This is a trained model that outputs cell evaluation information based on the results or data generated based on the results.

The cell evaluation information generation processing unit 1033 of this example generates cell evaluation information from the cell region in the anterior chamber region in the Scheimpflug image of the subject's eye by using such a third inference model (third neural network). Execute the processing to be performed.

FIG. 9 shows an example of the configuration of the evaluation processing section 1030. The evaluation processing section 1030B of this example includes a first segmentation section 1041, a conversion processing section 1042, a second segmentation section 1043, and a cell evaluation information generation processing section 1044.

The first segmentation unit 1041 has the same configuration and function as the first segmentation unit 1031 of the evaluation processing unit 1030A, and performs first segmentation for identifying the anterior chamber region from the Scheimpflug image acquired by the image acquisition unit 1010. is configured to run.

The conversion processing unit 1042 converts the anterior chamber region specified by the first segmentation unit 1041 into data having a structure according to the second segmentation performed by the second segmentation unit 1043. The second segmentation unit 1043 of this example is configured to perform the second segmentation using a neural network (second neural network) constructed by machine learning, like the second segmentation unit 1032 of the evaluation processing unit 1030A. has been done. The conversion processing unit 1042 performs conversion for converting the anterior chamber region specified from the Scheimpflug image by the first segmentation unit 1041 into image data having a structure corresponding to the input layer of the second neural network of the second segmentation unit 1043. configured to perform processing.

For example, the input layer of the second neural network (convolutional neural network) of the second segmentation unit 1043 may be configured to accept data with a predetermined structure (form, format). This predetermined data structure may be, for example, a predetermined image size (e.g., number of vertical pixels and number of horizontal pixels), a predetermined image shape (e.g., square or rectangle), and the like. On the other hand, the image size and image shape of the anterior chamber region specified by the first segmentation unit 1041 vary depending on the specifications of the ophthalmological device, the conditions and settings at the time of imaging, and individual differences in the size and shape of the eye to be examined. . The conversion processing unit 1042 allows the input layer of the second neural network of the second segmentation unit 1043 to accept the structure (for example, image size and/or image shape) of the anterior chamber region specified by the first segmentation unit 1041. Convert to structure.

Image size conversion may be performed using any known image size conversion technique, for example, converting the anterior chamber region identified by the first segmentation unit 1041 into a plurality of partial images with image sizes according to the input layer. It may include a process of dividing or a process of resizing the anterior chamber region specified by the first segmentation unit 1041 into a single image with an image size according to the input layer. Image shape transformation may be performed using any known image transformation technique. The same may be applied to conversion processing of other data structures.

Note that in this disclosure, several examples are described in detail regarding the case where a conversion process corresponding to the structure of a neural network is applied to the anterior chamber region of a Scheimpflug image, but the aspects of the conversion process and the configuration thereof are explained in detail. Not limited to those.

For example, if the image input to the neural network is a Scheimpflug image, it is possible to adopt a configuration in which similar conversion processing is applied to the input Scheimpflug image. Furthermore, if the image input to the neural network is arbitrary processed image data of Scheimpflug images, it is possible to adopt a configuration to apply similar conversion processing to the input processed image data. .

Furthermore, the arrangement of elements that execute the conversion process (an element that includes a processor that executes the conversion process, and is referred to as a conversion processing unit) may be arbitrary. For example, the conversion processing unit may be used at a stage before the target neural network (for example, at a stage before the inference model including this neural network) in a series of processes executed based on the obtained Scheimpflug image. , or inside this inference model and at a stage before this neural network), or it may be placed inside the target neural network. When placed inside the target neural network, the conversion processing unit is placed at a stage before the input layer that receives input that directly corresponds to the output of this neural network.

The second segmentation unit 1043 has the same configuration and function as the second segmentation unit 1032 of the evaluation processing unit 1030A, and is used to specify a cell region from the anterior chamber region whose data structure has been converted by the conversion processing unit 1042. 2 is configured to perform segmentation. The second neural network of the second segmentation unit 1043 is configured to receive input of the image data (anterior chamber area whose data structure has been converted) generated by the conversion processing unit 1042 and output a cell area. The machine learning for constructing the second neural network of this example may be performed in the same manner as the machine learning for constructing the second neural network of the evaluation processing unit 1030A.

The cell evaluation information generation processing section 1044 has the same configuration and function as the cell evaluation information generation processing section 1033 of the evaluation processing section 1030A, and is used to generate cell evaluation information from the cell region specified by the second segmentation section 1043. The cell evaluation information generating process is configured to execute the cell evaluation information generation process.

In this way, the evaluation processing section 1030B of this example may have a configuration in which the conversion processing section 1042 is arranged between the first segmentation section 1031 and the second segmentation section 1032 of the evaluation processing section 1030A. However, the configuration of the evaluation processing unit 1030B is not limited to this.

FIG. 10 shows an example of the configuration of the evaluation processing section 1030. The evaluation processing section 1030C of this example includes a second segmentation section 1051 and a cell evaluation information generation processing section 1052.

The second segmentation unit 1051 includes a processor that performs second segmentation to identify a cell area, and is configured to identify the cell area from the Scheimpflug image acquired by the image acquisition unit 1010.

When performing the second segmentation of this example using machine learning, the second segmentation unit 1051 is configured to perform the second segmentation using a previously constructed inference model (fourth inference model). . The fourth inference model is trained to include at least an eye image (e.g., a Scheimpflug image of the eye, an eye image acquired by another modality, or a Scheimpflug image of the eye and an eye image acquired by another modality). It includes a neural network (fourth neural network) constructed by machine learning using data.

In some exemplary aspects, the fourth neural network may include at least a portion of the first neural network and at least a portion of the second neural network in the evaluation processing unit 1030A. For example, the fourth neural network may be a neural network in which a first neural network and a second neural network are arranged in series. The fourth neural network having such a configuration has a function of specifying the anterior chamber region from the Scheimpflug image and a function of specifying the cell region from the anterior chamber region.

In some exemplary embodiments, the fourth neural network may be subjected to machine learning so as to directly identify the cell area from the Scheimpflug image without identifying the anterior chamber area. The aspect of the fourth neural network is not limited to these, and may include a neural network to which any machine learning for identifying a cell region from a Scheimpflug image is applied.

The data input to the fourth neural network is a Scheimpflug image, and the data output from the fourth neural network is a cell region. That is, the second segmentation unit 1051 receives the Scheimpflug image, inputs the Scheimpflug image into the fourth neural network of the fourth inference model, and outputs the output data from the fourth neural network (cells in the input Scheimpflug image). area).

The construction of the fourth inference model (fourth neural network) may be performed in the same manner as the construction of the first inference model (first neural network) of the evaluation processing unit 1030A. For example, construction of the fourth inference model (fourth neural network) is executed by the above-mentioned inference model construction device. Unless otherwise mentioned, the inference model construction device (learning processing unit and neural network) of this example may be the same as the inference model construction device in constructing the first inference model (first neural network) of the evaluation processing unit 1030A. .

The training data used to construct the fourth neural network may include one or more Scheimpflug images acquired for one or more eyes. The types of images included in the training data are not limited to Scheimpflug images; for example, the training data may include images acquired by other ophthalmology modalities, images acquired by an image diagnostic modality in any clinical department, or images acquired in actual practice. The image may include an image generated by processing an eye image, a pseudo image, or the like.

Any of the images included in the training data may have information attached thereto to assist the processing performed by the fourth neural network. For example, the anterior chamber region in the image may be labeled by prior annotation.

The training method (machine learning method) for constructing the fourth neural network may be arbitrary, for example, any one of supervised learning, unsupervised learning, and reinforcement learning, or a combination of two or more of them. It may be.

In some example aspects, supervised learning is performed using training data generated by annotations that label input images. In this annotation, for example, for each image included in the training data, a cell region in the image is identified and labeled. Identification of the cellular region is performed, for example, by at least one of a physician, a computer, and other inference models. The inference model construction device can construct the fourth neural network by applying supervised learning using such training data to the neural network.

The fourth inference model including the fourth neural network constructed in this way takes as input the Scheimpflug image (or its processed image data, etc.) acquired by the image collection unit 1010, and also uses the input Scheimpflug image as input. This is a trained model whose output is a cell region in an image (for example, information indicating the range or position of a cell region).

The second segmentation unit 1051 of this example executes a process of identifying a cell region from the Scheimpflug image of the eye to be examined by using such a fourth inference model (fourth neural network).

The cell evaluation information generation processing unit 1052 includes a processor that executes cell evaluation information generation processing for generating cell evaluation information, and generates cell evaluation information from the cell region specified by the second segmentation unit 1051.

When performing the cell evaluation information generation process of this example using machine learning, the cell evaluation information generation processing unit 1052 executes the cell evaluation information generation process using a previously constructed inference model (fifth inference model). It is configured as follows. The fifth inference model is trained to include at least an eye image (e.g., a Scheimpflug image of the eye, an eye image acquired by another modality, or a Scheimpflug image of the eye and an eye image acquired by another modality). It includes a neural network (fifth neural network) constructed by machine learning using data.

The data input to the fifth neural network is the output from the second segmentation unit 1051 or data generated based thereon (for example, data indicating the range, position, distribution, etc. of the cell region, or data containing the identification results of the cell region). The data output from the fifth neural network is cell evaluation information. That is, the cell evaluation information generation processing unit 1052 receives the cell region identification result from the second segmentation unit 1051 or the data generated based on it, and applies the cell region identification result or the data generated based thereon to the fifth inference model. It is configured to input into the fifth neural network and obtain output data (cell evaluation information) from the fifth neural network.

The machine learning method for constructing the fifth inference model (fifth neural network) is the same as the machine learning method for constructing the third neural network of the cell evaluation information generation processing section 1033 of the evaluation processing section 1030A. It's good. Further, the training data used for machine learning to construct the fifth inference model (fifth neural network) is similar to the training data used for machine learning to construct the third neural network of the evaluation processing unit 1030A. It's good.

The fifth inference model including the fifth neural network receives as input the result of identifying the cell region by the second segmentation unit 1051 or the data generated based thereon, and the result of identifying the cell region inputted or the data generated based thereon. This is a trained model that outputs cell evaluation information based on data.

The cell evaluation information generation processing unit 1052 of this example executes the process of generating cell evaluation information from the cell region in the Scheimpflug image of the eye to be examined by using such a fifth inference model (fifth neural network). do.

FIG. 11 shows an example of the configuration of the evaluation processing section 1030. The evaluation processing section 1030D of this example includes a cell evaluation information generation processing section 1061.

The cell evaluation information generation processing unit 1061 includes a processor that executes cell evaluation information generation processing for generating cell evaluation information, and generates cell evaluation information from the Scheimpflug image acquired by the image collection unit 1010.

When executing the cell evaluation information generation process of this example using machine learning, the cell evaluation information generation processing unit 1061 executes the cell evaluation information generation process using a previously constructed inference model (sixth inference model). It is configured as follows. The sixth inference model is trained to include at least an eye image (e.g., a Scheimpflug image of the eye, an eye image acquired by another modality, or a Scheimpflug image of the eye and an eye image acquired by another modality). It includes a neural network (sixth neural network) constructed by machine learning using data.

In some exemplary aspects, the sixth neural network includes at least a portion of the first neural network, at least a portion of the second neural network, and at least a portion of the third neural network in the evaluation processing unit 1030A. You can stay there. For example, the sixth neural network may be a neural network in which a first neural network, a second neural network, and a third neural network are arranged in series. The sixth neural network having such a configuration has a function of specifying an anterior chamber region from a Scheimpflug image, a function of specifying a cell region from the anterior chamber region, and a function of generating cell evaluation information from the cell region.

In some example aspects, the sixth neural network is machine learned to generate cellular assessment information directly from the Scheimpflug image without identifying anterior chamber regions and/or identifying cellular regions. may have been done. The aspect of the sixth neural network is not limited to these, and may include a neural network to which any machine learning for identifying cell evaluation information from a Scheimpflug image is applied.

The data input to the sixth neural network is the output from the image collection unit 1010 or data generated based on it, and the data output from the sixth neural network is cell evaluation information. That is, the cell evaluation information generation processing unit 1061 receives the Scheimpflug image acquired by the image collection unit 1010 (and/or data generated based on this Scheimpflug image), and receives the Scheimpflug image or data generated based on this Scheimpflug image. The data is input to the sixth neural network of the sixth inference model, and output data (cell evaluation information) from the sixth neural network is obtained.

The machine learning method for constructing the sixth inference model (sixth neural network) is the same as the machine learning method for constructing the third neural network of the cell evaluation information generation processing section 1033 of the evaluation processing section 1030A. It's good to be there. Further, the training data used for machine learning to construct the sixth inference model (sixth neural network) is the same as the training data used for machine learning to construct the third neural network of the evaluation processing unit 1030A. It's good.

The sixth inference model including the sixth neural network takes as input the Scheimpflug image acquired by the image collection unit 1010 (and/or data generated based on this Scheimpflug image), and uses the input Scheimpflug image as input. This is a trained model that outputs cell evaluation information based on an image (and/or data generated based on this Scheimpflug image).

The cell evaluation information generation processing unit 1061 of this example uses the sixth inference model (sixth neural network) to generate the Scheimpflug image of the eye to be examined (and/or the Scheimpflug image generated based on this Scheimpflug image). data) to generate cell evaluation information.

FIG. 12 shows an example of the configuration of the evaluation processing section 1030. The evaluation processing section 1030E of this example includes a first segmentation section 1071 and a cell evaluation information generation processing section 1072.

The first segmentation unit 1071 includes a processor that performs first segmentation to identify the anterior chamber region, and is configured to identify the anterior chamber region from the Scheimpflug image acquired by the image acquisition unit 1010.

When performing the first segmentation of this example using machine learning, the first segmentation unit 1071 is configured to perform the first segmentation using a previously constructed inference model (seventh inference model). . The seventh inference model is trained to include at least an eye image (e.g., a Scheimpflug image of the eye, an eye image acquired by another modality, or a Scheimpflug image of the eye and an eye image acquired by another modality). It includes a neural network (seventh neural network) constructed by machine learning using data.

The machine learning method for constructing the seventh inference model (seventh neural network) is the same as the machine learning method for constructing the first neural network of the first segmentation unit 1031 of the evaluation processing unit 1030A. good. Further, the training data used for machine learning to construct the seventh inference model (seventh neural network) is the same as the training data used for machine learning to construct the first neural network of the evaluation processing unit 1030A. It's good to be there. In some example aspects, the seventh neural network may be the same as or similar to the first neural network, and the seventh inference model may be the same as or similar to the first inference model.

The seventh inference model including the seventh neural network takes as input the Scheimpflug image (or its processed image data, etc.) acquired by the image acquisition unit 1010, and also uses the anterior chamber region ( For example, it is a trained model whose output is information indicating the range or position of the anterior chamber region.

The first segmentation unit 1071 of this example executes the process of identifying the anterior chamber region from the Scheimpflug image of the eye to be examined by using such a seventh inference model (seventh neural network).

Cell evaluation information generation processing section 1072 includes a processor that executes cell evaluation information generation processing for generating cell evaluation information, and generates cell evaluation information from the anterior chamber region specified by first segmentation section 1071.

When executing the cell evaluation information generation process of this example using machine learning, the cell evaluation information generation processing unit 1072 executes the cell evaluation information generation process using a previously constructed inference model (eighth inference model). It is configured as follows. The eighth inference model is trained to include at least an eye image (e.g., a Scheimpflug image of the eye, an eye image acquired by another modality, or a Scheimpflug image of the eye and an eye image acquired by another modality). It includes a neural network (eighth neural network) constructed by machine learning using data.

In some exemplary aspects, the eighth neural network may include at least a portion of the second neural network and at least a portion of the third neural network of the evaluation processing unit 1030A. For example, the eighth neural network may be a neural network in which a second neural network and a third neural network are arranged in series. The eighth neural network having such a configuration has a function of specifying a cell region from the anterior chamber region and a function of generating cell evaluation information from the cell region.

In some exemplary embodiments, the eighth neural network may be subjected to machine learning so as to directly generate cell evaluation information from the anterior chamber region without specifying the cell region. The aspect of the eighth neural network is not limited to these, and may include a neural network to which any machine learning for identifying cell evaluation information from the anterior chamber region is applied.

The data input to the eighth neural network is the output from the first segmentation unit 1071 or data generated based on it, and the data output from the eighth neural network is cell evaluation information. That is, the cell evaluation information generation processing unit 1072 receives the anterior chamber region specified from the Scheimpflug image by the first segmentation unit 1071 (and/or data generated based on this anterior chamber region), and Alternatively, data generated based thereon is input to the eighth neural network of the eighth inference model, and output data (cell evaluation information) from the eighth neural network is acquired.

The machine learning method for constructing the eighth inference model (eighth neural network) is similar to the machine learning method for constructing the third neural network of the cell evaluation information generation processing section 1033 of the evaluation processing section 1030A. It's good to be there. Further, the training data used for machine learning to construct the eighth inference model (eighth neural network) is the same as the training data used for machine learning to construct the third neural network of the evaluation processing unit 1030A. It's good to be there.

The eighth inference model including the eighth neural network takes as input the anterior chamber region specified by the first segmentation unit 1071 (and/or data generated based on this anterior chamber region), and uses the input anterior chamber region as input. This is a trained model that outputs cell evaluation information based on the chamber region (and/or data generated based on the anterior chamber region).

The cell evaluation information generation processing unit 1072 of this example uses such an eighth inference model (eighth neural network) to calculate the anterior chamber region (and/or this anterior chamber region) in the Scheimpflug image of the subject's eye. The process of generating cell evaluation information from the data generated based on the above data is executed.

FIG. 13 shows an example of the configuration of the evaluation processing section 1030. The evaluation processing section 1030F of this example includes a first segmentation section 1081, a conversion processing section 1082, and a cell evaluation information generation processing section 1083.

The first segmentation unit 1081 has the same configuration and function as the first segmentation unit 1031 of the evaluation processing unit 1030A, and performs first segmentation for identifying the anterior chamber region from the Scheimpflug image acquired by the image acquisition unit 1010. is configured to run.

The conversion processing unit 1082 converts the anterior chamber region specified by the first segmentation unit 1081 into data having a structure according to the cell evaluation information generation process executed by the cell evaluation information generation processing unit 1083. The cell evaluation information generation processing unit 1083 of this example, like the cell evaluation information generation processing unit 1072 of the evaluation processing unit 1030E, performs cell evaluation information generation processing using a neural network (eighth neural network) constructed by machine learning. is configured to run. The conversion processing unit 1082 converts the anterior chamber region specified from the Scheimpflug image by the first segmentation unit 1081 into image data having a structure corresponding to the input layer of the eighth neural network of the cell evaluation information generation processing unit 1083. is configured to perform the conversion process.

For example, the input layer of the eighth neural network (convolutional neural network) of the cell evaluation information generation processing unit 1083 may be configured to accept data with a predetermined structure (form, format). This predetermined data structure may be, for example, a predetermined image size (e.g., number of vertical pixels and number of horizontal pixels), a predetermined image shape (e.g., square or rectangle), and the like. On the other hand, the image size and image shape of the anterior chamber region specified by the first segmentation unit 1081 vary depending on the specifications of the ophthalmological equipment, the conditions and settings at the time of imaging, and individual differences in the size and shape of the eye to be examined. . The conversion processing unit 1082 causes the input layer of the eighth neural network of the cell evaluation information generation processing unit 1083 to receive the structure (for example, image size and/or image shape) of the anterior chamber region specified by the first segmentation unit 1081. Convert it to a possible structure.

Image size conversion may be performed using any known image size conversion technique, for example, converting the anterior chamber region identified by the first segmentation unit 1081 into a plurality of partial images with image sizes according to the input layer. It may include a process of dividing or a process of resizing the anterior chamber region specified by the first segmentation unit 1081 into a single image with an image size according to the input layer. Image shape transformation may be performed using any known image transformation technique. The same may be applied to conversion processing of other data structures.

The cell evaluation information generation processing section 1083 has the same configuration and functions as the cell evaluation information generation processing section 1072 of the evaluation processing section 1030E, and processes the anterior chamber region specified by the first segmentation section 1081 using the conversion processing section 1082. The cell evaluation information generating process is configured to execute cell evaluation information generation processing for generating cell evaluation information from data obtained by performing the following steps.

In this way, the evaluation processing section 1030F of this example may have a configuration in which the conversion processing section 1082 is arranged between the first segmentation section 1071 and the cell evaluation information generation processing section 1072 of the evaluation processing section 1030E. However, the configuration of the evaluation processing unit 1030F is not limited to this.

Up to this point, several examples of the evaluation processing unit 1030 including an inference model (neural network) constructed using machine learning have been mainly described. However, the evaluation processing unit 1030 is not limited to such a machine learning-based configuration. The evaluation processing unit 1030 according to the present disclosure may be implemented using only a machine learning-based configuration, a combination of a machine learning-based configuration and a non-machine learning-based configuration, or a non-machine learning It may also be implemented by only the base configuration.

Hereinafter, some examples of the evaluation processing unit 1030 having only a non-machine learning-based configuration will be described. Those skilled in the art will be able to understand aspects of the evaluation processing unit 1030 based on a combination of a machine learning-based configuration and a non-machine learning-based configuration, as well as the various examples of the machine learning-based configuration described above and the non-machine learning-based configuration described below. The evaluation processing unit 1030 can be understood based on some examples of the evaluation processing unit 1030 having only the configuration shown in FIG.

FIG. 14 shows an example of the configuration of the evaluation processing section 1030. The evaluation processing section 1030G in this example includes a first analysis processing section 1091, a second analysis processing section 1092, and a third analysis processing section 1093.

The first analysis processing unit 1091 includes a processor that performs first segmentation to identify the anterior chamber region, and applies a predetermined function to the Scheimpflug image (and/or its processed image data) acquired by the image acquisition unit 1010. It is configured to apply an analysis process (referred to as a first analysis process) to identify the anterior chamber region in this Scheimpflug image.

The first analysis process may include any known segmentation for identifying the anterior chamber region in the Scheimpflug image. For example, segmentation for identifying the anterior chamber region is different from segmentation for identifying the image region corresponding to the cornea (especially the posterior corneal surface), and segmentation for identifying the image region corresponding to the lens (especially the anterior surface of the lens). It includes segmentation. The image area corresponding to the cornea is called the corneal area, the image area corresponding to the posterior surface of the cornea is called the posterior corneal area, the image area corresponding to the crystalline lens is called the crystalline lens area, and the image area corresponding to the anterior surface of the crystalline lens is called the anterior lens area. call.

Segmentation of the posterior corneal region may include any known segmentation. Segmentation of the posterior corneal region poses problems such as artifacts in Scheimpflug images and saturation of pixel values. In order to solve these problems, for example, the configuration shown in FIG. 4C can be adopted. That is, by combining the photographing method using the first photographing system 1014 and the second photographing system 1015 and the Scheimpflug image selection process using the image selection unit 1110, a Scheimpflug image free of artifacts and saturation is selected. In addition, it becomes possible to specify the posterior corneal area from the selected Scheimpflug image.

Segmentation of the anterior lens region may include any known segmentation. In segmentation of the anterior lens region, the expression state of the Scheimpflug image (how the Scheimpflug image looks) changes depending on the state of the pupil of the eye being examined (e.g., mydriatic state, non-mydriatic state, small pupil state, etc.). becomes a problem. For example, when the eye to be examined has a non-mydriatic state or a small pupil, the range of the crystalline lens to be imaged becomes smaller than when the eye to be examined has a mydriatic state. In order to solve this problem, for example, the position and shape of the part of the front surface of the lens that is not imaged (the part covered by the pupil) can be estimated based on the front surface area of the crystalline lens depicted in the Scheimpflug image. It is possible to apply processing to equalize the expression state of the Scheimpflug image, such as processing to uniformize the expression state of the Scheimpflug image. The processing for uniformizing the representation state of the Scheimpflug image may be performed on a machine learning basis or on a non-machine learning basis. Further, the process of estimating the position and shape of the front surface of the crystalline lens may include, for example, any known extrapolation process.

When the image collection unit 1010 collects a series of Scheimpflug images by slit scanning, there may be a mixture of images with problems and images without problems, or there may be variations in the degree of problems among the images. For example, images with and without artifacts or saturation may coexist, or several images may contain artifacts in various different states (position, size, shape, etc.). These phenomena may adversely affect the quality (eg, stability, robustness, reproducibility, accuracy, precision, etc.) of the processing executed by the evaluation processing unit 1030G. In some exemplary embodiments, steps can be taken to prevent these phenomena from occurring or to reduce the negative effects caused by these phenomena. As an example of the former measure, a photographing method using the first photographing system 1014 and the second photographing system 1015 and Scheimpflug image selection processing using the image selection unit 1110 can be combined. Examples of the latter measures include image correction, noise removal, noise reduction, and adjustment of image parameters.

The second analysis processing unit 1092 includes a processor that executes second segmentation to identify cell regions, and applies the second analysis processing to the anterior chamber region identified from the Scheimpflug image by the first analysis processing unit 1091. The system is configured to identify cell regions using

In some exemplary aspects, the second analysis processing unit 1092 performs analysis based on the value of each pixel in the anterior chamber region (e.g., at least one of a luminance value, an R value, a G value, and a B value). may be configured to identify a cell region. In some exemplary aspects, the second analysis processor 1092 may be configured to apply segmentation to the anterior chamber region to identify cellular regions. This segmentation is performed, for example, according to a program created based on the standard morphology (eg, size, shape, etc.) of inflammatory cells (cell regions). In some exemplary aspects, the second analysis processing unit 1092 may be configured to identify the cell region by at least a partial combination of these two techniques.

Measures that can be taken when the image collection unit 1010 collects a series of Scheimpflug images by slit scanning may be the same as those taken by the first analysis processing unit 1091. Further, considering that a cell region is generally a minute image region, measures may be taken to distinguish between a cell region and a minute artifact. For example, by performing processing to remove artifacts (ghosts, etc.), it is possible to prevent artifacts from being erroneously detected in cell region detection.

The third analysis processing unit 1093 includes a processor that executes a cell evaluation information generation process for generating cell evaluation information, and the third analysis processing unit 1093 includes a processor that performs cell evaluation information generation processing to generate cell evaluation information, and the third analysis processing unit 1093 applies a 3 analysis processing is applied to generate cell evaluation information.

As described above, the cell evaluation information may be any evaluation information regarding inflammatory cells, and may include, for example, information representing the state of inflammatory cells (for example, arbitrary parameters such as density, number, position, distribution, etc.). Alternatively, the evaluation information may include evaluation information generated based on information on predetermined parameters regarding the state of inflammatory cells.

In some exemplary embodiments, the third analysis processing unit 1093 can determine the density, number, position, distribution, etc. of one or more cell regions specified by the second analysis processing unit 1092.

The process of determining the density of inflammatory cells includes, for example, a process of setting an image area of a predetermined size (for example, an image area of 1 mm square), and a process of determining the cell area detected by the second analysis processing unit 1092 in the set image area. This includes a process of counting the number of items. Here, the dimensions of the image area (the dimensions in real space, such as "1 millimeter") are, for example, the specifications of the optical system of the ophthalmological apparatus 1500 (for example, the design data of the optical system and/or the actual measurement data of the optical system). ), and is typically defined as a correspondence between pixels and dimensions in real space (for example, dot pitch). The cell evaluation information may include information on the density of inflammatory cells determined in this manner, or may include evaluation information obtained from the information on this density. This evaluation information may include, for example, evaluation results using classification criteria for uveitis diseases proposed by the SUN Working Group. This classification standard defines grades according to the number of inflammatory cells (i.e., the density (concentration) of inflammatory cells) present in one visual field (a visual field of 1 mm square size), and grade "0" is less than 1 cell, grade "0.5+" is 1 to 5 cells, grade "1+" is 6 to 15 cells, grade "2+" is 16 to 25 cells, grade "3+" is The number of cells is 26 to 50, and grade "4+" is defined as the number of cells 50 or more. Note that the grade classification of this classification standard may be made finer or coarser. Furthermore, cell evaluation information may be generated based on other classification criteria.

In some exemplary embodiments, the evaluation processing unit 1030G (third analysis processing unit 1093) calculates a partial area (for example, 1 mm square) of the anterior chamber region identified from the Scheimpflug image by the first analysis processing unit 1091. the image region), the process of determining the number of cell regions belonging to this partial region, and the processing of calculating the density of inflammatory cells based on this number and the dimensions of the partial region. It's okay to stay. Here, the evaluation processing section 1030G selects a cell region located within the partial region from among the cell regions detected from the entire anterior chamber region by the second analysis processing section 1092, and based on the selected cell region. The third analysis processing unit 1093 may be configured to calculate the density using the third analysis processing unit 1093. Alternatively, the evaluation processing unit 1030G causes the second analysis processing unit 1092 to analyze the partial area to identify the cell area, and the third analysis processing unit 1093 determines the density based on the cell area specified from the partial area. It may be configured as follows.

The process of determining the number of inflammatory cells includes, for example, a process of counting the number of cell regions detected by the second analysis processing unit 1092. The cell evaluation information may include information on the number of inflammatory cells determined in this manner, or may include evaluation information obtained from information on this number. For example, cell evaluation information can be calculated by calculating the average density of inflammatory cells in the entire anterior chamber region by dividing the number of cell regions detected in the entire anterior chamber region by the dimensions (e.g., area, volume, etc.) of the anterior chamber region. You can ask for it. Further, the cell evaluation information may include an evaluation result (e.g., grade) based on the number of cell areas detected from the entire anterior chamber area, or the number of cell areas and/or the number of cell areas in a partial area of the anterior chamber area. It may also include evaluation results based on it.

The process of determining the position of the inflammatory cells may include, for example, the process of specifying the position of the cell area detected by the second analysis processing unit 1092. The position of the cell region may be expressed, for example, as a coordinate in the defined coordinate system of the Scheimpflug image, or it may be expressed as a relative position (for example, a distance , direction, etc.). This reference area may be, for example, the cornea area, the posterior corneal area, the crystalline lens area, the anterior lens area, or the image area corresponding to the axis of the eye (for example, a straight line connecting the apex position of the cornea and the apex position of the anterior surface of the crystalline lens). It's fine. The cell evaluation information may include information on the position of the inflammatory cells determined in this manner, or may include evaluation information obtained from the information on this position. For example, the cell evaluation information may include information representing the distribution of inflammatory cells (distribution of multiple cell regions), or may include evaluation results (e.g., grade) based on the location of one or more inflammatory cells. It may also include evaluation results (for example, grade) based on the location (distribution) of a plurality of inflammatory cells.

According to this aspect, evaluation information can be generated based on the Scheimpflug image in which blurring of the image of intraocular floaters (inflammatory cells, etc.) is reduced, so in any of the examples described above, It becomes possible to perform evaluations with higher quality. It should be noted that those skilled in the art will understand that even when other evaluation information generation processing is adopted, evaluation can be performed with higher quality than before.

Several examples will be described regarding the operation of the ophthalmological apparatus (1000, 1500) according to the embodiment. Any matter related to this disclosure, any matter related to documents cited in this disclosure, any matter related to a technical field to which an embodiment of this disclosure belongs, or any matter related to a technical field related to an embodiment of this disclosure. Any items etc. can be combined with the following operation examples.

An example of the operation of the ophthalmologic apparatus 1500 is shown in FIG. Note that, as can be seen from the description below, steps S1 and S2 in FIG. 15 can also be executed by the ophthalmologic apparatus 1000.

It is assumed that some operations (preparatory operations) performed by the ophthalmological apparatus 1500 have been completed before photographing (scanning) the eye to be examined. Preparatory operations include adjusting the table on which the ophthalmological device 1500 is installed, adjusting the chair used by the patient, adjusting the face rest (chin rest, forehead rest, etc.) of the ophthalmological device 1500, and performing ophthalmology on the eye to be examined. There is alignment of the device 1500, etc.

In this operation example, first, conditions for scanning the subject's eye (scan conditions) are set (S1). At least a part of the condition setting may be performed manually, or at least a part of the condition setting may be performed automatically. Exemplary methods and operations for setting conditions will be described later.

Examples of scan conditions include conditions related to the illumination system 1011, conditions related to the imaging system 1012, and conditions related to the moving mechanism (or an element having the same function as the moving mechanism: an illumination scanner, a movable illumination mirror, a photographing scanner, a movable photographing mirror, etc.) , the illumination system 1011, the photographing system 1012, and the conditions regarding coordination (synchronization) of any two or three of the moving mechanisms.

Examples of conditions related to the illumination system 1011 include intensity of illumination light (light amount: illumination control pulse height in FIGS. 5 and 6), pulse width of illumination light (projection time: illumination control pulse width in FIGS. 5 and 6), and slit. These include the dimensions of the light and the direction of the slit light.

Examples of conditions related to the imaging system 1012 include the exposure time of the image sensor 1013 and the gain (sensitivity) of the image sensor 1013.

Examples of conditions related to the moving mechanism include the moving distance of the illumination light (scan range in FIGS. 5 and 6), the moving speed of the illumination light (scan speed in FIG. 6), and the like.

Examples of conditions regarding the coordination (synchronization) of any two or three of the illumination system 1011, the photographing system 1012, and the moving mechanism include the coordination (synchronization) of the projection time of the illumination light and the exposure time of the image sensor 1013; There is a linkage (synchronization) between the projection period and the exposure period of the image sensor 1013. In this operation example, the illumination light projection time is set to be shorter than the exposure time of the image sensor 1013, and at least a part of the illumination light projection period and at least a part of the exposure period of the image sensor 1013 overlap. The conditions are set as follows. For example, a part of the exposure period of the image sensor 1013 corresponds to a period of illumination light projection, and the total time of the illumination light projection time and non-projection time and the exposure time and non-exposure time of the image sensor 1013 are Conditions are set so that the total time of As a result, as shown in FIG. 2, a sequence of illumination light projection (multiple projections of illumination light arranged in chronological order) and a sequence of camera exposure (multiple exposures of the image sensor 1013 arranged in chronological order) are synchronized with each other.

Upon receiving the instruction to start imaging, the control unit 1020 of the ophthalmological apparatus 1500 controls the image acquisition unit 1010 to execute a scan of the subject's eye based on the scan conditions set in step S1. Through this scan, a series of Scheimpflug images of the eye to be examined are collected (S2).

Subsequently, the evaluation processing unit 1030 of the ophthalmological apparatus 1500 generates evaluation information for the eye to be examined based on the series of Scheimpflug images acquired in step S2 (S3).

As shown in FIG. 4, when the imaging system 1012 of the ophthalmological apparatus 1500 includes a first imaging system 1014 and a second imaging system 1015, and the ophthalmological apparatus 1500 includes or can use an image selection section 1110, the ophthalmological apparatus 1500 collects the first Scheimpflug image group and the second Scheimpflug image group by the first imaging system 1014 and the second imaging system 1015, respectively, in step S2, and the image selection unit 1110 collects the first Scheimpflug image group and the second Scheimpflug image group. A series of Scheimpflug images may be generated from the Scheimpflug image group, and evaluation information may be generated from the series of Scheimpflug images by the evaluation processing unit 1030 in step S3.

The ophthalmological apparatus 1500 can display the Scheimpflug image acquired in step S2 and/or the evaluation information generated in step S3 on a display device. The display device may be an element of the ophthalmological apparatus 1500 or may be an external device connected to the ophthalmic apparatus 1500. Further, control for displaying information on the display device is executed by a display control processor (not shown). For example, this display control processor may be included in the control unit 1020, may be an element of the ophthalmological apparatus 1500 separate from the control unit 1020, or may be provided in a separate device from the ophthalmological apparatus 1500. You can leave it there.

Below, some examples of information display that can be performed by the ophthalmologic apparatus 1500 will be described. The form of information display is not limited to these examples. It is possible to at least partially combine at least two of these examples.

In the first example of information display, the ophthalmological apparatus 1500 causes the Scheimpflug image acquired in step S2 and/or the evaluation information generated in step S3 to be displayed as is on the display device. The ophthalmological apparatus 1500 may display other information (additional information) along with the Scheimpflug image and/or evaluation information. The additional information may be any type of information that is useful for medical treatment of the eye to be examined when referenced together with the Scheimpflug image and/or evaluation information.

In the second example of information display, the ophthalmological apparatus 1500 generates an image that simulates an eye image (slit lamp image) acquired by a conventional slit lamp microscope from the Scheimpflug image, and displays the generated simulated image. is displayed on the display device. This makes it possible to provide an image that imitates the slit lamp image that has been conventionally used to observe the subject's eye and that many doctors are accustomed to seeing.

The process of generating a simulated image from a slit lamp image is formed by machine learning-based processing and/or non-machine learning-based processing.

Machine learning-based processing is performed using, for example, a neural network constructed by machine learning using training data including multiple pairs of Scheimpflug images and slit lamp images. This neural network includes a convolutional neural network. This convolutional neural network is configured to receive input of a Scheimpflug image and output a simulated image.

The non-machine learning-based processing may include, for example, processing for generating a simulated blur, processing for converting the appearance of an image, such as color conversion and image quality conversion. Exemplary aspects of non-machine learning-based processing include constructing a three-dimensional image (e.g., an 8-bit grayscale volume) from a series of Scheimpflug images collected with a slit scan and determining the maximum value of the pixel value range. (for example, processing to reduce 256 gradations to 10 gradations) and a region of interest (for example, a rectangular parallelepiped region of a predetermined size) of a 3D image whose gradation has been converted. It includes a setting process and a process (for example, maximum intensity projection process (MIP)) of constructing an enface image of the set region of interest.

The simulated image may be an image representing the same region as the Scheimpflug image that is in focus over a wide range, or may be an image representing a partial region of the region represented by the Scheimpflug image. This partial area may be, for example, one visual field, that is, a visual field of 1 mm square size, which is the evaluation range in the classification criteria for uveitis diseases proposed by the SUN Working Group.

The ophthalmologic apparatus 1500 can, for example, highlight a portion of interest in a simulated image. Examples of this portion of interest include an image region corresponding to inflammatory cells, an image region corresponding to anterior chamber flare, and an image region corresponding to opacity of the crystalline lens.

In the third example of information display, the ophthalmologic apparatus 1500 causes the display device to display information that visualizes the evaluation information generated in step S3. This visualization information may be, for example, a map representing the distribution of a predetermined object or a map representing the distribution of values of a predetermined parameter.

An example of such a map is a map representing the inflammatory state of the subject's eye (inflammatory state map). Examples of this inflammatory state map include an inflammatory cell map that shows the location (distribution) of inflammatory cells in the anterior chamber, and an inflammatory cell density map that shows the distribution of the density (or number) of inflammatory cells in the anterior chamber. number map). The process of creating a map regarding these inflammatory cells is, for example, a process of identifying an image area (cell area) corresponding to an inflammatory cell from each of a series of Scheimpflug images obtained using a slit scan (second segmentation). and a process of determining the position of each identified cell region (for example, two-dimensional coordinates in a defined coordinate system of a Scheimpflug image, or three-dimensional coordinates in a defined coordinate system of a series of Scheimpflug-based three-dimensional images), The process includes a process of creating a map based on the determined positions of each cell area.

The ophthalmological apparatus 1500 can display an inflammatory state map along with the Scheimpflug image and/or inflammatory state information. For example, the ophthalmologic apparatus 1500 can display a frontal image based on a series of Scheimpflug images collected by slit scanning and an inflammatory state map generated based on the same series of Scheimpflug images. As a specific example, the inflammatory state map can be displayed superimposed on the frontal image, or the frontal image and the inflammatory state map can be displayed side by side.

The process of generating information to be displayed on the display device is executed by an information generation processor (not shown). For example, this display control processor may be included in the evaluation processing unit 1030, may be an element of the ophthalmological apparatus 1500 separate from the evaluation processing unit 1030, or may be a separate element from the ophthalmological apparatus 1500. may be provided. This concludes the explanation of the operation example in FIG. 15.

An example of the operation of the ophthalmologic apparatus 1500 is shown in FIG. 16. This operation example provides a specific example of step S1 (condition setting) and step S2 (scan) of the operation example of FIG. 15 described above. As in the case of the operation example in FIG. 15, this operation example can also be executed by the ophthalmologic apparatus 1000. Further, as in the case of the operation example in FIG. 15, it is assumed that some preparatory operations have been completed. It is also assumed that the ophthalmological apparatus 1500 can refer to imaging mode information (for example, imaging mode information 1200 in FIG. 5 or imaging mode information 1210 in FIG. 6).

In this example of operation, first, a shooting mode is specified (S11). The shooting mode is designated manually or automatically. Manual specification is performed using a user interface (not shown). Automatic designation is performed based on predetermined reference information such as diagnosis name (disease name), test type, test record, etc., for example. This reference information is obtained, for example, from medical records regarding the subject (eye to be examined).

Next, the control unit 1020 of the ophthalmological apparatus 1500 controls the pulse height of the control pulse signal applied to the pulse-driven illumination light source (illumination control pulse high) and pulse width (illumination control pulse width) (S12).

Furthermore, the control unit 1020 of the ophthalmological apparatus 1500 controls the optical systems (illumination system 1011, imaging system 1012) that are moved by the movement mechanism for scanning based on the imaging mode specified in step S11 and the imaging mode information. The moving distance (scan range) and moving speed (scan speed) are set (S13).

In addition, the control unit 1020 of the ophthalmological apparatus 1500 selects one of the imaging mode specified in step S11, the conditions related to the illumination system 1011 set in step S12, and the conditions related to the movement mechanism set in step S13. The exposure time of the image sensor 1013 is set based on the above information (S14).

The order of steps S12 to S13 is not limited to this example, and the conditions to be set are not limited to this example. For example, conditions regarding coordination (synchronization) of any two or three of the illumination system 1011, the imaging system 1012, and the movement mechanism may be set. Note that the condition setting in this example is executed so as to satisfy the following two conditions: (1) the projection time of the illumination light to the subject's eye is shorter than the exposure time of the image sensor 1013; (2) At least a portion of the period during which the illumination light is projected onto the eye to be examined overlaps with at least a portion of the exposure period of the image sensor. The condition settings in this example may be executed so as to satisfy the following arbitrary conditions: (3) The total time of the illumination light projection time and non-projection time to the subject's eye, and the exposure time and non-projection time of the image sensor 1013. be equal to the total non-exposure time.

After completing the condition setting, alignment of the optical system (illumination system 1011, imaging system 1012) of the ophthalmological apparatus 1500 with respect to the eye to be examined is performed (S15). This alignment may be automatic alignment or manual alignment. For details on alignment, please refer to Patent Document 3 (Japanese Patent Application Laid-Open No. 2019-213733).

Next, the optical system of the ophthalmologic apparatus 1500 is placed at a predetermined scan start position (S16). This process may be performed automatically or manually.

If the alignment in step S15 is performed to guide the optical system to the scan start position, there is no need to perform step S16, but the results of the alignment in step S15 may be checked and adjusted in step S16. . This confirmation and adjustment may be performed automatically or manually. Note that for details of the alignment for guiding the optical system to the scan start position, please refer to Patent Document 3 (Japanese Patent Application Laid-Open No. 2019-213733).

After the optical system is placed at the scan start position, the image acquisition unit 1010 starts scanning the subject's eye (slit scan) (S17). For example, scanning is started in response to a user's instruction operation, or in response to detection that the optical system is placed at a scan start position.

When the scan is started, the control unit 1020 first causes the image sensor 1013 to start exposure based on the scan conditions set in steps S12 to S14 (see FIG. 2) (S18). This step corresponds to the start of one exposure period.

Next, the control unit 1020 causes the illumination light source to emit pulsed illumination light based on the scan conditions set in steps S12 to S14 (see FIG. 2) (S19). This step corresponds to one pulse emission.

Next, the control unit 1020 causes the image sensor 1013 to end exposure based on the scan conditions set in steps S12 to S14 (see FIG. 2) (S20). This step corresponds to the end of one exposure period started in step S18.

In this manner, in this operation example (steps S18 to S20), as shown in FIG. 17A, one entire projection period corresponds to a part of one exposure period (temporally overlapping , parallel in time).

In general, in the scan according to the embodiment, in addition to the projection time of illumination light to the subject's eye being shorter than the exposure time of the image sensor 1013, at least part of the period of illumination light projection to the subject's eye and the exposure period of the image sensor 1013 The condition is that at least a portion of the In the example shown in FIG. 17B, a partial period of one projection time corresponds to a partial period of one exposure period.

Note that it is also necessary to satisfy the condition that the length of one projection period (projection time) is shorter than the length of one exposure period (exposure time), so at least part of the projection period and at least part of the exposure period must be satisfied. Among the conditions that the projection period overlaps with the entire exposure period, cases where part of the projection period overlaps with the entire exposure period, and cases where the entire projection period overlaps with the entire exposure period are logically excluded. It turns out.

Therefore, in the embodiment, "the projection time of the illumination light is shorter than the exposure time of the image sensor, and at least a part of the period of projection of the illumination light to the eye to be examined overlaps with at least a part of the exposure period of the image sensor". The condition is, for example, "the projection time of the illumination light is shorter than the exposure time of the image sensor, and at least a part of the period of projection of the illumination light to the eye to be examined overlaps with a part of the exposure period of the image sensor". , ``The projection time of the illumination light is shorter than the exposure time of the image sensor, and part of the period of illumination light projection to the eye to be examined overlaps with part of the exposure period of the image sensor, or the illumination light to the eye to be examined is shorter than the exposure time of the image sensor. "The entire projection period and a part of the exposure period of the image sensor overlap", "At least a part of the projection period of the illumination light to the eye to be examined and a part of the exposure period of the image sensor overlap", "The whole projection period and a part of the exposure period of the image sensor overlap" A part of the illumination light projection period for the eye to be examined overlaps with a part of the exposure period of the image sensor, or a part of the illumination light projection period (total) for the eye to be examined overlaps with a part of the image sensor exposure period. ”, etc.

Steps S18 to S20 are repeated until a predetermined condition for ending the scan is satisfied (S21: No). The scan termination conditions include, for example, that a predetermined number of Scheimpflug images have been acquired, that the number of repetitions of steps S18 to S20 has reached a predetermined number, that the execution time of steps S18 to S20 has reached a predetermined time, It may be that the user has performed a predetermined instruction operation (scan end operation), or the like.

When the scan end condition is satisfied (S21: Yes), the scan for the subject's eye started in step S17 ends (S22). Step S22 corresponds to the end of step S2 in FIG.

After step S22, the ophthalmologic apparatus 1500 may be configured to perform arbitrary processing such as the evaluation information generation process, information display process, information storage process, and information analysis process in step S3 of FIG. This concludes the explanation of the operation example in FIG. 16.

FIG. 18 shows an example of a specific configuration of an ophthalmologic apparatus that can function as the ophthalmologic apparatus 1000 or 1500 described above. FIG. 18 is a top view, in which the direction along the axis of the eye E to be examined is defined as the Z direction, and among the directions orthogonal to this, the left and right directions for the subject are defined as the X direction, which is orthogonal to both the X direction and the Z direction. The direction (vertical direction, body axis direction) is defined as the Y direction.

The ophthalmological apparatus of this example is a slit lamp microscope system 1 having a configuration similar to that disclosed in Patent Document 3 (Japanese Patent Application Laid-open No. 2019-213733), and includes an illumination system 2, an imaging system 3, and a video camera system. It includes an imaging system 4, an optical path coupling element 5, a moving mechanism 6, a control section 7, a data processing section 8, a communication section 9, and a user interface 10. The cornea of the eye E to be examined is indicated by the symbol C, and the crystalline lens is indicated by the symbol CL. The anterior chamber corresponds to the area between the cornea C and the crystalline lens CL (the area between the cornea C and the iris). For details of each element of the slit lamp microscope system 1, please refer to Patent Document 3 (Japanese Patent Application Laid-Open No. 2019-213733).

The combination of the illumination system 2, the imaging system 3, and the moving mechanism 6 is an example of the image collection unit 1010 of the ophthalmological apparatus 1000 (1500). Illumination system 2 is an example of illumination system 1011, and imaging system 3 is an example of imaging system 1012.

The illumination system 2 projects slit light onto the anterior segment of the eye E to be examined. Reference numeral 2a indicates the optical axis (illumination optical axis) of the illumination system 2. The photographing system 3 photographs the anterior segment of the eye onto which the slit light from the illumination system 2 is projected. Reference numeral 3a indicates the optical axis (photographing optical axis) of the photographing system 3. The optical system 3A guides light from the anterior segment of the subject's eye E onto which the slit light is projected to the image sensor 3B. The image sensor 3B receives the light guided by the optical system 3A on its imaging surface. The image sensor 3B includes an area sensor (CCD area sensor, CMOS area sensor, etc.) having a two-dimensional imaging area. The image sensor 3B is an example of the image sensor 1013 of the ophthalmological apparatus 1000 (1500).

The illumination system 2 and the photographing system 3 function as a Scheimpflug camera, and are arranged so that the object plane along the illumination optical axis 2a, the optical system 3A, and the imaging surface of the image sensor 3B satisfy Scheimpflug conditions. That is, it is configured such that the YZ plane (including the object plane) passing through the illumination optical axis 2a, the main surface of the optical system 3A, and the imaging surface of the image sensor 3B intersect on the same straight line. Thereby, imaging can be performed in a state where at least the range from the back surface of the cornea C to the front surface of the crystalline lens CL (anterior chamber) is in focus. Furthermore, the illumination system 2 and the imaging system 3 can perform imaging in a state in which at least the range from the anterior vertex of the cornea C (Z=Z1) to the posterior vertex of the crystalline lens CL (Z=Z2) is in focus. . Note that the coordinate Z=Z0 indicates the intersection of the illumination optical axis 2a and the photographing optical axis 3a.

The video shooting system 4 is a video camera, and takes a video of the anterior segment of the eye E to be examined in parallel with the imaging of the eye by the illumination system 2 and the imaging system 3. The optical path coupling element 5 couples the optical path of the illumination system 2 (illumination optical path) and the optical path of the video shooting system 4 (video shooting optical path).

A specific example of an optical system including an illumination system 2, a photographing system 3, a moving image photographing system 4, and an optical path coupling element 5 is shown in FIG. The optical system shown in FIG. 19 includes an illumination system 20 which is an example of the illumination system 2, and a left photographing system 30L and a right photographing system 30R which are examples of the photographing system 3 (first photographing system 1014 and second photographing system 1015). , a moving image shooting system 40 which is an example of the moving image shooting system 4, and a beam splitter 47 which is an example of the optical path coupling element 5.

Reference numeral 20a indicates the optical axis (illumination optical axis) of the illumination system 20, reference numeral 30La indicates the optical axis (left imaging optical axis) of the left imaging system 30L, and reference numeral 30Ra indicates the optical axis (right imaging optical axis) of the right imaging system 30R. axis). The angle θL indicates the angle between the illumination optical axis 20a and the left photographing optical axis 30La, and the angle θR indicates the angle between the illumination optical axis 20a and the right photographing optical axis 30Ra. Coordinate Z=Z0 indicates the intersection of the illumination optical axis 20a, the left photographing optical axis 30La, and the right photographing optical axis 30Ra.

The moving mechanism 6 moves the illumination system 20, the left imaging system 30L, and the right imaging system 30R in the direction shown by an arrow 49 (X direction).

The illumination light source 21 of the illumination system 20 outputs illumination light (for example, visible light), and the positive lens 22 refracts the illumination light. The slit forming section 23 forms a slit to allow part of the illumination light to pass through. The generated slit light is refracted by the objective lens groups 24 and 25, reflected by the beam splitter 47, and projected onto the anterior segment of the eye E to be examined.

The reflector 31L and the imaging lens 32L of the left imaging system 30L guide light from the anterior segment of the eye onto which the slit light is projected by the illumination system 20 (light traveling in the direction of the left imaging system 30L) to the imaging device 33L. . The image sensor 33L receives the guided light at an image pickup surface 34L. The left photographing system 30L repeatedly performs photographing in parallel with the movement of the illumination system 20, the left photographing system 30L, and the right photographing system 30R by the moving mechanism 6. As a result, a plurality of anterior segment images (a series of Scheimpflug images) are obtained. The object surface along the illumination optical axis 20a, the optical system including the reflector 31L and the imaging lens 32L, and the imaging surface 34L satisfy the Scheimpflug condition. The right photographing system 30R also has a similar configuration and function.

Scheimpflug image collection by the left imaging system 30L and Scheimpflug image collection by the right imaging system 30R are performed in parallel with each other. The combination of the series of Scheimpflug images collected by the left imaging system 30L and the series of Scheimpflug images collected by the right imaging system 30R corresponds to the combination of the first Scheimpflug image group and the second Scheimpflug image group. do.

The control unit 7 can synchronize repeated imaging by the left imaging system 30L and repeated imaging by the right imaging system 30R. Thereby, a correspondence relationship between a series of Scheimpflug images obtained by the left imaging system 30L and a series of Scheimpflug images obtained by the right imaging system 30R is obtained. Note that the process of determining the correspondence between the plurality of anterior eye segment images obtained by the left imaging system 30L and the plurality of anterior eye segment images obtained by the right imaging system 30R is performed by the control unit 7 or the data processing unit. 8 may be executed.

The video imaging system 40 takes video of the anterior segment of the eye E from a fixed position in parallel with the imaging by the left imaging system 30L and the imaging by the right imaging system 30R. The light transmitted through the beam splitter 47 is reflected by a reflector 48 and enters the moving image shooting system 40. The light incident on the moving image photographing system 40 is refracted by an objective lens 41 and then imaged by an imaging lens 42 on an imaging surface of an image sensor 43 (area sensor). The video shooting system 40 is used for monitoring the movement of the eye E to be examined, alignment, tracking, processing of collected Scheimpflug images, and the like.

Referring back to FIG. The moving mechanism 6 moves the illumination system 2 and the photographing system 3 integrally in the X direction. The control unit 7 controls each part of the slit lamp microscope system 1. The control unit 7 controls the illumination system 2, the imaging system 3, and the moving mechanism 6, and controls the video imaging system 4 in parallel, so that the image acquisition unit 1010 of the ophthalmological apparatus 1000 (1500) Slit scanning (collecting a series of Scheimpflug images) and video shooting (collecting a series of time-series images) can be performed in parallel. Further, the control unit 7 synchronizes the slit scan and video shooting by controlling the illumination system 2, the imaging system 3, and the moving mechanism 6, and controlling the video imaging system 4 in synchronization with each other. be able to.

When the imaging system 3 includes a left imaging system 30L and a right imaging system 30R, the control unit 7 controls repeated imaging by the left imaging system 30L (collection of the first Scheimpflug image group) and repeated imaging by the right imaging system 30R (collection of the first Scheimpflug image group). (collection of two Scheimpflug image groups) can be synchronized with each other.

The control unit 7 includes a processor, a storage device, and the like. The storage device stores computer programs such as various control programs. The functions of the control unit 7 are realized by cooperation between software such as a control program and hardware such as a processor. The control unit 7 controls the illumination system 2, the imaging system 3, and the movement mechanism 6 in order to scan a three-dimensional region of the eye E to be examined with slit light. For details of this control, please refer to Patent Document 3 (Japanese Unexamined Patent Publication No. 2019-213733).

The data processing unit 8 performs various data processing. The data processing unit 8 includes a processor, a storage device, and the like. The storage device stores computer programs such as various data processing programs. The functions of the data processing section 8 are realized by cooperation between software such as a data processing program and hardware such as a processor. The data processing section 8 may have the functions of the image selection section 1110 and/or the evaluation processing section 1030 of the ophthalmological apparatus 1000 (1500).

The communication unit 9 performs data communication between the slit lamp microscope system 1 and other devices. The user interface 10 includes any user interface devices such as a display device and an operation device.

The slit lamp microscope system 1 shown in FIGS. 18 and 19 is merely an example, and the configuration for implementing the ophthalmologic apparatus 1000 or 1500 is not limited to the slit lamp microscope system 1.

Some functions and effects of the ophthalmological device according to the embodiment will be described.

An ophthalmological apparatus (1000, 1500) according to one embodiment includes an image acquisition section (1010) and a control section (1020). The image collecting unit includes an optical system (illumination system 1011, photographing system 1012) that satisfies Scheimpflug conditions and projects illumination light onto the subject's eye to take an image with an image sensor (1013). The image collection unit collects a series of images (a series of Scheimpflug images) while changing the illumination light projection position and photographing position under the control of the control unit. In other words, the image collecting section collects a series of images by scanning the subject's eye while maintaining the state in which the optical system satisfies Scheimpflug conditions under the control of the control section. The control unit is configured such that the projection time of the illumination light to the eye to be examined is shorter than the exposure time of the image sensor, and at least part of the period of time the illumination light is projected to the eye to be examined and at least part of the exposure period of the image sensor. The image collection unit (optical system) is controlled so that the images overlap.

According to an embodiment with such a configuration, since exposure is substantially performed only during a projection period that is shorter than the exposure period, it is possible to reduce image blurring caused by movement of the scan position or eye movement during exposure. This makes it possible to provide high-quality images. In this way, the ophthalmologic apparatus according to the embodiment contributes to improving the image quality of an ophthalmologic apparatus that uses an imaging method using optical scanning. Thereby, the ophthalmological apparatus according to the embodiment can perform image analysis such as corneal detection and cell detection with high quality. Furthermore, according to the ophthalmological apparatus according to the embodiment, the time during which the illumination light is projected onto the eye to be examined can be shortened, so there is an advantage that it does not impose a large burden on the examinee. In addition, according to the ophthalmological apparatus according to the embodiment, the optical system does not repeatedly start and stop suddenly for scanning, so vibrations caused by this do not occur during scanning, which adversely affects imaging quality. It also has the advantage of not giving any

An ophthalmologic apparatus (1) according to one embodiment includes an illumination system (2), an imaging system (3), a movement mechanism (6), and a control section (7). The illumination system is configured to project illumination light onto the eye to be examined. The photographing system is configured to photograph the eye to be examined. The photographing system includes an image sensor (3B). The illumination system and photographing system are configured to satisfy Scheimpflug conditions. The moving mechanism is configured to move the illumination system and the imaging system. The control unit is configured to control the illumination system, the imaging system, and the movement mechanism to cause the imaging system to collect a series of images. In controlling the imaging system to collect a series of images of the eye to be examined, the control unit controls the projection time of the illumination light to the eye to be examined to be shorter than the exposure time of the image sensor, and At least one of the illumination system and the photographing system is controlled so that at least part of the projection period and at least part of the exposure period of the image sensor overlap.

The effects of the embodiment with such a configuration include improving the image quality of an ophthalmological device that uses an imaging method using optical scanning, performing image analysis with high quality, and not placing a large burden on the patient. , vibrations that adversely affect shooting quality do not occur.

Some optional aspects of embodiments are described below. Note that matters related to arbitrary aspects are not limited to the matters described below, and may be matters included in the present disclosure, matters derived from the matters included in the present disclosure, and the like. Also, two or more optional aspects can be combined.

In the embodiment, the control unit controls the imaging system (image collection unit) to repeat exposure of the image sensor in controlling the imaging system (image collection unit) to collect a series of images of the eye to be examined. It may be configured to do so. The control unit may further be configured to control the illumination system to modulate the intensity of the illumination light in parallel with the control of the imaging system to repeat the exposure of the image sensor. The control unit may further be configured to control the illumination system so that the illumination light is intermittently projected onto the eye to be examined.

In the embodiment, the control unit may be configured to perform synchronous control between the illumination system and the imaging system in controlling the imaging system (image collection unit) to collect a series of images of the eye to be examined. . The control unit further performs synchronization control between the illumination system and the imaging system so that the total time of the illumination light projection time and non-projection time is equal to the total time of the exposure time and non-exposure time of the image sensor. It may be configured to do so.

In the embodiment, the control unit may be configured to control the illumination system (image collection unit) to change the projection time of the illumination light. The control unit may further be configured to control the illumination system (image collection unit) and change the projection time of the illumination light based on the speed of the illumination system and imaging system moved by the moving mechanism. The control unit may further be configured to determine the speed of the illumination system and the imaging system to be moved by the moving mechanism based on the imaging mode selected from two or more preset imaging modes. good.

In the embodiment, the control unit may be configured to control the illumination system (image acquisition unit) to change the projection time of the illumination light, and further control two or more preset shooting modes. The illumination system (image collection unit) may be controlled based on the photographing mode selected from the above, and the projection time of the illumination light may be changed.

In the embodiment, the control unit may be configured to control the imaging system (image acquisition unit) to change the exposure time of the image sensor. The control unit may further be configured to control the imaging system (image acquisition unit) based on the speed of the illumination system and imaging system moved by the moving mechanism, and change the exposure time of the image sensor. The control unit may further be configured to determine the speed of the illumination system and the imaging system to be moved by the moving mechanism based on the imaging mode selected from two or more preset imaging modes. good.

In the embodiment, the control unit may be configured to control the imaging system (image acquisition unit) to change the exposure time of the image sensor, and further control two or more preset imaging modes. The camera may be configured to control the imaging system (image acquisition unit) based on the imaging mode selected from the imaging mode to change the exposure time of the image sensor.

In embodiments, the ophthalmological apparatus (1500) may further include an evaluation processing section (1030). The evaluation processing unit is configured to generate evaluation information of the eye to be examined based on a series of images of the eye to be examined collected by the imaging system (image collection unit). The evaluation processing unit may further be configured to generate evaluation information of floating objects present in the eye to be examined as evaluation information for the eye to be examined.

In the embodiment, the illumination light projected onto the subject's eye by the illumination system (image collection unit) may be slit light.

These optional aspects make it possible to further improve the image quality of an ophthalmological device that uses optical scanning to capture images, and to provide applications for ophthalmological devices that use optical scanning to capture images. , will be understood by those skilled in the art.

<Other embodiments>
Although embodiments of ophthalmological devices have been described so far, embodiments according to the present disclosure are not limited to ophthalmological devices. Embodiments other than ophthalmological apparatuses include methods for controlling ophthalmological apparatuses, programs, recording media, and the like. Similar to the embodiments of the ophthalmological device, these embodiments also aim to improve the image quality of an ophthalmological device that uses an imaging method using optical scanning, perform image analysis with high quality, and avoid placing a large burden on the patient. Effects such as the absence of vibrations that adversely affect photographic quality are achieved.

A method for controlling an ophthalmologic apparatus according to one embodiment is a method for controlling an ophthalmologic apparatus including an illumination system, an imaging system, a movement mechanism, and a processor. The illumination system projects illumination light onto the eye to be examined, and the imaging system photographs the eye to be examined using an image sensor. The illumination system and photographing system are configured to satisfy Scheimpflug conditions. The moving mechanism moves the illumination system and the imaging system. The method according to the present embodiment causes the processor to execute the following three controls in parallel: (1) Illumination system and imaging to make the illumination light projection time to the subject's eye shorter than the exposure time of the image sensor; Control of at least one of the systems; (2) Control of at least one of the illumination system and the imaging system so that at least part of the period of illumination light projected onto the eye to be examined overlaps with at least part of the exposure period of the image sensor; (3) Control of the illumination system, imaging system, and movement mechanism in order to cause the imaging system to collect a series of images of the eye to be examined.

A method for controlling an ophthalmologic apparatus according to one embodiment is a method for controlling an ophthalmologic apparatus including an image acquisition unit and a processor. The image collection unit includes an optical system that projects illumination light onto the eye to be examined and photographs the eye with an image sensor, which satisfies Scheimpflug conditions. The image collection unit further collects a series of images while changing the illumination light projection position and photographing position. In other words, the image collecting unit collects a series of images by scanning the subject's eye while maintaining the state in which the optical system satisfies Scheimpflug conditions. The program according to the present embodiment causes the processor to execute the following two controls in parallel in order to cause the image collection unit to collect a series of images of the eye to be examined: (1) the time period for which illumination light is projected onto the eye to be examined; Control of the image collection unit (optical system) to make the exposure time shorter than the exposure time of the image sensor; (2) Overlap at least part of the period of illumination light projection onto the eye to be examined and at least part of the exposure period of the image sensor; Control of the image acquisition unit (optical system) for

A program according to one embodiment is a program that causes a computer to control an ophthalmologic apparatus. The ophthalmologic apparatus includes an illumination system, an imaging system, and a movement mechanism. The illumination system projects illumination light onto the eye to be examined, and the imaging system photographs the eye to be examined using an image sensor. The illumination system and photographing system are configured to satisfy Scheimpflug conditions. The moving mechanism moves the illumination system and the imaging system. The program according to this embodiment causes the computer to execute the following three controls in parallel: (1) Illumination system and imaging to make the projection time of illumination light to the subject's eye shorter than the exposure time of the image sensor; Control of at least one of the systems; (2) Control of at least one of the illumination system and the imaging system so that at least part of the period of illumination light projected onto the eye to be examined overlaps with at least part of the exposure period of the image sensor; (3) Control of the illumination system, imaging system, and movement mechanism in order to cause the imaging system to collect a series of images of the eye to be examined.

A program according to one embodiment is a program that causes a computer to control an ophthalmologic apparatus. The ophthalmological device includes an image acquisition section. The image collection unit includes an optical system that projects illumination light onto the eye to be examined and photographs the eye with an image sensor, which satisfies Scheimpflug conditions. The image collection unit further collects a series of images while changing the illumination light projection position and photographing position. In other words, the image collecting unit collects a series of images by scanning the subject's eye while maintaining the state in which the optical system satisfies Scheimpflug conditions. The program according to the present embodiment causes the computer to execute the following two controls in parallel in order to cause the image collection unit to collect a series of images of the eye to be examined: (1) the time of illumination light projection to the eye to be examined; Control of the image collection unit (optical system) to make the exposure time shorter than the exposure time of the image sensor; (2) Overlap at least part of the period of illumination light projection onto the eye to be examined and at least part of the exposure period of the image sensor; Control of the image acquisition unit (optical system) for

The recording medium according to one embodiment is a computer-readable non-transitory recording medium in which a program for causing a computer to control an ophthalmologic apparatus is recorded. The ophthalmologic apparatus includes an illumination system, an imaging system, and a movement mechanism. The illumination system projects illumination light onto the eye to be examined, and the imaging system photographs the eye to be examined using an image sensor. The illumination system and photographing system are configured to satisfy Scheimpflug conditions. The moving mechanism moves the illumination system and the imaging system. The program recorded on the recording medium according to this embodiment causes the computer to execute the following three controls in parallel: (1) make the projection time of illumination light on the subject's eye shorter than the exposure time of the image sensor; (2) Control of at least one of the illumination system and the imaging system for the purpose of overlapping at least a part of the illumination light projection period to the eye to be examined and at least a part of the exposure period of the image sensor; Control of at least one of the systems; (3) Control of the illumination system, the imaging system, and the movement mechanism in order to cause the imaging system to collect a series of images of the eye to be examined.

The recording medium according to one embodiment is a computer-readable non-transitory recording medium in which a program for causing a computer to control an ophthalmologic apparatus is recorded. The ophthalmological device includes an image acquisition section. The image collection unit includes an optical system that satisfies Scheimpflug conditions for projecting illumination light onto the eye to be examined and photographing the eye using an image sensor. The image collection unit further collects a series of images while changing the illumination light projection position and photographing position. In other words, the image collecting unit collects a series of images by scanning the subject's eye while maintaining the state in which the optical system satisfies Scheimpflug conditions. The program recorded on the recording medium according to this embodiment causes the computer to execute the following two controls in parallel in order to cause the image collection unit to collect a series of images of the eye to be examined: (1) the eye to be examined; (2) Control of the image collection unit (optical system) to make the projection time of the illumination light shorter than the exposure time of the image sensor; Control of the image acquisition unit (optical system) to overlap at least a portion of the image.

Note that the computer-readable non-temporary recording medium that can be used as the recording medium according to the embodiments may be any type of recording medium, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory. It may be

Any of the items described in the ophthalmological device embodiments can be combined in non-ophthalmological device embodiments. For example, any of the items described above as optional aspects of the ophthalmologic apparatus according to the embodiments can be combined with the embodiment of the ophthalmologic apparatus control method, the program embodiment, the recording medium embodiment, and the like. Further, any of the matters described in the present disclosure can be combined with an embodiment of a method for controlling an ophthalmologic apparatus, an embodiment of a program, an embodiment of a recording medium, and the like.

This disclosure presents several embodiments and several exemplary aspects thereof. These embodiments and aspects are merely illustrative of the invention. Therefore, any modification (omission, substitution, addition, etc.) within the scope of the gist of the present invention can be applied to the embodiments and aspects presented in this disclosure.

1 Slit lamp microscope system (ophthalmological equipment)
2 Illumination system 3 Imaging system 3B Imaging device 6 Movement mechanism 7 Control section 1000, 1500 Ophthalmological apparatus 1010 Image collection section 1011 Illumination system 1012 Imaging system 1013 Imaging device 1020 Control section 1030 Evaluation processing section

Claims (20)

an illumination system that projects illumination light onto the eye to be examined;
an imaging system that photographs the eye to be examined;
a moving mechanism that moves the illumination system and the imaging system;
a control unit that controls the illumination system, the imaging system, and the movement mechanism to cause the imaging system to collect a series of images;
The imaging system includes an image sensor,
The illumination system and the photographing system are configured to satisfy Scheimpflug conditions,
In the control for causing the photographing system to collect the series of images, the control unit controls the illumination light so that the projection time of the illumination light onto the eye to be examined is shorter than the exposure time of the image sensor; controlling at least one of the illumination system and the photographing system so that at least a part of the period during which the illumination light is projected onto the optometrist overlaps with at least a part of the exposure period of the image sensor;
Ophthalmology equipment. The control unit controls the imaging system to repeat exposure of the image sensor in the control for causing the imaging system to collect the series of images.
The ophthalmological device of claim 1. The control unit executes control of the illumination system for modulating the intensity of the illumination light in parallel with the control of the imaging system for repeating the exposure of the image sensor.
The ophthalmological device according to claim 2. The control unit executes the control of the illumination system so that the illumination light is intermittently projected onto the eye to be examined.
The ophthalmological device according to claim 3. The control unit executes synchronous control of the illumination system and the imaging system in the control for causing the imaging system to collect the series of images.
The ophthalmological device of claim 1. The control unit executes the synchronization control so that the total time of the projection time and non-projection time of the illumination light is equal to the total time of the exposure time and non-exposure time of the image sensor.
The ophthalmological device according to claim 5. The control unit controls the illumination system to change the projection time of the illumination light.
The ophthalmological device of claim 1. The control unit controls the illumination system based on the speed of the illumination system and the imaging system moved by the moving mechanism, and changes the projection time of the illumination light.
The ophthalmological device according to claim 7. The control unit determines the speeds of the illumination system and the imaging system based on a shooting mode selected from two or more preset shooting modes.
The ophthalmological device according to claim 8. The control unit controls the illumination system based on a photographing mode selected from two or more preset photographing modes to change the projection time of the illumination light.
The ophthalmological device according to claim 7. The control unit controls the imaging system to change the exposure time of the image sensor.
The ophthalmological device of claim 1. The control unit controls the imaging system based on the speed of the illumination system and the imaging system moved by the moving mechanism, and changes the exposure time of the image sensor.
The ophthalmological device of claim 11. The control unit determines the speeds of the illumination system and the imaging system based on a shooting mode selected from two or more preset shooting modes.
The ophthalmological device of claim 12. The control unit controls the photographing system based on a photographing mode selected from two or more preset photographing modes to change the exposure time of the image sensor.
The ophthalmological device of claim 11. further comprising an evaluation processing unit that generates evaluation information of the eye to be examined based on the series of images collected by the imaging system;
The ophthalmological device of claim 1. The evaluation processing unit generates evaluation information of floating objects present in the eye to be examined as the evaluation information for the eye to be examined.
The ophthalmological device of claim 15. the illumination light is slit light;
The ophthalmological device of claim 1. The illumination system includes an illumination system that projects illumination light onto the eye to be examined, a photography system that photographs the eye to be examined using an image sensor, a movement mechanism that moves the illumination system and the photography system, and a processor. A method for controlling an ophthalmological apparatus configured such that the system satisfies Scheimpflug conditions, the method comprising:
The processor is configured to control the projection time of the illumination light onto the subject's eye to be shorter than the exposure time of the image sensor, and to control at least a part of the projection period of the illumination light onto the eye to be tested and the exposure of the image sensor. While at least one of the illumination system and the photographing system is controlled so that at least a part of the period overlaps, the illumination system, the photographing system, and the moving mechanism are controlled so that the photographing system produces a series of images. to collect,
Method. The lighting system includes an illumination system that projects illumination light onto the eye to be examined, a photography system that photographs the eye to be examined using an image sensor, and a movement mechanism that moves the illumination system and the photography system, and the illumination system and the photography system are arranged in a shine mode. A program that causes a computer to control an ophthalmological device configured to satisfy proofing conditions,
The computer is configured to control the exposure time of the image sensor so that the projection time of the illumination light to the eye to be examined is shorter than the exposure time of the image sensor, and at least a part of the period of projection of the illumination light to the eye to be examined. While at least one of the illumination system and the photographing system is controlled so that at least a part of the period overlaps, the illumination system, the photographing system, and the moving mechanism are controlled so that the photographing system produces a series of images. to collect,
program. The lighting system includes an illumination system that projects illumination light onto the eye to be examined, a photography system that photographs the eye to be examined using an image sensor, and a movement mechanism that moves the illumination system and the photography system, and the illumination system and the photography system are arranged in a shine mode. A computer-readable non-transitory recording medium on which a program for causing a computer to control an ophthalmological device configured to satisfy proofing conditions is recorded,
The computer is configured to control the exposure time of the image sensor so that the projection time of the illumination light to the eye to be examined is shorter than the exposure time of the image sensor, and at least a part of the period of projection of the illumination light to the eye to be examined. While at least one of the illumination system and the photographing system is controlled so that at least a part of the period overlaps, the illumination system, the photographing system, and the moving mechanism are controlled so that the photographing system produces a series of images. to collect,
recoding media.

Images