CN117177703A - Inspection system, inspection device, inspection method, and inspection program - Google Patents

Abstract

The inspection system (1000) is provided with an imaging device (16), a processor (21) for inspecting the wettability of the eye on the basis of an imaging image obtained by imaging the eye by the imaging device (16) and an estimated model (232) including a neural network (2321), and a display (15) for displaying the inspection result of the processor (21).

Description

Inspection system, inspection device, inspection method, and inspection program

Technical Field

The present disclosure relates to an inspection system, an inspection apparatus, an inspection method, and an inspection program for inspecting wettability of an eye. In addition, the present disclosure relates to an inspection system, an inspection apparatus, an inspection method, and an inspection program for inspecting stability of a tear liquid layer. The present disclosure also relates to an inspection system, an inspection apparatus, an inspection method, and an inspection program for inspecting the presence or absence of dry eye.

Background

In recent years, people who complain of unpleasant feeling of eyes have increased due to the effects of aging, drying of rooms caused by use of air conditioners, use of personal computers, wearing of contact lenses, and the like. Such ophthalmic diseases are, for example, dry eye.

Dry eye is a disease in which tears are unevenly spread over the surface of the eye (e.g., cornea) due to insufficient tear volume or a disruption of tear quality balance. Patients with dry eye sometimes have unpleasant sensations to the eye, abnormal visual function, or injured surfaces of the eye.

Japanese patent application laid-open No. 7-136120 (patent document 1) discloses an ophthalmic device capable of diagnosing dry eye.

Prior art literature

Patent literature

Patent document 1: japanese patent laid-open No. 7-136120

Disclosure of Invention

Problems to be solved by the invention

An image pickup device for an ophthalmic device disclosed in japanese patent application laid-open No. 7-136120 picks up an interference pattern formed by light reflected on the surface of an eye. The user can check whether dry eye exists by observing the interference pattern imaged by the imaging device of the ophthalmic device.

However, the ophthalmic device disclosed in japanese patent application laid-open No. 7-136120 is a user-specific device such as an ophthalmologist, which has knowledge of the extent to which the presence or absence of dry eye can be determined. On the other hand, a general person or patient lacking knowledge about an ophthalmic disease requires a technique capable of more easily self-testing his own eyes.

The present disclosure has been made to solve the above-described problems, and an object thereof is to provide an inspection system, an inspection apparatus, an inspection method, and an inspection program capable of performing self-inspection of eyes.

Means for solving the problems

The present disclosure relates to an inspection system that inspects the wettability of an eye. The inspection system is provided with: an image pickup device; an inspection unit that inspects the wettability of the eye based on a captured image obtained by capturing an image of the eye with the imaging device and an estimated model including a neural network; and a display for displaying the inspection result of the inspection unit.

Preferably, the estimation model performs learning so as to check the wettability of the eye from the captured image based on learning data including the captured image and the result of checking the wettability of the eye.

Preferably, the photographed image includes a first still image immediately after blinking and a second still image when a prescribed time has elapsed after blinking.

Preferably, each of the first still image and the second still image is extracted from a moving image obtained by capturing an eye by the imaging device.

Preferably, the first still image and the second still image are each images including at least a face of an eye. The eye image is extracted from the face image included in each of the first still image and the second still image.

Preferably, the inspection unit inspects the wettability of the eye based on the degree of fluctuation of the eye image extracted from the first still image and the degree of fluctuation of the eye image extracted from the second still image.

Preferably, the display further displays an image of an eye included in the captured image.

Preferably, the display also displays information related to recommended ophthalmic drugs.

Preferably, the display also outputs information related to the ophthalmic hospital.

Preferably, the information related to the ophthalmic hospital includes at least any one of information prompting diagnosis and treatment in the ophthalmic hospital, information related to diagnosis and treatment in the ophthalmic hospital, and information related to a recommended ophthalmic hospital.

Preferably, the display displays an inspection result based on at least one of a subjective symptom associated with the eye and a possibility that the eye has an ophthalmic disease, and an inspection result of the inspection by the inspection section.

Preferably, the inspection system further includes an inspection device and a server device capable of communicating with the inspection device. The inspection device includes an imaging device and a display. The server device includes an inspection unit.

The present disclosure relates to an inspection apparatus that inspects wettability of an eye. The inspection device is provided with: an image pickup device; an inspection unit that inspects the wettability of the eye based on a captured image obtained by capturing an image of the eye with the imaging device and an estimated model including a neural network; and a display for displaying the inspection result of the inspection unit.

The present disclosure relates to an inspection apparatus that inspects wettability of an eye. The inspection device is provided with: an image pickup device; communication means for communicating with a server apparatus provided with an estimation model including a neural network; and a display. The communication device transmits a captured moving image obtained by capturing an image of an eye by the image capturing device to the server device. The communication device receives an inspection result of the wettability of the eye obtained based on the captured image of the eye extracted from the captured moving image by the server device and the estimation model. The display displays the inspection results.

The present disclosure relates to an inspection method for inspecting wettability of an eye using a computer. The inspection method comprises the following steps: a step of inputting a photographed image obtained by photographing an eye by an imaging device; a step of checking the wettability of the eye based on the photographed image and the estimated model including the neural network; and outputting the inspection result of the step of inspecting.

The present disclosure relates to an inspection procedure for inspecting the wettability of an eye. The inspection program causes the computer to execute the steps of: a step of inputting a photographed image obtained by photographing an eye by an imaging device; a step of checking the wettability of the eye based on the photographed image and the estimated model including the neural network; and outputting the inspection result of the step of inspecting.

The present disclosure relates to an inspection system that inspects stability of a tear fluid layer. The inspection system is provided with: an image pickup device; an inspection unit that inspects the stability of the tear liquid layer based on a captured image obtained by capturing an image of an eye with an imaging device and an estimated model including a neural network; and a display for displaying the inspection result of the inspection unit.

The present disclosure relates to an inspection system that inspects the presence or absence of dry eye. The inspection system is provided with: an image pickup device; an inspection unit for inspecting the presence or absence of dry eye based on a captured image obtained by capturing an image of an eye with an imaging device and an estimated model including a neural network; and a display for displaying the inspection result of the inspection unit.

ADVANTAGEOUS EFFECTS OF INVENTION

According to the inspection system, the inspection apparatus, the inspection method, and the inspection program of the present disclosure, a user can perform self-inspection of eyes.

Drawings

Fig. 1 is a diagram for explaining an inspection of wettability of an eye using the inspection apparatus according to the present embodiment.

Fig. 2 is a diagram showing a configuration of the inspection system according to the present embodiment.

Fig. 3 is a diagram for explaining learning of the estimation model in the learning stage.

Fig. 4 is a diagram showing the structure of the inspection system in the operation stage.

Fig. 5 is a diagram showing a user interface screen displayed on a display of the inspection apparatus at the time of self-inspection.

Fig. 6 is a diagram showing a user interface screen displayed on a display of the inspection apparatus at the time of self-inspection.

Fig. 7 is a diagram showing a user interface screen displayed on a display of the inspection apparatus at the time of self-inspection.

Fig. 8 is a diagram showing a user interface screen displayed on a display of the inspection apparatus at the time of self-inspection.

Fig. 9 is a diagram showing a user interface screen displayed on a display of the inspection apparatus at the time of self-inspection.

Fig. 10 is a diagram showing a user interface screen displayed on a display of the inspection apparatus at the time of self-inspection.

Fig. 11 is a diagram showing a user interface screen displayed on a display of the inspection apparatus at the time of self-inspection.

Fig. 12 is a diagram showing a user interface screen displayed on a display of the inspection apparatus at the time of self-inspection.

Fig. 13 is a diagram showing a user interface screen displayed on a display of the inspection apparatus at the time of self-inspection.

Fig. 14 is a diagram showing a user interface screen displayed on a display of the inspection apparatus at the time of self-inspection.

Fig. 15 is a diagram showing a user interface screen displayed on a display of the inspection apparatus at the time of self-inspection.

Fig. 16 is a flowchart relating to the inspection process of the wettability of the eye performed by the server apparatus.

Fig. 17 is a diagram for explaining the examination of the wettability of the eye.

Fig. 18 is a diagram showing a configuration of an inspection system according to another embodiment.

Detailed Description

Embodiments of the present disclosure are described in detail below with reference to the drawings. Note that the same or corresponding portions in the drawings are denoted by the same reference numerals, and description thereof is not repeated. In the present disclosure, the term "eye" is used as a term including the peripheral tissues of the eyeball such as eyelid, inner canthus, and outer canthus, in addition to the eyeball. The term "eye" is mainly used as a term representing the eyeball of a human. The term "pupil" is mainly used as a term indicating the iris of a person. The "wettability of the eye" is not limited to "wettability of the eyeball", but includes "wettability of the pupil" or "wettability of the iris".

[ outline of examination of eye wettability ]

Fig. 1 is a diagram for explaining an inspection of wettability of eyes using the inspection apparatus 1 of the present embodiment. Regarding the eye with dry eye, the secreted amount of the tear is insufficient compared with the normal eye, or the state of the tear such as immediate evaporation of the tear due to the decrease in tear quality is unstable although the secreted amount is sufficient. Therefore, in the case of dry eye, even if the state of the tear immediately after blinking is stable, the state of the tear becomes unstable after a lapse of time without blinking.

If the above-mentioned unpleasant person complaining of the eye can self-test his own eye, this technique can enhance the enthusiasm of the eye's examination, purchase of ophthalmic drugs, etc., and lead to early detection, prevention and treatment of ophthalmic diseases.

Therefore, in the present embodiment, the user 10 can obtain an objective examination result of the wettability of the eye (pupil) by starting an application program for performing self-examination of the eye using the examination apparatus 1 (hereinafter, also referred to as "self-examination application") and operating the examination apparatus 1 according to the self-examination application.

Specifically, as shown in fig. 1, the inspection apparatus 1 includes an imaging device 16. The lens of the imaging device 16 is disposed on the back side (opposite to the side on which the display 15 is disposed) of the inspection device 1. The inspection apparatus 1 captures a face of the user 10 as a moving image by the imaging apparatus 16. The moving image obtained by photographing by the image pickup device 16 (hereinafter, also referred to as "photographed moving image") includes still images of a plurality of faces obtained in time series. The inspection apparatus 1 is communicably connected to a server apparatus 2 described later. The inspection device 1 transmits the data of the captured moving image obtained by the image capturing device 16 to the server device 2. The server device 2 extracts a plurality of still images from the photographed moving image, which is acquired from the inspection device 1. Further, the server apparatus 2 extracts an eye portion from each of the plurality of still images by image recognition based on AI (artificial intelligence: artificial Intelligence), and determines a change in time series of the extracted images of the plurality of eyes, thereby checking the wettability of the eyes of the user 10. The server device 2 transmits the inspection result to the inspection device 1. The inspection device 1 displays the inspection result obtained from the server device 2 on the display 15.

Thus, the user 10 can objectively check the wettability of his/her own eye by checking the state (stability) of his/her own eye tear using the checking device 1. In this way, the user 10 can easily perform self-test of eyes using the test device 1 without depending on the level of knowledge of himself.

The inspection result output by the inspection device 1 is not limited to the determination result of the wettability of the eye, and includes the determination result of the stability of the tear layer (tear layer) covering the surface of the eye. The inspection result output by the inspection device 1 includes the presence or absence of dry eye, that is, the result of determination as to whether dry eye is detected or the result of determination as to whether dry eye is suspected. Here, the wettability of the eye (pupil) refers to the degree to which the surface of the eye is wetted with tears. The lower the wettability, the more unstable the tear state becomes, and the tear layer becomes uneven. On the other hand, the higher the wettability, the more moist the surface of the eye is always wetted by the tear, and the less the tear component is, the more stable the tear state is. In addition, the lens of the imaging device 16 may be disposed on the side of the inspection device 1 on which the display 15 is disposed. In this case, the user 10 may take a moving image of the face with the display 15 side of the inspection apparatus 1 facing the front of the user.

[ Structure of inspection System ]

Fig. 2 is a diagram showing a configuration of an inspection system 1000 according to the present embodiment. As shown in fig. 2, the inspection system 1000 includes a plurality of inspection apparatuses 1 (in the example of fig. 2, the inspection apparatuses 1A, 1B, 1C) and a server apparatus 2 communicably connected to each of the plurality of inspection apparatuses 1.

The inspection apparatus 1 is constructed in accordance with a general-purpose computer architecture. In the present embodiment, a mobile terminal such as a smart phone that can be carried by the user 10 is exemplified as the inspection apparatus 1. The inspection apparatus 1 may be an apparatus other than a smart phone such as a desktop computer, a notebook computer, and a tablet computer.

The inspection apparatus 1 includes a processor 11, a communication device 12, a memory 13, an input interface 14, a display 15, and an imaging device 16.

The processor 11 is an operation body (computer) that executes various processes according to various programs. The processor 11 is constituted by at least any one of CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), GPU (Graphics Processing Unit), and MPU (Multi Processing Unit), for example. The processor 11 may be configured by an arithmetic circuit (Processing Circuitry).

The communication device 12 transmits and receives data (information) to and from the server device 2 through a wired connection or a wireless connection. In the present embodiment, the communication device 12 transmits and receives data (information) to and from the communication device 22 of the server device 2 by wireless communication via the network 5. Specifically, the communication device 12 transmits the captured moving image including the captured image of the eye acquired by the imaging device 16 to the server device 2 via the network 5 at the time of self-test. The communication device 12 receives data including the inspection result of the eye wettability from the server device 2 via the network 5 at the time of self-inspection.

The memory 13 is constituted by a volatile memory such as DRAM (Dynamic Random Access Memory) and SRAM (Static Random Access Memory) or a nonvolatile memory such as ROM (Read Only Memory). The memory 13 stores various programs and data such as a self-test application 131 for performing self-test of eyes. The self-check application 131 is an application program for the user 10 to check the wettability of the eye by himself, and includes a program for photographing the eye with the image pickup device 16.

The input interface 14 is an interface for receiving an input from the user 10, such as a button or a touch panel. The input interface 14 outputs a signal based on the input of the user to the processor 11.

The display 15 is a display device such as a liquid crystal display, a plasma display, or an organic EL (Electro Luminescence) display, and displays a predetermined screen under the control of the processor 11.

The imaging device 16 images an object to be imaged in a moving image manner. The moving image data 236 of the captured moving image obtained by the image capturing apparatus 16 is transmitted to the server apparatus 2 via the network 5.

The server device 2 is configured according to a general computer architecture. In the present embodiment, the server apparatus 2 is a server apparatus owned by a manufacturer that provides the user 10 with a self-check application 131 for checking the wettability of the eye. The server device 2 includes a processor 21, a communication device 22, and a memory 23.

The processor 21 is an example of a "checking unit". The processor 21 is a calculation body that executes various processes (for example, inspection processes described later) according to various programs (for example, inspection program 231 described later). The processor 21 is composed of at least one of CPU, FPGA, GPU and MPU, for example. The processor 21 may be configured by an arithmetic circuit.

The communication device 22 transmits and receives data (information) to and from each of the plurality of inspection devices 1 through a wired connection or a wireless connection. In the present embodiment, the communication device 22 transmits and receives data (information) to and from the communication device 12 of the inspection device 1 by wireless communication via the network 5. Specifically, the communication device 22 receives the captured moving image including the captured image of the eye acquired by the imaging device 16 from the inspection device 1 via the network 5 at the time of self-inspection. The communication device 22 transmits data including the inspection result of the eye wettability to the inspection device 1 via the network 5 at the time of self-inspection.

The memory 23 is constituted by volatile memory such as DRAM and SRAM, or nonvolatile memory such as ROM. The memory 23 stores various programs and data such as an inspection program 231 for inspecting the wettability of the eye, a model 232 for estimating the wettability of the eye, eye medicine information 233 including information related to eye medicine, ophthalmic hospital information 234 including information related to ophthalmic hospital, user information 235 including information related to the user 10 of each of the plurality of inspection apparatuses 1, moving image data 236 of a captured moving image obtained by the imaging apparatus 16 of the inspection apparatus, and a calculation table 237 for calculating a score used in calculating a result of the self-test. Here, the inspection program is a program for inspecting the wettability of the eye by analyzing the photographed moving image of the eye acquired from the inspection apparatus 1 by AI.

[ learning of the estimation model in the learning stage ]

Fig. 3 is a diagram for explaining learning of the estimation model 232 in the learning stage. The learning phase refers to a prior learning phase in which the estimation model 232 is learned before the self-test application 131 is provided to the inspection apparatus 1 of the user 10. As shown in fig. 3, the estimation model 232 is learned by the learning device 31 so as to check the wettability of the eye from the photographed image of the eye.

As a learning algorithm for learning the estimation model 232, a known algorithm such as supervised learning, unsupervised learning, or reinforcement learning can be used. In the present embodiment, the learning device 31 learns the estimation model 232 by supervised learning using the learning data 4.

The learning data 4 is prepared in advance for learning the estimation model 232, and the learning data 4 includes a captured moving image of the eye and a result of checking the wettability of the eye. For example, the designer of the inspection program 231 captures eyes of a plurality of persons having different eye wettabilities in a moving image manner, and correlates the obtained captured image of the eyes with the inspection result (correct data) of the eye wettabilities to obtain learning data 4. A designer prepares a plurality of such learning data 4 in advance.

The estimation model 232 includes a neural network 2321 and parameters 2322 used by the neural network 2321. The neural network 2321 employs a known neural network used in image recognition processing based on deep learning, such as a convolutional neural network (CNN: convolution Neural Network), a cyclic neural network (recurrent neural network) (RNN: recurrent Neural Network), or an LSTM network (Long Short Term Memory Network). The estimation model 232 performs deep learning by using the neural network 2321 described above. The parameter 2322 includes a weighting coefficient or the like used in calculation using the neural network 2321. The estimation model 232 is not limited to the model that is learned by deep learning using a neural network, and may be a model that is learned by other machine learning.

The learning device 31 receives an input of a captured image of an eye in the learning data 4. The learning device 31 performs processing for checking the wettability of the eye represented by the captured image of the eye based on the captured image of the eye input and the estimation model 232 including the neural network 2321.

Specifically, the learning device 31 acquires a still image immediately after blinking and a still image when a predetermined time (for example, 5 seconds) has elapsed after blinking. The two acquired still images are still images including at least an eye image. The learning device 31 inputs the acquired two still images into the estimation model 232. The estimation model 232 determines a change in time series of images of both eyes through image recognition, thereby estimating the wettability of the eyes of the user 10. The learning device 31 obtains the estimation result of the eye wettability obtained by the estimation model 232.

The learning device 31 may estimate the wettability of the eye based on a plurality of still images obtained immediately after blinking until a predetermined time (for example, 5 seconds) elapses. For example, the learning device 31 may acquire all still images obtained for each frame immediately after blinking until a predetermined time (for example, 5 seconds) elapses, and determine a change in the time series of the acquired images of the plurality of eyes, thereby estimating the wettability of the eyes of the user 10.

The learning device 31 learns the estimation model 232 based on the estimation result of the eye wettability and the correct data (the result obtained by predicting the eye wettability expressed by the captured image) included in the learning data 4. Specifically, the learning device 31 learns the estimation model 232 by adjusting the parameter 2322 (for example, a weighting coefficient) so that the estimation result of the wettability of the eye obtained by the estimation model 232 approximates to the accurate data.

[ Structure of inspection device in operation stage ]

Fig. 4 is a diagram showing the structure of the inspection apparatus 1 in the operation stage. The operation stage is a stage of estimating the wettability of the eye using the estimation model 232 after the self-test application 131 is provided to the inspection apparatus 1 of the user 10. As shown in fig. 4, the server device 2 stores the estimation model 232 learned by the learning device 31 shown in fig. 3 in the memory 23. For example, the server device 2 acquires the estimation model 232 from the learning device 31, and stores the acquired estimation model 232 in the memory 23. The learning device 31 may be the server device 2, and the function of the learning device 31 may be the function of the processor 21 of the server device 2.

The processor 21 of the server device 2 includes an input unit 211, a processing unit 212, and an output unit 213.

The input unit 211 receives an input of a shot moving image including a shot image obtained by shooting the eyes of the user 10 by the imaging device 16 of the inspection device 1. The processing unit 212 performs processing for checking the wettability of the eye represented by the captured image, which is input from the input unit 211, based on the captured image extracted from the captured dynamic image and the estimation model 232 including the neural network 2321. As described above, the estimation model 232 is not limited to the model that is learned by deep learning using a neural network, and may be a model that is learned by other machine learning.

Specifically, the processing unit 212 acquires a still image of the eye immediately after blinking and a still image of the eye when a predetermined time (for example, 5 seconds) has elapsed after blinking. The still image of the eye immediately after blinking is an example of the "first still image". The still image of the eye when a predetermined time (for example, 5 seconds) has elapsed after blinking is an example of the "second still image". The two acquired still images are still images including at least an eye image. The processing unit 212 inputs the acquired two still images into the estimation model 232. The estimation model 232 determines a change in time series of images of both eyes through image recognition, thereby estimating the wettability of the eyes of the user 10. The processing unit 212 obtains the estimation result of the eye wettability obtained by the estimation model 232.

The processing unit 212 may estimate the wettability of the eye based on a plurality of still images obtained immediately after blinking until a predetermined time (for example, 5 seconds) elapses. For example, the processing unit 212 may acquire all still images obtained for each frame immediately after blinking until a predetermined time (for example, 5 seconds) elapses, and determine a change in the time series of the acquired images of the plurality of eyes, thereby estimating the wettability of the eyes of the user 10.

The output unit 213 outputs the inspection result obtained by the processing unit 212 to the inspection apparatus 1. The display 15 of the inspection device 1 displays a screen indicating the inspection result of the obtained eye wettability, thereby providing the inspection result of the eye wettability to the user 10.

[ specific example of self-test of eye ]

A specific example of performing the self-test of the eyes by the user 10 using the inspection apparatus 1 will be described with reference to fig. 2 and fig. 5 to 15. Fig. 5 to 15 are views showing user interface screens displayed on the display 15 of the inspection apparatus 1 at the time of self-inspection. The user can perform an operation or the like of an icon displayed on the display 15 by using the input interface 14. The user interface screens shown in fig. 5 to 15 are examples, and can be changed as appropriate by a designer of the self-test application 131 or the like. Hereinafter, the eye examination performed by the user 10 itself using the self-examination application 131 is also referred to as "self-examination".

As shown in fig. 5 (a), when the user 10 starts the self-check application 131 using the check device 1, a login screen is displayed on the display 15.

The login screen includes a field for inputting an ID and a password for identifying the user 10. The user 10 registers information (e.g., name, sex, age, telephone number, mail address, etc.) of the user himself to the server apparatus 2, and thereby obtains an ID and a password from the server apparatus 2. The user 10 may be able to set a desired ID and password. Information registered by the user 10 in the server apparatus 2 is stored in the memory 23 as user information 235.

On the login screen, when the user 10 inputs an ID and a password, the inspection device 1 transmits the ID and the password to the server device 2. When the server apparatus 2 authenticates the user 10 based on the ID and the password, the inspection apparatus 1 displays a menu screen on the display 15 as shown in fig. 5 (B). The menu screen includes an icon 151 for starting a self-test, an icon 152 for viewing the history of the self-test performed in the past, and an icon 153 for performing various settings.

When the user 10 operates the icon 151, as shown in fig. 5 (C) and (D), the inspection apparatus 1 displays a risk check screen for the user 10 to check the risk on the display 15. Risk screening refers to screening for a likelihood that the user 10 has an ophthalmic disease. In the present embodiment, the risk checkup includes a plurality of checkup items including the sex of the user 10, the age group, the time of use of the PC, the smartphone, and the like, the presence or absence of shoulder stiffness, the presence or absence of headache, the presence or absence of contact lens wear, the presence or absence of eye drops, and the presence or absence of diagnosis of past dry eye.

When the user 10 completes the risk hook, as shown in fig. 6 (E), the inspection apparatus 1 displays a selection screen on the display 15. The selection screen includes an icon 148 for viewing a course and an icon 149 for taking a photograph.

When the user 10 operates the icon 148, the examination apparatus 1 displays a course screen on the display 15 as shown in fig. 6 (F1) to 7 (F6). The course screen contains an image for explaining the photographing method of the eye for self-test to the user 10.

As shown in fig. 6 (F1), the inspection apparatus 1 will take the eyes to rest as "please see outside". "a message prompting the user 10 to rest the eyes when capturing a moving image in this way is displayed on the display 15. Also, the inspection apparatus 1 will take a photograph using natural light, for example, "during the daytime". "a message informing the user 10 of the use of natural light in moving image shooting in this way is displayed on the display 15.

As shown in fig. 6 (F2), the inspection apparatus 1 photographs the window in the direction of the window, for example, "please stand on the front surface of the window, approach the distance to one arm. A message "notifying the user 10 of the standing position at the time of moving image shooting and the like in this way is displayed on the display 15.

As shown in fig. 7 (F3), the inspection apparatus 1 will check the shooting distance as "please confirm. The thumb is erected by hand, and the thumb is erected as tightly as possible. "a message notifying the user 10 of the positional relationship between the user 10 and the image pickup device 16 at the time of moving image capturing in this way is displayed on the display 15.

As shown in fig. 7 (F4), the inspection apparatus 1 will be as "please gently stick the thumb to the cheekbones, and put the little finger side on the back of the terminal. "a message notifying the user 10 of the positional relationship between the user 10 and the image pickup device 16 at the time of moving image capturing in this way is displayed on the display 15.

As shown in fig. 7 (F5), the inspection apparatus 1 displays a schematic view of the case where the thumb is gently stuck to the cheekbones correctly on the display 15.

As shown in fig. 7 (F6), the inspection apparatus 1 displays a schematic view on the display 15 in the case where the thumb is erroneously lightly stuck on the cheekbones. Specifically, an example in which the lower eyelid is pulled by the thumb attached to the cheekbone is shown on the display 15.

In the selection screen of fig. 6 (E), when the user 10 operates the icon 149, the inspection apparatus 1 displays a self-inspection screen on the display 15 as shown in fig. 8 (G1) to 9 (G8). The self-check screen includes an image for causing the user 10 to capture a moving image of the eye using the imaging device 16 and measuring the wettability of the eye based on the captured moving image.

As shown in fig. 8 (G1), the inspection apparatus 1 will take a moving image as "please blink slowly" and about 5 seconds thereafter. During shooting, please not blink. "a message prompting the user 10 to shoot a face in a state of gently sticking the thumb to the cheekbones" is displayed on the display 15. Thus, the user 10 views the outside scene with natural light in the daytime, and captures a face including his eyes in a moving image within 5 seconds with the imaging device 16.

As shown in fig. 8 (G2), the inspection apparatus 1 displays an image captured by the imaging apparatus 16 on the display 15. When the photographing by the image pickup device 16 is completed, as shown in fig. 8 (G3), the inspection device 1 displays a message informing the user 10 of the completion of photographing as such on the display 15.

As shown in fig. 8 (G4), the inspection apparatus 1 displays an image after the start of shooting on the display 15 after the completion of shooting. At this time, the inspection apparatus 1 detects the pupil (iris) based on the still image after the start of photographing, and extracts an image of the peripheral portion of the eye (for example, a portion of the eye) including the pupil. The inspection apparatus 1 represents the extracted image portion by a box. The user 10 can adjust by moving or zooming in, out of the frame in such a way that at least the image portion of the eye gets into the frame. The user 10 can cut out the image portion located in the frame by selecting the icon 147 and extract at least the photographed image of the eye. On the other hand, the user 10 can re-photograph by selecting the icon 146.

As shown in fig. 9 (G5), the inspection apparatus 1 confirms whether or not the image is proper for inspection. "such a message is shown on the display 15. As shown in fig. 9 (G6), the inspection apparatus 1 displays a photographed image of the eye at a time after 5 seconds from photographing on the display 15. The inspection device 1 displays on the display 15 an example of an eye image taken when the pupil (iris) is detected and focused appropriately, and an example of an eye image taken when the pupil (iris) is detected and focused inappropriately.

As shown in fig. 9 (G7), the inspection apparatus 1 will "is an image appropriate for inspection? "a message for inquiring whether or not an image appropriate for the examination is acquired, an icon 154 indicating that an image appropriate for the examination is acquired, and an icon 155 indicating that a moving image of the eye is captured again are displayed on the display 15" are inquired of the user 10.

When the user operates the icon 154, measurement of the wettability of the eye by AI is started. As shown in fig. 9 (G8), in the self-test, the inspection apparatus 1 displays a message informing the user 10 that the determination is being made, as in the case of "AI determination", on the display 15. The inspection apparatus 1 displays a message prompting the user 10 to select subjective symptoms related to eyes, such as "request to select subjective symptoms to this side", and an icon 156 for the user 10 to select subjective symptoms on the display 15.

When the user 10 operates the icon 156, as shown in fig. 10 (H1) and (H2), the inspection apparatus 1 displays a plurality of subjective symptom checking screens on the display 15 in stages.

As shown in fig. 10 (H1) and (H2), the inspection apparatus 1 displays a question of 12 items for asking for subjective symptoms of the user 10 related to eyes, such as "eyes are dry" and "eyes are tired easily", as a subjective symptom hook, on the display 15. The user 10 can input the subjective symptom-checkup result by checkup the item having the subjective symptom. As shown in fig. 10 (H1), the user 10 can also use the subjective symptom picking result inputted in the past.

When the user 10 completes the input of the all subjective symptom checkup, as shown in fig. 11 (H3), the inspection apparatus 1 will respond as "thank you answer-! The result of the image-based inspection is obtained. "a message notifying the user 10 of the result of the self-test and an icon 157 for viewing the result of the self-test are displayed on the display 15.

When the user 10 operates the icon 157, as shown in fig. 12 and 13, the inspection device 1 displays a plurality of inspection result screens indicating inspection results of the self-inspection on the display 15 in stages.

Fig. 12 shows the inspection result in the case where the level of the wettability of the eyes of the user 10 is low. As shown in fig. 12 (J1), the examination apparatus 1 displays an image 1511 showing the comprehensive evaluation of the examination result, an image 1521 showing the examination result of tear quality, and an image 1531 showing the enlarged eye portion on the display 15.

Image 1511 contains a message indicating a lower level of wettability of the eyes of user 10, such as "your pupil wettability" low ", as a comprehensive evaluation such as" suggest earlier to ophthalmic subjects. The cause of the "true" that reduces the wettability of the pupil may be known. "thus prompting the user 10 to a message to an ophthalmic hospital.

Image 1521 includes a graph informing user 10 that the tear quality of user 10 is unstable.

The image 1531 contains an enlarged image of the eye of the user 10 used in the self test. The magnified image represents the portion of the eye that is the subject of image recognition in the self-test. Thus, the user 10 can confirm the image of his own eye used in the self-test by using the display 15.

As shown in fig. 12 (J2), the inspection apparatus 1 displays an image 1541 on the display 15, the image 1541 representing the inspection result of the self-test based on the subjective symptom checkup shown in fig. 10.

Image 1541 contains, for example, "you are more than 5 items, and tears may be unstable. "thus informing the user 10 of the result of the subjective symptom hook".

Fig. 13 shows the result of the examination in the case where the level of the wettability of the eyes of the user 10 is medium. As shown in fig. 13 (K1), the examination apparatus 1 displays an image 1512 showing the comprehensive evaluation of the examination result, an image 1522 showing the examination result of tear quality, and an image 1532 showing the enlarged eye portion on the display 15.

The image 1512 contains a message indicating that the level of the wettability of the eyes of the user 10 is medium, such as "in the wettability of your pupil", as a comprehensive evaluation of how the eye medicine in the pharmacy is being cared for. If symptoms are not to be taken care of, please talk in the ophthalmology department. "thus prompting the user 10 to conduct a self-care message containing the use of an ophthalmic drug.

Image 1522 includes a graph informing user 10 that the tear quality of user 10 is substantially normal.

The image 1532 contains an enlarged image of the eye of the user 10 used in the self-test. In addition, the enlarged image shows a portion of the eye that is the object of image recognition in the self-test. Thus, the user 10 can confirm the image of his own eye used in the self-test by using the display 15.

As shown in fig. 13 (K2), the inspection apparatus 1 displays an image 1542 on the display 15, the image 1542 representing the inspection result of the self-test based on the subjective symptom checkup shown in fig. 10.

Image 1542 contains as "you are 2 items, tears are likely to be unstable. "thus informing the user 10 of the result of the subjective symptom hook".

As shown in fig. 14 (L), the inspection device 1 displays an image 1543 on the display 15, the image 1543 representing the inspection result of the self-inspection based on the risk checkup shown in fig. 5 (C) and (D).

The image 1543 includes a message such as "the risk of dry eye is considered high when the following items are met, and therefore, please note that" the user 10 is notified of an item whose risk of dry eye is high due to the coincidence among the items to be the risk-checked object in this way.

As shown in fig. 12 (J1) and 13 (K1), the inspection apparatus 1 displays an icon 158 for viewing a method of handling based on the inspection result on the display 15 on the inspection result screen.

When the user 10 operates the icon 158 on the inspection result screen (inspection result screen in fig. 12 (J1)) in the case where the wettability of the eyes of the user 10 is low, the inspection apparatus 1 displays a correspondence screen indicating a correspondence based on the inspection result on the display 15 as shown in fig. 15 (M1).

The coping picture of the user 10 in the case where the wettability of the eyes is low includes at least one of information prompting diagnosis and treatment by the ophthalmic hospital, information related to diagnosis and treatment in the ophthalmic hospital, and information related to a recommended ophthalmic hospital as information related to the ophthalmic hospital. In the present disclosure, the term "diagnosis and treatment" is used as a term including "diagnosis" and "treatment". That is, the information related to the ophthalmic hospital may include at least any one of information prompting diagnosis (diagnosis or treatment) by the ophthalmic hospital, information related to diagnosis (diagnosis or treatment) in the ophthalmic hospital, and information related to a recommended ophthalmic hospital.

Specifically, as shown in fig. 15 (M1), the coping picture in the case where the wettability of the eyes of the user 10 is low will be as "first recommend ophthalmic examination". "a message prompting the user 10 to diagnose by the ophthalmic hospital, for example," check lacrimal substance while checking symptoms by inquiry ", etc., and diagnose dry eye. A message informing the user 10 of the information related to the diagnosis and treatment in the ophthalmic hospital in this way is displayed on the display 15.

Further, the examination apparatus 1 displays an icon 159 on the display 15, the icon 159 informing the user 10 of the latest ophthalmic hospital of the user 10 as information related to the recommended ophthalmic hospital. When the user 10 operates the icon 159, for example, the inspection apparatus 1 acquires information of the nearest ophthalmic hospital of the user 10 from the server apparatus 2 via the network 5, and displays the acquired information on the display 15. The inspection apparatus 1 may search for the nearest ophthalmic hospital of the user 10 by connection via the internet of the network 5, and display the search result on the display 15. The examination apparatus 1 can search the nearest ophthalmic hospital of the user 10 based on the position information of the examination apparatus 1. The examination apparatus 1 may retrieve the nearest ophthalmic hospital of the user 10 based on the information input by the user 10 (e.g., the residence of the user 10).

In this way, the inspection device 1 provides, as information related to the ophthalmic hospital, information prompting diagnosis and treatment by the ophthalmic hospital, information related to diagnosis and treatment in the ophthalmic hospital, and information related to the recommended ophthalmic hospital to the user 10 based on the inspection result of the self-test. Thus, the user 10 is urged to go to the ophthalmic hospital, and can acquire information about the nearest ophthalmic hospital.

As shown in fig. 15 (M1), the inspection apparatus 1 may display all of the information prompting the diagnosis and treatment by the ophthalmic hospital, the information related to the diagnosis and treatment in the ophthalmic hospital, and the information related to the recommended ophthalmic hospital on the display 15 as the information related to the ophthalmic hospital, or may display at least any one of these information related to the ophthalmic hospital on the display 15.

The inspection device 1 may display at least one of information prompting diagnosis and treatment by the ophthalmologic hospital, information related to diagnosis and treatment in the ophthalmologic hospital, and information related to recommended ophthalmologic hospital on the display 15 as information related to the ophthalmologic hospital on a coping picture when the wettability of eyes of the user 10 is medium or high. In this way, even when the wettability of the eyes of the user 10 is medium or high, the examination apparatus 1 can urge the user 10 to perform medical treatment in the ophthalmic hospital for the sake of care.

In the inspection result screen (the inspection result screen in fig. 13 (K1)) when the wettability of the eyes of the user 10 is intermediate, when the user 10 operates the icon 158, the inspection apparatus 1 displays a coping process screen indicating a coping process based on the inspection result on the display 15 as shown in fig. 15 (M2).

The countermeasure screen in the case where the wettability of the eyes of the user 10 is medium contains information on recommended eye drops.

Specifically, as shown in fig. 15 (M2), the coping picture in the case where the eye wettability of the user 10 is moderate displays a message that introduces recommended eye drops to the user 10, such as "general medicine recommended in self-care", and a message that introduces the recommended eye drop function to the user 10, such as "eye drops with a high water retention function for a person who wants to keep the eye wet".

Further, the inspection device 1 displays an icon 160 for notifying the user 10 of detailed information of recommended ophthalmic drugs on the display 15. When the user 10 operates the icon 160, for example, the inspection apparatus 1 acquires detailed information of recommended eye drops from the server apparatus 2 via the network 5, and displays the acquired information on the display 15. The inspection device 1 may search for detailed information of recommended eye drops through an internet connection via the network 5, and display the search result on the display 15. The eye drops recommended by the inspection device 1 may be eye drops or eye ointments.

In this way, the inspection device 1 provides the user 10 with information on recommended eye drops based on the inspection result of the self-inspection. Thus, the user 10 can acquire information on an ophthalmic drug optimal for self-care of eyes.

In the inspection apparatus 1, information on recommended eye drops may be displayed on the display 15 on a countermeasure screen when the wettability of the eyes of the user 10 is low. In this way, when the wettability of the eyes of the user 10 is low, the user 10 can be prompted to make a diagnosis and treatment by the ophthalmologic hospital, and information on an eye medicine optimal for self-care of the eyes can be provided to the user 10.

[ inspection treatment of eye wettability ]

Fig. 16 is a flowchart relating to the eye wettability inspection process performed by the server apparatus 2. The inspection process shown in fig. 16 is performed by the processor 21 of the server apparatus 2 executing the inspection program 231. The server device 2 executes the inspection process shown in fig. 16 on condition that a predetermined start condition is satisfied. As a start condition, as shown in fig. 5 (B), an icon 151 or the like for starting self-test is enumerated by the user 10. In fig. 16, "S" is used as an abbreviation for "STEP".

As shown in fig. 16, the server device 2 obtains a risk assessment result (S1). Specifically, the server device 2 obtains the risk assessment result of the user 10 input from the inspection device 1 via the network 5. The server device 2 determines whether or not a captured moving image including a captured image of the eyes of the user 10 obtained by the imaging device 16 of the inspection device 1 is input (S2). The photographed moving image of the eye includes a still image of the eye immediately after blinking and a still image of the eye when a prescribed time (for example, 5 seconds) has elapsed after blinking. When the captured moving image is not input (no in S2), the server device 2 repeats the processing of S2.

When the captured moving image is input (yes in S2), the server device 2 extracts a still image of the eye immediately after blinking and a still image of the eye when a predetermined time (for example, 5 seconds) has elapsed after blinking from the captured moving image (S3). The server device 2 extracts the degree of fluctuation of each of the images of the two eyes by image recognition (S4). The server device 2 checks the wettability of the eyes based on the change in the time series of the fluctuation degrees in the images of the two eyes (S5).

Here, a process for checking the wettability of the eye based on the captured moving image will be described with reference to fig. 17. Fig. 17 is a diagram for explaining the examination of the wettability of the eye.

As shown in fig. 17, the inspection apparatus 1 starts shooting at a timing t1 immediately after blinking and then ends shooting at a timing t2 after 5 seconds by using the imaging apparatus 16. The inspection device 1 transmits the data of the photographed moving image to the server device 2.

The server device 2 extracts a still image ("first still image" for example) at a timing t1 immediately after blinking and a still image ("second still image" for example) at a timing t2 after a predetermined time (5 seconds in the example of fig. 17) has elapsed after blinking from among a plurality of still images included in the captured moving image acquired from the inspection device 1 (processing corresponding to S3 of fig. 16). As shown in fig. 8 (G2), these two still images each contain an image of the face of the user 10.

The server apparatus 2 extracts an image 301 of the eye from the still image at timing t1 by image recognition, and extracts an image 302 of the eye from the still image at timing t 2. The server device 2 extracts the extent of fluctuation of the image 301 of the eye at the timing t1 and extracts the extent of fluctuation of the image 302 of the eye at the timing t2 (processing corresponding to S4 in fig. 16).

In the case of dry eye, even if the state of the tear immediately after blinking is stable, the state of the tear becomes unstable after a lapse of time without blinking. Therefore, in the case of dry eye, the extent of fluctuation of an image of the eye after the lapse of time without blinking is greater than that of an image of the eye immediately after blinking. That is, the greater the extent of fluctuation of the eye image, the more unstable the tear state, which suggests that the eye image is low in wettability of the eye and/or is suffering from dry eye or is likely to be suffering from dry eye.

For example, in the image 301 of the eye at the timing t1, a scene (in this example, a background of a street) that is reflected on the surface of the eye (for example, cornea) is approximately clearly represented, and in the image 302 of the eye at the timing t2, a scene (a background of a street) that is reflected on the surface of the eye (cornea) is represented in a permeated form. The "degree of fluctuation of an image of an eye" refers to the degree of blurring, penetration, degree of fluctuation, or the like of an object reflected on the surface of the eye (e.g., cornea) as shown in the image.

The server device 2 compares the image 301 of the eye at the timing t1 and the image 302 of the eye at the timing t2 by using the estimation model 232, and observes the change in the fluctuation degree, thereby checking the wettability of the eye of the user 10 (processing corresponding to S5 in fig. 16).

The timing t2 is not limited to the timing at which 5 seconds elapse immediately after blinking, and may be any timing as long as it is a timing at which the degree of wetting of the eye can be checked based on a change in the degree of fluctuation of the image of the eye.

The eye image 302 to be compared with the eye image 301 immediately after blinking is not limited to one, and may be plural. For example, the server device 2 may check the wettability of the eyes by extracting the eye image 301 at the timing t1 immediately after blinking, then extracting the eye image 302 at the timing t2 after a first predetermined time (for example, 5 seconds) has elapsed, extracting the eye image at the timing t3 after a second predetermined time (for example, 7 seconds) has elapsed, and observing the change in the degree of fluctuation in these multiple eye images. In this way, the server device 2 can check the wettability of the eye based on the images of the plurality of eyes extracted at each of a plurality of timings passing immediately after blinking, so that the wettability of the eye can be checked with higher accuracy.

The inspection apparatus 1 is not limited to the process of extracting images of a plurality of eyes from moving images of faces (eyes). For example, the inspection apparatus 1 can take a first still image (photograph) by taking a face of the user 10 at a timing t1 immediately after blinking, and then take a second still image (photograph) by taking a face of the user 10 at a timing t2 after a predetermined time (for example, 5 seconds) has elapsed. The server device 2 can extract an eye image from each of the plurality of still images thus obtained.

Returning to fig. 16, after the processing of S5, the server device 2 obtains the subjective symptom assessment result (S6). Specifically, the server device 2 obtains the subjective symptom assessment result of the user 10 input from the inspection device 1 via the network 5.

The server device 2 calculates a score used when estimating the final self-test result (S7).

Here, calculation of a score used in estimating a self-test result will be described. In a calculation table (not shown) for calculating the score, a predetermined score is assigned to the result of the examination of the wettability of the eye using the image, a predetermined score is assigned to each item for subjective symptom assessment, and a predetermined score is assigned to each item for risk assessment. The server device 2 refers to the calculation table, and calculates the score in an additive manner based on the inspection result of the wettability of the eye using the image, the subjective symptom assessment result, and the risk assessment result. For example, the server device 2 adds a predetermined score when the subjective symptom checkup user 10 has checked the item "eyes appear dry", and adds a predetermined score when the risk checkup user 10 has checked the item "use of eye medicine" has ". In this way, the server device 2 obtains the score based on the examination result of the wettability of the eye using the image, the subjective symptom assessment result, and the risk assessment result.

The server device 2 extracts the determination message as a result of the self-test based on the score obtained by referring to the calculation table and a predetermined reference value.

For example, if the score is equal to or higher than the reference value, the server device 2 extracts a message such as "your pupil wettability" low "and" advice to the ophthalmic subject earlier, as shown in fig. 12. The cause of the "true" that reduces the wettability of the pupil may be known. "such message, and outputs the result to the inspection apparatus 1.

In this way, since the server device 2 generates the final result in consideration of the inspection result of the wettability of the eye using the image, the risk assessment result, and the subjective symptom assessment result, the self-test result can be output with higher accuracy than the final result generated based on the inspection result of the wettability of the eye using the image alone.

Returning to fig. 16, the server apparatus 2 determines whether the score calculated in S7 is equal to or greater than a reference value (S8). When the score is equal to or greater than the reference value, that is, when the wettability of the eye is low (yes in S8), the server device 2 acquires information on the ophthalmic hospital shown in fig. 15 (M1) and includes the information in the examination result (S9).

When the score calculated in S7 is smaller than the reference value, that is, when the eye wettability is medium or high (no in S8), the server device 2 acquires information on the recommended eye drop shown in fig. 15 (M2) and includes the information in the examination result (S10).

Even when the wettability of the eyes is low, the server device 2 may acquire information on recommended eye drops shown in fig. 15 (M2) and include the information in the examination result. The server device 2 can acquire information on the ophthalmic hospital shown in fig. 15 (M1) and include the information in the examination result even when the eye wettability is medium or high. The server device 2 is not limited to generating a final result based on the subjective symptom assessment result and the risk assessment result, and the result of checking the wettability of the eye using the image. The server device 2 may generate a final result based on at least one of the subjective symptom assessment result and the risk assessment result, and the result of checking the wettability of the eye using the image.

The server device 2 outputs the generated inspection result to the inspection device 1 (S11). Thus, the inspection results shown in fig. 13 to 15 are displayed on the display 15 of the inspection apparatus 1. Thereafter, the server device 2 ends the present process.

In this way, the server device 2 performs the inspection processing in accordance with the inspection program 231, and thus the user 10 can easily perform the self-inspection of the eyes using the inspection device 1.

Other embodiments

In the present embodiment, the server device 2 performs the inspection process to inspect the wettability of the eyes of the user 10, but the inspection device 1 may perform the inspection process to inspect the wettability of the eyes of the user 10.

Fig. 18 is a diagram showing a configuration of an inspection system 1000a according to another embodiment. As shown in fig. 18, the inspection device 1a may store an inspection program 231, a presumption model 232, eye medicine information 233, ophthalmic hospital information 234, and a calculation table 237, which are provided in the server device 2 shown in fig. 2, in the memory 13. The processor 11 of the inspection device 1a may inspect the wettability of the eye based on the captured image included in the captured moving image obtained by capturing the eye by the imaging device 16 and the estimated model 232 including the neural network 2321. Further, the examination result thereof may be displayed on the display 15 together with the eye medicine information 233 or the ophthalmic hospital information 234 as a final examination result. That is, the inspection apparatus 1a may execute processing corresponding to the inspection processing of the server apparatus 2 shown in fig. 16.

As described above, the inspection device 1 can inspect the wettability of the eye based on the change in the fluctuation degree of the image of the eye. Here, a large degree of fluctuation of the image of the eye means a state in which the stability of the tear layer is reduced (dry eye). Therefore, the inspection device 1 can also perform inspection of the stability of the tear layer (dry eye) based on the change in the extent of fluctuation of the eye image.

The stability of the tear layer was also evaluated by measuring the Time from opening of the eye to the Break of the tear layer on the surface of the eye (tear layer Break Time (also referred to as Break Up Time or but). In the present disclosure, the examination apparatus 1 can also measure BUT based on a change in the degree of fluctuation of an image of an eye and evaluate the stability of the tear liquid layer. In particular, the inspection device 1 can also measure the non-invasive tear layer breakdown time (NIBUT) and evaluate the stability of the tear layer without using a fluorescent dye.

Thus, the descriptions of the inspection system 1000, the inspection device 1, the inspection method, and the inspection program for inspecting the wettability of the eye can be applied to the inspection system, the inspection device, the inspection method, and the inspection program for inspecting the stability of the tear liquid layer.

The descriptions of the inspection system 1000, the inspection device 1, the inspection method, and the inspection program themselves for inspecting the wettability of the eye can be applied to the inspection system, the inspection device, the inspection method, and the inspection program themselves for inspecting the presence or absence of dry eye.

[ summary ]

The present disclosure relates to an inspection system 1000 that inspects the wettability of the eye (stability of tear layer, presence or absence of dry eye). As shown in fig. 2 and 4, the inspection system 1000 includes an imaging device 16, a processor 21 (inspection unit) for inspecting the wettability of the eye (stability of tear layer, presence or absence of dry eye) based on a captured image obtained by capturing an eye by the imaging device 16 and an estimated model 232 including a neural network 2321, and a display 15 for displaying the inspection result of the processor 21.

Thus, the user 10 can objectively check the wettability of his/her own eyes (stability of tear layer, presence or absence of dry eye) by photographing his/her own eyes using the imaging device 16 of the inspection device 1. In this way, the user 10 can easily perform self-test of eyes using the inspection apparatus 1.

The term "captured image" may refer to any of "still image" (so-called photograph) captured by an imaging device and "still image" included in "captured moving image" captured by an imaging device. The inspection unit is configured to inspect the wettability of the eye (stability of tear layer, presence or absence of dry eye) based on at least "still image" (image included in photograph or moving image).

Preferably, as shown in fig. 3, the estimation model 232 performs learning so as to check the wettability of the eye (stability of tear layer, presence or absence of dry eye) from the captured image based on learning data 4, and the learning data 4 includes the captured image and the check result of the wettability of the eye (stability of tear layer, presence or absence of dry eye).

Thus, the inspection system 1000 can inspect the wettability of the eyes (stability of tear layer, presence or absence of dry eye) of the user 10 from the captured image using the learned estimation model 232.

Preferably, as shown in fig. 17, the captured image includes a first still image at a timing t1 immediately after blinking and a second still image at a timing t2 when a predetermined time (for example, 5 seconds) has elapsed after blinking.

Thus, the inspection system 1000 can inspect the wettability of the eyes (stability of tear layer, presence or absence of dry eye) of the user 10 by comparing the still image immediately after blinking with the still image when a predetermined time (for example, 5 seconds) has elapsed after blinking.

Preferably, each of the first still image and the second still image is extracted from a moving image obtained by capturing an eye by the imaging device 16.

As a result, the user 10 may capture his or her own eyes as moving images using the imaging device 16, and the inspection system 1000 may inspect the wettability (stability of tear layer, presence or absence of dry eye) of the eyes of the user 10 by extracting two still images from the captured moving images and comparing the two still images. Therefore, the user 10 can check the wettability of the eye (stability of tear layer, presence or absence of dry eye) extremely easily without taking a still image immediately after blinking or a still image when a predetermined time (for example, 5 seconds) has elapsed after blinking with the image pickup device 16.

Preferably, the first still image and the second still image are each images including at least a face of an eye. The eye image is extracted from the face image included in each of the first still image and the second still image.

Thus, the inspection system 1000 can inspect the wettability of the eyes (stability of tear layer, presence or absence of dry eye) of the user 10 by extracting images of the eye portion from each of the two still images and comparing the two images. Therefore, the user 10 can check the wettability of the eye (stability of tear layer, presence or absence of dry eye) extremely easily without photographing only a part of the eye with the image pickup device 16.

Preferably, as shown in S4 and S5 of fig. 16, the processor 21 checks the wettability of the eye (stability of tear layer, presence or absence of dry eye) based on the fluctuation degree of the eye image extracted from the first still image and the fluctuation degree of the eye image extracted from the second still image.

Thus, the inspection system 1000 can inspect the wettability of the eyes (stability of tear layer, presence or absence of dry eye) of the user 10 by comparing the fluctuation degrees of the eye images extracted from the two still images. Therefore, the inspection system 1000 can inspect the wettability of the eye (stability of tear layer, presence or absence of dry eye) with higher accuracy.

Preferably, as shown in fig. 12 (J1) and 13 (K1), the display 15 also displays an image of the eye included in the captured image.

Thus, the user 10 can confirm the image of his own eye used in the self-test by using the display 15. The image of the eye displayed on the display 15 may be the second still image at the timing t2 shown in fig. 17.

Preferably, as shown in fig. 15 (M2), the display 15 also displays information related to recommended ophthalmic drugs.

Thus, the user 10 can acquire information on an ophthalmic drug optimal for self-care of eyes.

Preferably, as shown in fig. 15 (M1), the display 15 also outputs information related to the ophthalmic hospital.

Thus, the user 10 is urged to go to the ophthalmic hospital, and can acquire information about the ophthalmic hospital.

Preferably, as shown in fig. 15 (M1), the information related to the ophthalmic hospital includes at least any one of information prompting diagnosis and treatment by the ophthalmic hospital, information related to diagnosis and treatment in the ophthalmic hospital, and information related to a recommended ophthalmic hospital.

Thus, the user 10 is urged to go to the ophthalmologic hospital, and can acquire information about the recommended ophthalmologic hospital.

Preferably, as shown in fig. 12 and 13, the display 15 displays the inspection result based on at least one of the subjective symptoms associated with the eye and the possibility of the eye having an ophthalmic disease, and the inspection result of the processor 11.

Thus, the user 10 can obtain a result obtained by checking the wettability of the eye (stability of tear layer, presence or absence of dry eye) in consideration of at least one of the subjective symptom-checkup result and the risk-checkup result.

Preferably, as shown in fig. 2, the inspection system 1000 further includes an inspection device 1 and a server device 2 capable of communicating with the inspection device 1. The inspection device 1 includes an imaging device 16 and a display 15. The server device 2 includes a processor 21 (inspection unit).

Thus, the user 10 can easily perform the self-test of the eyes by using the inspection device 1 and the server device 2 included in the inspection system 1000.

The present disclosure relates to an inspection device 1a for inspecting the wettability of the eye (stability of tear layer, presence or absence of dry eye). As shown in fig. 18, the inspection device 1a includes an imaging device 16, a processor 11 (inspection unit) for inspecting the wettability of the eye (stability of tear layer, presence or absence of dry eye) based on a captured image obtained by capturing an eye by the imaging device 16 and an estimated model 232 including a neural network 2321, and a display 15 for displaying the inspection result of the processor 11.

Thus, the user 10 can objectively check the wettability of his/her own eyes (stability of tear layer, presence or absence of dry eye) by photographing his/her own eyes using the imaging device 16 of the inspection device 1a. In this way, the user 10 can easily perform self-test of eyes using the test device 1a.

The present disclosure relates to an inspection device 1 for inspecting the wettability of the eye (stability of tear layer, presence or absence of dry eye). As shown in fig. 2, the inspection apparatus 1 includes an imaging apparatus 16, a communication apparatus 12 for communicating with the server apparatus 2, and a display 15, and the server apparatus 2 includes an estimation model 232 including a neural network 2321. The communication device 12 transmits the photographed moving image obtained by photographing the eye by the image pickup device 16 to the server device 2. The communication device 12 receives the inspection result of the wettability of the eye (stability of tear layer, presence or absence of dry eye) obtained based on the captured image of the eye extracted from the captured moving image by the server device 2 and the estimation model 232. The display 15 displays the inspection result.

Thus, the user 10 can objectively check the wettability of his/her own eyes (stability of tear layer, presence or absence of dry eye) by photographing his/her own eyes using the imaging device 16 of the inspection device 1. In this way, the user 10 can easily perform self-test of eyes using the inspection apparatus 1.

The present disclosure relates to an inspection method for inspecting the wettability (stability of tear layer, presence or absence of dry eye) of an eye by using a processor 21 (computer). The inspection method comprises the following steps: a step of inputting a captured image obtained by capturing an eye with the imaging device 16 (S2 of fig. 16), a step of checking the wettability of the eye (stability of tear layer, presence or absence of dry eye) based on the captured image and the estimation model 232 including the neural network 2321 (S5 of fig. 16), and a step of outputting a result of the checking step (S11 of fig. 16).

Thus, the user 10 can objectively check the wettability of his/her own eyes (stability of tear layer, presence or absence of dry eye) by photographing his/her own eyes using the imaging device 16 of the inspection device 1. In this way, the user 10 can easily perform self-test of eyes using the inspection apparatus 1.

The present disclosure relates to an inspection program 231 that inspects the wettability of the eye (stability of tear layer, presence or absence of dry eye). The inspection program 231 causes the processor 21 (computer) to execute a step of inputting a captured image obtained by capturing an eye by the imaging device 16 (S2 of fig. 16), a step of inspecting the wettability of the eye (stability of tear layer, presence or absence of dry eye) based on the captured image and the estimation model 232 including the neural network 2321 (S5 of fig. 16), and a step of outputting an inspection result of the step of inspecting (S11 of fig. 16).

Thus, the user 10 can objectively check the wettability of his/her own eyes (stability of tear layer, presence or absence of dry eye) by photographing his/her own eyes using the imaging device 16 of the inspection device 1. In this way, the user 10 can easily perform self-test of eyes using the inspection apparatus 1.

The presently disclosed embodiments are considered in all respects to be illustrative and not restrictive. The scope of the present disclosure is indicated by the claims, rather than the description of the embodiments described above, and all changes that come within the meaning and range of equivalency of the claims are intended to be embraced therein.

Description of the reference numerals

1. 1A, 1B, 1C, 1A inspection device, 2 server device, 4 learning data, 5 network, 10 user, 11, 21 processor, 12, 22 communication device, 13, 23 memory, 14 input interface, 15 display, 16 camera device, 31 learning device, 131 self-test application, 146, 147, 148, 149, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160 icon, 211 input unit, 212 processing unit, 213 output unit, 231 inspection program, 232 estimation model, 233 eye medicine information, 234 ophthalmic hospital information, 235 user information, 236 dynamic image data, 301, 302, 1511, 1512, 1521, 1522, 1531, 1532, 1541, 1543 image, 1000a inspection system, 2321 neural network, 2322 parameters.

Claims (18)

1. An inspection system for inspecting wettability of an eye, the inspection system comprising:

an image pickup device;

an inspection unit that inspects the wettability of an eye based on a captured image obtained by capturing the eye with the imaging device and an estimated model including a neural network; and

and a display for displaying the inspection result of the inspection unit.

2. The inspection system of claim 1, wherein,

The estimation model performs learning so as to check the wettability of the eye from the captured image based on learning data including the captured image and the check result of the wettability of the eye.

3. The inspection system of claim 1 or 2, wherein,

the photographed image includes a first still image immediately after blinking and a second still image when a prescribed time has elapsed after blinking.

4. The inspection system of claim 3 wherein,

the first still image and the second still image are each extracted from a moving image obtained by capturing the eye by the image capturing device.

5. The inspection system of claim 4, wherein,

the first still image and the second still image are each images including at least a face of the eye,

the eye image is extracted from the face image included in each of the first still image and the second still image.

6. The inspection system of claim 5, wherein,

the inspection unit inspects the wettability of the eye based on the degree of fluctuation of the eye image extracted from the first still image and the degree of fluctuation of the eye image extracted from the second still image.

7. The inspection system of any one of claims 1-6, wherein,

the display also displays an image of the eye contained in the captured image.

8. The inspection system of any one of claims 1-7, wherein,

the display also displays information related to recommended ophthalmic drugs.

9. The inspection system of any one of claims 1-8, wherein,

the display also outputs information related to the ophthalmic hospital.

10. The inspection system of claim 9, wherein,

the information related to the ophthalmic hospital includes at least any one of information prompting diagnosis and treatment in the ophthalmic hospital, information related to diagnosis and treatment in the ophthalmic hospital, and information related to a recommended ophthalmic hospital.

11. The inspection system of any one of claims 1-10, wherein,

the display displays an inspection result based on at least one of a subjective symptom associated with the eye and a possibility that the eye has an ophthalmic disease, and an inspection result of the inspection by the inspection unit.

12. The inspection system according to any one of claims 1 to 11, further comprising:

An inspection device; and

a server device capable of communicating with the inspection device,

the inspection device comprises the camera device and the display,

the server device includes the inspection unit.

13. An inspection device for inspecting wettability of an eye, the inspection device comprising:

an image pickup device;

an inspection unit that inspects the wettability of an eye based on a captured image obtained by capturing the eye with the imaging device and an estimated model including a neural network; and

and a display for displaying the inspection result of the inspection unit.

14. An inspection device for inspecting wettability of an eye, the inspection device comprising:

an image pickup device;

a communication device for communicating with a server device provided with an estimation model including a neural network; and

the display device is provided with a display device,

the communication device transmits a photographed moving image obtained by photographing an eye by the photographing device to the server device,

And receiving a result of checking the wettability of the eye obtained based on the captured image of the eye extracted from the captured moving image by the server device and the estimation model,

The display displays the inspection result.

15. An inspection method for inspecting wettability of an eye by a computer, the inspection method comprising:

a step of inputting a photographed image obtained by photographing an eye by an imaging device;

a step of checking the wettability of the eye based on the photographed image and an estimated model including a neural network; and

and outputting the inspection result of the inspecting step.

16. An inspection program that inspects wettability of an eye, the inspection program causing a computer to execute:

a step of inputting a photographed image obtained by photographing an eye by an imaging device;

a step of checking the wettability of the eye based on the photographed image and an estimated model including a neural network; and

and outputting the inspection result of the inspecting step.

17. An inspection system for inspecting stability of a tear liquid layer, the inspection system comprising:

an image pickup device;

an inspection unit that inspects the stability of the tear liquid layer based on a captured image obtained by capturing an image of an eye with the imaging device and an estimated model including a neural network; and

and a display for displaying the inspection result of the inspection unit.

18. An inspection system for inspecting the presence or absence of dry eye, the inspection system comprising:

an image pickup device;

an inspection unit that inspects the presence or absence of the dry eye based on a captured image obtained by capturing an image of an eye with the imaging device and an estimated model including a neural network; and

and a display for displaying the inspection result of the inspection unit.