JP7096020B2 - Mobile terminal and control method of mobile terminal - Google Patents

Description

The present invention relates to a mobile terminal and a method for controlling a mobile terminal.

In recent years, the number of patients with dry eye due to VDT (Visual Display Terminal Syndrome) work, long-term use of smartphones and the like, and use of contact lenses is increasing. The human tear film is composed of an oil layer (also referred to as a lipid layer), an aqueous layer, mucinous substance, and the like from the front surface side to the back surface side. The oil layer on the surface is secreted from the meibomian glands on the back of the eyelid and covers the surface of the tear layer to prevent evaporation of tears. The water layer is also secreted from the lacrimal glands, moisturizes the corneal surface, and then is excreted from the punctum. Dry eye occurs when the oil content secreted by the decrease in the meibomian glands decreases and the tear fluid becomes easier to dry, or the tear film layer (water layer) secreted from the lacrimal glands decreases. Therefore, in order to diagnose dry eye, it is necessary to observe changes in the thickness of the tear film on the surface of the cornea over time (state of destruction of the tear film, etc.).

Generally, as a method for measuring the amount of tears secreted, Schirmer's test in which tears secreted from the eye to be inspected are impregnated into an exposed paper to measure the amount of tears, and a fluorosane stain is instilled on the corneal surface of the eye to be inspected. There are known methods for observing the state of the tear film, but all of them are burdensome for the user. In addition, a method of observing the meibomian glands on the back of the eyelid with infrared light is also known, but it is not possible to directly observe tears by this method.

Therefore, in Patent Document 1 and Patent Document 2, an illumination unit that illuminates the anterior eye portion (corneal surface) of the eye to be inspected with white light, an imaging unit that photographs the anterior eye portion, and an imaging unit photographed by the photographing unit. Disclosed is an ophthalmic apparatus including a detection unit that analyzes an image and detects the state of the tear film of the eye to be inspected (thickness of the oil layer, etc.). According to this ophthalmologic device, the condition of the tear layer of the user can be inspected in a non-contact and non-invasive manner.

Special Table 2010-530282 Gazette Japanese Unexamined Patent Publication No. 2017-136212

By the way, the ophthalmic apparatus described in Patent Document 1 and Patent Document 2 requires a device for fixing the user's face, a mechanism for aligning the illumination unit and the imaging unit with respect to the eye to be inspected, a focus adjusting mechanism for the imaging unit, and the like. Therefore, the device becomes large-scale. For this reason, when performing a dry eye examination, the user has to go to an ophthalmologist and undergo an examination with a dedicated ophthalmology device.

The present invention has been made in view of such circumstances, and an object of the present invention is to provide a mobile terminal and a control method for the mobile terminal, which can easily inspect the tear film of the eye to be inspected.

The portable terminal for achieving the object of the present invention is provided on the housing and one side of the housing, emits a predetermined pattern light, and projects the pattern light onto the anterior eye portion of the eye to be inspected. And, the reflected image of the pattern light is detected from the photographing unit provided on one side of the housing and photographing the front eye portion on which the pattern light from the light emitting unit is projected, and the photographed image photographed by the photographing unit. It includes an image detection unit and a state detection unit that analyzes the reflected image detected by the image detection unit and detects the state of the tear film layer covering the corneal surface of the anterior eye portion.

According to this portable terminal, pattern light is projected onto the anterior eye portion, the anterior eye portion is photographed, and the captured image is analyzed (reflection image detection and reflection image detection) without requiring a large-scale mechanism as in a conventional ophthalmic device. The state of the tear film) can be performed.

In the portable terminal according to another aspect of the present invention, the image detection unit detects the anterior eye portion image corresponding to the anterior eye portion from the captured image, and based on the anterior eye portion image detected by the image detection unit, the subject to the eye is examined. It includes an alignment detection unit that detects the alignment state of the imaging unit, and an instruction unit that outputs an alignment instruction that is an index of relative alignment between the eye to be inspected and the imaging unit based on the detection result of the alignment detection unit. As a result, the user can manually align the imaging unit with respect to the eye to be inspected according to the alignment instruction.

In the portable terminal according to another aspect of the present invention, the light emitting unit can adjust the light emitting mode of the pattern light, and the image detection unit obtains a reflected image from the captured image and an anterior eye portion image corresponding to the anterior eye portion. Based on the detection result of the image detection unit, the projection state detection unit that detects the projection state of the pattern light on the anterior eye portion and the light emission mode of the light emitting unit are adjusted based on the detection result of the projection state detection unit. It is provided with a projection control unit that maintains the projection state of the pattern light in a specific state. As a result, even if the relative positional relationship between the user and the mobile terminal changes, the projected state of the pattern light projected from the light emitting portion to the anterior eye portion is maintained in a specific state, so that various states of the tear film can be maintained. It is possible to accurately detect changes over time.

In the portable terminal according to another aspect of the present invention, the projection state detection unit detects at least the projection range of the pattern light projected on the anterior eye portion as the projection state, and the projection control unit adjusts the light emission mode. At least adjust the emission range of the pattern light. This makes it possible to accurately detect changes over time in various states of the tear film.

In the mobile terminal according to another aspect of the present invention, the light emitting unit is a display device that displays a pattern corresponding to the pattern light, and the illuminance distribution detection unit that detects the illuminance distribution of the reflected image detected by the image detection unit. A display control unit that controls the display of a pattern by a display device based on the detection result of the illuminance distribution detection unit to make the illuminance distribution uniform is provided. As a result, the illuminance distribution of the reflected image can be made uniform, so that the state of the tear film can be accurately detected from the reflected image.

In the mobile terminal according to another aspect of the present invention, the light emitting unit is a display device that displays a pattern corresponding to the pattern light and a fixative. This allows the user to fix the fixative during the examination of the tear film.

In the portable terminal according to another aspect of the present invention, the image detection unit detects an anterior eye portion image corresponding to the anterior eye portion from the captured image, and the image of the anterior eye portion detected by the image detection unit is used for the eye to be inspected. It includes an alignment detection unit that detects the alignment state of the photographing unit, and a display control unit that controls a display device to change the display mode of the fixative based on the detection result of the alignment detection unit. This allows the user to easily grasp the alignment state.

In a portable terminal according to another aspect of the present invention, the light emitting unit is a liquid crystal display that projects patterned light polarized in a specific polarization direction onto the anterior segment of the eye, and the photographing unit emits light in a specific polarization direction. The anterior segment of the eye is photographed through a transparent polarizing plate. As a result, a part of the reflected light of the pattern light in the eyelids, the iris and the like can be dimmed by the polarizing plate, so that the generation of noise due to the scattered component of the reflected light is reduced.

In the mobile terminal according to another aspect of the present invention, the state detection unit detects an interference image generated on the corneal surface from the reflection image, and detects the thickness of the tear film as the state of the tear film based on the interference image. ..

In the control method of the mobile terminal for achieving the object of the present invention, a predetermined pattern light is emitted from a light emitting portion provided on one surface side of the housing, and the pattern light is projected onto the anterior eye portion of the eye to be inspected. The pattern light is taken from the shooting step of shooting the front eye part where the pattern light from the light emitting part is projected by the projection step and the shooting unit provided on one side of the housing, and the shot image taken in the shooting step. It has an image detection step of detecting the reflected image of the light, and a state detecting step of analyzing the reflected image detected in the image detecting step to detect the state of the tear film covering the corneal surface of the anterior eye portion.

INDUSTRIAL APPLICABILITY The present invention can easily inspect the tear film of the eye to be inspected.

It is a front view of the smartphone of 1st Embodiment. It is a schematic sectional drawing of a part of the smartphone of 1st Embodiment. It is a schematic diagram of the photographed image data. It is explanatory drawing for demonstrating the posture change of a smartphone. It is a schematic diagram of the captured image data 34 obtained by photographing after changing the posture of the smartphone. It is a block diagram which shows the electric structure of the smartphone of 1st Embodiment. It is a flowchart which showed the flow of the examination of the tear film layer of the eye under test by the smartphone of 1st Embodiment. It is a block diagram which shows the electric structure of the smartphone of 2nd Embodiment. It is explanatory drawing for demonstrating the first example of the adjustment of the display mode of a pattern by a projection control unit. It is explanatory drawing for demonstrating the 2nd example of the adjustment of the display mode of a pattern by a projection control unit. It is explanatory drawing for demonstrating the 3rd example of the adjustment of the display mode of a pattern by a projection control unit. It is a flowchart which showed the flow of the examination of the tear film layer of the eye under test by the smartphone of 2nd Embodiment. It is explanatory drawing for demonstrating the illuminance distribution of the pattern light projected from the display panel to the anterior eye portion. It is a block diagram which shows the electric structure of the smartphone of 3rd Embodiment. It is a flowchart which showed the flow of the examination of the tear film layer of the eye under test by the smartphone of 3rd Embodiment. It is a front view of the smartphone of 4th Embodiment. It is a front view of the smartphone of 5th Embodiment. It is a block diagram which showed the modification of the control part of each embodiment.

[Structure of smartphone of the first embodiment]
FIG. 1 is a front view of the smartphone 10 of the first embodiment corresponding to the mobile terminal of the present invention. FIG. 2 is a schematic cross-sectional view of a part of the smartphone 10 of the first embodiment. The smartphone 10 has a function of inspecting the tear film layer covering the corneal surface of the anterior eye portion Ef of the user to be inspected E, which is the subject.

As shown in FIGS. 1 and 2, the smartphone 10 includes a portable housing 12 that is formed in a flat plate shape and is held by a user. A display input unit 14, a speaker 16, a microphone 18, and a so-called in-camera camera unit 20 are provided on one side of the housing 12 facing the user. Further, a so-called out-camera is provided on the side opposite to the one side of the housing 12, although not shown.

The display input unit 14 displays various images (still images and moving images), character information, and the like. Further, the display input unit 14 has a so-called touch panel structure that detects a user operation for the displayed information. The display input unit 14 includes a display panel 22 and an operation panel 24 (not shown in FIG. 2).

The display panel 22 corresponds to the light emitting unit, the indicator unit, and the display device of the present invention, and various displays such as a liquid crystal display (LCD) and an organic EL display (OELD: Organic Electro-Luminescence Display). The device is used.

The display panel 22 displays the white pattern 28 in at least a part of the display surface at the time of the tear film examination. As a result, the white pattern light 28L based on the pattern 28 is emitted from the display panel 22, and the pattern light 28L is further projected onto the front eye portion Ef. The reference numeral R1 in FIG. 2 refers to the projection range of the pattern light 28L projected from the display panel 22 onto the anterior eye portion Ef.

As for the display position of the pattern 28 in the display panel 22, the pattern light 28L is the front eye portion Ef when the optical axis OA of the camera unit 20 and the reference position of the eye to be inspected E, which will be described later, substantially coincide with each other. It is pre-adjusted so that it is projected onto a predetermined position on the surface of the cornea (see FIG. 3). Further, the color, shape, size, etc. of the pattern 28 (pattern light 28L) are not particularly limited.

The operation panel 24 has light transmission and is provided on the display surface of the display panel 22. The operation panel 24 receives various input operations input by a user's finger, a stylus, or the like by a known detection method. Then, the operation panel 24 outputs the detection signal of the input operation to the control unit 46 (see FIG. 6) described later. In addition, various hardware keys such as a push button type switch and a cross key may be provided on one side of the housing 12.

The speaker 16 outputs various sounds under the control unit 46 (see FIG. 6) described later. The speaker 16 also functions as an instruction unit of the present invention, which will be described in detail later. The microphone 18 accepts a user's voice input.

The camera unit 20 corresponds to the photographing unit of the present invention, and electronically photographs the user's face. In the present embodiment, the camera unit 20 is used for photographing the anterior eye portion Ef (corneal surface) of the user's eye E to be inspected at the time of the tear film layer examination. The camera unit 20 includes a lens group 30 including a focus lens and the like, a lens drive mechanism (not shown) for driving each lens of the lens group 30, a shutter mechanism (not shown), and an image sensor 32.

The lens group 30 has an optical axis OA, and images of various subjects are imaged on the image pickup surface of the image pickup element 32. At this time, the camera unit 20 has a known autofocus function. Therefore, the focus of the camera unit 20 is automatically adjusted by moving the focus lens of the lens group 30 along the optical axis OA by a lens drive mechanism (not shown) under the control unit 46 (see FIG. 6). .. In this way, the camera unit 20 can perform focus adjustment (alignment in the optical axis direction) in the same manner as the stationary ophthalmic apparatus by performing the so-called autofocus process. Since the autofocus process is a known technique, a specific description thereof will be omitted.

The image sensor 32 is a CMOS (Complementary Metal Oxide Semiconductor) type or CCD (Charge-Coupled Device) type color image sensor (image sensor), and captures image light of various subjects through the lens group 30.

At the time of the tear film examination, the camera unit 20 performs main imaging (continuous imaging) of the corneal surface of the anterior eye portion Ef on which the pattern light 28L is projected by the display panel 22, and the captured image data 34 of the anterior eye portion Ef. Is output. The main imaging referred to here is imaging of captured image data 34 used for detecting the state of the tear film.

Further, the camera unit 20 performs preliminary shooting (continuous shooting) of the front eye portion Ef before the start of the main shooting, and outputs the captured image data 34 of the front eye portion Ef. The captured image data 34 obtained in this preliminary imaging is used for detecting the alignment state of the housing 12 (smartphone 10) with respect to the eye to be inspected E and for detecting the blink of the eye to be inspected E before the start of the main imaging. The detection of the alignment state here means the alignment state in the direction perpendicular to the optical axis perpendicular to the optical axis OA, that is, the deviation between the optical axis OA and the reference position (pupil, corneal apex, etc.) of the eye to be inspected E. Detection of quantity and deviation direction.

In the present embodiment, the preliminary shooting for blink detection by the camera unit 20 and the main shooting for detecting the state of the tear film are described separately, but both may be one continuous shooting. ..

FIG. 3 is a schematic view of the captured image data 34 obtained by the main imaging. As shown in FIG. 3, the captured image data 34 includes an anterior eye portion image 35 corresponding to the anterior eye portion Ef and a reflected image 36 of the pattern light 28L reflected on the corneal surface of the anterior eye portion Ef. ing. Then, the smartphone 10 of the present embodiment detects the alignment state of the camera unit 20 in the vertical and horizontal directions (substantially vertical direction of the optical axis) with respect to the eye E to be inspected, based on the result of detecting the anterior eye portion image 35 from the captured image data 34. I do. Further, the smartphone 10 detects and analyzes the reflected image 36 from the captured image data 34 to detect the state of the tear film layer covering the projection range R1 on the corneal surface of the anterior eye portion Ef.

FIG. 4 is an explanatory diagram for explaining the posture change of the smartphone 10 at the time of the actual shooting. FIG. 5 is a schematic view of the captured image data 34 obtained by the main shooting after the posture of the smartphone 10 is changed.

As shown in FIG. 4, when the user changes the posture of the smartphone 10, for example, by rotating the smartphone 10 in the direction around the axis of the optical axis OA, the pattern 28 also rotates in the same direction. As a result, the projection position and shape of the pattern light 28L projected on the anterior eye portion Ef can be changed. Then, as shown in FIG. 5, by detecting and analyzing the reflected image 36 from the captured image data 34 after the posture change, the tear film layer covering the new projection range R1 in the corneal surface of the anterior eye portion Ef. The state of is detected. Thereby, the user can make the smartphone 10 detect the state of the tear film in a desired region on the corneal surface of the anterior eye portion Ef by changing the posture of the smartphone 10.

FIG. 6 is a block diagram showing an electrical configuration of the smartphone 10 of the first embodiment. In FIG. 6, in order to prevent the drawings from becoming complicated, the configurations related to the functions other than the tear film layer inspection are not shown as appropriate among the configurations related to the various functions of the smartphone 10.

As shown in FIG. 6, in addition to the display input unit 14, the speaker 16, and the microphone 18 described above, the smartphone 10 includes a motion sensor 40, a communication unit 42, a storage unit 44, and a control unit 46. Be prepared.

The motion sensor 40 includes, for example, a three-axis acceleration sensor, detects the physical movement of the smartphone 10, and outputs the detection result to the control unit 46.

The communication unit 42 performs wireless communication (wired communication is also possible) with the base station device or router device housed in the mobile communication network. As a result, the communication unit 42 transmits / receives various data such as voice data and image data.

The storage unit 44 stores the program 48, the test results of the tear film (not shown), and the like. Program 48 is application software for inspecting the tear film with the smartphone 10. This program 48 may be pre-installed in the storage unit 44, or may be acquired via the Internet.

The control unit 46 is a calculation circuit composed of various calculation units including a CPU (Central Processing Unit) or an FPGA (field-programmable gate array), a memory, and the like, and controls each part of the smartphone 10 in an integrated manner.

Then, when the start operation of the program 48 is input to the operation panel 24, the control unit 46 reads the program 48 from the storage unit 44 and executes the program 48 to inspect the smartphone 10 for the tear film layer of the eye E to be inspected. Make it function as a device. In this case, the control unit 46 controls each part of the smartphone 10 to perform preliminary imaging and detection of the alignment state, projection of the pattern light 28L onto the anterior eye portion Ef, preliminary imaging and blink detection of the eye to be inspected E. The main imaging and the state detection of the tear film are performed in order.

By executing the program 48, the control unit 46 includes a display control unit 50, a voice control unit 52, an image acquisition unit 54, an image detection unit 56, an alignment detection unit 58, an instruction determination unit 60, an inspection control unit 62, and a state. It functions as a detection unit 64.

The display control unit 50 controls the image display of the display panel 22. The display control unit 50 displays various images or information on the display panel 22 based on various instructions or various information input from the instruction determination unit 60, the inspection control unit 62, the state detection unit 64, etc. at the time of the tear film inspection. Is displayed.

The voice control unit 52 controls the operation of the speaker 16 and the microphone 18. At the time of the tear film inspection, the voice control unit 52 causes the speaker 16 to output various instructions or information input from the instruction determination unit 60, the inspection control unit 62, and the state detection unit 64.

The image acquisition unit 54 is connected to the image pickup element 32 of the camera unit 20, and sequentially acquires and acquires the captured image data 34 of the anterior eye portion Ef obtained by the preliminary imaging or the main imaging described above from the image pickup element 32. The captured image data 34 is sequentially output to the image detection unit 56.

When the captured image data 34 for preliminary shooting is sequentially input from the image acquisition unit 54, the image detection unit 56 may use, for example, a known pattern matching method (template matching method) or a known eye (pupil) for each captured image data 34. The anterior ocular segment image 35 is detected from the captured image data 34 by performing analysis by the detection method. Then, the image detection unit 56 outputs the position detection result of the anterior eye portion image 35 in the captured image data 34 to the alignment detection unit 58 at the time of alignment detection, and detects from the captured image data 34 at the time of blink detection of the eye subject ER. The image data of the anterior eye portion image 35 (extracted) is output to the inspection control unit 62.

Further, when the captured image data 34 of the main shooting is sequentially input from the image acquisition unit 54, the image detection unit 56 analyzes each captured image data 34 by a pattern matching method or the like, and performs analysis from the captured image data 34 to the front eye. The part image 35 and the reflected image 36 are detected. Here, the posture of the reflected image 36 in the captured image data 34 changes according to the posture of the smartphone 10 as shown in FIGS. 1, 3, and 5 described above. Therefore, the image detection unit 56 discriminates the posture and shape of the reflected image 36 in the captured image data 34 based on the detection result of the motion sensor 40, and detects the reflected image 36 from the captured image data 34 based on this discrimination result. do.

Then, the image detection unit 56 outputs the image data of the anterior eye portion image 35 detected from the captured image data 34 to the inspection control unit 62, and outputs the image data of the reflection image 36 detected from the captured image data 34 to the state detection unit 64. Output to. The image data of the anterior eye portion image 35 is used for blink detection of the eye to be inspected E after the start of main imaging by the inspection control unit 62 described later, and the image data of the reflection image 36 is tears by the state detection unit 64 described later. It is used to detect the state of the liquid layer.

The alignment detection unit 58 detects the alignment state of the camera unit 20 with respect to the eye E to be inspected based on the position detection result of the anterior eye portion image 35 sequentially input from the image detection unit 56. For example, the alignment detection unit 58 detects the position of the anterior eye portion image 35 (pupil or the like) in the captured image data 34 from the detection result of the image detection unit 56. Next, the alignment detection unit 58 determines the relative positional deviation direction and position of the reference position of the eye E to be inspected with respect to the optical axis OA based on the position detection result of the anterior eye portion image 35 and the setting information of the known lens group 30. The amount of deviation is detected, and this detection result is output to the instruction determination unit 60 as an alignment detection result.

The instruction determination unit 60 determines an alignment instruction as an index of relative alignment between the eye E to be inspected and the camera unit 20 based on the alignment detection result input from the alignment detection unit 58. For example, when the reference position of the eye E to be inspected is displaced relative to the optical axis OA based on the alignment detection result, the instruction determination unit 60 can correct the misalignment as an alignment instruction. Determine the position adjustment instruction of. Further, when the instruction determination unit 60 substantially matches the reference position of the optical axis OA and the eye E to be inspected based on the alignment detection result, the instruction determination unit 60 maintains the position to maintain the current position of the housing 12 as an alignment instruction. Determine the instructions.

Then, the instruction determination unit 60 outputs the determined alignment instruction (position adjustment instruction or position maintenance instruction) to the voice control unit 52. As a result, the voice control unit 52 executes voice output of the alignment instruction on the speaker 16. Therefore, the instruction determination unit 60 constitutes the instruction unit of the present invention together with the speaker 16 and the voice control unit 52.

The inspection control unit 62 controls the screen display of the display panel 22 via the display control unit 50, controls the audio output of the speaker 16 via the audio control unit 52, and controls the preliminary shooting and the main shooting by the camera unit 20. conduct. Hereinafter, each control by the inspection control unit 62 will be specifically described.

Before the alignment detection, the inspection control unit 62 outputs a gaze instruction to the user to gaze at the camera unit 20 to at least one of the display control unit 50 and the voice control unit 52. As a result, at least one of the screen display of the gaze instruction on the display panel 22 by the display control unit 50 and the voice output of the gaze instruction on the speaker 16 by the voice control unit 52 is executed. As a result, the user can gaze at the camera unit 20 before the alignment detection.

After outputting the gaze instruction, the inspection control unit 62 controls the camera unit 20 to execute a preliminary image for alignment detection. As a result, the front eye portion Ef is continuously photographed by the camera unit 20, and the captured image data 34 is sequentially input from the camera unit 20 to the image acquisition unit 54. As a result, the position detection of the anterior eye portion image 35 by the above-mentioned image detection unit 56, the detection of the alignment state by the alignment detection unit 58, the output of the alignment instruction by the instruction determination unit 60, and the alignment instruction by the speaker 16 or the like. Audio output and are executed repeatedly. Therefore, manual alignment by the user is possible.

The inspection control unit 62 controls the display control unit 50 when the alignment is completed, that is, when the optical axis OA and the reference position of the eye E to be inspected substantially match based on the alignment detection result of the alignment detection unit 58. The pattern 28 is displayed on the display panel 22. As a result, the pattern light 28L is projected from the display panel 22 onto the front eye portion Ef. During alignment, a pattern for alignment different from the pattern 28 may be displayed and output on the display panel 22, and alignment may be performed based on the corneal reflection image of this alignment pattern.

After displaying the pattern 28 on the display panel 22, the inspection control unit 62 outputs a blink instruction to the user to blink to the voice control unit 52. As a result, the voice control unit 52 executes the voice output of the blink instruction by the speaker 16. As a result, the user can be made to blink before the start of the main shooting.

Further, the inspection control unit 62 controls the camera unit 20 at the same time as outputting the blink instruction to execute the preliminary shooting for blink detection. As a result, the front eye portion Ef is continuously photographed by the camera unit 20, and the captured image data 34 is sequentially input from the camera unit 20 to the image acquisition unit 54. As a result, each time new captured image data 34 is input to the image acquisition unit 54, the image detection unit 56 detects the anterior eye portion image 35 and the image detection unit 56 inspects the image data of the anterior eye portion image 35. The output to the control unit 62 and the output to the control unit 62 are repeatedly executed.

Then, the inspection control unit 62 performs blink detection of the eye to be inspected E based on the image data of the anterior eye portion image 35 sequentially input from the image detection unit 56. For example, the inspection control unit 62 calculates the size of the pupil of the eye to be inspected E from the image data of the anterior eye portion image 35 each time the detection result of the anterior eye portion image 35 is input from the image detection unit 56. Then, the inspection control unit 62 determines that there is a blink when an extreme change occurs in the size of the pupil. Various methods are known for detecting blinks (Japanese Patent Laid-Open No. 9-215662), and any known method may be used.

When the inspection control unit 62 detects the blink of the user, the inspection control unit 62 controls the camera unit 20 to execute the main shooting. As a result, the front eye portion Ef is continuously photographed by the camera unit 20, and the captured image data 34 is sequentially input from the camera unit 20 to the image acquisition unit 54. As a result, each time new captured image data 34 is input to the image acquisition unit 54, the image detection unit 56 detects the anterior eye portion image 35 and the reflection image 36, and the inspection control unit 62 detects the anterior eye portion image 35. The output of the image data of the above and the output of the image data of the reflection image 36 to the state detection unit 64 are repeatedly executed.

The inspection control unit 62 ends the main shooting by the camera unit 20 when either of the conditions from the start of the main shooting until the user blinks again or until a certain time elapses is satisfied. Therefore, the inspection control unit 62 continuously performs blink detection based on the image data of the anterior eye portion image 35 sequentially input from the image detection unit 56 even after the start of the main imaging.

The state detection unit 64 analyzes the image data of the reflection image 36 continuously input from the image detection unit 56 during the main imaging to detect the state of the tear film layer covering the corneal surface of the anterior eye portion Ef.

For example, in the projection range R1 on the surface of the cornea of the anterior eye portion Ef, the pattern light 28L is reflected on the front and back surfaces of the oil layer of the tear film, so that an interference image (interference fringe) is generated on the surface of the cornea. This interference image has a color corresponding to the thickness of the oil layer of the tear film at each position in the projection range R1. Therefore, by detecting the color at each position of the reflected image 36, the thickness of the oil layer at each position within the projection range R1 can be detected. Therefore, the state detection unit 64 detects the thickness of the oil layer at each position by detecting the color of each position in the projection range R1 for each image data of the reflection image 36. As a result, the state detection unit 64 can detect the change over time in the thickness of the oil layer from the start time to the end time of the main imaging as the state of the tear film.

Further, the state detection unit 64 determines the shape and formation position of the dry spot, which is a region on the corneal surface where the tear layer is destroyed, as the state of the tear layer based on the detection result of the change in the thickness of the oil layer with time. Can be detected. Then, the state detection unit 64 outputs the detection result of the state of the tear film to the display control unit 50 and the storage unit 44 as the inspection result of the tear film. As a result, the inspection result of the tear film is displayed on the display panel 22 by the display control unit 50 and stored in the storage unit 44. When the tear film is destroyed, the interference pattern of the oil layer disappears, and the reflectance of the fractured portion decreases due to the destruction of the aqueous layer. Therefore, the tear film is also destroyed by the change in the brightness of the reflected image 36. Can be detected.

[Action of smartphone of the first embodiment]
FIG. 7 is a flowchart showing the flow of inspection of the tear film layer of the eye E to be inspected by the smartphone 10 of the first embodiment (corresponding to the control method of the mobile terminal of the present invention).

As shown in FIG. 7, the user performs an operation to start the program 48 on the display input unit 14 of the smartphone 10 (step S1). In response to this operation, the control unit 46 reads the program 48 from the storage unit 44 and executes it, so that the examination of the tear film by the smartphone 10 is started.

First, the inspection control unit 62 outputs a gaze instruction to at least one of the display control unit 50 and the voice control unit 52. As a result, at least one of the screen display of the gaze instruction on the display panel 22 by the display control unit 50 and the voice output of the gaze instruction on the speaker 16 by the voice control unit 52 is executed, so that the user (eye to be inspected E) is executed. ) Can gaze at the camera unit 20 (step S2).

Next, the inspection control unit 62 starts preliminary imaging of the anterior eye portion Ef by the camera unit 20 (step S3). As a result, the front eye portion Ef is continuously photographed by the camera unit 20, and the captured image data 34 of the front eye portion Ef is sequentially output from the camera unit 20 to the image acquisition unit 54. Then, the image acquisition unit 54 acquires the captured image data 34, the image detection unit 56 detects the anterior eye portion image 35 (step S4), the alignment detection unit 58 detects the alignment state (step S5), and the instruction determination unit. The output of the alignment detection result to 60 and the execution are performed.

When the reference position of the eye E to be inspected is displaced relative to the optical axis OA, the instruction determination unit 60 that receives the input of the alignment detection result gives an alignment instruction as a position adjustment instruction capable of correcting the misalignment. Is determined, and this position adjustment instruction is output to the voice control unit 52 (NO in step S6). As a result, the audio output of the alignment instruction (position adjustment instruction) on the speaker 16 is executed (step S7). As a result, the user moves the face or the smartphone 10 according to the alignment instruction, so that the relative alignment (manual alignment) between the eye E to be inspected and the camera unit 20 is executed.

Hereinafter, each process from step S3 to step S7 is repeatedly executed until the optical axis OA and the reference position of the eye to be inspected E substantially match (YES in step S6). Then, based on the alignment detection result, the instruction determination unit 60 determines a position maintenance instruction as an alignment instruction when the optical axis OA and the reference position of the eye to be inspected E substantially match, and the voice control unit 52 issues this position maintenance instruction. Output to. As a result, the speaker 16 executes the voice output of the position maintenance instruction, so that the user can determine that the alignment is completed.

Regarding the alignment between the front eye portion Ef and the camera unit 20 in the direction of the optical axis OA, first, this voice is output by outputting a voice instruction so that the imaging range of the front eye portion Ef falls within the specified range. Rough alignment is performed by the user moving the smartphone 10 back and forth according to the instruction, or by changing the imaging magnification so that the observation range becomes a specified range by using the zoom function of the camera unit 20. Next, precision alignment is executed using the autofocus function of the camera unit 20.

When the alignment is completed, the inspection control unit 62 controls the display control unit 50 to display the pattern 28 on the display panel 22, so that the display panel 22 emits the pattern light 28L. As a result, the pattern light 28L is projected from the display panel 22 onto the anterior eye portion Ef (step S8, corresponding to the projection step of the present invention).

Next, the inspection control unit 62 outputs a blink instruction to the voice control unit 52. As a result, the voice output of the blink instruction by the speaker 16 is executed, so that the user can be urged to execute the blink (step S9).

Further, the inspection control unit 62 starts the preliminary imaging of the anterior eye portion Ef by the camera unit 20 at the same time as the output of the blink instruction. As a result, the front eye portion Ef is continuously photographed by the camera unit 20, and the captured image data 34 is sequentially output from the camera unit 20 to the image acquisition unit 54. Further, each time the captured image data 34 is output from the camera unit 20, the image acquisition unit 54 acquires the captured image data 34, the image detection unit 56 detects the anterior eye portion image 35, and the inspection control unit 62 blinks. Detection and execution are repeated (NO in step S10).

Then, when the inspection control unit 62 detects the blink of the eye to be inspected E, the camera unit 20 starts the main imaging of the anterior eye portion Ef (YES in step S10, step S11, corresponding to the imaging step of the present invention). As a result, the front eye portion Ef is continuously photographed by the camera unit 20, and the captured image data 34 is sequentially output from the camera unit 20 to the image acquisition unit 54. The captured image data 34 may be a moving image or a still image captured at regular time intervals.

Further, each time the captured image data 34 is output from the camera unit 20, the image acquisition unit 54 acquires the captured image data 34, and the image detection unit 56 detects the anterior eye portion image 35 and the reflected image 36 (step S12, (Corresponding to the image detection step of the present invention), blink detection by the inspection control unit 62, and output of the image data of the reflection image 36 to the state detection unit 64 are repeatedly executed (NO in step S15).

After the start of the main shooting, when the inspection control unit 62 detects another blink of the user, the inspection control unit 62 ends the main shooting by the camera unit 20 (YES in step S15).

When the main imaging is completed, the state detection unit 64 analyzes the image data of the reflection image 36 continuously input from the image detection unit 56, and changes the thickness of the tear film (oil layer) with time and the dry spot. Various states of the tear film including the shape and formation position are detected (step S16, corresponding to the state detection step of the present invention). As a result, the state of the tear film in the projection range R1 is detected. The state detection result of the tear layer is output from the state detection unit 64 to the display control unit 50 and the storage unit 44, respectively, as the inspection result of the tear layer. Then, the test result of the tear film is displayed on the display panel 22 and stored in the storage unit 44.

[Effect of the first embodiment]
As described above, in the smartphone 10 of the first embodiment, the pattern light 28L is projected onto the anterior eye portion Ef, the anterior eye portion Ef is photographed, and the captured image data 34 is analyzed using the existing members of the smartphone 10. (Detection of the reflection image 36 and detection of the state of the tear film) can be performed. For this reason, a large-scale mechanism such as a device for fixing the user's face as provided in a conventional ophthalmic device, a mechanism for aligning the illumination unit and the imaging unit with respect to the eye E to be inspected, and a focus adjustment mechanism for the imaging unit. Is unnecessary. As a result, the user can easily inspect the tear film without going to an ophthalmologist.

[Smartphone of the second embodiment]
FIG. 8 is a block diagram showing an electrical configuration of the smartphone 10 of the second embodiment. In the first embodiment, it is assumed that the relative positional relationship between the user and the smartphone 10 (camera unit 20) is kept constant while the main shooting is being executed, but the user is during the main shooting. There is a risk of moving the face or the smartphone 10. In this case, the projection state (projection range R1 and brightness) of the pattern light 28L projected from the display panel 22 onto the anterior eye portion Ef changes. It is not possible to accurately detect changes in the state of the tear layer in the area over time.

At this time, the display panel 22 can arbitrarily adjust the display mode of the pattern 28 to be displayed, that is, the light emission mode of the pattern light 28L can be arbitrarily adjusted. Therefore, in the smartphone 10 of the second embodiment, even if the user accidentally moves the face or the smartphone 10 during the main shooting (even during the preliminary shooting after the start of projection of the pattern light 28L), the anterior eye portion Ef A function is provided to maintain the projected state of the pattern light 28L projected on the smartphone in a predetermined specific state.

As shown in FIG. 8, the smartphone 10 of the second embodiment is basically the same as the smartphone 10 of the first embodiment except that the control unit 46 functions as the projection state detection unit 70 and the projection control unit 72. It is a composition. Therefore, those having the same function or configuration as the first embodiment are designated by the same reference numerals and the description thereof will be omitted.

The image detection unit 56 of the second embodiment is basically the same as the image detection unit 56 of the first embodiment. However, the image detection unit 56 of the second embodiment detects the detection results of both the anterior eye portion image 35 and the reflection image 36 each time the captured image data 34 is input from the image acquisition unit 54 during the main shooting (hereinafter, The image detection result) and the captured image data 34 are output to the projection state detection unit 70.

Each time the image detection result and the captured image data 34 are input from the image detection unit 56 during the main imaging, the projection state detection unit 70 is based on the image detection result from the captured image data 34 to the corneal surface of the anterior eye portion Ef. The projection state of the pattern light 28L with respect to is detected. In the projection state of the pattern light 28L, the relative position between the user and the smartphone 10 (camera unit 20) such as the projection range R1 (projection position, size, shape, etc.) of the pattern light 28L, brightness, and the like. It contains one of the items that changes as the relationship changes.

For example, the projection state detection unit 70 discriminates from the captured image data 34 the image range of the reflected image 36 with respect to the anterior eye portion image 35 and the brightness of the reflected image 36, respectively, based on the above-mentioned image detection result. Next, the projection state detection unit 70 detects the projection range R1 of the pattern light 28L with respect to the anterior eye portion Ef based on the image range of the reflection image 36 described above, and the pattern light 28L based on the brightness of the pixels of the reflection image 36 described above. Detects the brightness of.

Then, each time the image detection result and the captured image data 34 are input from the image detection unit 56, the projection state detection unit 70 outputs the detection result of the projection state (projection range R1 and brightness) of the pattern light 28L to the projection control unit. Output to 72.

The projection control unit 72 maintains the projection state of the pattern light 28L in a specific state based on the detection result of the projection state of the pattern light 28L sequentially input from the projection state detection unit 70 during the main shooting. , The projection control of the pattern light 28L projected from the display panel 22 onto the anterior segment Ef is performed. The specific state may be a projection state predetermined by the manufacturer or the like, or may be a projection state of the pattern light 28L first input from the projection state detection unit 70 at the start of the main shooting, and is particularly limited. Will not be done.

The projection control unit 72 detects the difference between the projection state and the specific state each time the detection result of the projection state of the pattern light 28L is input from the projection state detection unit 70 during the main shooting. Then, the projection control unit 72 displays the display mode (display range and brightness) of the pattern 28 by the display panel 22 via the display control unit 50 based on the difference between the projection state and the specific state, that is, the pattern light by the display panel 22. By adjusting the light emission mode (light emission range and brightness) of 28L, the projection control of the pattern light 28L projected on the anterior eye portion Ef is performed.

For example, when the projection range R1 of the pattern light 28L is different from the range defined in a specific state, the projection control unit 72 is displayed on the display panel 22 via the display control unit 50 based on the difference between the two ranges. The display range of the pattern 28 (the emission range of the pattern light 28L) is adjusted. As a result, the projection range R1 of the pattern light 28L projected from the display panel 22 onto the front eye portion Ef is adjusted to a range defined in a specific state.

Further, when the brightness of the pattern light 28L is different from the brightness defined in a specific state, the projection control unit 72 controls and displays the display control unit 50 according to the difference between the brightness of both. The brightness of the pattern 28 displayed on the panel 22 is adjusted. As a result, the brightness of the pattern light 28L projected from the display panel 22 onto the front eye portion Ef is adjusted to the brightness defined in a specific state.

FIG. 9 is an explanatory diagram for explaining a first example of adjusting the display mode of the pattern 28 by the projection control unit 72. As shown by reference numeral 9A in FIG. 9, when the eye E to be inspected is displaced relative to the smartphone 10 by ΔL along the direction of the optical axis OA from the positional relationship satisfying the above-mentioned specific state. If the projection control is not performed by the projection control unit 72, the projection range R1 of the pattern light 28L projected from the display panel 22 onto the anterior eye portion Ef changes.

Therefore, as shown by reference numeral 9B in FIG. 9, the projection control unit 72 shifts the projection range R1 of the pattern light 28L based on the detection result of the projection state of the pattern light 28L sequentially input from the projection state detection unit 70. The display range of the pattern 28 on the display panel 22 is changed from the range W0 to the range W1 via the display control unit 50 so as to be maintained constant before and after.

FIG. 10 is an explanatory diagram for explaining a second example of adjusting the display mode of the pattern 28 by the projection control unit 72. As shown in FIG. 10, in the second example, in the second example, the eye E to be inspected is relative to the smartphone 10 by Δh in the direction perpendicular to the optical axis from the positional relationship satisfying the above-mentioned specific state (see reference numeral 9A in FIG. 9). Is out of position. In this case, the projection control unit 72 passes through the display control unit 50 so that the projection range R1 is maintained constant before and after the positional shift based on the detection result of the projection state sequentially input from the projection state detection unit 70. The display range of the pattern 28 on the display panel 22 is changed from the range W0 to the range W2.

FIG. 11 is an explanatory diagram for explaining a third example of adjusting the display mode of the pattern 28 by the projection control unit 72. As shown in FIG. 11, in the third example, the eye E to be inspected is tilted relative to the smartphone 10 by an angle θ from the positional relationship (see reference numeral 9A in FIG. 9) that satisfies the above-mentioned specific state. .. In this case, the projection control unit 72 keeps the projection range R1 constant before and after the relative inclination of the eye E to be inspected based on the detection result of the projection state sequentially input from the projection state detection unit 70. The display range of the pattern 28 on the display panel 22 is changed from the range W0 to the range W3 via the display control unit 50.

Further, the projection control unit 72 before and after the change in the relative positional relationship between the user and the smartphone 10 based on the detection result of the projection state sequentially input from the projection state detection unit 70 in the above-mentioned first to third examples. The brightness of the pattern 28 on the display panel 22 is appropriately adjusted via the display control unit 50 so that the brightness of the reflected image 36 is kept constant.

FIG. 12 is a flowchart showing the flow of inspection of the tear film layer of the eye E to be inspected by the smartphone 10 of the second embodiment. In FIG. 9, the processes from step S1 to step S12 are the same as those of the first embodiment shown in FIG. 7 described above, and thus the illustration is omitted as appropriate.

As shown in FIG. 12, in the smartphone 10 of the second embodiment, the front eye portion image 35 and the reflection image 36 are detected by the above-mentioned image detection unit 56 from the start of the main shooting to the end of the main shooting (step). S12), the projection state detection by the projection state detection unit 70 (step S13A), and the projection control by the projection control unit 72 (step S13B) are repeatedly executed.

As a result, in the smartphone 10 of the second embodiment, even when the relative positional relationship between the user and the smartphone 10 changes, the pattern light projected from the display panel 22 to the anterior segment Ef by the projection control by the projection control unit 72. The projection state (projection range R1 and brightness) of 28L is maintained in a specific state, that is, a constant state. This makes it possible to accurately detect changes over time in various states of the tear film.

[Smartphone of the third embodiment]
FIG. 13 is an explanatory diagram for explaining the illuminance distribution of the pattern light 28L projected from the display panel 22 onto the anterior eye portion Ef in the smartphone 10 of each of the above embodiments.

As shown in FIG. 13, the display panel 22 has a certain area. Therefore, the incident angle θi at which the pattern light 28L emitted from each position P on the display surface of the display panel 22 (only two points are exemplified in the figure) is incident on the anterior eye portion Ef is set for each position P on the display surface. Will change to. Then, the intensity (luminosity) of the pattern light 28L incident on the anterior eye portion Ef from each position P is reduced by COS (θi) according to the incident angle θi of each position P. As a result, the intensity of the pattern light 28L projected on the anterior eye portion Ef changes for each position P, so that the illuminance distribution of the reflected image 36 projected on the anterior eye portion Ef becomes non-uniform. It gets darker in the peripheral area. As a result, the state of the tear film may not be accurately detected from the image data of the reflected image 36.

Therefore, in the smartphone 10 of the third embodiment, the image sensor 32 of the reflected image (reflected light) of the pattern light 28L reflected by the cornea during the main shooting (even during the preliminary shooting after the start of projection of the pattern light 28L). The display control of the pattern 28 displayed on the display panel 22 is performed so that the above illuminance distribution becomes uniform.

FIG. 14 is a block diagram showing an electrical configuration of the smartphone 10 of the third embodiment. As shown in FIG. 14, the smartphone 10 of the third embodiment has basically the same configuration as the smartphone 10 of the first embodiment, except that the control unit 46 functions as the illuminance distribution detection unit 76. Therefore, those having the same function or configuration as the first embodiment are designated by the same reference numerals and the description thereof will be omitted.

The image detection unit 56 of the third embodiment is basically the same as the image detection unit 56 of the first embodiment. However, the image detection unit 56 of the third embodiment detects the illuminance distribution of the image data of the reflection image 36 detected from the captured image data 34 each time the captured image data 34 is input from the image acquisition unit 54 during the main shooting. Output to unit 76.

Each time the image data of the reflected image 36 is input from the image detecting unit 56 during the main shooting, the illuminance distribution detecting unit 76 analyzes the image data of the reflected image 36 and detects the illuminance for each pixel to reflect the reflection. The illuminance distribution of the image 36 is detected. Then, each time the illuminance distribution detection unit 76 detects the illuminance distribution of the reflected image 36, the detection result is output to the display control unit 50.

The display control unit 50 of the third embodiment makes the illuminance distribution of the reflected image 36 uniform every time the detection result of the luminance distribution of the reflected image 36 is input from the illuminance distribution detecting unit 76 during the main shooting. , The display control of the pattern 28 displayed on the display panel 22 is executed, and the brightness of each pixel of the pattern 28 is adjusted.

For example, the display control unit 50 performs a discrimination process for discriminating a target pixel region that requires luminance adjustment in the reflected image 36 based on the illuminance distribution of the reflected image 36, and each pixel of the pattern 28 displayed on the display panel 22. The brightness adjustment process for adjusting the brightness of the pixel corresponding to the target pixel area is performed. The brightness adjustment process adjusts the transmittance of the liquid crystal element when the display panel 22 is a liquid crystal display, and adjusts the light emission intensity of the light emitting element when the display panel 22 is an organic EL display.

FIG. 15 is a flowchart showing the flow of inspection of the tear film layer of the eye E to be inspected by the smartphone 10 of the third embodiment. In FIG. 15, the processes from step S1 to step S12 are the same as those of the first embodiment shown in FIG. 7 described above, and thus the illustration is omitted as appropriate.

As shown in FIG. 15, in the smartphone 10 of the third embodiment, the front eye portion image 35 and the reflection image 36 are detected by the above-mentioned image detection unit 56 from the start of the main shooting to the end of the main shooting (step). S12), the detection of the illuminance distribution by the illuminance distribution detection unit 76 (step S14A), and the display control by the display control unit 50 (step S14B) are repeatedly executed.

As a result, in the smartphone 10 of the third embodiment, the illuminance distribution of the reflected image 36 in the acquired image of the anterior eye portion Ef can be made uniform. Further, by repeatedly executing the processes from step S12 to step S14B during the main shooting, even if the user moves the face or the smartphone 10 during the main shooting, the illuminance distribution of the reflected image 36 is made uniform. The state of being is maintained.

[Smartphone of the fourth embodiment]
FIG. 16 is a front view of the smartphone 10 of the fourth embodiment. In the smartphone 10 of each of the above embodiments, the user is made to gaze at the camera unit 20 at the time of the tear film examination. On the other hand, as shown in FIG. 16, in the smartphone 10 of the fourth embodiment, the inspection control unit 62 controls the display control unit 50 from before the alignment detection to the end of the main shooting, and the display panel is displayed. Display the fixative 78 on 22. This allows the user to fix the fixative 78 during the examination of the tear film.

The smartphone 10 of the fourth embodiment has basically the same configuration as the smartphone 10 of each of the above-described embodiments, except that the fixative target 78 is displayed on the display panel 22. Therefore, those having the same function or configuration as each of the above embodiments are designated by the same reference numerals and the description thereof will be omitted.

At this time, the display control unit 50 of the fourth embodiment displays the fixative target 78 to be displayed and output on the display panel 22 based on the alignment detection result (alignment instruction by the instruction determination unit 60) by the alignment detection unit 58. Change the aspect (color or shape, etc.). In this case, the display panel 22 functions as an instruction unit of the present invention.

For example, the display control unit 50 displays the green fixative 78 on the display panel 22 when the camera unit 20 is aligned with the eye E to be inspected. Further, the display control unit 50 displays a blue fixative 78 on the display panel 22 when the camera unit 20 is deviated from the alignment position in any one direction, and the camera unit 20 is unidirectional from the alignment position. When it is deviated to the other direction opposite to the side, the red fixative 78 is displayed on the display panel 22. Instead of displaying the green, blue, and red fixatives 78 on the display panel 22, dot-shaped, ring-shaped, and cross-shaped fixatives 78 may be displayed on the display panel 22. Further, a shape (arrow or triangle) indicating a direction of deviation and a direction of movement may be displayed on the display panel 22 as a fixative 78.

As described above, in the fourth embodiment, the user can easily grasp the alignment state by changing the display mode of the fixative target 78 displayed on the display panel 22 according to the alignment detection result by the alignment detection unit 58. be able to. As a result, the user can be guided so that the camera unit 20 is aligned with the eye E to be inspected.

[Smartphone of the fifth embodiment]
FIG. 17 is a front view of the smartphone 10 of the fifth embodiment. As shown in FIG. 5, in the smartphone 10 of the fifth embodiment, except that the liquid crystal display 22A is used as the display panel 22 and the polarizing plate 80 is provided on the front surface of the camera unit 20, each of the above-described embodiments is performed. It has basically the same configuration as the smartphone 10 in the form. Therefore, those having the same function or configuration as each of the above embodiments are designated by the same reference numerals and the description thereof will be omitted.

As is known, the liquid crystal display 22A has a structure in which a liquid crystal panel in which a large number of liquid crystal elements are two-dimensionally arranged is sandwiched between a pair of polarizing filters. Therefore, the pattern light 28L projected from the liquid crystal display 22A toward the anterior segment Ef becomes linearly polarized light polarized in a specific polarization direction by passing through the polarization filter.

The camera unit 20 of the fifth embodiment takes an image of the anterior eye portion Ef through the polarizing plate 80. The polarizing plate 80 transmits only light in the polarization direction that is the same as (including substantially the same) in the polarization direction of the pattern light 28L. Here, since the specularly reflected light of the pattern light 28L specularly reflected on the corneal surface of the anterior segment Ef is specularly reflected on the corneal surface, its polarization state is not disrupted. Therefore, the specularly reflected light passes through the polarizing plate 80 and is incident on the camera unit 20.

On the other hand, since the reflection of the pattern light 28L by the iris of the eyelid and the anterior segment Ef becomes diffuse reflection, the polarized light of the pattern light 28L by the eyelid and the iris collapses. Therefore, this reflected light contains a polarizing component that cannot pass through the polarizing plate 80. As a result, a part of the reflected light of the pattern light 28L in the eyelids and the iris incident on the camera unit 20 can be dimmed by the polarizing plate 80, so that noise is generated due to the scattering component of the reflected light. It will be reduced. As a result, high-quality photographed image data 34 can be obtained, so that the accuracy of detecting the state of the tear film is improved.

[others]
The functions or configurations of the smartphone 10 described in each of the above embodiments may be combined as appropriate.

FIG. 18 is a block diagram showing a modified example of the control unit 46 of each of the above embodiments. In each of the above embodiments, even the state detection of the tear film is executed, but as shown in FIG. 18, for example, the control unit 46 may also function as the determination unit 82. The determination unit 82 compares the state detection result of the tear film detected by the state detection unit 64 with a predetermined diagnostic standard for dry eye, and the eye E to be inspected is in a state requiring an expert diagnosis. Judge whether or not. Then, the determination unit 82 outputs the determination result to the display control unit 50. As a result, the determination result by the determination unit 82 is displayed on the display panel 22 by the display control unit 50.

Further, when the determination unit 82 determines that the eye E to be inspected is in a state requiring expert diagnosis, the determination unit 82 obtains the photographed image data 34, the inspection result of the tear film, and the like after the user's approval operation. To send data to the server of a predetermined medical institution.

In each of the above embodiments, the display panel 22, that is, the display device has been described as an example of the light emitting unit of the present invention, but various light emitting units such as an illumination light source may be used. Further, in each of the above embodiments, white light has been described as an example of the pattern light 28L, but any color other than white light, flashing light (the color may change each time it flashes), or the like is arbitrary. It may be light.

In the above embodiment, the camera unit 20 (in-camera) has been described as an example of the photographing unit of the present invention, but the present invention is not particularly limited as long as it has an electronic photographing function.

In each of the above embodiments, for example, as shown in FIG. 7, analysis by the state detection unit 64 (condition detection of the tear film) is performed after the main shooting, but new shot image data 34 is shot in the main shooting. The analysis by the state detection unit 64 may be executed every time.

In each of the above embodiments, the case where the thickness of the tear layer and the shape and formation position of the dry spot are detected as the state of the tear layer has been described as an example, but for example, the tear layer destruction time and the like have been described. Other conditions of the tear film may be detected.

In each of the above embodiments, the smartphone 10 has been described as an example of the mobile terminal of the present invention, but a shooting unit such as a mobile phone, a tablet terminal, a portable personal computer, and a digital camera capable of self-shooting. The present invention can be applied to various mobile terminals having a light emitting unit (display device).

10 ... Smartphone,
12 ... Housing,
14 ... Display input unit,
16 ... Speaker,
20 ... Camera part,
22 ... Display panel,
22A ... Liquid crystal display,
24 ... Operation panel,
28 ... pattern,
28L ... Pattern light,
34 ... Photographed image data,
35 ... Anterior segment image,
36 ... Reflection image,
46 ... Control unit,
50 ... Display control unit,
52 ... Voice control unit,
56 ... Image detector,
58 ... Alignment detector,
60 ... Instruction decision unit,
62 ... Inspection control unit,
64 ... Status detector,
70 ... Projection state detector,
72 ... Projection control unit,
76 ... Illuminance distribution detector,
78 ... Fixative,
80 ... Polarizing plate

Claims (13)

With the housing
A light emitting unit provided on one surface side of the housing, which emits a predetermined pattern light and projects the pattern light onto the anterior eye portion of the eye to be inspected.
An imaging unit provided on one surface side of the housing and photographing the anterior eye portion on which the pattern light from the light emitting unit is projected.
An image detection unit that detects a reflected image of the pattern light from the captured image captured by the imaging unit, and an image detection unit.
A state detection unit that analyzes the reflection image detected by the image detection unit to detect the state of the tear film layer covering the corneal surface of the anterior eye portion, and a state detection unit.
Equipped with
The light emitting unit can adjust the light emitting mode of the pattern light.
The image detection unit detects the reflected image and the anterior eye portion image corresponding to the anterior eye portion from the captured image.
A projection state detection unit that detects the projection state of the pattern light on the front eye portion based on the detection result of the image detection unit, and a projection state detection unit.
A projection control unit that adjusts the light emission mode of the light emitting unit based on the detection result of the projection state detection unit to maintain the projection state of the pattern light in a specific state.
A mobile terminal equipped with. The projection state detecting unit detects at least the projection range of the pattern light projected on the anterior eye portion as the projection state.
The mobile terminal according to claim 1 , wherein the projection control unit adjusts at least the emission range of the pattern light as the adjustment of the emission mode. The light emitting unit is a display device that displays a pattern corresponding to the pattern light.
An illuminance distribution detection unit that detects the illuminance distribution of the reflected image detected by the image detection unit, and an illuminance distribution detection unit.
A display control unit that controls the display of the pattern by the display device based on the detection result of the illuminance distribution detection unit to make the illuminance distribution uniform.
The mobile terminal according to claim 1 or 2 . With the housing
A light emitting unit provided on one surface side of the housing, which emits a predetermined pattern light and projects the pattern light onto the anterior eye portion of the eye to be inspected.
An imaging unit provided on one surface side of the housing and photographing the anterior eye portion on which the pattern light from the light emitting unit is projected.
An image detection unit that detects a reflected image of the pattern light from the captured image captured by the photographing unit, and an image detecting unit.
A state detection unit that analyzes the reflection image detected by the image detection unit to detect the state of the tear film layer covering the corneal surface of the anterior eye portion, and a state detection unit.
Equipped with
The light emitting unit is a display device that displays a pattern corresponding to the pattern light.
An illuminance distribution detection unit that detects the illuminance distribution of the reflected image detected by the image detection unit, and an illuminance distribution detection unit.
A display control unit that controls the display of the pattern by the display device based on the detection result of the illuminance distribution detection unit to make the illuminance distribution uniform.
A mobile terminal equipped with. The mobile terminal according to any one of claims 1 to 4 , wherein the light emitting unit is a display device that displays a pattern corresponding to the pattern light and a fixative. With the housing
A light emitting unit provided on one surface side of the housing, which emits a predetermined pattern light and projects the pattern light onto the anterior eye portion of the eye to be inspected.
An imaging unit provided on one surface side of the housing and photographing the anterior eye portion on which the pattern light from the light emitting unit is projected.
An image detection unit that detects a reflected image of the pattern light from the captured image captured by the photographing unit, and an image detecting unit.
A state detection unit that analyzes the reflection image detected by the image detection unit to detect the state of the tear film layer covering the corneal surface of the anterior eye portion, and a state detection unit.
Equipped with
A mobile terminal in which the light emitting unit is a display device that displays a pattern corresponding to the pattern light and a fixative. The image detection unit detects an anterior eye portion image corresponding to the anterior eye portion from the captured image,
An alignment detection unit that detects the alignment state of the imaging unit with respect to the eye to be inspected based on the image of the anterior eye portion detected by the image detection unit.
A display control unit that controls the display device to change the display mode of the fixative based on the detection result of the alignment detection unit.
The mobile terminal according to claim 6. The light emitting portion is a liquid crystal display that projects the pattern light polarized in a specific polarization direction onto the front eye portion.
The mobile terminal according to any one of claims 1 to 7, wherein the photographing unit photographs the anterior eye portion through a polarizing plate that transmits light in the specific polarization direction. With the housing
A light emitting unit provided on one surface side of the housing, which emits a predetermined pattern light and projects the pattern light onto the anterior eye portion of the eye to be inspected.
An imaging unit provided on one surface side of the housing and photographing the anterior eye portion on which the pattern light from the light emitting unit is projected.
An image detection unit that detects a reflected image of the pattern light from the captured image captured by the photographing unit, and an image detecting unit.
A state detection unit that analyzes the reflection image detected by the image detection unit to detect the state of the tear film layer covering the corneal surface of the anterior eye portion, and a state detection unit.
Equipped with
The light emitting portion is a liquid crystal display that projects the pattern light polarized in a specific polarization direction onto the front eye portion.
A portable terminal in which the photographing unit photographs the anterior eye portion through a polarizing plate that transmits light in the specific polarization direction. 13 . The mobile terminal according to any one of the items. With the housing
A light emitting unit provided on one surface side of the housing, which emits a predetermined pattern light and projects the pattern light onto the anterior eye portion of the eye to be inspected.
An imaging unit provided on one surface side of the housing and photographing the anterior eye portion on which the pattern light from the light emitting unit is projected.
An image detection unit that detects a reflected image of the pattern light from the captured image captured by the photographing unit, and an image detecting unit.
A state detection unit that analyzes the reflection image detected by the image detection unit to detect the state of the tear film layer covering the corneal surface of the anterior eye portion, and a state detection unit.
Equipped with
A mobile terminal in which the state detection unit detects an interference image generated on the corneal surface from the reflection image, and detects the thickness of the tear film as the state of the tear film based on the interference image. The image detection unit detects an anterior eye portion image corresponding to the anterior eye portion from the captured image,
An alignment detection unit that detects the alignment state of the imaging unit with respect to the eye to be inspected based on the image of the anterior eye portion detected by the image detection unit.
An instruction unit that outputs an alignment instruction that is an index of relative alignment between the eye to be inspected and the imaging unit based on the detection result of the alignment detection unit.
The mobile terminal according to any one of claims 1 to 11 . A projection step in which a predetermined pattern light is emitted from a light emitting portion provided on one surface side of the housing and the pattern light is projected onto the anterior eye portion of the eye to be inspected.
A photographing step of photographing the anterior eye portion on which the pattern light from the light emitting portion is projected by the photographing unit provided on one surface side of the housing.
An image detection step of detecting a reflected image of the pattern light from the captured image captured in the imaging step,
A state detection step of analyzing the reflected image detected in the image detection step to detect the state of the tear film layer covering the corneal surface of the anterior eye portion, and a state detection step.
Have,
The light emitting unit can adjust the light emitting mode of the pattern light.
In the image detection step, the reflected image and the anterior eye portion image corresponding to the anterior eye portion are detected from the captured image.
A projection state detection step for detecting the projection state of the pattern light on the anterior eye portion based on the detection result of the image detection step, and a projection state detection step.
Based on the detection result of the projection state detection step, the projection control step of adjusting the light emission mode of the light emitting unit to maintain the projection state of the pattern light in a specific state,
A method of controlling a mobile terminal having .

Images