KR20210084965A - System and method that monitoring myopia patient according to usage environment of terminal - Google Patents

Abstract

The present invention provides a method and system for monitoring a myopic patient according to a use environment of a terminal, which can monitor the myopic patient by generating terminal use environment information according to the use environment when the user uses the terminal in the terminal. The method includes the steps of: detecting a screen on signal indicating an activated state of the display of the terminal; obtaining sensor data by sensing the use environment using a sensor during a screen on time in which the screen on signal is sensed; generating the terminal use environment information, which is information monitored by analyzing the terminal use environment of the user based on the acquired sensor data; sequentially accumulating and storing the generated terminal use environment information; transmitting the accumulated and stored terminal use environment information to a myopia monitoring server; and obtaining and displaying expected myopia progress information based on the constantly transmitted terminal use environment information from the myopia monitoring server.

Description

Myopia patient monitoring method and system according to the user environment of the terminal {SYSTEM AND METHOD THAT MONITORING MYOPIA PATIENT ACCORDING TO USAGE ENVIRONMENT OF TERMINAL}

The present invention relates to a method and system for monitoring a myopic patient according to a user environment of a terminal. More particularly, it relates to a monitoring method and system for managing a user's myopia by identifying a correlation between an environment in which a user uses a terminal and myopia.

In recent years, with the development of information and communication technology, the spread of mobile communication terminals has been made rapidly. Here, the mobile communication terminal is an intelligent terminal that includes a data communication function such as Internet communication or information search in a mobile phone, and is an intelligent terminal with various computer support functions added, and can provide various services desired by a user by using various applications. can

However, when the application is executed through such a mobile communication terminal, the user's gaze is fixed on the display screen of the mobile communication terminal, so that the user unknowingly and unconsciously stares at the display screen at a short distance for a long time continuously and repeatedly. and blinking will be drastically reduced. As such, when the mobile communication terminal and the user's eyes are kept at an appropriate distance (for example, 30 cm when the mobile terminal is a smart phone) for a long time, there is a problem that myopia (or glaucoma) may increase in the user. Recently, statistics on the prevalence of high myopia have been continuously published in East Asia, and in particular, in the case of high myopia, the risk of ophthalmic diseases such as retinal detachment, retinal tear, and subretinal neovascularization increases.

In addition, when the mobile communication terminal is used, the degree of difference between the ambient brightness (exposure illuminance) of the mobile communication terminal and the brightness output from the terminal display (display illuminance) or the degree of movement of the mobile communication terminal, that is, how shaken the mobile communication terminal is The visual acuity of a user using a mobile communication terminal is greatly affected according to various usage environments such as whether it is used in a mobile communication terminal. However, in the related art, there is insufficient technology for considering various usage environments of a mobile communication terminal that can affect a user's eyesight in various ways.

In addition, due to excessive use of the eyes due to the use of a mobile communication terminal, eye fatigue may increase as well as dry eyes, and if the dryness of the eyes increases, there is a problem in that the visual field is blurred and the visual acuity is greatly deteriorated.

In addition, in recent years, the age group who uses display-oriented applications through mobile communication terminals is gradually getting younger, and even infants or children with weak self-control are experiencing deterioration of eyesight, so it is urgent to introduce a technology to solve this problem. .

However, the conventional method of simply stopping or limiting the operation according to the usage time of the mobile communication terminal is not very helpful in reducing the deterioration of eyesight.

On the other hand, even under the same viewing conditions, each person feels differently according to their experiences. For example, a person may be able to easily read displayed text under certain conditions so that the eyes may not be too tired, and vision problems such as the onset of myopia may not be caused. On the other hand, others may struggle to read the same marked text under the same conditions as above, which may increase the likelihood of developing vision problems such as myopia. However, in the conventional system, these individual factors are not considered, and development of a technology for this is required.

In detail, although the use of devices such as mobile communication terminals is caused as a cause of vision problems due to the induction of myopia (or glaucoma), precisely in any usage environment (eg, low light, close reading distance, fixed focus for a long time, Since there is insufficient technology to prove the correlation between individual users' myopia (or glaucoma) becoming serious in excessive blue light, excessive display shake, etc., prevention of myopia (or glaucoma) is carried out by considering individual factors of each user. There is a need to introduce technology that can achieve this.

10-2015-0098542 A

The present invention has been devised to solve the above problems, and provides a monitoring method and system for managing the user's myopia by monitoring the environment in which the user uses the terminal and understanding the correlation between the user's terminal use environment and myopia but it has a purpose.

In detail, an object of the present invention is to provide a method and system for monitoring a myopia patient based on the user's terminal use environment, which provides base data for determining the correlation between the user's terminal use environment and myopia.

In addition, the present invention finds the cause of the user's myopia based on the base data obtained by monitoring the environment in which the user uses the terminal, and uses the terminal to provide customized myopia prevention education and prescription for each user based on this An object of the present invention is to provide a method and system for monitoring a myopia patient based on the environment.

However, the technical problems to be achieved by the present invention and embodiments of the present invention are not limited to the technical problems as described above, and other technical problems may exist.

A method and system for monitoring a myopic patient according to a usage environment of a terminal according to an embodiment of the present invention is a method for monitoring a myopic patient by generating terminal usage environment information according to a usage environment when a user uses the terminal in the terminal, detecting a screen on signal indicating an activated state of the display of the ; acquiring sensor data by sensing the use environment using a sensor during a screen on time in which the screen on signal is sensed; generating the terminal use environment information, which is information monitored by analyzing the terminal use environment of the user based on the acquired sensor data; sequentially accumulating and storing the generated terminal use environment information; transmitting the accumulated and stored terminal use environment information to a myopia monitoring server; and displaying, from the myopia monitoring server, expected myopia progress information based on the constantly transmitted terminal usage environment information. includes

In this case, the terminal usage environment information includes at least one of terminal usage distance information generated based on the sensor data, usage time information, ambient illuminance information, motion information, focus fixation time information, and blue light amount information. .

In addition, the acquiring of the sensor data includes sensor data measuring the distance between the terminal and the user, sensor data measuring the movement of the terminal, sensor data measuring the ambient brightness value of the terminal, and the terminal and acquiring at least one sensor data from sensor data measuring the amount of blue light output from the display, sensor data measuring whether or not the user's focus is fixed on the terminal, or measuring a fixed time.

In addition, the generating of the terminal usage environment information includes obtaining the terminal usage distance information using a camera, and the obtaining of the terminal usage distance information includes: obtaining grid-distance information; obtaining a photographed image from the camera; detecting a user's face area, which is an area classified as the user's face, based on the acquired captured image; obtaining grid occupancy rate information for determining an area occupied by the detected user face region on a grid preset in the camera; and obtaining the terminal usage distance information according to the grid-distance information based on the obtained grid occupancy rate information, wherein the grid-distance information includes: ) is information on the distance between the specific object and the terminal matched according to the ratio of the area occupying it.

In addition, the predicted myopia progress information is determined by the myopia monitoring server using a deep learning neural network to determine a correlation between each parameter of the terminal use environment information and the user's myopia, and based on the determined correlation It is an index indicating worsening or improvement of the user's myopia.

On the other hand, the myopia patient monitoring method and system according to the terminal use environment according to an embodiment of the present invention, the sensor unit for acquiring sensor data by sensing the user's terminal use environment, and the user based on the acquired sensor data a control unit for generating terminal use environment information, which is information monitored by analyzing the terminal use environment of , and transmits the generated terminal use environment information to a myopia monitoring server, and the terminal use environment information and the user's myopia from the myopia monitoring server A terminal comprising: a communication unit for receiving expected myopia progress information, which is information for determining a correlation between the two; and a display unit for outputting an output based on the received predicted myopia progress information; and a data receiver for receiving the terminal use environment information, a data transmitter for transmitting the expected myopia progress information, and a deep learning neural network based on the received terminal use environment information to generate the expected myopia progress information Data processing unit; includes a myopia monitoring server comprising a.

A method and system for monitoring a myopia patient based on a terminal use environment according to an embodiment of the present invention, by identifying a correlation between a user's terminal use environment and myopia, and managing the user's myopia based on this, the individual factors of the user It is possible to find the cause of myopia by taking this into account, and there is an effect of providing customized myopia prevention education and prescription for each user based on this.

In addition, the method and system for monitoring a myopic patient based on a terminal use environment according to an embodiment of the present invention obtains and provides base data for understanding the correlation between the user's terminal use environment and myopia in various ways, so that the user's There is an effect that it is possible to grasp the correlation between the terminal use environment and myopia more delicately.

In addition, the myopia patient monitoring method and system based on the terminal use environment according to an embodiment of the present invention provides information that identifies the correlation between the user's terminal use environment and myopia, thereby providing general statistics related to myopia or statistics for each user. There is an effect that can be derived more accurately.

In addition, the myopia patient monitoring method and system based on the terminal usage environment according to the embodiment of the present invention, the distance between the user's face and the terminal when the user uses the terminal, the user's face area detected from the photographed image obtained from the camera By providing the base, it is possible to more accurately determine the terminal use distance information included in the user's terminal use environment.

However, the effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood from the description below.

1 is a conceptual diagram of a method and system for monitoring a myopic patient based on a user environment of a terminal according to an embodiment of the present invention.
2 is an internal block diagram of a terminal according to an embodiment of the present invention.
3 is an internal block diagram of a myopia monitoring server according to an embodiment of the present invention.
4 is a flowchart illustrating a method of providing a myopia patient monitoring service based on a user environment of a terminal in a terminal according to an embodiment of the present invention.
5 is a flowchart illustrating a method of obtaining a distance between a user and a terminal based on a captured image obtained from a camera according to an embodiment of the present invention.
6 is a diagram for explaining a method of obtaining a distance between a user and a terminal based on a captured image obtained from a camera according to an embodiment of the present invention.
7 is an example of detecting a user's face region from a photographed image obtained from a camera according to an embodiment of the present invention.
8 is an example of deriving a grid occupation ratio of a user's face area detected from a captured image obtained from a camera according to an embodiment of the present invention.
9 is a flowchart illustrating a method of providing a myopia patient monitoring service based on a user environment of a terminal in a myopia monitoring server according to an embodiment of the present invention.
10 is a diagram for explaining one block of a neural network structure consisting of a plurality of blocks implementing a deep learning neural network according to an embodiment of the present invention.

Since the present invention can apply various transformations and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. Effects and features of the present invention, and a method of achieving them, will become apparent with reference to the embodiments described below in detail in conjunction with the drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various forms. In the following embodiments, terms such as first, second, etc. are used for the purpose of distinguishing one component from another, not in a limiting sense. Also, the singular expression includes the plural expression unless the context clearly dictates otherwise. In addition, terms such as include or have means that the features or components described in the specification are present, and do not preclude the possibility that one or more other features or components will be added. In addition, in the drawings, the size of the components may be exaggerated or reduced for convenience of description. For example, since the size and thickness of each component shown in the drawings are arbitrarily indicated for convenience of description, the present invention is not necessarily limited to the illustrated bar.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, and when described with reference to the drawings, the same or corresponding components are given the same reference numerals, and the overlapping description thereof will be omitted. .

1 is a conceptual diagram of a method and system for monitoring a myopic patient based on a user environment of a terminal according to an embodiment of the present invention.

Referring to FIG. 1 , a method and system for monitoring a myopia patient based on a user environment of a terminal according to an embodiment of the present invention may include a terminal 100 and a myopia monitoring server 200 .

Here, each component of FIG. 1 may be connected through a network. The network means a connection structure capable of exchanging information between each node, such as the terminal 100 and the myopia monitoring server 200, and an example of such a network is a 3rd Generation Partnership Project (3GPP) network, a Long Term (LTE) network, and the like. Evolution) network, WIMAX (World Interoperability for Microwave Access) network, Internet (Internet), LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), Bluetooth (Bluetooth) network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, and the like are included, but are not limited thereto.

- terminal

First, the terminal 100 according to an embodiment of the present invention may perform a series of operations for providing a myopia patient monitoring service based on the user's terminal 100 usage environment.

In detail, the terminal 100 may first determine whether the screen of the terminal 100 is on/off. And when the screen is in an on state, the terminal 100 may acquire sensor data, which is information sensing a usage environment when the user uses the terminal 100 .

In detail, the terminal 100 may acquire sensor data that is basic data for generating terminal use environment information to be described later by using a plurality of sensors.

Also, the terminal 100 may acquire information about the user's terminal use environment based on the acquired sensor data.

Here, the terminal usage environment information is based data for detecting factors affecting myopia by deriving a correlation between the usage environment of the terminal 100 and myopia, and sensing the usage environment when the user uses the terminal 100 . It is generated based on sensor data, which is one piece of information, and can be used as input data for a deep learning neural network later. A detailed description thereof will be provided later.

Also, the terminal 100 may sequentially accumulate and store the obtained terminal use environment information according to time. In addition, the terminal 100 may transmit terminal use environment information accumulated and stored for a predetermined period to the myopia monitoring server 200 .

Thereafter, the terminal 100 that has transmitted the terminal usage environment information to the myopia monitoring server 200 may receive and output expected myopia progress information based on the terminal usage environment information from the myopia monitoring server 200 .

Here, the predicted myopia progress information is information obtained by the myopia monitoring server 200 performing deep learning using the terminal use environment information, and the terminal use environment derived by inputting the user's terminal use environment information into the deep learning neural network. It may be information for determining deterioration or improvement information of the user's myopia based on the correlation between the user's myopia and the myopia. For example, the worsening or improvement information of the user's myopia may include aggravation or improvement degree of the user's myopia, information on a condition and/or an influence factor, and the like. A detailed description thereof will be provided later.

On the other hand, such a terminal 100 is a portable terminal 100 installed with a program for performing a myopia patient monitoring service according to the usage environment of the terminal 100, a smart phone, a digital broadcasting terminal 100 device, a mobile phone, a PDA ( It may include personal digital assistants, portable multimedia players (PMPs), navigation devices, tablet PCs, wearable devices, and smart glasses.

Hereinafter, each component constituting the terminal 100 will be described in detail with reference to the accompanying drawings.

2 is an internal block diagram of the terminal 100 according to an embodiment of the present invention.

Referring to FIG. 2 , the terminal 100 includes a communication unit 110 , an input unit 120 , a display unit 130 , a touch screen 135 , a sensor unit 140 , a camera 150 , and a storage unit. 160 and a control unit 170 may be included.

First, the communication unit 110 may transmit and receive various data and/or information for providing a myopic patient monitoring service according to the usage environment of the terminal 100 .

In an embodiment, the communication unit 110 may communicate with the myopia monitoring server 200 to transmit/receive data related to the myopia patient monitoring service (eg, terminal use environment information, etc.).

This communication unit 110, the technical standards or communication methods for mobile communication (eg, Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), High Speed Downlink Packet Access (HSDPA), HSUPA ( High Speed Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), etc.) Signals can be sent and received.

Next, the input unit 120 may detect a user input related to the myopia patient monitoring service according to the usage environment of the terminal 100 .

In an embodiment, the input unit 120 may include a device on/off button to detect an input for on/off of the device, that is, an on/off input for the screen. .

Next, the display unit 130 may output various information related to the myopia patient monitoring service according to the usage environment of the terminal 100 as graphic images.

The display unit 130 includes a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display. (a flexible display), a three-dimensional display (3D display), and may include at least one of an e-ink display (e-ink display).

In addition, the input unit 120 and the display unit 130 may be combined to be implemented as a touch screen 135 .

Next, the sensor unit 140 may acquire basic data for providing a myopia patient monitoring service according to the usage environment of the terminal 100 .

In detail, the sensor unit 140 may acquire sensor data, which is basic data for generating the user's terminal use environment information.

In an embodiment, the sensor unit 140 may include a motion measuring sensor, an illuminance (light) sensor, a blue light measuring sensor, a distance measuring sensor, and/or an iris recognition sensor.

In more detail, first, the motion measurement sensor may include an acceleration sensor, a gyroscope sensor, and/or a gravity sensor (G-sensor), and basic data obtained by measuring the movement of the terminal 100 . can be obtained.

Also, the illuminance (light) sensor may sense a brightness value around the terminal 100 to obtain basic data obtained by measuring the ambient illuminance of the terminal 100 .

In addition, the blue light measuring sensor may include a tristimulus value color sensor and the like, and may acquire basic data obtained by measuring the amount of blue light output from the display of the terminal 100 .

In addition, the distance measuring sensor may include a proximity sensor, an infrared sensor, and/or a laser sensor, and may acquire basic data obtained by measuring the distance between the terminal 100 and the user.

Also, the iris recognition sensor may sense the user's iris to obtain basic data obtained by measuring whether or not the user's focus is fixed and/or fixed time.

Next, in the embodiment, the camera 150 may acquire a photographed image for providing a myopia patient monitoring service according to the usage environment of the terminal 100 .

In detail, the camera 150 may be disposed on the front and/or rear of the terminal 100 and may acquire an image by photographing the disposed direction side. In this case, in particular, in an embodiment of the present invention, the camera 150 may acquire a photographed image of a user using the terminal 100 . In addition, the terminal 100 that has acquired the photographed image may include the acquired photographed image in sensor data to use it when generating terminal use environment information later.

Also, the camera 150 may include an image sensor and an image processing module.

Specifically, the camera 150 may process a still image or a moving image obtained by an image sensor (eg, CMOS or CCD).

In addition, the camera 150 may process a still image or a moving image obtained through an image sensor using an image processing module to extract necessary information, and transmit the extracted information to the controller 170 .

Next, the storage unit 160 may store any one or more of various application programs, data, and commands for providing a myopia patient monitoring service according to the usage environment of the terminal 100 according to an embodiment of the present invention.

In an embodiment, the storage unit 160 may store and manage sensor data, terminal use environment information, and/or grid-distance information, and the like.

The storage unit 160 may be various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, and the like, and web storage that performs the storage function of the storage unit 160 on the Internet (internet). ) may be

Finally, the controller 170 may control the overall operation of each of the above-described components in order to provide a monitoring service for the myopia patient according to the usage environment of the terminal 100 .

In this case, the controller 170 may perform image deep learning in conjunction with a deep running neural network.

Here, according to an embodiment, the deep learning neural network may be directly installed in the terminal 100 or may perform deep learning by receiving data as a device separate from the terminal 100 .

Hereinafter, in an embodiment of the present invention, an embodiment in which a deep learning neural network is directly installed in the terminal 100 to perform image deep learning will be described.

In detail, the control unit 170 may read the deep learning neural network driving program from the storage unit 160 and may perform image deep learning according to the deep learning neural network system built in the read driving program.

In an embodiment, the controller 170 may input a captured image obtained through the camera 150 as input data of a deep learning neural network built in a driving program, and a user on the captured image through image deep learning based on this A face region can be acquired as output data.

These control units 170, ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers (controllers), It may be implemented using at least one of micro-controllers, microprocessors, and other electrical units for performing functions.

- Myopia monitoring server

On the other hand, the myopia monitoring server 200 according to an embodiment of the present invention may perform deep learning based on terminal usage environment information for a myopia patient monitoring service according to the usage environment of the terminal 100, and based on this, the user It is possible to provide expected myopia progression information, which is information for determining deterioration or improvement information of myopia based on the correlation between the use environment of the terminal 100 and myopia.

In detail, the myopia monitoring server 200 may obtain a myopia test result for the user from the terminal 100 , an external device, and/or a server. Here, the myopia test result value may be result information on the myopia test performed by the user.

Also, the myopia monitoring server 200 may obtain terminal use environment information from the terminal 100 .

In addition, the myopia monitoring server 200 may determine the correlation between the terminal 100 using environment and myopia by performing deep learning based on the obtained myopia test result value and the terminal use environment information.

For example, the myopia monitoring server 200 uses a deep learning neural network to generate a table for determining a weight value that affects the progress of myopia for each parameter of the terminal use environment information, and the terminal 100 use environment and myopia correlation between them can be shown. Alternatively, the myopia monitoring server 200 uses a deep learning neural network to use the terminal 100 as a correlation value that numerically indicates the degree of influence on the progress of myopia for each parameter of the terminal use environment information according to a predetermined standard. It can show the correlation between the environment and myopia.

In addition, the myopia monitoring server 200, based on the correlation between the terminal 100 use environment and myopia determined through deep learning, provides information on worsening or improving the user's myopia according to the user's terminal 100 use environment It is possible to generate expected myopia progression information.

3 is an internal block diagram of the myopia monitoring server 200 according to an embodiment of the present invention.

In addition, referring to FIG. 3 , the myopia monitoring server 200 may include a data receiver 210 , a data transmitter 220 , a data processor 230 , and a database 240 .

First, the data receiving unit 210 and the data transmitting unit 220 transmit and receive various data for performing a myopia patient monitoring service according to the usage environment of the terminal 100 and/or an external server and the terminal 100 through a network. can

In addition, the data processing unit 230 may perform a series of data processing for performing a myopia patient monitoring service according to the usage environment of the terminal 100 .

In this case, the data processing unit 230 may perform deep learning in conjunction with a deep running neural network.

Here, according to an embodiment, the deep learning neural network may be directly installed in the myopia monitoring server 200 or may perform deep learning by receiving data as a device separate from the myopia monitoring server 200 .

Hereinafter, in an embodiment of the present invention, a deep learning neural network is directly installed in the myopia monitoring server 200 to perform deep learning.

In detail, the data processing unit 230 may read a deep learning neural network driving program from the database 240 and may perform deep learning according to a deep learning neural network system built on the read driving program.

In an embodiment, the data processing unit 230 may input the myopia test result value and terminal use environment information obtained through the data receiving unit 210 as input data of the deep learning neural network built in the driving program, and based on this Through deep learning, it is possible to determine the correlation between the user's terminal 100 usage environment and myopia, and based on this, the expected myopia progress information, which is an index indicating worsening or improvement information of myopia according to the user's terminal usage environment, is output data can be obtained with

In addition, these data processing unit 230, ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controller ( controllers), micro-controllers, microprocessors, and other electrical units for performing functions.

In addition, the database 240 may store various data related to the myopia patient monitoring service according to the usage environment of the terminal 100 .

In an embodiment, the database 240 may store a deep learning neural network driving program, and may provide it at the request of the data processing unit 230 .

The database 240 may be various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, and the like, and may be a web storage that performs a storage function of the database 240 on the Internet. may be

- Monitoring method for myopic patients according to the terminal usage environment

Hereinafter, a method of monitoring a myopic patient based on the usage environment of the terminal 100 will be described in detail with reference to the accompanying drawings.

<Terminal>

4 is a flowchart illustrating a method of providing a myopia patient monitoring service based on a usage environment of the terminal 100 in the terminal 100 according to an embodiment of the present invention.

Referring to FIG. 4 , in order to provide a myopia patient monitoring service based on the usage environment of the terminal 100 , the terminal 100 may first detect a screen on signal. (S101)

In detail, the terminal 100 may basically detect a screen off signal through the device on/off button of the input unit 120 or the sensor unit 140 .

In detail, the terminal 100, the device is in an off state by the user's input to the device on/off button of the input unit 120, or the movement of the device obtained through the motion measurement sensor of the terminal 100, or When the vibration is not detected, a screen-off signal indicating a situation in which the screen is turned off may be detected.

In this case, the terminal 100 may acquire standby time measurement data when a screen off signal is detected.

Here, the standby time measurement data refers to an off time based on information obtained through the device on/off button of the terminal 100 or the sensor unit 140 , that is, the non-use time of the terminal 100 is measured. It can be data.

And the terminal 100 may utilize the obtained standby time measurement data as basic data for generating terminal use environment information later.

On the other hand, in the terminal 100 , the device (ie, the terminal 100 ) operates through the device on/off button of the input unit 120 or the sensor unit 140 to display the display unit 130 of the terminal 100 . A screen on signal indicating a situation in which the screen in the active state is turned on may be detected.

In detail, the terminal 100 is the device in an on state by the user's input to the device on/off button of the input unit 120, or the movement of the device obtained through the motion measurement sensor of the terminal 100, or When vibration is detected, a screen-on signal indicating a screen-on state may be detected.

In this case, the terminal 100 detecting the screen-on signal may stop detecting the standby time measurement data and acquire sensor data during the screen on time. (S103)

Here, the sensor data is data obtained through a plurality of sensors for information on the usage environment when the user uses the terminal 100, and may be data used as basic data for generating the user's terminal usage environment information. have.

In detail, the terminal 100 may acquire sensor data during the screen on time through a plurality of sensors of the sensor unit 140 .

In more detail, the terminal 100 may acquire sensor data obtained by measuring the movement and/or vibration of the terminal 100 through a motion measurement sensor.

Also, the terminal 100 may obtain sensor data obtained by measuring the ambient illuminance of the terminal 100 by sensing a brightness value around the terminal 100 through an illuminance (light) sensor.

Also, the terminal 100 may obtain sensor data obtained by measuring the amount of blue light output from the display of the terminal 100 through the blue light measurement sensor.

Also, the terminal 100 may acquire sensor data obtained by measuring the distance between the terminal 100 and the user through a distance measuring sensor. For example, the terminal 100 may detect the approach of an object in a non-contact manner through the proximity sensor of the distance measuring sensor. In addition, the terminal 100 may detect the distance to the object by emitting infrared (IR) through the infrared sensor of the distance measuring sensor and measuring the emergency time (projection and return time). In addition, the terminal 100 may detect the distance to the object by emitting (second) sound waves through the laser sensor of the distance measuring sensor and measuring the emergency time (projection and return time). In this case, in the embodiment, the infrared sensor and/or the laser sensor may be implemented in a time of flight (TOF) method and interlock with the TOF camera 150 .

Also, the terminal 100 may sense the user's iris through the iris recognition sensor to obtain sensor data measuring whether the user's focus is fixed and/or fixed time.

In this way, the terminal 100 uses a plurality of sensors for sensing data specialized for each of the various terminal 100 usage environments (eg, usage time, ambient illuminance, distance between the terminal 100 and the user, movement, and focus fixation). time, amount of blue light, etc.) can be obtained, and through this, the user's terminal 100 usage environment can be analyzed from various angles.

Next, the terminal 100 having acquired the sensor data as described above may generate terminal use environment information based on the acquired sensor data. (S105)

In detail, the terminal 100 may analyze the user's terminal 100 usage environment based on the sensor data and generate terminal usage environment information that is monitored information.

In an embodiment, the terminal usage environment information may include at least one of usage time information, ambient illuminance information, motion information, focus fixation time information, blue light amount information, and terminal usage distance information for the terminal 100 .

In more detail, the terminal 100 may obtain information on the usage time of the terminal 100 , which is information obtained by measuring the time the user uses the terminal 100 by measuring the screen on time.

In this case, the terminal 100 may measure the screen on time through the device on/off button of the input unit 120 and/or the sensor unit 140 . For example, the terminal 100 may derive the screen on time by calculating a difference value between the detection time of the screen on signal detected through the device on/off button and the detection time of the screen off signal. In another example, the terminal 100, based on the standby time measurement data obtained through the motion measurement sensor of the sensor unit 140, a predetermined reference time (eg, 24 hours) and the standby time calculated as the standby time measurement data The screen-on time may be derived by calculating a difference value between them.

Also, the terminal 100 may determine the user's fixed time of the screen-on time as the usage time information of the terminal 100 to more accurately determine the user's usage time of the terminal 100 .

In this way, the terminal 100 monitors the user's terminal 100 usage time information and manages the user's myopia based thereon, so that myopia based on the correlation between the time the user uses the terminal 100 and myopia Patient monitoring services can be provided effectively.

In addition, the terminal 100 may acquire peripheral illuminance information of the terminal 100 , which is information obtained by measuring ambient brightness when a screen is on.

In this case, the terminal 100 may generate ambient illuminance information by acquiring a brightness value around the terminal 100 from the illuminance (light) sensor of the sensor unit 140 .

As such, the terminal 100 monitors the peripheral illuminance information and manages the user's myopia based on this, so that the myopia patient based on the correlation between the degree of ambient brightness and the myopia when the user uses the terminal 100 Monitoring services can be provided effectively.

Also, the terminal 100 may acquire motion information of the terminal 100 , which is information obtained by measuring the movement and/or vibration of the terminal 100 .

In this case, the terminal 100 may generate motion information by detecting continuous motion and/or vibration of the terminal 100 by the user based on the sensor data of the motion measurement sensor of the sensor unit 140 .

As such, the terminal 100 may monitor the movement information of the terminal 100 to determine whether the user is using the terminal 100 in an environment in which the user moves or shakes, and based on this, manages the user's myopia to the terminal 100 ) can effectively provide a monitoring service for myopia patients based on the correlation between the degree of shaking and myopia.

In addition, the terminal 100 may acquire focus fixation time information, which is information obtained by measuring the focus fixation time with respect to the user's focus fixed on the display of the terminal 100 for a long time.

At this time, the terminal 100 may detect that the user is looking at the screen based on the sensor data of the iris recognition sensor of the sensor unit 140 , and when the user's focus is fixed on the screen for a long time, the corresponding time By measuring, it is possible to generate focus fixation time information.

In this way, the terminal 100 monitors the user's focus fixation time information for the terminal 100 and manages the user's myopia based on this, so that the fixed time of the focus on the display of the terminal 100 and the myopia and the nearsightedness It is possible to effectively provide monitoring services for myopia patients based on the correlation of

Also, the terminal 100 may obtain blue light amount information, which is information obtained by measuring the amount of blue light output from the display of the terminal 100 .

In this case, the terminal 100 may generate blue light spirit information by detecting the amount of blue light output from the screen of the terminal 100 based on the sensor data of the blue light measuring sensor of the sensor unit 140 .

In this way, the terminal 100 monitors the blue light amount information of the terminal 100 and manages the user's myopia based on this, based on the correlation between the degree of blue light output from the screen the user is watching and the myopia. It is possible to effectively provide monitoring services for myopia patients.

Also, the terminal 100 may acquire terminal usage distance information, which is information obtained by measuring the distance between the terminal 100 and the user's face, terminal usage distance information.

In detail, the terminal 100 may obtain the terminal usage distance information by using the sensor unit 140 and/or the camera 150 in at least one of a plurality of terminal usage distance information acquisition methods described below. have.

First, the terminal 100 may 1) acquire terminal usage distance information through the sensor unit 140 . In detail, the terminal 100 measures the emergency time (projection and return time) value based on at least one of a proximity sensor, an infrared sensor, and a laser sensor, which are distance measuring sensors of the sensor unit 140 . It is possible to obtain terminal usage distance information.

Also, the terminal 100 may acquire terminal usage distance information through the camera 150 . In detail, in the embodiment, the terminal 100 may obtain terminal usage distance information by using the camera 150 disposed on the front side of the terminal 100 that can easily identify the user who gazes at the display of the terminal 100 . have. That is, the control unit of the terminal 100 may detect use distance information that is the distance between the terminal 100 and the user through only a single camera even when there is no distance measuring sensor on the front of the terminal 100 .

In more detail, first, when the front camera 150 of the terminal 100 is a dual camera 150, the terminal 100 may 2) acquire terminal usage distance information in a stereo method. For example, the terminal 100 may include a user-side point obtained through the dual camera 150 (eg, an intermediate position between the detected user's eyes, etc.) and a focus point of each of the two cameras 150 (eg, each camera ( 150), it is possible to generate terminal usage distance information by performing triangular measurement based on the lens position, etc.).

However, in general, most of the cameras 150 disposed on the front side of the terminal 100 are implemented as a single camera 150 .

Therefore, the terminal 100, when the front camera 150 of the terminal 100 is a single camera 150 3) through an auto focus function on the image photographed by the camera 150, the photographed image terminal usage distance information can be generated based on the phase difference between objects of

In addition, when the front camera 150 of the terminal 100 is a single camera 150 4) the terminal 100 obtains an emergency time value from the auto-focus distance measurement sensor of the camera 150 to obtain terminal use distance information can create

In addition, the terminal 100, when the front camera 150 of the terminal 100 is a single camera 150, the corresponding camera 150 is a 3D camera 150 that detects a depth (depth) TOF (Time Of Flight) ), 5) an emergency time value through the TOF sensor of the TOF (Time Of Flight) camera 150 may be acquired to generate terminal usage distance information.

In addition, in particular, in the embodiment of the present invention, the terminal 100 is classified as a user from a photographed image obtained through 6) the camera 150 when the front camera 150 of the terminal 100 is a single camera 150 . The area may be detected, and terminal usage distance information may be obtained based on the sensed area.

5 and 6 are diagrams for explaining a method of obtaining a distance between a user and the terminal 100 based on a captured image obtained from the camera 150 according to an embodiment of the present invention.

5 and 6 , the terminal 100 according to an embodiment of the present invention may first obtain grid-distance information for the type of the front camera 150 of the terminal 100 . have. (S201)

In general, the size of a grid preset in the camera 150 may be different depending on the type (model) of the camera 150 . Therefore, in the embodiment of the present invention, the distance between the user and the terminal 100 is obtained in consideration of different grids for each type of camera 150 .

That is, in the embodiment, the grid-distance information may be distance information between a predetermined object and the terminal 100 according to a ratio of the grid area in which the object is sensed to the total grid area. At this time, since specifications such as the number and size of grids differ depending on the type (model) of the camera 150 disposed in the terminal 100, it is preferable that the grid-distance information is set for each camera 150 type. Grid-distance information for each type of camera 150 may be stored in the terminal 100 and/or the myopia monitoring server 200 .

For example, grid-distance information includes a total of 100 pixel grids when a grid matching the camera 150 type is implemented with 10 horizontal and 10 vertical straight lines as shown in (2) of FIG. 6 . The captured image obtained as shown in (1) of FIG. 6 is superimposed as shown in FIG. 6 (3), and the distance between the terminal 100 and the corresponding object is determined according to how much area the object of the corresponding captured image occupies on the grid. The distance may be preset information. That is, the distance between the object and the terminal 100 may be calculated in inverse proportion to the ratio of the number of grids occupied by the object of the photographed image to the total number of grids.

For example, in the grid-distance information, when a pixel grid of 0-20% or more of a total of 100 pixel grids is filled with a specific object, the distance between the terminal 100 and the specific object is at least 100cm, and a pixel grid of 20%-50% or more is specified. When the pixel grid is filled with an object, the distance between the terminal 100 and a specific object is at least 50 cm, and when the pixel grid is filled with a specific object, the distance between the terminal 100 and the specific object is at least 20 cm, 80% to 100% or more. When the pixel grid is filled with a specific object, it may be preset information for determining the distance between the terminal 100 and the specific object to be at least 0 cm.

In addition, the terminal 100 having obtained the grid-distance information may obtain a photographed image from the front camera 150 of the terminal 100 . (S203)

In detail, the terminal 100 may acquire a photographed image generated by photographing an external object by the front camera 150 of the terminal 100 as shown in FIG. 6 ( 1 ).

In addition, the terminal 100 may detect the user's face region based on the acquired captured image. (S205)

Here, the user's face area means an area representing the user's face on a photographed image.

In detail, the terminal 100 may detect the user's face region from the captured image through a deep-learning neural network that performs image deep-learning.

Here, the image deep learning neural network according to an embodiment of the present invention is a convolutional neural network (CNN), Regions with CNN features (R-CNN), Fast R-CNN, Faster R-CNN and Mask R. -CNN may be at least one or more.

However, since a convolutional neural network (CNN) is effective in performing semantic segmentation according to an embodiment, in the following description, an image deep learning neural network is used as a convolutional neural network (CNN). Network) will be limited to the description.

Referring to FIG. 7 , in the embodiment, the terminal 100 performs object detection for detecting the presence or absence of a specific object, that is, a user's face, by using the captured image first as input data of the deep learning neural network. can do.

In addition, the terminal 100 may perform object classification for classifying the detected user face into a user face region.

Also, the terminal 100 may determine the detected user face region as a meaningful region and perform object segmentation for detecting a boundary thereto. That is, the terminal 100 may detect the user's face region as an object of a meaningful region rather than a simple distinction based on a threshold or an edge.

Thus, the terminal 100 may detect the user's face region on the captured image through a deep learning neural network using the captured image as input data.

In this case, when there are a plurality of regions classified as user faces, that is, when multiple faces are detected, the terminal 100 may determine the face region occupying the largest area and determine the user face region based on the corresponding face region. have.

Here, the embodiment of the present invention describes that terminal usage distance information is obtained based on the distance between the terminal 100 and the user's face by limiting to the user's face area on the photographed image, but the entire area of the user detected on the photographed image ( For example, various embodiments are also possible, such as calculating the distance between the terminal 100 and the user based on the face and body, etc.) to obtain terminal use distance information.

Next, the terminal 100 that has detected the user's face area may acquire grid occupancy rate information for the detected user's face area. (S207)

In detail, referring to FIG. 8 , the terminal 100 overlaps the detected user face region with a grid preset for the corresponding camera 150 , and determines the area occupied by the user face region on the grid, the grid occupancy ratio. information can be obtained.

As such, the terminal 100 having obtained the grid occupancy ratio information may acquire terminal usage distance information according to the grid-distance information based on the obtained grid occupancy ratio information. (S209)

In detail, the terminal 100 may determine the distance between the user's face and the terminal 100 based on the obtained grid occupancy rate information and grid-distance information preset for the corresponding camera 150 .

For example, when it is determined that the user's face area occupies 80% of the area of the grid of the corresponding camera 150 through the grid occupancy ratio information, the terminal 100 determines the preset grid-distance information (eg, grid When 50% to 80% or more of is filled with a specific object, the terminal use distance information that determines the distance between the user's face and the terminal 100 as 20cm according to the information that determines the distance between the terminal 100 and the specific object at least 20cm) can be obtained

As such, the terminal 100 detects an area classified as a user's face from a photographed image obtained through the camera 150 through deep learning, and obtains terminal usage distance information based on the detected area, thereby obtaining a photographed image It is possible to measure the distance based on an object that is clearly determined to be a person among the objects on the face, and to more accurately determine the distance between the terminal 100 and the user's face to generate terminal usage distance information.

In addition, as described above, the terminal 100 monitors the terminal usage distance information of the terminal 100 and manages the user's myopia based on this, so that the user's face and the terminal 100 when the user uses the terminal 100 . It is possible to effectively provide a myopia patient monitoring service based on the distance between the eyes and the correlation between myopia.

Returning to FIG. 4 again, the terminal 100 that has generated at least one of usage time information, ambient illuminance information, motion information, focus fixation time information, blue light amount information, and terminal usage distance information based on the sensor data as above is, Terminal usage environment information including at least one or more of the generated information may be generated.

Next, the terminal 100 that has generated the terminal use environment information may sequentially accumulate and store the generated terminal use environment information. (S107)

In detail, the terminal 100 may sequentially accumulate at least one terminal use environment information generated for each screen on time of the terminal 100 according to the order of time and store it in the storage unit 160 . .

Also, the terminal 100 that sequentially accumulates and stores the terminal use environment information may transmit the terminal use environment information for a predetermined period to the myopia monitoring server 200 . (S109)

In detail, the terminal 100 may transmit, to the myopia monitoring server 200 , the terminal use environment information accumulated and stored for a predetermined period among at least one or more terminal use environment information accumulated and stored in chronological order.

Here, the predetermined period may be set through the terminal 100 and/or the myopia monitoring server 200, for example, the predetermined period is a period between the user's first myopia test time and the second myopia test time. can

In this way, the terminal 100 may provide the user's terminal use environment information accumulated and stored for a predetermined period, thereby effectively supplying evidence data for deriving a correlation between the user's terminal use environment information and the nearsightedness later.

Next, the terminal 100 that has transmitted the terminal use environment information for a predetermined period may receive and display expected myopia progress information based on the transmitted terminal use environment information from the myopia monitoring server 200 . (S111)

Here, the predicted myopia progress information is information obtained by the myopia monitoring server 200 performing deep learning using the terminal usage environment information, and the terminal 100 derived by inputting the user's terminal usage environment information into the deep learning neural network. ) may be an indicator of worsening or improvement of the user's myopia according to the correlation between the use environment and myopia.

In detail, the predicted myopia progress information includes each parameter of the terminal use environment information (eg, use time information parameter, ambient illuminance information parameter, motion information parameter, focus fixation time information parameter, blue light amount information parameter, or terminal use distance information parameter ) and the user's myopia may be information indicating the progress of the user's myopia obtained according to the correlation. A detailed description of this will be described in the detailed description of a method of providing a myopia patient monitoring service based on the usage environment of the terminal 100 in the server, which will be described later.

That is, the terminal 100, based on the expected myopia progress information received from the myopia monitoring server 200, provides the user with information on deterioration or improvement of the user's myopia according to the correlation between each parameter of the terminal use environment information and the nearsightedness. can

For example, the terminal 100 may preset the correlation between one parameter and myopia to be higher as it approaches step 10 based on steps 1 to 10. And the terminal 100, when the correlation value indicating the degree of correlation between one parameter and myopia is determined to be 8 through the expected myopia progress information received from the myopia monitoring server 200, the correlation between the corresponding one parameter and myopia can be provided to the user by displaying in 8 steps.

In this way, the terminal 100 provides a degree of correlation in which the environment in which the user uses the terminal 100 affects myopia based on the expected myopia progress information, so that according to the terminal use environment information measured for each user It is possible to provide customized prescription and myopia prevention education for each user, and data can be utilized in various ways, such as calculating general statistics related to myopia or calculating individual statistics.

In the above, the terminal 100 generates terminal use environment information based on sensor data, receives expected myopia progress information based on the generated terminal use environment information, and receives nearsightedness based on the correlation between terminal use environment information and myopia. Although the description is limited to the embodiment in which exacerbation or improvement information is provided, in another embodiment, the terminal 100 generates terminal use environment information based on sensor data, and predicted glaucoma progress information based on the generated terminal use environment information. An embodiment in which information on deterioration or improvement of glaucoma based on the correlation between terminal use environment information and glaucoma by receiving the glaucoma is also possible.

<Myopia Monitoring Server>

Meanwhile, FIG. 9 is a flowchart illustrating a method of providing a myopia patient monitoring service based on the usage environment of the terminal 100 in the myopia monitoring server 200 according to an embodiment of the present invention.

Referring to FIG. 9 , in order to provide a myopia patient monitoring service based on the usage environment of the terminal 100, the myopia monitoring server 200 first acquires the user's first myopia test result value and the second myopia test result value. can (S301)

Here, the myopia test result value may refer to result information on the myopia test performed by the user. That is, the myopia monitoring server 200 provides information on a first myopia test result value, which is result information on a first myopia test performed by the user at a first time point, and a second myopia test result value performed by the user at a second time point thereafter. A second myopia test result value, which is the result information, may be acquired.

In this case, the myopia monitoring server 200 may obtain first and second myopia test result values from a user input, an external device, and/or a server according to an embodiment.

Next, the myopia monitoring server 200 may acquire terminal usage environment information of the user (ie, the subject) from a first time point at which the first myopia test is performed to a second time point at which the second myopia test is performed. . (S303)

In detail, the myopia monitoring server 200 is configured for a first time point to a second time point period, which is a predetermined period, among at least one or more terminal use environment information accumulated and stored in chronological order from the terminal 100 to the terminal 100 . It is possible to receive terminal usage environment information that is accumulated and stored.

For example, the myopia monitoring server 200, the first terminal use environment information, the second terminal use environment information, . , it is possible to receive the nth terminal use environment information.

Subsequently, the myopia monitoring server 200 may generate a training data set based on the obtained first myopia test result value, the second myopia test result value, and terminal use environment information. (S305)

In detail, the myopia monitoring server 200 may generate a training data set with a plurality of learning data generated based on the first myopia test result value, the second myopia test result value, and the terminal use environment information.

For example, the training data set includes {the first terminal use environment information, the first myopia test result value, the second myopia test result value}, {the second terminal use environment information, the first myopia test result value, and the second myopia test result value. inspection result}, … , {the nth terminal usage environment information, the first myopia test result value, and the second myopia test result value}.

Next, the myopia monitoring server 200 may train a deep-learning neural network based on the generated training data set. (S307)

In detail, the myopia monitoring server 200 may train the deep learning neural network to derive a correlation between the user's terminal usage environment information and myopia based on the training data set.

In this case, in the embodiment, the myopia monitoring server 200 may learn a deep learning neural network for deriving a correlation between terminal use environment information and myopia based on a Resnet (resnet, residual net) structure. This is because, when the deep learning neural network adopts the LessNet structure, 1) the learning speed is improved, and 2) the learning effect according to the depth can be obtained while implementing a deep network (eg, a network of 100 layers or more). This is because advantages such as performance can be improved as the depth of the network is implemented, and 3) a deep network can be easily optimized, and the accuracy can be improved due to the increased depth of the network.

10 is a diagram for explaining one block of a neural network structure consisting of a plurality of blocks implementing a deep learning neural network according to an embodiment of the present invention.

For example, referring to FIG. 10 , a deep learning neural network based on the Resnet structure may be composed of 19 residual blocks.

At this time, one residual block includes 256 3X3 convolutional layers, batch normalization layers, Relu activation function layers, 256 3X3 convolutional layers, batch normalization layers, skip connections, and Relu activation function layers. may be arranged in order.

Here, the batch normalization layer is to prevent covariate shift, which is a phenomenon in which the distribution of the input of the current layer is changed due to the parameter change of the previous layer during learning.

In addition, skip connection prevents the performance of the neural network from decreasing even when the block layer becomes thick, and makes the block layer thicker to increase the overall neural network performance. In this case, the skip connection may be in a form in which the first input data of the residual block is combined with the output of the second batch normalization layer and input to the second Relu activation function layer.

Back again, the myopia monitoring server 200 uses a deep learning neural network based on the above Resnet as input data, and uses a terminal for the difference between the first myopia test result and the second myopia test result. By determining the degree of influence of each parameter of the environmental information, it is possible to learn to derive a correlation value with myopia for each parameter.

Here, the correlation value is information indicating the degree of correlation between one parameter of the terminal use environment information and the nearsightedness, and for example, may be derived as a larger numerical value as the correlation is higher, and a smaller numerical value as the correlation is lower.

For example, the deep learning neural network may perform element-by-element transformation of each parameter of the terminal use environment information to obtain a correlation value between each parameter and myopia, and may perform learning through this.

For example, the deep learning neural network uses time information, ambient illuminance information, motion information, focus fixation time information, blue light amount information, and terminal usage distance information parameters of the first to n terminal usage environment information of the training data set, one parameter transformation can be performed on In addition, the deep learning neural network measures the change in the difference between the first myopia test result and the second myopia test result when the one parameter is modified to obtain a correlation value between the corresponding one parameter and myopia. You can do it, and you can learn based on it.

In addition, the myopia monitoring server 200, by repeatedly performing the learning operation of the deep learning neural network as described above for a plurality of other training data sets, deep learning to derive a correlation between the user's terminal usage environment information and myopia Neural networks can be trained.

On the other hand, the myopia monitoring server 200 that performs deep learning according to the deep learning neural network system learned as described above uses the learned deep learning neural network to generate expected myopia progress information according to the user's terminal usage environment information. It can be transmitted to the terminal 100 . (S309)

Here, the predicted myopia progress information is information obtained by determining the correlation between each parameter of the terminal use environment information and the user's myopia by using a deep learning neural network, for example, use time information, peripheral illuminance information, motion information, focus fixation The time information, the blue light amount information, and/or the terminal usage distance information may be information providing information on worsening or improving myopia of the user based on a correlation value indicating a degree to which each parameter affects myopia.

For example, the myopia monitoring server 200 inputs a training data set based on the first user's first myopia test result value, the second myopia test result value, and the terminal use environment information to the learned deep learning neural network, 1 The correlation value between the usage time information of the user and myopia is 3, the correlation value between the first user's peripheral illuminance information and myopia is 4, the correlation value between the movement information of the first user and myopia is 5, and the first user's Output data derived from a correlation value between focus fixation time information and myopia of 6, a correlation value between blue light amount information of the first user and myopia 7, and a correlation value between terminal usage distance information and myopia of the first user 8 can be obtained.

And the myopia monitoring server 200, based on the output data obtained through the deep learning neural network, the use of the first user according to a preset criterion (eg, the higher the number, the greater the effect on myopia, etc.) The degree to which time information affects myopia is 3, the degree to which peripheral illuminance information of the first user affects myopia 4, the degree to which movement information of the first user affects myopia 5, and focus of the first user The degree of influence of fixed time information on myopia is 6, the degree of blue light intensity information of the first user affecting myopia is 7, and the degree of influence of terminal usage distance information of the first user is 8. Myopia progress information can be generated.

In addition, the myopia monitoring server 200 that has generated the expected work progress information may transmit the generated expected work progress information to the terminal 100 .

As such, the myopia monitoring server 200 derives a correlation based on deep learning that the user's terminal 100 usage environment affects the user's myopia, and predicts myopia progression information based on the derived correlation. By creating and providing it, it is possible to provide base data for customized prescription for each user and myopia prevention education, and it is possible to provide data that can be used in various ways, such as calculating general statistics related to myopia or calculating individual statistics. .

In the above, the myopia monitoring server 200 is the first myopia test result value, the second myopia test result value and the embodiment of generating expected myopia progression information based on the terminal use environment information, but in another embodiment, the myopia monitoring server 200 performs the first glaucoma test result value and the second glaucoma test result value and predicted glaucoma progress information may be generated and provided based on the terminal use environment information.

In addition, in the above description, it has been described that the myopia monitoring server 200 performs a series of operations for generating predicted myopia progress information using deep learning based on the myopia test result value and terminal use environment information. Various embodiments may be possible, such as performing a part of a series of operations in the terminal 100 .

As described above, the myopia patient monitoring method and system based on the usage environment of the terminal 100 according to an embodiment of the present invention manages the user's myopia based on the correlation between the user's terminal 100 usage environment and myopia By doing so, it is possible to find the cause of myopia in consideration of the user's individual factors, and there is an effect of providing customized myopia prevention education and prescription for each user based on this.

In addition, the myopia patient monitoring method and system based on the use environment of the terminal 100 according to an embodiment of the present invention obtains the base data for determining the correlation between the user's terminal 100 use environment and myopia in various ways By providing this, there is an effect that it is possible to grasp the correlation between the user's terminal 100 usage environment and myopia in a more subtle way.

In addition, the myopia patient monitoring method and system based on the usage environment of the terminal 100 according to an embodiment of the present invention provides information that identifies the correlation between the user's terminal 100 usage environment and myopia, thereby providing general information related to myopia. It has the effect of more accurately deriving statistics or statistics for each user.

In addition, the myopia patient monitoring method and system based on the use environment of the terminal 100 according to an embodiment of the present invention, when the user uses the terminal 100, the distance between the user's face and the terminal 100 is measured by the camera 150 By providing based on the user's face region detected from the photographed image obtained from

In addition, the embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention, or may be known and used by those skilled in the computer software field. Examples of the computer-readable recording medium include hard disks, magnetic media such as floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floppy disks. medium), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. A hardware device may be converted into one or more software modules to perform processing in accordance with the present invention, and vice versa.

The specific implementations described in the present invention are only examples, and do not limit the scope of the present invention in any way. For brevity of the specification, descriptions of conventional electronic components, control systems, software, and other functional aspects of the systems may be omitted. In addition, the connection or connection members of the lines between the components shown in the drawings exemplarily represent functional connections and/or physical or circuit connections, and in an actual device, various functional connections, physical connections that are replaceable or additional may be referred to as connections, or circuit connections. In addition, unless there is a specific reference such as “essential” or “important”, it may not be a necessary component for application of the present invention.

In addition, although the detailed description of the present invention has been described with reference to a preferred embodiment of the present invention, those skilled in the art or those having ordinary knowledge in the art will appreciate the spirit of the present invention described in the claims to be described later. And it will be understood that various modifications and variations of the present invention can be made without departing from the technical scope. Accordingly, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be defined by the claims.

Claims (6)

A method of monitoring a myopic patient by generating terminal usage environment information according to the usage environment when the user uses the terminal in the terminal,
detecting a screen on signal indicating an activated state of the display of the terminal;
acquiring sensor data by sensing the use environment using a sensor during a screen on time in which the screen on signal is sensed;
generating the terminal use environment information, which is information monitored by analyzing the terminal use environment of the user based on the acquired sensor data;
sequentially accumulating and storing the generated terminal use environment information;
transmitting the accumulated and stored terminal use environment information to a myopia monitoring server; and
obtaining and displaying expected myopia progress information based on the constantly transmitted terminal usage environment information from the myopia monitoring server; containing
A method for monitoring a myopic patient according to the user environment of the terminal. The method of claim 1,
The terminal use environment information,
Terminal use distance information generated based on the sensor data, use time information, ambient illuminance information, motion information, focus fixation time information, and blue light amount information including at least one or more information
A method for monitoring a myopic patient according to the user environment of the terminal. 3. The method of claim 2,
Acquiring the sensor data includes:
Sensor data measuring the distance between the terminal and the user, sensor data measuring the movement of the terminal, sensor data measuring the ambient brightness value of the terminal, and sensor data measuring the amount of blue light output from the display of the terminal , Acquiring at least one sensor data of whether or not the user's focus on the terminal is fixed or sensor data measured for a fixed time
A method for monitoring a myopic patient according to the user environment of the terminal. 3. The method of claim 2,
The step of generating the terminal use environment information,
Acquiring the terminal usage distance information using a camera,
The step of obtaining the terminal usage distance information includes:
obtaining grid-distance information for the type of camera;
obtaining a photographed image from the camera;
detecting a user's face area, which is an area classified as the user's face, based on the acquired captured image;
obtaining grid occupancy rate information for determining an area occupied by the detected user face region on a grid preset in the camera; and
Comprising the step of obtaining the terminal usage distance information according to the grid-distance information on the basis of the obtained grid occupancy ratio information,
The grid-distance information is
Information on the distance between the specific object and the terminal matched according to the ratio of the area occupied by the specific object to the grid with respect to the type of camera
A method for monitoring a myopic patient according to the user environment of the terminal. The method of claim 1,
The predicted myopia progression information is
The myopia monitoring server determines a correlation between each parameter of the terminal use environment information and the user's myopia by using a deep learning neural network, and indicates worsening or improvement of the user's myopia based on the determined correlation indicator
A method for monitoring a myopic patient according to the user environment of the terminal. A sensor unit for sensing the user's terminal usage environment to obtain sensor data, and a control unit for analyzing and monitoring the user's terminal usage environment based on the acquired sensor data and generating terminal usage environment information that is monitored information; A communication unit for transmitting terminal usage environment information to a myopia monitoring server and receiving expected myopia progression information that is information for determining a correlation between the terminal usage environment information and the user's myopia from the myopia monitoring server; a terminal including a display unit for performing information-based output; and
A data receiving unit for receiving the terminal usage environment information, a data transmitting unit for transmitting the expected myopia progress information, and data for generating the expected myopia progress information using a deep learning neural network based on the received terminal usage environment information Processing unit; including a myopia monitoring server comprising
Myopia patient monitoring system according to the user environment of the terminal.

Images