KR20230145629A - Lifelog caring system for a user with deeplearning algorithm and a method for contrlling the system - Google Patents

Abstract

The present invention provides an activity sensor 200-1 that detects at least activity in the user's living space, and at least water, gas and electricity usage and illumination conditions and the user's breathing rate and A monitoring sensor unit 200 including an environment sensor unit 200-4 that detects heart rate, monitoring life log data detected as at least activity from the monitoring sensor unit 200, and monitoring indicating the user's physical condition. A monitoring terminal 100 that collects monitoring data including bio log data, and the monitoring terminal 100 receives the monitoring data to include at least an activity index indicating whether the user is active, bedtime information indicating bedtime information, and external data. A monitoring management server 400 that determines the stability of the user's daily state by checking the going out index indicating activity information and the bio-index indicating the user's physical condition through the user's breathing rate and heart rate and transmits the judgment to at least the guardian terminal unit 500. Provides a deep learning user life status monitoring management device and a control method thereof, further comprising a monitoring module 4100 and a probability module 4700.

Description

Deep learning user life status monitoring management device and control method {LIFELOG CARING SYSTEM FOR A USER WITH DEEPLEARNING ALGORITHM AND A METHOD FOR CONTRLLING THE SYSTEM}

The present invention relates to a system and control method that detects and confirms the status of a user, for example, a patient or elderly person to be monitored, and performs safety management.

With the advancement of medical technology in modern society, survival age continues to increase, leading to a society where life is extended. However, as a counter-payment, the population with diseases different from before is rapidly increasing, for example, the population of dementia patients and people requiring psychological or psychosomatic observation is increasing. For example, due to the rapid increase in the number of infringing patients and the increase in the number of elderly people living alone, the demand for protection or observation in medical and daily life aspects is rapidly increasing.

Mainly, caregivers are responsible for the usual protective and observation of dementia patients or elderly people who have difficulty living alone and need help from others. However, compared to the rate of increase in the elderly population, the rate of increase in the influx of productive workers is significantly low, and the health of the elderly is significantly low. There is a significant lack of professional manpower in charge of management.

To solve these problems, it was necessary to commercialize high-quality care services using ICT, and various technologies have been researched and developed accordingly. Although the development of technology-based wearable healthcare devices is increasing, actual use by the elderly is low, and usability and maintenance are low. Management is in a remarkably low state and is unable to provide practical help. In other words, the conditions of use that require complicated usage, frequent charging management, and constant carrying pose a significant burden or burden to elderly people who have difficulty moving and living alone, resulting in extremely low actual usability and usability. It has been done.

Therefore, the need to utilize non-contact, non-perceptive sensors emerged as an alternative to the older generation's aversion to wearables, and the need for permanent and inexpensive life log sensors and methods to be applied to the majority of the elderly population was formed, and this was resolved. This is the purpose of the present invention.

The present invention collects IoT sensor-based information for long-term, continuous life log collection, detects health abnormalities and performs safety management through life log analysis, and provides appropriate information to caregivers about changes in living patterns for the elderly. The purpose is to provide a system and control method to implement a collaborative health recommendation service through feedback from caregivers. In other words, the present invention aims to develop technology for detecting health abnormalities and recommending individual health-related services based on 24-hour life log monitoring, for example, in the elderly. In other words, for example, through long-term and continuous monitoring of elderly people or those subject to psychosomatic detection, the activity level of subjects such as elderly people and each individual's unique life pattern are analyzed, brain health-related status evaluation and its variations are analyzed, and appropriate measures are provided accordingly. The goal is to develop technology to help seniors maintain healthy and active lives by recommending personalized health services and providing collaborative feedback accordingly.

In addition, the goal is to provide a configuration that enables risk elimination through preparedness by providing advance notice of the possibility of risk on the day of the user by utilizing data from other users with high data robustness.

The present invention is an IoT sensor-based information collection unit equipped with a non-contact sensor and a Narrow Area Active Detector (N-AD) for collecting long-term and continuous life logs of motoring subjects, and big data processing. 24-hour lifelog monitoring including a health abnormality detection and safety management unit through lifelog analysis using detection information, and a customized/collaborative health management service recommendation unit that performs recommendation functions according to the lifelog and brain health status. and an abnormality detection and response system and control method thereof.

That is, the present invention includes a monitoring sensor unit 200 including an activity sensor 200-1 that detects at least activity in the user's living space, and a monitoring life sensor unit 200 that detects at least activity. A monitoring terminal unit 100 that collects monitoring data including log data, and the monitoring terminal unit 100 receives monitoring data such as at least an activity index indicating whether the user is active, bedtime information indicating bedtime information, and external activities. Provides a deep learning user life status monitoring and management device (10) including a monitoring management server (400) that checks the outing index indicating information, determines the stability of the user's daily state, and transmits it to at least the guardian terminal unit (500). do.

In the deep learning user living status monitoring and management device 10, the activity sensor 200-1 may be disposed in a plurality of zones of the user's living space to detect information on the user's occupied space by time.

In the deep learning user living status monitoring and management device 10, the monitoring sensor unit 200: detects at least water, gas, and electricity usage and illumination conditions that reflect the user's daily living status in the user's living space. It may further include an implementation sensor unit 200-4 and a surveillance sensor unit 200-3 that detects whether the user goes out from the living space.

In the deep learning user life status monitoring and management device 10, the monitoring terminal unit 100 includes: a monitoring terminal communication unit 110 that receives the monitoring data from the monitoring sensor unit 200, and the monitoring sensor unit. A monitoring terminal storage unit 130 that stores the monitoring data received from (200), and a monitoring terminal control unit that controls transmission of the received monitoring data to the monitoring management server 400. 120) may also be included.

In the deep learning user life status monitoring and management device 10, the monitoring management server 40 includes: a server communication module 4001 that receives the monitoring data transmitted from the monitoring terminal 100, and the monitoring A monitoring module that monitors and confirms the stability of the user's daily state by using data to check at least the activity index indicating whether the user is active, the bedtime information indicating bedtime information, and the outing index indicating external activity information, and analyzes and determines the stability state. It may include (4100) and a corresponding module (4200) that transmits at least the analysis and judgment result of the monitoring module (4100) to the guardian terminal unit (500).

In the deep learning user life status monitoring and management device 10, the monitoring module 4100 performs: spatial analysis to check space occupancy status information of the user by time from at least the monitoring life log data detected as activity among the monitoring data; A monitoring submodule 4110, a behavior analysis monitoring submodule 4120 that confirms and analyzes the user's behavior information using the space occupancy status information of the user by time confirmed by the space analysis monitoring submodule 41110, and the space An abnormality analysis monitoring submodule 4130 that monitors and analyzes whether the user is in an abnormal state using the user's space occupancy status information by time from the analysis monitoring submodule 4110 and the user's behavior information from the behavior analysis monitoring submodule 4120. It may also include .

In the deep learning user life status monitoring and management device 10, the response module 4200 includes: an abnormality judgment response sub-module 4210 that executes a corresponding monitoring mode according to the analysis judgment of the monitoring module 4100; , an alarm execution response sub-module 4220 that executes an emergency alert mode when it is determined to be an emergency situation in the analysis judgment of the monitoring module 4100, and a corresponding monitoring mode according to the analysis judgment of the monitoring module 4100. It may also include a reporting response sub-module 4230 that transmits a report to the guardian terminal unit 500.

In the deep learning user life status monitoring and management device 10, other users' monitoring data can be input to the monitoring module 4100, and the user's monitoring data and other users' monitoring data are used to monitor the user. It may further include an AI learning module 4300 that executes AI learning to derive correlations between labels and features in the data and other users' monitoring data.

In the deep learning user life status monitoring and management device 10, the AI learning module 4300: uses the user's monitoring data and other users' monitoring data to obtain the user's monitoring data and other users' monitoring data and other users through AI learning learning. An AI learning model storage unit 4323 that stores an AI learning model for deriving correlations between labels and features in the user's monitoring data, and the user's monitoring data and other data derived from the AI learning model storage unit 4423. It may also include an AI learning data storage unit 4321 that stores correlations between labels and features in the user's monitoring data.

In the deep learning user life status monitoring and management device 10, including time-based space occupancy status information of the user of the spatial analysis monitoring sub-module 4110 and user behavior information of the behavior analysis monitoring sub-module 4120. Based on the core feature data and the core feature data of other users, the similarity analysis monitoring service confirms the similarity between the user's core feature data and the core feature data of other users, and confirms and extracts labeling information for the core feature data of other users. It may further include a module 4140.

In the deep learning user life status monitoring and management device 10, the user's monitoring data and other users' monitoring data are used to extract and label data events from the user's monitoring data and other users' monitoring data and label them. It may further include an auto labeling module 4400 that generates.

In the deep learning user life status monitoring and management device 10, the auto labeling module 4400: uses the user's monitoring data and other users' monitoring data to collect data among the user's monitoring data and other users' monitoring data. A data event extraction submodule 4410 that extracts an event, a labeling submodule 4420 that extracts the data event and assigns a label, and verifies the consistency of the labeling assigned by the labeling submodule 4420. It may also include a labeling verification submodule 4430.

According to another aspect of the present invention, the present invention includes a monitoring sensor unit 200 including an activity sensor 200-1 that detects at least activity in the user's living space, and a monitoring sensor unit 200 that detects at least activity in the user's living space. A monitoring terminal 100 that collects monitoring data including monitoring life log data detected by the monitoring terminal 100 and receives the monitoring data to indicate at least an activity index indicating whether the user is active, and bedtime information. A deep learning user life status monitoring and management device that includes a monitoring management server 400 that checks the going out index indicating sleeping information and external activity information, determines the stability of the user's daily state, and transmits it to at least the guardian terminal 500. A provision step (S10) of providing (10), and at least activity from the monitoring sensor unit 200 received from the monitoring terminal unit 100 through the server communication module 4001 of the monitoring management server 400. Using monitoring data including detected monitoring life log data, the stability of the user's daily state is confirmed by checking at least the activity index indicating whether the user is active, the bedtime information indicating bedtime information, and the going out index indicating external activity information. A monitoring step (S20) in which monitoring is confirmed and the stability state is analyzed and determined by the monitoring module 4100, and a response mode determination step in which the response module 4200 determines a response mode according to the analysis determination result of the monitoring module 4100 ( Provides a deep learning user life status monitoring and management device control method comprising a response mode execution step (S30) and a response mode execution step (S40) of executing the response mode determined in the response mode determination step (S30).

In the deep learning user life status monitoring and management device control method, the monitoring step (S20) includes: the monitoring received from the monitoring terminal 100 through the server communication module 4001 of the monitoring management server 400; A monitoring input step (S21) in which monitoring data including at least monitoring life log data detected as activity is received and input from the sensor unit 200, and spatial analysis monitoring from at least the monitoring life log data detected as activity among the monitoring data. A spatial analysis monitoring step (S23) in which the sub-module 4110 checks the user's space occupancy status information by time, and behavior analysis monitoring using the user's space occupancy status information by time confirmed by the spatial analysis monitoring sub-module 41110. A behavior analysis monitoring step (S25) in which the user's behavior information is confirmed and analyzed in the submodule 4120, and the user's space occupancy status information by time in the space analysis monitoring submodule 4110 and the behavior analysis monitoring submodule 4120. It may also include an abnormality analysis monitoring step (S27) in which the abnormality analysis monitoring submodule 4130 monitors and analyzes whether the user is in an abnormal state using the user's behavior information.

In the deep learning user life status monitoring and management device control method, the abnormality analysis monitoring step (S27) includes: receiving monitoring data from the monitoring terminal 100 and generating at least an activity index indicating the user's activity and bedtime information. It may also include a rule base abnormality analysis monitoring step (S271) in which the stability of the user's daily state is confirmed and determined by comparing the going-out index representing the sleeping information and the external activity information with a set reference value.

In the deep learning user life status monitoring and management device control method, the deep learning user life status monitoring and management device can input monitoring data of another user into the monitoring module 4100, and the user's monitoring data and the other user's monitoring data can be input to the deep learning user life status monitoring and management device. An AI learning module 4300 that uses monitoring data to perform AI learning to derive correlations between labels and features in the user's monitoring data and other users' monitoring data is further included, and the spatial analysis monitoring sub-module ( Based on the user's time-based space occupancy status information and the core feature data including the user's behavior information of the behavior analysis monitoring submodule 4120 and the core feature data of other users, the user's core feature data and Check the similarity of other users' core feature data, confirm and extract labeling information for other users' core feature data, and receive the monitoring data with the monitoring terminal 100, at least an activity index indicating whether the user is active or not, and bedtime information. It may further include an artificial intelligence abnormality analysis monitoring step (S273) that confirms and determines the stability of the user's daily state with respect to the going-out index representing sleeping information and external activity information.

In the deep learning user life status monitoring and management device control method, the response mode determination step (S30) is: according to the rule base anomaly analysis monitoring confirmation result confirmed in the rule base anomaly analysis monitoring step (S271), the response mode It may also include a determining rule base response mode determination step (S30-1).

In the deep learning user life status monitoring and management device control method, an analysis monitoring data input step (S31) in which the rule base abnormality analysis monitoring confirmation result confirmed in the rule base abnormality analysis monitoring step (S271) is input, and the corresponding module A confirmation result determination step (S33) for determining whether the rule base abnormality analysis monitoring confirmation result at (4200) is normal, and the response module (4200) for determining whether the rule base abnormality analysis monitoring confirmation result requires attention. It may also include a caution determination step (S35).

In the deep learning user life status monitoring and management device control method, the response mode determination step (S30) is: according to the artificial intelligence anomaly analysis monitoring confirmation result confirmed in the artificial intelligence anomaly analysis monitoring step (S273), the response mode is selected. It may also include a decision step (S30-2) on the artificial intelligence response mode.

In the deep learning user life status monitoring and management device control method, the analysis monitoring data input step (S301) in which the artificial intelligence abnormality analysis monitoring confirmation result confirmed in the artificial intelligence abnormality analysis monitoring step (S273) is input, and the corresponding module In (4200), a core feature similarity high determination step (S303) of determining whether the core feature similarity indicating core feature similarity as a result of the artificial intelligence anomaly analysis monitoring confirmation is higher than the preset core feature high similarity, and the corresponding module (4200) It may include a core feature similarity medium determination step (S305) that determines whether the core feature similarity indicating core feature similarity as a result of the artificial intelligence abnormality analysis monitoring confirmation is higher than the preset core feature medium similarity.

In the deep learning user life status monitoring and management device control method, the monitoring management server 400 further includes an auto labeling module 4400, is executed at least after the providing step (S10), and monitors the user's monitoring data. And an auto labeling step (S50) in which data events are extracted from the user's monitoring data and the other user's monitoring data using the other user's monitoring data and labeling is performed to generate a label may be further provided.

In the deep learning user life status monitoring and management device control method, the auto labeling step (S50) includes: an auto labeling monitoring input step (S51) in which the monitoring data is received and input to the auto label module 4400, and the user's A data event extraction step (S53) in which a data event among the user's monitoring data and the other user's monitoring data is extracted in the data event extraction submodule 4410 using the monitoring data and the monitoring data of another user, and a labeling submodule A preliminary labeling step (S55) in which (4420) assigns a label to the data event extracted in the data event extraction step (S53), and the labeling submodule 4420 in the preliminary labeling step (S55) ), a labeling verification step (S57) in which the labeling verification submodule 4430 verifies the consistency of the labeling assigned through ), and a labeling verification step (S55) according to the verification result of the labeling verification step (S59). A labeling confirmation step (S59) of confirming the labeling may be included.

On the other hand, according to another aspect of the present invention, the present invention includes a monitoring sensor unit 200 including an activity sensor 200-1 that detects at least activity in the user's living space, and the monitoring sensor unit 200. ), a monitoring terminal 100 that collects monitoring data including at least monitoring life log data detected as activity, and an activity index that receives the monitoring data from the monitoring terminal 100 and indicates at least whether the user is active, It includes a monitoring management server 400 that verifies the stability of the user's daily state by checking bedtime information indicating bedtime information and an outing index indicating external activity information and transmits the judgment to at least the guardian terminal unit 500, and manages the monitoring. Monitoring data of other users can be input to the server 400, and the correlation between labels and features in the user's monitoring data and the monitoring data of other users is derived using the user's monitoring data and the other user's monitoring data. It further includes an AI learning module 4300 that executes AI learning learning, and the monitoring management server 400 includes a device that de-identifies the monitoring data of the other users according to the minimum de-identification setting. Deep learning user life status monitoring, comprising a ID module 4600 that performs identification and re-identifies the de-identified monitoring data of other users to check whether the other users can be identified. A management device (10) is provided.

In the deep learning user life status monitoring and management device, the monitoring management server 400 includes: a server communication module 4001 that receives the monitoring data transmitted from the monitoring terminal 100, and uses the monitoring data. A monitoring module 4100 that monitors and confirms the stability of the user's daily state by checking at least the activity index indicating whether the user is active, the bedtime information indicating bedtime information, and the going out index indicating external activity information, and analyzes and determines the stability state. And, it may include a corresponding module 4200 that transmits at least the analysis and judgment result of the monitoring module 4100 to the guardian terminal unit 500.

In the deep learning user life status monitoring and management device, other users' monitoring data can be input to the monitoring module 4100, and the AI learning module 4300: monitors the user's monitoring data and other users' monitoring data. An AI learning model storage unit 4323 that stores an AI learning model for deriving correlations between labels and features in the user's monitoring data and other users' monitoring data through AI learning learning, and storing the AI learning model It may also include an AI learning data storage unit 4321 that stores correlations between labels and features in the user's monitoring data and other users' monitoring data derived from the unit 4423.

In the deep learning user life status monitoring and management device, the DID module 4600: performs de-identification to de-identify the other user's monitoring data according to the minimum de-identification setting. A D-identification sub-module 4610 that re-identifies the de-identified monitoring data of other users, a RE-ID confirmation module 4620 that re-identifies the other users, and a D-ID similarity check module that checks whether the other users can be identified ( 4630) may also be included.

In the deep learning user life status monitoring and management device, the di-identification sub-module 4610 includes: a di-identification sub-minimum limit that confirms the minimum requirements required to prevent re-identification of the monitoring data of other users. A non-displayed table of personal information-related data features of the monitoring data of other users based on the minimum requirements confirmed by the requirement confirmation unit 4611 and the di-identification subminimum requirement confirmation unit 4611 A di-identification sub-feature check unit 4613 that verifies data features related to municipal personal information, and a di-identification sub-feature check unit 4613 that non-displays the encrypted data features associated with the personal information to be de-displayed. It may also include a di-identification sub-screening unit 4617.

In the deep learning user life status monitoring and management device, the di-identification sub-module 4610 selects an individual subject to non-display among the personal information-related data features identified in the di-identification sub-feature check unit 4613. It may further include a di-identification sub-crypto unit 4615 that encrypts information-related data features.

In the deep learning user life status monitoring and management device, the DID module 4600 may further include: a minimum settings storage module 4640 that updates the minimum settings to de-identify the monitoring data of other users.

In the deep learning user living status monitoring and management device, the activity sensor 200-1 may be disposed in a plurality of zones of the user's living space to detect information on the user's occupied space by time.

In the deep learning user living status monitoring and management device, the monitoring sensor unit 200 includes: an environment sensor that detects at least water, gas, and electricity usage and illumination conditions that reflect the user's daily living status in the user's living space. It may further include a unit 200-4 and a surveillance sensor unit 200-3 that detects whether the user goes out from the living space.

In the deep learning user life status monitoring and management device, the monitoring terminal unit 100 includes: a monitoring terminal communication unit 110 that receives the monitoring data from the monitoring sensor unit 200, and the monitoring sensor unit 200. A monitoring terminal storage unit 130 that stores monitoring data received from the monitoring terminal control unit 120 that controls transmission of the received monitoring data to the monitoring management server 400. It may also be included.

In the deep learning user life status monitoring and management device, the monitoring management server 400 uses the user's monitoring data and other users' monitoring data to select data events among the user's monitoring data and other users' monitoring data. It may further include an auto labeling module 4400 that extracts and performs labeling to generate a label.

In the deep learning user life status monitoring and management device, the auto labeling module 4400: extracts a data event from the user's monitoring data and other users' monitoring data using the user's monitoring data and other users' monitoring data. A data event extraction sub-module 4410, a labeling sub-module 4420 for extracting the data event and assigning a label, and a labeling verification sub-module for verifying the consistency of the labeling assigned by the labeling sub-module 4420. It may also include module 4430.

In the deep learning user life status monitoring and management device, the monitoring module 4100 includes: a spatial analysis monitoring sub-module that checks space occupancy status information by time of the user from at least the monitoring life log data detected as activity among the monitoring data; (4110) and a behavior analysis monitoring sub-module 4120 that confirms and analyzes the user's behavior information using the user's time-specific space occupancy status information confirmed by the space analysis monitoring sub-module 41110, and the space analysis monitoring sub-module 4110. It may include an anomaly analysis monitoring sub-module 4130 that monitors and analyzes whether the user is in an abnormal state using the user's time-based space occupancy status information of the module 4110 and the user's behavior information of the behavior analysis monitoring sub-module 4120. It may be possible.

In the deep learning user life status monitoring and management device, core feature data including time-based space occupancy status information of the user of the spatial analysis monitoring sub-module 4110 and user behavior information of the behavior analysis monitoring sub-module 4120. And based on the core feature data of other users, a similarity analysis monitoring submodule 4140 that checks the similarity between the user core feature data of the user and the core feature data of other users, and confirms and extracts labeling information about the core feature data of other users. ) may further be included.

In the deep learning user life status monitoring and management device, the response module 4200 includes: an abnormality determination response sub-module 4210 that executes a corresponding monitoring mode according to the analysis judgment of the monitoring module 4100, and the monitoring An alarm execution response sub-module 4220 that executes an emergency alert mode when the analysis judgment of the module 4100 determines that it is an emergency situation, and a report corresponding to the corresponding monitoring mode according to the analysis judgment of the monitoring module 4100. It may also include a reporting response sub-module 4230 that transmits to the guardian terminal unit 500.

According to another aspect of the present invention, the present invention includes a monitoring sensor unit 200 including an activity sensor 200-1 that detects at least activity in the user's living space, and at least A monitoring terminal 100 that collects monitoring data including monitoring life log data detected as activity, and the monitoring terminal 100 receives the monitoring data to provide at least an activity index indicating whether the user is active and bedtime information. It includes a monitoring management server 400 that verifies the stability of the user's daily state by checking the going-out index indicating the sleeping information and the external activity information and transmits it to at least the guardian terminal unit 500, and the monitoring management server 400 ), other users' monitoring data can be input, and AI learning learning that uses the user's monitoring data and other users' monitoring data to derive correlations between labels and features in the user's monitoring data and other users' monitoring data. It further includes an AI learning module 4300 that executes, and the monitoring management server 400 includes a de-identification device that de-identifies the monitoring data of other users according to a minimum setting. Deep learning user life status monitoring and management device ( 10) a provision step (S10) of providing, and at least activity detected from the monitoring sensor unit 200 received from the monitoring terminal unit 100 through the server communication module 4001 of the monitoring management server 400 Using monitoring data including monitoring life log data, the stability of the user's daily state is monitored by checking at least the activity index indicating whether the user is active, the bedtime information indicating bedtime information, and the going out index indicating external activity information. A monitoring step (S20) in which the stability status is analyzed and determined by the monitoring module 4100, and a response mode determination step (S30) in which the response module 4200 determines a response mode according to the analysis and determination result of the monitoring module 4100. ), and a response mode execution step (S40) of executing the response mode determined in the response mode determination step (S30).

In the deep learning user life status monitoring and management device control method, the monitoring step (S20) includes: the monitoring received from the monitoring terminal 100 through the server communication module 4001 of the monitoring management server 400; A monitoring input step (S21) in which monitoring data including at least monitoring life log data detected as activity is received and input from the sensor unit 200, and spatial analysis monitoring from at least the monitoring life log data detected as activity among the monitoring data. A spatial analysis monitoring step (S23) in which the sub-module 4110 checks the user's space occupancy status information by time, and behavior analysis monitoring using the user's space occupancy status information by time confirmed by the spatial analysis monitoring sub-module 41110. A behavior analysis monitoring step (S25) in which the user's behavior information is confirmed and analyzed in the submodule 4120, and the user's space occupancy status information by time in the space analysis monitoring submodule 4110 and the behavior analysis monitoring submodule 4120. It may also include an abnormality analysis monitoring step (S27) in which the abnormality analysis monitoring submodule 4130 monitors and analyzes whether the user is in an abnormal state using the user's behavior information.

In the deep learning user life status monitoring and management device control method, the abnormality analysis monitoring step (S27) includes: receiving monitoring data from the monitoring terminal 100 and generating at least an activity index indicating the user's activity and bedtime information. It may also include a rule base abnormality analysis monitoring step (S271) in which the stability of the user's daily state is confirmed and determined by comparing the going-out index representing the sleeping information and the external activity information with a set reference value.

In the deep learning user life status monitoring and management device control method, the deep learning user life status monitoring and management device can input monitoring data of another user into the monitoring module 4100, and the user's monitoring data and the other user's monitoring data can be input to the deep learning user life status monitoring and management device. An AI learning module 4300 that uses monitoring data to perform AI learning to derive correlations between labels and features in the user's monitoring data and other users' monitoring data is further included, and the spatial analysis monitoring sub-module ( Based on the user's time-based space occupancy status information and the core feature data including the user's behavior information of the behavior analysis monitoring submodule 4120 and the core feature data of other users, the user's core feature data and Check the similarity of other users' core feature data, confirm and extract labeling information for other users' core feature data, and receive the monitoring data with the monitoring terminal 100, at least an activity index indicating whether the user is active or not, and bedtime information. It may further include an artificial intelligence abnormality analysis monitoring step (S273) that confirms and determines the stability of the user's daily state with respect to the going-out index representing sleeping information and external activity information.

In the deep learning user life status monitoring and management device control method, the response mode determination step (S30) is: according to the rule base anomaly analysis monitoring confirmation result confirmed in the rule base anomaly analysis monitoring step (S271), the response mode It may also include a determining rule base response mode determination step (S30-1).

In the deep learning user life status monitoring and management device control method, an analysis monitoring data input step (S31) in which the rule base abnormality analysis monitoring confirmation result confirmed in the rule base abnormality analysis monitoring step (S271) is input, and the corresponding module A confirmation result determination step (S33) for determining whether the rule base abnormality analysis monitoring confirmation result at (4200) is normal, and the response module (4200) for determining whether the rule base abnormality analysis monitoring confirmation result requires attention. It may also include a caution determination step (S35).

In the deep learning user life status monitoring and management device control method, the response mode determination step (S30) is: according to the artificial intelligence anomaly analysis monitoring confirmation result confirmed in the artificial intelligence anomaly analysis monitoring step (S273), the response mode is selected. It may also include a decision step (S30-2) on the artificial intelligence response mode.

In the deep learning user life status monitoring and management device control method, the analysis monitoring data input step (S301) in which the artificial intelligence abnormality analysis monitoring confirmation result confirmed in the artificial intelligence abnormality analysis monitoring step (S273) is input, and the corresponding module In (4200), a core feature similarity high determination step (S303) of determining whether the core feature similarity indicating core feature similarity as a result of the artificial intelligence anomaly analysis monitoring confirmation is higher than the preset core feature high similarity, and the corresponding module (4200) It may include a core feature similarity medium determination step (S305) that determines whether the core feature similarity indicating core feature similarity as a result of the artificial intelligence abnormality analysis monitoring confirmation is higher than the preset core feature medium similarity.

In the deep learning user life status monitoring and management device control method, it is executed at least after the provision step (S10) and de-identifies the monitoring data of the other user in the ID module 4600 of the monitoring management server 400. A DI ID step in which de-identification is performed to de-identify the monitoring data of the other user according to the minimum settings, and re-identification of the de-identified monitoring data of the other user is performed to confirm whether the other user can be identified. (S60) may be further included.

In the deep learning user life status monitoring and management device control method, the DI step (S60) includes: a DI ID monitoring input step (S61) in which the monitoring data is received and input to the DI module 4600, and the other user's A de-identification step (S63) in which de-identification is performed to de-identify the monitoring data of other users in the de-identification submodule 4610 according to the minimum setting for de-identifying the monitoring data; A minimum setting requirement verification step (S65) in which the identified monitoring data of other users is re-identified and compared with the minimum setting requirements to confirm and verify whether re-identification is possible, and the requirements in the minimum setting requirement verification step (S65) It may also include a verification response step (S67) in which verification response items set according to the verification results are executed.

In the deep learning user life status monitoring and management device control method, the di-identification step (S63) includes: the di-identification sub-minimum requirement confirmation unit 4611 of the di-identification sub-module 4610. A di-identification sub-minimum requirement confirmation step (S631) of confirming the minimum requirements required to prevent re-identification of the monitoring data of other users, and di-identification of the di-identification sub-module 4610. Non-display of personal information-related data features of other users' monitoring data in the cation sub-feature check unit 4613 based on the minimum requirements confirmed in the di-identification sub-minimum requirement check unit 4611. A di-identification sub-feature check step (S633) for confirming personal information-related data features, and the di-identification sub-feature in the di-identification sub-screening unit 4617 of the di-identification sub-module 4610. It may also include a di-identification sub-screening step (S635) of non-displaying personal information-related data features subject to non-display among the personal information-related data features identified in the check unit 4613.

In the deep learning user life status monitoring and management device control method, the di-identification step (S63) is: the di-identification sub-crypto unit 4615 of the di-identification sub-module 4610 It may further include a de-identification sub-crypto step (S367) of encrypting a personal information-related data feature to be undisplayed among the personal information-related data features identified by the information sub-feature check unit 4613.

In the deep learning user life status monitoring and management device control method, the minimum setting requirement verification step (S65) is: the non-identified other person in the ID verification module 4620 of the di-identification sub-module 4610. A ID confirmation step (S651) of re-identifying the user's monitoring data, and the ID similarity confirmation module 4630 calculating a ID similarity indicating whether another user can be identified from the re-identified monitoring data of another user. A DID similarity check step (S653) and a DID completion check step (S655) that determines whether the minimum setting requirements for D-identification are satisfied based on whether the DID similarity calculated in the DID similarity check step (S653) is less than the preset similarity. ) may further be included.

In addition, according to another aspect of the present invention, the present invention includes an activity sensor 200-1 that detects at least activity in the user's living space and at least water, gas, and A monitoring sensor unit 200 including an environment sensor unit 200-4 that detects electricity usage and illumination conditions and the user's breathing rate and heart rate, and monitoring that detects at least activity from the monitoring sensor unit 200. A monitoring terminal 100 that collects monitoring data including life log data and monitoring bio log data indicating the user's physical condition, and receives the monitoring data with the monitoring terminal 100 to indicate at least the user's activity. The stability of the user's daily state is confirmed and determined by checking the activity index, bedtime information indicating bedtime information, going out index indicating external activity information, and the bio index indicating the user's physical condition through the user's breathing rate and heart rate, and at least the guardian terminal ( 500) and includes a monitoring management server 400 that transmits information to the user, wherein the monitoring management server 400 uses the monitoring data to determine at least an activity index indicating whether the user is active, bedtime information indicating bedtime information, and external activities. Monitoring and confirming the stability of the user's daily state by checking the outing index and bio index indicating information, analyzing and determining the stability state, and monitoring data including call events including death of other users, emergency condition, and life support status. A monitoring module 4100 further comprising an event update and classification submodule 4150 that can be input, updated, and classified, and monitoring data of other users can be input, using the user's monitoring data and the other user's monitoring data. An AI learning module (4300) that performs AI learning to derive correlations between labels and features in the user's monitoring data and other users' monitoring data, and call events classified in the event update and classification submodule (4150) A deep learning user life status monitoring and management device ( 10) is provided.

In the deep learning user life status monitoring and management device 10, the probability module 4700 includes: an emergency call identified as an emergency among call events classified in the event update and classification sub-module 4150; An emergency situation probability sub-module (4710) that generates the probability of an emergency situation occurring on the day of the user based on AI learning performed in the AI learning module (4300) from an event, and the event update and classification sub-module (4150) ) A death situation probability sub that generates the probability of occurrence of a death situation on the day of the user based on AI learning executed in the AI learning module 4300 from a death situation call event identified as a death situation among call events classified in ). It may also include module 4720.

In the deep learning user life status monitoring and management device 10, the monitoring management server 400 includes: a server communication module 4001 that receives the monitoring data transmitted from the monitoring terminal 100, and at least the A response module that transmits the probability of an emergency situation occurring on the day of the user of the probability module 4700 and the probability of a death situation occurring on the user's day to at least the guardian terminal unit 500 according to the analysis judgment result of the monitoring module 4100. It may also include (4200).

In the deep learning user life status monitoring and management device 10, other users' monitoring data can be input to the monitoring module 4100, and the AI learning module 4300 includes: the user's monitoring data and other users' monitoring data. An AI learning model storage unit 4323 that stores an AI learning model for deriving correlations between labels and features in the user's monitoring data and other users' monitoring data through AI learning using monitoring data, and the AI It may also include an AI learning data storage unit 4321 that stores correlations between labels and features in the user's monitoring data and other users' monitoring data derived from the learning model storage unit 4423.

In the deep learning user living status monitoring and management device 10, the activity sensor 200-1 may be disposed in a plurality of zones of the user's living space to detect information on the user's occupied space by time.

In the deep learning user living status monitoring and management device 10, the monitoring sensor unit 200 may further include a surveillance sensor unit 200-3 that detects whether the user goes out from the living space. .

In the deep learning user life status monitoring and management device 10, the environment sensor unit 200-4 detects at least water, gas, and electricity usage reflecting the user's daily life condition in the user's living space. It may also include an implementation weighing sensor (200-4b, 200-4d, 200-4e, 200-4f).

In the deep learning user living condition monitoring and management device 10, the environment sensor unit 200-4 is an environment illuminance sensor 200-4 that detects the illuminance state of the user's daily life condition in the user's living space. 4a) may also be included.

In the deep learning user life status monitoring and management device 10, the environment sensor unit 200-4 may include an environment laser sensor 200-4g that detects the user's breathing rate and heart rate. there is.

In the deep learning user life status monitoring and management device 10, the monitoring terminal unit 100 includes: a monitoring terminal communication unit 110 that receives the monitoring data from the monitoring sensor unit 200, and the monitoring sensor unit. A monitoring terminal storage unit 130 that stores the monitoring data received from (200), and a monitoring terminal control unit that controls transmission of the received monitoring data to the monitoring management server 400. 120) may also be included.

In the deep learning user life status monitoring and management device 10, the monitoring management server 400 uses the user's monitoring data and other users' monitoring data to select among the user's monitoring data and the other user's monitoring data. It may further include an auto labeling module 4400 that extracts data events and performs labeling to generate labels.

In the deep learning user life status monitoring and management device 10, the auto labeling module 4400: uses the user's monitoring data and other users' monitoring data to collect data among the user's monitoring data and other users' monitoring data. A data event extraction submodule 4410 that extracts an event, a labeling submodule 4420 that extracts the data event and assigns a label, and verifies the consistency of the labeling assigned by the labeling submodule 4420. It may also include a labeling verification submodule 4430.

In the deep learning user life status monitoring and management device 10, the monitoring module 4100 performs: spatial analysis to check space occupancy status information of the user by time from at least the monitoring life log data detected as activity among the monitoring data; A monitoring submodule 4110, a behavior analysis monitoring submodule 4120 that confirms and analyzes the user's behavior information using the space occupancy status information of the user by time confirmed by the space analysis monitoring submodule 41110, and the space An abnormality analysis monitoring submodule 4130 that monitors and analyzes whether the user is in an abnormal state using the user's space occupancy status information by time from the analysis monitoring submodule 4110 and the user's behavior information from the behavior analysis monitoring submodule 4120. It may also include .

In the deep learning user life status monitoring and management device 10, including time-based space occupancy status information of the user of the spatial analysis monitoring sub-module 4110 and user behavior information of the behavior analysis monitoring sub-module 4120. Based on the core feature data and the core feature data of other users, the similarity analysis monitoring service confirms the similarity between the user's core feature data and the core feature data of other users, and confirms and extracts labeling information for the core feature data of other users. It may further include a module 4140.

In the deep learning user life status monitoring and management device 10, the response module 4200 includes: an abnormality judgment response sub-module 4210 that executes a corresponding monitoring mode according to the analysis judgment of the monitoring module 4100; , an alarm execution response sub-module 4220 that executes an emergency alert mode when it is determined to be an emergency situation in the analysis judgment of the monitoring module 4100, and a corresponding monitoring mode according to the analysis judgment of the monitoring module 4100. It may also include a reporting response sub-module 4230 that transmits a report to the guardian terminal unit 500.

According to another aspect of the present invention, the present invention provides an activity sensor 200-1 that detects at least activity in the user's living space and at least water, gas, and electricity usage that reflects the daily living conditions in the user's living space. and a monitoring sensor unit 200 including an environment sensor unit 200-4 that detects the illumination state and the user's breathing rate and heart rate, and a monitoring life log that is detected as at least activity from the monitoring sensor unit 200. A monitoring terminal 100 that collects monitoring data including data and monitoring bio log data indicating the user's physical condition, and an activity index that receives the monitoring terminal 100 and the monitoring data to indicate at least whether the user is active. , the stability of the user's daily state is confirmed and determined by checking the bedtime information indicating sleeping information, the going out index indicating external activity information, and the bio-index indicating physical condition through the user's breathing rate and heart rate, and at least the guardian terminal 500 The monitoring management server 400 includes: a monitoring management server 400 that transmits the monitoring data to: at least an activity index indicating whether the user is active, sleeping information indicating sleeping information, and external activity information; By checking the going out index and bio index, the stability of the user's daily state is monitored and confirmed, the stability state is analyzed and judged, and monitoring data including call events including death, emergency condition, and life support status of other users are entered. A monitoring module 4100 further comprising an event update and classification submodule 4150 capable of updating and classifying, and capable of inputting monitoring data of other users, and using the user's monitoring data and other users' monitoring data An AI learning module (4300) that performs AI learning to derive correlations between labels and features in the monitoring data and other users' monitoring data, and call events classified in the event update and classification submodule (4150). Deep learning user life status monitoring and management device (10), further comprising a probability module (4700) that generates the possibility of emergency and death situations based on AI learning executed in the AI learning module (4300). A provision step (S10) of providing, and monitoring that is detected as at least active from the monitoring sensor unit 200 received from the monitoring terminal unit 100 through the server communication module 4001 of the monitoring management server 400. Using monitoring data including life log data, the stability of the user's daily state is confirmed by checking at least the activity index indicating whether the user is active, bedtime information indicating bedtime information, and going out index and bio-index indicating external activity information. A monitoring step (S20) in which monitoring is confirmed and the stability state is analyzed and determined by the monitoring module 4100, and a response mode determination step in which the response module 4200 determines a response mode according to the analysis determination result of the monitoring module 4100 ( Provides a deep learning user life status monitoring and management device control method comprising a response mode execution step (S30) and a response mode execution step (S40) of executing the response mode determined in the response mode determination step (S30).

In the deep learning user life status monitoring and management device control method, the monitoring step (S20) includes: the monitoring received from the monitoring terminal 100 through the server communication module 4001 of the monitoring management server 400; A monitoring input step (S21) in which monitoring data including at least monitoring life log data detected as activity is received and input from the sensor unit 200, and spatial analysis monitoring from at least the monitoring life log data detected as activity among the monitoring data. A spatial analysis monitoring step (S23) in which the sub-module 4110 checks the user's space occupancy status information by time, and behavior analysis monitoring using the user's space occupancy status information by time confirmed by the spatial analysis monitoring sub-module 41110. A behavior analysis monitoring step (S25) in which the user's behavior information is confirmed and analyzed in the submodule 4120, and the user's space occupancy status information by time in the space analysis monitoring submodule 4110 and the behavior analysis monitoring submodule 4120. It may also include an abnormality analysis monitoring step (S27) in which the abnormality analysis monitoring submodule 4130 monitors and analyzes whether the user is in an abnormal state using the user's behavior information.

In the deep learning user life status monitoring and management device control method, the abnormality analysis monitoring step (S27) includes: receiving monitoring data from the monitoring terminal 100 and generating at least an activity index indicating the user's activity and bedtime information. It may also include a rule base abnormality analysis monitoring step (S271) in which the stability of the user's daily state is confirmed and determined by comparing the going-out index representing the sleeping information and external activity information with a set reference value.

In the deep learning user life status monitoring and management device control method, the deep learning user life status monitoring and management device can input monitoring data of another user into the monitoring module 4100, and the user's monitoring data and the other user's monitoring data can be input to the deep learning user life status monitoring and management device. An AI learning module 4300 that uses monitoring data to perform AI learning to derive correlations between labels and features in the user's monitoring data and other users' monitoring data is further included, and the spatial analysis monitoring sub-module ( Based on the user's time-based space occupancy status information and the core feature data including the user's behavior information of the behavior analysis monitoring submodule 4120 and the core feature data of other users, the user's core feature data and Check the similarity of other users' core feature data, confirm and extract labeling information for other users' core feature data, and receive the monitoring data with the monitoring terminal 100, at least an activity index indicating whether the user is active or not, and bedtime information. It may further include an artificial intelligence abnormality analysis monitoring step (S273) that confirms and determines the stability of the user's daily state with respect to the going out index and bio index representing sleeping information and external activity information.

In the deep learning user life status monitoring and management device control method, the response mode determination step (S30) is: according to the rule base anomaly analysis monitoring confirmation result confirmed in the rule base anomaly analysis monitoring step (S271), the response mode It may also include a determining rule base response mode determination step (S30-1).

In the deep learning user life status monitoring and management device control method, an analysis monitoring data input step (S31) in which the rule base abnormality analysis monitoring confirmation result confirmed in the rule base abnormality analysis monitoring step (S271) is input, and the corresponding module A confirmation result determination step (S33) for determining whether the rule base abnormality analysis monitoring confirmation result at (4200) is normal, and the response module (4200) for determining whether the rule base abnormality analysis monitoring confirmation result requires attention. A caution determination step (S35) may also be included.

In the deep learning user life status monitoring and management device control method, the response mode determination step (S30) is: according to the artificial intelligence anomaly analysis monitoring confirmation result confirmed in the artificial intelligence anomaly analysis monitoring step (S273), the response mode is selected. It may also include a decision step (S30-2) on the artificial intelligence response mode.

In the deep learning user life status monitoring and management device control method, the analysis monitoring data input step (S301) in which the artificial intelligence abnormality analysis monitoring confirmation result confirmed in the artificial intelligence abnormality analysis monitoring step (S273) is input, and the corresponding module In (4200), a core feature similarity high determination step (S303) of determining whether the core feature similarity indicating core feature similarity as a result of the artificial intelligence anomaly analysis monitoring confirmation is higher than the preset core feature high similarity, and the corresponding module (4200) It may include a core feature similarity medium determination step (S305) that determines whether the core feature similarity indicating core feature similarity as a result of the artificial intelligence abnormality analysis monitoring confirmation is higher than the preset core feature medium similarity.

In the deep learning user life status monitoring and management device control method, it is executed at least after the provision step (S10), updates a call event requesting emergency or support for the user and other users, and determines the type of the call event. It may further include a probability deep learning step (S70) in which the probability is calculated based on information on the occurrence of emergency treatment or death of other users and deep learning is performed.

In the deep learning user life status monitoring and management device control method, the probability deep learning step (S70) includes: a call event result update step of updating a call event requesting emergency or support for the user and other users. (S71), a call event classification step (S73) for classifying emergency call events of death and first aid and life support call events processed by requesting life support, and other users' information including information on occurrence of first aid or death. A probability data extraction step (S75) of extracting probability data for calculating the probability of an emergency situation and the probability of a death situation from monitoring data, and a probability data learning step of deep learning the probability data. (S77) may be further included.

According to the present invention, in accordance with the need to develop IoT sensor-based information collection technology for long-term and continuous life log collection considering the elderly, based on information extracted through sustainable sensing using an activity detector as a non-contact sensor In addition to the provision of an activity sensor that studies activity and lifestyle patterns through combination with other sensors, the acquired data is used to secure big data on the lifestyle patterns of the elderly in a non-conscious, non-contact state and to measure activity and lifestyle patterns over a long period of time. Clinical information can be obtained that correlates changes with the user's health status, for example, brain health status.

According to the present invention, by securing an inexpensive monitoring means to cover the majority of the elderly, developing a health abnormality detection and safety management system through life log analysis, medical information and constant monitoring life log through identification of the lifestyle habits of super agers Based on this information, we can provide customized health programs for the elderly. In other words, it is necessary to analyze changes in patterns in the lives of patients with diseases, especially those with dementia, Parkinson's disease, and chronic diseases related to brain cognition, and apply them to the general elderly. A model that predicts health abnormalities in the general elderly is needed. Patients' life logs By analyzing the correlation between data (lifestyle patterns) and brain health status through clinical trials and applying it to the general elderly, it is possible to run a prediction model for the general public's health status, and minimize the cost burden of use through minimal optimization of installation and operation costs. By minimizing and expanding user base coverage, it is possible to provide sufficient monitoring and management services to more elderly users.

In addition, the need for rapid accident response and prevention through emergency situation detection through lifestyle pattern analysis can increase the importance of continuous health care even in places other than medical institutions. It is necessary to acquire/analyze data using mobile devices capable of continuous monitoring and interaction, and it is necessary to provide customized healthcare services appropriate for each individual's health status, emotional state, and preferences, and reduce activity over 24 hours and over a long period of time, and lifestyle patterns. It is necessary to measure the amount of change in (activity, sleep, going out, toilet use, etc.), which can provide psychological and emotional stability to the elderly by knowing that they can receive help in emergency situations when living at home.

The present invention has accumulated many years of care experience in silver care and has developed the first telecare communication infrastructure and equipment using a wireless network through technology for safety services for the elderly through its own capabilities of equipment, protocols, and telecare platforms. Ability provides customized health services based on the lifestyle habits of the very elderly.

The present invention implements a rule base-based monitoring function and simultaneously or selectively implements an artificial intelligence monitoring function, and monitors data including the user's own monitoring life log data by comparing a preset reference value with the user's own monitor data. A rule base-based monitoring and response process may be performed, and the user's current living status can be monitored using AI learning on other users' monitoring data in addition to the user's own monitoring data, using other users' a priori events to detect danger or caution. It may enable advance notification of situations or the implementation of preventive measures.

In addition, the present invention provides labels to the monitoring data to maintain a reasonable amount of data processing calculations and improve the accuracy of data analysis when using the user's monitoring data and/or other users' monitoring data, and automatically labels the monitoring data. By enabling this task, it may be possible to monitor and manage the user's more accurate life information through rapid updates with meaningful labels of the monitoring data.

In addition, the present invention performs AI learning using the user's monitoring data as well as other users' monitoring data, and de-identifies for personal information protection measures when using other users' monitoring data, including computer medical records, etc. Implement compliance with the minimum de-identification measures and de-identification requirements according to regulations through the components and methods that perform the function, check whether re-identification is possible, and review and respond to the need for additional de-identification measures. By doing so, the amount of calculation can be optimized while the risk of personal information leakage can be minimized.

In addition, the present invention updates events such as emergencies or deaths of other users, extracts highly correlated data from the current or previous monitoring data, learns it through deep learning, provides it to the guardian through comparison with the user's monitoring data, and provides it to the guardian on the same day. It may be possible to provide an opportunity for action to eliminate or minimize the risk.

In addition, it is possible to eliminate risk through preparation by notifying the user in advance of the possibility of risk on the day by utilizing data from other users with high data robustness.

Figure 1 is a schematic configuration diagram of a deep learning user life status monitoring and management device according to an embodiment of the present invention.
2 and 3 are schematic and detailed configuration diagrams of the monitoring sensor unit and the monitoring terminal unit of the deep learning user life status monitoring and management device according to an embodiment of the present invention.
Figure 4 is a schematic block diagram of a monitoring management server of a deep learning user life status monitoring and management device according to an embodiment of the present invention.
5 to 8 are schematic block diagrams of the detailed configuration of the monitoring management server of the deep learning user life status monitoring and management device according to an embodiment of the present invention.
Figure 9 is a schematic flowchart of a control method of a deep learning user life status monitoring and management device according to an embodiment of the present invention.
10 to 17 are schematic flowcharts and detailed flowcharts of a control method of a deep learning user life status monitoring and management device according to an embodiment or modified example of the present invention.
Figure 18 is a detailed block diagram of a deep learning user life status monitoring and management device according to a modification of an embodiment of the present invention.
Figure 19 is a flowchart of a control method of a deep learning user life status monitoring and management device according to a modification of an embodiment of the present invention.
Figure 20 is a schematic configuration diagram of a monitoring sensor unit and a monitoring terminal unit of a deep learning user life status monitoring and management device according to another embodiment of the present invention.
Figure 21 is a schematic block diagram of a monitoring management server of a deep learning user life status monitoring and management device according to another embodiment of the present invention.
22 and 23 are schematic block diagrams of the detailed configuration of a deep learning user life status monitoring and management device according to an embodiment of the present invention.
Figure 24 is a flowchart of a control method of a deep learning user life status monitoring and management device according to a modification of another embodiment of the present invention.
Figure 25 shows the relationship between user and other user monitoring data with respect to time and peer/peer group for AI deep learning learning of the control method of the deep learning user life status monitoring management device according to a modification of another embodiment of the present invention. These are the lines listed.
Figures 26 and 27 are examples of emergency or death situation probability and caution display images transmitted to the guardian terminal of a deep learning user life status monitoring and management device according to a modification of another embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings. First, when adding reference numerals to components in each drawing, it should be noted that identical components are given the same reference numerals as much as possible even if they are shown in different drawings. Additionally, in describing the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present invention, the detailed description will be omitted.

The deep learning user life status monitoring and management device 10 of the present invention includes a monitoring sensor unit 200, a monitoring terminal unit 100, and a monitoring management server 400, and the monitoring management server 400 monitors the user's Communication is achieved with the preset guardian terminal unit 300 on the guardian side.

As shown in FIGS. 1 and 2, the monitoring sensor unit 200 of the present invention is placed in the user's living space. The monitoring sensor unit 200 is designed for users who require management or protection, that is, people with mild cognitive impairment, people with musculoskeletal diseases, people with respiratory diseases such as chronic obstructive pulmonary disease (COPD), people with mental diseases such as digestive disorders, or the elderly or the very elderly. Obtain information from users' living spaces, such as That is, the monitoring sensor unit 200 is placed in the living space of a sick person or an elderly person and acquires the user's life activity information, so-called life log data. The monitoring sensor unit 200 of the present invention has a non-possessive, non-contact sensing structure. achieves In the case of the sick or the elderly, there is the inconvenience of installing a separate sensor device to detect and understand the life log as a lifestyle pattern, the decrease in detection accuracy due to frequent non-possession, and the inconvenience of charging the battery or non-detection due to battery discharge. To solve problems such as state formation, the monitoring sensor unit 200 of the present invention adopts a structure that is not carried by the user and is directly mounted in the user's living space.

That is, it includes an activity sensor 200-1 that detects at least activity in the user's living space. The activity sensor 200-1 is placed in the user's living space, and its placement position can be adjusted depending on the sensing environment. The activity sensor 200-1 can be selected from a variety of options, such as being configured as an infrared sensor within the range of detecting the user's activity. In this embodiment, the activity sensors 200-1 are shown as being disposed one by one in each space, but in some cases, a narrow-area activity sensor that narrows the detection area to detect detailed motion and a wide-area activity sensor with a wide detection area are used, respectively. Various implementations are possible depending on the design specifications, such as a configuration in which a plurality of devices are arranged.

Meanwhile, the activity sensor 200-1 may include an activity sensor detector 201, an activity sensor control unit 203, and an activity sensor communication unit 205. Other sensors of the monitoring sensor unit 200 described below may also be configured to include a detector, a control unit, and a communication unit. Here, the communication unit can adopt various types of communication structures such as Zigbee and 5G, and establishes a communication method to implement IOT functions to enable data transmission and reception.

The monitoring module 4100 of the monitoring management server 400 of the present invention utilizes the detection signal of the activity sensor 200-1 to monitor the space occupied by the user, the occupancy time in the occupied space, and changes in the occupied space. It enables acquisition of secondary information such as mobility. In this way, based on the information extracted through sustainable sensing through activity detection through the activity sensor 200-1 of the non-possessed contactless sensor of the present invention, the amount of activity and lifestyle patterns are studied through combination with other sensors. In addition to providing the activity sensor, the acquired data is used to secure big data on the lifestyle patterns of the elderly in an unconscious, non-contact state, and by securing clinical information on the correlation between changes in activity level and lifestyle patterns over a long period of time and brain health status. , It is possible to detect health abnormalities and implement a safety management system through life log analysis by securing inexpensive monitoring means to cover the majority of the elderly, and to extract meaningful medical information or events through tracking the lifestyle habits of the sick or the elderly. Based on information from the constant monitoring life log, customized health programs for the elderly can be provided.

That is, the present invention collects lifelog data related to the user's daily life and compares it with a preset reference value to monitor the user's current state, or compares it with the lifelog data of the user's previous monitoring data, and/or It provides a structure to monitor the user's current status by comparing it with other people's monitoring data. The user checks similar or different patterns from the previous life pattern, that is, the life log pattern, through his or her time series of monitoring data points, compares it with the standard setting value, determines whether there is an abnormality, and determines whether there is an abnormality. can make countermeasures feasible.

In addition, in some cases, the user can compare the user's time series of monitoring data points with other people's monitoring data to determine whether there is a similarity or abnormality, warn the user, and execute certain countermeasures. You may. In other words, by comparing the monitoring data of others of the same age or with the same disease with the current time standard or time series time series monitoring data, it is possible to determine whether there is similarity or abnormality and warn, thereby enabling the execution of countermeasures.

Through this configuration, through the user's own monitoring data or other people's monitoring data, for example, other people's monitoring information, patterns related to daily information about the lives of patients with mild cognitive impairment, Parkinson's patients, and chronic diseases are identified. It is also possible to mark changes as analysis references and compare them with the monitoring data of general users to monitor the possibility of preliminary disease due to changes in the general user's daily life patterns.

For example, predicting health abnormalities of general elderly people by comparing and applying information about general elderly people or other people, or conversely, monitoring the estimated possibility of disease by comparing daily information of general elderly people with information about patients with diseases, or monitoring the possibility of disease by comparing information on patients with diseases. It may be possible to provide preliminary warnings or take precautions for situations that are expected or likely to occur by comparing information about the disease with the information of other people with the disease, through life log data (life patterns) of the user and other people and clinical trials. By analyzing the correlation between the health status of the general public and applying it to the general elderly, it is possible to execute the operation of the health status prediction model of the general public. In particular, considering that users of the present invention are likely to be elderly, we implement a non-possessed, contactless detection structure using IoT sensor-based information collection technology for long-term, continuous life log collection.

The activity sensor 200-1 may be configured to detect both a narrow area and a wide area by mounting both a narrow area activity sensor and a wide area activity sensor as described above in the user space. The activity sensor 200-1 -1) A plurality of units may be placed in the user's living space. In other words, the activity sensor 200-1 can be placed in a plurality of areas of the user's living space to minimize blind areas, thereby enabling accurate collection of lifelog data and monitoring data regarding the user's activity, and Detects occupied space information by time.

In addition to detecting the user's activity, the present invention may be further equipped with various sensors for collecting various life log data, that is, monitoring data. That is, the monitoring sensor unit 200 further includes an environment sensor unit 200-4 and a surveillance sensor unit 200-3, where the environment sensor unit 200-4 monitors the user's life. The surveillance sensor unit 200-3 detects at least water, gas, and electricity usage and illumination conditions that reflect the daily living conditions in the space, and the surveillance sensor unit 200-3 detects whether the user has gone out from the living space.

Like the activity sensor 200-1, the environment sensor unit 200-4 and the surveillance sensor unit 200-3 also include a sensor detector, a sensor control unit, and a sensor communication unit to collect detected life log data. It enables transmission to the monitoring management server 400.

The environment sensor unit 200-4 acquires information about the user's daily living environment in the space where the user resides, such as gas, electricity, water, and illuminance and/or temperature and humidity. In this embodiment, the user's living space mainly includes a bedroom space (Abd), a living space (Alv), a bathroom space (Abt), a kitchen space (Akc), and an entrance space (Adr), which is an example of the present invention. The user's living space is not limited to this and can be modified in various ways within the scope of detecting the user's activity, but in this embodiment, it is a bedroom space (Abd), a living room space (Alv), a bathroom space (Abt), and a kitchen space (Akc). , Set the access space (Adr) as the user activity space or living space.

That is, the environment sensor unit 200-4 includes an environment illuminance sensor 200-4a, an environment temperature and humidity sensor 200-4c, and an environment metering sensor 200-4b, 200-4d, and 200. -4e,200-4f),

The environment illuminance sensor 200-4a detects the illuminance in each living space where the lighting lamp 160 is placed. In particular, illuminance is one of the important living environment factors that affect the mental state of users who are mentally ill or need attention or are elderly. For example, in the case of elderly people without mild cognitive impairment, sudden changes in illumination may cause physical and mental changes such as sudden seizures in some cases, and the user's activity status and direction of activity should be taken into consideration to prevent sudden changes in illumination. By controlling the operation of the lighting lamp 160, it is possible to provide an optimized living environment to the user.

Additionally, the environment temperature and humidity sensor 200-4c detects the temperature and humidity in the user's active living space. This can be used to derive a correlation with each user's activity through comparison of the temperature and humidity data from the environment temperature and humidity sensor 200-4c and the user's activity life log data.

In addition, the environment metering sensors (200-4b, 200-4d, 200-4e, 200-4f) detect water, gas, and electricity usage in the user's living space and monitor it to the monitoring management server 40. Enables data transmission. The environmental metering sensors (200-4b, 200-4d, 200-4e, 200-4f) are connected to the environmental electric metering sensor (200-4d), the environmental water metering sensor (200-4e) and the environmental metering sensor (200-4e). It includes a gas metering sensor 200-4f, and in some cases, it may also include an environment kitchen activity metering sensor 200-4b.

The environmental electricity metering sensor 200-4d detects the amount of electricity used in the user's living space, and the environmental water metering sensor 200-4e detects the amount of water used in the user's living space, The environment gas metering sensor 200-4f detects the amount of gas usage in the user's living space. In some cases, the use of gas for a heater or a gas range for a kitchen range is taken into consideration, and in the case of use of a district heating or induction range, each situation may be considered and reflected in the daily life log.

In addition, as an environmental kitchen activity metering sensor (200-4b), it detects essential items such as home appliances used in the kitchen, such as the number of openings and closings of home appliance doors of refrigerated home appliances such as refrigerators and kimchi refrigerators. The same environment kitchen activity metering sensor 200-4b may be further provided.

The surveillance sensor unit 200-3 can detect whether the user goes out from the living space. That is, the surveillance sensor unit (200-3) includes a surveillance door lock sensor unit (200-3a), a surveillance surveillance door sensor unit (200-3b), and a surveillance entry/exit activity sensor unit (200-3c). Includes.

The surveillance door lock sensor unit (200-3a) is a sensor for operating the door lock of the user's home, and the surveillance door sensor unit (200-3b) is a sensor for checking whether the user's door is open. The entry activity sensor unit 200-3c includes a sensor that checks whether the user enters or exits through the door, and the surveillance sensor unit 200-3 may also include a separate sensor with an anti-theft function. there is.

In addition, the monitoring sensor unit 200 may further include a safety sensor unit 200-2 for ensuring the safety of the user's living space. The safety sensor unit 200-2 is implemented as a gas fire-related sensor in this embodiment, and the safety sensor unit 200-2 includes a safety gas blocking sensor unit 200-2a and a safety gas sensor unit 200-2b. , and includes a safety fire sensor unit (200-2c). The safety gas shutoff sensor unit (200-2a) detects whether a switch for a gas device such as a gas range in the user space is shut off, and the safety gas sensor unit (200-2b) detects gas leakage, and a safety fire sensor. The unit 200-2c may detect whether a fire has occurred in the user's indoor space and transmit fire-related information to monitor gas leaks and fire-related safety information.

The arrangement and configuration of the monitoring sensor unit 200 is an example, and various modifications are possible within the scope of obtaining various information about the monitoring data from the user's life log data.

Meanwhile, the monitoring terminal unit 100 collects monitoring data including at least monitoring life log data detected as activity from the monitoring sensor unit 200 and transmits it to the monitoring management server 400. This monitoring terminal unit 100 functions as a monitoring management server gateway for information detected by the monitoring sensor unit 200, which detects monitoring data, which is life log data in the user space where the user lives.

The monitoring terminal unit 100 includes a monitoring terminal communication unit 110, a monitoring terminal storage unit 130, and a monitoring terminal control unit 120.

The monitoring terminal communication unit 110 receives monitoring data from the monitoring sensor unit 200 and transmits the received monitoring data to the monitoring management server 400 according to the transmission control of the monitoring terminal control unit 120 as described below. The monitoring terminal storage unit 130 stores the monitoring data received from the monitoring sensor unit 200, and the monitoring terminal control unit 120 transmits the monitoring data received from the monitoring management server 40 to the monitoring management server 400. control.

The monitoring management server 400 receives the monitoring terminal 100 and monitoring data. The monitoring management server 400 confirms and determines the stability of the user's daily state by checking at least the activity index indicating whether the user is active, the bedtime information indicating bedtime information, and the going out index indicating external activity information. The monitoring management server 400 transmits or processes the confirmation decision to at least the guardian terminal unit 300 based on the confirmation decision.

The monitoring management server 40 includes a server communication module 4001, a monitoring module 4100, and a corresponding module 4200, each of which forms a modular configuration, but in some cases, the overall server control unit of the monitoring management server. Various modifications are possible, such as a configuration in which each module is controlled.

The server communication module 4001 receives monitoring data transmitted from the monitoring terminal 100. In some cases, the monitoring management server 40 includes a separate server signal processing module 4003 to signal-process the monitoring data received from the monitoring terminal 100 and send the data to other modules of the monitoring management server 40. Transmission may also be possible.

The monitoring module 4100 uses monitoring data to monitor and confirm the stability of the user's daily state by checking at least the activity index indicating whether the user is active, the sleeping information indicating sleeping information, and the going out index indicating external activity information, and checking the stability state. Analyze and judge.

In this embodiment, the monitoring module 4100 includes a spatial analysis monitoring sub-module 4110, a behavior analysis monitoring sub-module 4120, and an abnormality analysis monitoring sub-module 4130.

The spatial analysis monitoring submodule 4110 checks the user's space occupancy status information by time from at least the monitoring life log data that is detected as activity among the monitoring data. That is, the spatial analysis monitoring sub-module 4110 monitors the user's activity among the user's monitoring data received from the sub-communication module 4001 of the monitoring management server 40 and signal-processed by the sub-signal processing module 4003. By detecting and confirming the user's activity in each living space from life log data, the user's activity level, user's occupancy by space, bedtime information, etc. through information on the space currently occupied by the user and changes in the occupied space, and external Processed data such as entry and exit from or emergency matters can be formed.

The behavior analysis monitoring submodule 4120 confirms and analyzes the user's behavior information using the user's space occupancy status information by time confirmed by the space analysis monitoring submodule 4110. In other words, the behavior analysis monitoring sub-module 4120 uses the user's space occupancy status information by time confirmed by the spatial analysis monitoring sub-module 4110 and calculates the user's status activity amount, bedtime information, activity index for external activities, etc. The index and outing index can be calculated.

The abnormal analysis monitoring sub-module 4130 monitors and analyzes whether the user is in an abnormal state by using the user's time-based space occupancy status information of the spatial analysis monitoring sub-module 4110 and the user's behavior information of the behavior analysis monitoring sub-module 4120. do. That is, the abnormality analysis monitoring submodule 4130 compares the user's behavior information calculated and analyzed by the behavior analysis monitoring submodule 4120, that is, the calculated activity index, sleeping index, and going out index, with preset values, etc. to determine the user's current status. Abnormal state analysis may also be performed.

The response module 4200 transmits the analysis and judgment result of the monitoring module 4100 to at least the guardian terminal unit 300. That is, the response module 4200 is based on analysis judgment results from the spatial analysis monitoring sub-module 4110, the behavior analysis monitoring sub-module 4120, and the abnormal analysis monitoring sub-module 4130 of the monitoring module 4100, If the user is in a normal state, monitoring information about the normal state is reported to at least the guardian's terminal, and if the user is in an abnormal state, follow-up procedures can be executed to take preset emergency response measures.

The response module 4200 includes an abnormality determination response submodule 4210, an alarm execution response submodule 4220, and a reporting response submodule 4230.

The abnormality determination response submodule 4210 executes the corresponding monitoring mode according to the analysis judgment of the monitoring module 4100. The response mode executed in this embodiment includes a rule base response mode based on comparison of preset values, and in some cases, may further include an artificial intelligence response mode when processing data including other users' data.

The rule base response mode includes three response modes in this embodiment, including a normal response mode, a caution response mode, and an emergency response mode.

The normal response mode determines that the user is not in a special abnormal state and executes a reporting response operation that transmits information about the user's life log to the guardian terminal unit 300 on the preset guardian side at a preset time.

The caution response mode determines that the user is not yet in a special abnormal state, but is in a state of life that requires unusual attention, and transmits information about the user's life log to the guardian terminal unit 300 on the preset guardian side at a preset time. , a reporting response operation is performed to advise the guardian or user of careful observation.

The emergency response mode determines that the user is in an abnormal condition that requires a special emergency response, and immediately transmits information about the user's life log indicating that the situation requires emergency response action to the preset guardian terminal unit 300 on the guardian's side. Executes reporting response actions.

The alarm execution response sub-module 4220 executes an emergency response mode when it is determined to be an emergency situation in the analysis judgment of the monitoring module 4100. That is, when it is determined that the response mode to be executed is an emergency response mode, the alarm execution response sub-module 4220 executes an emergency interrupt rather than a regular monitoring operation and requests an emergency response operation to the guardian terminal unit 300 on the guardian's side. A notification is issued, and in some cases, an emergency contact is made to a preset emergency agency, for example, 119 or the institutional server 500 of the hospital where the usual medical examination was performed, to execute an emergency response follow-up operation.

The reporting response sub-module 4230 transmits a report corresponding to the corresponding monitoring mode to the guardian terminal unit 300 according to the analysis judgment of the monitoring module 4100. In this embodiment, in the case of normal response mode and caution response mode, the function of transmitting the user's monitoring life log data or processed reporting data to the guardian terminal unit 300 is performed. In some cases, the reporting cycle and timing are determined in advance. It can be adjusted according to the settings, and settings can also be made possible by adjusting the settings of the user or guardian.

Meanwhile, the response mode executed in this embodiment includes a rule base response mode based on comparison of preset values, and in some cases, may further include an artificial intelligence response mode when processing includes other users' data. Various modifications are possible, such as a configuration in which only the artificial intelligence response mode is executed independently, or a configuration in which it runs simultaneously with the rule base response mode. When further equipped with such an artificial intelligence response mode, the monitoring management server 400 of the deep learning user life status monitoring and management device 10 of the present invention may further include an AI learning module 4300. The monitoring module 4100 may have a structure that allows monitoring data from other users to be input. In other words, the monitoring management server 40 of the deep learning user life status monitoring and management device 10 of the present invention provides a more stable artificial intelligence so that the amount of data can be increased to execute the artificial intelligence response mode through the AI learning module 4300. It is possible to implement an intelligent response mode, but since there are limitations to its implementation only with the user's past monitoring lifelog data, the monitoring module 4100 of the monitoring management server 40 includes other people's monitoring lifelog data through the monitoring module input unit. It adopts a structure that can receive monitoring data.

The AI learning module 4300 uses the user's monitoring data and the input monitoring data of other users to learn AI learning, that is, machine learning, that derives correlations between labels and features in the user's monitoring data and other users' monitoring data. Execute deep learning learning. Here, the user's monitoring data and other users' monitoring data may include EMR (Electronic Medical Record) in addition to monitoring life log data.

In addition, the monitoring data can be modified in various ways, such as forming another data feature by receiving additional climate information about the location of the user's living space through communication with an external server.

Labels in the user's monitoring data and other users' monitoring data are formed for events included in the user's monitoring data and other users' monitoring data, for example, hospital outpatient or hospitalization records of the user or other users, doctor's records, etc. It is a meaningful classified event such as diagnosis information, medication information, call record information of emergency response agencies such as 119, information about going out during late night hours, etc., and is specified through labeling of the user's monitoring data and the monitoring data of other users. It refers to an event, and the feature refers to each feature data of the user's monitoring data and other users' monitoring data, that is, each feature detected through the monitoring sensor unit 200, and the core feature, described later, refers to the user's monitoring data. Among the various features that make up monitoring data and other users' monitoring data, in some cases, features with high correlation calculated through Ai learning such as machine learning and/or deep learning are selected between labels and features derived through data processing described later. refers to

The AI learning module 4300 includes an AI learning model storage unit 4323 and an AI learning data storage unit 4321. The AI learning model storage unit 4323 stores the AI learning model. AI learning is performed using the user's monitoring data and other users' monitoring data by the AI learning model, and the user's monitoring data and other users' monitoring are performed. Correlations between labels and features in the data are derived.

The AI learning models stored in the AI learning model storage unit 4323 are AI learning models such as machine learning or deep learning algorithm models such as Regression, Logistic Regression, Auto Encoder, CNN (Convvolutional Neural Network), and RNN (Reccurrent Neural Network). can be used, and is not limited to a specific model, but various modifications are possible to the extent of improving the efficiency of learning or deriving correlations using each data.

The AI learning data storage unit 4321 stores the correlation between labels and features in the user's monitoring data derived from the AI learning model storage unit 4423 and other users' monitoring data. In other words, by executing AI learning through this AI learning module 4300, a correlation between labels and features is derived.

When the AI learning module 4300 includes an AI learning model storage unit 4323 and an AI learning data storage unit 4321, the monitoring module 4100 may further include a similarity analysis monitoring sub-module 4140. there is.

In the similarity analysis monitoring submodule 4140, core feature data including the user's space occupancy status information by time in the spatial analysis monitoring submodule 4110 and the user's behavior information in the behavior analysis monitoring submodule 4120 and other users' core Based on the feature data, the similarity between the user's user core feature data and the core feature data of other users is confirmed. The similarity analysis monitoring submodule 4140 may check and extract labeling information about other user core feature data that has high similarity between the user's user core feature data and the other user's core feature data. In other words, through AI learning of the AI learning module 4300, the user's core features among the user's monitoring data are compared with the core features of other users to extract highly correlated labeling information to determine whether there are currently events that the user should pay attention to. By analyzing and determining whether an event is likely to occur not only from the current user's past history, but also from monitoring data of other users in different spaces, the process of avoiding risky situations through a proactive or caution process is provided in advance. It can be achieved.

On the other hand, the deep learning user life status monitoring and management device 10 of the present invention extracts core features through AI learning and correlation analysis through the AI learning module 4300 and the similarity analysis monitoring sub-module 4130, In executing the process of avoiding the current user's risky situation by extracting similar labeling information through similarity analysis of the core features of the current user and other users, labels of the user's monitoring data and other users' monitoring data may be required. . In some cases, the AI learning model as an AI engine stored in the AI learning model storage unit 4323 of the AI learning module 4300 for AI learning may include a model for processing structured data, In some cases, it may include a model for processing unstructured data, but in this embodiment, a model for processing semi-structured data when structured data and unstructured data are mixed. It is structured around . In other words, the monitoring data of the current user's past history is easily classified and labeled as a standardized data structure, but the monitoring data or EMR data of other users are highly likely to be unlabeled and unstructured data. In order to improve the reliability of correlation analysis between features and labels of each monitoring data in a mixed state of structured data and unstructured data, the AI learning module 4300 of the present invention is designed for processing semi-structured data. It is implemented by including a model.

At this time, the monitoring management server 400 of the present invention may further include an auto labeling module 4400, which automatically labels the monitoring data of the user and/or other users. In other words, the auto labeling module 4400 uses the user's monitoring data and other users' monitoring data to extract data events from the user's monitoring data and other users' monitoring data and perform labeling. Create and assign labels to monitoring data.

More specifically, the auto labeling module 4400 includes a data event extraction sub-module 4410, a labeling assignment sub-module 4420, and a labeling verification sub-module 4430. The data event extraction submodule 4410 uses the user's monitoring data and other users' monitoring data to extract data events from the user's monitoring data and other users' monitoring data. That is, the data event extraction submodule 4410 extracts meaningful labels from among the user's monitoring data and other users' monitoring data, for example, hospital outpatient or hospitalization records of the user or other users, doctor's diagnosis information, medication information, 119 Information such as call record information from emergency response agencies, information about going out during late-night hours, and welfare worker's consultation records are extracted.

Additionally, the labeling submodule 4420 of the auto labeling module 4400 assigns labels to data events. That is, the labeling sub-module 4420 of the auto-labeling module 4400 classifies or re-extracts aesthetically significant data events from among the data events extracted in the data event extraction sub-module 4410 and assigns a label to them.

At this time, in some cases, the monitoring management server 400 may further include a semantic module 4500 equipped with a semantic engine, and the semantic module 4500 may be used in the data event extraction process and/or the labeling process. In other words, the process of extracting information such as the user's or other users' hospital outpatient or hospitalization records, doctor's diagnosis information, medication information, call record information from emergency response agencies such as 119, information about going out during late night hours, and welfare worker's consultation records, etc. Among them, data events are extracted by executing data mining, such as extracting and classifying semantic text from monitoring data through a semantic module 4500 equipped with a semantic engine, and labels are assigned to meaningful data events among the extracted data events. It could be.

When the labeling process is performed in the labeling sub-module 4420, the labeling verification sub-module 4430 verifies whether a meaningful labeling process has been performed. That is, the labeling verification submodule 4430 can verify the consistency of the labeling assigned by the labeling granting submodule 4420 by comparing it with the previously stored label stored on the monitoring management server 400 and performing overall monitoring on the new label. Considering the frequency of the corresponding label in the data, it can be configured to determine and verify whether or not to grant a new label by comparing it with a preset frequency that can be determined to have a significant frequency.

Through this AI learning process, for example, core features that have a correlation with an event called 'fall' that occurred to another user are extracted, and if the user forms a life pattern similar to the core feature, the event called 'fall' is extracted. By detecting the possibility of occurrence, notifications such as cautions or warnings are sent to the terminal of the user or guardian, and preliminary response measures are taken for matters that are likely to occur based on judgments made based on correlations extracted through big data processing. You can also prevent it from happening.

Hereinafter, the deep learning user life status monitoring and management device control method of the present invention will be described with reference to the drawings.

The deep learning user life status monitoring and management device control method of the present invention includes a provision step (S10), a monitoring step (S20), a response mode determination step (S30), and a response mode execution step (S40).

In the provision step (S10), the monitoring sensor unit 200 includes an activity sensor 200-1 that detects at least activity in the user's living space, and a monitoring life log that detects at least activity from the monitoring sensor unit 200. A monitoring terminal 100 that collects monitoring data including data, and a monitoring terminal 100 that receives the monitoring data and provides at least an activity index indicating whether the user is active, sleeping information indicating sleeping information, and external activity information. A deep learning user life status monitoring and management device 10 is provided, including a monitoring management server 400 that checks the indicating outing index to confirm and determine the stability of the user's daily state and transmits it to at least the guardian terminal unit 300. The deep learning user life status monitoring and management device 10 includes a monitoring sensor unit 200, a monitoring terminal unit 100, and a monitoring management server 400. The specific description thereof is replaced with the previous description and is not duplicated. The explanation is omitted.

Then, the monitoring step (S20) is executed. In the monitoring step (S20), the stability of the user's daily state is monitored and the stability state is analyzed and determined. in other words,

The monitoring sensor unit 200 detects monitoring data including at least monitoring life log data that is detected as active and transmits it to the monitoring management server 400. The monitoring management server 400 of the monitoring module 4100 performs server communication. Monitoring data including at least monitoring life log data detected as activity is received from the monitoring terminal 100 through the module 4001. Using the received monitoring data, the monitoring module 4100 monitors the stability of the daily state and analyzes and determines whether it is stable. That is, the monitoring module 4100 monitors and confirms the stability of the user's daily state by checking at least the activity index indicating whether the user is active, the bedtime information indicating bedtime information, and the going out index indicating external activity information, and monitors the stability state. Analytical judgment is made at (4100).

More specifically, the monitoring step (S20) includes a monitoring input step (S21), a spatial analysis monitoring step (S23), a behavior analysis monitoring step (S25), and an abnormality analysis monitoring step (S27). Includes.

First, in the monitoring input step (S21), monitoring data is received and input from the monitoring terminal 100 through the server communication module 4001 of the monitoring management server 400. Here, the monitoring data includes monitoring life log data, and the monitoring life log data indicates at least activity and is detected by the monitoring sensor unit 200. That is, the monitoring data includes monitoring life log data, and in some cases, in addition to directly sensed data, the monitoring data may also include the previously described EMR (Electronic Medical Record).

The spatial analysis monitoring step (S23) calculates spatial information about the space occupied by the user from the user's monitoring data, that is, monitoring life log data. That is, the spatial analysis monitoring sub-module 4110 determines the user's space occupancy status by time from at least the monitoring life log data that is detected as active among the monitoring data received and input in the spatial analysis monitoring step (S23). Check the information. In other words, by using data that detects the user's location by time, we can determine which space the user occupied at what time, for example, whether the user was mainly located in the living room or kitchen in the morning, and the bedroom space in the afternoon. From location information by time zone, such as the occupation time or frequency, it is possible to determine the user's 24-hour space occupancy rate or space occupancy rate by time zone, that is, spatial information including space occupancy status information by time.

Then, the behavior analysis monitoring step (S25) is executed. In the behavior analysis monitoring step (S25), the behavior analysis monitoring submodule 4120 extracts and analyzes the user's behavior information from the spatial information acquired in step S23. That is, in the behavior analysis monitoring step (S25), the behavior analysis monitoring submodule 4120 collects the user's behavior information using spatial information including the user's space occupancy status information by time confirmed by the spatial analysis monitoring submodule 41110. Confirm and analyze. Since the user's movement has no choice but to take a continuous motion structure, it is possible to extract the user's behavior information from the spatial information detected by time zone through the direction in which the user moves and information about the user space in which the user is located. For example, when the user's occupied space moves through the bedroom space, the living room space, and then the kitchen space and stays in the kitchen space for a certain period of time, behavioral information is extracted, such as determining that the user will perform cooking actions such as eating or cooking. This is possible.

After the behavior analysis monitoring step (S25) is executed, the anomaly analysis monitoring step (S27) is executed. In the anomaly analysis monitoring step (S27), the anomaly analysis monitoring submodule 4130 uses spatial information, especially time-specific space occupancy status information and behavior information, and monitors and analyzes the user's normal state, caution state, or abnormal state to determine the user's normal state, caution state, or abnormal state. do. At this time, when analyzing behavior information, activity information, sleeping information, going out information, and restroom information are used to determine whether the current user's status is normal response mode, caution response mode, or emergency response mode. In making the decision, a judgment process regarding the current user's current state may be performed through comparison with a preset reference value.

As described above, the anomaly analysis monitoring step (S27) may include the rule base anomaly analysis monitoring step (S271). In the rule base abnormality analysis monitoring step (S271), the abnormality analysis monitoring sub-module 4130 receives monitoring data from the monitoring terminal 100 and includes at least an activity index indicating the user's activity, bedtime information indicating bedtime information, and external data. The stability of the user's daily state is confirmed and determined by comparing the going out index representing activity information with the set standard value.

For example, if the space occupancy in the user's living space drops sharply, the space occupancy in the sleeping space sharply decreases, and the user's activity level drops sharply, the sleeping index increases, the activity index decreases, and the frequency of going out decreases. This leads to a decline in the outing index. Here, this index is calculated in the behavior analysis monitoring sub-module 4120, and the abnormality analysis monitoring sub-module 4130 determines whether there is an abnormality according to the rule base criteria in the rule base abnormality analysis monitoring step (S271), which the user Comparison with a reference value such as the user's own reference value, for example, the cumulative average value over a certain period such as the cumulative average value for 10 years, the user's cumulative average value for the same quarter of the previous year, or the cumulative average value during a certain period of the general population of the same age group or a person with the same disease A comparative analysis step is performed using a rule base method to analyze whether the current user is in an abnormal state.

Then, the response mode determination step (S30) is executed. In the response mode determination step (S30), the response mode is determined by the response module 4200 according to the analysis determination result of the monitoring module 4100, which is the step determined in step S20. ), it is determined whether the user's current execution response mode is a normal response mode, a caution response mode, or an emergency response mode.

More specifically, the response mode determination step (S30) includes a rule base response mode determination step (S30-1). In the rule base response mode determination step (S30-1), the response module 4200 analyzes and monitors rule base abnormalities. A response mode is determined according to the rule base abnormality analysis monitoring confirmation result confirmed in step S271.

The rule base response mode determination step (S30-1) includes an analysis monitoring data input step (S31), a verification result judgment step (S33), and a caution determination step (S35). In the analysis monitoring data input step (S31), the rule base abnormality analysis and monitoring confirmation result confirmed in the rule base abnormality analysis and monitoring step (S271) is input. Analysis monitoring confirmation results are entered.

Then, the confirmation result determination step (S33) is executed in the abnormality judgment response submodule 4210. The abnormality judgment response submodule 4210 of the response module 4200 determines whether the rulebase abnormality analysis monitoring confirmation result is normal. It is judged. If the abnormality determination response submodule 4210 determines that the rulebase abnormality analysis monitoring confirmation result is not normal, the abnormality response submodule 4210 sets the rulebase mode to be executed to the emergency response mode (step S37). . On the other hand, if the abnormality determination response submodule 4210 determines in step S33 that the rule base abnormality analysis monitoring confirmation result is normal, the control flow is switched to step S35 and the caution determination step (S35) is executed.

That is, in the caution determination step (S35), the response module 4200 determines whether the rule base abnormality analysis monitoring confirmation result requires caution. In the caution determination step (S35), the rule base abnormality analysis monitoring confirmation result is caution. If it is determined that there is a need, the anomaly response submodule 4210 sets the rule base mode to be executed as a caution response mode (step S36), and in the caution determination step (S35), the rule base anomaly analysis monitoring confirmation result indicates that caution is required. If it is determined that there is no, the abnormal response submodule 4210 sets the rule base mode that should be executed as the normal response mode (step S38).

After this correspondence mode determination step (S30), the correspondence mode execution step (S40) is executed, and the correspondence mode determined in the correspondence mode determination step (S30) is executed. In other words, the corresponding response mode determined according to the result of determining whether the user's current execution response mode is a normal response mode, a caution response mode, or an emergency response mode is executed.

For example, if it is determined that the response mode to be executed is a normal response mode, monitoring data including the user's monitoring lifelog data is continuously detected and monitored through the monitoring sensor unit 200 without a separate interrupt. The terminal unit 100 transmits the information to the monitoring management server 400, and at a preset time point, at least the guardian terminal unit 300 on the guardian side executes reporting of the user on the same day or on a preset periodic basis. That is, the user's previous day's activity amount, space occupancy for each living space, sleeping index, going out index, activity index, bathroom index, etc. are provided to the guardian terminal unit 300 in the form of a text or push-up message, and at the same time, Simultaneous provision can also improve the guardian's ease of recognition of the user's status.

In addition, if the execution response mode to be executed is the caution response mode, the routine of the previous normal response mode is processed, but at least the same day or preset periodic reporting is provided to the guardian terminal unit 300 on the guardian side at a preset time. In addition, steps are taken to send additional caution messages. That is, the user's previous day's activity amount, space occupancy for each living space, sleeping index, going out index, activity index, bathroom index, etc. are provided to the guardian terminal unit 300 in the form of a text or push-up message, and at the same time, In addition to simultaneous provision, it is also possible to increase the need for guardians to pay attention to conditions that require user attention by transmitting a caution notification function for insufficient instructions of the user. For example, when the user's going out index decreases rapidly due to a decrease in the user's going out frequency, the user's going out frequency decreases by notifying the user of the change in going out status and providing comparative information with the frequency compared to the same period of time for people of the same age group or with the same disease. Additionally, information may be provided to enable an inference to be made as to whether the change is due to the user's own physical changes or climatic conditions due to a sudden drop in winter weather. Meanwhile, in some cases, in the case of the caution response mode, various modifications are possible, such as deviating from the routine of the normal response mode and transmitting an attention response message through an interrupt to the guardian terminal unit 300 on the guardian's side.

In addition, when the execution response mode to be executed is an emergency response mode, the routine of the preceding normal response mode is switched to the emergency mode through an interrupt, and an emergency response notification message is transmitted to the guardian terminal unit 300 on the guardian's side. , an emergency response request notification message can be transmitted to the institutional server 500 of the institution, such as a regular outpatient hospital or 119.

Meanwhile, in the previous embodiment, the deep learning user life status monitoring and management device includes a rule base-based response mode determination and response mode execution steps (S340 and S40) with a structure including a monitoring module 4100 and a response module 4200. Although the configuration was adopted as a base method, the present invention is not limited to this and various modifications are possible.

For example, as described above, the response mode determination and response mode execution steps executed in this embodiment include a rule base response mode according to comparison of preset values, and in some cases, data of other users is included and processed. In this case, an artificial intelligence response mode may be further included, and various modifications are possible, such as a configuration that runs simultaneously with the rule base response mode, or a configuration in which only the artificial intelligence response mode is executed independently.

First, in the case where such an artificial intelligence response mode step is further provided, monitoring of the deep learning user life status monitoring and management device 10 provided in the providing step (S10) of the deep learning user life status monitoring and management device control method of the present invention The management server 400 may further include an AI learning module 4300, and the monitoring module 4100 may have a structure in which other users' monitoring data can be input. The monitoring management server 40 of the deep learning user life status monitoring and management device 10 of the present invention provides more stable artificial intelligence response so that the amount of data can be increased to execute the artificial intelligence response mode through the AI learning module 4300. It is possible to implement this mode, but since there are limitations to its implementation only with the user's own past monitoring lifelog data, the monitoring module 4100 of the monitoring management server 40 is equipped with a monitoring mode that includes other people's monitoring lifelog data through the monitoring module input unit. The fact that it takes a structure to receive data is the same as described above as an example.

As a premise for executing this artificial intelligence response mode, the monitoring step (S20) may include an artificial intelligence abnormality analysis monitoring step (S273). In this embodiment, the artificial intelligence anomaly analysis monitoring step (S273) may be executed after or simultaneously with the rule base anomaly monitoring step (S271). In the artificial intelligence anomaly analysis monitoring step (S273), the user's user core feature data and other The similarity of user core feature data can be confirmed, and labeling information about other user core feature data can be confirmed and extracted. Here, these steps are based on core feature data including the user's time-based space occupancy status information of the spatial analysis monitoring submodule 4110 and the user's behavior information of the behavior analysis monitoring submodule 4120, and core feature data of other users. do. That is, the feature data is each data information of the monitoring data, for example, detected by the activity sensor unit 200-1, the environment sensor unit 200-4, etc., and transmitted to the monitoring terminal unit 100. Sensor value data refers to the characteristics of everyday life, and core feature data refers to key features selected among sensor value data items that are judged to have a high degree of correlation, that is, a high degree of correlation, through correlation analysis between the event and feature data, which will be described later. refers to data.

Based on the core feature data including the user's time-based space occupancy status information of the spatial analysis monitoring sub-module 4110 and the user's behavior information of the behavior analysis monitoring sub-module 4120, and the core feature data of other users, the monitoring module ( The similarity analysis monitoring sub-module 4140 of 4100) checks the similarity between the user's user core feature data and other users' core feature data in the artificial intelligence abnormality analysis monitoring step (S273) and compares it to other users' core feature data to determine the relevance. Labeling information can be confirmed and extracted, and the monitoring terminal 100 and monitoring data are received using this highly correlated labeling information, and at least an activity index indicating whether the user is active, bedtime information indicating bedtime information, and external activity. The stability of the user's daily state with respect to the outing index indicating information may be confirmed and determined. In other words, in the artificial intelligence anomaly analysis monitoring step (S273), it is checked whether highly relevant labeling information exists by comparing the core feature data of the current user and other users and checking for similarity, and responding to the extracted labeling information. By comparing the core feature data of other users and the current current user monitoring data, monitoring is performed to see if other highly relevant user labels can occur in the current user.

In this case, in addition to the monitoring stage, specific configuration changes are also involved in the response mode determination and response mode execution stages (S30, S40). That is, the response mode determination step (S30) may include an artificial intelligence response mode determination step (S30-2) in addition to the rule base response mode determination step described above. In the artificial intelligence response mode determination step (S30-2), the response mode is determined according to the results of the artificial intelligence anomaly analysis monitoring confirmation confirmed in the artificial intelligence anomaly analysis monitoring step (S273).

More specifically, the artificial intelligence response mode determination step (S30-2) may include an analysis monitoring data input step (S301), a core feature similarity high determination step (S303), and a core feature similarity medium determination step (S305). there is. In this case, in the analysis monitoring data input step (S301), the artificial intelligence anomaly analysis monitoring confirmation result confirmed in the artificial intelligence anomaly analysis monitoring step (S273) is input.

Then, the core feature similarity high determination step (S303) is executed. In the core feature similarity high determination step (S303), the artificial intelligence error of the similarity analysis monitoring submodule 4140 of the monitoring module 4100 in the response module 4200 is performed. As a result of the analysis monitoring confirmation, it is determined whether the core feature similarity, which indicates core feature similarity, is higher than the preset core feature high similarity. That is, when the corresponding module 4200 determines that the core feature similarity is higher than the preset core feature high similarity, the control flow switches to step S308 and the artificial intelligence mode (AI mode) to be executed is set to the high similarity pattern mode ( S308).

On the other hand, if the corresponding module 4200 determines that the core feature similarity is not higher than the preset core feature high similarity, the control flow switches to step S305. In the core feature similarity medium determination step (S305), the response module 4200 determines whether the core feature similarity indicating core feature similarity as a result of artificial intelligence anomaly analysis monitoring confirmation is higher than the preset core feature medium similarity, and the response module 4200 If it is determined at 4200 that the core feature similarity is not higher than the preset core feature medium similarity, the control flow switches to step S310 and the artificial intelligence mode (AI mode) to be executed is set to the low similarity pattern mode (S310).

On the other hand, in the core feature similarity medium determination step (S305), if the response module 4200 determines that the core feature similarity indicating core feature similarity as a result of artificial intelligence abnormality analysis monitoring confirmation is higher than the preset core feature medium similarity, the control flow is The artificial intelligence mode (AI mode) to be switched to step S309 and executed is set to medium similarity pattern mode (S309).

According to the response mode confirmed and judged in this way, the response mode selected in step S40 is executed. As in the previous response decision step, the response mode execution step is performed in each mode, that is, the rule base response mode execution step and the artificial intelligence response mode execution step. The steps may be executed individually, or in some cases, simultaneous notification actions may be performed as integrated execution steps, and by giving the user a choice for each execution type and the function of selecting an operation mode, an operation mode suitable for the user, In other words, various modifications are possible depending on the design specifications, such as selectively executing either a rule base-based rule base response execution mode or an artificial intelligence-based artificial intelligence response execution mode.

Meanwhile, in the case of the execution type of the artificial intelligence response mode of the present invention described above, it may be executed in the direction of utilizing a large amount of big data of other users to resolve the restrictions accompanying the application of only the user's data. As described, whether it is the user's monitoring data or other users' monitoring data, each monitoring data has meaningful meaning so that the computational load of each data can be reduced, enabling a more rapid response and improving reliability according to data classification. A label for feature data is formed, and in another embodiment of the present invention, components for automating the labeling task of forming such a label may be further provided.

That is, the monitoring management server 400 provided in the provision step of the present invention may further include an auto-labeling module 4400, and the auto-labeling step (S50) may be executed in the auto-labeling module 4400. In the auto labeling step (S50), which is executed at least after the provision step (S10), data events are extracted from the user's monitoring data and other users' monitoring data using the user's monitoring data and other users' monitoring data, and labeling is performed. A label is created. That is, the auto-labeling step (S50) of this embodiment may be executed at any stage after the provision step (S10), preferably in the monitoring step (S20) when the monitoring data input after the monitoring input step (S21) is executed exists. In this case, it is desirable to run them in parallel.

More specifically, the auto labeling step (S50) includes an auto labeling monitoring input step (S51), a data event extraction step (S53), a preliminary labeling step (S55), a labeling verification step (S57), and a labeling confirmation step. Includes step S59.

In the auto labeling monitoring input step (S51), monitoring data is received and input to the auto label module (4400). This input monitoring data can be done in the monitoring step (S20) as described above, or directly through a separate input unit. Various modifications are possible, such as a configuration input to the monitoring management server 400. As described above, the input monitoring data may be the user's past and present detected monitoring lifelog data, other users' monitoring lifelog data, or the electronic obligations of this user and/or other users. Various data that provides meaningful information about the user's daily life may be selected, such as records (EMR).

After the monitoring data is input in the auto labeling monitoring input step (S51), the data event extraction step (S53) is executed. In the data event extraction step (S53), the data event extraction sub-module 4410 uses the monitoring data input in the auto labeling monitoring input step (S51), that is, the user's monitoring data and other users' monitoring data, to obtain the user's monitoring data and Extract data events from other users' monitoring data. As described above, in the data event extraction step (S53), the data event extraction sub-module 4410 selects a meaningful label among the user's monitoring data and the monitoring data of other users, for example, the user or another user's hospital outpatient department. Significant information or information events are extracted, such as hospitalization records, doctor's diagnosis information, medication information, call record information from emergency response agencies such as 119, information about going out during late-night hours, and welfare worker's consultation records, etc. These are extracted from the user or other users. Information or information events that affect or reflect the impact of daily life are classified and extracted as meaningful events so that the correlation between events and feature data can be studied and reviewed in the AI learning module.

In the preliminary labeling step (S55), the labeling submodule 4420 assigns a label to the data event extracted in the data event extraction step (S53). At this time, in some cases, the monitoring management server 400 may further include a semantic module 4500 equipped with a semantic engine, and the semantic module 4500 may be used in the data event extraction process and/or labeling process. , in the process of extracting information such as the user's or other users' hospital outpatient or hospitalization records, doctor's diagnosis information, medication information, call record information from emergency response agencies such as 119, information about going out during late night hours, and welfare worker's consultation records, etc. , Extract data events by performing data mining, such as extracting and classifying semantic text from monitoring data through the semantic module 4500 equipped with a semantic engine, and assign labels to meaningful data events among the extracted data events. It may be possible.

After the preliminary labeling step (S55) is executed, the labeling verification step (S57) is executed. In the preliminary labeling step (S55), the consistency of the labeling assigned through the labeling submodule 4420 is verified by the labeling verification submodule. Module 4430 verifies.

The execution method of the labeling verification step (S57) can be selected in various ways depending on the design specifications. In the labeling verification step (S57) in this embodiment, the labeling verification submodule 4430 performs verification based on the name of the label. Take . For example, in this embodiment, in the labeling verification step (S57), the labeling verification submodule 4430 compares the label with the previously stored label stored on the monitoring management server 400 to verify whether a meaningful labeling process has been performed. Use the method. In other words, the previously stored label name is used first, and if the name of the preliminarily assigned label does not exist among the names of the previously stored labels, a new label name is assigned, and the frequency is accumulated by creating a new label name for the new label name. I do it. The frequency of the label in this overall monitoring data is checked, and the frequency of the label given a new name is compared with the preset frequency. The preset frequency determines that the label corresponding to the name has a meaningful frequency. It is a possible reference value and can be adjusted according to the design method. In this way, data events are extracted and labels are temporarily assigned to them, and it is checked whether the label is a label name that exists in the existing data or a label to which a new name is assigned, and if it is judged to be a label with a significant frequency, the newly named name is new. You can also take a configuration that is judged and verified by giving it a label. In some cases, various modifications are possible, such as a step of simply assigning a separate check label to the preliminary label that has not reached the frequency so that the manager can subsequently check it.

Then, a labeling confirmation step (S59) is executed to confirm the labeling assigned in the preliminary labeling step (S55) according to the verification result of the labeling verification step (S59).

In this embodiment, verification was performed focusing on the name of the label, but this is just an example and various choices are possible in the range where data events are extracted more accurately and their labeling is performed.

On the other hand, in the above embodiment, a configuration using monitoring data of other users in addition to the monitoring data of the present user has been described. The deep learning user life status monitoring and management device and its control method of the present invention are used to protect personal information such as the Personal Information Protection Act. Components for de-identification function for protection may be further provided. Descriptions that overlap with the content described above will be replaced with the previous technology, and the following description will focus on the configuration that performs the de-identification function.

That is, the deep learning user life status monitoring and management device 10 of the present invention with a de-identification function includes a monitoring sensor unit 200, a monitoring terminal unit 100, and a monitoring management server 400, and monitors The fur management server 400 includes an AI learning module 4300 and further includes a DID module 4600.

The monitoring sensor unit 200, the monitoring terminal unit 100, and the monitoring management server 400 are the same as described above. The AI learning module 4300 is a component further provided in the monitoring management server 400, and allows other users' monitoring data to be input, and uses the user's monitoring data and other users' monitoring data to provide the user's monitoring data and other users' monitoring data. AI learning learning that derives correlations between labels and features in the user's monitoring data is possible, and the specific implementation of this is as described above.

At this time, the monitoring management server 400 of the present invention is further equipped with a DID module 4600. The DID module 4600 de-identifies other users' monitoring data according to the minimum de-identification setting. Perform de-identification and re-identify the de-identified monitoring data of other users to check whether other users can be identified.

More specifically, the DID module 4600 includes a DID identification sub-module 4610, a REID verification module 4620, and a DID similarity verification module 4630.

Other user monitoring data from other users is received and input into the identification submodule 4610. The di-identification submodule 4610 performs di-identification to de-identify the monitoring data of other users according to the minimum de-identification setting. Here, de-identification refers to personal information that can identify other users among other users' monitoring data, such as name, address, resident registration number, vehicle number, telephone number, cell phone number, etc. This refers to removing or executing an encrypted screening operation of the personal information to prevent the execution of a process that enables specific, presumption, or identification. This de-identification is an important process to prevent the leakage of personal information. am.

The di-identification sub-module 4610 according to an embodiment of the present invention includes a di-identification sub-minimum requirement confirmation unit 4611, a di-identification sub-feature check unit 4613, and a di-identification sub-minimum requirement confirmation unit 4611. It includes a transaction sub-crypto unit 4615 and a di-identification sub-screening unit 4616.

The di-identification sub-minimum requirement confirmation unit 4611 of the di-identification sub-module 4610 verifies the minimum requirements required to prevent re-identification of other users' monitoring data. Here, the minimum requirements refer to essential items that must be removed technically or legally to protect personal information, and the minimum requirements use data stored in the minimum settings storage module 4640 provided in the DID module 4600. In some cases, the minimum setting storage module 4640 is updated with changes in minimum requirements due to changes in laws and regulations, and when a change in minimum conditions occurs, the minimum setting storage module 4640 immediately checks the di-identification sub-minimum requirement confirmation unit of the di-identification sub-module 4610 ( 4611) checks this and updates the de-identification function of the de-identification submodule 4610 to enable application of the latest regulations.

The di-identification sub-feature check unit 4613 non-displays the personal information-related data features of other users' monitoring data based on the minimum requirements confirmed by the di-identification sub-minimum requirement check unit 4611. Check the data features associated with non-displayed personal information. In other words, among the monitoring data of other users, personal information that can identify other users, such as name, address, resident registration number, vehicle number, telephone number, cell phone number, etc., is an item based on the minimum requirements for non-display. Check the data features related to personal information, such as name, address, resident registration number, phone number, cell phone number, vehicle number, skin color, race, age, disease, etc., among the data features related to personal information, such as name, address, resident registration number, and phone number. , identify data features associated with de-identified personal information, which are items that are de-identified according to minimum requirements, such as cell phone number and vehicle registration number.

The di-identification sub-screening unit 4617 non-displays the personal information-related data features that are subject to non-display among the personal information-related data features identified in the di-identification sub-feature checking unit 4613. At this time, the non-display process may be performed for all data features related to personal information subject to non-display, or may be performed for only some information. In other words, the entire resident registration number may be encrypted, or in some cases, only specific parts, such as the last digit of the resident registration number or the first two digits of the resident registration number, may be screened.

Meanwhile, in some cases, the di-identification sub-module 4610 may further include a component that executes an encryption process for the non-displayed portion. In other words, the di-identification sub-screening unit 4617 non-displays the personal information-related data features to be non-displayed among the personal information-related data features identified in the di-identification sub-feature check unit 4613, but the non-displayed portion A di-identification sub-crypto unit 4615 that performs encryption may be further provided.

That is, the di-identification sub-crypto unit 4615 encrypts the personal information-related data features to be non-displayed among the personal information-related data features confirmed in the di-identification sub-feature check unit 4613. At this time, as in the case of the previously screened portion, the encryption process may be performed on all data features related to personal information to be undisplayed, or may be performed on only some information. In other words, various modifications are possible, such as the entire resident registration number may be encrypted, or in some cases, only specific parts such as the last digit of the resident registration number or the first two digits of the resident registration number may be executed.

The REID verification module 4620 re-identifies de-identified monitoring data of other users and determines whether the user can be re-identified from the monitoring data. The re-identification decision process of the REID verification module 4620 includes Requires DID similarity confirmation module (4630). In other words, the DID similarity confirmation module 4630 identifies data from de-identified monitoring data of other users, and more specifically, from non-identified data features other than the de-identified data features of de-identified monitoring data of other users. In the process of checking the possibility of extraction of other users before being identified, the probability of matching other users before being de-identified is calculated from the de-identified other user monitoring data, and the REID confirmation module 4620 determines the preset reference value. The possibility of re-identification is determined by comparing (SIDS) with the probability (SID) of matching other users before de-identification from de-identified other user monitoring data.

in other words. Regarding the monitoring data of other users for a total of 20,000 people, including A, personal information related data features such as name, address, resident registration number, phone number, cell phone number, vehicle number, skin color, race, age, disease, etc., name, Only addresses, resident registration numbers, phone numbers, cell phone numbers, and vehicle numbers meet the minimum requirements, and after screening and encryption for de-identification, these are traced back from the data and from the data features of the remaining de-identified monitoring data. This is done by calculating the perceivable probability for A and comparing whether its probability value is less than a preset reference value. For example, after screening and encryption through data features on disease and age among de-identified data features, the number of other users with the same disease and age is calculated, the reciprocal of this number is calculated as the probability of identification, and this is calculated as a dictionary. If it is lower than the set reference value, for example, 5%, the probability of A being identified from unidentified data features is determined to be extremely small, and an additional identification step for additional data features is set not to be provided.

On the other hand, in the above example, if the identification probability of identifying A is judged to be greater than the preset value, the identification probability of A is determined to be high and the additional de-identification process is repeated for other data features according to the preset order. do.

This de-identification process may be performed for all users or may be applied to local users. However, since there is a difference in the number of users when executed for all users and when executed for local users, the preset value compared to the identification probability value may change depending on the type.

In other words, the identification probability value of the DID similarity confirmation module 4630 is transmitted to the REID confirmation module 4620 to determine whether re-identification is possible or not through comparison with the identification probability of the currently de-identified item and a preset value. The judgment is made, and based on the judgment result, the REID confirmation module 4620 performs additional screening or encryption of personal information-related data features or maintains the current screening encryption state without the need for additional actions.

Below, the diidentification process will be described with reference to the drawings. Matters regarding the deep learning user life status monitoring and management device of the present invention will be replaced with the above, and redundant description will be omitted.

First, a DI step (S60) performed at least after the providing step (S10) of the present invention may be further included. In the DI step (S60), the DI module 4600 of the monitoring management server 400 performs de-identification to de-identify the monitoring data of other users according to the minimum de-identification setting. , de-identified monitoring data of other users is re-identified to confirm whether other users can be identified.

More specifically, the DI step (S60) includes a DI monitoring input step (S61), a DI identification step (S63), a minimum setting requirement verification step (S65), and a verification response step (S67).

In the DID monitoring input step (S61), the monitoring data is received and input to the DID module 4600, and in the DID identification step (S63), the monitoring data of other users is de-identified and, according to the minimum setting, the DID submodule. At 4610, de-identification is performed to de-identify other users' monitoring data. Then, the minimum setting requirement verification step (S65) is executed. In the minimum setting requirement verification step (S65), the de-identified monitoring data of other users is re-identified and compared with the minimum setting requirements to determine whether re-identification is possible. It is confirmed and verified, and in the verification response step (S67), the verification response items set according to the requirements verification result in the minimum set requirement verification step (S65) are executed.

More specifically, the di-identification step (S63) includes a di-identification sub-minimum requirement confirmation step (S631), a di-identification sub-feature check step (S633), and a di-identification sub-screening step ( S635).

In the di-identification sub-minimum requirement confirmation step (S631), the di-identification sub-minimum requirement confirmation unit 4611 of the di-identification sub-module 4610 re-identifies the monitoring data of the other user. The minimum requirements required to make it impossible to configure are identified.

Then, the di-identification sub-feature check step (S633) is executed, and the di-identification sub-feature check unit 4613 of the di-identification sub-module 4610 checks the di-identification sub-minimum requirement. Based on the minimum requirements identified in unit 4611, unmarked personal information-related data features of other users' monitoring data are identified.

Then, the di-identification sub-screening step (S635) is executed. In the di-identification sub-screening step (S635), the di-identification sub-screening unit 4617 of the di-identification sub-module 4610 Among the personal information-related data features identified in the identification sub-feature check unit 4613, the personal information-related data features to be non-displayed are non-displayed.

Meanwhile, in the de-identification step (S63), in addition to screening, an encryption process may be additionally performed to prevent personal information leakage due to data leakage from the monitoring management server to the outside. That is, the di-identification step (S63) may further include a di-identification sub-crypto step (S367), where the di-identification sub-crypto step (S367) The identification sub-crypto unit 4615 encrypts the personal information-related data features to be non-displayed among the personal information-related data features identified in the di-identification sub-feature check unit 4613.

The minimum setting requirement verification step (S65) includes a REID confirmation step (S651), a DID similarity confirmation step (S653), and a ID completion confirmation step (S655). In some cases, a minimum setup requirements update step (S650) may be further included to check and update the latest minimum setup requirements before the REID confirmation step (S651). In the minimum setup requirements update step (S650), the minimum setup requirements update step (S650) Data stored in the settings storage module 4610 can be used.

The REID confirmation step (S651) may be executed in the REID confirmation module 4620, where the data features of a specific age and disease are randomly selected by a preset number of people among the data features of disease and age. Identify third-party users and perform re-identification. For example, for monitoring data of other users other than target user A, for example a total of 20,000 people, information such as name, address, resident registration number, phone number, cell phone number, vehicle number, skin color, race, age, disease, etc. Among the data features related to personal information, only name, address, resident registration number, phone number, cell phone number, and vehicle number are set to meet the minimum setting requirements, and after screening and encryption to de-identify them, the remaining non-identification data are collected from these data. As a data feature of unidentified monitoring data, the age and disease are random or the same value as the age or disease with a certain change before and after the age applied to user A is specified and corresponds to this. Re-identification is performed to identify third-party users. That is, after screening and encryption through data features on disease and age among de-identified data features, a preset number of other users with the same disease and age are identified and re-identification is performed. That is, in the REID confirmation step (S651), as previously described with respect to the re-identification process, other users' monitoring data is checked through non-identified data features in the REID confirmation module 4620 of the DI module 4600. A re-identification process is carried out.

Then, in the DID similarity confirmation step (S653), a DID similarity (SID) indicating the possibility of identification of the other user is calculated from the monitoring data of the re-identified other user. This DID similarity is calculated by the DID similarity confirmation module 4630. It is calculated from That is, in the DID similarity confirmation step (S653), the DID similarity (SID) is calculated. For example, the SID is calculated by calculating the number of specified other users compared to the total number of users in the REID confirmation step (S651) as the probability of identification. may be calculated. For example, after screening and encryption through data features on disease and age among de-identified data features, the number of other users with the same disease and age is calculated, and the reciprocal of this number is identified as SID similarity. It is calculated as a probability.

Then, the DID completion confirmation step (S655) is executed in the REID confirmation module 4620, and in some cases, it may be performed in another sub-module of the DID module.

The DI ID completion confirmation step (S655) includes a DI ID similarity comparison judgment step (S6551) and a DI ID completion confirmation step (S6553).

First, in the DID similarity comparison judgment step (S6551), the REID confirmation module 4620 compares the calculated DID similarity (SID) with the preset standard DID similarity (SIDS). According to the result of comparing the DI similarity (SID) and the preset standard DI similarity (SIDS) in step S6551, the control flow is transferred to the DI ID completion check step (S6553) to check whether the DI identification process is complete. The confirmation step (S6553) includes a DI completion confirmation step (S65531) and a DI ID non-completion confirmation step (S65533). As a result of the comparison, if the DI similarity (SID) is less than the preset standard DI similarity (SIDS), the control flow is It transitions to the DID completion confirmation stage (S65531) and determines that the DID identification process has been properly completed and no additional de-identification measures are necessary. For example, the DID similarity as the identification probability is the preset standard DID similarity (SIDS). If it is lower than the exemplary value of 5%, the probability of A being identified from the unidentified data feature is determined to be extremely small, and the DID completion confirmation step (S65531) is set to not require an additional identification step for additional data features. ) is confirmed.

On the other hand, as a result of comparing the DI similarity (SID) and the preset standard DI similarity (SIDS) in step S6551, if the DI similarity (SID) is not less than the preset standard DI similarity (SIDS), that is, if the value is greater than or equal to the preset standard DI similarity (SIDS), the control flow is It is converted to the non-complete confirmation stage (S65533), and it is determined that an additional de-identification process must be performed and that additional de-identification measures are unnecessary.

Then, the control flow switches to step S67, and execution is performed according to the selected confirmation result among the DID completion confirmation step (S65531) and the DID non-completion confirmation step (S65533) of the DID completion confirmation step (S6553). If the step (S65531) is checked, the control flow to end the de-identification process is performed, and if the DID non-completion check step (S65533) is checked, the control flow is sent to step S63 to further execute the de-identification process, or a preset It can also be done by sequentially excluding non-core features by lowering the DID similarity.

On the other hand, the present invention uses a call event, that is, a request event made by the user or another user to receive emergency or help, to probabilistically provide the user's risk level on the day, thereby minimizing risk by requiring more careful attention. You can also take a configuration to prevent it. That is, the deep learning user life status monitoring and management device 10 of the present invention is provided with a monitoring sensor unit 200 including an environment sensor unit 200-4. Monitors the monitoring data by detecting at least water, gas and electricity usage and illumination conditions and the user's breathing rate and heart rate, which reflect the user's daily living conditions in the user's living space, and generating monitoring biolog data indicating the user's physical condition. Bio log data is included, and this monitoring bio log data is collected by the monitoring terminal 100 described above.

More specifically, Figure 20 shows an example of a part of the deep learning user living status monitoring and management device installed in the living environment of the user or other users. The environment sensor unit (200- 4) is placed. The environment sensor unit (200-4) disposed in this living space is an environment metering sensor (200-4b, 200) that detects at least water, gas, and electricity usage reflecting the user's daily living conditions in the living space. -4d, 200-4e, 200-4f). In addition, the environment sensor unit 200-4 includes an environment illuminance sensor 200-4a that detects the illuminance state of the user's daily life in the user's living space, and detects the user's breathing rate and heart rate. Includes an environment laser sensor (200-4g) that At this time, the laser sensor (200-4g) enables the acquisition of monitoring data of bio log data through the user's breathing rate or heart rate through a non-contact method. This environment laser sensor (200-4g) uses Doppler or IR- A variety of methods can be selected within the range of being able to monitor the breathing rate or heart rate of the user or other users in a non-contact state, such as using the UWB (Impule radio ultra wideband) method, and thereby confirming the bio-index indicating the physical condition. .

In addition, the monitoring management server 400 receives monitoring data from the monitoring terminal 100 and receives at least an activity index indicating whether the user is active, bedtime information indicating sleeping information, an outing index indicating external activity information, and the user's breathing rate. And by checking the bio-index indicating the physical condition through heart rate, the stability of the user's daily state is confirmed and judged and transmitted to at least the guardian terminal unit 500. The monitoring management server 400 in this embodiment is shown in FIG. 22. As described above, it may include a monitoring module 4100 that further includes an event update and classification sub-module 4150. The monitoring module 4100 uses monitoring data to check the stability of the user's daily state by checking at least the activity index indicating whether the user is active, bedtime information indicating sleeping information, and going out index indicating external activity information, and the bio index. Monitoring is confirmed, stability status is analyzed and determined, and the event update and classification submodule 4150 can update and classify by inputting monitoring data including call events including other users' deaths, emergency conditions, and life support status. That is, the event update and classification submodule 4150 classifies emergency call events of death and first aid and life support call events processed by requesting life support according to instructions from the probability module 700, which will be described later. Call events that occur by the user or other users include life support events that can be resolved through support with a simple request for help, emergency situation events that indicate emergency treatment situations requiring emergency treatment at a hospital, and death of other users. It includes a death situation event, and after the occurrence of a call event, the result is updated and classified by type.

In addition, the AI learning module 4300 of the present invention can input other users' monitoring data, and uses the user's monitoring data and other users' monitoring data to connect labels and features in the user's monitoring data and other users' monitoring data. AI learning learning to derive correlations is performed. As described above, the AI learning module 4300 can be equipped with various models for AI deep learning, and among these models, continuous data is generated using RNN (Recursive Neural Network). That is, a recursive and circular learning structure of continuously occurring monitoring data may be formed.

In addition, the present invention includes a probability module 4700, as shown in FIGS. 21 and 23, and classifies call events in the event update and classification submodule 4150 through the probability module 4700. Possibilities of emergency and death situations are generated based on AI learning performed in the AI learning module 4300, and these possibilities of emergency and death situations may be provided in the form of probability numbers.

The probability module 4700 includes an emergency probability submodule 4710 and a death probability submodule 4720. The emergency probability submodule 4710 is an event update and classification submodule. Based on the AI learning executed in the AI learning module 4300 from the emergency call event identified as an emergency among the call events classified in the module 4150, the probability of an emergency occurring on the user's day is generated, and the probability of an emergency occurring on the user's day is generated, and the death situation pro The mobility sub-module 4720 is a death situation confirmed as a death situation among call events classified in the event update and classification sub-module 4150. Based on the AI learning executed in the AI learning module 4300, the user's Generates the probability of death occurring on the same day. That is, as shown in FIG. 25, based on the AI learning executed in the AI learning module 4300, the probability of occurrence of an emergency situation on the user's day and the probability of occurrence of a death situation on the user's day are updated temporal and regional monitoring data. Through deep learning, monitoring data at or before the event of a town-level situation or death situation within a peer or peer group with high similarity is learned through deep learning, and from this, a lifelog or bio that appears to be highly correlated with the occurrence of an emergency situation or death event is generated. By extracting monitoring data such as logs as probability data and executing deep learning on the probability data, the probability of an emergency situation and the probability of a death situation for the user on the day can be provided in the form of numbers.

At this time, the monitoring management server 400 includes a server communication module 4001 and a corresponding module 4200. The server communication module 4001 receives monitoring data transmitted from the monitoring terminal 100, and the corresponding module ( 4200) transmits at least the probability of an emergency situation occurring on the day of the user of the probability module 4700 and the probability of a death situation occurring on the user's day to at least the guardian terminal unit 500 according to the analysis judgment result of the monitoring module 4100. do. In other words, it is desirable to transmit the user's probability of an emergency situation or death situation to a guardian such as a family member rather than to the user. Numerical values or notification information such as those shown in FIGS. 26 and 27 can be transmitted to the guardian terminal in the form of display information. there is. In other words, the user's probability of an emergency situation and death situation on the day are provided in numerical form, and if the numerical value is higher than the normal value or represents an unusual numerical value that exceeds the preset range compared to the user's previous numerical value, it is sent to the guardian's terminal. A cautionary text in the form of FIG. 27 is additionally provided and connection information for re-notification or protective measures is provided so that careful attention can be paid to minimize or prevent the occurrence of risk to the user on that day.

This series of deep learning user life status monitoring and management device control methods is

A provision step (S10) of providing a deep learning user living status monitoring and management device 10 further including a probability module 4700, and monitoring and confirming the stability of the user's daily status by checking the going out index and bio index. A monitoring step (S20) in which the stability state is analyzed and determined by the monitoring module 4100, and a response mode determination step (S30) in which the response module 4200 determines a response mode according to the analysis and determination result of the monitoring module 4100, The execution includes a corresponding mode execution step (S40) of executing the corresponding mode determined in the corresponding mode determination step (S30).

In addition, the deep learning user life status monitoring and management device control method of the present invention is executed at least after the provision step (S10), updates call events requesting emergency or support for the user and other users, and updates the type of call event. It may further include a probability deep learning step (S70) in which the probability is calculated based on information on the occurrence of emergency treatment or death of other users and deep learning is performed.

At this time, as described above, the probability deep learning step (S70) includes a call event result update step (S71), a probability data extraction step (S75), and a probability data learning step (S77). .

In the call event result update step (S71), call events requesting emergency or support for the user and other users are updated, and after executing the call event result update step (S71), the call event classification step (S73) is executed. At this time, emergency call events of death and first aid and life support call events processed by requesting life support are classified, and among these classified data, events regarding emergency situations and deaths are monitored by other users at the relevant time or before. A probability data extraction step (S75) is performed to extract data highly correlated with the event from the data. That is, the probability data extraction step (S75) is performed to extract probability data for calculating the probability of an emergency situation and the probability of a death situation from other users' monitoring data including first aid or death occurrence information, From this, the probability data learning step (S77) is executed through AI deep learning learning, and the probability data is deep learning, so that the probability of an emergency situation on the day and the probability of a death situation on the day can be provided to the guardian terminal. do.

The contents described above are examples for explaining the present invention, and the present invention is not limited thereto. For example, in the process of executing AI learning using the monitoring data of the user and/or other users, the correlation between data core features and labels, etc. may be structured in the form of a function, and the numerical value for the data core feature may be expressed as a function. Although an example of warning about the possibility of occurrence of a specific label by inputting the input was described, the present invention is not limited to this, but visually graphs the degree of occupancy in the living space occupied by the user or other users and creates a visual graph image of the occupancy in the living space occupied by the user or other users. By forming AI learning again, learning or caution warnings can be taken by extracting changes in items for core features that affect when a specific event, or an event named as a label when extracted, occurs, Various modifications are possible depending on the design specifications, such as monitoring bio log data to form various bio indices in addition to heart rate or respiratory rate.

In other words, the above description is merely an illustrative explanation of the technical idea of the present invention, and various modifications and variations will be possible by those skilled in the art without departing from the essential characteristics of the present invention. . Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but are for illustrative purposes, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope should be construed as being included in the scope of rights of the present invention.

Claims (25)

An activity sensor 200-1 that detects at least activity in the user's living space and detects at least water, gas and electricity usage and illumination conditions and the user's breathing rate and heart rate that reflect the daily living conditions in the user's living space. A monitoring sensor unit 200 including an environment sensor unit 200-4,
a monitoring terminal unit 100 that collects monitoring data including at least monitoring life log data detected as activity and monitoring bio log data indicating the user's physical condition from the monitoring sensor unit 200;
The monitoring terminal 100 and the monitoring data are received to indicate at least an activity index indicating whether the user is active, bedtime information indicating sleeping information, an outing index indicating external activity information, and a user's physical condition through the breathing rate and heart rate. It includes a monitoring management server (400) that checks the biometric index, determines the stability of the user's daily state, and transmits it to at least the guardian terminal unit (500),
The monitoring management server 400:
Using the monitoring data, at least the activity index indicating whether the user is active, the bedtime information indicating sleeping information, and the going out index and bio index indicating external activity information are checked to monitor and confirm the stability of the user's daily state and check the stability state. A monitoring module 4100 that analyzes and determines, and further includes an event update and classification submodule 4150 capable of inputting, updating, and classifying monitoring data including call events including the death of another user, emergency condition, and life support status. class,
Other users' monitoring data can be input, and AI learning is performed to derive correlations between labels and features in the user's monitoring data and other users' monitoring data using the user's monitoring data and other users' monitoring data. An AI learning module (4300) that
A probability module 4700 that generates the possibility of emergency situations and death situations based on AI learning performed in the AI learning module 4300 for call events classified in the event update and classification sub-module 4150 is further provided. A deep learning user life status monitoring and management device (10), comprising:
According to clause 1,
The reliability module 4700:
The probability of an emergency occurring on the day of the user based on the AI learning performed in the AI learning module 4300 from the emergency call event identified as an emergency among the call events classified in the event update and classification sub-module 4150. An emergency situation probability submodule 4710 that generates,
Death situation identified as a death situation among call events classified in the event update and classification sub-module 4150 Probability of occurrence of death situation of the user on the same day based on AI learning executed in the AI learning module 4300 from the call event Deep learning user life status monitoring and management device (10), characterized in that it includes a death situation probability submodule (4720) that generates.
According to clause 2,
The monitoring management server 400:
a server communication module 4001 that receives the monitoring data transmitted from the monitoring terminal 100;
At least the probability of occurrence of an emergency situation on the day of the user of the probability module 4700 and the probability of occurrence of a death situation on the day of the user are transmitted to the guardian terminal unit 500 at least according to the analysis judgment result of the monitoring module 4100. A deep learning user life status monitoring and management device (10) comprising a response module (4200).
In the third,
Monitoring data from other users can be input into the monitoring module 4100,
The AI learning module 4300:
An AI learning model that stores an AI learning model to derive correlations between labels and features in the user's monitoring data and other users' monitoring data through AI learning using the user's monitoring data and other users' monitoring data. a storage unit 4323;
Deep learning comprising an AI learning data storage unit 4321 that stores correlations between labels and features in the user's monitoring data and other users' monitoring data derived from the AI learning model storage unit 4423. User living condition monitoring management device (10).
According to clause 1,
The activity sensor (200-1) is a deep learning user living status monitoring and management device (10), wherein the activity sensor (200-1) is disposed in a plurality of zones of the user's living space and detects information on the user's occupied space by time.
According to clause 1,
The monitoring sensor unit 200:
A deep learning user living status monitoring and management device (10) further comprising a surveillance sensor unit (200-3) that detects whether the user goes out from the living space.
According to clause 1,
The environment sensor unit 200-4 is an environment metering sensor (200-4b, 200-4d, 200- 4e, 200-4f) A deep learning user life status monitoring and management device (10).
According to clause 1,
The environment sensor unit 200-4 is a deep learning user living condition monitoring system, characterized in that it includes an environment illuminance sensor 200-4a that detects the illuminance condition of the user's daily life condition in the user's living space. Management device (10).
According to clause 1,
The environment sensor unit (200-4) is a deep learning user life status monitoring and management device (10), characterized in that it includes an environment laser sensor (200-4g) that detects the user's breathing rate and heart rate.
According to clause 1,
The monitoring terminal 100:
A monitoring terminal communication unit 110 that receives the monitoring data from the monitoring sensor unit 200,
a monitoring terminal storage unit 130 that stores monitoring data received from the monitoring sensor unit 200;
Deep learning user life status monitoring and management device (10), comprising a monitoring terminal control unit (120) that controls transmission of the received monitoring data to the monitoring management server (400). .
According to clause 1,
The monitoring management server 400 uses the user's monitoring data and other users' monitoring data to extract data events from the user's monitoring data and other users' monitoring data and perform labeling to create a label. Deep learning user life status monitoring and management device (10), further comprising (4400).
According to clause 11,
The auto labeling module (4400):
A data event extraction submodule 4410 that extracts data events from the user's monitoring data and other users' monitoring data using the user's monitoring data and other users' monitoring data;
a labeling submodule 4420 that extracts the data event and assigns a label;
A deep learning user life status monitoring and management device (10), comprising a labeling verification sub-module (4430) that verifies consistency of the labeling assigned by the labeling sub-module (4420).
According to clause 1,
The monitoring module 4100:
A spatial analysis monitoring sub-module 4110 that checks space occupancy status information by time of the user from at least the monitoring life log data detected as activity among the monitoring data;
A behavior analysis monitoring sub-module (4120) that confirms and analyzes the user's behavior information using the user's space occupancy status information by time confirmed by the space analysis monitoring sub-module (41110);
An abnormality analysis monitoring submodule ( 4130) A deep learning user life status monitoring and management device (10) comprising:
According to clause 13,
Based on the core feature data including the space occupancy status information by time of the user of the space analysis monitoring sub-module 4110 and the user's behavior information of the behavior analysis monitoring sub-module 4120, and the core feature data of other users, A deep learning user, characterized in that it further includes a similarity analysis monitoring sub-module 4140 that checks the similarity between the user's user core feature data and other users' core feature data and confirms and extracts labeling information for other users' core feature data. Life status monitoring management device (10).
According to clause 3,
The corresponding module 4200:
an abnormality determination response sub-module 4210 that executes a corresponding monitoring mode according to the analysis judgment of the monitoring module 4100;
an alarm execution response sub-module (4220) that executes an emergency alarm mode when it is determined to be an emergency situation in the analysis judgment of the monitoring module (4100);
Deep learning user life status monitoring management, comprising a reporting response sub-module 4230 that transmits a report corresponding to the corresponding monitoring mode to the guardian terminal 500 according to the analysis judgment of the monitoring module 4100. Device (10).
An activity sensor 200-1 that detects at least activity in the user's living space and detects at least water, gas and electricity usage and illumination conditions and the user's breathing rate and heart rate that reflect the daily living conditions in the user's living space. a monitoring sensor unit 200 including an environment sensor unit 200-4, monitoring life log data detected as at least activity from the monitoring sensor unit 200, and monitoring bio log data indicating the user's physical condition. A monitoring terminal 100 that collects monitoring data including, and receives the monitoring data with the monitoring terminal 100 to provide at least an activity index indicating whether the user is active, bedtime information indicating bedtime information, and external activity information. It includes a monitoring management server 400 that verifies the stability of the user's daily state by checking the going out index and the bio index indicating the user's physical condition through the user's breathing rate and heart rate and transmits the judgment to at least the guardian terminal unit 500; , the monitoring management server 400: uses the monitoring data to check at least the activity index indicating whether the user is active, the bedtime information indicating bedtime information, and the going out index and bio index indicating external activity information, thereby confirming the user's daily life. The event update and classification submodule (4150) monitors and confirms the stability of the status, analyzes and determines the stability status, and inputs, updates, and classifies monitoring data including call events including death of other users, emergency status, and life support status. ), and a monitoring module 4100 that further includes a monitoring data of another user can be input, and the monitoring data of the user and the monitoring data of other users are used to create a label in the user's monitoring data and the monitoring data of another user. An AI learning module 4300 that executes AI learning to derive correlations between features, and AI learning performed by the AI learning module 4300 on call events classified in the event update and classification sub-module 4150 A provision step (S10) of providing a deep learning user life status monitoring and management device (10), further comprising a probability module (4700) that generates the possibility of emergency situations and death situations based on
Using monitoring data including at least monitoring life log data detected as activity from the monitoring sensor unit 200 received from the monitoring terminal unit 100 through the server communication module 4001 of the monitoring management server 400 Thus, the stability of the user's daily state is monitored and confirmed by checking at least the activity index indicating whether the user is active, the bedtime information indicating bedtime information, and the going out index indicating external activity information, and the stability state is analyzed in the monitoring module 4100. A monitoring step to determine (S20),
A response mode determination step (S30) in which the response module 4200 determines a response mode according to the analysis and determination result of the monitoring module 4100;
A deep learning user life status monitoring and management device control method comprising a response mode execution step (S40) of executing the response mode determined in the response mode determination step (S30).
According to clause 16,
The monitoring step (S20) is:
Monitoring data including at least monitoring life log data detected as activity is received from the monitoring sensor unit 200, which is received from the monitoring terminal unit 100 through the server communication module 4001 of the monitoring management server 400. A monitoring input step (S21),
A spatial analysis monitoring step (S23) in which the spatial analysis monitoring sub-module 4110 checks the user's space occupancy status information by time from at least the monitoring life log data detected as activity among the monitoring data;
A behavior analysis monitoring step (S25) in which the user's behavior information is confirmed and analyzed in the behavior analysis monitoring sub-module 4120 using the user's space occupancy status information by time confirmed by the space analysis monitoring sub-module 41110;
The abnormality analysis monitoring submodule 4130 determines whether the user is in an abnormal state using the user's space occupancy status information by time in the space analysis monitoring submodule 4110 and the user's behavior information in the behavior analysis monitoring submodule 4120. A deep learning user life status monitoring and management device control method comprising an abnormality analysis monitoring step (S27) of monitoring and analyzing.
According to clause 17,
The abnormality analysis monitoring step (S27) is:
The monitoring terminal 100 receives the monitoring data and compares the set reference value with at least an activity index indicating whether the user is active, bedtime information indicating sleeping information, and an outing index indicating external activity information, thereby stabilizing the user's daily state. A deep learning user life status monitoring and management device control method comprising a rule base abnormality analysis and monitoring step (S271) for confirming and determining.
According to clause 18,
In the deep learning user life status monitoring and management device, other users' monitoring data can be input to the monitoring module 4100, and the user's monitoring data and other users' monitoring data are used to obtain the user's monitoring data and other users' monitoring data. An AI learning module 4300 that executes AI learning to derive correlations between labels and features in the monitoring data is further included,
Based on the core feature data including the space occupancy status information by time of the user of the space analysis monitoring sub-module 4110 and the user's behavior information of the behavior analysis monitoring sub-module 4120, and the core feature data of other users, The similarity between the user's user core feature data and the core feature data of other users is confirmed, labeling information for the other user's core feature data is confirmed and extracted, and the monitoring terminal 100 and the monitoring data are received to determine at least the user's activity. Characterized by further comprising an artificial intelligence abnormality analysis monitoring step (S273) that confirms and determines the stability of the user's daily state with respect to the activity index representing, sleeping information representing bedtime information, and the going out index and bio index representing external activity information. Deep learning user life status monitoring and management device control method.
According to clause 19,
The response mode determination step (S30) is:
Deep learning user life, comprising a rule base response mode determination step (S30-1) of determining a response mode according to the rule base abnormality analysis monitoring confirmation result confirmed in the rule base abnormality analysis monitoring step (S271). Condition monitoring management device control method.
According to clause 20,
An analysis monitoring data input step (S31) in which the rule base abnormality analysis monitoring confirmation result confirmed in the rule base abnormality analysis monitoring step (S271) is input,
A confirmation result determination step (S33) in which the response module 4200 determines whether the rule base abnormality analysis monitoring confirmation result is normal;
A deep learning user life status monitoring and management device control method comprising a caution determination step (S35) in which the response module 4200 determines whether the rule base abnormality analysis monitoring confirmation result requires caution.
According to clause 19,
The response mode determination step (S30) is:
Deep learning user life comprising an artificial intelligence response mode determination step (S30-2) of determining a response mode according to the artificial intelligence anomaly analysis monitoring confirmation result confirmed in the artificial intelligence anomaly analysis monitoring step (S273). Condition monitoring management device control method.
According to clause 22,
An analysis monitoring data input step (S301) in which the artificial intelligence anomaly analysis monitoring confirmation result confirmed in the artificial intelligence anomaly analysis monitoring step (S273) is input,
A core feature similarity high determination step (S303) in which the response module 4200 determines whether the core feature similarity indicating core feature similarity as a result of the artificial intelligence anomaly analysis monitoring confirmation is higher than the preset core feature high similarity;
Characterized by comprising a core feature similarity medium determination step (S305) in which the response module 4200 determines whether the core feature similarity indicating the core feature similarity as a result of the artificial intelligence anomaly analysis monitoring confirmation is higher than the preset core feature medium similarity. Deep learning user life status monitoring and management device control method.
According to clause 19,
Executed at least after the provision step (S10),
Probability updates call events that request emergency or support for the user and other users, classifies them according to the type of call event, calculates the probability through information on the occurrence of first aid or death of other users, and learns deep learning. A deep learning user life status monitoring and management device control method further comprising a deep learning step (S70).
According to clause 24,
The probability deep learning step (S70) is:
A call event result update step (S71) of updating call events requesting emergency or support for the user and other users;
A call event classification step (S73) for classifying emergency call events of death and first aid and life support call events processed by requesting life support;
A probability data extraction step (S75) of extracting probability data for calculating the probability of an emergency situation and the probability of a death situation from other users' monitoring data including first aid or death occurrence information;
A deep learning user life status monitoring and management device control method further comprising a probability data learning step (S77) of deep learning the probability data.

Images