KR20210057332A - Smart Mirror Based Behavior Pattern Management System - Google Patents

Abstract

The present invention is a system capable of behavior pattern recognition and voice recognition with a sensor disposed on the front of a display which also functions as a mirror. More specifically, the present invention relates to a monitoring system which is integrated technology of Bluetooth, wireless communication, and IoT technology, and earthquake sensor, temperature sensor, radar biometric sensor and microphone technology. Moreover, the monitoring system can monitor health status by receiving health information of information providers and people living alone, can prepare for emergency situations by learning repetitive behavior patterns, and can analyze cognitive information through voice of the information providers.

Description

Health care monitoring system using smart mirror-based radar, biometric information, and voice {Smart Mirror Based Behavior Pattern Management System}

The present invention is a smart mirror applied with a monitor with a mirror function. It is a health care system in which biometric information can be collected through a sensor mounted on the smart mirror, and a radar sensor is applied among the collectable sensors. It is a system integrating environmental sensor technology such as earthquake sensor, temperature sensor, fine dust sensor, etc., which monitors the health of information providers, analyzes health status or activity patterns, and manages them effectively.

Radar sensors are capable of recognizing proximity patterns, so if the information provider's behavior pattern is converted into data as a pattern of movement over time, it is possible to collect and process data that can be analyzed. With this data, an artificial intelligence notification service is possible, and through a microphone and temperature sensor. It is a system that quickly responds to conditions according to the situation with a recognition method appropriate to the situation. At this time, it is a system that converts the radio waves from the sensor into data according to a specific value and informs the caregiver who perceives the state of health as the data.

Conventionally, human health monitoring uses a sensor that detects only the motion of an information provider, and uses motion data, images, and infrared rays to determine whether there is an abnormality due to a short visual change in the activity state, and a system that informs the administrator or guardian of it. However, according to the angle and position of the installed sensor, the status of the information provider could not be known only by inaccurate movement.

It was difficult to manage abnormal conditions as the sensor detects the movement of objects other than the movement of people, and because it only detects the activity state of the information provider, it was not able to detect predictable behavior, and actual biometric information (heart rate, respiration), etc. It did not reach the level of detecting and analyzing.

In particular, when an emergency situation occurs, the method of sending a rescue signal to the observer or the 119 through the telephone was only possible through a separate sending device, and the information provider can request a rescue only in a state of consciousness or ability to move. In the state of being conscious but unable to move, there was no way to notify the emergency situation and request rescue, so the situation of missing the timing of rescue frequently occurred.

It is an object of the present invention to provide an emergency relief notification service by collecting more accurate information than the situation from existing motion detection in order to more actively cope with situations that may arise from information providers and socially marginalized subjects who cannot use information devices. The purpose is to analyze the situation more efficiently through the collection of certain activity patterns and voices of the user, and to implement this, contactless radar, biometric information sensor, and environmental information collection sensor (temperature sensor, earthquake sensor, and fine dust detection sensor) rather than movement. You need an algorithm that can analyze the data collected through the device, and use a device such as a microphone to analyze biometric information and surrounding information through middleware and communication AP, and check for predictable data only when it is possible to check comprehensive health status and activity patterns. It is possible.

It is to provide a management system capable of monitoring information providers with pattern data and biometric information collected for predictable situations.

In the present invention, life managers, social workers, guardians, etc. can easily and conveniently inspect and manage various information of information providers in real time anytime, anywhere, and it is possible to check predictable pattern information through analysis of repetitive patterns. It is a system that can actively respond to emergency situations only by analyzing comparative data.

The present invention converts and disseminates the data from the comparative analysis of the changed data rather than the visual elements of the subject and its surroundings, and propagates the data as a sign of abnormality. The purpose of this is to provide more efficient and emergency response from related expert groups.

In the present invention, when an information provider needs help, unlike the method of requesting help using a conventional telephone or button, the behavior pattern and information from the living body (pulse, breathing, movement) and voice or environmental information are collected (temperature, Earthquake) alone can be analyzed and predicted, so you can get help even if you do not request relief at an appropriate time, and when relief actions are not possible, contact a guardian who can contact you by text, alarm (push message), voice, etc. for urgent situations. Propagation is possible.

In the present invention, action patterns and biometric information (pulse, breathing, movement) and voice, environmental information (temperature, earthquake) possessed by the information provider can be used to take action, and only irregular data generated from this information is urgent. Push notification service is possible so that dispatch service is performed automatically, and predictable monitoring and management system can be established only with provided data.

Specifically, the present invention has a certain radio wave between a certain device and a person,

Information generated by movement can be classified into biometric information (pulse, breath, movement), and information collected from environmental sensors (temperature, earthquake) makes it easy to recognize the situation. By incorporating IOT technology into a specific field of health and activity monitoring, it is possible to effectively grasp the monitoring status of the target through the already commercialized machine learning technology through online application interfaces such as websites and mobile services.

In addition, in order to advance the technology by efficient data, it is to provide an information provider monitoring and management system in order to implement a technology capable of deep learning in machine learning.

Various non-contact sensors must be installed and operated in the vicinity of the smart mirror, and the direct and indirect data obtained from the installed sensors are used to convert into encrypted data that can be transmitted. Changes can be made, and in order for a general monitor to have a mirror effect, a film capable of mirror effect must be installed on the front side. The biometric information propagation sensor controller must collect the minute movements of a person as biometric information that can be analyzed with a constant radio wave, but for this, a 60hz~80hz radio wave is required. This radio wave is included in a frequency of 24 GHz or higher to facilitate human access. Information is sensed through and the same pattern can be analyzed and defined by converting the protocol into a database through movement patterns coming out of a certain distance.

If the pattern behavior and body information other than the collected data are different, separate functional management should be performed by separating them into abnormal signals, and AI functions can be implemented with the result values analyzed after comparing and analyzing the current state and the previous state. 24Ghz In the W70Ghz band, the sensor must be detected at the same height as the heart, and there is no frequency interference due to interference with other sensors. It is a sensor that can detect rapid changes by collecting moisture data and analyze the temperature rise due to fire whether the indoor temperature is appropriate. It is installed in the lower part of the housing and detects the temperature rising from the floor first to prevent fire and prevent changes in the indoor temperature. Sensing unit that can respond

The seismic sensor controller installs a vibration (motion) sensor on the upper part that can detect even the small movement width of the housing than the lower part of the housing, and the vibration sensor detects the movement of the housing itself and detects the band with a large movement width and classifies it as an earthquake. By installing a resistor in the sensor controller and the sensor part, the temperature of the resistance is artificially buoyant, so that friction with the air is possible, so that the temperature of the resistance is sensed by the LED resistance and flickering is calculated numerically; an analysis unit capable of analyzing the concentration of fine dust;

A voice recognition processor for interactive conversation with a microphone corresponding to a voice command of a management subject and a voice command (speaker interlocks with an external Bluetooth speaker);

Aligosm for storing and analyzing the environmental information, the biometric information and motion information, and the analysis data

Database; It provides an information provider monitoring and management system comprising a; transmitting unit for providing the analysis data to the user terminal.

The analysis unit identifies an abnormal symptom or emergency situation by tracking the accumulated analysis data, and the transmission unit provides a notification message for the identified abnormal symptom or emergency situation to the user terminal. The analysis data is the analysis data. It is generated while maintaining the environmental information, the biometric information, the motion information and the real-time property, and the transmitting unit provides the analysis data maintaining the real-time property to the user terminal. The present invention includes the steps of receiving the temperature information from a temperature sensor for collecting temperature information;

Receiving the biometric information and the motion information from a biometric sensor that collects biometric information, biometric information of a management subject, and motion information of an activity state for whether an activity/rest/sleep or not;

Generating analysis data through analysis of the environmental information and the biometric information and motion information;

Storing the environmental information, the biometric information and motion information, and the analysis data;

Providing the analysis data to a user terminal; Provides an information provider monitoring and management system through biometric information recognition and behavior patterns including.

The present invention provides a voice recognition processing unit capable of collecting a microphone and voice corresponding to biometric information (breathing, pulse, movement) and voice commands coming out to a management subject; the voice recognition processing unit uses a multi-directional microphone to make a user's speech (voice Command), and when an ignition is detected, the content of the utterance is analyzed and transmitted to the person subject to management, and the contents analyzed by the fine dust sensor can be monitored by the person subject to management to understand the situation of the person subject to management, and the guardian or manager designated by text and alarm Send a text message or an alarm (push service) to the phone, or if an emergency call is needed, it enables the emergency dispatch service in connection with the rescue agency dispatch service (119 emergency center, emergency dispatch service provider), and monitors and manages through voice recognition that is transmitted. System.

As described above, according to the present invention, the integrated technology of a radar-based contactless bio-signal sensor and a temperature sensor, an earthquake sensor, a fine dust sensor, and a Wi-Fi-based communication AP and middleware is applied to the information provider management field in the welfare industry. By doing so, it is possible to perform care management management with AI services, which are more efficient and the next step in deep learning.

In addition, according to the present invention, by linking the smart bio management system with a real-time web service, it is possible to naturally monitor and manage the subject's biometric information and living environment for 24 hours. By notifying the message in real time to the person, an urgent response can be made immediately. In addition, through the establishment of such a system, an environment in which all information can be managed continuously and systematically can be provided.

According to the present invention, it is possible to help maintain the subject's condition and improve the health condition through a notification system that induces periodic exercise through monitoring the subject's motion and condition.

According to the present invention, by monitoring the temperature information of the subject's residential environment to manage the temperature, it is possible to check and prevent whether the subject is exposed to high temperature or low temperature.

According to the present invention, a service capable of evacuating the subject to a safe place in advance in case of an earthquake by detecting whether the subject's residential environment is earthquake through an earthquake sensor is possible.

According to the present invention, when a subject needs assistance, it is possible to request assistance by detecting voice and abnormal behavior without using an existing telephone or emergency button. Therefore, it is appropriate even when the use of electronic devices is difficult or when it is away from devices such as toilets. You can get help.

According to the present invention, the non-contact biometric signal sensor extracts biometric information using radio waves that are harmless to the human body without direct contact with the subject, that is, data such as respiration algorithm, repeated pattern learning using proximity functions, movement, etc. By providing real-time status information through measured data to medical staff and managers as well as to the targets that are transmitted to the server to be measured, it enables constant monitoring of health status, early recognition of emergencies, and prediction and prevention of abnormal signs to improve blind spots in management, etc. It is expected to dramatically improve service quality.

In addition, the temperature sensor provides measurement data of various sensing sensors such as temperature for the indoor air quality of the subject's residential environment, thereby maintaining and managing a comfortable living environment of the subject through monitoring of the environmental distribution that is difficult to recognize. There is.

The representative diagram is a diagram that explains the behavior pattern of biometric information, and it is a representative diagram that explains the degree of movement on the wall with the smart mirror attached, and the components and operating principles of the information provider monitoring and management system through collection of biometric information and environmental information and voice recognition. FIG. 1 is a diagram illustrating the configuration of a central server, which is a core component, according to functions. FIG. 2 is an operation flow diagram specifying a method for monitoring and managing information providers in an embodiment of the present invention. FIG. 3 is the present invention A flow chart specifying a flow for collection and analysis according to another embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail with reference to the accompanying drawings. In describing the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the subject matter of the present invention, the detailed description thereof will be provided. Middleware mentioned in the description of the present invention is software that has appeared to solve various problems arising from implementing a distributed computing environment, and is applied by connecting different types of hardware, protocols, and communication environments in a distributed computing environment. Refers to software that enables smooth communication between a program and the environment in which the program is operated. Hereinafter, a biometric monitoring and management system according to an embodiment of the present invention will be described in detail with reference to FIGS. 1 to 4 attached thereto. 1 is a diagram showing the components and operating principles of an information provider monitoring and management system through voice recognition according to an embodiment of the present invention. In a residential environment in which the information provider resides, the subject's breathing rate and movement A radar biometric signal sensor 110 capable of detecting information, etc., a temperature sensor 120 capable of detecting the constituents of air in a residential environment, and an earthquake sensor 130 capable of detecting an earthquake around the residential environment. A microphone 180 is installed. The radar bio-signal sensor 110 and the temperature sensor 120 are respectively, and the communication may also be designed in a commonly used Bluetooth and WIFI method. The temperature sensor 120 detects environmental factors such as indoor temperature and so on. Users can monitor the temperature in the subject's residence. Wi-Fi-based data collection units and communication APs are installed in the residential environment facilities. Communication AP is an abbreviation of communication'Access Point' and means a wireless communication access point. The data collection unit and the communication AP are provided from one or more radar biometric signal sensors 110, one or more temperature sensors 120, one or more seismic sensors 130, and one or more motion sensors 140. The collected information is collected and transmitted to the central server 140 through the network 150. The information transmitted in this way is sent to the central server 140 and undergoes an analysis process. The subject management-related information extracted from the data analysis process in the central server 140 is provided by a life manager, a social worker, and a guardian through the network 150. It is possible to efficiently and quickly manage the subject by providing environmental information, earthquake information, movement information, and biometric information of the subject's living environment to various user terminals 160 or groups of related persons/experts 170 to related parties. Provide services to do so. Examples of various user terminals 160 include mobile terminals, notebook computers, and desktop PCs. These user terminals 160 provide information to users through applications of mobile terminals, notebook web interfaces, and desktop PC web interfaces. In the case of an emergency situation requiring the assistance of the related expert 170, the central server 140 provides the biometric information of the subject who needs emergency treatment and Send out a message saying it is an emergency situation.

In addition, it is very important that the present living body monitoring and management system maintains real-time data. Data transmission to interfaces of various user terminals 160, such as a mobile application, a notebook web interface, and a desktop PC web interface, may be performed in both directions instead of one direction. A group of experts in related fields 170 receives information detected by the radar bio-signal sensor 110, the environmental sensor 120, the earthquake sensor 130, the microphone 180, and the fine dust sensor 190, and provides an emergency situation. It is possible to determine whether or not, if necessary, check the status of the target person through the microphone 180, and reply or broadcast the identified situation information through the network 150.

2 is a block diagram showing the configuration of the central server 140, which is a core component of the information provider monitoring and management system through voice recognition according to an embodiment of the present invention, according to functions. Based on the database 210 analysis algorithm/processing module 220 and the analyzed content, the database 210, which is essential for analyzing the vast data of the database 210 and the database 210, is an event that describes the current condition of the subject and the condition in the living environment It consists of a table 230. In addition, it is possible to send various messages to the relevant person using the SMS or the Agent 240 of the Push server according to the user's setting conditions, schedule management, and emergency situations. Here, Push is a technology that automatically and periodically pushes the latest information directly to the computer screen by simply selecting the list of information that the user is interested in using the Internet environment, and the server automatically sends the information the user wants. It is a way to provide it. In other words, it is an automated technology that allows users to receive specific information without requesting each time. SMS is an abbreviation for System Management Server, which eliminates the inefficiency of managing each terminal individually, and provides hardware monitoring and automatic software distribution. It refers to a technology that can reduce TCO (Total Cost of Management), such as overseeing.

That is, the central server 140 according to an embodiment of the present invention collects from the radar bio-signal sensor 110 and the temperature sensor 120, the earthquake sensor 130 and the microphone 180, and the fine dust sensor 190 The information in the database 210 accumulated as the information sent from the data collection unit and the communication AP 150 is extracted and processed in the form of desired information using the database analysis algorithm/processing module 220, and then processed into the event table 230. After importing, it can be used for monitoring targets using the Push server or Agent 240 of the SMS server.

The database 210 analysis algorithm/processing module 220 may be implemented by one or more processors, and the event table 230 may store profile information on a plurality of events corresponding to emergency situations and abnormal symptoms. The event table 230 may store necessary information by assigning an index to each subject, or may be stored together with necessary event information by giving an index to the subject. A message including status information can be generated when an emergency or abnormal symptom occurs based on the information and the optimal value of the residential environment information. The optimal value may be stored in advance in the event table 230. In the biometric information, the activity/rest/sleep pattern analysis generated based on biometric information such as motion information, proximity information, and breathing of the subject is used. State motion information is included. The radar biometric sensor 110 may collect biometric information and motion information (degree of proximity) in a contactless manner. The database 210 may track and store the detected biometric information, motion information, and environment information. If necessary, information on a certain time period may be updated with the latest data. For a certain period of time in which no abnormal symptoms are detected, existing data stored in the database 210 may be deleted to prepare a space for storing new data. FIG. 3 is an information provider through voice recognition according to an embodiment of the present invention. It is an operation flowchart showing the monitoring and management system. Referring to FIG. 3, a non-contact radar biometric sensor 110 detects biometric information and motion information of a subject. The generated biometric information and motion information is transmitted from the biometric sensor 110 to the central server 140 (S310). The temperature sensor 120 detects environmental information including air component information such as fine dust, temperature, and humidity. (S320). At this time, the generated environmental information is transmitted from the temperature sensor 120 to the central server 140 (S320). The earthquake sensor 130 detects information on earthquakes and vibrations. At this time, the generated earthquake information is transmitted from the earthquake sensor 130 to the central server 140. (S330) The voice command of the information provider is detected and analyzed through the microphone 180. At this time, the generated voice information through the voice recognition is transmitted to the central server 140 through the microphone 180. (S360) The information collected by the sensor is transmitted through the network 150 in real time through the communication AP and middleware. It is transmitted to the server 140 (S330). Steps S310 and S320 are displayed as if S320 is executed after S310 for convenience of explanation, but the spirit of the present invention is not limited to this embodiment, and Step S310 may be executed independently and in parallel with each other. Central server The database 210 analysis algorithm/processing module 220 of 140 may identify abnormal signs or emergency situations through analysis of data created based on a protocol suitable for the accumulated sensor (S340) central server 140 Push server / Agent (240) can transmit an alarm message for the identified abnormal symptoms or emergency situations to the user terminal 160 or the related person / expert 170 group (S350) Figure 4 of the present invention It is an operation flowchart showing a method of monitoring and managing an information provider through a radar sensor and voice recognition according to another embodiment. In the step S410 of FIG. 4, the radar sensor and the micro voice recognition are enabled, so that the temperature information detection of S420 is crossed to provide the usual information to the real-time terminal 160 or the related person/expert 170 group. When there is a change in the algorithm in the information sensor and the voice recognition microphone, the S470 analysis module analyzes and predicts the change in the algorithm, stores the information in the S480 database, and performs a notification in step S490 with minimal processing, and the user terminal 160 Alternatively, it may be provided as a group of related persons/experts 170. Such minimal processing (processing of information that maintains real-time) or a database that can be analyzed in the S470 analysis module can be stored in the database S480, and this allows the data collected by S470 to operate in a parallel structure by interlocking with the existing database. As a result, deep learning can be implemented, and the implemented information can be provided to the user terminal 160 or a group of related persons/experts 170. If the S430 algorithm does not change, perform S440 seismic information collection and change the S450 algorithm. In case of occurrence, the S470 analysis module analyzes and cross-analyzes the S480 database data in parallel. The information generated here is connected to the occurrence of the S490 situation, which can also be provided to the user 160 and the administrator 170 through the S530 administrator UI. When S460 voice information is generated, it is possible to process first, and the generated voice information S460 repeats modulation and demodulation that converts analog signals into digital signals in the analysis module S470, and the final digital signal is changed once again to data that can be analyzed in the S470 analysis module. It can be implemented with database S480, and through this dedicated interface, the subject's biometric information and motion information can be provided in real time. For example, it can be implemented so that the subject's main management information can be accessed through websites and mobile applications. have.

The information provider management target monitoring and management method according to an embodiment of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium is a program

Commands, data files, data structures, etc. may be included alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the present invention, or may be known to and usable by a person in charge of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as flopticaldisks. And hardware devices specially configured to store and execute program instructions such as magneto-optical media and ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention.

As described above, in the present invention, specific matters such as specific components, etc., and limited embodiments and drawings have been described, but this is provided only to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments. , If a person of ordinary skill in the field to which the present invention belongs, various modifications and variations are possible from these descriptions.

The global population decline and the number of households living alone are increasing, and the trend of aging and disconnection from society is increasing. It is increasing rapidly. Dementia or Cho Heon-Byeong allows for machine learning of behavioral patterns and rush machine based on the learned data to promote social stability through early detection and treatment of the population disconnected from society. The current lone death prevention system is a human being's lone death prevention system. There are many systems that detect and inform only simple movements without obtaining body information, but deep learning of specific information on the human body now requires an advanced healthcare system with a frequency sensing system that can assist the population living alone with such an effect. It is a reality, so I think it will be useful at the point where the population decline continues.

110: radar bio-signal sensor, 120: temperature sensor, 130: earthquake sensor, 140 central server 150: network 160: user terminal, 170: expert, 190: fine dust sensor, 180: microphone, 210: database, 220: algorithm/ Processing module, 140: event table, s310: push server/agent, s320: sensing biometric information and motion information from a non-contact radar biometric sensor, s330: sensing environmental information including temperature information from a temperature environment sensor s340: Transmits information collected from sensors in real time through communication AP and middleware, s350: Identify abnormal signs or emergencies through analysis of accumulated data 　, s360: Alarm for abnormal signs or emergency situations, s410: Microphone voice recognition information Sensing, s420: Detects biometric information and motion information from a non-contact biometric sensor s430: detects environmental information from a temperature sensor, s431: transmits information collected from the sensor in real time through communication APs and middleware, s440: biometrics Provides information, motion information, and environment information to the user terminal in real time, s450: Executes analysis of biometric information, motion information, and environment information in the analysis module s460: Provides a message according to the analysis result of the analysis module to the user terminal

Claims (2)

Collects the information from the sensor 110 that collects biometric information, receives the biometric information of the information provider, behavior patterns and information on proximity activities, and collects and receives the temperature of the surrounding environment from the temperature sensor 120 Collecting unit to do;
An analysis unit that generates analysis data through analysis of the environmental information and the biometric information and motion information;
A voice recognition processing unit microphone 180 capable of collecting voice recognition from an information provider;
A database storing the environmental information, the biometric information and motion information, and the analysis data;
A transmitter for providing the analysis data to a user terminal;
Information provider monitoring and management system through voice recognition, characterized in that it comprises a. The method of claim 1, wherein the analysis data is generated while maintaining the biometric information and environmental information in real time, and a microphone 180 corresponding to the voice collection of a person subject to monitoring and management and a microphone 180 having a multi-directional voice recognition processing unit Use to detect the utterance (voice) of the information provider, analyze the pattern of the utterance and deliver it to the person subject to management, and if a text or alarm is needed, a text or alarm (push service) is sent to the designated guardian or manager, or emergency dispatch When this is necessary, an information provider monitoring and management system through voice recognition that transmits emergency dispatch services in connection with emergency dispatch services (119 emergency centers, emergency dispatch providers).

Images