WO2022231132A1 - Artificial intelligence smart remote control device and self-test method using same - Google Patents

Artificial intelligence smart remote control device and self-test method using same Download PDF

Info

Publication number
WO2022231132A1
WO2022231132A1 PCT/KR2022/003938 KR2022003938W WO2022231132A1 WO 2022231132 A1 WO2022231132 A1 WO 2022231132A1 KR 2022003938 W KR2022003938 W KR 2022003938W WO 2022231132 A1 WO2022231132 A1 WO 2022231132A1
Authority
WO
WIPO (PCT)
Prior art keywords
remote control
smart remote
signal
artificial intelligence
user
Prior art date
Application number
PCT/KR2022/003938
Other languages
French (fr)
Korean (ko)
Inventor
유재천
Original Assignee
성균관대학교산학협력단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 성균관대학교산학협력단 filed Critical 성균관대학교산학협력단
Publication of WO2022231132A1 publication Critical patent/WO2022231132A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H10/00—ICT specially adapted for the handling or processing of patient-related medical or healthcare data
    • G16H10/60—ICT specially adapted for the handling or processing of patient-related medical or healthcare data for patient-specific data, e.g. for electronic patient records
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
    • G16H40/67—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for remote operation
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]

Definitions

  • Smart remote control device that measures, stores and manages a user's health condition by means of a biosensor installed on the remote control while using the remote control in everyday life, and automatically analyzes it by artificial intelligence to identify risk factors for the user's health
  • the user's health status is measured and identification is authenticated by the biosensor installed on the remote control while using the remote control. It provides an artificial intelligence smart remote control device that can prevent a user's disease in advance and a self-test method using the same.
  • body heat, blood pressure, blood sugar, electrocardiogram, oxygen saturation, cholesterol, and body fat are major indicators of health management in daily life that inform our body condition well. you will be able to enjoy life
  • bio-signals eg, electrocardiogram, PPG signal, auscultation sound signal
  • the present invention provides a ceiling-type artificial intelligence health monitoring device and a remote medical diagnosis method using the same (application number: 10-2020-0055019), a self-isolation monitoring device and a method using the same (application number: 10-2020-0071282), distributed It is a continuation of a type movement tracking device and a method using the same (Application No.: 10-2020-0086171) and an AI-type asymptomatic movement tracking device and a method using the same (Application No.: 10-2021-0024156).
  • the present invention controls the function of a digital TV (or smart mirror) and performs user authentication without a separate authentication procedure in daily life, as well as body heat, which is a major indicator of health care
  • An object of the present invention is to provide an artificial intelligence smart remote control device that can provide remote medical diagnosis service through connection and a self-test method using the same.
  • the present invention provides an artificial intelligent smart remote control device capable of performing blood pressure measurement, cholesterol measurement, and blood sugar measurement by using a synchronized bio-signal measured from the user's body at the same time as an input signal of an artificial intelligence neural network, and a person using the same. We want to provide an inspection method.
  • the artificial intelligent smart remote control device performs a button for controlling a function of a digital TV and a bio-signal measurement and user authentication during the bio-signal measurement at the same time a smart remote control including a biosensor; and controlling the digital TV by voice commands, receiving medical data on the biosignal measured by the biosensor, analyzing it through an artificial intelligence neural network, and providing a remote medical diagnosis service through an Internet communication connection with a doctor It may include a voice recognition terminal.
  • the smart remote control may include: a biosignal collecting unit for collecting biosignals from the biosensor; and biosignal validity determining means for collecting valid biosignals for which user authentication has been completed among the collected biosignals, wherein the voice recognition terminal includes medical data measured from a plurality of medical devices through a short-range wireless communication connection to the It may include a medical data receiving unit for receiving medical data measured by the biosensor of the smart remote control.
  • the voice recognition terminal transmits the medical data to the server through the Internet network, and receives feedback on the analysis result of the medical data analyzed by the artificial intelligence neural network or expert system on the server to provide the user with information on whether or not there is a disease.
  • the level of risk may be notified through the digital TV.
  • the artificial intelligence neural network may be an artificial intelligence neural network app installed in the voice recognition terminal.
  • an artificial intelligent smart remote control device includes a biosensor for simultaneously measuring biosignals and user authentication during the biosignal measurement, a biosignal collecting unit for collecting biosignals from the biosensor, and the biometrics.
  • a biosignal validity determination means for filtering out only a valid biosignal part among biosignal components for which user authentication has been completed among biosignals collected by the signal collection unit, and medical care that receives medical data measured by medical devices through a short-distance wireless communication connection a medical data storage unit for storing data or medical data obtained through the biosignal collecting unit; a wireless communication connection means for transmitting the medical data to a server through the Internet network, and then receiving a feedback result of analysis of the medical data by an artificial intelligence neural network or an expert system on the server; and a smart remote controller including a smart remote controller for controlling the biosensor, biosignal collecting unit, medical data storage unit, and wireless communication connection means and providing the result of analyzing the medical data to the user through a touch screen.
  • the touch screen includes a menu button for switching between a remote control mode and a health care mode screen, and during the remote control mode, remote control selection buttons for controlling a function of a digital TV are displayed on the touch screen screen, During the health care mode, menu selection buttons for selecting one of health care items may be displayed on the touch screen screen.
  • the smart remote control control unit when a valid biosignal for a specific health care item is collected below a preset reference value, sends a text or voice message requesting measurement of a biosignal of the specific health care item to the user, or It may include a biosignal measurement requesting means for registering a specific health care item in the intensive care item to induce the user to self-examine or interactively request the user to measure the biosignal.
  • the left and right sides of the smart remote control include a reflective blood vessel optical sensor array module, and the reflective blood vessel optical sensor array module includes a plurality of SpO 2 sensors or a plurality of vein sensor units, and the user controls the side of the smart remote control.
  • a vein image of the blood vessel pattern of the vein inside the finger is obtained by a photodetector and used for vein authentication, or oxygen saturation and PPG signal can be measured from the artery of the finger.
  • the fingerprint authentication unit installed on the top, bottom, left and right sides of the smart remote control; and a plurality of hand electrodes surrounding an outer periphery of the fingerprint authentication unit providing simultaneous contact with the fingerprint authentication unit when the fingerprint authentication unit is in contact, wherein the fingerprint authentication unit is the user's hand when the user holds the smart remote control.
  • Fingerprint authentication is performed by the fingerprint of the thumb, and the plurality of hand electrodes can be operated as a driving electrode and a detection electrode for body fat measurement, or a first electrode of an ECG sensor can be formed.
  • the ECG sensor is installed on the rear side of the smart remote control, and the ECG measurement is performed by amplifying the voltage signal formed between the first electrode and the second electrode, and the second electrode is when the ECG sensor is in contact with the skin. can be formed.
  • the electrocardiogram is measured by amplifying the voltage signal formed between the first electrode and the second electrode, and the second electrode may be formed when the opposite finger of the hand is in contact with the touch-type ECG sensor.
  • the touch-type ECG sensor a display panel; a lower transparent film laminated on the display panel; an upper transparent film laminated on the lower transparent film; an ECG sensor laminated on the upper transparent film; a spacer separating the upper transparent film and the lower transparent film; a first resistive film pattern formed on the upper surface of the lower transparent film and providing X-axis coordinates when touched; a second resistive film pattern formed at a lower end of the upper transparent film and arranged to cross perpendicularly to the first resistive film pattern to provide a Y-axis coordinate when touched; a plurality of X-axis electrodes installed at the ends of the first resistive layer pattern; a plurality of Y-axis electrodes installed at the ends of the second resistive layer pattern; and an ECG electrode installed at one terminal of the ECG sensor.
  • the smart remote control includes a reflective blood vessel optical sensor array module on the left and right sides of the smart remote control, and the reflective blood vessel optical sensor array module is composed of a plurality of SpO 2 sensors, so that the user can control the side of the smart remote control. Simultaneously with fingerprint authentication when held with a finger, oxygen saturation and PPG signal can be measured from the finger's artery by a photodetector.
  • the smart remote control includes a stethoscope installed on the rear side, and when the user holds the side of the smart remote control with a finger and touches the stethoscope to the heart region, PCG signals emitted from the inside of the heart and lungs are measured by the stethoscope At the same time, fingerprint authentication may be performed.
  • the stethoscope is used as a microphone input for controlling the digital TV with a voice command
  • the smart remote control controller controls the volume of the digital TV to decrease during the period of using the stethoscope, and when the use of the stethoscope ends, the volume of the digital TV before use It can be set to automatically return to the size.
  • the stethoscope can be used as an ECG sensor.
  • the biosensor includes an infrared temperature sensor, a camera, an ECG sensor for measuring an electrocardiogram, an SpO2 sensor for measuring oxygen saturation, a PPG sensor for obtaining a photoplethysmography (PPG) signal, and a glucose light reflection signal. It may include any one or more selected from a blood glucose sensor for a blood glucose sensor, a stethoscope for obtaining a PCG signal, and hand electrodes for measuring body fat.
  • the artificial intelligence neural network receives the bio-signals and personal body information as inputs, and selects any one or more selected from an artificial intelligence neural network for blood pressure measurement, an artificial intelligence neural network for cholesterol measurement, an artificial intelligence neural network for blood sugar measurement, and a heart disease measurement unit.
  • an artificial intelligence neural network for blood pressure measurement may include, but not limited to, a blood pressure measurement, a blood pressure measurement, a blood pressure measurement, and a heart disease measurement unit.
  • the personal body information includes body fat information and a standard blood pressure value
  • the standard blood pressure value is calculated by applying [Equation 1]
  • the artificial intelligence neural network is a deep running neural network and LSTM for extracting a feature vector of a biosignal. (long short term memory), and [Equation 1] is
  • the input of the artificial intelligence neural network may use a biological signal synchronized with respect to the R point of the ECG signal.
  • the biosignal may include ECG signal, PPG signal, PCG signal, SpO2, PTT, and glucose light reflection information.
  • the blood glucose sensor may include a virtual finger pattern displayed on the touch screen; a plurality of near-infrared light emitting devices for irradiating near-infrared rays of different wavelengths to each finger part of the user aligned on the virtual finger pattern; and an optical sensor that receives the near-infrared light reflected from the finger and converts it into an electrical signal, and obtains glucose light reflection signal information from the optical sensor.
  • the voice recognition terminal or the smart remote control control unit is a health tracking management unit for observing changes in the bio-signals and medical data to inform the user of the degree of risk, inform the item requiring intensive care examination, or inform the user of the next examination schedule may include.
  • the synchronized living body measured from the user's body at the same point in time by the smart remote control of the present invention can be used as an input signal of the artificial intelligence neural network, so that accurate blood pressure measurement, cholesterol measurement, and blood sugar measurement are possible.
  • 1A and 1B are installed in an area within a patient's residence according to an embodiment of the present application, and have a voice recognition unit and a voice reproducing unit to control a digital TV by a voice command collected through a microphone, and to control a digital TV through a speaker.
  • This is an embodiment of a voice terminal that provides a feedback service.
  • 1C is an embodiment of a smart remote control according to an embodiment of the present application.
  • FIGS. 2A to 2D are an embodiment in which a biosensor including any one selected from an infrared temperature sensor, a camera, a blood sugar sensor, a hand electrode for measuring body fat, an ECG sensor, and a stethoscope according to an embodiment of the present application is integrated on a smart remote control. It is a drawing showing an example.
  • FIG. 2E is a diagram illustrating an embodiment of vein authentication by recognizing vein images of a vein blood vessel pattern of a user's fingers holding a smart remote control with a hand 800 according to an embodiment of the present application.
  • FIG. 3 is an embodiment of recognizing and authenticating a vein image of a vein blood vessel pattern of a palm of a user holding a smart remote control with a hand according to an embodiment of the present application.
  • FIG. 4 is a view illustrating various embodiments of measuring a patient's body heat using an infrared temperature sensor integrated on a smart remote controller according to an embodiment of the present application.
  • FIG. 5 is an embodiment of a body fat measuring unit that measures body fat by hand electrodes installed on a smart remote control according to an embodiment of the present application.
  • 6A to 6C are an embodiment of measuring an ECG signal by a non-contact ECG sensor installed on a smart remote control according to an embodiment of the present application.
  • FIG 7 is an embodiment of the oxygen saturation and PPG signal measurement unit using the SpO2 sensor installed in the reflective blood vessel optical sensor array module installed on both left and right sides of the smart remote control according to an embodiment of the present application.
  • FIG. 8 is a diagram showing when a user holds the smart remote control so that fingerprint authentication is performed through the thumb according to an embodiment of the present application, the remaining fingers except the thumb naturally come into contact with the reflective blood vessel optical sensor array module, so that life while using the smart remote control. This is an example of measuring the user's oxygen saturation and PPG signal in the
  • 9 is an embodiment of non-invasively obtaining a glucose light reflection signal from a user's finger by a blood glucose sensor installed on a smart remote control according to an embodiment of the present application.
  • 11A to 11D are exemplary embodiments for measuring blood pressure, cholesterol, and blood sugar by using biosignals obtained from a biosensor installed on a smart remote controller according to an exemplary embodiment of the present disclosure.
  • FIG. 12 is a schematic diagram of an embodiment in which a doctor performs remote medical diagnosis not only to self-examine the health of the fetus by using the stethoscope on the smart remote control according to the embodiment of FIG. 10 but also to remotely check the health of the fetus It is a drawing shown as
  • the artificial intelligent smart remote control device in the present invention may be used interchangeably with the smart remote control device.
  • the artificial intelligence neural network in the present invention includes an expert system.
  • the blood pressure measurement unit in the present invention may be used in combination with an artificial intelligence neural network for blood pressure measurement.
  • the cholesterol measurement unit in the present invention may be mixed with an artificial intelligence neural network for cholesterol measurement.
  • the blood glucose measurement unit in the present invention may be used in combination with an artificial intelligence neural network for blood glucose measurement.
  • the artificial intelligence neural network of the present invention may include an artificial intelligence neural network for blood pressure measurement, an artificial intelligence neural network for cholesterol measurement, an artificial intelligence neural network for blood sugar measurement, and a heart disease measurement unit.
  • the biosignal in the present invention may include information on ECG signal, PPG signal, PCG signal, oxygen saturation, PTT, and glucose light reflection signal.
  • the patient, the subject or the user in the present invention may be used interchangeably with the smart remote control user.
  • the digital TV may be replaced with a smart mirror or a computer monitor.
  • the voice recognition terminal may be integrated into the digital TV. Therefore, the voice recognition terminal can be mixed with a digital TV, a smart mirror, and a computer monitor.
  • the voice recognition terminal may be integrated into the smart remote control.
  • the personal body information 69f is information that selects at least one of body heat, sex, age, height, weight, standard blood pressure, body fat information, and HRV (heart rate variability). do.
  • FIG. 1A and 1B show a digital TV 300 installed in an area within the patient's residence, provided with a voice recognition unit 27a and a voice reproducing unit 28a, and a voice command collected through a microphone 23.
  • a voice terminal 17 that controls and provides a voice feedback service through the speaker 28, a short-range wireless communication connection with external medical devices (eg, NFC (Near Field Communication) interface, Bluetooth, Internet of Things or infrared rays) a digital communication module 8 for providing a communication connection), and a medical data receiving unit 12 for receiving medical data measured from the patient by the medical devices through the digital communication module 8 .
  • NFC Near Field Communication
  • Bluetooth Internet of Things or infrared rays
  • the voice recognition terminal 17 not only controls the digital TV 300 according to a voice command, but also performs remote medical diagnosis through the digital TV 300 .
  • the digital communication module 8 provides an Internet and Wi-Fi communication connection that allows remote medical diagnosis with a medical professional, or provides a short-range wireless communication connection with an external medical device.
  • the digital TV 300 may be replaced with a smart mirror.
  • control unit 30 has the result of analyzing the medical data from the artificial intelligence neural network 16, and according to the patient's health condition, the control unit 30 provides a guideline for health management, telemedicine and medical device usage to the patient through audio and video.
  • the digital communication module (8), the speaker (28) and the digital TV (300) are controlled.
  • control unit 30 has the result of analyzing the medical data from the artificial intelligence neural network 16, determines the need for remote medical diagnosis according to the patient's health condition, and performs remote medical diagnosis between the doctor and the patient if necessary.
  • the digital communication module 8 the speaker 28 and the digital TV (300).
  • the medical device 430 is a wireless transmitter (eg, NFC interface, Bluetooth) that transmits medical data measured from the patient's specimen or the affected part to the medical data receiving unit 12 to the medical data storage unit 15 .
  • POCT point of care testing
  • the smart remote control 400 may transmit medical data on body temperature, blood sugar, body fat, cholesterol, electrocardiogram, oxygen saturation, PCG signal, and blood pressure to the medical data receiver 12 of the voice recognition terminal 17 .
  • the wearable device is preferably a patch-type wearable device attached to the patient's skin, and data obtained by non-invasively measuring the electrocardiogram, blood pressure, cholesterol, or blood sugar on the skin surface of the patient by a sensor built into the patch. It is preferred to wirelessly transmit to the medical data receiving unit 12 to the medical data storage unit 15 through a Bluetooth communication connection.
  • the patch-type wearable device attaches a patch with a built-in blood glucose sensor to the skin with tape, and then brings the smart remote control 400 to the patch. It may be a device that provides a blood sugar level to the medical data storage unit 15 of the smart remote control 400 by measuring blood sugar by measuring the ratio of water and glucose in the blood.
  • Another aspect of the patch-type wearable device is to measure the glucose concentration of the interstitial fluid (a liquid component filling between cells) at periodic intervals through a sensor attached to subcutaneous fat such as the abdomen, arm, and buttock, so that the measurement result is measured in a short distance.
  • a sensor attached to subcutaneous fat such as the abdomen, arm, and buttock, so that the measurement result is measured in a short distance.
  • It may be a continuous glucose monitoring (CGM) that wirelessly transmits the medical data storage unit 15 to the medical data receiving unit 12 through a wireless communication connection.
  • CGM continuous glucose monitoring
  • the voice recognition terminal 17 may include a body heat diagnosis unit 19 for determining an abnormal body heat suspect.
  • the voice recognition terminal 17 observes the change in medical data according to the periodic examination by the biosensor and medical device, and additional information including the risk to the patient (user), items requiring intensive care examination, and the next examination schedule may include a health tracking management unit 11 that informs the patient.
  • the digital TV 300 collects, organizes, and analyzes the medical data collected from the smart remote control 400 and displays the results on the screen or additional information provided by the health tracking management unit 11 .
  • the voice recognition terminal 17 has an artificial intelligence neural network 16 or an expert system 18 therein, and medical data measured by the smart remote control 400 is received by the medical data receiving unit 12. After this, an embodiment in which the artificial intelligence neural network 16 or the expert system 18 can analyze this to notify the patient of the risk of the disease is shown.
  • the artificial intelligence neural network 16 is deep learning learned by medical data for learning previously collected by medical devices.
  • the voice recognition terminal 17 may apply the patient's medical data measured from the medical devices to the pre-learned artificial neural network 16 to automatically determine whether the patient has a disease or not and the risk of the disease.
  • the voice recognition terminal 17 may include an expert system 18 for determining a risk factor of a patient's disease with respect to the data collected by the medical data receiving unit 12 .
  • the expert system 18 may be a computer application program that makes decisions or solves problems by utilizing knowledge accumulated through the intellectual activities and experiences of the medical experts and inference rules defined by the medical experts.
  • the artificial intelligence neural network 16 may be an artificial intelligence neural network app installed as an application software on the voice recognition terminal 17 .
  • the data collected by the medical data receiving unit 12 may be analyzed by the artificial intelligence neural network app and the result may be provided to the user.
  • the medical data receiving unit 12 of the voice recognition terminal 17 receives the medical data measured by the smart remote control 400, and retransmits it to a server through the Internet network, and then the voice recognition terminal (17) shows an embodiment of notifying the patient of the presence or absence of the disease and the risk of the disease to the patient by receiving feedback from the results analyzed by the artificial intelligence neural network 16 or the expert system 18 on the server 13.
  • the bio-signal collecting unit 420 includes a bio-signal valid determining means 420a to discriminate valid bio-signals among bio-signals collected by the bio-signal collecting unit 420 from the bio-sensor 440, Among the bio-signals collected by the bio-sensor 440 on the remote control 400, user authentication is completed and valid bio-signals (ECG signal, PPG signal, oxygen saturation, glucose light reflection signal information, PCG signal, body fat, body temperature) Only every filter is stored in the medical data storage unit 15 .
  • ECG signal, PPG signal, oxygen saturation, glucose light reflection signal information, PCG signal, body fat, body temperature Only every filter is stored in the medical data storage unit 15 .
  • the bio-signal collection unit 420 filters out only a valid bio-signal portion from among bio-signal components for which user authentication has been completed from among the bio-signals collected from the bio-sensor 440 .
  • the biosignal validity determination means 420a includes a 1-D (Dimension) artificial intelligence neural network, LSTM (Long short time memory), Auto Correlation Coefficient, an effective range check method, and a standard (standard) biosignal characteristic. It is preferred to use any one or more techniques selected from the cross-correlation coefficient between the vector and the feature vector of the collected biosignal.
  • the cross-correlation coefficient is calculated using any one of Sum of Squared Difference (SSD), Sum of Absolute Difference (SAD), Euclidean Distance, K-nearest neighbor algorithm (KNN), and Normalized Cross Correlation (NCC). It is preferred to calculate the degree of similarity or correlation between components.
  • SSD Sum of Squared Difference
  • SAD Sum of Absolute Difference
  • KNN K-nearest neighbor algorithm
  • NCC Normalized Cross Correlation
  • the biosignal collected through the biosensor 440 is a valid biometric signal. If it a signal.
  • Standard (standard) biosignal is a biosignal (ECG signal, PPG signal, oxygen saturation, glucose light reflection signal information, PCG signal, body fat, body temperature) that shows the characteristics and values of the biosignal of a statistically normal person, standard (reference)
  • ECG signal ECG signal
  • PPG signal oxygen saturation
  • glucose light reflection signal information PCG signal
  • body fat body temperature
  • standard (reference) As the feature vector of the biosignal, average values of the feature vector for the biosignal of a normal person may be used, and a plurality of standard (reference) biosignal feature vectors may be used.
  • the feature vector of the biosignal may be extracted by a deep learning neural network or spectrum analysis in the frequency domain.
  • the deep learning neural network may be a 1-D convolutional neural network (CNN).
  • CNN convolutional neural network
  • the feature vector by spectrum analysis of the frequency domain is 1-D (Dimension) artificial intelligence neural network, frequency component analysis, STFT (Short time Fourier Transform), wavelet transform (Wavelet transform), MFCC (Mel-Frequency Cepstral Coefficient), It may be extracted by Linear Prediction Codes (LPC), Mel-Frequency Cepstral Coefficient (MFCC), Mel-Coefficient, Preceptual Linear Prediction (PLP), or Bark Frequency Cepstral Coefficients (BFCC).
  • LPC Linear Prediction Codes
  • MFCC Mel-Frequency Cepstral Coefficient
  • PDP Preceptual Linear Prediction
  • BFCC Bark Frequency Cepstral Coefficients
  • the 1-D artificial intelligence neural network or the LSTM-based biosignal validity determination means 420a is a 1-D artificial intelligence neural network learned using ECG signals, PPG signals, and PCG signals labeled with valid or invalid biosignals. By this, it is possible to determine the validity of the biosignal of the subject.
  • the effective range is preferably 10 to 40 degrees, and when the body temperature within this range maintains the same temperature state for 1 second or more, it is preferably recognized as an effective body temperature signal.
  • the user authentication may be performed by the fingerprint recognition unit 24, 25, 26, 27 or the vein sensor unit 54a.
  • the biosensor 440 includes an electrocardiogram measuring unit 71 , oxygen saturation and PPG signal measuring unit 72 , body fat measuring unit 70 , and auscultation sound measuring unit 73 , and the electrocardiogram measuring unit 71 .
  • ECG sensor 48 for measuring the electrocardiogram, oxygen saturation and PPG signal measuring unit 72, SpO2 sensor 54b and PPG sensor for measuring oxygen saturation and photo-only red pulse wave signal, and auscultation sound measuring unit 73 is a stethoscope 44 for obtaining a heart sound signal
  • the body fat measuring unit 70 includes a plurality of hand electrodes 24a, 25a, 26a, and 27a for measuring body fat.
  • the smart remote control controller 53 may include a biosignal measurement request means to transmit a touch screen text or voice message requesting biosignal measurement to the user.
  • the intensive care item is blinked, or the user interactively uses the speaker unit 38a for voice reproduction and the microphone unit 38b for voice recognition. You may request to measure biosignals.
  • the smart remote control control unit 53 may include a health tracking management unit that notifies the user of the degree of risk by observing changes in biosignals and medical data, informs the user of items requiring intensive care examination, or informs the user of the next examination schedule. have.
  • Another aspect of the smart remote control 400 according to FIG. 1c is an artificial intelligence neural network app that resides on the smart remote control instead of using the artificial intelligence neural network 16 on the server 13 and is installed as application software. is provided, analyzing the medical data measured by the biosensor, and providing the result to the user.
  • the artificial intelligence neural network app provides the advantage of continuously improving the performance of the smart remote control according to the version upgrade.
  • Another aspect of the smart remote control 400 according to FIG. 1C is to receive medical data measured by external medical devices 430 through a short-range communication connection or use the bio-signal collection unit 420 from the biosensor 440 . and a medical data storage unit 15 for storing the medical data obtained through the wireless communication connection means 37 for transmitting the medical data to the voice recognition terminal 17, wherein the voice recognition terminal 17 includes the wireless communication
  • the medical data provided from the connection means 37 is transmitted to the artificial intelligence neural network 16 to the expert system 18 on the server 13 through the Internet 202, and the artificial intelligence neural network 16 to the expert system 18 ) and provides feedback to the user through the digital TV 300 .
  • the bio-signals measured from the patient by the smart remote control 400 can be used for medical treatment by short-range wireless communication connection. It can be wirelessly transmitted to the data receiving unit 12 or stored in the medical data storage unit 15 .
  • 2a (a) and 2b are one embodiment of a front view of the smart remote control 400, a touch screen 401 for selecting health care items or displaying remote control selection buttons on the front of the smart remote control 400 provided
  • FIG. 2A (a) is an embodiment of the remote control mode in which remote control buttons for controlling functions of the digital TV 300 are displayed on the touch screen 401 during the remote control mode, and (b) of FIG. 2A is in the health care mode This is an embodiment in which health care items are displayed on the touch screen 401 of
  • the smart remote control 400 is provided with a menu button for alternately switching between the remote control mode and the health care mode screen on the touch screen 401, and during the remote control mode, remote control selection buttons for controlling the functions of the digital TV 300 are provided. Displayed on the touch screen 401 screen, menu selection buttons for selecting one of health care items including an electrocardiogram, oxygen saturation test, blood sugar test, body fat test, cholesterol test, blood pressure test, etc. are provided during the health management mode.
  • the touch screen 401 may be displayed on the screen.
  • the remote control mode user authentication and medical data collection and transmission of biosignals for health care items including electrocardiogram, oxygen saturation test, blood sugar test, body fat test, cholesterol test, and blood pressure test are unconsciously performed in daily life.
  • the artificial intelligence neural network analyzes these transmitted medical data and feeds back the result, and the health tracking management unit of the smart remote control 400 discovers health care items requiring intensive care, and discovers intensive care items It is preferred to induce self-examination through the health management mode.
  • the smart remote control controller 53 displays the intensive care items by blinking on the touch screen 401 so that the user can self-examine the intensive care items. Biosignal measurement can be requested.
  • Reference numeral 39 denotes a remote control transmitter that transmits a signal for controlling a digital TV.
  • 2B shows the fingerprint authentication units 24, 25, 26, 27 and four hand electrodes 24a, 25a, 26a, and 27a installed on the upper, lower, left, and right sides of the smart remote control 400 .
  • the plurality of hand electrodes 24a, 25a, 26a, and 27a are configured in such a way that the outer periphery of the fingerprint authentication unit 24, 25, 26, and 27 is wrapped with a metal conductor, and the fingerprint portion of the finger is the hand electrode during fingerprint authentication. are also brought into contact at the same time. In this case, these hand electrodes operate as biosensors when measuring body fat or measuring an electrocardiogram.
  • the hooks 14a, 14b, 14c, and 14d of the hand electrodes 24a, 25a, 26a, and 27a are physically engaged with the connection electrodes 24b, 25b, 26b, and 27b when assembling and are electrically connected at the same time. .
  • 2c is a rear view of the smart remote control 400 according to various embodiments of the present invention, and the stethoscope 44 and the ECG sensor 48 are provided on the rear portion of the smart remote control 400 .
  • the stethoscope 44 can also be used as a microphone input for controlling the digital TV 300 with a voice command.
  • there is a microphone input at a place somewhat distant from the digital TV 300 and thus mixing of TV noise is minimized, which is advantageous for voice command control for the digital TV 300 even when the TV is on.
  • the smart remote control controller 53 controls the volume of the digital TV 300 to automatically decrease during the period of use of the stethoscope 44, amplifies the sound of the heart or lungs, and transmits the sound to the medical data receiver 12 or medical data It is preferred to store it in the storage unit 15 .
  • the stethoscope 44 usage period determination is preferably calculated when a valid ECG signal is acquired.
  • the smart remote control control unit 53 controls the stethoscope 44 while using the stethoscope to control the bell mode suitable for heart sounds (20 Hz to 200 Hz), diaphragm mode suitable for lung sounds (200 Hz to 1 KHz), the above
  • the measured stethoscope measurement data may be transmitted to the medical data receiver 12 or stored in the medical data storage unit 15 .
  • the electrocardiogram measurement is performed by amplifying the voltage signal formed between the first electrode and the second electrode, and the first electrode uses the fingerprint authentication unit 24, 25, 26, 27), the thumb naturally makes contact with any one of the four hand electrodes 24a, 25a, 26a, 27a, and the second electrode is formed by the ECG sensor 48 on the skin (eg chest). formed when in contact.
  • the user's electrocardiogram (ECG) signal may be measured through the two electrodes (the first electrode and the second electrode) spaced apart from each other. That is, while the user is holding the smart remote control 400 , the first electrode formed by contacting the thumb with any one of the four hand electrodes 24a , 25a , 26a , and 27a , another part of the user's body (eg, The second electrode formed by bringing the chest) into contact with the ECG sensor 48, in which case the voltage signal formed between the first electrode and the second electrode is amplified to measure the user's electrocardiogram (ECG) signal.
  • ECG electrocardiogram
  • the ECG sensor 48 may be replaced with an ECG electrode surface formed by coating the surface of the thin film sounding plate 44a of the stethoscope 44 with a conductive material and coating an insulating film thereon again.
  • the thin film sounding plate 44a The cardiopulmonary sound signal (PCG signal) and the electrocardiogram signal are simultaneously collected.
  • ECG measurement is to amplify the voltage signal formed between the first electrode and the second electrode
  • the first electrode is the hand electrodes installed on the side of the smart remote control 400 while the user is holding the smart remote control 400 .
  • (24a, 25a, 26a, 27a) is formed by contact between any one of the thumb and the right thumb
  • the second electrode is formed by contact between the touch screen (touch-type ECG sensor) provided with the ECG sensor and the left finger.
  • the user's electrocardiogram (ECG) signal is measured through these two electrodes (the first electrode and the second electrode). For example, while watching TV, the user of the smart remote control 400 frequently touches a button on the touch screen 401 with the opposite finger (eg, index finger) while holding the smart remote control in one hand to select a function of the TV.
  • ECG electrocardiogram
  • Example 2 compared to Example 1, the surface area of the hand electrodes 24a, 25a, 26a, and 27a disposed on the side of the smart remote control 400 is increased to maximize the contact area with the user's hand.
  • Figure 2d is an embodiment of a side view of the smart remote control 400, provided with fingerprint recognition units 24, 25, 26, 27 for performing fingerprint authentication of the thumb installed on the top, bottom, left and right sides of the smart remote control 400, , who, and when the smart remote control 400 can be grasped in real time.
  • the fingerprint recognition units 24, 25, 26, and 27 of the smart remote control 400 have a rather small fingerprint recognition area, but there is no practical problem in personal authentication between a small number of families (less than 10 people).
  • the fingerprint recognition units 24, 25, 26, and 27 for authenticating the fingerprint of the thumb are installed in four places on the upper, lower, left, and right places, so that the user can hold the smart remote control 400 with his hand in any direction, in four places.
  • the thumb is positioned in at least one of the fingerprint recognition units, so that fingerprint authentication is performed regardless of the shape and direction of holding the smart remote control 400 .
  • the fingerprint recognition unit 24, 25, 26, 27 of the present invention preferably uses a capacitive, ultrasonic or optical fingerprint recognition sensor.
  • the ultrasonic fingerprint sensor generates a fingerprint image by measuring the time it takes to return after hitting the curve of the fingerprint after emitting ultrasonic waves from the fingerprint sensor when a finger is placed on the fingerprint sensor window, and then compares and matches with the previously registered fingerprint DB ( matching) to authenticate the input fingerprint.
  • the optical type is a method of authenticating an ID by obtaining a fingerprint image by an image sensor, comparing and matching (matching) it with a pre-registered fingerprint DB.
  • the fingerprint recognition unit of the present invention clarifies ridges and valleys on the fingerprint image by using a thinning algorithm for the fingerprint image.
  • the comparison matching for fingerprint authentication includes a process of extracting minutiae information from ridges and valleys, and determining the similarity (matching score) between fingerprint images by comparing it with minutiae information of a previously registered fingerprint image.
  • the key point information may include the type of the key point, the direction of the ridge where the key point is located, and the location of the key point in the fingerprint image.
  • the type of the characteristic point includes an ending point at which the flow of the ridge boils or a bifurcation at which the ridge is split.
  • hand electrodes 24a, 25a, 26a, and 27a are provided on the edges of the fingerprint recognition unit 24, 25, 26, and 27, so that when the fingerprint part of the finger comes into contact with the fingerprint authentication unit, the hand electrode is also simultaneously provided. have a structure
  • fingerprint authentication is performed and the user's body fat or ECG signal can be measured by the hand electrodes 24a, 25a, 26a, and 27a.
  • reflective blood vessel optical sensor array modules 35 and 36 are installed on the left and right sides of the smart remote control 400 .
  • the reflective blood vessel optical sensor array modules 35 and 36 may include a plurality of vein sensor units 54a and a plurality of SpO 2 sensors 54b.
  • the vein sensor unit 54a is composed of a near-infrared LED 52a and a photo detector 50, and when the user holds the side of the smart remote control 400 with a finger 66, A vein image for the blood vessel pattern of the vein 56 on the inside of the finger 66 is obtained and used for vein authentication.
  • the SpO2 sensor 54b is composed of a red light LED 51a, an infrared LED 51b, and a photodetector 50.
  • the photodetector 50 When the user holds the side of the smart remote control 400 with a finger 66, the photodetector 50 ) to measure the oxygen saturation from the peripheral vascular artery (55) of the finger.
  • the vein sensor unit 54a of the present invention includes a near-infrared LED 52a for projecting near-infrared (eg, 850 nm) to the finger 66 to obtain a vein blood vessel pattern image, and then a photodetector for acquiring a reflected finger vein image. It consists of (50).
  • the photodetector 50 for acquiring the vein image is preferably an image sensor.
  • veins contain hemoglobin, they have the property of absorbing near-infrared rays, so veins generate a darker image compared to the skin area in the vein image.
  • a branching point set is formed by extracting branch points of the venous vascular path from the vein image through a thinning algorithm and noise removal for the vein image obtained through the photodetector 50 .
  • vein authentication of the present invention it is preferred to authenticate the user by calculating a matching score based on the similarity between the branch point set extracted from the vein image and the branch point set of other previously registered vein images.
  • the threshold value of the matching score for identifying the same person may be adjusted in consideration of the number of users sharing the smart remote control 400 together.
  • the oxygen saturation measurement of the present invention when the user holds the smart remote control 400 so that fingerprint authentication is performed through the thumb, the fingers other than the thumb naturally come into contact with the reflective blood vessel optical sensor array modules 35 and 36, It is possible to measure the oxygen saturation of the user in daily life while using the smart remote control 400 .
  • the oxygen saturation (SpO2) is irradiated to the peripheral arterial blood vessel 55 of the finger 66 by sequentially emitting the red light LED 51a and the infrared LED 51b sequentially by one cycle, and then reflected and reflected by the photodetector 50 It can be measured by observing the change in the amount of light received.
  • the oxygen saturation level of the present invention is determined using one or more fingers 66 of an index finger, a middle finger, and a ring finger in contact with the reflective blood vessel optical sensor array modules 35 and 36. It is preferred to measure
  • FIG. 2e shows vein images 120b, 120c, 120d, and 120e showing the venous blood vessel pattern of the user's fingers 800b, 800c, 800d, and 800e holding the smart remote control 400 with the hand 800.
  • An example of vein authentication is shown. After seeing blood vessels using near-infrared rays, the user's identity of the smart remote control 400 may be authenticated by comparing the reflected vein image with the template image to calculate a matching score.
  • vein image having the highest matching score among the vein images 120b, 120c, 120d, and 120e for user authentication.
  • 3(a) to 3(b) show a vein image 120f from a vein blood vessel pattern of a user's palm 800f holding the smart remote control 400 with a hand 800 as another embodiment of vein authentication.
  • An example of recognizing and authenticating is shown. After seeing blood vessels using near-infrared rays, the identity of the smart remote control user can be authenticated by comparing the reflected vein image with the template image to calculate a matching score.
  • the vein sensor unit 54a disposed on the back of the smart remote control 400 includes a near-infrared LED 52a for projecting near-infrared light on the palm 800f and a photodetector 50 for acquiring a vein image of the palm 800f. is configured, and when the user holds the smart remote control 400, a vein image 120f for the vein blood vessel pattern in the inside of the palm 800f is obtained by the photodetector 50 and used for vein authentication.
  • the stethoscope 44 also serves as the ECG sensor 48 by coating the surface of the thin film sounding plate 44a of the stethoscope 44 with a conductive material and coating an insulating film thereon.
  • FIG. 4 shows various embodiments of measuring a patient's body heat using the infrared temperature sensor 29 integrated on the smart remote control 400 .
  • the patient's temperature information measured by the infrared temperature sensor 29 is transferred to the medical data receiving unit 12 of the voice recognition terminal 17. transmit wirelessly.
  • the voice recognition terminal 17 When the temperature is measured by the fingerprint recognition unit 24, 25, 26, 27 or the vein sensor unit 54a on the smart remote control 400, the voice recognition terminal 17 through a short-range wireless communication connection is notified of who is measuring the temperature. can tell you
  • the patient When measuring the temperature, the patient should touch the thumb to the fingerprint recognition unit 24, 25, 26, 27 so that the fingerprint recognition can be performed smoothly.
  • the infrared temperature sensor 29 is preferably disposed on the front of the smart remote control 400 like the camera 22 .
  • the infrared temperature sensor 29 measures body heat with the amount of infrared radiation emitted from the user of the smart remote control 400 without contacting the measurement target.
  • the infrared temperature sensor may measure body heat at a distance of 3 cm to 5 cm from the subject's (user's) forehead.
  • the camera 22 may provide a guide for optical alignment with the measurement position of the infrared temperature sensor by recognizing the patient's face region and forehead region to provide the user with a guide.
  • the user ID information and the measured temperature value authenticated by the fingerprint recognition unit 24, 25, 26, 27 or the vein sensor unit 54a are wirelessly transmitted to the medical data receiving unit 12 and ,
  • the speaker 28 of the voice recognition terminal 17 provides voice feedback to the patient about the temperature value or the measurement error occurrence history (measurement area error, authentication error).
  • FIG. 4 (b) shows a case in which the voice recognition terminal 17 of the present invention is integrated and integrated inside the digital TV 300.
  • the digital communication module 8 is located inside the digital TV 300, A medical data receiving unit 12 , a body heat diagnosis unit 19 , an artificial intelligence neural network 16 , and a control unit 30 may be included.
  • the body heat diagnosis unit 19 checks whether the user is pre-registered by the face recognition unit, the fingerprint recognition unit 24, 25, 26, 27, or the vein sensor unit 54a to identify the identity. It can be activated after performing authentication.
  • the body heat diagnosis unit 19 may assume (determine) a subject suspected of having an abnormal body heat.
  • 5A and 5B show an embodiment of the body fat measuring unit 70 for measuring body fat by hand electrodes 24a, 25a, 26a, and 27a installed on the smart remote control 400 .
  • the hand electrodes 24a, 25a, 26a, and 27a are used as driving electrodes and detection electrodes.
  • the remaining hand electrodes 26a and 27a are used as detection electrodes.
  • a method of measuring body fat using the driving electrodes 24a and 25a and the detection electrodes 26a and 27a will be described.
  • the user holds the driving electrodes 24a and 25a on the left side of the smart remote control 400 using the thumb and index finger of the left hand, and at the same time the thumb and index finger of the right hand are on the right side of the smart remote control 400 . Hold the detection electrodes 26a and 27a on one side.
  • the current passed through the user's left thumb in contact with the driving electrodes 24a and 25a passes through the detection electrodes 26a and 27a, and the impedance value of the current passed into the user's body from the passed current save
  • the body fat measuring unit 70 applies a minute current (eg, about 500 ⁇ A) into the body while the person to be measured uses the thumb and index finger of both hands to hold the driving electrodes 24a and 25a and the detection electrodes 26a and 27a.
  • a minute current eg, about 500 ⁇ A
  • body fat information including body fat percentage, body mass index (BMI), and muscle mass is obtained, and this is transmitted to the medical data receiver 12 It is preferred to do
  • the body fat measuring unit 70 includes driving electrodes 24a and 25a for applying a driving signal for measuring bio-impedance to the user, driving electrodes ( A sine wave oscillator 40 for generating a sine wave driving signal (eg, 10 to 50 KHz) to be applied to 24a, 25a), detection electrodes 26a and 27a for detecting a signal received through the human body, detection electrodes An RMS (Root-Mean-Square) voltage detector 41 for detecting the rms voltage of the signal received through 26a, 27a, an AD converter 46 for converting the output of the RMS voltage detector into a digital signal, and a voice Consists of a wireless communication connection means 37 that transmits the data converted by the AD converter 46 to the medical data receiver 12 through the smart remote control controller 53 through a short-range wireless communication connection with the recognition terminal 17 do.
  • a sine wave oscillator 40 for generating a sine wave driving signal (eg, 10 to 50 KHz) to be applied to 24a, 25a)
  • bio-signal collecting unit 420 determines whether the body fat information converted by the AD converter 46 has completed user authentication and whether it is a valid bio-signal, and the smart remote control controller 53 determines whether the user-authenticated valid bio-signal is valid. Only information may be stored in the medical data storage unit 15 and transmitted to the medical data receiving unit 12 .
  • the ECG sensor 48 of the present invention is composed of an insulating film 48a and an ECG electrode surface 48b that allow for capacitive coupling, and shows minute electrical fluctuations emitted from myocardial cells by capacitive coupling with the human skin surface 47. ECG signal is sensed.
  • the human skin surface 47 is preferably the skin surface of the chest area or the area of finger prints.
  • the ECG electrode surface 48b is preferably a conductive copper thin film, and is input to the differential amplifier 61 through the ECG electrode 48c.
  • the ECG electrode surface 48b is coated with an insulating film 48a.
  • 6A (a) is an embodiment in which the ECG sensor 48 is installed on the rear part of the smart remote control 400 and acquires an ECG signal by capacitive coupling when it comes into contact with the human skin surface (eg, chest).
  • the human skin surface eg, chest
  • the thumb naturally makes contact with any one of the four hand electrodes 24a, 25a, 26a, and 27a to form the first electrode.
  • the ECG sensor 48 is in contact with the chest region to form a second electrode.
  • the first electrode and the second electrode may form two electrodes spaced apart from each other to measure the user's electrocardiogram (ECG) signal.
  • ECG electrocardiogram
  • FIG. 6a (b) is an embodiment of a touch-type ECG sensor 49 in which the ECG sensor 48 is integrated with the touch screen 401 on the front of the smart remote control 400, and the touch screen 401 An ECG signal can be acquired by capacitive coupling with a finger at every touch.
  • the touch-type ECG sensor 49 is integrated with the touch screen 401 and the ECG sensor 48 on the front face of the smart remote control 400, so that the application corresponding to the coordinates of the contact location according to the finger touch. Not only motion control is possible, but the user's ECG signal is sensed at the same time.
  • any one of the four hand electrodes 24a, 25a, 26a, 27a and the thumb come into contact with the first electrode
  • the finger also comes into contact with the ECG sensor 48 integrated on the touch screen 401 to form a second electrode, and the user's electrocardiogram ( ECG) signal can be measured.
  • ECG electrocardiogram
  • the user of the smart remote control 400 frequently touches the function buttons on the touch screen with the finger (eg, index finger) of the opposite side while holding the smart remote control 400 with one hand. It is possible to collect the user's ECG signal without an additional personal authentication procedure in the
  • 6b is an embodiment of the electrocardiogram measuring unit 71 for removing and amplifying the noise of the ECG signal obtained from the ECG sensor 48 and transmitting it to the voice recognition terminal 17.
  • Ripple noise mixed in the supply power A differential amplifier 61 having a common mode rejection ratio (CMRR) circuit for removing , High Pass Filter, 62), a Band Rejection Filter (BRF, Band Rejection Filter, 63) for removing the 50Hz or 60Hz power component, and the ECG signal output through the high pass filter 62 and the band rejection filter 63 Wireless transmitting the ECG signal converted by the AD converter 64 to the medical data receiving unit 12 through a short-range wireless communication connection with the AD converter 64 and the voice recognition terminal 17 for amplifying and converting to a digital signal and communication connection means (37).
  • CMRR common mode rejection ratio
  • BRF Band Rejection Filter
  • fingerprint authentication is performed by the thumb 800a in contact with the first electrode (eg, reference numeral 24a) during the electrocardiogram measurement, or the user's fingers 800b, 800c, 800d holding the smart remote control 400
  • the user may be authenticated by the vein sensor unit 54a for authenticating the vein pattern.
  • the biosignal collecting unit 420 determines whether the ECG signal digitized by the AD converter 64 is user authentication completed and a valid ECG signal, and the smart remote control controller 53 is a valid ECG signal authenticated by the user. is stored in the medical data storage unit 15 and transmitted to the medical data receiving unit 12 through the wireless communication connection means 37 .
  • 6c is an embodiment of the touch-type ECG sensor 49 in which the ECG sensor 48 and the resistive touch screen 401 are integrated, the display panel 100 is laminated on the display panel 100
  • a first resistive film pattern 200a that is formed on the upper surface of the lower transparent film 200 and informs the coordinates of the X-axis when touched, is formed at the lower end of the upper transparent film 500 and is a first resistive film pattern 200a and the second resistive layer pattern 500a indicating Y-axis coordinates when touched, the X-axis electrodes 200b installed at the ends of the first resistive layer pattern 200a, and the second Y-axis electrodes 500b installed at the ends of the resistive film pattern 500a and the ECG electrode 48c installed at one terminal of the ECG sensor 48 may be included.
  • the ECG sensor 48 can be obtained by forming an insulating film 48a on the ECG electrode surface 48b, and by forming an ECG electrode 48c at one end of the ECG electrode surface 48b, it is applied to the input of the differential amplifier 61. connected,
  • the ECG electrode surface 48b provides capacitive coupling when in contact with the skin.
  • the scan timing of the TX switching unit 144a that sequentially applies scan pulses to the Y-axis electrodes 500b of the second resistive film pattern 500a in order to calculate the finger touch coordinates on the touch screen 401
  • the second resistive film pattern 500a and the first resistive film pattern 200a are in contact with the TX scan unit 140 that controls and generates a scan pulse and the pressure at the touched point, and at this time, the X-axis electrode 200b and an RX scan unit 142 that controls the switching timing of the RX switching unit 144b to receive the transmitted scan pulse, and the smart remote control unit 53 includes a TX switching unit ( 144a) and the RX switching unit 144b can calculate the time synchronization relationship to calculate the X, Y coordinates of the finger contact position.
  • the first resistive layer pattern 200a, the second resistive layer pattern 500a, and the ECG electrode surface 48b are formed of ITO ((Indium tin oxide), TAO (tin antinomy oxide), TO (tin oxide), Indium-zinc-oxide (IZO), AZO (Al-doped ZnO), ZnO (zinc odixe), metal mesh, nano ink, Ag Nano Wire, conductive polymer, graphene, carbon nanotube Or it may be formed of any one or more transparent conductive materials selected from these composites.
  • the transparent films 200 and 500 may be formed of an insulating film using at least one of glass, polyethylene terephthalate (PET), polyethylene naphthalate (PEN), polyimide (PI), and acryl.
  • PET polyethylene terephthalate
  • PEN polyethylene naphthalate
  • PI polyimide
  • the first resistive film pattern 200a and the second resistive film pattern 500a may be obtained by depositing or coating a transparent conductive material on the insulating transparent film.
  • FIG. 7 shows an embodiment of the oxygen saturation and PPG signal measurement unit 72 using the SpO2 sensor 54b in the reflective blood vessel optical sensor array modules 35 and 36 installed on both left and right sides of the smart remote control 400. .
  • Blood oxygen saturation measurement is a method of acquiring a signal using the change in blood volume due to contraction and relaxation of the heart and the change in the relative ratio between oxyhemoglobin and reduced hemoglobin. On the other hand, it takes advantage of the fact that in the infrared region (eg 940 nm), the light absorption of oxyhemoglobin is higher than that of hemoglobin.
  • oxygen saturation is measured by a plurality of SpO2 sensors 54b composed of a red light LED 51a, an infrared LED 51b and a photo detector 50 at the fingertip, and smart
  • the red light LED 51a and the infrared LED 51b are sequentially staggered by one cycle by the switching circuit 85 controlled by the remote control controller 53, respectively, to the peripheral blood vessels of the fingers 800b, 800c, and 800d.
  • the photodetector 50 After irradiation, the photodetector 50 detects the amount of light reflected from the blood vessel in which the irradiated light flows in the blood, and the amount of oxygenated hemoglobin and reduced hemoglobin is calculated from the electrical signal changed into a current according to the intensity of light, and from this, oxygen Calculate the saturation.
  • the smart remote control controller 53 controls the red light LED 51a and the infrared LED 51b to alternately irradiate light with a set time difference by a switch operation of the switching circuit 85 .
  • the differential amplifier 84 amplifies the electrical signal output from the photodetector 50 after current-voltage conversion, removes noise by the filter 81 and converts the analog signal to digital by the AD converter 82 .
  • the light emitting diode driver 80 receives the feedback of the output value of the photodetector 50 and then controls the driving current of the red light LED 51a and the infrared LED 51b so that an electric signal of sufficient intensity is output from the photodetector 50 . to be. This is to adjust the amount of light applied to the skin of the finger because the absorption rate of light is different for each skin of a person's finger.
  • the smart remote control control unit 53 controls the driving intensity of the infrared light and the RED light with the DC level read through the AD converter 82 so that the red light LED and the infrared LED have the same DC level. It is preferred to run
  • the smart remote control control unit 53 is synchronized with the switch operation of the switching circuit 85, converted into a digital signal by the AD converter 82 from the photodetector 50, and from the received composite reception signal to IR light. After separating the signal component (IR component) by the RED light and the signal component by the RED light (the RED component), the oxygen saturation calculation is performed by the oxygen saturation calculation means (not shown) provided in the smart remote control controller 53 .
  • the oxygen saturation calculation means after inverting the polarity of the digitized IR component and the RED component, respectively, extracting only the AC component from which the DC component is removed and using the obtained oxygen saturation value is stored in the medical data storage unit 15 Preferred.
  • the alternating current (AC) component represents the amount of light absorbed by blood passing through peripheral blood vessels
  • the direct current (DC) component represents the amount of light absorbed by capillaries or tissues such as muscle, epidermis, and bone.
  • the oxygen saturation calculation means calculates the oxyhemoglobin (HbO2) concentration and the deoxyhemoglobin (Hb) concentration by the intensity of the signal received by the photodetector 50, in which case the oxygen saturation (SpO2) can be calculated by the same formula as HbO2 concentration/(HbO2 concentration+Hb concentration).
  • the oxyhemoglobin (HbO2) concentration is the intensity of the signal received by the photodetector 50 when the red light LED is on
  • the deoxyhemoglobin (Hb) concentration is when the infrared LED is on. It is preferred to use the intensity of the signal received by the photodetector 50 at the time. At this time, it is preferable to calculate the oxyhemoglobin (HbO2) concentration and the deoxyhemoglobin (Hb) concentration using only the AC component by excluding the DC component from the intensity of the signal received by the photodetector 50 .
  • Another aspect of the oxygen saturation calculation means is to first calculate the Red2IR ratio as shown in Equation 2 below, and then convert the Red2IR ratio value by the Loop up table based on the Beer-Lambert law to obtain the oxygen saturation. have.
  • RED DC is the average value of the direct current components (DCs) of the red light detected by the photodetector 50 when the red light LED 51a is on
  • IR DC is detected by the photodetector 50 when the infrared LED 51b is turned on. It is the average value of the direct current components (DC) of the infrared rays.
  • RED AC is the average value of AC components (AC) of the red light detected by the photodetector 50 when the red light LED 51a is On
  • IR AC is the average value of the AC components (AC) of the red light detected by the photodetector 50 when the infrared LED 51b is On. It is the average value of the alternating current components (AC) of infrared rays.
  • oxygen saturation and PPG signal measuring unit 72 using the SpO2 sensor 54b can also measure the PPG signal.
  • the photoplethysmographic wave (PPG) signal continuously emits only the red light LED 51a of the SpO2 sensor 54b installed in the smart remote control 400, irradiates the finger, and then is reflected and received.
  • the PPG signal can be measured by continuously observing the change in the amount of light by the photodetector 50 .
  • the smart remote control controller 53 controls the switch operation of the switching circuit 85 so that only the red light LED 51a continuously emits light.
  • the PPG signal reflected from the finger vein is amplified after current-voltage conversion through the differential amplifier 84, and the amplified signal is configured to pass through a high-pass filter (eg Cutoff 0.1Hz) and a low-pass filter (eg Cutoff 12Hz). .
  • a high-pass filter eg Cutoff 0.1Hz
  • a low-pass filter eg Cutoff 12Hz
  • the PPG signal that has passed through the filter 81 is converted into a digital signal by the AD converter 82 and is input to the smart remote controller 53 through the biosignal collecting unit 420 .
  • the smart remote control controller 53 After inverting the polarity of the digitized PPG signal, the smart remote control controller 53 extracts only the AC component from which the DC component is removed and stores the final PPG signal in the medical data storage unit 15 .
  • bio-signal collection unit 420 determines whether the IR component or the RED component digitized by the AD converter 82 has completed user authentication and whether it is a valid bio-signal, and the smart remote control unit 53 performs user authentication Only the valid biosignal information (oxygen saturation or PPG signal) obtained is stored in the medical data storage unit 15 and transmitted to the medical data receiving unit 12 through the wireless communication connection means 37 .
  • FIG. 8 shows that when a user holds the smart remote control 400 so that fingerprint authentication is performed through the thumb 800a, the fingers 800b, 800c, and 800d except the thumb 800a naturally become reflective blood vessel optical sensor array modules.
  • An embodiment is shown in which the user's oxygen saturation and PPG signal 68c are measured in daily life while the smart remote control 400 is used by making contact with (35,36).
  • a plurality of SpO2 sensors are integrated in the reflective blood vessel optical sensor array modules 35 and 36 installed on both left and right sides of the smart remote control 400 .
  • the red light LED 51a and the infrared LED 51b are sequentially emitted alternately by one cycle, irradiated to the peripheral arterial blood vessels 55 of the fingers 800b, 800c, 800d, and then reflected IR light and RED light are received by the photodetector 50 and calculated as a component ratio therebetween, while the PPG signal 68c emits only the red light LED 51a to the peripheral arterial vessels 55 of the fingers 800b, 800c, and 800d.
  • SpO2 oxygen saturation
  • the PPG signal reflected from the finger vein includes both the AC component 68b and the DC component 68a, only the AC component 68b is extracted and used to obtain a final PPG signal.
  • Oxygen saturation and PPG measurement of the present invention is performed using any one or more of the index finger, middle finger, and ring finger in contact with the reflective blood vessel optical sensor array modules 35 and 36. It is preferable to measure and use statistical values.
  • the statistical value is preferably calculated as an average value or a median value of measured values within 3-sigma ( ⁇ : standard deviation).
  • the PPG signal can be measured using the SpO2 sensor 54b. As shown in the embodiment of FIG. 8, while measuring the PPG signal using the middle finger 800c, oxygen saturation can be measured using the ring finger 800d. . In addition, the PPG signal and the oxygen saturation measurement can be periodically alternated with each other by changing fingers.
  • the blood glucose sensor 31 includes a plurality of near-infrared light emitting devices 31a, 31b, and 31c irradiating near-infrared rays to a finger through an opening on the touch screen 401, and a muscle reflected from the finger portions 801b, 801c, and 801d. It may include optical sensors 32a, 32b, and 32c that receive infrared light and convert it into an electrical signal.
  • the electrical signals obtained from the optical sensors 32a, 32b, and 32c are amplified after current-voltage conversion, noise is removed by a filter, and then digitally converted by an AD converter to obtain glucose light reflection signal information 69e. .
  • the AC component as the glucose light reflection signal information 69e by excluding the DC component from the intensity of the signal received by the optical sensors 32a, 32b, and 32c.
  • the user's blood sugar can be measured by using the glucose light reflection signal information 69e obtained by collecting the near-infrared signal reflected from the finger.
  • the sensitivity of glucose and the skin depth through which near-infrared rays can penetrate depend on the wavelength, when glucose light reflection signal information 69e collected using light emitting devices of different near-infrared wavelengths is used, when measuring blood glucose accuracy can be increased.
  • a virtual finger pattern 34 is displayed on the touch screen 410 and At this time, when the user aligns his/her finger with the virtual finger pattern 34 and puts it on the touch screen 401, the glucose sensor 31 collects glucose light reflection signal information 69e from each finger. Thus, it is provided to the smart remote control controller 53 .
  • the near-infrared light emitting devices 31a, 31b, and 31c are arranged to be optically aligned with each finger on the virtual finger pattern 34 .
  • the opening of the touch screen 401 may be formed as a cover glass layer of the touch screen 401 with respect to the optical axis of the near-infrared light emitting devices 31a, 31b, and 31c.
  • the vein sensor unit 54a While obtaining the glucose light reflection signal information 69e by the blood glucose sensor 31, the vein sensor unit 54a authenticates the venous blood vessel pattern of the user's fingers 800b, 800c, and 800d holding the smart remote control 400. ), the user can be authenticated.
  • fingerprint authentication may be performed through the user's thumb 800a holding the smart remote control 400 .
  • the glucose light reflection signal information 69e of the present invention irradiates near infrared rays to three fingers including the index finger 801b, middle finger 801c, and ring finger 801d, and collects the near infrared rays reflected from the three fingers by an optical sensor. It can be obtained by receiving light with (32a, 32b, 32c).
  • the near-infrared light emitting device preferably has a laser or LED having a wavelength between 900 nm and 2000 nm, and more preferably has a wavelength range of 940 nm, 1320 nm, and 1550 nm.
  • reference numeral 31a denotes an LED that emits near-infrared rays in a wavelength range of 920 nm to 960 nm, and the near-infrared rays reflected from the index finger end 801b may be sensed by the photosensor 32a.
  • Reference numeral 31b denotes an LED that emits near-infrared rays in a wavelength range of 1300 nm to 1340 nm, and the near-infrared rays reflected from the middle end 801c may be sensed by the photosensor 32b.
  • Reference numeral 31c denotes an LED that emits near-infrared rays in a wavelength region of 1530 nm to 1570 nm, and the near-infrared rays reflected from the ring finger end 801d may be sensed by the photosensor 32c.
  • Stethoscope 10 is an embodiment of the stethoscope sound measurement unit 73 using the stethoscope 44 installed on the rear part of the smart remote control 400.
  • Auscultation sound measurement unit 73 is a transducer (transducer, 171) that converts the minute physical sound wave vibration from the stethoscope input unit 170 into an electrical signal, and obtains a PCG signal amplified by the electrical signal obtained from the transducer 171 a signal amplifying unit 172 for; a noise removal filter 173 for removing noise and power noise included in the PCG signal; at least one or more frequency filters 174 for extracting PCG signals for each frequency band; and an AD converter 176 for converting the extracted PCG signal into a digital signal.
  • transducer transducer
  • the biosignal collection unit 420 determines whether the PCG signal digitized by the AD converter 176 is user authentication completed and whether it is a valid PCG signal, and the smart remote control control unit 53 is a valid PCG signal authenticated by the user. is stored in the medical data storage unit 15 and transmitted to the medical data receiving unit 12 through the wireless communication connection means 37 .
  • the frequency filter 174 may use a filter having a frequency band suitable for the atrium, ventricle, lung, and mixed mode (band including all of the atrium, ventricle, and lung).
  • Another aspect of the frequency filter 174 is to use a filter having a frequency band of 30 Hz to 100 Hz in a bell mode for detecting a heart sound, and a diaphragm for detecting a lung sound ( diaphragm) mode, a frequency filter 174 having a frequency band of 100 Hz to 500 Hz may be used.
  • the stethoscope input unit 170 is composed of a thin-film sounding plate 44a, and collects sound from changes in internal air pressure caused by the sounding of the thin-film sounding plate 44a, and the transducer 171 is a microscopic sound generated from the stethoscope input unit 170. It is for converting the displacement of the thin film sounding plate 44a due to sound wave vibration into an electric signal, and may be a piezo film or a microphone.
  • the surface of the thin-film sounding plate 44a can serve as an ECG sensor.
  • the ECG signal can be simultaneously obtained while the stethoscope signal is obtained from the skin surface (eg, the chest area).
  • the simultaneous measurement of the PCG signal and the ECG signal using the stethoscope 44 is performed when the patient holds the smart remote control 400 and touches the skin of the chest region so that fingerprint authentication is performed, the thumb 800a naturally moves the four hand electrodes
  • a first electrode is formed by making contact with any one of the electrodes 24a, 25a, 26a, and 27a.
  • a second electrode is formed and these two electrodes (the first electrode) are formed.
  • ECG electrocardiogram
  • 11A to 11D show an embodiment for measuring blood pressure, cholesterol, and blood sugar by using a bio-signal obtained from a bio-sensor installed on the smart remote controller 400.
  • Blood pressure for measuring blood pressure by the artificial intelligence neural network 16 This is an embodiment in which a measuring unit, a cholesterol measuring unit measuring cholesterol, and a blood glucose measuring unit measuring blood sugar are implemented.
  • the blood pressure measuring unit (not shown) predicts the blood pressure of the person to be measured by the artificial intelligence neural network 20a for blood pressure measurement learned using bio-signals and personal body information 69f labeled according to various blood pressure values. It is preferred
  • the biosignal of the present invention may include ECG signal 69a, PPG signal 68c, PCG signal 69b, oxygen saturation 69c, PTT 69d, and glucose light reflection signal information 69e.
  • the cholesterol measurement unit (not shown) measures the cholesterol of the person to be measured by the artificial intelligence neural network 20b for cholesterol measurement learned using biosignals and personal body information 69f labeled according to various cholesterol levels. Prediction is preferred.
  • TG triglyceride
  • HDL-C high-density lipoprotein-cholesterol
  • LDL-C low-density lipoprotein-L cholesterol
  • the blood sugar measurement unit (not shown) predicts the blood sugar of the person to be measured by the artificial intelligence neural network 20c for blood sugar measurement learned using bio signals and personal body information labeled according to various blood sugar levels. Preferred.
  • the artificial intelligence neural network 16 of the present invention may include an artificial intelligence neural network 20a for blood pressure measurement, an artificial intelligence neural network 20b for cholesterol measurement, and an artificial intelligence neural network 20c for blood sugar measurement.
  • the artificial intelligence neural network 16 synchronizes the PPG signal 68c, the PCG signal 69b, the oxygen saturation 69c, the PTT 69d, and the glucose light reflection signal information based on the two R points of the ECG signal 69a. It is preferred to use (69e), that is, PPG signal (68c), PCG signal (69b), oxygen saturation (69c), PTT (69d), glucose light reflection signal information ( 69e) is preferably used as the input signal of the artificial intelligence neural networks 20a, 20b, and 20c.
  • the synchronized bio-signals are bio-signals measured from the user's body at the same time point, it is more accurate than using unsynchronized bio-signals (biometric signals acquired at different time points) as the input signal of the artificial intelligence neural network 16 . Blood pressure measurement, cholesterol measurement, and blood sugar measurement are possible. Since the patient's bio-signals change frequently according to the patient's body condition and time, accurate diagnosis results can be obtained only by using the bio-signals collected at the same time as the input signals of the artificial intelligence neural network 16 .
  • Another aspect of the artificial intelligence neural network 16 is to use the PCG signal 69b, oxygen saturation 69c, and glucose light reflection signal information 69e synchronized based on the peak point of the PPG signal 68c. do.
  • ECG signal 69a, PPG signal 68C, oxygen saturation level 69c, PCG signal 69b, and glucose light reflection signal information 69e are measured from the user, and such biosignal information and personal body information 69f It is configured to estimate the user's blood pressure in real time by applying the previously learned artificial intelligence neural network 20a for blood pressure measurement.
  • 11B to 11D show feature vectors extracted from ECG signal 69a, PPG signal 68c, PCG signal 69b, SpO2 69c, PTT 69d, glucose light reflection information 69e, and personal body information.
  • the artificial intelligence neural network 16 that calculates blood pressure, cholesterol, and blood sugar by the classifier 21a using 69f.
  • the feature vector extraction unit 1 extracts a feature vector from the ECG signal 69a
  • the feature vector extraction unit 2 extracts the feature vector from the PPG signal (68c)
  • the feature vector extraction unit 3 extracts the feature vector from the PCG signal (69b).
  • the feature vector extractor 3 (33c) is configured to extract the bell mode signal from the PCG signal 69b separated into a bell mode signal corresponding to a heart sound and a diaphram mode signal corresponding to a lung sound. It is possible to extract the feature vector and the feature vector of the diaphragm mode signal.
  • the feature vectors of the ECG signal 69a, the PPG signal 68c, and the PCG signal 69b may be extracted by a deep learning neural network or spectrum analysis in the frequency domain.
  • the deep learning neural network may be a 1-D convolutional neural network (CNN).
  • CNN convolutional neural network
  • LPC Linear Prediction Codes
  • MFCC Mel-Coefficient
  • PFP Mel-Frequency Cepstral Coefficient
  • PPP Preceptual Linear Prediction
  • BFCC Bark Frequency Cepstral Coefficients
  • the voice recognition terminal 17 provides bio-signals and personal body information 699f to the server 13 through the digital communication module 8, and thereafter, blood pressure, cholesterol, Receive feedback on diagnostic results for blood sugar.
  • the feature vectors extracted by the feature vector extraction units 33a, 33b, and 33c are SpO2 (69c), PTT (69d), glucose light reflection information 69e, and personal body
  • the information 69f is input together into the classifier 21a to calculate blood pressure, cholesterol, and blood sugar.
  • reference numeral 21a may be used instead of LSTM.
  • blood pressure, cholesterol, and blood sugar may be calculated by analyzing time series information of the feature vectors extracted by the feature vector extraction units 33a, 33b, and 33c.
  • the artificial intelligence neural network 16 illustrated in FIG. 11c it is an embodiment used by connecting a long short term memory (LSTM) or a gated recurrent unit (GRU) 21b to the output of the classifier of FIG. 11b,
  • LSTM long short term memory
  • GRU gated recurrent unit
  • the LSTM or GRU 21b analyzes the time sequence information output from the classifier 21a to calculate blood pressure, cholesterol, and blood sugar.
  • the blood glucose measurement unit may calculate HbA1c (glycated hemoglobin) for evaluating the average value of blood glucose for 2 to 3 months.
  • HbA1c glycated hemoglobin
  • the PPT (69d) may be defined as a time difference between an ECG peak (point R) and a peak point of the PPG signal, or a time difference between an ECG peak and a peak of Second Derivative Photoplethysmographic (SDPPG).
  • SDPPG Second Derivative Photoplethysmographic
  • the ventricles begin to push blood into the artery as the ventricles contract, so the starting point of the PPT measurement is used as the R peak, and the peak of the PPG signal to the peak of the SDPPG are used as the end point of the PTT measurement.
  • the personal body information 69f of the present invention is information that selects at least one or more of body heat, sex, age, height, weight, standard blood pressure, body fat information, and heart rate variability (HRV). do it with
  • the body fat information is the latest user's body fat information obtained by the body fat measuring unit 70 of the smart remote control 400 .
  • the glucose light reflection information 69e is preferably the recently measured glucose light reflection information of the user.
  • the thumb 800a of one hand of the user is in contact with the hand electrode (any one of 24a, 25a, 26a, 27a) formed on the side of the smart remote control 400 , another part of the user's body (eg, the chest area) ) in contact with the ECG sensor 48, the user's ECG signal 69a and PCG signal 69b can be simultaneously measured at the same time as fingerprint authentication.
  • the PPG signal 68c, the oxygen saturation level 69c, and the PTT 69d can be measured from the remaining fingers 800b, 800c, and 800d by the reflective blood vessel optical sensor array modules 35 and 36 .
  • the standard blood pressure value of the present invention is the blood pressure of a normal person known statistically and can be calculated as in Equation 3 below.
  • coefficients ⁇ and ⁇ are parameters that can be set by statistical information.
  • Figure 11d is another embodiment of the artificial intelligence neural network 16 for calculating blood pressure, cholesterol, blood sugar, Linear Prediction Codes (LPC), Mel-Coefficient, Mel-Frequency Cepstral Coefficient (MFCC), Cepstral Coefficient, Preceptual Linear A spectrogram calculated by Prediction (PLP), Short Time Fourier Transform (STFT), or Bark Frequency Cepstral Coefficients (BFCC) may be used as an input of the CNN 57 .
  • LPC Linear Prediction Codes
  • MFCC Mel-Frequency Cepstral Coefficient
  • PFP Preceptual Linear A spectrogram calculated by Prediction
  • STFT Short Time Fourier Transform
  • BFCC Bark Frequency Cepstral Coefficients
  • the feature vector extraction unit 4 (33d) outputs a spectrogram of the ECG signal from the ECG signal (69a),
  • the feature vector extraction unit 5 (33e) outputs a spectrogram of the PPG signal from the PPG signal 68c, and the feature vector extraction unit 6 (33f) outputs a spectrogram of the PCG signal from the PCG signal 69b.
  • the feature vector extractor 6 (33f) is configured to extract the bell mode signal from the PCG signal 69b separated into a bell mode signal corresponding to a heart sound and a diaphram mode signal corresponding to a lung sound.
  • the spectrogram and the spectrogram of the diaphragm mode signal can be extracted.
  • CNN Convolutional Neural Network, 57
  • feature vectors are output.
  • the CNN 57 may apply a separate independent CNN to each of the ECG signal 69a, the PPG signal 68c, and the PCG signal 69b.
  • These feature vectors are input to the classifier 21a together with SpO2 (69c), PTT (69d), glucose light reflection information 69e and personal body information 69f to calculate blood pressure, cholesterol, and blood sugar.
  • the CNN 57 and the classifier 21a may be modified and replaced with LSTMs.
  • the time series information of the spectrogram extracted by the feature vector extraction units 33d, 33e, and 33f may be used as input information of the LSTM to calculate blood pressure, cholesterol, and blood sugar.
  • the classifier 21a of the present invention may be a fully connected network (FCN) or a support vector machine (SVM).
  • FCN fully connected network
  • SVM support vector machine
  • FIG. 12 shows not only self-examination of the fetus' health status by using the stethoscope 44 on the smart remote control 400 according to the embodiment of FIG. It is a diagram schematically showing an embodiment to be performed.
  • FIG. 12 shows the fetus of the mother 86 remotely by using the stethoscope 44 on the smart remote control 400 , the doctor 201 using the smart remote control 400 and the digital TV 300 connected via Bluetooth.
  • remote medical diagnosis is performed to check the health status of a person online.
  • the digital TV 300 may be replaced with a smart mirror.
  • the voice recognition terminal 17 Integrated and embedded within the digital TV 300 , the voice recognition terminal 17 according to FIG. 1B provides an Internet 202 and Wi-Fi communication connection that allows remote medical diagnosis with a physician 201 or with a stethoscope 44 .
  • a digital communication module 8 that provides a short-range wireless communication connection, a medical data receiver 12 that receives medical data measured from the mother by a stethoscope 44, and a remote medical diagnosis between the doctor 201 and the mother 86 It may include a control unit 30 for controlling the digital communication module 8, the speaker 28, and the screen display unit 10 for execution.
  • the identity authentication of the mother 86 is made by the fingerprint recognition unit 24, 25, 26, 27 or the vein sensor unit 54a on the smart remote control 400.
  • the digital TV 300 receives the medical data measured by the smart remote control 400 and retransmits it to the server 13 through the Internet network, and the artificial intelligence neural network 16 or the expert system on the server 13 ( 18), the analysis result may be fed back, and the risk level of the patient's disease may be notified through the digital TV 300 or the touch screen 401 .
  • the digital TV 300 may share the screen between the mother 86 and the doctor 201 by providing visual information helpful for the remote medical diagnosis to the screen display unit 10 during the remote medical diagnosis.
  • the digital TV 300 may provide the mother through the speaker 28 with a guideline for health care, a telemedicine diagnosis method, and a guideline for using the stethoscope.
  • the deep learning learned artificial intelligence neural network 16 analyzes the medical data of the stethoscope 44 received by the medical data receiving unit 12 to automatically determine the presence or absence of disease and the risk of disease in the mother and fetus. have.
  • control unit 30 may drive the digital communication module 8 so that the information displayed on the screen display unit 10 and the information that the doctor sees on the monitor 206 are identical to each other.
  • the doctor shares real-time information with the mother 86 through the monitor 206 so that the doctor directly instructs the mother 86 how to use the stethoscope 44 through the Internet 202, which is the communication network. Therefore, it is possible to assist the mother in fetal health diagnosis as if the doctor 201 is next to the mother 86.
  • Reference numeral 60 denotes that the doctor can remotely hear the fetal heart pulse sound heard from the stethoscope 44 on the smart remote control 400 through headphones.
  • the voice recognition terminal receives the medical data for the bio-signal measured by the bio-sensor of the smart remote control and retransmits it to the server through the Internet network, and then the result analyzed by the artificial intelligence neural network or expert system on the server It is possible to notify the patient of the risk of the patient's disease by receiving feedback.
  • the voice recognition terminal can be integrated into the inside of the digital TV or the inside of the smart mirror.
  • Another aspect of the artificial intelligent smart remote control device of the present invention is that the voice recognition terminal can be integrated into the smart remote control.
  • the smart remote control device includes a biosensor that can check the vital signs of body heat, blood pressure, blood sugar, electrocardiogram, heart pulse, oxygen saturation, cholesterol, and body fat, which are major indicators of health management, from time to time in daily life.
  • a smart remote controller that is integrated and can perform biometric signal measurement and user authentication for these health care items at the same time; And the function of the digital TV (or smart mirror) is controlled by voice commands, and the medical data for the bio-signals measured by the bio-sensor of the smart remote control are received and analyzed with artificial intelligence, as well as abnormal signs are detected.
  • a voice recognition terminal that provides a remote medical diagnosis service through an Internet communication connection with a doctor; includes
  • the voice recognition terminal receives medical data measured by the smart remote control, retransmits it to a server through the Internet network, and then feeds back the analysis result of the medical data by an artificial intelligence neural network or expert system on the server can receive
  • the smart remote control can not only control digital TV functions by button input, but also use the biosensor installed on the smart remote control to operate the smart remote control while watching TV.
  • Medical data related to blood sugar, heart pulse, electrocardiogram, oxygen saturation, cholesterol, and body fat may be measured and collected at any time, and the collected data may be transmitted to the medical data receiver of the voice recognition terminal.
  • the patient can share information with the doctor through the screen of the digital TV during the remote medical diagnosis service.
  • a voice recognition terminal deep learning learning in advance by the medical data receiving unit and the medical data for learning to receive the medical data measured from a plurality of medical devices by a short-distance wireless communication connection with the medical devices.
  • the voice recognition terminal of the present invention may be integrated into a digital TV or a smart mirror.
  • the voice recognition terminal of the present invention may be integrated into the smart remote control.
  • the smart remote control receives medical data measured by the biosensor, retransmits it to a server through the Internet network, and then feeds back the analysis result of the medical data by an artificial intelligence neural network or expert system on the server. It can be received and presented to the user through the touch screen.
  • the biosensor of the present invention includes an infrared temperature sensor, a camera, an ECG sensor for measuring an electrocardiogram, a SpO2 sensor for measuring oxygen saturation, a PPG sensor for obtaining a photoplethysmography (PPG) signal, and a glucose light reflection signal. It is characterized in that it is equipped with at least one selected from a blood glucose sensor, a stethoscope for obtaining a heart sound signal, and hand electrodes for measuring body fat.
  • the biosensor may further include a fingerprint recognition unit or a vein sensor unit for user authentication, and in this case, measurement of health care items and user authentication may be performed at the same time.
  • the biosensor includes at least one measuring unit selected from an electrocardiogram measuring unit, a heart disease measuring unit, an oxygen saturation and PPG signal measuring unit, a body fat measuring unit, a blood glucose measuring unit, a blood pressure measuring unit, a cholesterol measuring unit, and a auscultation sound measuring unit. characterized by including.
  • the PPG sensor is preferably implemented using an SpO2 sensor.
  • One of the important roles of blood is to provide oxygen to each part of the body.
  • hemoglobin in red blood cells combines with oxygen to supply oxygen to each part of the body, and the amount of oxygen supplied in the blood is measured as Saturation of partial pressure oxygen (SpO2).
  • oxygen saturation is defined as the ratio of the concentration of hemoglobin containing oxygen to the concentration of total hemoglobin. The ratio is used to measure blood oxygen saturation without directly taking blood.
  • hemoglobin bound to oxygen (HbO2) absorbs infrared light (eg, 940 nm) well, and reduced hemoglobin (Hb) not bound to oxygen better absorbs red light (eg, 660 nm).
  • HbO2 hemoglobin bound to oxygen
  • red light eg, 660 nm
  • the amount of hemoglobin bound to oxygen increases relatively, and most of the infrared rays emitted by the infrared light emitting device are absorbed while passing through the blood vessels, so that the infrared rays are not detected by the photodetector.
  • the reflective blood vessel photosensor array module of the present invention includes: an infrared LED irradiating infrared (hereinafter referred to as 'IR light') including light in an infrared wavelength region (eg, 900 nm to 950 nm) to a finger region; a red light LED that irradiates red light (hereinafter referred to as 'RED light') including light in a red wavelength region (eg, 640 nm to 670 nm) to a finger region; and a photodetector for receiving the IR light and the RED light reflected from the finger, wherein the oxygen saturation is calculated based on the ratio of the components between the received IR light and the RED light.
  • 'IR light' infrared LED irradiating infrared
  • 'RED light' red light LED that irradiates red light
  • a photodetector for receiving the IR light and the RED light reflected from the finger, wherein the oxygen saturation
  • an image sensor In the present invention, it is preferable to use an image sensor, a photo diode or a photo transistor as the photo detector.
  • Blood oxygen saturation is observed between 98 and 99%, which is close to 100%, in healthy people.
  • the photoplethysmography (PPG) signal irradiates the red light generated from the light emitting element of the SpO2 sensor installed in the smart remote control to the human body, and then reflects the change in the amount of light received by the human body.
  • a photoplethysmogram signal can be measured by observation with a photodetector.
  • the photoplethysmographic pulse wave signal is a pulse wave signal measured in peripheral blood vessels when ejected blood is delivered to the peripheral blood vessels during ventricular systole. It can be measured by collecting by a detector.
  • the PPG signal is a biosignal that converts a change in blood flow, which is a change in the volume of peripheral blood vessels, into a change in light intensity at a location where peripheral blood vessels are distributed, such as the arteries of the fingertips, according to the heartbeat of a person.
  • ECG electrocardiogram
  • PTT Pulse Transit Time
  • PWV Pulse Wave Velocity
  • body fat measurement of the present invention it is preferable to measure body fat by a known technique BIA (Bioelectrical Impedance Analysis) method.
  • BIA Bioelectrical Impedance Analysis
  • the BIA method sees the human body as a kind of resistance, as the electrical resistance varies depending on the amount of body water, flows a weak alternating current (about 400 microampere) to the measurement part of the human body, detects the potential difference at this time, and calculates the resistance value.
  • biometric information such as body fat mass, muscle mass, and body water content is calculated by a predetermined algorithm along with the detected resistance value and variables such as height, weight, age, and sex of the subject.
  • the body fat measuring unit is disposed on the smart remote control and in direct contact with the target's finger to apply current to the target's body
  • the driving electrode is disposed on the smart remote control to directly contact the target's body a detection electrode for obtaining a body fat measurement signal by not only being touched but also measuring the electrical resistance value of the body of the person to be measured
  • AD converter Analog to Digital converter
  • electrocardiogram ECG signal, Electrocardiogram
  • ECG sensor installed on a smart remote control.
  • the heart is a muscle organ that acts as a pump to circulate blood, and its muscle movements are regulated by micro electricity, and ECG signals showing minute electrical fluctuations from myocardial cells that make up the heart are generated according to the contraction and relaxation of the heart.
  • the signal can be read by an ECG sensor that detects the skin surface.
  • the touch screen of the present invention is an input device capable of inputting a user's command by recognizing a contact position of a finger.
  • the touch screen of the present invention is characterized in that any one selected from a capacitive type, a resistive type, an ultrasonic type, and an infrared type (Infrared Beam) is used.
  • a resistive touch screen two upper and lower surfaces coated with a transparent conductive material having a resistive component on a transparent film face each other, but spacing dots are installed at regular intervals so as not to touch each other.
  • the two surfaces come into contact with each other, and the resistance value changes depending on the touch position.
  • the touch screen of the present invention may be integrated with the ECG sensor and converted into a touch-type ECG sensor.
  • the ECG signal transmitted from the heart to the fingertip can be detected.
  • the smart remote control user since the smart remote control user frequently touches the touch screen with the opposite finger (for example, index finger) while holding the hand electrode of the smart remote control with one hand to select the function of the TV, the user's ECG signal can be easily accessed in daily life. can be collected.
  • the stethoscope is installed on the smart remote control to obtain a PCG (Phono CardioGram) signal, which is a signal from the inside of the body, such as the heart and lungs.
  • PCG Pano CardioGram
  • the heart disease measuring unit of the present invention uses the labeled ECG signal, PPG signal, PCG signal, oxygen saturation, PTT, and personal body information according to the type and grade of heart disease to measure the heart of the subject by learning the artificial intelligence neural network. Predicting the disease is preferred.
  • the heart disease includes heart failure, arrhythmia, myocardial infarction, and angina pectoris.
  • the blood pressure measurement of the present invention predicts the blood pressure level of the subject by the artificial intelligence neural network learned using labeled ECG signal, PPG signal, PCG signal, oxygen saturation, PTT and personal body information according to various blood pressure values. It is preferred to do
  • Blood pressure is the pressure the blood pumped from the heart exerts on the walls of the arteries.
  • the ventricles of the heart contract, the amount of blood flowing increases and the pressure increases.
  • the ventricles dilate the amount of blood flowing decreases and the pressure decreases.
  • blood pressure measurement is indicated by the magnitude of the systolic pressure and the diastolic pressure.
  • the systolic pressure refers to the maximum blood pressure when the ventricles contract, and the diastolic pressure refers to the minimum blood pressure when the ventricles expand.
  • the cholesterol measurement of the present invention it is preferred to predict the cholesterol level of the subject by the artificial intelligence neural network learned using ECG signal, PPG signal, PCG signal, oxygen saturation, PTT and personal body information labeled by various cholesterol levels. do.
  • TG triglyceride
  • HDL-C high-density lipoprotein-cholesterol
  • LDL-C low-density lipoprotein-L cholesterol
  • a glucose light reflection signal is obtained from a finger part of a user touched on a touch screen by a plurality of blood glucose sensors installed on a smart remote control, and a glucose light reflection signal marked by various grades of blood glucose level and It is preferred to predict the blood sugar level of the subject using an artificial intelligence neural network learned from personal body information.
  • the absorbance of near-infrared rays is different depending on the blood glucose concentration and the wavelength of the near-infrared rays
  • a plurality of blood glucose sensors that output different near-infrared wavelengths are used simultaneously and the near-infrared signals reflected from the finger (glucose light reflection signal) information) is applied to the artificial intelligence neural network to measure the user's blood sugar.
  • the user identity authentication of the present invention is characterized in that it is performed using any one or more recognition methods selected from fingerprint authentication, vein authentication, and face recognition.
  • the fingerprint authentication is preferably performed by recognizing the fingerprint of the user's thumb holding the smart remote control.
  • the vein authentication is authentication by recognizing and authenticating the vein pattern of the user's fingers holding the smart remote control. It is preferred to do
  • the voice recognition terminal of the present invention observes the change in medical data according to the periodic examination by the biosensor and medical device to inform the patient of the degree of risk, inform the patient of items requiring intensive care examination, or inform the patient of the next examination schedule may include a health tracking management unit.
  • the bio-signal collection unit of the present invention is installed on the smart remote control, and among the bio-signals collected by the bio-sensor on the smart remote control while the subject uses the smart remote control in daily life, valid bio-signals (ECG signal, PPG signal) , oxygen saturation, glucose light reflection signal information, PCG signal, body fat, body temperature) are collected and stored in the medical data storage unit. These collected medical data are used to analyze and manage the health of the subject by the artificial intelligence neural network, expert system, and health tracking management unit.
  • the smart remote control controller which controls each part of the smart remote control (touch screen, biosensor, biosignal collection unit, medical data storage unit, wireless communication connection means) , it may be provided with a bio-signal measurement request means for inducing the user to self-exam through the health management mode of the touch screen by additionally registering it in the intensive care item.
  • the intensive care item may be flickered to request the user to measure the biosignal.
  • the touch screen screen may be forcibly switched to a health care item, and then the blood sugar item may be blinked to request the user to check the blood sugar.
  • the biosignal measurement requesting means for postprandial blood glucose check, when glucose light reflection signal information required for a periodic blood glucose test is not secured, the biosignal measurement requesting means may be used to determine the user's recent meal time (or recent postprandial blood glucose). In consideration of the check time), a text or voice message requesting to check blood glucose after a meal may be delivered to the user around the time of the meal.
  • biosignal measurement request means includes a voice recognition unit and a voice reproducing unit, so that the user can interactively request the biosignal measurement or guide the measurement method.
  • the biosignal measurement request means is: “Master, it is time to check blood glucose after a meal”. User: “Tell me back in 10 minutes”. Smart Remote: “Okay. master! See you in ten minutes.” As such, it is possible to interactively request the user to measure biosignals.
  • the smart remote control control unit of the present invention displays the medical data collected by the medical data storage unit on the touch screen screen or displays additional information (eg, intensive care items and the next examination schedule) provided from the health tracking management unit of the voice recognition terminal on the touch screen. It can be displayed on the screen.
  • additional information eg, intensive care items and the next examination schedule
  • bio-signal information measured or obtained by the smart remote control ECG signal, heart pulse, PPG signal, oxygen saturation, PCG signal, body fat, body temperature, blood pressure, blood sugar, cholesterol information, personal body information, etc.
  • additional information (intensive care) item and next inspection schedule) can be displayed on the touch screen of the smart remote control.
  • the cardiac pulse measurement of the present invention can be calculated as the number of beats per minute (beats/min) using the peak (point R) of the ECG signal.
  • HRV Heart Rate Variability
  • the artificial intelligence neural network of the present invention may be a service application built on a server.
  • the voice recognition terminal transmits medical data measured by a smart remote control or medical device to a server through the Internet network.
  • the artificial intelligence neural network on the server provides a service that feeds back the analysis result of the medical data to the voice terminal.
  • Another aspect of the artificial intelligence neural network of the present invention is to transmit medical data measured by the biosensor on the smart remote control to a server through the Internet network, and then analyze the medical data obtained by the artificial intelligence neural network on the server. The result can be provided to the user through the touch screen by receiving feedback from the smart remote control.
  • Another aspect of the artificial intelligence neural network of the present invention is to transmit medical data measured by the biosensor on the smart remote control to the mobile phone, and then to the mobile phone obtained by the artificial intelligence neural network app installed as application software on the mobile phone. An analysis result of the medical data may be provided to the user.
  • control unit of the smart remote control and all electrical circuits wake up from sleep mode and operate normally by the motion signal or interrupt signal of the smart remote control, and after obtaining valid biometric information of the registered user, It is preferred to return to sleep mode again to conserve battery power.
  • the smart remote control of the present invention may include a gyro sensor, an acceleration sensor, or a tilt sensor inside the smart remote control to detect a motion (shake) signal of the smart remote control.
  • Another aspect of the smart remote control of the present invention may be modified so that the bio-sensor of the smart remote control is installed on the smart phone to collect the bio-signals of the smart phone user in daily life.
  • the self-testing method using the smart remote control 400 may be performed by the above-described voice recognition terminal 17 and the digital TV 300 having the voice recognition terminal 17 embedded therein. Therefore, even if omitted below, the description of the voice recognition terminal 17 and the digital TV 300 may be equally applied to the description of the self-test method using the smart remote control.
  • a smart remote control device not shown.
  • step S101 the user's bio-signals may be collected from the bio-sensors included in the smart remote control 400 .
  • step S102 a valid biosignal for which user authentication has been completed may be determined from among the collected biosignals by using the biosignal validity determining means.
  • step S103 the medical data received through the short-range wireless communication connection with the medical data measured by the medical devices and the medical data obtained through the biosignal collecting unit may be stored.
  • step S104 the medical data stored through the wireless communication connection means may be transmitted to the server through the Internet network, and the analysis result of the medical data may be fed back by the artificial intelligence neural network and the expert system on the server.
  • step S105 the analysis result of the medical data may be provided to the user through the touch screen.
  • steps S101 to S108 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present application.
  • some steps may be omitted if necessary, and the order between the steps may be changed.
  • the self-test method using the smart remote control 400 may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium.
  • the computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination.
  • the program instructions recorded on the medium may be specially designed and configured for the present invention, or may be known and available to those skilled in the art of computer software.
  • Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks.
  • - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like.
  • Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.
  • the hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.
  • the digital TV of the present invention can be replaced with a smart mirror or computer monitor.
  • the self-test method using the above-described smart remote control 400 may be implemented in the form of a computer program or application executed by a computer stored in a recording medium.
  • 420a means for determining the validity of a biosignal

Abstract

The present invention relates to a smart remote control device which not only enables user authentication without a separate user authentication process, but also measures a user's bio-signals, by means of bio-sensors installed on a remote control, while the remote control is being used in daily life, stores and manages the bio-signals, and automatically analyzes the bio-signals by means of artificial intelligence to identify risk factors to the user's health. In particular, the present invention provides an artificial intelligence smart remote control device and a self-test method using same, wherein the artificial intelligence smart remote control device can measure a user's health status, including body heat, blood pressure, blood sugar, heart beat, oxygen saturation, cholesterol, and body fat, which are key indicators for health care, by means of bio-sensors installed on the remote control, while the remote control is used, and perform user authentication at the same time, and thus can prevent diseases of a user in daily life.

Description

인공지능형 스마트 리모컨 장치 및 이를 이용한 자가 검사 방법Artificial intelligence smart remote control device and self-test method using the same
본원은, 생활 속에서 리모컨 사용 중에 리모컨 상에 설치된 바이오 센서에 의해 사용자의 건강 상태를 측정하여 보관 및 관리하고, 이를 인공지능에 의해 자동 분석하여 사용자의 건강의 위험요소를 파악할 수 있는 스마트 리모컨 장치에 대한 것으로, 특히, 건강관리의 주요 지표인 체열, 혈압, 혈당, 심전도, 산소포화도, 콜레스테롤, 체지방 등을 체크하기 위해 리모컨 사용 중에 리모컨 상에 설치된 바이오 센서에 의해 사용자의 건강 상태 측정 및 신분 인증을 동시에 수행할 수 있어, 생활 속에서 사용자의 질병을 사전에 예방할 수 있는 인공지능형 스마트 리모컨 장치와 이를 이용한 자가 검사 방법을 제공한다.Smart remote control device that measures, stores and manages a user's health condition by means of a biosensor installed on the remote control while using the remote control in everyday life, and automatically analyzes it by artificial intelligence to identify risk factors for the user's health In particular, in order to check the main indicators of health management, such as body heat, blood pressure, blood sugar, electrocardiogram, oxygen saturation, cholesterol, body fat, etc., the user's health status is measured and identification is authenticated by the biosensor installed on the remote control while using the remote control. It provides an artificial intelligence smart remote control device that can prevent a user's disease in advance and a self-test method using the same.
최근 첨단 의료장비 기술과 함께 병원 인프라 증대로 많은 사람들이 풍부한 의료혜택을 받으면서, 인류는 수명 연장과 함께 삶의 질 향상으로 과거보다 많은 행복을 도모하게 되었다.Recently, as many people receive abundant medical benefits due to the increase of hospital infrastructure along with cutting-edge medical equipment technology, mankind has been promoting greater happiness than in the past by extending life expectancy and improving the quality of life.
그러나 현대 사회는 식생활 문화의 향상 및 운동 부족 그리고 고령화 사회로의 급속한 진행에 따라, 정기적인 건강검진의 필요성은 날로 중요시되어 가고 있다. However, in modern society, with the improvement of dietary culture, lack of exercise, and rapid progress toward an aging society, the need for regular health checkups is becoming increasingly important.
또한, 건강검진을 하기 위해서는 별도의 시간을 내야하는 불편함이 따르기 때문에 대부분은 건강에 이상이 생긴 경우 뒤늦게 병원을 찾고 있다. In addition, since it is inconvenient to have to pay extra time for a health check-up, most of them are late to the hospital if there is an abnormality in their health.
따라서 우리가 수시로 생활 속에서 몸의 건강 상태를 파악할 수 있다면, 건강을 위협하는 많은 상황에 대해 미리 대처가 가능하다.Therefore, if we can check the health condition of our body from time to time in our daily life, we can deal with many situations that threaten our health in advance.
특히 체열, 혈압, 혈당, 심전도, 산소포화도, 콜레스테롤, 체지방은 우리 몸 상태를 잘 알려주는 생활 속 건강관리의 주요 지표로서, 이들을 수시로 관리하는 잘하는 경우 많은 질병을 사전에 예방할 수 있을 뿐만 아니라 보다 건강한 삶을 누릴 수 있을 것이다.In particular, body heat, blood pressure, blood sugar, electrocardiogram, oxygen saturation, cholesterol, and body fat are major indicators of health management in daily life that inform our body condition well. you will be able to enjoy life
요즈음, 병원에 가지 않고서도 자신의 체지방, 혈당과 혈압을 언제 어디서나 확인할 수 있는 의료기기들이 많이 보급되어, 환자들은 자신의 집이나 사무실에서 이를 사용하고 있다. 그러나 이러한 의료기기들이 각기 다른 장소에 흩어져 있어, 일반인이 생활 속 건강관리의 주요 지표를 꾸준히 챙기는데 에는 많은 번거로움이 따른다. 예컨대 환자는 각각의 의료기기의 보관 장소로부터 일일이 의료기기를 꺼내서 사용해야 하는 번거로움은 둘째치고 라도, 사용시마다 매번 각각의 의료기기별로 사용자 인증을 해야 하는 불편이 따른다.Nowadays, many medical devices are available that allow you to check your body fat, blood sugar and blood pressure anytime and anywhere without going to the hospital, and patients are using them at their home or office. However, since these medical devices are scattered in different places, it is very cumbersome for the general public to keep up with the main indicators of health management in their daily life. For example, aside from the inconvenience of having to take out each medical device from the storage place of each medical device and use it, the patient has the inconvenience of having to authenticate the user for each medical device each time it is used.
오늘날의 디지털 의료기기는 여러 사람 (예컨대 가족구성원)이 같이 사용하도록 허용할 뿐만 아니라, 각각의 환자로부터 측정된 데이터는 환자에 대한 지속적 추적 관찰을 위해 개인별로 저장 및 관리되고 있으며, 이를 위해 의료기기 사용 전에는 반드시 사용자 등록 및 인증 절차를 따라야 한다. 그러나 이 경우, 사용자는 사용시마다 매번 각각의 의료기기에 대해 사용자 인증을 수행해야 하는 불편이 따른다.Today's digital medical devices not only allow multiple people (eg, family members) to use together, but the data measured from each patient is stored and managed individually for continuous follow-up of the patient. Before use, you must follow the user registration and authentication procedures. However, in this case, the user has the inconvenience of having to perform user authentication for each medical device every time it is used.
또한, 기존의 디지털 의료기기들은 측정 항목마다 각기 다른 제품의 디지털 의료기기들을 사용하기 때문에, 동일한 시점에서 측정된 동기화된 생체 신호들(예컨대, 심전도, PPG신호, 청진음 신호)을 얻기가 힘들다. 이들 생체신호가 환자의 바디 컨디션과 시간에 따라 수시로 변한다는 점을 고려하면 각기 다른 시점에 측정된 생체신호들을 이용해 진단한다는 것은 신뢰도를 떨어뜨릴 뿐만 아니라 부정확한 결과를 초래한다.In addition, since the existing digital medical devices use digital medical devices of different products for each measurement item, it is difficult to obtain synchronized bio-signals (eg, electrocardiogram, PPG signal, auscultation sound signal) measured at the same time point. Considering that these biosignals change frequently depending on the patient's body condition and time, making a diagnosis using biosignals measured at different times not only lowers reliability but also leads to inaccurate results.
한편, 본 발명은 천정형 인공지능 건강 모니터링 장치 및 이를 이용한 원격 의료 진단 방법(출원번호: 10-2020-0055019), 자가 격리자 모니터링 장치 및 이를 이용한 방법(출원번호: 10-2020-0071282), 분산형 동선 추적 장치 및 이를 이용한 방법(출원번호:10-2020-0086171)과 인공지능형 무증상자 동선 추적 장치 및 이를 이용한 방법(출원번호:10-2021-0024156)의 연속이다.On the other hand, the present invention provides a ceiling-type artificial intelligence health monitoring device and a remote medical diagnosis method using the same (application number: 10-2020-0055019), a self-isolation monitoring device and a method using the same (application number: 10-2020-0071282), distributed It is a continuation of a type movement tracking device and a method using the same (Application No.: 10-2020-0086171) and an AI-type asymptomatic movement tracking device and a method using the same (Application No.: 10-2021-0024156).
본 발명은 전술한 종래 기술의 문제점을 해결하기 위해, 디지털 TV (또는 스마트 미러)의 기능을 제어하고, 생활 속에서 별도의 인증 절차 없이 사용자 인증이 이루어질 뿐만 아니라, 건강관리의 주요 지표인 체열, 혈압, 혈당, 심장맥박, 산소포화도, 콜레스테롤, 체지방을 수시로 간편하게 측정할 수 있는 바이오 센서를 구비한 스마트 리모컨, 및 이상징후가 발견된 경우 디지털 TV(또는 스마트 미러)와 연동하여 의사와의 인터넷 통신 연결을 통해 원격 의료 진단 서비스를 제공할 수 있는 인공지능형 스마트 리모컨 장치 및 이를 이용한 자가 검사방법을 제공하고자 한다.In order to solve the problems of the prior art described above, the present invention controls the function of a digital TV (or smart mirror) and performs user authentication without a separate authentication procedure in daily life, as well as body heat, which is a major indicator of health care, A smart remote control equipped with a biosensor that can easily measure blood pressure, blood sugar, heart rate, oxygen saturation, cholesterol and body fat from time to time, and when abnormal symptoms are found, it is linked with a digital TV (or smart mirror) to communicate with a doctor An object of the present invention is to provide an artificial intelligence smart remote control device that can provide remote medical diagnosis service through connection and a self-test method using the same.
또한, 본 발명은 동일한 시점에서 사용자의 몸으로부터 측정된 동기화된 생체신호를 인공지능 신경망의 입력신호로 사용하여 혈압 측정, 콜레스테롤 측정, 혈당 측정을 수행할 수 있는 인공지능형 스마트 리모컨 장치 및 이를 이용한 자가 검사방법을 제공하고자 한다.In addition, the present invention provides an artificial intelligent smart remote control device capable of performing blood pressure measurement, cholesterol measurement, and blood sugar measurement by using a synchronized bio-signal measured from the user's body at the same time as an input signal of an artificial intelligence neural network, and a person using the same. We want to provide an inspection method.
다만, 본원의 실시예가 이루고자 하는 기술적 과제는 상기된 바와 같은 기술적 과제들로 한정되지 않으며, 또 다른 기술적 과제들이 존재할 수 있다.However, the technical problems to be achieved by the embodiments of the present application are not limited to the technical problems as described above, and other technical problems may exist.
상기한 기술적 과제를 달성하기 위한 기술적 수단으로서, 본원의 일 실시예에 따른 인공지능형 스마트 리모컨 장치는, 디지털 TV의 기능을 제어하기 위한 버튼 및 생체 신호 측정과 상기 생체신호 측정 동안 사용자 인증을 동시에 수행하는 바이오 센서를 포함하는 스마트 리모컨; 및 상기 디지털 TV를 음성 명령에 의해 제어하고, 상기 바이오 센서에 의해 측정된 생체 신호에 대한 의료 데이터를 수신하여 인공지능 신경망을 통해 분석하고, 의사와의 인터넷 통신 연결을 통해 원격 의료 진단 서비스를 제공하는 음성인식 단말기를 포함할 수 있다. As a technical means for achieving the above technical problem, the artificial intelligent smart remote control device according to an embodiment of the present application performs a button for controlling a function of a digital TV and a bio-signal measurement and user authentication during the bio-signal measurement at the same time a smart remote control including a biosensor; and controlling the digital TV by voice commands, receiving medical data on the biosignal measured by the biosensor, analyzing it through an artificial intelligence neural network, and providing a remote medical diagnosis service through an Internet communication connection with a doctor It may include a voice recognition terminal.
또한, 상기 스마트 리모컨은, 상기 바이오 센서로부터 생체 신호를 수집하는 생체 신호 수집부; 및 상기 수집된 생체 신호 중 사용자 인증이 완료된 유효한 생체신호를 수집하기 위한 생체신호 유효 판별 수단을 포함하고, 상기 음성인식 단말기는, 근거리 무선 통신 연결에 의해 복수의 의료기기로부터 측정된 의료 데이터 내지 상기 스마트 리모컨의 바이오 센서에 의해 측정된 의료 데이터를 수신하는 의료 데이터 수신부를 포함할 수 있다. In addition, the smart remote control may include: a biosignal collecting unit for collecting biosignals from the biosensor; and biosignal validity determining means for collecting valid biosignals for which user authentication has been completed among the collected biosignals, wherein the voice recognition terminal includes medical data measured from a plurality of medical devices through a short-range wireless communication connection to the It may include a medical data receiving unit for receiving medical data measured by the biosensor of the smart remote control.
또한, 상기 음성인식 단말기는, 상기 의료 데이터를 인터넷망을 통해 서버에 전송하고, 상기 서버상의 인공지능 신경망 내지 전문가 시스템에 의해 분석된 상기 의료 데이터의 분석 결과를 피드백 받아 사용자에게 질병 유무와 질병의 위험도를 상기 디지털 TV를 통해 통지할 수 있다. In addition, the voice recognition terminal transmits the medical data to the server through the Internet network, and receives feedback on the analysis result of the medical data analyzed by the artificial intelligence neural network or expert system on the server to provide the user with information on whether or not there is a disease. The level of risk may be notified through the digital TV.
또한, 상기 인공지능 신경망은 상기 음성인식 단말기에 설치되는 인공지능 신경망 앱(app)일 수 있다. In addition, the artificial intelligence neural network may be an artificial intelligence neural network app installed in the voice recognition terminal.
본원의 일 실시예에 따르면, 인공지능형 스마트 리모컨 장치는, 생체 신호 측정과 상기 생체 신호 측정 동안 사용자 인증을 동시에 수행하는 바이오 센서, 상기 바이오 센서로부터 생체 신호를 수집하기 위한 생체 신호 수집부, 상기 생체 신호 수집부에 의해 수집된 생체 신호 중 사용자 인증이 완료된 생체 신호 성분 중 유효한 생체신호 부분만을 걸러내는 생체신호 유효 판별 수단, 의료기기들에 의해 측정된 의료 데이터를 근거리 무선 통신 연결에 의해 수신한 의료데이터 내지 상기 생체 신호 수집부를 통해 얻어진 의료데이터를 저장하기 위한 의료 데이터 저장부; 상기 의료 데이터를 인터넷망을 통해 서버에 전송하고, 이후 상기 서버상의 인공지능 신경망 내지 전문가 시스템에 의해 상기 의료 데이터의 분석한 결과를 피드백 받기 위한 무선 통신 연결 수단; 및 상기 바이오 센서, 생체 신호 수집부, 의료데이터 저장부, 무선 통신 연결 수단을 제어하고 상기 의료 데이터의 분석한 결과를 터치 스크린을 통해 사용자에게 제공하는 스마트 리모컨 제어부를 포함하는 스마트 리모컨을 포함할 수 있다. According to an embodiment of the present application, an artificial intelligent smart remote control device includes a biosensor for simultaneously measuring biosignals and user authentication during the biosignal measurement, a biosignal collecting unit for collecting biosignals from the biosensor, and the biometrics. A biosignal validity determination means for filtering out only a valid biosignal part among biosignal components for which user authentication has been completed among biosignals collected by the signal collection unit, and medical care that receives medical data measured by medical devices through a short-distance wireless communication connection a medical data storage unit for storing data or medical data obtained through the biosignal collecting unit; a wireless communication connection means for transmitting the medical data to a server through the Internet network, and then receiving a feedback result of analysis of the medical data by an artificial intelligence neural network or an expert system on the server; and a smart remote controller including a smart remote controller for controlling the biosensor, biosignal collecting unit, medical data storage unit, and wireless communication connection means and providing the result of analyzing the medical data to the user through a touch screen. have.
또한, 상기 터치스크린은, 리모컨 모드와 건강관리 모드 화면을 전환할 수 있는 메뉴버튼을 포함하고, 상기 리모컨 모드 동안에는 디지털 TV의 기능을 제어하기 위한 리모컨 선택 버튼들이 상기 터치 스크린 화면상에 표시되고, 상기 건강관리 모드 동안에는 건강관리 항목 중 하나를 선택하기 위한 메뉴 선택 버튼들이 상기 터치 스크린 화면상에 표시될 수 있다. In addition, the touch screen includes a menu button for switching between a remote control mode and a health care mode screen, and during the remote control mode, remote control selection buttons for controlling a function of a digital TV are displayed on the touch screen screen, During the health care mode, menu selection buttons for selecting one of health care items may be displayed on the touch screen screen.
또한, 상기 스마트 리모컨 제어부는, 특정 건강관리 항목에 대해 유효한 생체 신호가 미리 설정된 기준 값 이하로 수집되는 경우, 상기 특정 건강관리 항목의 생체신호 측정을 요청하는 문자나 음성 메시지를 사용자에게 발송하거나 상기 특정 건강관리 항목을 집중 케어 항목에 등록시켜 사용자에게 자가 검사하도록 유도하거나 대화형으로 사용자에게 생체 신호 측정을 요구하는 생체신호 측정 요구 수단을 포함할 수 있다. In addition, the smart remote control control unit, when a valid biosignal for a specific health care item is collected below a preset reference value, sends a text or voice message requesting measurement of a biosignal of the specific health care item to the user, or It may include a biosignal measurement requesting means for registering a specific health care item in the intensive care item to induce the user to self-examine or interactively request the user to measure the biosignal.
또한, 상기 스마트 리모컨의 좌우 측면에는 반사형 혈관 광센서 어레이 모듈을 포함하고, 상기 반사형 혈관 광센서 어레이 모듈은 복수개의 SpO2센서 또는 복수개의 정맥 센서부를 포함하고, 사용자가 스마트 리모컨의 측면을 손가락으로 잡았을 때, 광 검출기에 의해 손가락의 내부에 있는 정맥의 혈관 패턴에 대한 정맥 이미지를 얻어 정맥 인증에 사용하거나 손가락의 동맥으로부터 산소 포화도와 PPG신호를 측정할 수 있다. In addition, the left and right sides of the smart remote control include a reflective blood vessel optical sensor array module, and the reflective blood vessel optical sensor array module includes a plurality of SpO 2 sensors or a plurality of vein sensor units, and the user controls the side of the smart remote control. When held by a finger, a vein image of the blood vessel pattern of the vein inside the finger is obtained by a photodetector and used for vein authentication, or oxygen saturation and PPG signal can be measured from the artery of the finger.
또한, 상기 스마트 리모컨의 상하 좌우측면에 설치된 지문 인증부; 및 상기 지문 인증부 접촉시, 상기 지문 인증부와의 동시 접촉을 제공하는 상기 지문 인증부의 외주 테두리를 감싸는 복수의 손 전극을 포함하고, 상기 지문 인증부는 사용자가 스마트 리모컨을 손으로 잡았을 때 사용자의 엄지손가락의 지문에 의해 지문 인증을 수행하고, 상기 복수의 손 전극을 체지방 측정을 위한 구동 전극과 검출 전극으로서 동작하거나 ECG센서의 제1전극을 형성할 수 있다. In addition, the fingerprint authentication unit installed on the top, bottom, left and right sides of the smart remote control; and a plurality of hand electrodes surrounding an outer periphery of the fingerprint authentication unit providing simultaneous contact with the fingerprint authentication unit when the fingerprint authentication unit is in contact, wherein the fingerprint authentication unit is the user's hand when the user holds the smart remote control. Fingerprint authentication is performed by the fingerprint of the thumb, and the plurality of hand electrodes can be operated as a driving electrode and a detection electrode for body fat measurement, or a first electrode of an ECG sensor can be formed.
또한, 상기 ECG센서는 상기 스마트 리모컨의 후면부에 설치되고, 심전도 측정은 상기 제1전극과 제2전극 간에 형성된 전압 신호를 증폭하여 측정하되, 상기 제2전극은 상기 ECG 센서가 피부에 접촉되었을 때 형성될 수 있다. In addition, the ECG sensor is installed on the rear side of the smart remote control, and the ECG measurement is performed by amplifying the voltage signal formed between the first electrode and the second electrode, and the second electrode is when the ECG sensor is in contact with the skin. can be formed.
또한, 심전도 측정은 상기 제1전극과 제2전극 간에 형성된 전압 신호를 증폭하여 측정하되, 상기 제2전극은 상기 손의 반대편 손가락이 터치형 ECG센서에 접촉되었을 때 형성될 수 있다. In addition, the electrocardiogram is measured by amplifying the voltage signal formed between the first electrode and the second electrode, and the second electrode may be formed when the opposite finger of the hand is in contact with the touch-type ECG sensor.
또한, 상기 터치형 ECG 센서는, 디스플레이 패널; 상기 디스플레이 패널에 적층되는 하층 투명필름; 상기 하층 투명필름상에 적층되는 상층 투명필름; 상기 상층 투명 필름상에 적층되는 ECG 센서; 상기 상층 투명필름과 하층 투명 필름을 분리하는 스페이서; 상기 하층 투명필름의 윗면에 형성되어, 터치시 X축의 좌표를 제공하는 제1 저항막 패턴; 상기 상층 투명필름의 하단에 형성되며 상기 제1 저항막 패턴과는 서로 수직으로 교차하도록 배열하여, 터치시 Y축의 좌표를 제공하는 제2 저항막 패턴; 상기 제1 저항막 패턴의 말단에 설치되는 복수의 X축 전극; 상기 제2 저항막 패턴의 말단에 설치되는 복수의 Y축 전극; 및 상기 ECG 센서의 한쪽 단말에 설치되는 ECG전극,을 포함할 수 있다. In addition, the touch-type ECG sensor, a display panel; a lower transparent film laminated on the display panel; an upper transparent film laminated on the lower transparent film; an ECG sensor laminated on the upper transparent film; a spacer separating the upper transparent film and the lower transparent film; a first resistive film pattern formed on the upper surface of the lower transparent film and providing X-axis coordinates when touched; a second resistive film pattern formed at a lower end of the upper transparent film and arranged to cross perpendicularly to the first resistive film pattern to provide a Y-axis coordinate when touched; a plurality of X-axis electrodes installed at the ends of the first resistive layer pattern; a plurality of Y-axis electrodes installed at the ends of the second resistive layer pattern; and an ECG electrode installed at one terminal of the ECG sensor.
또한, 상기 스마트 리모컨은, 상기 스마트 리모컨의 좌우 측면에는 반사형 혈관 광센서 어레이 모듈을 포함하고, 상기 반사형 혈관 광센서 어레이 모듈은 복수개의 SpO2센서로 구성되어, 사용자가 상기 스마트 리모컨의 측면을 손가락으로 잡았을 때 지문 인증과 동시에, 광 검출기에 의해 손가락의 동맥으로부터 산소 포화도와 PPG신호를 측정할 수 있다. In addition, the smart remote control includes a reflective blood vessel optical sensor array module on the left and right sides of the smart remote control, and the reflective blood vessel optical sensor array module is composed of a plurality of SpO 2 sensors, so that the user can control the side of the smart remote control. Simultaneously with fingerprint authentication when held with a finger, oxygen saturation and PPG signal can be measured from the finger's artery by a photodetector.
또한, 상기 스마트 리모컨은 후면부에 설치되는 청진기를 포함하고, 사용자가 상기 스마트 리모컨의 측면을 손가락으로 잡아 상기 청진기를 심장 부위에 접촉시, 상기 청진기에 의해 심장 및 폐의 내부에서 나오는 PCG신호를 계측과 동시에 지문 인증이 이루어질 수 있다. In addition, the smart remote control includes a stethoscope installed on the rear side, and when the user holds the side of the smart remote control with a finger and touches the stethoscope to the heart region, PCG signals emitted from the inside of the heart and lungs are measured by the stethoscope At the same time, fingerprint authentication may be performed.
또한, 상기 청진기는 디지털 TV를 음성 명령으로 제어하기 위한 마이크 입력으로 사용되고, 상기 스마트 리모컨 제어부는, 상기 청진기 사용 기간 동안 디지털 TV의 볼륨이 줄어들도록 제어하고, 청진기 사용 종료시에는 사용 전의 디지털 TV의 볼륨 크기로 자동 복귀되도록 할 수 있다.In addition, the stethoscope is used as a microphone input for controlling the digital TV with a voice command, and the smart remote control controller controls the volume of the digital TV to decrease during the period of using the stethoscope, and when the use of the stethoscope ends, the volume of the digital TV before use It can be set to automatically return to the size.
또한, 상기 청진기의 박막 울림판 표면을 도전성 재료로 코팅하고, 상기 도전성 재료의 상측에 절연막을 코팅하여 ECG 전극면을 형성함으로써, 상기 청진기를 ECG 센서로 겸용할 수 있다. In addition, by coating the surface of the thin film sounding board of the stethoscope with a conductive material and coating an insulating film on the upper side of the conductive material to form an ECG electrode surface, the stethoscope can be used as an ECG sensor.
또한, 상기 바이오 센서는 적외선 온도 센서, 카메라, 심전도를 측정하기 위한 ECG센서, 산소 포화도를 측정하기 위한 SpO2센서, 광전용적맥파(PPG, Photoplethysmography) 신호를 얻기 위한 PPG센서, 글루코스 광반사 신호들을 얻기 위한 혈당 센서, PCG 신호를 얻기 위한 청진기, 체지방 측정용 손 전극들 중 선택된 어느 하나 이상을 구비할 수 있다. In addition, the biosensor includes an infrared temperature sensor, a camera, an ECG sensor for measuring an electrocardiogram, an SpO2 sensor for measuring oxygen saturation, a PPG sensor for obtaining a photoplethysmography (PPG) signal, and a glucose light reflection signal. It may include any one or more selected from a blood glucose sensor for a blood glucose sensor, a stethoscope for obtaining a PCG signal, and hand electrodes for measuring body fat.
또한, 상기 인공 지능 신경망은 상기 생체신호 및 개인 신체 정보를 입력으로 하여, 혈압 측정용 인공지능 신경망, 콜레스테롤 측정용 인공지능 신경망, 혈당 측정용 인공지능 신경망, 심장 질환 측정부 중 선택된 어느 하나 이상을 포함할 수 있다. In addition, the artificial intelligence neural network receives the bio-signals and personal body information as inputs, and selects any one or more selected from an artificial intelligence neural network for blood pressure measurement, an artificial intelligence neural network for cholesterol measurement, an artificial intelligence neural network for blood sugar measurement, and a heart disease measurement unit. may include
또한, 상기 개인 신체 정보는 체지방 정보 및 표준 혈압치를 포함하고, 상기 표준 혈압치는 [수학식 1]에 적용하여 계산되고, 상기 인공지능 신경망은 생체 신호의 특징 벡터를 추출하는 딥 런닝 신경 회로망 및 LSTM(long Short term Memory)를 포함하고, [수학식 1]은In addition, the personal body information includes body fat information and a standard blood pressure value, the standard blood pressure value is calculated by applying [Equation 1], and the artificial intelligence neural network is a deep running neural network and LSTM for extracting a feature vector of a biosignal. (long short term memory), and [Equation 1] is
Figure PCTKR2022003938-appb-I000001
일 수 있다.
Figure PCTKR2022003938-appb-I000001
can be
또한, 상기 인공 지능 신경망의 입력은 ECG 신호의 R지점을 기준으로 동기화된 생체 신호를 사용할 수 있다. In addition, the input of the artificial intelligence neural network may use a biological signal synchronized with respect to the R point of the ECG signal.
또한, 상기 생체신호는 ECG신호, PPG신호, PCG신호, SpO2, PTT, 글루코스 광 반사 정보를 포함할 수 있다. In addition, the biosignal may include ECG signal, PPG signal, PCG signal, SpO2, PTT, and glucose light reflection information.
또한, 상기 혈당 센서는, 터치 스크린상에 표시되는 가상의 손가락 패턴; 상기 가상의 손가락 패턴 상에 정렬된 사용자의 각각의 손가락 부위에 서로 다른 파장의 근 적외선을 조사하는 복수개의 근 적외선 발광소자; 및 상기 손가락 부위로부터 반사한 근 적외선을 수광하여 전기적 신호로 변환하는 광 센서를 포함하고, 상기 광 센서로부터 글루코스 광 반사 신호 정보를 획득할 수 있다. In addition, the blood glucose sensor may include a virtual finger pattern displayed on the touch screen; a plurality of near-infrared light emitting devices for irradiating near-infrared rays of different wavelengths to each finger part of the user aligned on the virtual finger pattern; and an optical sensor that receives the near-infrared light reflected from the finger and converts it into an electrical signal, and obtains glucose light reflection signal information from the optical sensor.
또한, 상기 음성 인식 단말기 내지 스마트 리모컨 제어부는, 상기 생체 신호 및 의료 데이터의 변화 추이를 관찰하여 사용자에게 위험도를 알려주거나 집중케어 검사가 필요한 항목을 알려주거나 다음 검사 일정을 사용자에게 알려주는 건강 추적 관리부를 포함할 수 있다. In addition, the voice recognition terminal or the smart remote control control unit is a health tracking management unit for observing changes in the bio-signals and medical data to inform the user of the degree of risk, inform the item requiring intensive care examination, or inform the user of the next examination schedule may include.
상술한 과제 해결 수단은 단지 예시적인 것으로서, 본원을 제한하려는 의도로 해석되지 않아야 한다. 상술한 예시적인 실시예 외에도, 도면 및 발명의 상세한 설명에 추가적인 실시예가 존재할 수 있다.The above-described problem solving means are merely exemplary, and should not be construed as limiting the present application. In addition to the exemplary embodiments described above, additional embodiments may exist in the drawings and detailed description.
전술한 본원의 과제 해결 수단에 의하면, 별도의 사용자 인증 과정없이 사용자 인증이 될 뿐만 아니라, 생활속에서 리모컨 사용 중에 리모컨 상에 설치된 바이오 센서에 의해 사용자의 생체 신호를 측정함으로써, 건강관리의 주요 지표인 체열, 혈압, 혈당, 심장맥박, 산소포화도, 콜레스테롤, 체지방을 리모컨 사용 중에 생활 속에서 관리함으로서 사용자의 질병을 사전에 예방할 수 있다.According to the above-described problem solving means of the present application, not only user authentication is performed without a separate user authentication process, but also the user's bio-signals are measured by the bio-sensor installed on the remote control while using the remote control in daily life, a major indicator of health management By managing body heat, blood pressure, blood sugar, heart pulse, oxygen saturation, cholesterol, and body fat in daily life while using the remote control, user's disease can be prevented in advance.
또한, 본 발명의 스마트 리모컨에 의해 동일한 시점에서 사용자의 몸으로부터 측정된 동기화된 생체를 인공지능 신경망의 입력신호로 사용할 수 있어 정확한 혈압 측정, 콜레스테롤 측정, 혈당 측정이 가능하다.In addition, the synchronized living body measured from the user's body at the same point in time by the smart remote control of the present invention can be used as an input signal of the artificial intelligence neural network, so that accurate blood pressure measurement, cholesterol measurement, and blood sugar measurement are possible.
다만, 본원에서 얻을 수 있는 효과는 상기된 바와 같은 효과들로 한정되지 않으며, 또 다른 효과들이 존재할 수 있다.However, the effects obtainable herein are not limited to the above-described effects, and other effects may exist.
도 1a 및 도 1b는 본원의 일 실시예에 따른 환자의 거주지 내의 일 영역에 설치되며, 음성 인식부와 음성 재생부를 구비하여 마이크를 통해 수집되는 음성 명령에 의해 디지털 TV를 제어하고 스피커를 통해 음성 피드백 서비스를 제공하는 음성 단말기의 일 실시예이다.1A and 1B are installed in an area within a patient's residence according to an embodiment of the present application, and have a voice recognition unit and a voice reproducing unit to control a digital TV by a voice command collected through a microphone, and to control a digital TV through a speaker. This is an embodiment of a voice terminal that provides a feedback service.
도 1c는 본원의 일 실시예에 따른 스마트 리모컨의 일 실시예이다.1C is an embodiment of a smart remote control according to an embodiment of the present application.
도 2a 내지 도 2d는 본원의 일 실시예에 따른 적외선 온도센서, 카메라, 혈당 센서, 체지방 측정용 손 전극, ECG 센서, 청진기 중 선택된 어느 하나를 구비한 바이오 센서를 스마트 리모컨상에 집적화한 일실시예를 나타낸 도면이다.2A to 2D are an embodiment in which a biosensor including any one selected from an infrared temperature sensor, a camera, a blood sugar sensor, a hand electrode for measuring body fat, an ECG sensor, and a stethoscope according to an embodiment of the present application is integrated on a smart remote control. It is a drawing showing an example.
도 2e는 본원의 일 실시예에 따른 손(800)으로 스마트 리모컨을 쥐고 있는 사용자의 손가락들의 정맥 혈관 패턴의 정맥 이미지들을 인식하여 정맥 인증하는 일 실시예를 나타낸 도면이다.FIG. 2E is a diagram illustrating an embodiment of vein authentication by recognizing vein images of a vein blood vessel pattern of a user's fingers holding a smart remote control with a hand 800 according to an embodiment of the present application.
도 3은 본원의 일 실시예에 따른 손으로 스마트 리모컨을 쥐고 있는 사용자의 손바닥의 정맥 혈관 패턴의 정맥 이미지를 인식하여 인증하는 실시예이다.3 is an embodiment of recognizing and authenticating a vein image of a vein blood vessel pattern of a palm of a user holding a smart remote control with a hand according to an embodiment of the present application.
도 4는 본원의 일 실시예에 따른 스마트 리모컨 상에 집적화된 적외선 온도센서를 사용하여 환자의 체열을 측정하는 여러 실시 예이다.4 is a view illustrating various embodiments of measuring a patient's body heat using an infrared temperature sensor integrated on a smart remote controller according to an embodiment of the present application.
도 5는 본원의 일 실시예에 따른 스마트 리모컨 상에 설치된 손 전극들에 의해 체지방을 측정하는 체지방 측정부의 일실시예이다.5 is an embodiment of a body fat measuring unit that measures body fat by hand electrodes installed on a smart remote control according to an embodiment of the present application.
도 6a 내지 도 6c는 본원의 일 실시예에 따른 스마트 리모컨 상에 설치된 비 접촉 ECG센서에 의해 ECG 신호를 측정하는 일실시예이다.6A to 6C are an embodiment of measuring an ECG signal by a non-contact ECG sensor installed on a smart remote control according to an embodiment of the present application.
도 7은 본원의 일 실시예에 따른 스마트 리모컨의 양쪽 좌우 측면에 설치된 반사형 혈관 광센서 어레이 모듈 내에 설치된 SpO2센서를 이용한 산소 포화도 및 PPG신호 측정부 일실시예이다.7 is an embodiment of the oxygen saturation and PPG signal measurement unit using the SpO2 sensor installed in the reflective blood vessel optical sensor array module installed on both left and right sides of the smart remote control according to an embodiment of the present application.
도 8은 본원의 일 실시예에 따른 엄지손가락을 통해 지문 인증이 이루어지도록 사용자가 스마트 리모컨을 잡으면, 자연스럽게 엄지를 제외한 나머지 손가락들은 반사형 혈관 광센서 어레이 모듈과 접촉이 이루어져, 스마트 리모컨 사용 동안 생활속에서 사용자의 산소포화도 와 PPG신호를 계측하는 일실시예이다.8 is a diagram showing when a user holds the smart remote control so that fingerprint authentication is performed through the thumb according to an embodiment of the present application, the remaining fingers except the thumb naturally come into contact with the reflective blood vessel optical sensor array module, so that life while using the smart remote control. This is an example of measuring the user's oxygen saturation and PPG signal in the
도 9는 본원의 일 실시예에 따른 스마트 리모컨 상에 설치된 혈당 센서에 의해 사용자의 손가락으로부터 비침습적으로 글루코스 광 반사 신호를 얻는 일실시예이다.9 is an embodiment of non-invasively obtaining a glucose light reflection signal from a user's finger by a blood glucose sensor installed on a smart remote control according to an embodiment of the present application.
도 10은 본원의 일 실시예에 따른 스마트 리모컨의 후면부에 설치된 청진기의 일 실시예이고,10 is an embodiment of the stethoscope installed on the rear part of the smart remote control according to an embodiment of the present application,
도 11a 내지 도 11d는 본원의 일 실시예에 따른 스마트 리모컨 상에 설치된 바이오 센서로부터 얻어진 생체 신호를 이용하여 혈압, 콜레스테롤, 혈당을 측정하기 위한 일 실시예이다.11A to 11D are exemplary embodiments for measuring blood pressure, cholesterol, and blood sugar by using biosignals obtained from a biosensor installed on a smart remote controller according to an exemplary embodiment of the present disclosure.
도 12는 도 10의 실시예에 따른 스마트 리모컨 상의 청진기를 활용하여 태아의 건강 상태를 자가 검사할 뿐만 아니라, 원격으로 의사가 태아의 건강상태를 체크하기 위해 원격 의료 진단을 수행하는 실시예를 개략적으로 나타낸 도면이다.12 is a schematic diagram of an embodiment in which a doctor performs remote medical diagnosis not only to self-examine the health of the fetus by using the stethoscope on the smart remote control according to the embodiment of FIG. 10 but also to remotely check the health of the fetus It is a drawing shown as
아래에서는 첨부한 도면을 참조하여 본원이 속하는 기술 분야에서 통상의 지식을 가진 자가 용이하게 실시할 수 있도록 본원의 실시예를 상세히 설명한다. 그러나 본원은 여러 가지 상이한 형태로 구현될 수 있으며 여기에서 설명하는 실시예에 한정되지 않는다. 그리고 도면에서 본원을 명확하게 설명하기 위해서 설명과 관계없는 부분은 생략하였으며, 명세서 전체를 통하여 유사한 부분에 대해서는 유사한 도면 부호를 붙였다.Hereinafter, embodiments of the present application will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art to which the present application pertains can easily implement them. However, the present application may be implemented in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present application in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.
본원 명세서 전체에서, 어떤 부분이 다른 부분과 "연결"되어 있다고 할 때, 이는 "직접적으로 연결"되어 있는 경우뿐 아니라, 그 중간에 다른 소자를 사이에 두고 "전기적으로 연결" 또는 “간접적으로 연결”되어 있는 경우도 포함한다. Throughout this specification, when a part is "connected" with another part, it is not only "directly connected" but also "electrically connected" or "indirectly connected" with another element interposed therebetween. “Including cases where it is
본원 명세서 전체에서, 어떤 부재가 다른 부재 "상에", "상부에", "상단에", "하에", "하부에", "하단에" 위치하고 있다고 할 때, 이는 어떤 부재가 다른 부재에 접해 있는 경우뿐 아니라 두 부재 사이에 또 다른 부재가 존재하는 경우도 포함한다.Throughout this specification, when it is said that a member is positioned "on", "on", "on", "under", "under", or "under" another member, this means that a member is located on the other member. It includes not only the case where they are in contact, but also the case where another member exists between two members.
본원 명세서 전체에서, 어떤 부분이 어떤 구성 요소를 "포함"한다고 할 때, 이는 특별히 반대되는 기재가 없는 한 다른 구성 요소를 제외하는 것이 아니라 다른 구성 요소를 더 포함할 수 있는 것을 의미한다.Throughout this specification, when a part "includes" a component, it means that other components may be further included, rather than excluding other components, unless otherwise stated.
이하 첨부된 도면을 사용하여 본 발명의 바람직한 실시예에 대하여 상세하게 설명한다.Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings.
이하, 본 발명에서의 인공지능형 스마트 리모컨 장치는 스마트 리모컨 장치와 혼용될 수 있다.Hereinafter, the artificial intelligent smart remote control device in the present invention may be used interchangeably with the smart remote control device.
이하, 본 발명에서의 인공지능 신경망은 전문가 시스템을 포함한다.Hereinafter, the artificial intelligence neural network in the present invention includes an expert system.
이하, 본 발명에서의 혈압 측정부는 혈압 측정용 인공지능 신경망과 혼용되어 사용될 수 있다.Hereinafter, the blood pressure measurement unit in the present invention may be used in combination with an artificial intelligence neural network for blood pressure measurement.
이하, 본 발명에서의 콜레스테롤 측정부는 콜레스테롤 측정용 인공지능 신경망과 혼용될 수 있다.Hereinafter, the cholesterol measurement unit in the present invention may be mixed with an artificial intelligence neural network for cholesterol measurement.
이하, 본 발명에서의 혈당 측정부는 혈당 측정용 인공지능 신경망과 혼용되어 사용될 수 있다.Hereinafter, the blood glucose measurement unit in the present invention may be used in combination with an artificial intelligence neural network for blood glucose measurement.
본 발명의 인공지능 신경망은 혈압 측정용 인공지능 신경망, 콜레스테롤 측정용 인공지능 신경망, 혈당 측정용 인공지능 신경망, 심장 질환 측정부를 포함할 수 있다.The artificial intelligence neural network of the present invention may include an artificial intelligence neural network for blood pressure measurement, an artificial intelligence neural network for cholesterol measurement, an artificial intelligence neural network for blood sugar measurement, and a heart disease measurement unit.
본 발명에서의 생체신호는 ECG신호, PPG신호, PCG신호, 산소포화도, PTT, 글루코스 광 반사신호 정보를 포함할 수 있다.The biosignal in the present invention may include information on ECG signal, PPG signal, PCG signal, oxygen saturation, PTT, and glucose light reflection signal.
이하, 본 발명에서의 환자, 피 측정자 또는 사용자는 스마트 리모컨 사용자와 혼용되어 사용될 수 있다.Hereinafter, the patient, the subject or the user in the present invention may be used interchangeably with the smart remote control user.
이하, 디지털 TV는 스마트 미러 내지 컴퓨터 모니터로 대체될 수 있다.Hereinafter, the digital TV may be replaced with a smart mirror or a computer monitor.
이하, 음성인식 단말기는 디지털 TV 내에 집적화되어 일체화될 수 있다. 따라서 음성인식 단말기는 디지털 TV, 스마트 미러, 컴퓨터 모니터와 혼용될 수 있다.Hereinafter, the voice recognition terminal may be integrated into the digital TV. Therefore, the voice recognition terminal can be mixed with a digital TV, a smart mirror, and a computer monitor.
이하, 음성인식 단말기는 스마트 리모컨 내에 집적화되어 일체화될 수 있다.Hereinafter, the voice recognition terminal may be integrated into the smart remote control.
이하, 개인 신체 정보(69f)는 체열, 성(sex), 나이, 키(body height), 몸무게, 표준 혈압치, 체지방 정보, HRV(heart Rate Variability) 중 적어도 어느 하나 이상을 선택한 정보인 것을 특징으로 한다. Hereinafter, the personal body information 69f is information that selects at least one of body heat, sex, age, height, weight, standard blood pressure, body fat information, and HRV (heart rate variability). do.
도 1a 및 도 1b는 환자의 거주지 내의 일 영역에 설치되며, 음성 인식부(27a)와 음성 재생부(28a)를 구비하여 마이크(23)를 통해 수집되는 음성 명령에 의해 디지털 TV(300)를 제어하고 스피커(28)를 통해 음성 피드백 서비스를 제공하는 음성 단말기(17)의 일 실시예로, 외부 의료 기기들과 근거리 무선 통신 연결 (예컨대 NFC(Near Field Communication) 인터페이스, 블루투스, 사물 인터넷 또는 적외선 통신 연결)을 제공하는 디지털 통신 모듈(8), 상기 디지털 통신 모듈(8)을 통해 상기 의료기기들에 의해 환자로부터 측정된 의료 데이터를 수신하는 의료 데이터 수신부(12)를 포함한다.1A and 1B show a digital TV 300 installed in an area within the patient's residence, provided with a voice recognition unit 27a and a voice reproducing unit 28a, and a voice command collected through a microphone 23. As an embodiment of the voice terminal 17 that controls and provides a voice feedback service through the speaker 28, a short-range wireless communication connection with external medical devices (eg, NFC (Near Field Communication) interface, Bluetooth, Internet of Things or infrared rays) a digital communication module 8 for providing a communication connection), and a medical data receiving unit 12 for receiving medical data measured from the patient by the medical devices through the digital communication module 8 .
본원의 일 실시예에 따르면, 음성 인식 단말기(17)는 음성 명령에 의해 디지털 TV(300)에 제어할 뿐만 아니라, 디지털 TV(300)를 통해 원격 의료 진단을 수행하는 것을 특징으로 한다. According to an embodiment of the present application, the voice recognition terminal 17 not only controls the digital TV 300 according to a voice command, but also performs remote medical diagnosis through the digital TV 300 .
상기 디지털 통신 모듈(8)은 의료 전문가와의 원격 의료 진단을 허여하는 인터넷 및 와이 파이 통신 연결을 제공하거나 외부 의료기기와의 근거리 무선 통신 연결을 제공한다.The digital communication module 8 provides an Internet and Wi-Fi communication connection that allows remote medical diagnosis with a medical professional, or provides a short-range wireless communication connection with an external medical device.
본 실시예에서는 디지털 TV(300)는 스마트 미러로 대체되어 사용될 수 있다.In this embodiment, the digital TV 300 may be replaced with a smart mirror.
또한, 제어부(30)는 인공지능 신경망(16)으로부터의 의료 데이터 분석 결과를 가지고, 환자의 건강 상태에 따라 환자에게 건강관리에 대한 지침, 원격 의료 및 의료기기 사용법에 대한 가이드라인을 음성과 영상 서비스로 제공하기 위해, 상기 디지털 통신 모듈(8), 스피커(28) 와 디지털 TV(300)를 제어하는 것을 특징으로 한다.In addition, the control unit 30 has the result of analyzing the medical data from the artificial intelligence neural network 16, and according to the patient's health condition, the control unit 30 provides a guideline for health management, telemedicine and medical device usage to the patient through audio and video. In order to provide a service, it is characterized in that the digital communication module (8), the speaker (28) and the digital TV (300) are controlled.
또한, 제어부(30)는 상기 인공지능 신경망(16)으로부터의 의료 데이터 분석 결과를 가지고, 환자의 건강 상태에 따라 원격 의료 진단의 필요성을 판단하고, 필요한 경우 의사와 환자 간의 원격 의료 진단을 수행을 위해 디지털 통신 모듈(8), 스피커(28) 와 디지털 TV(300)를 제어한다.In addition, the control unit 30 has the result of analyzing the medical data from the artificial intelligence neural network 16, determines the need for remote medical diagnosis according to the patient's health condition, and performs remote medical diagnosis between the doctor and the patient if necessary. To control the digital communication module 8, the speaker 28 and the digital TV (300).
본원의 일 실시예에 따르면 의료기기(430)는 환자의 검체 내지 환부로부터 측정된 의료 데이터를 의료 데이터 수신부(12) 내지 의료 데이터 저장부(15)에 송출하는 무선 송신부(예컨대 NFC 인터페이스부, 블루투스 통신 연결부 또는 적외선 통신 부)를 구비한 기기로서 스마트 리모컨(400), 헬스 케어 어플이 포함된 핸드폰, 스마트 워치, 안경형 스마트 기기, 초음파 스캐너, 열화상 카메라, 변기에 설치된 대변 검사기, 변기에 설치된 소변 검사기, 혈압 측정 장치, 혈당 측정기, 체중계, 인후통 내지 치아 상태를 보여주는 이미지 센서, 자동화된 혈액 분석기, DNA 증폭 검사 장치, 바이러스의 특이 항원을 이용해 진단하는 바이러스 진단 키트 기기, 바이오 마커(bio marker)를 사용한 래피드 테스트(Rapid test) 기기, 스마트 워치, 웨어러블(wearable) 기기, 암 진단 기기, POCT(Point of Care Testing) 기기 중 선택된 의료 기기 중 적어도 어느 하나를 포함할 수 있다.According to an embodiment of the present application, the medical device 430 is a wireless transmitter (eg, NFC interface, Bluetooth) that transmits medical data measured from the patient's specimen or the affected part to the medical data receiving unit 12 to the medical data storage unit 15 . A device equipped with a communication connection unit or infrared communication unit), a smart remote control 400, a mobile phone with a health care application, a smart watch, a smart glasses-type device, an ultrasound scanner, a thermal imager, a stool tester installed in the toilet, urine installed in the toilet Tester, blood pressure measuring device, blood glucose meter, weight scale, image sensor showing sore throat or tooth condition, automated blood analyzer, DNA amplification test device, virus diagnostic kit device that diagnoses using virus specific antigen, biomarker It may include at least one selected from among a used rapid test device, a smart watch, a wearable device, a cancer diagnosis device, and a point of care testing (POCT) device.
스마트 리모컨(400)은 체온, 혈당, 체지방, 콜레스테롤, 심전도, 산소포화도, PCG신호, 혈압에 대한 의료데이터를 음성인식 단말기(17)의 의료 데이터 수신부(12)에 송출할 수 있다.The smart remote control 400 may transmit medical data on body temperature, blood sugar, body fat, cholesterol, electrocardiogram, oxygen saturation, PCG signal, and blood pressure to the medical data receiver 12 of the voice recognition terminal 17 .
상기 웨어러블(wearable) 기기는 환자의 피부에 부착되는 패치(patch)형 웨어러블 기기가 선호되며, 패치 내에 내장된 센서에 의해 환자의 피부 표면에서 심전도, 혈압 콜레스테롤 또는 혈당을 비침습적으로 측정한 데이터를 블루투스 통신 연결을 통해, 의료 데이터 수신부(12) 내지 의료 데이터 저장부(15)에 무선 전송하는 것이 선호된다.The wearable device is preferably a patch-type wearable device attached to the patient's skin, and data obtained by non-invasively measuring the electrocardiogram, blood pressure, cholesterol, or blood sugar on the skin surface of the patient by a sensor built into the patch. It is preferred to wirelessly transmit to the medical data receiving unit 12 to the medical data storage unit 15 through a Bluetooth communication connection.
상기 패치형 웨어러블 기기는 혈당 센서를 내장한 패치를 피부에 테이프로 붙인 다음, 상기 패치에 스마트 리모컨(400)을 갖다 대면, NFC인터페이스에 의해 스마트 리모컨(400)에 내장된 리더기로 상기 패치에 동력을 공급해 혈액 내 물과 포도당의 비율을 측정해 혈당을 측정하여 스마트 리모컨(400)의 의료데이터 저장부(15)에 혈당 수치를 제공하는 기기일수 있다.The patch-type wearable device attaches a patch with a built-in blood glucose sensor to the skin with tape, and then brings the smart remote control 400 to the patch. It may be a device that provides a blood sugar level to the medical data storage unit 15 of the smart remote control 400 by measuring blood sugar by measuring the ratio of water and glucose in the blood.
상기 패치형 웨어러블 기기의 또 다른 측면은 복부, 팔, 엉덩이 등 피하지방에 부착된 센서를 통해 세포 간질액(세포와 세포 사이를 채우는 액체 성분)의 포도당 농도를 주기적 간격으로 측정하여, 측정 결과를 근거리 무선 통신 연결을 통해 무선으로 의료 데이터 저장부(15) 내지 의료데이터 수신부(12)에 전송하는 연속혈당측정기(CGM, Continuous Glucose Monitoring) 일수 있다.Another aspect of the patch-type wearable device is to measure the glucose concentration of the interstitial fluid (a liquid component filling between cells) at periodic intervals through a sensor attached to subcutaneous fat such as the abdomen, arm, and buttock, so that the measurement result is measured in a short distance. It may be a continuous glucose monitoring (CGM) that wirelessly transmits the medical data storage unit 15 to the medical data receiving unit 12 through a wireless communication connection.
또한, 음성인식 단말기(17)는, 의료 데이터 수신부(12)에 의해 수집된 환자의 온도가 비정상적인 체열 온도인 경우, 이상 체열 의심 대상자로 판별하기 위한 체열 진단부(19)를 포함할 수 있다. Also, when the patient's temperature collected by the medical data receiving unit 12 is an abnormal body heat temperature, the voice recognition terminal 17 may include a body heat diagnosis unit 19 for determining an abnormal body heat suspect.
또한, 음성인식 단말기(17)는 바이오 센서 및 의료기기에 의한 주기적 검사에 따른 의료 데이터의 변화 추이를 관찰하여 환자(사용자)에게 위험도, 집중케어 검사가 필요한 항목 및 다음 검사 일정을 포함하는 부가정보를 환자에게 알려주는 건강 추적 관리부(11)를 포함할 수 있다.In addition, the voice recognition terminal 17 observes the change in medical data according to the periodic examination by the biosensor and medical device, and additional information including the risk to the patient (user), items requiring intensive care examination, and the next examination schedule may include a health tracking management unit 11 that informs the patient.
또한, 디지털 TV(300)는 스마트 리모컨(400)로부터 수집된 의료 데이터를 모아, 정리 및 분석한 결과를 화면상에 표시해 주거나 건강 추적 관리부(11)에서 제공해주는 부가 정보를 표시하는 것이 선호된다. In addition, it is preferred that the digital TV 300 collects, organizes, and analyzes the medical data collected from the smart remote control 400 and displays the results on the screen or additional information provided by the health tracking management unit 11 .
도 1a는, 음성인식 단말기(17)가 인공지능신경망(16) 내지 전문가 시스템(18)을 내부에 구비하여, 스마트 리모컨(400)에 의해 측정된 의료 데이터를 의료데이터 수신부(12)에 의해 수신한 후, 이를 상기 인공지능신경망(16) 내지 전문가 시스템(18)에 의해 분석하여 질병의 위험도를 환자에게 통지할 수 있는 실시예를 보인다.1A, the voice recognition terminal 17 has an artificial intelligence neural network 16 or an expert system 18 therein, and medical data measured by the smart remote control 400 is received by the medical data receiving unit 12. After this, an embodiment in which the artificial intelligence neural network 16 or the expert system 18 can analyze this to notify the patient of the risk of the disease is shown.
인공지능신경망(16)은 의료기기들에 의해 사전 수집된 학습용 의료 데이터들에 의해 딥런닝 학습된다.The artificial intelligence neural network 16 is deep learning learned by medical data for learning previously collected by medical devices.
이후, 음성 인식 단말기(17)는 의료기기들로부터 측정된 환자의 의료 데이터를 상기 기 학습된 인공 신경망(16)에 적용하여, 환자의 질병 유무와 질병의 위험도를 자동으로 판별할 수 있다.Thereafter, the voice recognition terminal 17 may apply the patient's medical data measured from the medical devices to the pre-learned artificial neural network 16 to automatically determine whether the patient has a disease or not and the risk of the disease.
또한, 음성인식 단말기(17)는 의료 데이터 수신부(12)에 의해 수집된 데이터에 대해 환자의 질병의 위험요소를 판별하기 위한 전문가 시스템(expert system, 18)을 포함할 수 있다. In addition, the voice recognition terminal 17 may include an expert system 18 for determining a risk factor of a patient's disease with respect to the data collected by the medical data receiving unit 12 .
상기 전문가 시스템(18)은 의료 전문가의 지적 활동과 경험을 통해서 축적된 지식과 의료 전문가에 의해 정의된 추론 규칙을 활용하여 결정을 내리거나 문제 해결을 하는 컴퓨터 응용 프로그램 일수 있다. The expert system 18 may be a computer application program that makes decisions or solves problems by utilizing knowledge accumulated through the intellectual activities and experiences of the medical experts and inference rules defined by the medical experts.
상기 인공지능 신경망(16)은 음성인식 단말기(17)상에 어플(application software)로서 설치되는 인공지능 신경망 앱(app) 일수 있다. 이 경우 의료 데이터 수신부(12)에 의해 수집된 데이터를 상기 인공지능 신경망 앱에 의해 분석하여 그 결과를 사용자에게 제공할 수 있다.The artificial intelligence neural network 16 may be an artificial intelligence neural network app installed as an application software on the voice recognition terminal 17 . In this case, the data collected by the medical data receiving unit 12 may be analyzed by the artificial intelligence neural network app and the result may be provided to the user.
도 1b는, 음성인식 단말기(17)의 의료 데이터 수신부(12)가 스마트 리모컨(400)에 의해 측정된 의료 데이터를 수신하고, 이를 인터넷망을 통해 서버(server)에 재전송하고, 이후 음성인식 단말기(17)는 서버(13)상의 인공지능신경망(16) 내지 전문가 시스템(18)에 의해 분석된 결과를 피드백 받아 환자에게 환자의 질병 유무와 질병의 위험도를 통지하는 실시예를 보인다.1b, the medical data receiving unit 12 of the voice recognition terminal 17 receives the medical data measured by the smart remote control 400, and retransmits it to a server through the Internet network, and then the voice recognition terminal (17) shows an embodiment of notifying the patient of the presence or absence of the disease and the risk of the disease to the patient by receiving feedback from the results analyzed by the artificial intelligence neural network 16 or the expert system 18 on the server 13.
도 1c는 본 발명을 따르는 스마트 리모컨(400)의 일 실시예로, 생체 신호 측정과 상기 생체신호 측정 동안 사용자 인증을 동시에 수행하는 바이오 센서, 외부 의료기기(430)들에 의해 측정된 의료 데이터를 근거리 무선 통신 연결에 의해 수신하거나 바이오 센서(440)로부터 생체 신호 수집부(420)를 통해 얻어진 의료 데이터를 저장하기 위한 의료 데이터 저장부(15), 상기 의료 데이터를 인터넷(202)을 통해 서버(13)에 재전송하고, 이후 서버상의 인공지능 신경망(16) 내지 전문가 시스템(18)에 의해 상기 의료 데이터의 분석한 결과를 피드백 받기 위한 무선 통신 연결 수단(37); 및 상기 각부를 제어하고 의료 데이터의 분석한 결과를 터치 스크린(401)을 통해 사용자에게 제공하는 스마트 리모컨 제어부(53)를 포함한다.1C is an embodiment of the smart remote control 400 according to the present invention, and the bio-signal measurement and the medical data measured by the bio-sensor that simultaneously performs user authentication during the bio-signal measurement and the external medical devices 430 are displayed. A medical data storage unit 15 for storing medical data received by a short-range wireless communication connection or obtained from the biosensor 440 through the biosignal collection unit 420, and the medical data through the Internet 202 to a server ( a wireless communication connection means 37 for retransmitting to 13) and then receiving a feedback result of the analysis of the medical data by the artificial intelligence neural network 16 or the expert system 18 on the server; and a smart remote control controller 53 that controls the respective parts and provides a result of analyzing the medical data to the user through the touch screen 401 .
상기 생체 신호 수집부(420)는 생체신호 유효 판별 수단(420a)을 구비하여, 바이오 센서(440)로부터 생체 신호 수집부(420)에 의해 수집된 생체 신호들 중 유효한 생체 신호를 판별하며, 스마트 리모컨(400)상의 바이오 센서(440)에 의해 수집된 생체 신호들 중 사용자 인증이 완료되고 동시에 유효한 생체신호 (ECG신호, PPG신호, 산소포화도, 글루코스 광 반사 신호 정보, PCG신호, 체지방, 체온)만을 걸러서 의료 데이터 저장부(15)에 저장한다.The bio-signal collecting unit 420 includes a bio-signal valid determining means 420a to discriminate valid bio-signals among bio-signals collected by the bio-signal collecting unit 420 from the bio-sensor 440, Among the bio-signals collected by the bio-sensor 440 on the remote control 400, user authentication is completed and valid bio-signals (ECG signal, PPG signal, oxygen saturation, glucose light reflection signal information, PCG signal, body fat, body temperature) Only every filter is stored in the medical data storage unit 15 .
다시 말해, 상기 생체 신호 수집부(420)는 바이오 센서(440)로부터 수집된 생체 신호들 중 사용자 인증이 완료된 생체 신호 성분 중 유효한 생체신호 부분만 걸러내는 역할을 하게 된다.In other words, the bio-signal collection unit 420 filters out only a valid bio-signal portion from among bio-signal components for which user authentication has been completed from among the bio-signals collected from the bio-sensor 440 .
상기 생체신호 유효 판별 수단(420a)은 1-D(Dimension) 인공지능 신경망, LSTM(Long short time Memory), 자기 상관 계수(Auto Correlation Coefficient), 유효 범위 체크 방법, 표준(기준) 생체 신호의 특징 벡터와 수집된 생체 신호의 특징벡터와의 상호 상관 계수 중 선택된 어느 하나 이상의 기법을 사용하는 것이 선호된다.The biosignal validity determination means 420a includes a 1-D (Dimension) artificial intelligence neural network, LSTM (Long short time memory), Auto Correlation Coefficient, an effective range check method, and a standard (standard) biosignal characteristic. It is preferred to use any one or more techniques selected from the cross-correlation coefficient between the vector and the feature vector of the collected biosignal.
바람직하게는, 상기 상호 상관 계수는 SSD(Sum of Squared Difference), SAD(Sum of Absolute Difference), Euclidean Distance, KNN (K-nearest neighbor algorithm), NCC(Normalized Cross Correlation) 중 어느 하나를 사용하여 두 성분 간에 유사도 내지 상관성 정도를 산출하는 것이 선호된다.Preferably, the cross-correlation coefficient is calculated using any one of Sum of Squared Difference (SSD), Sum of Absolute Difference (SAD), Euclidean Distance, K-nearest neighbor algorithm (KNN), and Normalized Cross Correlation (NCC). It is preferred to calculate the degree of similarity or correlation between components.
예컨대 표준(기준) 생체 신호의 특징 벡터와 바이오 센서(440)를 통해 수집된 생체 신호의 특징 벡터 간에 상호 상관 계수가 정해진 문턱치보다 큰 경우, 바이오 센서(440)를 통해 수집된 생체 신호는 유효한 생체 신호라 판단한다. For example, when the cross-correlation coefficient between the standard (reference) biosignal feature vector and the biosignal feature vector collected through the biosensor 440 is greater than a predetermined threshold, the biosignal collected through the biosensor 440 is a valid biometric signal. consider it a signal.
표준(기준) 생체 신호는 통계적으로 정상인의 생체 신호의 특성과 수치를 보여주는 생체 신호(ECG신호, PPG신호, 산소포화도, 글루코스 광 반사 신호 정보, PCG신호, 체지방, 체온)로, 표준(기준) 생체 신호의 특징 벡터는 정상인의 생체 신호에 대한 특성 벡터의 평균값들이 사용될 수 있으며 복수개의 표준(기준) 생체신호의 특징 벡터가 사용될 수 있다.Standard (standard) biosignal is a biosignal (ECG signal, PPG signal, oxygen saturation, glucose light reflection signal information, PCG signal, body fat, body temperature) that shows the characteristics and values of the biosignal of a statistically normal person, standard (reference) As the feature vector of the biosignal, average values of the feature vector for the biosignal of a normal person may be used, and a plurality of standard (reference) biosignal feature vectors may be used.
상기 생체 신호의 특징 벡터는 딥런닝 신경회로망(Deep learning Neural Network) 내지 주파수 영역의 스펙트럼(spectrum) 분석에 의해 추출될 수 있다. The feature vector of the biosignal may be extracted by a deep learning neural network or spectrum analysis in the frequency domain.
예컨대 상기 딥런닝 신경회로망(Deep learning Neural Network)은 1-D CNN(Convolutional Neural Network)이 될 수 있다For example, the deep learning neural network may be a 1-D convolutional neural network (CNN).
상기 주파수 영역의 스펙트럼 분석에 의한 특징 벡터는, 1-D(Dimension) 인공지능 신경망, 주파수 성분 분석, STFT(Short time Fourier Transform), 웨이블릿 변환(Wavelet transform), MFCC(Mel-Frequency Cepstral Coefficient), Linear Prediction Codes(LPC), Mel-Frequency Cepstral Coefficient(MFCC), Mel-Coefficient, Preceptual Linear Prediction(PLP) 또는 Bark Frequency Cepstral Coefficients (BFCC)에 의해 추출 된 것 일수 있다.The feature vector by spectrum analysis of the frequency domain is 1-D (Dimension) artificial intelligence neural network, frequency component analysis, STFT (Short time Fourier Transform), wavelet transform (Wavelet transform), MFCC (Mel-Frequency Cepstral Coefficient), It may be extracted by Linear Prediction Codes (LPC), Mel-Frequency Cepstral Coefficient (MFCC), Mel-Coefficient, Preceptual Linear Prediction (PLP), or Bark Frequency Cepstral Coefficients (BFCC).
예컨대, 1-D 인공지능 신경망 내지 LSTM에 의한 생체신호 유효 판별 수단(420a)은 유효 생체 신호 또는 무효 생체 신호로 표지된 ECG신호, PPG신호, PCG신호들을 이용하여 학습된 1-D 인공지능 신경망에 의해 피 측정자의 생체 신호의 유효성을 판별할 수 있다.For example, the 1-D artificial intelligence neural network or the LSTM-based biosignal validity determination means 420a is a 1-D artificial intelligence neural network learned using ECG signals, PPG signals, and PCG signals labeled with valid or invalid biosignals. By this, it is possible to determine the validity of the biosignal of the subject.
체온의 경우 유효범위는 10도에서 40도가 바람직하며, 이 범위 내의 체온이 1초 이상 같은 온도 상태를 유지되는 경우 유효한 체온 신호로서 인정되는 것이 바람직하다.In the case of body temperature, the effective range is preferably 10 to 40 degrees, and when the body temperature within this range maintains the same temperature state for 1 second or more, it is preferably recognized as an effective body temperature signal.
상기 사용자 인증은 지문 인식부(24,25,26,27) 내지 정맥 센서부(54a)에 의해 이루어 질 수 있다.The user authentication may be performed by the fingerprint recognition unit 24, 25, 26, 27 or the vein sensor unit 54a.
상기 바이오 센서(440)는 심전도 측정부(71), 산소 포화도 및 PPG신호 측정부(72), 체지방 측정부(70), 청진음 측정부(73)를 포함하며, 상기 심전도 측정부(71)는 심전도를 측정하기 위한 ECG센서(48), 산소 포화도 및 PPG신호 측정부(72)는 산소 포화도 및 광전용 적맥파 신호를 측정하기 위한 SpO2 센서(54b) 및 PPG센서, 청진음 측정부(73)는 심음 신호를 얻기 위한 청진기(44), 체지방 측정부(70)는 체지방을 측정하기 위한 복수의 손전극(24a,25a,26a,27a)을 포함한다.The biosensor 440 includes an electrocardiogram measuring unit 71 , oxygen saturation and PPG signal measuring unit 72 , body fat measuring unit 70 , and auscultation sound measuring unit 73 , and the electrocardiogram measuring unit 71 . ECG sensor 48 for measuring the electrocardiogram, oxygen saturation and PPG signal measuring unit 72, SpO2 sensor 54b and PPG sensor for measuring oxygen saturation and photo-only red pulse wave signal, and auscultation sound measuring unit 73 ) is a stethoscope 44 for obtaining a heart sound signal, and the body fat measuring unit 70 includes a plurality of hand electrodes 24a, 25a, 26a, and 27a for measuring body fat.
또한, 스마트 리모컨 제어부(53)는 생체신호 측정 요구 수단을 구비하여 생체 신호 측정을 요구하는 터치스크린 문자나 음성 메시지를 사용자에게 전달할 수 있다. In addition, the smart remote control controller 53 may include a biosignal measurement request means to transmit a touch screen text or voice message requesting biosignal measurement to the user.
예컨대 터치 스크린(401) 화면을 건강관리 항목으로 강제 전환한 후, 집중케어 항목을 점멸하거나, 음성 재생을 위한 스피커부(38a)와 음성인식을 위한 마이크부(38b)를 이용하여 대화형으로 사용자에게 생체신호 측정을 요구할 수 있다.For example, after the screen of the touch screen 401 is forcibly switched to a health care item, the intensive care item is blinked, or the user interactively uses the speaker unit 38a for voice reproduction and the microphone unit 38b for voice recognition. You may request to measure biosignals.
또한, 스마트 리모컨 제어부(53)는 생체 신호 및 의료 데이터의 변화 추이를 관찰하여 사용자에게 위험도를 알려주거나 집중케어 검사가 필요한 항목을 알려주거나 다음 검사 일정을 사용자에게 알려주는 건강 추적 관리부를 포함할 수 있다.In addition, the smart remote control control unit 53 may include a health tracking management unit that notifies the user of the degree of risk by observing changes in biosignals and medical data, informs the user of items requiring intensive care examination, or informs the user of the next examination schedule. have.
도 1c을 따르는 스마트 리모컨(400)의 또 다른 측면은, 서버(13)상의 인공지능 신경망(16)을 사용하는 대신에 스마트 리모컨 상에 상주하고, 어플(application software)로서 설치되는 인공지능 신경망 앱을 구비하여, 바이오 센서에 의해 측정된 의료 데이터를 분석하여 그 결과를 사용자에게 제공하는 것을 특징으로 한다.Another aspect of the smart remote control 400 according to FIG. 1c is an artificial intelligence neural network app that resides on the smart remote control instead of using the artificial intelligence neural network 16 on the server 13 and is installed as application software. is provided, analyzing the medical data measured by the biosensor, and providing the result to the user.
이 경우 상기 인공지능 신경망 앱은 버전 업그레이드에 따라 스마트 리모컨의 성능 개선을 꾸준히 개선할 수 있는 장점을 제공한다.In this case, the artificial intelligence neural network app provides the advantage of continuously improving the performance of the smart remote control according to the version upgrade.
도 1c을 따르는 스마트 리모컨(400)의 또 다른 측면은, 외부 의료기기(430)들에 의해 측정된 의료 데이터를 근거리 통신 연결에 의해 수신하거나 바이오 센서(440)로부터 생체 신호 수집부(420)를 통해 얻어진 의료 데이터를 저장하기 위한 의료 데이터 저장부(15) 및 상기 의료 데이터를 음성 인식 단말기(17)에 전송하기 위한 무선 통신 연결 수단(37)을 포함하고 음성 인식 단말기(17)는 상기 무선 통신 연결 수단(37)으로부터 제공 받은 의료 데이터를 인터넷(202)을 통해 서버(13)상의 인공지능 신경망(16) 내지 전문가 시스템(18)에 전송하고, 상기 인공지능 신경망(16) 내지 전문가 시스템(18)에 의해 분석된 결과를 피드백 받아 디지털 TV(300)을 통해 사용자에게 제공한다.Another aspect of the smart remote control 400 according to FIG. 1C is to receive medical data measured by external medical devices 430 through a short-range communication connection or use the bio-signal collection unit 420 from the biosensor 440 . and a medical data storage unit 15 for storing the medical data obtained through the wireless communication connection means 37 for transmitting the medical data to the voice recognition terminal 17, wherein the voice recognition terminal 17 includes the wireless communication The medical data provided from the connection means 37 is transmitted to the artificial intelligence neural network 16 to the expert system 18 on the server 13 through the Internet 202, and the artificial intelligence neural network 16 to the expert system 18 ) and provides feedback to the user through the digital TV 300 .
도 2a 내지 도 2d는 적외선 온도센서(29), 카메라(22), SpO2센서(54b), 혈당 센서(31), 체지방 측정용 손 전극(24a, 25a, 26a, 27a), ECG 센서(48), 청진기(44) 중 선택된 어느 하나를 구비한 바이오 센서를 스마트 리모컨(400) 상에 집적화한 일실시예로, 스마트 리모컨(400)에 의해 환자로부터 측정된 생체 신호를 근거리 무선 통신 연결에 의해 의료 데이터 수신부(12)에 무선 전송하거나 의료 데이터 저장부(15)에 저장할 수 있다.2a to 2d show an infrared temperature sensor 29, a camera 22, an SpO2 sensor 54b, a blood glucose sensor 31, hand electrodes 24a, 25a, 26a, 27a for measuring body fat, and an ECG sensor 48 As an embodiment in which a biosensor having any one selected from among , stethoscope 44 is integrated on the smart remote control 400, the bio-signals measured from the patient by the smart remote control 400 can be used for medical treatment by short-range wireless communication connection. It can be wirelessly transmitted to the data receiving unit 12 or stored in the medical data storage unit 15 .
도 2a의 (a)와 도 2b는 스마트 리모컨(400)의 정면도의 일실시예로, 스마트 리모컨(400)의 정면에 건강관리 항목을 선택하거나 리모컨 선택 버튼들을 표시하기 위한 터치 스크린(401)이 구비된다. 2a (a) and 2b are one embodiment of a front view of the smart remote control 400, a touch screen 401 for selecting health care items or displaying remote control selection buttons on the front of the smart remote control 400 provided
도 2a의 (a)는 리모컨 모드 동안에 디지털 TV(300)의 기능을 제어하기 위한 리모컨 버튼들이 터치 스크린(401)상에 표시된 리모컨 모드의 실시예이고, 도 2a의 (b)는 건강관리 모드시의 터치 스크린(401)상에 건강관리 항목들이 표시되는 실시 예이다. FIG. 2A (a) is an embodiment of the remote control mode in which remote control buttons for controlling functions of the digital TV 300 are displayed on the touch screen 401 during the remote control mode, and (b) of FIG. 2A is in the health care mode This is an embodiment in which health care items are displayed on the touch screen 401 of
스마트 리모컨(400)은 터치 스크린(401)상에 리모컨 모드와 건강관리 모드 화면을 서로 번갈아 가면서 전환하는 메뉴 버튼이 구비되고, 리모컨 모드 동안에는 디지털 TV(300)의 기능을 제어하기 위한 리모컨 선택 버튼들이 터치 스크린(401) 화면상에 표시되고, 건강관리 모드 동안에는 심전도 검사, 산소 포화도 검사, 혈당 검사, 체지방 검사, 콜레스테롤 검사, 혈압 검사 등을 포함하는 건강관리 항목 중 하나를 선택하기 위한 메뉴 선택 버튼들이 터치 스크린(401) 화면상에 표시될 수 있다. The smart remote control 400 is provided with a menu button for alternately switching between the remote control mode and the health care mode screen on the touch screen 401, and during the remote control mode, remote control selection buttons for controlling the functions of the digital TV 300 are provided. Displayed on the touch screen 401 screen, menu selection buttons for selecting one of health care items including an electrocardiogram, oxygen saturation test, blood sugar test, body fat test, cholesterol test, blood pressure test, etc. are provided during the health management mode. The touch screen 401 may be displayed on the screen.
본 발명에서, 리모컨 모드 동안에는 사용자 인증 및 심전도 검사, 산소 포화도 검사, 혈당 검사, 체지방 검사, 콜레스테롤 검사, 혈압 검사를 포함하는 건강관리 항목들에 대한 생체 신호의 의료 데이터 수집 및 전송이 무의식 중에 생활 속에서 자동으로 이루어지고, 인공지능 신경망은 이들 전송된 의료데이터를 분석하여 그 결과를 피드백하고, 스마트 리모컨(400)의 건강 추적 관리부는 집중 케어가 필요한 건강관리 항목을 발굴하고, 발굴된 집중 케어 항목에 대해 건강관리 모드를 통해 자가 검사하도록 유도하는 것이 선호된다. In the present invention, during the remote control mode, user authentication and medical data collection and transmission of biosignals for health care items including electrocardiogram, oxygen saturation test, blood sugar test, body fat test, cholesterol test, and blood pressure test are unconsciously performed in daily life. The artificial intelligence neural network analyzes these transmitted medical data and feeds back the result, and the health tracking management unit of the smart remote control 400 discovers health care items requiring intensive care, and discovers intensive care items It is preferred to induce self-examination through the health management mode.
또한, 스마트 리모컨 제어부(53)는 집중 케어 항목들을 터치 스크린(401)상에서 점멸하여 표시함으로써, 사용자에게 집중 케어 항목들을 자가 검사하도록. 생체신호 측정 요구를 할 수 있다.In addition, the smart remote control controller 53 displays the intensive care items by blinking on the touch screen 401 so that the user can self-examine the intensive care items. Biosignal measurement can be requested.
도면 부호 39는 디지털 TV를 제어하기 위한 신호를 송출하는 리모컨 송신부이다.Reference numeral 39 denotes a remote control transmitter that transmits a signal for controlling a digital TV.
도 2b는 스마트 리모컨(400)의 상하 좌우 측면에 설치된 지문 인증부(24,25,26,27)와 네개의 손 전극들(24a,25a,26a,27a)을 보인다.2B shows the fingerprint authentication units 24, 25, 26, 27 and four hand electrodes 24a, 25a, 26a, and 27a installed on the upper, lower, left, and right sides of the smart remote control 400 .
상기 복수의 손 전극(24a, 25a, 26a, 27a)은 지문 인증부(24,25,26,27)의 외주 테두리를 금속 도체로 감싸는 형식으로 구성되며, 지문 인증 동안 손가락의 지문 부위가 손 전극에도 동시에 접촉되게 된다. 이때 이들 손 전극들은 체지방 측정 내지 심전도 측정 시에 바이오 센서로서 동작하게 된다.The plurality of hand electrodes 24a, 25a, 26a, and 27a are configured in such a way that the outer periphery of the fingerprint authentication unit 24, 25, 26, and 27 is wrapped with a metal conductor, and the fingerprint portion of the finger is the hand electrode during fingerprint authentication. are also brought into contact at the same time. In this case, these hand electrodes operate as biosensors when measuring body fat or measuring an electrocardiogram.
손 전극들 (24a, 25a, 26a, 27a)의 후쿠(14a, 14b, 14c, 14d)는 연결 전극들(24b, 25b, 26b, 27b) 과의 조립 시 물리적으로 맞물려 체결되는 동시에 전기적으로 연결된다. The hooks 14a, 14b, 14c, and 14d of the hand electrodes 24a, 25a, 26a, and 27a are physically engaged with the connection electrodes 24b, 25b, 26b, and 27b when assembling and are electrically connected at the same time. .
도 2c는 스마트 리모컨(400)의 후면도의 여러 실시예로, 스마트 리모컨(400)의 후면부에 청진기(44), ECG 센서(48)가 구비된다. 2c is a rear view of the smart remote control 400 according to various embodiments of the present invention, and the stethoscope 44 and the ECG sensor 48 are provided on the rear portion of the smart remote control 400 .
상기 청진기(44)는 디지털 TV(300)를 음성 명령으로 제어하기 위한 마이크 입력으로도 사용 가능하다. 이 경우, 디지털 TV(300)와 다소 동떨어진 곳에 마이크 입력이 있게 되어, TV소음의 혼입이 최소화되어, TV 온 상태에서도 디지털 TV(300)에 대한 음성 명령 제어에 유리하다. The stethoscope 44 can also be used as a microphone input for controlling the digital TV 300 with a voice command. In this case, there is a microphone input at a place somewhat distant from the digital TV 300 , and thus mixing of TV noise is minimized, which is advantageous for voice command control for the digital TV 300 even when the TV is on.
또한, 스마트 리모컨 제어부(53)는 청진기(44) 사용 기간 동안 디지털 TV(300)의 볼륨이 자동으로 줄어들도록 제어하고, 심장이나 폐의 소리를 증폭하여 의료 데이터 수신부(12)에 전송하거나 의료 데이터 저장부(15)에 저장하는 것이 선호된다. 상기 청진기(44) 사용 기간 판단은 유효한 ECG 신호가 포착되었을 때로 산정하는 것이 선호된다 In addition, the smart remote control controller 53 controls the volume of the digital TV 300 to automatically decrease during the period of use of the stethoscope 44, amplifies the sound of the heart or lungs, and transmits the sound to the medical data receiver 12 or medical data It is preferred to store it in the storage unit 15 . The stethoscope 44 usage period determination is preferably calculated when a valid ECG signal is acquired.
또한, 청진기 사용 종료시에는 사용 전의 디지털 TV(300)의 볼륨 크기로 자동 복귀되도록 하는 것이 선호된다In addition, it is preferable to automatically return to the volume level of the digital TV 300 before use when the stethoscope is used.
또한, 스마트 리모컨 제어부(53)는 청진기 사용 동안에는 청진기(44)를 제어하여 심장 소리(20Hz 내지 200Hz)에 적합한 벨(Bell) 모드, 폐 소리(200Hz 내지 1KHz)에 적합한 다이어프램(Diaphram) 모드, 상기 벨 모드와 다이어프램 모드를 합친 혼합 모드를 일정시간 간격으로 자동으로 절환하여 측정된 청진기 측정 데이터를 의료 데이터 수신부(12)에 전송하거나 의료 데이터 저장부(15)에 저장할 수 있다.In addition, the smart remote control control unit 53 controls the stethoscope 44 while using the stethoscope to control the bell mode suitable for heart sounds (20 Hz to 200 Hz), diaphragm mode suitable for lung sounds (200 Hz to 1 KHz), the above By automatically switching the mixed mode combining the bell mode and the diaphragm mode at regular time intervals, the measured stethoscope measurement data may be transmitted to the medical data receiver 12 or stored in the medical data storage unit 15 .
또한, 심전도 측정은 제1 전극과 제2 전극 간에 형성된 전압 신호를 증폭하여 측정하되, 상기 제1전극은 지문 인증이 이루어지도록 환자가 스마트 리모컨(400)의 지문 인증부(24,25,26,27)를 잡으면, 자연스럽게 엄지 손가락이 네개의 손 전극들(24a, 25a, 26a, 27a) 중 어느 하나와 접촉이 이루어져 형성되고, 상기 제2 전극은 ECG 센서(48)가 피부(예컨대 가슴)에 접촉되었을 때 형성된다. In addition, the electrocardiogram measurement is performed by amplifying the voltage signal formed between the first electrode and the second electrode, and the first electrode uses the fingerprint authentication unit 24, 25, 26, 27), the thumb naturally makes contact with any one of the four hand electrodes 24a, 25a, 26a, 27a, and the second electrode is formed by the ECG sensor 48 on the skin (eg chest). formed when in contact.
이들 서로 이격된 두개의 전극(제1 전극과 제2 전극)을 통해 사용자의 심전도(ECG) 신호를 측정할 수 있다. 즉, 사용자가 스마트 리모컨(400)을 잡고 있는 동안, 엄지 손가락이 네개의 손 전극들(24a, 25a, 26a, 27a) 중 어느 하나와 접촉되어 형성된 제1 전극, 사용자의 신체의 다른 부분(예컨대 가슴)을 ECG센서(48)에 접촉 시켜 형성된 제2 전극, 이때 상기 제1 전극과 제2 전극 간에 형성된 전압 신호를 증폭하여 사용자의 심전도(ECG) 신호를 측정할 수 있다.The user's electrocardiogram (ECG) signal may be measured through the two electrodes (the first electrode and the second electrode) spaced apart from each other. That is, while the user is holding the smart remote control 400 , the first electrode formed by contacting the thumb with any one of the four hand electrodes 24a , 25a , 26a , and 27a , another part of the user's body (eg, The second electrode formed by bringing the chest) into contact with the ECG sensor 48, in which case the voltage signal formed between the first electrode and the second electrode is amplified to measure the user's electrocardiogram (ECG) signal.
또한, 상기 ECG 센서(48)는 청진기(44)의 박막 울림판(44a) 표면을 도전성 재료로 코팅하고, 그 위에 다시 절연막 코팅하여 형성된 ECG 전극면으로 대체될 수 있으며, 이 경우 박막 울림판(44a)에 의해 심폐음 신호(PCG신호)와 심전도 신호가 동시에 수집된다. In addition, the ECG sensor 48 may be replaced with an ECG electrode surface formed by coating the surface of the thin film sounding plate 44a of the stethoscope 44 with a conductive material and coating an insulating film thereon again. In this case, the thin film sounding plate 44a The cardiopulmonary sound signal (PCG signal) and the electrocardiogram signal are simultaneously collected.
심전도 측정의 또 다른 측면은 제1 전극과 제2 전극 간에 형성된 전압 신호를 증폭하여 측정하되, 제1전극은 사용자가 스마트 리모컨(400)을 잡고 있는 동안 스마트 리모컨(400) 측면에 설치된 손 전극들(24a, 25a, 26a, 27a) 중 어느 하나와 오른손 엄지손가락 간의 접촉에 의해 형성되고, 제2 전극은 ECG센서가 구비된 터치 스크린(터치형 ECG센서)과 왼쪽 손가락 간의 접촉에 의해 형성되는 것을 특징으로 한다. 이들 두개의 전극(제1전극과 제2 전극)을 통해 사용자의 심전도(ECG) 신호를 측정된다. 예컨대 TV시청 중 스마트 리모컨(400) 사용자는 TV의 기능을 선택하기 위해, 스마트 리모컨을 한 손으로 잡은 채 반대편 손가락(예를 들면, 검지)으로 터치 스크린(401) 상의 버튼을 빈번히 터치하게 된다. Another aspect of the ECG measurement is to amplify the voltage signal formed between the first electrode and the second electrode, and the first electrode is the hand electrodes installed on the side of the smart remote control 400 while the user is holding the smart remote control 400 . (24a, 25a, 26a, 27a) is formed by contact between any one of the thumb and the right thumb, and the second electrode is formed by contact between the touch screen (touch-type ECG sensor) provided with the ECG sensor and the left finger. characterized. The user's electrocardiogram (ECG) signal is measured through these two electrodes (the first electrode and the second electrode). For example, while watching TV, the user of the smart remote control 400 frequently touches a button on the touch screen 401 with the opposite finger (eg, index finger) while holding the smart remote control in one hand to select a function of the TV.
이 경우, 엄지 손가락은 제1전극을 형성하고, 반대편 손의 검지는 제2 전극을 형성하기 때문에, 터치 스크린(401) 상의 버튼을 터치할 때마다, 상기 제1 전극 및 제2 전극을 통해 사용자의 심전도(ECG) 신호를 측정할 수 있다. In this case, since the thumb forms the first electrode and the index finger of the opposite hand forms the second electrode, whenever a button on the touch screen 401 is touched, the user passes through the first electrode and the second electrode. can measure the electrocardiogram (ECG) signal of
실시예 2는 실시예 1에 비해, 스마트 리모컨(400) 측면에 배치된 손 전극들(24a, 25a, 26a, 27a)의 표면적을 넓혀서, 사용자의 손과의 접촉면적을 극대화한 경우이다. In Example 2, compared to Example 1, the surface area of the hand electrodes 24a, 25a, 26a, and 27a disposed on the side of the smart remote control 400 is increased to maximize the contact area with the user's hand.
도 2d는 스마트 리모컨(400)의 측면도의 일실시예로, 스마트 리모컨(400)의 상하 좌우측면에 설치된 엄지손가락의 지문 인증을 수행하는 지문 인식부(24,25,26,27)를 구비하여, 누가, 언제 스마트 리모컨(400)을 잡고 사용 중인지 실시간으로 파악할 수 있다. Figure 2d is an embodiment of a side view of the smart remote control 400, provided with fingerprint recognition units 24, 25, 26, 27 for performing fingerprint authentication of the thumb installed on the top, bottom, left and right sides of the smart remote control 400, , who, and when the smart remote control 400 can be grasped in real time.
스마트 리모컨(400)의 지문 인식부(24,25,26,27)는 지문 인식 면적이 다소 적으나, 소수의 가족들간(10인 미만)의 개인 인증에는 실용상 문제가 없다. The fingerprint recognition units 24, 25, 26, and 27 of the smart remote control 400 have a rather small fingerprint recognition area, but there is no practical problem in personal authentication between a small number of families (less than 10 people).
본 실시예에서는 상하 좌우 네 곳에 엄지손가락의 지문의 인증을 위한 지문 인식 부(24,25,26,27)가 설치되어 있어, 사용자가 어떠한 방향으로 스마트 리모컨(400)을 손으로 잡아도, 네 곳 중 적어도 한곳의 지문 인식부에 엄지 손가락이 위치하게 되어, 스마트 리모컨(400)을 잡는 모양과 방향에 관계없이 지문 인증이 이루어 지도록 하였다.In this embodiment, the fingerprint recognition units 24, 25, 26, and 27 for authenticating the fingerprint of the thumb are installed in four places on the upper, lower, left, and right places, so that the user can hold the smart remote control 400 with his hand in any direction, in four places. The thumb is positioned in at least one of the fingerprint recognition units, so that fingerprint authentication is performed regardless of the shape and direction of holding the smart remote control 400 .
본 발명의 지문 인식부(24,25,26,27)는 정전용량방식, 초음파식 또는 광학식 지문 인식 센서를 사용하는 것이 선호된다.The fingerprint recognition unit 24, 25, 26, 27 of the present invention preferably uses a capacitive, ultrasonic or optical fingerprint recognition sensor.
초음파식 지문인식센서는 지문 센서 창에 손가락을 갖다 대면 지문 인식 센서에서 초음파를 발사한 뒤, 지문의 굴곡을 맞고 돌아오는 시간을 측정하여 지문 이미지를 생성한 후, 기 등록된 지문 DB와 비교 정합(매칭)하여 입력된 지문을 인증하는 방식이다.The ultrasonic fingerprint sensor generates a fingerprint image by measuring the time it takes to return after hitting the curve of the fingerprint after emitting ultrasonic waves from the fingerprint sensor when a finger is placed on the fingerprint sensor window, and then compares and matches with the previously registered fingerprint DB ( matching) to authenticate the input fingerprint.
광학식은 이미지 센서에 의해 지문 이미지를 얻어, 기 등록된 지문 DB와 비교 정합(매칭)하여 ID를 인증하는 방식이다.The optical type is a method of authenticating an ID by obtaining a fingerprint image by an image sensor, comparing and matching (matching) it with a pre-registered fingerprint DB.
본 발명의 지문 인식부는 지문 이미지를 세선화 알고리즘(thinning algorithm)에 의해 지문 이미지상의 융선(ridge) 과 골(valley)을 명확히 한다. 또한, 상기 지문 인증을 위한 비교 정합은 융선과 골로부터 특징점(minutiae) 정보들을 추출하고, 기 등록된 지문 이미지의 특징점 정보들과 비교하여 지문 이미지들 간의 유사도(매칭 점수)를 판별하는 과정을 포함한다. 상기 특징점 정보는 특징점의 종류, 특징점이 위치한 융선의 방향, 지문 이미지 내의 특징점의 위치를 포함할 수 있다.The fingerprint recognition unit of the present invention clarifies ridges and valleys on the fingerprint image by using a thinning algorithm for the fingerprint image. In addition, the comparison matching for fingerprint authentication includes a process of extracting minutiae information from ridges and valleys, and determining the similarity (matching score) between fingerprint images by comparing it with minutiae information of a previously registered fingerprint image. do. The key point information may include the type of the key point, the direction of the ridge where the key point is located, and the location of the key point in the fingerprint image.
상기 특징점의 종류는 융선의 흐름이 끓기는 단점(ending point) 내지 융선이 갈라지는 분기점(bifurcation)을 포함한다.The type of the characteristic point includes an ending point at which the flow of the ridge boils or a bifurcation at which the ridge is split.
또한, 지문 인식부(24,25,26,27)의 테두리에는 손 전극들(24a, 25a, 26a, 27a)이 구비되어 있어, 손가락의 지문 부위가 지문 인증부에 접촉되면 손 전극에도 동시에 되는 구조를 갖는다. In addition, hand electrodes 24a, 25a, 26a, and 27a are provided on the edges of the fingerprint recognition unit 24, 25, 26, and 27, so that when the fingerprint part of the finger comes into contact with the fingerprint authentication unit, the hand electrode is also simultaneously provided. have a structure
따라서, 사용자가 스마트 리모컨(400)을 손으로 잡고 있는 동안 지문 인증이 이루어지는 동시에 상기 손 전극들(24a, 25a, 26a, 27a)에 의해 사용자의 체지방 내지 ECG 신호 측정이 이루어 질 수 있다.Accordingly, while the user holds the smart remote control 400 by hand, fingerprint authentication is performed and the user's body fat or ECG signal can be measured by the hand electrodes 24a, 25a, 26a, and 27a.
또한, 스마트 리모컨(400)의 좌우 측면에는 반사형 혈관 광센서 어레이 모듈(35,36)이 설치된다. In addition, reflective blood vessel optical sensor array modules 35 and 36 are installed on the left and right sides of the smart remote control 400 .
상기 반사형 혈관 광센서 어레이 모듈(35,36)은 복수개의 정맥 센서부(54a) 및 복수개의 SpO2센서(54b)로 구성될 수 있다.The reflective blood vessel optical sensor array modules 35 and 36 may include a plurality of vein sensor units 54a and a plurality of SpO 2 sensors 54b.
*상기 정맥 센서부(54a)는 근 적외선 LED(52a) 및 광 검출기(50)로 구성되며, 사용자가 스마트 리모컨(400)의 측면을 손가락(66)으로 잡았을 때 상기 광 검출기(50)에 의해 손가락(66)의 내부에 있는 정맥(56)의 혈관 패턴에 대한 정맥 이미지를 얻어 정맥 인증에 사용한다.* The vein sensor unit 54a is composed of a near-infrared LED 52a and a photo detector 50, and when the user holds the side of the smart remote control 400 with a finger 66, A vein image for the blood vessel pattern of the vein 56 on the inside of the finger 66 is obtained and used for vein authentication.
상기 SpO2센서(54b)는 적색광 LED(51a), 적외선 LED(51b), 광 검출기(50)로 구성되며, 사용자가 스마트 리모컨(400)의 측면을 손가락(66)으로 잡았을 때 상기 광 검출기(50)에 의해 손가락의 말초 혈관 동맥(55)으로부터의 산소 포화도를 측정한다. The SpO2 sensor 54b is composed of a red light LED 51a, an infrared LED 51b, and a photodetector 50. When the user holds the side of the smart remote control 400 with a finger 66, the photodetector 50 ) to measure the oxygen saturation from the peripheral vascular artery (55) of the finger.
본 발명의 정맥 센서부(54a)는 정맥 혈관 패턴 이미지를 얻기 위해 근적외선(예컨대 850nm)을 손가락(66)에 투사시키기 위한 근적외선 LED(52a), 이후 반사된 손가락의 정맥 이미지를 획득하기 위한 광 검출기(50)로 구성된다. 본 발명의 실시예에서는 정맥 이미지를 획득하기 위한 광 검출기(50)는 이미지 센서가 선호된다.The vein sensor unit 54a of the present invention includes a near-infrared LED 52a for projecting near-infrared (eg, 850 nm) to the finger 66 to obtain a vein blood vessel pattern image, and then a photodetector for acquiring a reflected finger vein image. It consists of (50). In the embodiment of the present invention, the photodetector 50 for acquiring the vein image is preferably an image sensor.
정맥에는 헤모글로빈성분이 포함되어 있기 때문에 근적외선을 흡수하는 성질을 가지고 있어, 정맥 이미지 상에서 정맥은 피부 영역에 비해 어두운 영상을 발생시킨다.Since veins contain hemoglobin, they have the property of absorbing near-infrared rays, so veins generate a darker image compared to the skin area in the vein image.
광 검출기(50)를 통해 얻어진 정맥 이미지에 대한 노이즈 제거 및 정맥 영역의 세선화 알고리즘(thinning algorithm)을 통해 정맥 이미지로부터 정맥 혈관 경로의 분기점들을 추출하여 분기점 집합을 형성한다.A branching point set is formed by extracting branch points of the venous vascular path from the vein image through a thinning algorithm and noise removal for the vein image obtained through the photodetector 50 .
본 발명의 정맥 인증은 정맥 이미지에서 추출된 분기점 집합과 기 등록된 다른 정맥 이미지의 분기점 집합간 유사도에 의해 매칭 점수 산정하여 사용자를 인증하는 것이 선호된다.In the vein authentication of the present invention, it is preferred to authenticate the user by calculating a matching score based on the similarity between the branch point set extracted from the vein image and the branch point set of other previously registered vein images.
본 발명의 경우 동일인 판명을 위한 매칭 점수의 임계값(threshold value)은 스마트 리모컨(400)을 같이 공유하는 사용자수를 감안하여 조절될 수 있다.In the present invention, the threshold value of the matching score for identifying the same person may be adjusted in consideration of the number of users sharing the smart remote control 400 together.
본 발명의 산소 포화도 계측은 엄지손가락을 통해 지문 인증이 이루어지도록 사용자가 스마트 리모컨(400)을 잡으면, 자연스럽게 엄지를 제외한 나머지 손가락은 반사형 혈관 광센서 어레이 모듈(35,36)과 접촉이 이루어져, 스마트 리모컨(400) 사용 동안 생활 속에서 사용자의 산소포화도를 계측할 수 있다. In the oxygen saturation measurement of the present invention, when the user holds the smart remote control 400 so that fingerprint authentication is performed through the thumb, the fingers other than the thumb naturally come into contact with the reflective blood vessel optical sensor array modules 35 and 36, It is possible to measure the oxygen saturation of the user in daily life while using the smart remote control 400 .
산소포화도(SpO2)는 적색광 LED(51a)와 적외선 LED(51b)을 한 주기씩 순차적으로 번갈아 발광시켜 손가락(66)의 말초 동맥 혈관(55) 부위에 조사한 다음 반사되어 광 검출기(50)에 의해 수광되는 광의 광량 변화를 관찰하여 측정될 수 있다. The oxygen saturation (SpO2) is irradiated to the peripheral arterial blood vessel 55 of the finger 66 by sequentially emitting the red light LED 51a and the infrared LED 51b sequentially by one cycle, and then reflected and reflected by the photodetector 50 It can be measured by observing the change in the amount of light received.
본 발명의 산소 포화도는 반사형 혈관 광센서 어레이 모듈(35,36)과 접촉하는 검지(index finger), 중지(middle finger), 약지(ring finger) 손가락 중 어느 하나이상 손가락(66)을 사용하여 측정하는 것이 선호된다.The oxygen saturation level of the present invention is determined using one or more fingers 66 of an index finger, a middle finger, and a ring finger in contact with the reflective blood vessel optical sensor array modules 35 and 36. It is preferred to measure
도 2e는 손(800)으로 스마트 리모컨(400)을 쥐고 있는 사용자의 손가락들(800b,800c, 800d, 800e)의 정맥 혈관 패턴을 보여주는 정맥 이미지들 (120b, 120c, 120d, 120e)을 인식하여 정맥 인증하는 일 실시예를 보인다. 근 적외선을 사용하여 혈관을 투시한 후 반사된 정맥 이미지를 템플릿 이미지와 비교하여 매칭 점수를 산정하여 스마트 리모컨(400) 사용자의 신분을 인증할 수 있다. FIG. 2e shows vein images 120b, 120c, 120d, and 120e showing the venous blood vessel pattern of the user's fingers 800b, 800c, 800d, and 800e holding the smart remote control 400 with the hand 800. An example of vein authentication is shown. After seeing blood vessels using near-infrared rays, the user's identity of the smart remote control 400 may be authenticated by comparing the reflected vein image with the template image to calculate a matching score.
이 경우 정맥 이미지들(120b, 120c, 120d, 120e)중 매칭 점수가 가장 큰 정맥 이미지를 사용자 인증에 사용하는 것이 선호된다.In this case, it is preferred to use the vein image having the highest matching score among the vein images 120b, 120c, 120d, and 120e for user authentication.
도 3(a) 내지 도 3(b)는 정맥 인증의 또 다른 실시예로 손(800)으로 스마트 리모컨(400)을 쥐고 있는 사용자의 손바닥(800f)의 정맥 혈관 패턴으로부터 정맥 이미지(120f)를 인식하여 인증하는 실시예를 보인다. 근 적외선을 사용하여 혈관을 투시한 후 반사된 정맥 이미지를 템플릿 이미지와 비교하여 매칭 점수를 산정하여 스마트 리모컨 사용자의 신분을 인증할 수 있다. 3(a) to 3(b) show a vein image 120f from a vein blood vessel pattern of a user's palm 800f holding the smart remote control 400 with a hand 800 as another embodiment of vein authentication. An example of recognizing and authenticating is shown. After seeing blood vessels using near-infrared rays, the identity of the smart remote control user can be authenticated by comparing the reflected vein image with the template image to calculate a matching score.
스마트 리모컨(400)의 후면에 배치된 정맥 센서부(54a)는 손바닥(800f)에 근 적외선을 투사시키기 위한 근적외선 LED(52a) 및 손바닥(800f)의 정맥 이미지를 획득하기 위한 광 검출기(50)로 구성되며, 사용자가 스마트 리모컨(400)을 잡았을 때 상기 광 검출기(50)에 의해 손바닥(800f)의 내부에 있는 정맥 혈관 패턴에 대한 정맥 이미지(120f)를 얻어 정맥 인증에 사용한다.The vein sensor unit 54a disposed on the back of the smart remote control 400 includes a near-infrared LED 52a for projecting near-infrared light on the palm 800f and a photodetector 50 for acquiring a vein image of the palm 800f. is configured, and when the user holds the smart remote control 400, a vein image 120f for the vein blood vessel pattern in the inside of the palm 800f is obtained by the photodetector 50 and used for vein authentication.
이 실시예의 경우, 청진기(44)의 박막 울림판(44a)의 표면을 도전성 재료로 코팅하고 그 위에 절연막을 입힘으로써 청진기(44)가 ECG 센서(48) 역할을 겸하는 것이 선호된다. In this embodiment, it is preferred that the stethoscope 44 also serves as the ECG sensor 48 by coating the surface of the thin film sounding plate 44a of the stethoscope 44 with a conductive material and coating an insulating film thereon.
도 4는 스마트 리모컨(400) 상에 집적화된 적외선 온도센서(29)를 사용하여 환자의 체열을 측정하는 여러 실시 예를 보인다. 4 shows various embodiments of measuring a patient's body heat using the infrared temperature sensor 29 integrated on the smart remote control 400 .
스마트 리모컨(400)과 음성인식 단말기(17) 간의 근거리 무선 통신 연결에 의해, 상기 적외선 온도센서(29)에 의해 측정된 환자의 온도 정보를 음성인식 단말기(17)의 의료 데이터 수신부(12)에 무선 전송한다.By short-range wireless communication connection between the smart remote control 400 and the voice recognition terminal 17, the patient's temperature information measured by the infrared temperature sensor 29 is transferred to the medical data receiving unit 12 of the voice recognition terminal 17. transmit wirelessly.
스마트 리모컨(400) 상의 지문 인식부(24,25,26,27) 또는 정맥 센서부(54a)에 의해 온도 측정시 누가 온도를 측정하고 있는지를 근거리 무선 통신 연결을 통해 음성인식 단말기(17)에게 알려줄 수 있다.When the temperature is measured by the fingerprint recognition unit 24, 25, 26, 27 or the vein sensor unit 54a on the smart remote control 400, the voice recognition terminal 17 through a short-range wireless communication connection is notified of who is measuring the temperature. can tell you
환자는 온도 측정시 지문 인식 부(24,25,26,27)에 엄지손가락을 접촉하여 지문 인식이 원활이 잘 되도록 해야 한다.When measuring the temperature, the patient should touch the thumb to the fingerprint recognition unit 24, 25, 26, 27 so that the fingerprint recognition can be performed smoothly.
상기 적외선 온도센서(29)는 스마트 리모컨(400)의 전면에 카메라(22)와 같이 배치되는 것이 선호된다.The infrared temperature sensor 29 is preferably disposed on the front of the smart remote control 400 like the camera 22 .
또한, 적외선 온도센서(29)는 측정 대상에 접촉하지 않고 스마트 리모컨(400) 사용자로부터의 방사되는 적외선량으로 체열을 측정한다. In addition, the infrared temperature sensor 29 measures body heat with the amount of infrared radiation emitted from the user of the smart remote control 400 without contacting the measurement target.
일예로, 적외선 온도 센서는 대상자(사용자의) 이마 부위로부터 3cm 내지 5cm 거리에서 체열을 측정할 수 있다. For example, the infrared temperature sensor may measure body heat at a distance of 3 cm to 5 cm from the subject's (user's) forehead.
또한, 카메라(22)는 환자의 얼굴 영역과 이마 영역 인식하여 적외선 온도 센서의 측정 위치와의 광학 정렬에 대한 가이드를 사용자에게 제공할 수 있다. 이때 사용자의 카메라 얼굴 화면에 얼굴 테두리의 묘사하는 선(line)을 중첩하여 터치 스크린(401) 화면에 표시하는 것이 선호된다.In addition, the camera 22 may provide a guide for optical alignment with the measurement position of the infrared temperature sensor by recognizing the patient's face region and forehead region to provide the user with a guide. In this case, it is preferable to display the touch screen 401 by superimposing a line depicting the edge of the face on the user's camera face screen.
광학 정렬이 제대로 이루어진 경우, 지문 인식부(24,25,26,27) 또는 정맥 센서부(54a)에 의해 인증된 사용자 ID 정보와 측정된 온도 값이 의료 데이터 수신부(12)에 무선 전송되어지고, 이때 음성인식 단말기(17)의 스피커(28)는 온도값 내지 측정 오류발생내역(측정 영역 오류, 인증 오류)에 대해 환자에게 음성 피드백하는 것이 선호된다.When the optical alignment is properly made, the user ID information and the measured temperature value authenticated by the fingerprint recognition unit 24, 25, 26, 27 or the vein sensor unit 54a are wirelessly transmitted to the medical data receiving unit 12 and , In this case, it is preferred that the speaker 28 of the voice recognition terminal 17 provides voice feedback to the patient about the temperature value or the measurement error occurrence history (measurement area error, authentication error).
도 4의 (a)는 디지털 TV(300)에 연결되는 음성인식 단말기(17)를 구비하고, 스마트 리모컨(400) 상의 적외선 온도센서(29)에 의해 사용자의 이마 부위에서 측정된 온도 값을 블루투스로 연결된 음성인식 단말기(17)에 전송하고, 체열 진단부(19)에 의해 환자의 체열의 이상 유무를 체크하는 일실시예를 보인다. 4 (a) is provided with a voice recognition terminal 17 connected to the digital TV 300, the temperature value measured in the user's forehead by the infrared temperature sensor 29 on the smart remote control 400, Bluetooth An exemplary embodiment is shown in which data is transmitted to the voice recognition terminal 17 connected to
도 4의 (b)는 본 발명의 음성인식 단말기(17)가 디지털 TV(300) 내부에 내장 집적화되어 일체화된 경우를 보이며, 이 경우 디지털 TV(300)의 내부에는 디지털 통신 모듈(8), 의료데이터 수신부(12), 체열 진단부(19), 인공지능 신경망(16), 및 제어부(30) 가 포함될 수 있다.4 (b) shows a case in which the voice recognition terminal 17 of the present invention is integrated and integrated inside the digital TV 300. In this case, the digital communication module 8 is located inside the digital TV 300, A medical data receiving unit 12 , a body heat diagnosis unit 19 , an artificial intelligence neural network 16 , and a control unit 30 may be included.
도 4의 일 실시예에 따르면, 체열 진단부(19)는 얼굴인식부, 지문 인식부(24,25,26,27) 또는 정맥 센서부(54a)에 의해 사전 등록된 사용자 인지를 체크하여 신원 인증을 수행 후 활성화될 수 있다. According to the embodiment of FIG. 4 , the body heat diagnosis unit 19 checks whether the user is pre-registered by the face recognition unit, the fingerprint recognition unit 24, 25, 26, 27, or the vein sensor unit 54a to identify the identity. It can be activated after performing authentication.
또한, 체열 진단부(19)는 사용자의 체열이 37.5도 이상인 경우 이상 체열 의심 대상자로 상정(결정)할 수 있다. Also, when the body temperature of the user is 37.5 degrees or more, the body heat diagnosis unit 19 may assume (determine) a subject suspected of having an abnormal body heat.
도 5의 (a)와 (b)는 스마트 리모컨(400) 상에 설치된 손 전극들(24a,25a, 26a, 27a)에 의해 체지방을 측정하는 체지방 측정부(70)의 일 실시예를 보인다.5A and 5B show an embodiment of the body fat measuring unit 70 for measuring body fat by hand electrodes 24a, 25a, 26a, and 27a installed on the smart remote control 400 .
체지방 측정을 위해 손 전극들(24a, 25a, 26a, 27a)은 구동 전극과 검출 전극으로 사용된다. 예컨대 손 전극들(24a, 25a)을 구동 전극으로 사용하는 경우, 나머지 손 전극들(26a, 27a)은 검출 전극으로 사용된다. 이하 구동 전극 (24a, 25a) 과 검출 전극(26a,27a)을 사용한 체지방 측정 방법을 설명하고자 한다.For body fat measurement, the hand electrodes 24a, 25a, 26a, and 27a are used as driving electrodes and detection electrodes. For example, when the hand electrodes 24a and 25a are used as driving electrodes, the remaining hand electrodes 26a and 27a are used as detection electrodes. Hereinafter, a method of measuring body fat using the driving electrodes 24a and 25a and the detection electrodes 26a and 27a will be described.
사용자는 체지방 측정 동안 왼손의 엄지 손가락과 검지를 사용하여 상기 스마트 리모컨(400)의 왼쪽 편에 있는 구동 전극(24a, 25a)을 잡고, 동시에 오른손의 엄지손가락과 검지는 스마트 리모컨(400)의 오른쪽 편에 있는 검출 전극(26a, 27a)을 잡는다.During body fat measurement, the user holds the driving electrodes 24a and 25a on the left side of the smart remote control 400 using the thumb and index finger of the left hand, and at the same time the thumb and index finger of the right hand are on the right side of the smart remote control 400 . Hold the detection electrodes 26a and 27a on one side.
상기 구동 전극(24a, 25a)에 접한 사용자의 왼손 엄지손가락을 통해 통전되는 전류는 상기 검출 전극(26a, 27a)으로 통전시키고, 이 통전된 전류로부터 상기 사용자의 신체 내로 통전된 전류의 임피던스 값을 구한다.The current passed through the user's left thumb in contact with the driving electrodes 24a and 25a passes through the detection electrodes 26a and 27a, and the impedance value of the current passed into the user's body from the passed current save
예컨대, 체지방 측정부(70)는 피 측정자가 양손의 엄지와 검지를 이용해 구동 전극(24a, 25a)과 검출 전극(26a,27a)을 잡고 있는 동안, 미세한 전류(예컨대 약 500㎂)를 체내에 흘려 피 측정자의 몸의 전기 저항을 검출 전극(26a,27a)을 통해 측정함으로써 체지방률, 체질량지수(BMI: Body Mass Index), 근육량을 포함하는 체지방 정보를 얻고, 이를 의료 데이터 수신부(12)에 전송하는 것이 선호된다.For example, the body fat measuring unit 70 applies a minute current (eg, about 500 μA) into the body while the person to be measured uses the thumb and index finger of both hands to hold the driving electrodes 24a and 25a and the detection electrodes 26a and 27a. By measuring the electrical resistance of the person to be measured through the detection electrodes 26a and 27a, body fat information including body fat percentage, body mass index (BMI), and muscle mass is obtained, and this is transmitted to the medical data receiver 12 It is preferred to do
본 발명의 예시에 따르는 체지방 측정부(70)는 도 5의 (a)에 도시된 바와 같이, 사용자에 대한 생체 임피던스를 측정하기 위한 구동신호를 인가하는 구동전극(24a,25a), 구동전극(24a,25a)에 인가할 사인파(sine wave) 구동신호 (예컨대 10~ 50 KHz )를 생성하는 사인파 발진기(40), 인체를 통해 수신되는 신호를 검출하기 위한 검출전극(26a,27a), 검출전극(26a,27a)을 통해 수신된 신호의 실효치 전압을 검출하기 위한 RMS(Root-Mean-Square) 전압 검출기(41), RMS 전압 검출기의 출력을 디지털 신호로 변환하기 위한 AD 변환기(46) 및 음성 인식 단말기(17)와의 근거리 무선 통신 연결을 통해 상기 AD 변환기(46)에 의해 변환된 데이터를 스마트 리모컨 제어부(53)를 통해 의료 데이터 수신부(12)로 전송하는 무선 통신 연결 수단(37)으로 구성된다.The body fat measuring unit 70 according to an exemplary embodiment of the present invention includes driving electrodes 24a and 25a for applying a driving signal for measuring bio-impedance to the user, driving electrodes ( A sine wave oscillator 40 for generating a sine wave driving signal (eg, 10 to 50 KHz) to be applied to 24a, 25a), detection electrodes 26a and 27a for detecting a signal received through the human body, detection electrodes An RMS (Root-Mean-Square) voltage detector 41 for detecting the rms voltage of the signal received through 26a, 27a, an AD converter 46 for converting the output of the RMS voltage detector into a digital signal, and a voice Consists of a wireless communication connection means 37 that transmits the data converted by the AD converter 46 to the medical data receiver 12 through the smart remote control controller 53 through a short-range wireless communication connection with the recognition terminal 17 do.
또한, 생체 신호 수집부(420)는 AD 변환기(46)에 의해 변환된 체지방 정보가 사용자 인증이 완료되었는지, 그리고 유효한 생체 신호인지를 판별하고, 스마트 리모컨 제어부(53)는 사용자 인증된 유효한 생체신호 정보만을 의료 데이터 저장부(15)에 저장하고 의료 데이터 수신부(12)로 전송할 수 있다.In addition, the bio-signal collecting unit 420 determines whether the body fat information converted by the AD converter 46 has completed user authentication and whether it is a valid bio-signal, and the smart remote control controller 53 determines whether the user-authenticated valid bio-signal is valid. Only information may be stored in the medical data storage unit 15 and transmitted to the medical data receiving unit 12 .
도 5의 (b)에서 보듯이, 체지방 측정 동안 지문 인증이 이루어지도록 피 측정자가 손(800)으로 스마트 리모컨(400)을 잡으면, 자연스럽게 좌측 내지 우측 엄지손가락이 지문 인식부들(24,25,26,27)과 접촉이 이루어져 지문 인증과 함께 체지방 측정이 이루어진다.As shown in (b) of FIG. 5 , when the person to be measured holds the smart remote control 400 with his hand 800 so that fingerprint authentication is performed during body fat measurement, the left or right thumb naturally moves the fingerprint recognition units 24 , 25 , 26 . ,27) and body fat measurement is performed along with fingerprint authentication.
도 6a 내지 도 6c는 스마트 리모컨(400) 상에 설치된 비 접촉 ECG센서(48)에 의해 ECG 신호를 측정하는 일실시예를 보인다. 본 발명의 ECG센서(48)는 Capacitive coupling에 허용하는 절연막(48a)과 ECG전극면(48b)으로 구성되며, 인체 피부 표면(47)과의 Capacitive coupling에 의해 심근 세포로부터 나오는 미세한 전기변동을 보여주는 ECG 신호를 센싱한다. 인체 피부 표면(47)은 가슴 부위의 피부 표면 내지 손가락 지문 부위가 선호된다. 상기 ECG 전극면(48b)은 도전성 구리 박막이 선호되며, ECG 전극(48c)을 통해 차동 증폭기(61)에 입력된다.6A to 6C show an embodiment of measuring the ECG signal by the non-contact ECG sensor 48 installed on the smart remote control 400 . The ECG sensor 48 of the present invention is composed of an insulating film 48a and an ECG electrode surface 48b that allow for capacitive coupling, and shows minute electrical fluctuations emitted from myocardial cells by capacitive coupling with the human skin surface 47. ECG signal is sensed. The human skin surface 47 is preferably the skin surface of the chest area or the area of finger prints. The ECG electrode surface 48b is preferably a conductive copper thin film, and is input to the differential amplifier 61 through the ECG electrode 48c.
상기 ECG전극면(48b)은 절연막(48a)으로 코팅된다.The ECG electrode surface 48b is coated with an insulating film 48a.
도 6a의 (a)는 ECG센서(48)가 스마트 리모컨(400)의 후면부에 설치되어, 인체 피부 표면(예컨대 가슴부위)과의 접촉시 Capacitive coupling에 의해 ECG신호를 획득하는 실시예로, 스마트 리모컨(400)의 사용자가 지문 인증이 이루어지도록 스마트 리모컨(400)을 잡으면, 자연스럽게 엄지손가락이 네 개의 손 전극들(24a, 25a, 26a, 27a) 중 어느 하나와 접촉이 이루어져 제1전극을 형성하고, ECG 센서(48)가 가슴부위에 접촉되어 제2 전극을 형성하게 된다. 이때 상기 제1전극과 제2전극은 서로 이격된 두 개의 전극을 형성하여, 사용자의 심전도(ECG) 신호를 측정할 수 있다.6A (a) is an embodiment in which the ECG sensor 48 is installed on the rear part of the smart remote control 400 and acquires an ECG signal by capacitive coupling when it comes into contact with the human skin surface (eg, chest). When the user of the remote control 400 holds the smart remote control 400 to perform fingerprint authentication, the thumb naturally makes contact with any one of the four hand electrodes 24a, 25a, 26a, and 27a to form the first electrode. and the ECG sensor 48 is in contact with the chest region to form a second electrode. In this case, the first electrode and the second electrode may form two electrodes spaced apart from each other to measure the user's electrocardiogram (ECG) signal.
도 6a의 (b)는 ECG센서(48)가 스마트 리모컨(400)의 전면에 있는 터치 스크린(401)과 일체화되어 형성되는 터치형 ECG센서(49)의 일실시예로, 터치 스크린(401) 터치시 마다 손가락과의 Capacitive coupling에 의해 ECG신호를 획득할 수 있다.6a (b) is an embodiment of a touch-type ECG sensor 49 in which the ECG sensor 48 is integrated with the touch screen 401 on the front of the smart remote control 400, and the touch screen 401 An ECG signal can be acquired by capacitive coupling with a finger at every touch.
터치형 ECG센서(49)는 스마트 리모컨(400)의 전면(front face)에 터치스크린(401)과 ECG센서(48)를 일체화되어 배치됨으로써, 손가락 터치에 따른 접촉위치의 좌표에 대응하는 어플리케이션의 동작 제어가 가능할 뿐만 아니라, 사용자의 ECG 신호의 센싱도 동시에 이루어 진다.The touch-type ECG sensor 49 is integrated with the touch screen 401 and the ECG sensor 48 on the front face of the smart remote control 400, so that the application corresponding to the coordinates of the contact location according to the finger touch. Not only motion control is possible, but the user's ECG signal is sensed at the same time.
예컨대, 스마트 리모컨(400)의 사용자가 지문 인증이 이루어지도록 스마트 리모컨(400)을 잡으면, 네 개의 손 전극들(24a, 25a, 26a, 27a) 중 어느 하나와 엄지 손가락과 접촉이 이루어져 제1전극을 형성하고, 이때 사용자가 반대편 손가락으로 터치 스크린(401)을 터치하는 순간, 터치스크린(401) 상의 일체화되어 있는 ECG 센서(48)와도 손가락이 접촉되어 제2 전극이 형성되어, 사용자의 심전도(ECG) 신호를 측정할 수 있다.For example, when the user of the smart remote control 400 holds the smart remote control 400 so that fingerprint authentication is performed, any one of the four hand electrodes 24a, 25a, 26a, 27a and the thumb come into contact with the first electrode At this time, when the user touches the touch screen 401 with the opposite finger, the finger also comes into contact with the ECG sensor 48 integrated on the touch screen 401 to form a second electrode, and the user's electrocardiogram ( ECG) signal can be measured.
터치형 ECG센서(49)의 경우, 스마트 리모컨(400) 사용자는 스마트 리모컨(400)을 한 손으로 잡은 채 반대편 쪽의 손가락(예컨대, 검지)으로 터치 스크린상의 기능 버튼을 빈번히 터치하기 때문에, 생활 속에서 별도의 추가적인 개인 인증 절차 없이 사용자의 ECG 신호를 수집할 수 있다. In the case of the touch-type ECG sensor 49, the user of the smart remote control 400 frequently touches the function buttons on the touch screen with the finger (eg, index finger) of the opposite side while holding the smart remote control 400 with one hand. It is possible to collect the user's ECG signal without an additional personal authentication procedure in the
도 6b는 ECG 센서(48)로부터 얻은 ECG신호의 노이즈를 제거하고 증폭하여 음성인식 단말기(17)에 전달하기 위한 심전도 측정부(71)의 실시예로, 공급 전원에 혼입된 리플(ripple) 잡음을 제거하기 위한 동상제거(CMRR; Common Mode rejection Ratio) 회로를 구비한 차동 증폭기(differential amplifier, 61), 차동증폭기(61)를 거친 ECG 신호를 0.1Hz 이상의 신호를 통과시켜주는 고역통과필터(HPF, High Pass Filter, 62), 50Hz 또는 60Hz의 전원성분을 제거하기 위한 밴드 제거 필터(BRF, Band Rejection Filter, 63), 고역통과필터(62)와 밴드 제거필터(63)를 거친 ECG 신호의 출력을 증폭하여 디지털 신호로 변환하기 위한 AD변환기(64) 및 음성 인식 단말기(17)와의 근거리 무선 통신 연결을 통해 AD변환기(64)에 의해 변환된 ECG 신호를 의료 데이터 수신부(12)로 전송하는 무선 통신 연결 수단(37)으로 구성된다.6b is an embodiment of the electrocardiogram measuring unit 71 for removing and amplifying the noise of the ECG signal obtained from the ECG sensor 48 and transmitting it to the voice recognition terminal 17. Ripple noise mixed in the supply power A differential amplifier 61 having a common mode rejection ratio (CMRR) circuit for removing , High Pass Filter, 62), a Band Rejection Filter (BRF, Band Rejection Filter, 63) for removing the 50Hz or 60Hz power component, and the ECG signal output through the high pass filter 62 and the band rejection filter 63 Wireless transmitting the ECG signal converted by the AD converter 64 to the medical data receiving unit 12 through a short-range wireless communication connection with the AD converter 64 and the voice recognition terminal 17 for amplifying and converting to a digital signal and communication connection means (37).
또한, 심전도 측정 동안 제1 전극(예컨대 도면부호 24a)에 접촉한 엄지손가락(800a)에 의해 지문 인증이 이루어지거나, 스마트 리모컨(400)을 쥐고 있는 사용자의 손가락들(800b, 800c, 800d)의 정맥 혈관 패턴을 인증하는 정맥 센서부(54a)에 의해 사용자의 인증이 이루어 질 수 있다. In addition, fingerprint authentication is performed by the thumb 800a in contact with the first electrode (eg, reference numeral 24a) during the electrocardiogram measurement, or the user's fingers 800b, 800c, 800d holding the smart remote control 400 The user may be authenticated by the vein sensor unit 54a for authenticating the vein pattern.
또한, 생체 신호 수집부(420)는 AD 변환기(64)에 의해 디지털화된 ECG 신호가 사용자 인증이 완료되었는지, 그리고 유효한 ECG 신호인지를 판별하고, 스마트 리모컨 제어부(53)는 사용자 인증된 유효한 ECG 신호만을 의료 데이터 저장부(15)에 저장하고 무선 통신 연결 수단(37)을 통해 의료 데이터 수신부(12)로 전송한다.In addition, the biosignal collecting unit 420 determines whether the ECG signal digitized by the AD converter 64 is user authentication completed and a valid ECG signal, and the smart remote control controller 53 is a valid ECG signal authenticated by the user. is stored in the medical data storage unit 15 and transmitted to the medical data receiving unit 12 through the wireless communication connection means 37 .
도 6c는 ECG센서(48)와 저항막 방식(resistive) 터치 스크린(401)이 일체화된 터치형 ECG센서(49)의 일 실시예로, 디스플레이 패널(100), 디스플레이 패널(100)에 적층되는 하층 투명필름(200), 하층 투명필름(200)상에 적층되는 상층 투명필름(500), 상층 투명 필름(500) 상에 적층되는 ECG 센서(48), 상층 투명필름(500)과 하층 투명 필름(200)을 분리하는 스페이서(spacer, 미도시); 상기 하층 투명필름(200)의 윗면에 형성되어, 터치시 X축의 좌표를 알려주는 제1 저항막 패턴(200a), 상기 상층 투명필름(500)의 하단에 형성되며 제1 저항막 패턴(200a)과는 서로 수직으로 교차하도록 배열하여, 터치시 Y축의 좌표를 알려주는 제2 저항막 패턴(500a), 제1 저항막 패턴(200a)의 말단에 설치되는 X축 전극들(200b), 제2 저항막 패턴(500a)의 말단에 설치되는 Y축 전극들(500b) 및 ECG 센서(48)의 한쪽 단말에 설치되는 ECG전극(48c)을 포함할 수 있다. 6c is an embodiment of the touch-type ECG sensor 49 in which the ECG sensor 48 and the resistive touch screen 401 are integrated, the display panel 100 is laminated on the display panel 100 The lower transparent film 200, the upper transparent film 500 laminated on the lower transparent film 200, the ECG sensor 48 laminated on the upper transparent film 500, the upper transparent film 500 and the lower transparent film a spacer (not shown) separating the 200; A first resistive film pattern 200a that is formed on the upper surface of the lower transparent film 200 and informs the coordinates of the X-axis when touched, is formed at the lower end of the upper transparent film 500 and is a first resistive film pattern 200a and the second resistive layer pattern 500a indicating Y-axis coordinates when touched, the X-axis electrodes 200b installed at the ends of the first resistive layer pattern 200a, and the second Y-axis electrodes 500b installed at the ends of the resistive film pattern 500a and the ECG electrode 48c installed at one terminal of the ECG sensor 48 may be included.
ECG 센서(48)는 ECG전극면(48b) 위에 절연막(48a)을 형성함으로써 얻을 수 있으며, ECG전극면(48b)의 한쪽 말단에서 ECG전극(48c)을 형성하여 차동 증폭기(61)의 입력에 연결된다, The ECG sensor 48 can be obtained by forming an insulating film 48a on the ECG electrode surface 48b, and by forming an ECG electrode 48c at one end of the ECG electrode surface 48b, it is applied to the input of the differential amplifier 61. connected,
ECG전극면(48b)은 피부와 접촉시 capacitive coupling을 제공한다.The ECG electrode surface 48b provides capacitive coupling when in contact with the skin.
또한, 터치스크린(401)상의 손가락 터치 좌표를 산출하기 위해, 제2 저항막 패턴(500a)의 Y축 전극들(500b)에 순차적으로 스캔 펄스를 인가하는 TX 스위칭부(144a)의 스캔 타이밍을 제어하고 스캔 펄스를 발생시키는 TX스캔부(140) 및 터치된 지점의 압력에 의해 제2 저항막 패턴(500a)과 제1 저항막 패턴 (200a)이 접촉되고, 이때 X축 전극(200b)으로 전달된 스캔 펄스를 수신하기 위해 RX스위칭부(144b)의 스위칭 타이밍을 제어하는 RX 스캔부(142)를 구비하고, 스마트 리모컨 제어부(53)는 상기 스캔 펄스를 수신하는 시점에서의 TX스위칭부(144a)와 RX스위칭부(144b)간의 시간적 동기(synchronization) 관계를 계산하여 손가락 접촉 위치의 X, Y 좌표를 계산할 수 있다In addition, the scan timing of the TX switching unit 144a that sequentially applies scan pulses to the Y-axis electrodes 500b of the second resistive film pattern 500a in order to calculate the finger touch coordinates on the touch screen 401 The second resistive film pattern 500a and the first resistive film pattern 200a are in contact with the TX scan unit 140 that controls and generates a scan pulse and the pressure at the touched point, and at this time, the X-axis electrode 200b and an RX scan unit 142 that controls the switching timing of the RX switching unit 144b to receive the transmitted scan pulse, and the smart remote control unit 53 includes a TX switching unit ( 144a) and the RX switching unit 144b can calculate the time synchronization relationship to calculate the X, Y coordinates of the finger contact position.
제1저항막 패턴(200a) 및 제2 저항막 패턴(500a), ECG 전극면(48b)은 ITO((Indium tin oxide, 인듐 주석 산화물), TAO(tin antinomy oxide), TO(tin oxide), 인듐-징크-옥사이드(IZO), AZO(Al-doped ZnO), ZnO(zinc odixe), 메탈메시(metal mesh), 나노 잉크, 은나노와이어(Ag Nano Wire), 전도성고분자, 그래핀, 탄소나노튜브 또는 이들의 복합체 중 선택된 어느 하나 이상의 투명 도전성 재료에 의해 형성될 수 있다.The first resistive layer pattern 200a, the second resistive layer pattern 500a, and the ECG electrode surface 48b are formed of ITO ((Indium tin oxide), TAO (tin antinomy oxide), TO (tin oxide), Indium-zinc-oxide (IZO), AZO (Al-doped ZnO), ZnO (zinc odixe), metal mesh, nano ink, Ag Nano Wire, conductive polymer, graphene, carbon nanotube Or it may be formed of any one or more transparent conductive materials selected from these composites.
투명필름(200,500)은, 글라스(Glass), PET(Polyethylene Terephthalate), PEN(Polyethylene Naphthalate), PI(Polyimide) 또는 아크릴(Acryl) 중 적어도 어느 하나를 이용한 절연성 필름으로 형성될 수 있다. The transparent films 200 and 500 may be formed of an insulating film using at least one of glass, polyethylene terephthalate (PET), polyethylene naphthalate (PEN), polyimide (PI), and acryl.
제1 저항막 패턴(200a), 제2 저항막 패턴(500a)은 상기 절연성 투명필름 위에 투명 도전성 재료를 증착 내지 코팅하여 얻을 수 있다.The first resistive film pattern 200a and the second resistive film pattern 500a may be obtained by depositing or coating a transparent conductive material on the insulating transparent film.
도 7은 스마트 리모컨(400)의 양쪽 좌우 측면에 설치된 반사형 혈관 광센서 어레이 모듈(35,36) 내의 SpO2센서(54b)를 이용한 산소 포화도 및 PPG신호 측정부(72)의 일 실시예를 보인다.7 shows an embodiment of the oxygen saturation and PPG signal measurement unit 72 using the SpO2 sensor 54b in the reflective blood vessel optical sensor array modules 35 and 36 installed on both left and right sides of the smart remote control 400. .
혈중 산소포화도 측정은, 심장의 수축과 이완에 따른 혈액용적 변화와 산소 헤모글로빈과 환원 헤모글로빈간의 상대적 비율 변화를 이용하여 신호를 획득하는 방법으로, 적색 영역(예컨대 660nm)에서는 헤모글로빈의 광 흡수율이 산소 헤모글로빈보다 높은 반면, 적외선 영역(예컨대 940nm)에서는 산소 헤모글로빈의 광 흡수율이 헤모글로빈보다 높다는 사실을 이용한다.Blood oxygen saturation measurement is a method of acquiring a signal using the change in blood volume due to contraction and relaxation of the heart and the change in the relative ratio between oxyhemoglobin and reduced hemoglobin. On the other hand, it takes advantage of the fact that in the infrared region (eg 940 nm), the light absorption of oxyhemoglobin is higher than that of hemoglobin.
본 발명의 일 실시예에 따르면, 산소포화도(SpO2)는 손가락 끝에서 적색광 LED(51a), 적외선 LED(51b) 및 광 검출기(50)로 구성된 복수개의 SpO2센서(54b)에 의해 계측하며, 스마트 리모컨 제어부(53)에 의해 제어되는 스위칭 회로(85)에 의해 적색광 LED(51a)와 적외선 LED(51b)를 한 주기씩 순차적으로 시차를 두고 각각 손가락(800b, 800c, 800d)의 말초혈관 부위에 조사한 후, 조사된 빛이 혈액에 흐르는 혈관 부분에서 반사되는 빛의 양을 광 검출기(50)에서 감지하여 광의 세기에 따라 전류로 변화된 전기적 신호로부터 산소 헤모글로빈과 환원 헤모글로빈량을 산출하고, 이로부터 산소포화도를 계산한다.According to an embodiment of the present invention, oxygen saturation (SpO2) is measured by a plurality of SpO2 sensors 54b composed of a red light LED 51a, an infrared LED 51b and a photo detector 50 at the fingertip, and smart The red light LED 51a and the infrared LED 51b are sequentially staggered by one cycle by the switching circuit 85 controlled by the remote control controller 53, respectively, to the peripheral blood vessels of the fingers 800b, 800c, and 800d. After irradiation, the photodetector 50 detects the amount of light reflected from the blood vessel in which the irradiated light flows in the blood, and the amount of oxygenated hemoglobin and reduced hemoglobin is calculated from the electrical signal changed into a current according to the intensity of light, and from this, oxygen Calculate the saturation.
스마트 리모컨 제어부(53)는 스위칭 회로(85)의 스위치 동작에 의해 적색광 LED(51a)와 적외선 LED(51b) 각각을 설정된 시간 차이를 두고 교번으로 광을 조사하도록 제어한다.The smart remote control controller 53 controls the red light LED 51a and the infrared LED 51b to alternately irradiate light with a set time difference by a switch operation of the switching circuit 85 .
차동 증폭기(84)는 광검출기(50)로부터 출력되는 전기적 신호를 전류-전압 변환 후 증폭하며, 필터(81)에 의해 노이즈를 제거한 후 AD 변환기(82)에 의해 아날로그 신호를 디지털로 변환한다.The differential amplifier 84 amplifies the electrical signal output from the photodetector 50 after current-voltage conversion, removes noise by the filter 81 and converts the analog signal to digital by the AD converter 82 .
발광 다이오드 구동부(80)는 광 검출기(50)의 출력값을 피드백 받은 후 광 검출기(50)에서 충분한 세기의 전기적 신호가 출력되도록 적색광 LED(51a)와 적외선 LED(51b)의 구동 전류를 조절하는 회로부이다. 이는 사람의 손가락 피부마다 광의 흡수율이 다르기 때문에 손가락 피부에 가해지는 광량의 크기를 조절하기 위한 것이다. The light emitting diode driver 80 receives the feedback of the output value of the photodetector 50 and then controls the driving current of the red light LED 51a and the infrared LED 51b so that an electric signal of sufficient intensity is output from the photodetector 50 . to be. This is to adjust the amount of light applied to the skin of the finger because the absorption rate of light is different for each skin of a person's finger.
또한, 스마트 리모컨 제어부(53)는 AD 변환기(82)를 통해 읽어들인 DC 레벨을 가지고 적외선광 과 RED 광의 구동 세기를 조절하여 적색광 LED와 적외선 LED가 같은 DC 레벨을 갖도록 발광 다이오드 구동부(80)를 구동시키는 것이 선호된다.In addition, the smart remote control control unit 53 controls the driving intensity of the infrared light and the RED light with the DC level read through the AD converter 82 so that the red light LED and the infrared LED have the same DC level. It is preferred to run
또한, 상기 스마트 리모컨 제어부(53)는 상기 스위칭 회로(85)의 스위치 동작에 동기화하여, 광 검출기(50)로부터 AD 변환기(82)에 의해 디지털 신호로 변환되어 수신된 복합 수신 신호로부터 IR 광에 의한 신호 성분(IR 성분)과 RED 광에 의한 신호 성분(RED 성분)을 각각 분리한 후 스마트 리모컨 제어부(53) 내에 구비된 산소 포화도 연산 수단(미도시)에 의해 산소포화도를 연산을 수행한다. In addition, the smart remote control control unit 53 is synchronized with the switch operation of the switching circuit 85, converted into a digital signal by the AD converter 82 from the photodetector 50, and from the received composite reception signal to IR light. After separating the signal component (IR component) by the RED light and the signal component by the RED light (the RED component), the oxygen saturation calculation is performed by the oxygen saturation calculation means (not shown) provided in the smart remote control controller 53 .
산소 포화도 연산수단에서는 디지털화된 IR 성분과 RED 성분을 각각 극성이 반전 (inverting)시킨 후, 직류 성분은 제거한 교류 성분만 추출하여 사용하여 얻은 산소포화도 값을 의료 데이터 저장부(15)에 저장하는 것이 선호된다.In the oxygen saturation calculation means, after inverting the polarity of the digitized IR component and the RED component, respectively, extracting only the AC component from which the DC component is removed and using the obtained oxygen saturation value is stored in the medical data storage unit 15 Preferred.
여기서, 교류(AC) 성분은 말초혈관을 통과하는 혈액에 의해 흡수된 광량을 나타내고, 반면 직류(DC) 성분은 모세혈관이나 근육, 표피, 뼈와 같은 조직에 의해 흡수된 광량을 나타낸다.Here, the alternating current (AC) component represents the amount of light absorbed by blood passing through peripheral blood vessels, while the direct current (DC) component represents the amount of light absorbed by capillaries or tissues such as muscle, epidermis, and bone.
상기 산소 포화도 연산 수단(미도시)은, 광검출기(50)에 수광되는 신호의 세기로 옥시헤모글로빈(HbO2) 농도와 디옥시헤모글로빈(Hb) 농도를 산정하는 것이 선호되며, 이때 산소 포화도(SpO2)는 HbO2농도/(HbO2농도+Hb농도) 와 같은 수식으로 계산될 수 있다.Preferably, the oxygen saturation calculation means (not shown) calculates the oxyhemoglobin (HbO2) concentration and the deoxyhemoglobin (Hb) concentration by the intensity of the signal received by the photodetector 50, in which case the oxygen saturation (SpO2) can be calculated by the same formula as HbO2 concentration/(HbO2 concentration+Hb concentration).
여기서, 옥시헤모글로빈(HbO2) 농도는 적색광 LED가 온(On)되었을 때의 광검출기(50)에 수광되는 신호의 세기이고, 반면 디옥시헤모글로빈(Hb) 농도는, 적외선 LED가 온(On)되었을 때의 광 검출기(50)에 수광되는 신호의 세기를 사용하는 것이 선호된다. 이때 광 검출기(50)에 수광되는 신호의 세기 중 DC성분 제외시켜 AC 성분만 사용하여 옥시헤모글로빈(HbO2) 농도와 디옥시헤모글로빈(Hb) 농도를 산정하는 것이 선호된다.Here, the oxyhemoglobin (HbO2) concentration is the intensity of the signal received by the photodetector 50 when the red light LED is on, whereas the deoxyhemoglobin (Hb) concentration is when the infrared LED is on. It is preferred to use the intensity of the signal received by the photodetector 50 at the time. At this time, it is preferable to calculate the oxyhemoglobin (HbO2) concentration and the deoxyhemoglobin (Hb) concentration using only the AC component by excluding the DC component from the intensity of the signal received by the photodetector 50 .
상기 산소 포화도 연산 수단(미도시)의 또 다른 측면은 먼저 아래 수학식2와 같이 Red2IR 비율을 산출하고, 그후 Beer-Lambert law에 기반한 Loop up 테이블에 의해 Red2IR 비율값을 변환하여 산소 포화도를 구할 수 있다.Another aspect of the oxygen saturation calculation means (not shown) is to first calculate the Red2IR ratio as shown in Equation 2 below, and then convert the Red2IR ratio value by the Loop up table based on the Beer-Lambert law to obtain the oxygen saturation. have.
[수학식 2][Equation 2]
Figure PCTKR2022003938-appb-I000002
Figure PCTKR2022003938-appb-I000002
여기서, REDDC는 적색광 LED(51a) On시 광검출기(50)에 의해 검출된 적색광의 직류성분(DC)들의 평균값이고, IRDC는 적외선 LED(51b) On시 광검출기(50)에 의해 검출된 적외선의 직류성분들(DC)의 평균값이다.Here, RED DC is the average value of the direct current components (DCs) of the red light detected by the photodetector 50 when the red light LED 51a is on, and IR DC is detected by the photodetector 50 when the infrared LED 51b is turned on. It is the average value of the direct current components (DC) of the infrared rays.
또한 REDAC는 적색광 LED(51a) On시 광검출기(50)에 의해 검출된 적색광의 교류성분(AC)들의 평균값이고, IRAC는 적외선 LED(51b) On시 광검출기(50)에 의해 검출된 적외선의 교류성분(AC)들의 평균값이다.In addition, RED AC is the average value of AC components (AC) of the red light detected by the photodetector 50 when the red light LED 51a is On, and IR AC is the average value of the AC components (AC) of the red light detected by the photodetector 50 when the infrared LED 51b is On. It is the average value of the alternating current components (AC) of infrared rays.
또한 SpO2센서(54b)를 이용한 산소 포화도 및 PPG신호 측정부(72)는 PPG신호도 계측할 수 있다.In addition, the oxygen saturation and PPG signal measuring unit 72 using the SpO2 sensor 54b can also measure the PPG signal.
본 발명의 일 실시예에 따르면, 광전용적맥파(PPG) 신호는 스마트 리모컨(400)에 설치된 SpO2 센서(54b)의 적색광 LED(51a)만 연속적으로 발광시켜 손가락 부위에 조사한 한 다음 반사되어 수광되는 광의 광량 변화를 광 검출기(50)에 의해 연속 관찰하여 PPG신호를 계측할 수 있다. 이때 스마트 리모컨 제어부(53)는 적색광 LED(51a)만 연속적으로 발광시키도록 상기 스위칭 회로(85)의 스위치 동작을 제어한다.According to an embodiment of the present invention, the photoplethysmographic wave (PPG) signal continuously emits only the red light LED 51a of the SpO2 sensor 54b installed in the smart remote control 400, irradiates the finger, and then is reflected and received. The PPG signal can be measured by continuously observing the change in the amount of light by the photodetector 50 . At this time, the smart remote control controller 53 controls the switch operation of the switching circuit 85 so that only the red light LED 51a continuously emits light.
손가락 혈관으로부터 반사된 PPG 신호는 차동 증폭기(84)를 통해 전류-전압 변환 후 증폭하고, 증폭된 신호는 고역통과필터(예컨대 Cutoff 0.1Hz)와 저역통과 필터(예컨대 Cutoff 12Hz)를 거치도록 구성된다.The PPG signal reflected from the finger vein is amplified after current-voltage conversion through the differential amplifier 84, and the amplified signal is configured to pass through a high-pass filter (eg Cutoff 0.1Hz) and a low-pass filter (eg Cutoff 12Hz). .
필터(81)를 거친 PPG 신호는 AD변환기(82)에 의해 디지털 신호로 되어 생체 신호 수집부(420) 거쳐 스마트 리모컨 제어부(53)에 입력된다. 스마트 리모컨 제어부(53)에서는 디지털화된 PPG신호의 극성이 반전 (inverting)시킨 후, 직류 성분은 제거한 교류 성분만 추출하여 최종적인 PPG신호를 의료 데이터 저장부(15)에 저장한다.The PPG signal that has passed through the filter 81 is converted into a digital signal by the AD converter 82 and is input to the smart remote controller 53 through the biosignal collecting unit 420 . After inverting the polarity of the digitized PPG signal, the smart remote control controller 53 extracts only the AC component from which the DC component is removed and stores the final PPG signal in the medical data storage unit 15 .
또한, 생체 신호 수집부(420)는 상기 AD 변환기(82)에 의해 디지털화된 IR 성분 내지 RED 성분이 사용자 인증이 완료되었는지, 그리고 유효한 생체 신호인지를 판별하고, 스마트 리모컨 제어부(53)는 사용자 인증된 유효한 생체신호 정보(산소 포화도 내지 PPG신호)만을 의료 데이터 저장부(15)에 저장하고 무선 통신 연결 수단(37)을 통해 의료 데이터 수신부(12)로 전송한다.In addition, the bio-signal collection unit 420 determines whether the IR component or the RED component digitized by the AD converter 82 has completed user authentication and whether it is a valid bio-signal, and the smart remote control unit 53 performs user authentication Only the valid biosignal information (oxygen saturation or PPG signal) obtained is stored in the medical data storage unit 15 and transmitted to the medical data receiving unit 12 through the wireless communication connection means 37 .
도 8은 엄지손가락(800a)을 통해 지문 인증이 이루어지도록 사용자가 스마트 리모컨(400)을 잡으면, 자연스럽게 엄지(800a)를 제외한 나머지 손가락들(800b,800c, 800d)은 반사형 혈관 광센서 어레이 모듈(35,36)과 접촉이 이루어져, 스마트 리모컨(400) 사용 동안 생활 속에서 사용자의 산소포화도 와 PPG신호(68c)측정이 이루어지는 일 실시예를 보인다.FIG. 8 shows that when a user holds the smart remote control 400 so that fingerprint authentication is performed through the thumb 800a, the fingers 800b, 800c, and 800d except the thumb 800a naturally become reflective blood vessel optical sensor array modules. An embodiment is shown in which the user's oxygen saturation and PPG signal 68c are measured in daily life while the smart remote control 400 is used by making contact with (35,36).
이를 위해, 복수개의 SpO2센서들이 스마트 리모컨(400)의 양쪽 좌우 측면에 설치된 반사형 혈관 광센서 어레이 모듈(35,36)내에 집적화된다.To this end, a plurality of SpO2 sensors are integrated in the reflective blood vessel optical sensor array modules 35 and 36 installed on both left and right sides of the smart remote control 400 .
산소포화도(SpO2)는 적색광 LED(51a)와 적외선 LED(51b)을 한 주기씩 순차적으로 번갈아 발광시켜 손가락들(800b, 800c, 800d)의 말초 동맥 혈관(55) 부위에 조사한 다음 반사된 IR 광과 RED 광을 광 검출기(50)로 수광하여, 이들간의 성분 비로 계산하는 반면, PPG신호(68c)는 적색광 LED(51a)만을 발광시켜 손가락들(800b,800c, 800d)의 말초 동맥 혈관(55) 부위에 조사한 다음 반사되어 광 검출기(50)에 수광되는 광량의 AC성분만(68b)을 관찰하여 측정될 수 있다.For oxygen saturation (SpO2), the red light LED 51a and the infrared LED 51b are sequentially emitted alternately by one cycle, irradiated to the peripheral arterial blood vessels 55 of the fingers 800b, 800c, 800d, and then reflected IR light and RED light are received by the photodetector 50 and calculated as a component ratio therebetween, while the PPG signal 68c emits only the red light LED 51a to the peripheral arterial vessels 55 of the fingers 800b, 800c, and 800d. ) can be measured by observing only the AC component 68b of the amount of light received by the photodetector 50 after being irradiated to the area.
박동성 동맥혈관의 팽창과 수축에 따라 혈액 변화는 AC 성분으로 표시된다. 이에 비해 혈액을 제외한 나머지 요소들(즉, 피부와 조직)은 DC 성분으로 나타나므로, Blood changes due to dilatation and contraction of pulsatile arterial vessels are expressed as AC components. In contrast, elements other than blood (that is, skin and tissue) appear as DC components,
손가락 혈관으로부터 반사된 PPG 신호는 상기 AC성분(68b)과 DC성분(68a) 모두를 포함하므로 최종적인 PPG신호를 얻기 위해서는 AC성분(68b)만 추출하여 사용한다.Since the PPG signal reflected from the finger vein includes both the AC component 68b and the DC component 68a, only the AC component 68b is extracted and used to obtain a final PPG signal.
본 발명의 산소 포화도 및 PPG측정은 반사형 혈관 광센서 어레이 모듈(35,36)에 접촉하는 검지(index finger), 중지(middle finger), 약지(ring finger) 손가락 중 어느 하나 이상 손가락을 사용하여 측정하여 통계화된 수치를 사용하는 것이 선호된다.Oxygen saturation and PPG measurement of the present invention is performed using any one or more of the index finger, middle finger, and ring finger in contact with the reflective blood vessel optical sensor array modules 35 and 36. It is preferable to measure and use statistical values.
상기 통계화된 수치는 3-시그마(σ:standard deviation) 내의 측정값들의 평균값 내지 median값으로 산출하는 것이 선호된다.The statistical value is preferably calculated as an average value or a median value of measured values within 3-sigma (σ: standard deviation).
또한 SpO2센서(54b)를 이용하여 PPG신호도 계측할 수 있는데, 도 8의 실시예에서 보듯이 중지(800c)를 이용해 PPG신호를 측정하는 동안 약지(800d)를 이용해 산소포화도를 측정할 수 있다. 또한, 손가락을 바꾸어 가면서 PPG신호 와 산소 포화도 측정을 서로 주기적으로 교대로 측정할 수 있다.In addition, the PPG signal can be measured using the SpO2 sensor 54b. As shown in the embodiment of FIG. 8, while measuring the PPG signal using the middle finger 800c, oxygen saturation can be measured using the ring finger 800d. . In addition, the PPG signal and the oxygen saturation measurement can be periodically alternated with each other by changing fingers.
도 9는 스마트 리모컨(400) 상에 설치된 혈당 센서(31)에 의해 터치 스크린(401)상에 터치된 사용자의 손가락 부위로부터 글루코스 광 반사 신호를 얻는 일 실시예로, 본 발명의 실시예를 따르는 혈당 센서(31)는 터치스크린(401)상의 개구부를 통해 손가락에 근 적외선을 조사하는 복수개의 근 적외선 발광 소자(31a,31b, 31c) 및 상기 손가락 부위(801b, 801c, 801d)로부터 반사한 근 적외선을 수광하여 전기적 신호로 변환하는 광 센서(32a, 32b, 32c)를 포함할 수 있다. 광 센서(32a,32b, 32c)로부터 얻은 전기적 신호를 전류-전압 변환 후 증폭하여, 필터에 의해 노이즈를 제거한 후 AD 변환기에 의해 디지털로 변환하여, 글루코스 광 반사 신호 정보(69e)를 얻을 수 있다.9 is an embodiment of obtaining a glucose light reflection signal from a user's finger portion touched on the touch screen 401 by the blood glucose sensor 31 installed on the smart remote control 400, according to an embodiment of the present invention. The blood glucose sensor 31 includes a plurality of near-infrared light emitting devices 31a, 31b, and 31c irradiating near-infrared rays to a finger through an opening on the touch screen 401, and a muscle reflected from the finger portions 801b, 801c, and 801d. It may include optical sensors 32a, 32b, and 32c that receive infrared light and convert it into an electrical signal. The electrical signals obtained from the optical sensors 32a, 32b, and 32c are amplified after current-voltage conversion, noise is removed by a filter, and then digitally converted by an AD converter to obtain glucose light reflection signal information 69e. .
이때 광 센서(32a, 32b, 32c)에 수광되는 신호의 세기 중 DC 성분을 제외시켜 AC 성분만 사용하여 글루코스 광 반사 신호 정보(69e)로 사용하는 것이 선호된다.In this case, it is preferable to use only the AC component as the glucose light reflection signal information 69e by excluding the DC component from the intensity of the signal received by the optical sensors 32a, 32b, and 32c.
사용자의 혈중의 글루코스 농도에 따라, 근 적외선의 흡광도가 다르기 때문에 상기 손가락 부위에서 반사한 근 적외선 신호를 수집하여 얻어진 글루코스 광 반사 신호 정보(69e)를 이용하면 사용자의 혈당을 계측할 수 있다. 또한, 글루코스의 민감도와 근적외선이 침투할 수 있는 피부깊이(skin depth)는 파장에 따르기 때문에 서로 다른 근적외선 파장 영역의 발광소자를 사용하여 수집된 글루코스 광 반사 신호 정보(69e)를 이용하면 혈당 측정 시 정확도를 올릴 수 있다. Since the absorbance of near-infrared rays differs according to the user's blood glucose concentration, the user's blood sugar can be measured by using the glucose light reflection signal information 69e obtained by collecting the near-infrared signal reflected from the finger. In addition, since the sensitivity of glucose and the skin depth through which near-infrared rays can penetrate depend on the wavelength, when glucose light reflection signal information 69e collected using light emitting devices of different near-infrared wavelengths is used, when measuring blood glucose accuracy can be increased.
예컨대, 스마트 리모컨(400)의 건강관리 모드에서 터치 스크린(401)상에 나타난 건강관리 항목중 혈당 메뉴(92)를 선택하면, 터치 스크린(410)상에 가상의 손가락 패턴(34)이 표시되고, 이때 사용자는 자신의 손가락을 상기 가상의 손가락 패턴(34)에 정렬시켜 터치 스크린(401)상에 올려놓으면 혈당 센서(31)에 의해 각각의 손가락으로 부터 글루코스 광 반사 신호 정보(69e)를 수집하여 스마트 리모컨 제어부(53)에 제공하게 된다.For example, when the blood sugar menu 92 is selected among the health care items displayed on the touch screen 401 in the health management mode of the smart remote control 400, a virtual finger pattern 34 is displayed on the touch screen 410 and At this time, when the user aligns his/her finger with the virtual finger pattern 34 and puts it on the touch screen 401, the glucose sensor 31 collects glucose light reflection signal information 69e from each finger. Thus, it is provided to the smart remote control controller 53 .
상기 근 적외선 발광소자(31a,31b,31c)는 상기 가상의 손가락 패턴(34)상의 각각의 손가락과 광학 정렬이 되도록 배치된다.The near-infrared light emitting devices 31a, 31b, and 31c are arranged to be optically aligned with each finger on the virtual finger pattern 34 .
상기 터치스크린(401)의 개구부는 근 적외선 발광소자(31a,31b,31c)의 광축에 대해 터치스크린(401)의 커버 유리층으로서 형성될 수 있다.The opening of the touch screen 401 may be formed as a cover glass layer of the touch screen 401 with respect to the optical axis of the near-infrared light emitting devices 31a, 31b, and 31c.
혈당센서(31)에 의해 글루코스 광 반사 신호 정보(69e)를 얻는 동안, 스마트 리모컨(400)을 쥐고 있는 사용자의 손가락들(800b,800c,800d)의 정맥 혈관 패턴을 인증하는 정맥 센서부(54a)에 의해 사용자의 인증이 이루어 질 수 있다.While obtaining the glucose light reflection signal information 69e by the blood glucose sensor 31, the vein sensor unit 54a authenticates the venous blood vessel pattern of the user's fingers 800b, 800c, and 800d holding the smart remote control 400. ), the user can be authenticated.
또한 혈당센서(31)에 의해 글루코스 광 반사 신호 정보(69e)를 얻는 동안, 스마트 리모컨(400)을 쥐고 있는 사용자의 엄지손가락(800a)을 통해 지문 인증이 이루어질 수 있다.Also, while the glucose light reflection signal information 69e is obtained by the blood glucose sensor 31 , fingerprint authentication may be performed through the user's thumb 800a holding the smart remote control 400 .
본 발명의 글루코스 광 반사 신호 정보(69e)는 검지(801b), 중지(801c), 약지(801d)를 포함하는 세 손가락에 근 적외선을 조사하고, 상기 세 손가락 부위에서 반사된 근 적외선을 광 센서(32a,32b,32c)로 수광하여 얻어 질 수 있다.The glucose light reflection signal information 69e of the present invention irradiates near infrared rays to three fingers including the index finger 801b, middle finger 801c, and ring finger 801d, and collects the near infrared rays reflected from the three fingers by an optical sensor. It can be obtained by receiving light with (32a, 32b, 32c).
상기 근 적외선 발광소자는 900nm 내지 2000nm 사이의 파장을 갖는 레이저 내지 LED가 선호되며, 940nm, 1320nm, 1550nm 파장영역을 갖는 것이 더욱 선호된다.The near-infrared light emitting device preferably has a laser or LED having a wavelength between 900 nm and 2000 nm, and more preferably has a wavelength range of 940 nm, 1320 nm, and 1550 nm.
예컨대, 도면부호 31a는 920nm 내지 960 nm 파장영역의 근적외선을 방출하는 LED이고, 검지 말단(801b)에서 반사된 근적외선은 광센서(32a)에 의해 센싱될 수 있다.For example, reference numeral 31a denotes an LED that emits near-infrared rays in a wavelength range of 920 nm to 960 nm, and the near-infrared rays reflected from the index finger end 801b may be sensed by the photosensor 32a.
도면부호 31b는 1300nm~1340 nm 파장영역의 근적외선을 방출하는 LED이고, 중지 말단(801c)에서 반사된 근적외선은 광센서(32b)에 의해 센싱될 수 있다. Reference numeral 31b denotes an LED that emits near-infrared rays in a wavelength range of 1300 nm to 1340 nm, and the near-infrared rays reflected from the middle end 801c may be sensed by the photosensor 32b.
도면부호 31c는 1530nm~1570 nm 파장영역의 근적외선을 방출하는 LED이고, 약지 말단(801d)에서 반사된 근적외선은 광센서(32c)에 의해 센싱될 수 있다. Reference numeral 31c denotes an LED that emits near-infrared rays in a wavelength region of 1530 nm to 1570 nm, and the near-infrared rays reflected from the ring finger end 801d may be sensed by the photosensor 32c.
도 10는 스마트 리모컨(400)의 후면부에 설치된 청진기(44)를 이용한 청진음 측정부(73)의 일 실시예로 심장 및 폐와 같은 신체의 내부에서 나오는 청진기 신호, 즉 PCG (PhonoCardioGram) 신호를 계측한다. 10 is an embodiment of the stethoscope sound measurement unit 73 using the stethoscope 44 installed on the rear part of the smart remote control 400. Stethoscope signals coming from the inside of the body such as the heart and lungs, that is, PCG (PhonoCardioGram) signals. measure
청진음 측정부(73)는 청진기 입력부(170)로 부터의 미세한 물리적 음파 진동을 전기신호로 바꾸는 트랜스듀서(transducer, 171), 상기 트랜스듀서(171)로부터 얻어진 전기 신호를 증폭한 PCG신호를 얻기 위한 신호 증폭부(172); 상기 PCG 신호에 포함된 노이즈 및 전원잡음 제거를 위한 노이즈 제거 필터(173); 주파수 대역별 PCG 신호를 추출하기 위한 적어도 한 개 이상의 주파수 필터(174); 상기 추출된 PCG 신호를 디지털 신호를 바꾸는 AD 변환기(176)를 포함한다.Auscultation sound measurement unit 73 is a transducer (transducer, 171) that converts the minute physical sound wave vibration from the stethoscope input unit 170 into an electrical signal, and obtains a PCG signal amplified by the electrical signal obtained from the transducer 171 a signal amplifying unit 172 for; a noise removal filter 173 for removing noise and power noise included in the PCG signal; at least one or more frequency filters 174 for extracting PCG signals for each frequency band; and an AD converter 176 for converting the extracted PCG signal into a digital signal.
또한 생체 신호 수집부(420)는 상기 AD 변환기(176)에 의해 디지털화된 PCG 신호가 사용자 인증이 완료되었는지, 그리고 유효한 PCG 신호인지를 판별하고, 스마트 리모컨 제어부(53)는 사용자 인증된 유효한 PCG신호만을 의료 데이터 저장부(15)에 저장하고 무선 통신 연결 수단(37)을 통해 의료 데이터 수신부(12)로 전송한다.In addition, the biosignal collection unit 420 determines whether the PCG signal digitized by the AD converter 176 is user authentication completed and whether it is a valid PCG signal, and the smart remote control control unit 53 is a valid PCG signal authenticated by the user. is stored in the medical data storage unit 15 and transmitted to the medical data receiving unit 12 through the wireless communication connection means 37 .
상기 주파수 필터(174)는 심방, 심실, 폐, 혼합 모드(심방, 심실, 폐를 모두 포함하는 대역)에 맞는 주파수 대역을 갖는 필터를 사용할 수 있다.The frequency filter 174 may use a filter having a frequency band suitable for the atrium, ventricle, lung, and mixed mode (band including all of the atrium, ventricle, and lung).
상기 주파수 필터(174)의 또 다른 측면은 심음(heart sound)의 검출을 위한 벨 (bell) 모드에서는 30Hz 내지 100Hz의 주파수 대역을 갖는 필터를 사용하고, 폐음(lung sound) 의 검출을 위한 다이어프램(diaphragm) 모드에서는 100Hz 내지 500Hz의 주파수 대역을 갖는 주파수 필터(174)를 사용할 수 있다.Another aspect of the frequency filter 174 is to use a filter having a frequency band of 30 Hz to 100 Hz in a bell mode for detecting a heart sound, and a diaphragm for detecting a lung sound ( diaphragm) mode, a frequency filter 174 having a frequency band of 100 Hz to 500 Hz may be used.
상기 청진기 입력부(170)는 박막 울림판(44a)으로 구성되며, 박막 울림판(44a)의 울림에 의한 내부 공기압의 변화로부터 소리를 집음하고, 상기 트랜스듀서(171)는 청진기 입력부(170)으로부터 발생한 미세 음파 진동에 의한 박막 울림판(44a)의 변위를 전기신호로 변환하기 위한 것으로 피에조 필름(piezo 필름) 또는 마이크로폰 일수 있다.The stethoscope input unit 170 is composed of a thin-film sounding plate 44a, and collects sound from changes in internal air pressure caused by the sounding of the thin-film sounding plate 44a, and the transducer 171 is a microscopic sound generated from the stethoscope input unit 170. It is for converting the displacement of the thin film sounding plate 44a due to sound wave vibration into an electric signal, and may be a piezo film or a microphone.
또한 상기 박막 울림판(44a)의 표면을 도전성 재료로 코팅하고 그 위에 절역막을 입힘으로써 ECG 센서의 역할을 겸할 수 있다. 이 경우 피부 표면 (예컨대 가슴부위)에서 청진기 신호를 얻는 동안 ECG신호를 동시에 얻을 수 있다.In addition, by coating the surface of the thin-film sounding plate 44a with a conductive material and applying a cut-off film thereon, it can serve as an ECG sensor. In this case, the ECG signal can be simultaneously obtained while the stethoscope signal is obtained from the skin surface (eg, the chest area).
예컨대, 청진기(44)를 사용한 PCG 신호와 ECG 신호의 동시 측정은 지문 인증이 이루어 지도록 환자가 스마트 리모컨(400)을 잡고 가슴부위의 피부에 접촉하면, 자연스럽게 엄지 손가락(800a)이 네 개의 손 전극들(24a,25a,26a,27a)중 어느 하나와 접촉이 이루어져 제 1전극을 형성하고, 이때 박막 울림판(44a)의 표면이 가슴 피부에 접촉되면 제2 전극이 형성되어 이들 두개의 전극(제 1전극과 제 2전극)을 통해 사용자의 심전도(ECG) 신호와, 동시에 박막 울림판(44a)으로 부터의 PCG 신호를 측정할 수 있다.For example, the simultaneous measurement of the PCG signal and the ECG signal using the stethoscope 44 is performed when the patient holds the smart remote control 400 and touches the skin of the chest region so that fingerprint authentication is performed, the thumb 800a naturally moves the four hand electrodes A first electrode is formed by making contact with any one of the electrodes 24a, 25a, 26a, and 27a. At this time, when the surface of the thin-film sounding plate 44a comes into contact with the skin of the chest, a second electrode is formed and these two electrodes (the first electrode) are formed. The user's electrocardiogram (ECG) signal and the PCG signal from the thin film sounding plate 44a can be simultaneously measured through the first electrode and the second electrode).
도 11a내지 도 11d는 스마트 리모컨(400)상에 설치된 바이오 센서로부터 얻어진 생체 신호를 이용하여 혈압, 콜레스테롤, 혈당을 측정하기 위한 일 실시예로, 인공지능 신경망(16)에 의해 혈압을 측정하는 혈압 측정부, 콜레스테롤를 측정하는 콜레스테롤 측정부, 혈당을 측정하는 혈당 측정부를 구현한 실시예이다.11A to 11D show an embodiment for measuring blood pressure, cholesterol, and blood sugar by using a bio-signal obtained from a bio-sensor installed on the smart remote controller 400. Blood pressure for measuring blood pressure by the artificial intelligence neural network 16 This is an embodiment in which a measuring unit, a cholesterol measuring unit measuring cholesterol, and a blood glucose measuring unit measuring blood sugar are implemented.
혈압 측정부(미도시)는 다양한 혈압 수치에 따라 표지된(labeled) 생체신호 및 개인 신체정보(69f)를 이용하여 학습된 혈압 측정용 인공지능 신경망(20a)에 의해 피 측정자의 혈압을 예측하는 것이 선호된다. The blood pressure measuring unit (not shown) predicts the blood pressure of the person to be measured by the artificial intelligence neural network 20a for blood pressure measurement learned using bio-signals and personal body information 69f labeled according to various blood pressure values. it is preferred
본 발명의 생체 신호는 ECG신호(69a), PPG신호(68c), PCG신호(69b), 산소포화도(69c), PTT(69d), 글루코스 광 반사신호 정보(69e)를 포함할 수 있다.The biosignal of the present invention may include ECG signal 69a, PPG signal 68c, PCG signal 69b, oxygen saturation 69c, PTT 69d, and glucose light reflection signal information 69e.
또한, 콜레스테롤 측정부(미도시)는 다양한 콜레스테롤 수치에 따라 표지된(labeled) 생체 신호 및 개인 신체정보(69f)를 이용하여 학습된 콜레스테롤 측정용 인공지능 신경망(20b)에 의해 피 측정자의 콜레스테롤을 예측하는 것이 선호된다. In addition, the cholesterol measurement unit (not shown) measures the cholesterol of the person to be measured by the artificial intelligence neural network 20b for cholesterol measurement learned using biosignals and personal body information 69f labeled according to various cholesterol levels. Prediction is preferred.
상기 콜레스테롤 측정부(미도시)의 또 다른 측면은 중성지방(Triglyceride: TG), HDL-C(high-density lipoprotein-cholesterol), LDL-C(low-density lipoprotein-L cholesterol)값 중 선택된 어느 하나 이상의 값에 따라 표지된 생체 신호 및 개인 신체정보(69f)를 이용하여 학습된 콜레스테롤 측정용 인공지능 신경망(20b)에 의해 피 측정자의 콜레스테롤을 예측하는 것이 선호된다.Another aspect of the cholesterol measurement unit (not shown) is any one selected from triglyceride (TG), high-density lipoprotein-cholesterol (HDL-C), and low-density lipoprotein-L cholesterol (LDL-C) values. It is preferred to predict the cholesterol of the subject by the artificial intelligence neural network 20b for cholesterol measurement, which is learned using the biosignals and personal body information 69f labeled according to the above values.
또한, 혈당 측정부(미도시)는 다양한 혈당 수치에 따라 표지된(labeled) 생체 신호 및 개인 신체정보를 이용하여 학습된 혈당 측정용 인공지능 신경망(20c)에 의해 피 측정자의 혈당을 예측하는 것이 선호된다.In addition, the blood sugar measurement unit (not shown) predicts the blood sugar of the person to be measured by the artificial intelligence neural network 20c for blood sugar measurement learned using bio signals and personal body information labeled according to various blood sugar levels. Preferred.
본 발명의 인공지능 신경망(16)은 혈압 측정용 인공지능 신경망(20a), 콜레스테롤 측정용 인공지능 신경망(20b), 혈당 측정용 인공지능 신경망(20c)을 포함할 수 있다.The artificial intelligence neural network 16 of the present invention may include an artificial intelligence neural network 20a for blood pressure measurement, an artificial intelligence neural network 20b for cholesterol measurement, and an artificial intelligence neural network 20c for blood sugar measurement.
상기 인공지능 신경망(16)은 ECG 신호(69a)의 두 R지점을 기준으로 동기화된 PPG신호(68c), PCG신호(69b), 산소포화도(69c), PTT(69d), 글루코스 광 반사신호 정보(69e)를 사용하는 것이 선호된다, 즉, R-to-R 구간 마다 발생하는 PPG신호(68c), PCG신호(69b), 산소포화도(69c), PTT(69d), 글루코스 광 반사신호 정보(69e)를 상기 인공지능 신경망(20a,20b,20c)의 입력신호로서 사용하는 것이 선호된다. 상기 동기화된 생체신호는 동일한 시점에서 사용자의 몸으로부터 측정된 생체 신호이기 때문에 동기화가 안된 생체신호(서로 다른 시점에서 획득된 생체 신호)를 인공지능 신경망(16)의 입력신호로 사용하는 것 보다 정확한 혈압 측정, 콜레스테롤 측정, 혈당 측정이 가능하다. 환자의 생체신호가 환자의 바디 컨디션과 시간에 따라 수시로 변하기 때문에 동일한 시점에서 수집된 생체신호를 인공지능 신경망(16)의 입력신호로 사용해야만 정확한 진단 결과를 얻을 수 있다.The artificial intelligence neural network 16 synchronizes the PPG signal 68c, the PCG signal 69b, the oxygen saturation 69c, the PTT 69d, and the glucose light reflection signal information based on the two R points of the ECG signal 69a. It is preferred to use (69e), that is, PPG signal (68c), PCG signal (69b), oxygen saturation (69c), PTT (69d), glucose light reflection signal information ( 69e) is preferably used as the input signal of the artificial intelligence neural networks 20a, 20b, and 20c. Since the synchronized bio-signals are bio-signals measured from the user's body at the same time point, it is more accurate than using unsynchronized bio-signals (biometric signals acquired at different time points) as the input signal of the artificial intelligence neural network 16 . Blood pressure measurement, cholesterol measurement, and blood sugar measurement are possible. Since the patient's bio-signals change frequently according to the patient's body condition and time, accurate diagnosis results can be obtained only by using the bio-signals collected at the same time as the input signals of the artificial intelligence neural network 16 .
상기 인공지능 신경망(16)의 다른 측면은 상기 PPG신호(68c)의 피크점을 기준으로 동기화된 PCG신호(69b), 산소포화도(69c), 글루코스 광 반사신호 정보(69e)를 사용하는 것이 선호된다.Another aspect of the artificial intelligence neural network 16 is to use the PCG signal 69b, oxygen saturation 69c, and glucose light reflection signal information 69e synchronized based on the peak point of the PPG signal 68c. do.
예컨대, 사용자로부터 ECG 신호(69a), PPG 신호(68C), 산소 포화도(69c), PCG신호(69b), 글루코스 광 반사신호 정보(69e)를 측정하고 이러한 생체 신호 정보와 개인 신체 정보(69f)를 기 학습된 혈압 측정용 인공지능 신경망(20a) 적용하여 사용자의 혈압을 실시간으로 추정할 수 있도록 구성된다.For example, ECG signal 69a, PPG signal 68C, oxygen saturation level 69c, PCG signal 69b, and glucose light reflection signal information 69e are measured from the user, and such biosignal information and personal body information 69f It is configured to estimate the user's blood pressure in real time by applying the previously learned artificial intelligence neural network 20a for blood pressure measurement.
도11b 내지 도11d는 ECG신호(69a), PPG신호(68c), PCG신호(69b)로부터 추출된 특징 벡터, SpO2(69c), PTT(69d), 글루코스 광 반사 정보(69e) 및 개인 신체 정보(69f)를 이용하여 분류기(Classifier)(21a)에 의해 혈압, 콜레스테롤, 혈당을 산정하는 인공지능 신경망(16)의 여러 실시예이다. 11B to 11D show feature vectors extracted from ECG signal 69a, PPG signal 68c, PCG signal 69b, SpO2 69c, PTT 69d, glucose light reflection information 69e, and personal body information. There are several embodiments of the artificial intelligence neural network 16 that calculates blood pressure, cholesterol, and blood sugar by the classifier 21a using 69f.
예컨대, 도 11b 와 도 11c에 있어서는, 특징 벡터 추출부1(33a)은 ECG신호(69a) 로부터 특징 벡터를 뽑아 내고, 특징 벡터 추출부2(33b)은 PPG신호(68c) 로부터 특징 벡터를 뽑아 내고, 특징 벡터 추출부3(33c)은 PCG신호(69b) 로부터 특징 벡터를 뽑아 낸다. For example, in FIGS. 11B and 11C, the feature vector extraction unit 1 (33a) extracts a feature vector from the ECG signal 69a, and the feature vector extraction unit 2 (33b) extracts the feature vector from the PPG signal (68c). and the feature vector extraction unit 3 (33c) extracts the feature vector from the PCG signal (69b).
또한, 선택 사항으로 상기 특징 벡터 추출부3(33c)는 심장 소리에 해당하는 벨 모드 신호와 폐 소리에 해당하는 다이어프램(Diaphram) 모드 신호로 분리된 PCG신호(69b)로 부터, 벨모드 신호의 특징 벡터와 다이어프램 모드 신호의 특징 벡터를 뽑아 낼 수 있다. Also, optionally, the feature vector extractor 3 (33c) is configured to extract the bell mode signal from the PCG signal 69b separated into a bell mode signal corresponding to a heart sound and a diaphram mode signal corresponding to a lung sound. It is possible to extract the feature vector and the feature vector of the diaphragm mode signal.
상기 ECG신호(69a), PPG신호(68c), PCG신호(69b)의 특징 벡터는 딥런닝 신경회로망(Deep learning Neural Network) 내지 주파수 영역의 스펙트럼 분석에 의해 추출 될 수 있다. The feature vectors of the ECG signal 69a, the PPG signal 68c, and the PCG signal 69b may be extracted by a deep learning neural network or spectrum analysis in the frequency domain.
예컨대 상기 딥런닝 신경회로망(Deep learning Neural Network)은 1-D CNN(Convolutional Neural Network)이 될 수 있다.For example, the deep learning neural network may be a 1-D convolutional neural network (CNN).
또한 상기 주파수 영역의 스펙트럼 분석에 의한 특징 벡터 추출은 , Linear Prediction Codes(LPC), Mel-Coefficient, Mel-Frequency Cepstral Coefficient(MFCC), Cepstral Coefficient, Preceptual Linear Prediction(PLP) 또는 Bark Frequency Cepstral Coefficients (BFCC)에 의해 이루어 질 수 있다.In addition, the feature vector extraction by spectrum analysis of the frequency domain is , Linear Prediction Codes (LPC), Mel-Coefficient, Mel-Frequency Cepstral Coefficient (MFCC), Cepstral Coefficient, Preceptual Linear Prediction (PLP) or Bark Frequency Cepstral Coefficients (BFCC) ) can be done by
음성인식 단말기(17)는 디지털 통신 모듈(8)을 통해 생체 신호 와 개인 신체 정보(699f)를 서버(13)에 제공하고, 이후 서버(13)상의 인공지능 신경망(16)으로부터 혈압, 콜레스테롤, 혈당에 대한 진단결과를 피드백 받는다. The voice recognition terminal 17 provides bio-signals and personal body information 699f to the server 13 through the digital communication module 8, and thereafter, blood pressure, cholesterol, Receive feedback on diagnostic results for blood sugar.
도 11b에 예시된 인공지능 신경망(16)에서는 특징 벡터 추출부(33a,33b,33c)에 의해 추출된 특징 벡터들이 SpO2(69c), PTT(69d), 글루코스 광 반사 정보(69e) 및 개인 신체 정보(69f)를 함께 분류기(21a) 에 입력되어 혈압, 콜레스테롤, 혈당을 산정하게 된다.In the artificial intelligence neural network 16 illustrated in FIG. 11B , the feature vectors extracted by the feature vector extraction units 33a, 33b, and 33c are SpO2 (69c), PTT (69d), glucose light reflection information 69e, and personal body The information 69f is input together into the classifier 21a to calculate blood pressure, cholesterol, and blood sugar.
또한 도 11b의 실시예에서 도면부호 21a는 LSTM으로 대체하여 사용 할 수 있다. 이 경우 특징 벡터 추출부(33a,33b,33c,)에 의해 추출된 특징 벡터들의 시계열 정보를 분석하여 혈압, 콜레스테롤, 혈당을 산정할 수 있다.Also, in the embodiment of FIG. 11B , reference numeral 21a may be used instead of LSTM. In this case, blood pressure, cholesterol, and blood sugar may be calculated by analyzing time series information of the feature vectors extracted by the feature vector extraction units 33a, 33b, and 33c.
도11c에 예시된 인공지능 신경망(16)에서는 도 11b의 분류기의 출력에 LSTM(long Short term Memory) 또는 GRU(Gated Recurrent Unit) (21b)를 연결하여 사용한 실시예로, In the artificial intelligence neural network 16 illustrated in FIG. 11c , it is an embodiment used by connecting a long short term memory (LSTM) or a gated recurrent unit (GRU) 21b to the output of the classifier of FIG. 11b,
분류기(21a) 으로 부터 출력되는 시계열(time sequence) 정보를 LSTM 또는 GRU(21b)가 분석하여 혈압, 콜레스테롤, 혈당을 산정하게 된다.The LSTM or GRU 21b analyzes the time sequence information output from the classifier 21a to calculate blood pressure, cholesterol, and blood sugar.
이 경우 혈당 측정부는 2개월 내지 3개월 동안의 혈당의 평균치를 평가하는 HbA1c(당화 혈색소)를 산출할 수 있다.In this case, the blood glucose measurement unit may calculate HbA1c (glycated hemoglobin) for evaluating the average value of blood glucose for 2 to 3 months.
PPT(69d)는 ECG 피크(R 지점)와 PPG신호의 피크 지점 간의 시간차 내지 ECG 피크와 SDPPG(Second Derivative Photoplethysmographic)의 피크간의 시간차로서 정의될 수 있다.The PPT (69d) may be defined as a time difference between an ECG peak (point R) and a peak point of the PPG signal, or a time difference between an ECG peak and a peak of Second Derivative Photoplethysmographic (SDPPG).
ECG 신호(69a)상의 R 피크 시점에서 심실이 수축하면서 동맥으로 혈액을 밀어 내기 시작하므로, PPT측정의 시작점 기준을 R피크로 사용하고, PPG신호의 피크 내지 SDPPG의 피크를 PTT 측정의 종료 시점으로 삼을 수 있다.At the R peak point on the ECG signal (69a), the ventricles begin to push blood into the artery as the ventricles contract, so the starting point of the PPT measurement is used as the R peak, and the peak of the PPG signal to the peak of the SDPPG are used as the end point of the PTT measurement. can take
본 발명의 개인 신체 정보(69f)는 체열, 성(sex), 나이, 키(body height), 몸무게, 표준 혈압치, 체지방 정보, HRV(heart Rate Variability) 중 적어도 어느 하나 이상을 선택한 정보인 것을 특징으로 한다. The personal body information 69f of the present invention is information that selects at least one or more of body heat, sex, age, height, weight, standard blood pressure, body fat information, and heart rate variability (HRV). do it with
상기 체지방 정보는 스마트 리모컨(400)의 체지방 측정부(70)에 의해 얻어진 최근의 사용자의 체지방 정보인 것이 선호된다.It is preferable that the body fat information is the latest user's body fat information obtained by the body fat measuring unit 70 of the smart remote control 400 .
상기 글루코스 광 반사 정보(69e)는 최근에 측정된 사용자의 글루코스 광 반사 정보인 것이 선호된다.The glucose light reflection information 69e is preferably the recently measured glucose light reflection information of the user.
사용자 한쪽손의 엄지 손가락(800a)을 스마트 리모컨(400)의 측면에 형성된 손 전극(도면부호 24a,25a,26a,27a 중 어느 하나)에 접촉시킨 상태에서 자신의 신체의 다른 부분(예컨대 가슴부위)을 ECG센서(48)에 접촉시키게 되면, 지문 인증과 동시에 사용자의 ECG신호(69a) 와 PCG신호(69b)를 동시에 측정될 수 있다.In a state in which the thumb 800a of one hand of the user is in contact with the hand electrode (any one of 24a, 25a, 26a, 27a) formed on the side of the smart remote control 400 , another part of the user's body (eg, the chest area) ) in contact with the ECG sensor 48, the user's ECG signal 69a and PCG signal 69b can be simultaneously measured at the same time as fingerprint authentication.
또한, 나머지 손가락(800b, 800c, 800d)으로부터 반사형 혈관 광센서 어레이 모듈(35,36)에 의해 PPG신호(68c), 산소포화도(69c), PTT(69d)를 계측할 수 있다. In addition, the PPG signal 68c, the oxygen saturation level 69c, and the PTT 69d can be measured from the remaining fingers 800b, 800c, and 800d by the reflective blood vessel optical sensor array modules 35 and 36 .
본 발명의 표준 혈압치는 통계적으로 알려진 정상인의 혈압으로 다음 수학식 3과 같이 계산될 수 있다.The standard blood pressure value of the present invention is the blood pressure of a normal person known statistically and can be calculated as in Equation 3 below.
[수학식3][Equation 3]
Figure PCTKR2022003938-appb-I000003
Figure PCTKR2022003938-appb-I000003
PTT(69d)와 키(height) 정보를 가지고, 상기 (2) 식으로부터 표준 혈압치를 계산하기 위해 d=0.6×height, h=0.3×height 를 사용하는 것이 선호된다.With the PTT (69d) and height information, it is preferable to use d=0.6×height and h=0.3×height to calculate the standard blood pressure value from Equation (2) above.
또한, 상기 계수들 σ와 β는 통계 정보에서 의해 설정될 수 있는 파라미터들이다.Also, the coefficients σ and β are parameters that can be set by statistical information.
도 11d은 혈압, 콜레스테롤, 혈당을 산정하기 위한 인공지능 신경망(16)의 또 다른 실시예로, Linear Prediction Codes(LPC), Mel-Coefficient, Mel-Frequency Cepstral Coefficient(MFCC), Cepstral Coefficient, Preceptual Linear Prediction(PLP), STFT(Short Time Fourier Transform) 또는 Bark Frequency Cepstral Coefficients (BFCC)에 의해 산출되는 spectrogram을 CNN(57)의 입력으로 사용할 수 있다.Figure 11d is another embodiment of the artificial intelligence neural network 16 for calculating blood pressure, cholesterol, blood sugar, Linear Prediction Codes (LPC), Mel-Coefficient, Mel-Frequency Cepstral Coefficient (MFCC), Cepstral Coefficient, Preceptual Linear A spectrogram calculated by Prediction (PLP), Short Time Fourier Transform (STFT), or Bark Frequency Cepstral Coefficients (BFCC) may be used as an input of the CNN 57 .
특징 벡터 추출부4(33d)은 ECG신호(69a)로부터 ECG신호의 spectrogram을 출력하고, The feature vector extraction unit 4 (33d) outputs a spectrogram of the ECG signal from the ECG signal (69a),
특징 벡터 추출부5(33e)는 PPG신호(68c)로부터 PPG신호의 spectrogram를 출력하고, 특징 벡터 추출부6(33f)은 PCG신호(69b)로부터 PCG신호의 spectrogram를 출력한다. The feature vector extraction unit 5 (33e) outputs a spectrogram of the PPG signal from the PPG signal 68c, and the feature vector extraction unit 6 (33f) outputs a spectrogram of the PCG signal from the PCG signal 69b.
또한, 선택 사항으로 상기 특징 벡터 추출부 6(33f)은 심장 소리에 해당하는 벨 모드 신호와 폐 소리에 해당하는 다이어프램(Diaphram) 모드 신호로 분리된 PCG신호(69b)로 부터, 벨모드 신호의 spectrogram과 다이어프램 모드 신호의 spectrogram를 뽑아 낼 수 있다.Also, optionally, the feature vector extractor 6 (33f) is configured to extract the bell mode signal from the PCG signal 69b separated into a bell mode signal corresponding to a heart sound and a diaphram mode signal corresponding to a lung sound. The spectrogram and the spectrogram of the diaphragm mode signal can be extracted.
이들 Spectrogram은 CNN(Convolutional Neural Network,57)에 입력되어, 특징 벡터를 출력하게 된다. 또한, 상기 CNN(57)은 ECG신호(69a), PPG신호(68c) PCG신호(69b) 각각에 대해 별도의 독립된 CNN을 적용할 수 있다.These spectrograms are input to CNN (Convolutional Neural Network, 57), and feature vectors are output. In addition, the CNN 57 may apply a separate independent CNN to each of the ECG signal 69a, the PPG signal 68c, and the PCG signal 69b.
이들 특징 벡터들은 SpO2(69c), PTT(69d), 글루코스 광 반사 정보(69e) 및 개인 신체 정보(69f)를 함께 분류기(21a)에 입력되어 혈압, 콜레스테롤, 혈당을 산정하게 된다.These feature vectors are input to the classifier 21a together with SpO2 (69c), PTT (69d), glucose light reflection information 69e and personal body information 69f to calculate blood pressure, cholesterol, and blood sugar.
또한, 도 11d의 실시예에서 CNN(57)와 분류기(21a)를 LSTM으로 개조 대체하여 사용할 수 있다. 이 경우 특징 벡터 추출부(33d,33e,33f)에 의해 추출된 spectrogram의 시계열 정보를 LSTM의 입력정보로 사용하여 혈압, 콜레스테롤, 혈당을 산정할 수 있다.In addition, in the embodiment of FIG. 11D , the CNN 57 and the classifier 21a may be modified and replaced with LSTMs. In this case, the time series information of the spectrogram extracted by the feature vector extraction units 33d, 33e, and 33f may be used as input information of the LSTM to calculate blood pressure, cholesterol, and blood sugar.
본 발명의 분류기(21a)는 FCN(Fully connected Network) 또는 SVM(support Vector Machine) 일수 있다.The classifier 21a of the present invention may be a fully connected network (FCN) or a support vector machine (SVM).
도 12는 도 10의 실시예에 따른 스마트 리모컨(400)상의 청진기(44)를 활용하여 태아의 건강 상태를 자가 검사할 뿐만 아니라, 원격으로 의사가 태아의 건강상태를 체크하기 위해 원격 의료 진단을 수행하는 실시예를 개략적으로 나타낸 도면이다. 달리 말해, 도 12는 스마트 리모컨(400)상의 청진기(44)를 사용하여, 의사(201)가 스마트 리모컨(400)과 블루투스로 연결된 디지털 TV(300)를 활용하여 원격으로 산모(86)의 태아의 건강상태를 온라인상에서 체크하기 위해 원격 의료 진단을 수행하는 일 실시예이다. 본 실시예에서는 디지털 TV(300)는 스마트 미러로 대체되어 사용될 수 있다.12 shows not only self-examination of the fetus' health status by using the stethoscope 44 on the smart remote control 400 according to the embodiment of FIG. It is a diagram schematically showing an embodiment to be performed. In other words, FIG. 12 shows the fetus of the mother 86 remotely by using the stethoscope 44 on the smart remote control 400 , the doctor 201 using the smart remote control 400 and the digital TV 300 connected via Bluetooth. According to an embodiment of the present disclosure, remote medical diagnosis is performed to check the health status of a person online. In this embodiment, the digital TV 300 may be replaced with a smart mirror.
디지털 TV(300) 내에 집적화되어 내장된 도 1b를 따르는 음성 인식 단말기(17)는, 의사(201)와의 원격 의료 진단을 허여하는 인터넷(202) 및 와이 파이 통신 연결을 제공하거나 청진기(44)와의 근거리 무선 통신 연결을 제공하는 디지털 통신 모듈(8), 청진기(44)에 의해 산모로부터 측정된 의료 데이터를 수신하는 의료 데이터 수신부(12), 의사(201)와 산모(86) 간의 원격 의료 진단을 수행을 위해 디지털 통신 모듈(8), 스피커(28) 와 화면표시부(10)를 제어하는 제어부(30)를 포함할 수 있다. 산모(86)의 신분 인증은 스마트 리모컨(400)상의 지문 인식부(24,25,26,27) 또는 정맥 센서부(54a)에 의해 이루어 진다.Integrated and embedded within the digital TV 300 , the voice recognition terminal 17 according to FIG. 1B provides an Internet 202 and Wi-Fi communication connection that allows remote medical diagnosis with a physician 201 or with a stethoscope 44 . A digital communication module 8 that provides a short-range wireless communication connection, a medical data receiver 12 that receives medical data measured from the mother by a stethoscope 44, and a remote medical diagnosis between the doctor 201 and the mother 86 It may include a control unit 30 for controlling the digital communication module 8, the speaker 28, and the screen display unit 10 for execution. The identity authentication of the mother 86 is made by the fingerprint recognition unit 24, 25, 26, 27 or the vein sensor unit 54a on the smart remote control 400.
또한, 디지털 TV(300)는 스마트 리모컨(400)에 의해 측정된 의료 데이터를 수신하여 이를 인터넷망을 통해 서버(13)에 재전송하고, 서버(13)상의 인공지능 신경망(16) 내지 전문가 시스템(18)에 의해 분석된 결과를 피드백 받아 디지털 TV(300) 내지 터치 스크린(401)을 통해 환자의 질환의 위험도 등급을 통지할 수 있다.In addition, the digital TV 300 receives the medical data measured by the smart remote control 400 and retransmits it to the server 13 through the Internet network, and the artificial intelligence neural network 16 or the expert system on the server 13 ( 18), the analysis result may be fed back, and the risk level of the patient's disease may be notified through the digital TV 300 or the touch screen 401 .
디지털 TV(300)는, 원격 의료 진단 동안 원격 의료 진단의 도움이 되는 시각정보를 화면 표시부(10)에 제공하여 산모(86)와 의사(201) 간의 화면을 공유할 수 있다. 또한, 디지털 TV(300)는, 스피커(28)를 통해 산모에게 건강관리에 대한 지침, 원격의료 진단 방법 및 청진기 사용법에 대한 가이드라인을 제공할 수 있다. 또한, 딥런닝 학습된 인공지능 신경망(16)은 의료 데이터 수신부(12)에 의해 수신된 청진기(44)의 의료 데이터를 분석하여, 산모 및 태아의 질병 유무와 질병의 위험도를 자동으로 판별할 수 있다.The digital TV 300 may share the screen between the mother 86 and the doctor 201 by providing visual information helpful for the remote medical diagnosis to the screen display unit 10 during the remote medical diagnosis. In addition, the digital TV 300 may provide the mother through the speaker 28 with a guideline for health care, a telemedicine diagnosis method, and a guideline for using the stethoscope. In addition, the deep learning learned artificial intelligence neural network 16 analyzes the medical data of the stethoscope 44 received by the medical data receiving unit 12 to automatically determine the presence or absence of disease and the risk of disease in the mother and fetus. have.
또한, 제어부(30)는 화면 표시부(10)상에 표시된 정보와 의사가 모니터(206)상에서 보는 정보가 서로 동일하도록 상기 디지털 통신 모듈(8)을 구동할 수 있다.Also, the control unit 30 may drive the digital communication module 8 so that the information displayed on the screen display unit 10 and the information that the doctor sees on the monitor 206 are identical to each other.
이 경우, 의사는 모니터(206)를 통해 산모(86)와 실시간 정보를 공유함으로써 의사가 직접 산모(86)에게 청진기(44)의 사용방법을 상기 통신 네트워크인 인터넷(202)을 통해 음성 지시할 수 있어, 마치 산모(86) 옆에 의사(201)가 있는 것처럼 산모의 태아 건강 진단을 도와줄 수 있다. In this case, the doctor shares real-time information with the mother 86 through the monitor 206 so that the doctor directly instructs the mother 86 how to use the stethoscope 44 through the Internet 202, which is the communication network. Therefore, it is possible to assist the mother in fetal health diagnosis as if the doctor 201 is next to the mother 86.
도면부호 60은 헤드폰으로 스마트 리모컨(400)상의 청진기(44)로부터 들려오는 태아의 심장 맥박 소리를 의사가 원격으로 들을 수 있다. Reference numeral 60 denotes that the doctor can remotely hear the fetal heart pulse sound heard from the stethoscope 44 on the smart remote control 400 through headphones.
상기 음성인식 단말기는 스마트 리모컨의 바이오 센서에 의해 측정된 생체신호에 대한 의료 데이터를 수신하여 이를 인터넷망을 통해 서버(server)에 재전송하고, 이후 서버상의 인공지능 신경망 내지 전문가 시스템에 의해 분석된 결과를 피드백 받아 환자에게 환자의 질병의 위험도를 통지할 수 있다.The voice recognition terminal receives the medical data for the bio-signal measured by the bio-sensor of the smart remote control and retransmits it to the server through the Internet network, and then the result analyzed by the artificial intelligence neural network or expert system on the server It is possible to notify the patient of the risk of the patient's disease by receiving feedback.
본 발명의 인공지능형 스마트 리모컨 장치의 또 다른 측면은 음성 인식 단말기가 디지털 TV 내부 내지 스마트 미러 내부에 집적화되어 일체화 될 수 있다. Another aspect of the artificial intelligence smart remote control device of the present invention is that the voice recognition terminal can be integrated into the inside of the digital TV or the inside of the smart mirror.
본 발명의 인공지능형 스마트 리모컨 장치의 또 다른 측면은 음성 인식 단말기가 스마트 리모컨 내부에 집적화되어 일체화 될 수 있다.Another aspect of the artificial intelligent smart remote control device of the present invention is that the voice recognition terminal can be integrated into the smart remote control.
본원의 일 실시예에 따른 스마트 리모컨 장치는, 건강 관리의 주요 지표인 체열, 혈압, 혈당, 심전도, 심장맥박, 산소포화도, 콜레스테롤, 체지방의 생체 신호를 생활 속에서 수시로 체크할 수 있는 바이오 센서가 집적화되어, 이들 건강관리 항목에 대한 생체 신호 측정과 사용자 인증을 동시에 수행할 수 있는 스마트 리모컨; 및 디지털 TV(또는 스마트 미러)의 기능을 음성 명령에 의해 제어할 뿐만 아니라 상기 스마트 리모컨의 바이오 센서에 의해 측정된 생체신호에 대한 의료 데이터를 수신하여 인공지능으로 분석할 뿐만 아니라, 이상징후가 발견된 경우, 의사와의 인터넷 통신 연결을 통해 원격 의료 진단 서비스를 제공하는 음성인식 단말기; 를 포함한다.The smart remote control device according to an embodiment of the present application includes a biosensor that can check the vital signs of body heat, blood pressure, blood sugar, electrocardiogram, heart pulse, oxygen saturation, cholesterol, and body fat, which are major indicators of health management, from time to time in daily life. a smart remote controller that is integrated and can perform biometric signal measurement and user authentication for these health care items at the same time; And the function of the digital TV (or smart mirror) is controlled by voice commands, and the medical data for the bio-signals measured by the bio-sensor of the smart remote control are received and analyzed with artificial intelligence, as well as abnormal signs are detected. a voice recognition terminal that provides a remote medical diagnosis service through an Internet communication connection with a doctor; includes
상기 음성인식 단말기는 스마트 리모컨에 의해 측정된 의료 데이터를 수신하고, 이를 인터넷망을 통해 서버(server)에 재전송하고, 이후 서버상의 인공지능 신경망 내지 전문가 시스템에 의해 상기 의료 데이터의 분석한 결과를 피드백 받을 수 있다.The voice recognition terminal receives medical data measured by the smart remote control, retransmits it to a server through the Internet network, and then feeds back the analysis result of the medical data by an artificial intelligence neural network or expert system on the server can receive
상기 스마트 리모컨은 디지털 TV를 기능을 버튼 입력에 의해 제어할 수 있을 뿐만 아니라, 스마트 리모컨 상에 설치된 바이오 센서에 의해 TV시청중 스마트 리모컨을 조작하는 과정 속에서 건강 관리의 주요 지표인 체열, 혈압, 혈당, 심장맥박, 심전도, 산소포화도, 콜레스테롤, 체지방에 관련한 의료 데이터를 수시로 측정 및 수집할 수 있고 수집된 데이터를 음성 인식 단말기의 의료데이터 수신부로 전송될 수 있다.The smart remote control can not only control digital TV functions by button input, but also use the biosensor installed on the smart remote control to operate the smart remote control while watching TV. Medical data related to blood sugar, heart pulse, electrocardiogram, oxygen saturation, cholesterol, and body fat may be measured and collected at any time, and the collected data may be transmitted to the medical data receiver of the voice recognition terminal.
또한, 환자는 원격 의료 진단 서비스 동안 상기 디지털 TV의 화면을 통해 의사와 정보를 공유할 수 있다. In addition, the patient can share information with the doctor through the screen of the digital TV during the remote medical diagnosis service.
본 발명의 일 실시예를 따르는 음성 인식 단말기는, 의료 기기들과 근거리 무선 통신 연결에 의해 복수의 의료기기로부터 측정된 의료 데이터를 수신하는 의료 데이터 수신부 및 학습용 의료 데이터들에 의해 사전에 딥런닝 학습된 인공 신경망을 포함할 수 있고, 상기 딥런닝 학습된 인공 신경망은, 상기 의료 데이터 수신부에 의해 수신된 상기 의료 데이터를 분석하여, 환자의 질병 유무와 질병의 위험도를 자동으로 판별할 수 있다.A voice recognition terminal according to an embodiment of the present invention, deep learning learning in advance by the medical data receiving unit and the medical data for learning to receive the medical data measured from a plurality of medical devices by a short-distance wireless communication connection with the medical devices. may include an artificial neural network, and the deep-running artificial neural network may analyze the medical data received by the medical data receiving unit to automatically determine the presence or absence of disease and the risk of disease.
본 발명의 음성 인식 단말기는 디지털 TV 내부 내지 스마트 미러 내부에 집적화되어 일체화될 수 있다.The voice recognition terminal of the present invention may be integrated into a digital TV or a smart mirror.
본 발명의 음성 인식 단말기는 스마트 리모컨 내부에 집적화되어 일체화될 수 있다.The voice recognition terminal of the present invention may be integrated into the smart remote control.
이 경우 스마트 리모컨은 바이오 센서에 의해 측정된 의료 데이터를 수신하고, 이를 인터넷망을 통해 서버(server)에 재전송하고, 이후 서버상의 인공지능 신경망 내지 전문가 시스템에 의해 상기 의료 데이터의 분석한 결과를 피드백 받아 터치 스크린을 통해 사용자에게 제공할 수 있다.In this case, the smart remote control receives medical data measured by the biosensor, retransmits it to a server through the Internet network, and then feeds back the analysis result of the medical data by an artificial intelligence neural network or expert system on the server. It can be received and presented to the user through the touch screen.
본 발명의 바이오 센서는 적외선 온도 센서, 카메라, 심전도를 측정하기 위한 ECG센서, 산소 포화도를 측정하기 위한 SpO2센서, 광전용적맥파(PPG, Photoplethysmography) 신호를 얻기 위한 PPG센서, 글루코스 광반사 신호들을 얻기 위한 혈당 센서, 심음(heart sound) 신호를 얻기 위한 청진기, 체지방 측정용 손 전극들 중 선택된 어느 하나 이상을 구비한 것을 특징으로 한다.The biosensor of the present invention includes an infrared temperature sensor, a camera, an ECG sensor for measuring an electrocardiogram, a SpO2 sensor for measuring oxygen saturation, a PPG sensor for obtaining a photoplethysmography (PPG) signal, and a glucose light reflection signal. It is characterized in that it is equipped with at least one selected from a blood glucose sensor, a stethoscope for obtaining a heart sound signal, and hand electrodes for measuring body fat.
상기 바이오 센서는 사용자 인증을 위한 지문 인식부 또는 정맥 센서부를 더 구비할 수 있고, 이경우 건강관리 항목에 대한 측정과 사용자 인증을 동시에 할 수 있다.The biosensor may further include a fingerprint recognition unit or a vein sensor unit for user authentication, and in this case, measurement of health care items and user authentication may be performed at the same time.
또한, 상기 바이오 센서는 심전도 측정부, 심장 질환 측정부, 산소 포화도 및 PPG신호 측정부, 체지방 측정부, 혈당 측정부, 혈압 측정부, 콜레스테롤 측정부, 청진음 측정부 중 선택된 어느 하나 이상의 측정부를 포함하는 것을 특징으로 한다.In addition, the biosensor includes at least one measuring unit selected from an electrocardiogram measuring unit, a heart disease measuring unit, an oxygen saturation and PPG signal measuring unit, a body fat measuring unit, a blood glucose measuring unit, a blood pressure measuring unit, a cholesterol measuring unit, and a auscultation sound measuring unit. characterized by including.
본 발명에서, 상기 PPG센서는 SpO2센서을 사용하여 구현하는 것이 선호된다.In the present invention, the PPG sensor is preferably implemented using an SpO2 sensor.
혈액의 중요한 역할 중 한가지는 몸의 각 부분에 산소를 공급하는 것이다.One of the important roles of blood is to provide oxygen to each part of the body.
이를 위해, 적혈구속의 헤모글로빈은 산소와 결합하여 산소를 몸의 각 부분에 산소를 공급하며, 이때 혈중의 산소 공급량은 산소 포화도(Saturation of partial pressure oxygen, SpO2) 로서 계측한다. To this end, hemoglobin in red blood cells combines with oxygen to supply oxygen to each part of the body, and the amount of oxygen supplied in the blood is measured as Saturation of partial pressure oxygen (SpO2).
즉, 산소포화도는 총 헤모글로빈의 농도에 대하여 산소를 포함하고 있는 헤모글로빈 농도의 비율로서 정의되는데, 혈액 내의 헤모글로빈이 산소와 결합해 있을 때의 광흡수도와 결합하고 있지 않을 때의 광흡수도를 측정하여 그 비를 이용하여 혈액을 직접 채취하지 않고 혈중 산소포화도를 측정한다.That is, oxygen saturation is defined as the ratio of the concentration of hemoglobin containing oxygen to the concentration of total hemoglobin. The ratio is used to measure blood oxygen saturation without directly taking blood.
본 발명의 산소포화도 측정은, 산소와 결합한 헤모글로빈(HbO2)은 적외선(예컨대, 940nm)를 잘 흡수하고, 산소와 결합하지 않은 환원 헤모글로빈(Hb)은 적색 광(예컨대, 660nm)를 더 잘 흡수하는 성질을 착안하여, 발광소자와 광 검출기로 구성된 반사형 혈관 광센서 어레이 모듈을 사용하여 박동성 동맥혈관의 팽창과 수축에 따라서 감지되는 빛의 양을 수치화 하여 비침습적(non-invasive)으로 동맥 혈액의 산소포화도를 측정한다. In the measurement of oxygen saturation of the present invention, hemoglobin bound to oxygen (HbO2) absorbs infrared light (eg, 940 nm) well, and reduced hemoglobin (Hb) not bound to oxygen better absorbs red light (eg, 660 nm). Focusing on the properties, using a reflective blood vessel optical sensor array module composed of a light emitting element and a photo detector, the amount of light detected according to the expansion and contraction of the pulsatile arterial blood vessel is quantified and non-invasively Measure oxygen saturation.
예컨대, 혈액 속에 충분히 산소가 많이 있다면, 산소와 결합한 헤모글로빈이 상대적으로 많아지고, 적외선 발광소자가 쏘아준 적외선이 혈관을 통과하면서 대부분 흡수되어 광 검출기에는 적외선이 검출되지 않는다.For example, if there is a sufficient amount of oxygen in the blood, the amount of hemoglobin bound to oxygen increases relatively, and most of the infrared rays emitted by the infrared light emitting device are absorbed while passing through the blood vessels, so that the infrared rays are not detected by the photodetector.
본 발명의 반사형 혈관 광센서 어레이 모듈은 적외선 파장 영역(예컨대, 900nm 내지950nm)의 빛을 포함하는 적외선(이하 ‘IR 광’이라 함)을 손가락 부위에 조사하는 적외선 LED; 적색 파장 영역(예컨대, 640nm 내지670nm)의 빛을 포함하는 적색 광(이하 ‘RED 광’이라 함)을 손가락 부위에 조사하는 적색광 LED; 및 상기 손가락 부위에서 반사한 상기 IR 광과 RED 광을 수광하기 위한 광 검출기를 포함하되, 이들 수광된 IR광과 RED광간의 성분의 비를 바탕으로 산소포화도를 계산한다.The reflective blood vessel photosensor array module of the present invention includes: an infrared LED irradiating infrared (hereinafter referred to as 'IR light') including light in an infrared wavelength region (eg, 900 nm to 950 nm) to a finger region; a red light LED that irradiates red light (hereinafter referred to as 'RED light') including light in a red wavelength region (eg, 640 nm to 670 nm) to a finger region; and a photodetector for receiving the IR light and the RED light reflected from the finger, wherein the oxygen saturation is calculated based on the ratio of the components between the received IR light and the RED light.
본 발명에서 광 검출기는 이미지 센서, 포도 다이오드(photo diode) 또는 포토 트랜지스터(phot transistor)를 사용하는 것이 선호된다.In the present invention, it is preferable to use an image sensor, a photo diode or a photo transistor as the photo detector.
혈중 산소 포화도는 건강한 사람의 경우 산소 포화도는 대개 100%에 가까운 98~99% 사이로 관찰된다. Blood oxygen saturation is observed between 98 and 99%, which is close to 100%, in healthy people.
일반적으로, SpO2가 95% 이하가 되면, 뇌나 장기에 공급되는 산소가 부족하여 주의력이나 집중력이 떨어지며, 쉽게 피로감을 느낀다. In general, when SpO2 is 95% or less, the oxygen supplied to the brain or organs is insufficient, so attention or concentration decreases, and fatigue is easily felt.
또한, 본 발명의 일실시예에 따르면, 광전용적맥파(PPG, Photoplethysmography) 신호는 스마트 리모컨에 설치된 SpO2 센서의 발광 소자에서 발생된 적색 광을 인체에 조사한 다음 인체에서 반사되어 수광되는 광의 광량 변화를 광 검출기에 의해 관찰하여 광전용적맥파 신호를 계측할 수 있다. In addition, according to an embodiment of the present invention, the photoplethysmography (PPG) signal irradiates the red light generated from the light emitting element of the SpO2 sensor installed in the smart remote control to the human body, and then reflects the change in the amount of light received by the human body. A photoplethysmogram signal can be measured by observation with a photodetector.
광전용적맥파 신호(PPG 신호)는 심실 수축기 동안 박출(ejection)된 혈액이 말초혈관까지 전달될 때 말초혈관에서 측정되는 맥파 신호로, SpO2 센서의 적색 광 LED 조사동안, 손가락에서 반사된 신호를 광검출기에 의해 수집함으로써 측정될 수 있다. The photoplethysmographic pulse wave signal (PPG signal) is a pulse wave signal measured in peripheral blood vessels when ejected blood is delivered to the peripheral blood vessels during ventricular systole. It can be measured by collecting by a detector.
예컨대, PPG신호는 사람의 심박동에 따라 손끝 동맥과 같이 말초혈관이 분포된 위치에서 말초혈관의 용적 변화인 혈류량 변화 양상을 광량 변화로 변환시켜 나타낸 생체신호로, 이것을 통해 사용자의 맥박을 측정할 수 있을 뿐만 아니라, 심전도(ECG) 신호와 상관관계를 비교함으로서 맥파 전달 시간(PTT, Pulse Transit Time) 및 맥파 전달 속도(PWV, Pulse Wave Velocity)를 측정하여 혈관의 상태, 동맥경화, 말초 순환 장애 진단 등 같은 심혈관 질환 진단에 사용될 수 있다.For example, the PPG signal is a biosignal that converts a change in blood flow, which is a change in the volume of peripheral blood vessels, into a change in light intensity at a location where peripheral blood vessels are distributed, such as the arteries of the fingertips, according to the heartbeat of a person. In addition, by comparing the correlation with the electrocardiogram (ECG) signal, the pulse wave transmission time (PTT, Pulse Transit Time) and the pulse wave transmission speed (PWV, Pulse Wave Velocity) are measured to diagnose the condition of blood vessels, arteriosclerosis, and peripheral circulation disorders. It can be used to diagnose cardiovascular diseases such as
본 발명의 체지방 측정은 기술 공지된 BIA(Bioelectrical impedance analysis) 방식으로 체지방을 측정하는 것이 선호된다.For body fat measurement of the present invention, it is preferable to measure body fat by a known technique BIA (Bioelectrical Impedance Analysis) method.
예컨대 BIA 방식은 체수분량에 따라 전기 저항이 달라지므로 인체를 일종의 저항으로 보고, 인체의 측정부위에 약한 교류 전류(400마이크로암페어 정도)를 흘려주고 이때의 전위차를 검출하여 그 저항값을 산출한 후, 여기서 검출된 저항값과 피측정자의 신장, 몸무게, 나이, 성별등의 변수와 함께 소정의 알고리즘에 의하여 체지방량, 근육량, 체수분량 등의 생체 정보를 산출하게 된다. For example, the BIA method sees the human body as a kind of resistance, as the electrical resistance varies depending on the amount of body water, flows a weak alternating current (about 400 microampere) to the measurement part of the human body, detects the potential difference at this time, and calculates the resistance value. , Here, biometric information such as body fat mass, muscle mass, and body water content is calculated by a predetermined algorithm along with the detected resistance value and variables such as height, weight, age, and sex of the subject.
본 발명의 일실시예에 따르면, 체지방 측정부는 스마트 리모컨상에 배치되어 피 측정자의 손가락과 직접 접촉되어 피 측정자의 신체에 전류를 가하는 구동 전극, 상기 스마트 리모컨상에 배치되어 피 측정자의 신체와 직접 접촉될 뿐만 아니라 피 측정자의 신체의 전기적 저항값을 측정하여 체지방 측정 신호를 얻기 위한 검출 전극, 및According to an embodiment of the present invention, the body fat measuring unit is disposed on the smart remote control and in direct contact with the target's finger to apply current to the target's body, the driving electrode is disposed on the smart remote control to directly contact the target's body a detection electrode for obtaining a body fat measurement signal by not only being touched but also measuring the electrical resistance value of the body of the person to be measured; and
상기 체지방 측정 신호를 디지털 데이터 신호로 변환하기 위한 AD 변환기(Analog to Digital converter)를 포함하는 것을 특징으로 한다. and an AD converter (Analog to Digital converter) for converting the body fat measurement signal into a digital data signal.
본 발명의 일실시예에 따르면, 심전도(ECG 신호, Electrocardiogram) 측정은 스마트 리모컨상에 설치된 ECG 센서를 사용하여 측정된다.According to an embodiment of the present invention, electrocardiogram (ECG signal, Electrocardiogram) measurement is measured using an ECG sensor installed on a smart remote control.
심장은 혈액을 순환시키는 펌프 역할을 하는 근육 장기로서, 그 근육 운동이 미세한 전기로 조절되고 있으며, 심장의 수축과 이완에 따라 심장을 이루는 심근 세포로부터 나오는 미세한 전기변동을 보여주는 ECG 신호가 생성되고 이 신호를 피부 표면에서 감지하는 ECG센서에 의해 읽을 수 있다. The heart is a muscle organ that acts as a pump to circulate blood, and its muscle movements are regulated by micro electricity, and ECG signals showing minute electrical fluctuations from myocardial cells that make up the heart are generated according to the contraction and relaxation of the heart. The signal can be read by an ECG sensor that detects the skin surface.
본 발명의 터치 스크린은 접촉하는 손가락 접촉 위치를 인식하여, 사용자의 명령을 입력할 수 있도록 한 입력장치이다. The touch screen of the present invention is an input device capable of inputting a user's command by recognizing a contact position of a finger.
본 발명의 터치 스크린은 정전용량 방식 (Capacitive), 저항막 방식(Resistive), 초음파 방식, 적외선 방식 (Infrared Beam) 중 선택된 어느 하나를 사용하는 것을 특징으로 한다. The touch screen of the present invention is characterized in that any one selected from a capacitive type, a resistive type, an ultrasonic type, and an infrared type (Infrared Beam) is used.
예컨대, 저항막 방식의 터치 스크린은 투명 필름 위에 저항 성분을 갖는 투명 도전성 물질을 코팅한 상하 두 면이 서로 마주보고 있으나 서로 닿지 않도록 일정한 간격으로 스페이싱 도트들(spacing dots)이 설치되어 있는데, 이때 투명 필름 위에 손가락 터치시, 상기 두면이 서로 맞닿게 되고, 터치 위치에 따라 저항값이 변화하게 되고 이를 전압의 변화 정도로 계측하여 접촉된 손가락의 위치를 인식한다. For example, in a resistive touch screen, two upper and lower surfaces coated with a transparent conductive material having a resistive component on a transparent film face each other, but spacing dots are installed at regular intervals so as not to touch each other. When a finger touches the film, the two surfaces come into contact with each other, and the resistance value changes depending on the touch position.
본 발명의 터치스크린은 ECG센서와 일체화되어 터치형 ECG 센서로 개조될 수 있다.The touch screen of the present invention may be integrated with the ECG sensor and converted into a touch-type ECG sensor.
이 경우, 스마트 리모컨의 사용자가 터치 스크린을 터치할때 마다 심장으로부터 손가락 끝으로부터 전달되어 오는 ECG신호를 감지할 수 있게 된다. 즉, 스마트 리모컨 사용자가 TV의 기능을 선택하기 위해, 스마트 리모컨의 손 전극을 한손으로 잡은채 반대편 손가락(예를들면 검지)으로 터치 스크린을 빈번히 터치하기 때문에, 생활속에서 사용자의 ECG 신호를 쉽게 채집할 수 있게 된다. In this case, whenever the user of the smart remote control touches the touch screen, the ECG signal transmitted from the heart to the fingertip can be detected. In other words, since the smart remote control user frequently touches the touch screen with the opposite finger (for example, index finger) while holding the hand electrode of the smart remote control with one hand to select the function of the TV, the user's ECG signal can be easily accessed in daily life. can be collected.
본 발명의 일실시예에 따르면, 청진기는 스마트 리모컨에 설치되어 심장 및 폐와 같은 신체의 내부에서 나오는 신호인 PCG (PhonoCardioGram)신호를 얻을 수 있다. According to an embodiment of the present invention, the stethoscope is installed on the smart remote control to obtain a PCG (Phono CardioGram) signal, which is a signal from the inside of the body, such as the heart and lungs.
본 발명의 심장 질환 측정부는 심장 질환 종류와 등급에 따라 표지된(labeled) ECG신호, PPG신호, PCG신호, 산소포화도, PTT 및 개인 신체정보를 이용하여 학습된 인공지능 신경망에 의해 피 측정자의 심장 질환을 예측하는 것이 선호된다.The heart disease measuring unit of the present invention uses the labeled ECG signal, PPG signal, PCG signal, oxygen saturation, PTT, and personal body information according to the type and grade of heart disease to measure the heart of the subject by learning the artificial intelligence neural network. Predicting the disease is preferred.
상기 심장 질환은 심부전, 부정맥, 심근경색, 협심증을 포함하는 것이 선호된다.Preferably, the heart disease includes heart failure, arrhythmia, myocardial infarction, and angina pectoris.
본 발명의 혈압 측정은 다양한 혈압 수치에 따라 표지된(labeled) ECG신호, PPG신호, PCG신호, 산소포화도, PTT 및 개인 신체 정보를 이용하여 학습된 인공지능 신경망에 의해 피 측정자의 혈압 수치를 예측하는 것이 선호된다.The blood pressure measurement of the present invention predicts the blood pressure level of the subject by the artificial intelligence neural network learned using labeled ECG signal, PPG signal, PCG signal, oxygen saturation, PTT and personal body information according to various blood pressure values. It is preferred to do
혈압이란 심장에서 보낸 혈액이 동맥의 혈관 벽에 가하는 압력을 말한다. 심장의 심실이 수축할 때는 혈액의 흐르는 양이 많아져서 압력은 높아진다. 반대로 심실이 확장할 때는 흐르는 혈액량이 적어져서 압력은 낮아진다. Blood pressure is the pressure the blood pumped from the heart exerts on the walls of the arteries. When the ventricles of the heart contract, the amount of blood flowing increases and the pressure increases. Conversely, when the ventricles dilate, the amount of blood flowing decreases and the pressure decreases.
본 발명에서 혈압 측정은 수축기 압력과 이완기 압력의 크기로 나타낸다.In the present invention, blood pressure measurement is indicated by the magnitude of the systolic pressure and the diastolic pressure.
수축기 혈압(Systolic Pressure)은 심실이 수축할 때의 최대혈압을 나타내고, 이완기 혈압(Diastolic Pressure)은 심실이 확장할 때의 최소혈압을 말한다.The systolic pressure refers to the maximum blood pressure when the ventricles contract, and the diastolic pressure refers to the minimum blood pressure when the ventricles expand.
본 발명의 콜레스테롤 측정은 다양한 콜레스테롤 수치에 의해 표지된 ECG신호, PPG신호, PCG신호, 산소 포화도, PTT 및 개인 신체 정보를 이용하여 학습된 인공지능 신경망에 의해 피 측정자의 콜레스테롤 수치를 예측하는 것이 선호된다.In the cholesterol measurement of the present invention, it is preferred to predict the cholesterol level of the subject by the artificial intelligence neural network learned using ECG signal, PPG signal, PCG signal, oxygen saturation, PTT and personal body information labeled by various cholesterol levels. do.
정상적인 혈관에 콜레스테롤이 쌓이게 되면 혈관 탄력성이 떨어져 탄성 계수(압력에 저항하는 정도)가 낮아지는데, 심장의 구동 파워를 보여주는 PCG 신호, 이에 따른 PPG신호를 가지고 콜레스테롤 수치를 추정할 수 있다. 콜레스테롤이 쌓이는 경우, 혈관 탄력이 떨어져 기준 PCG신호 대비 말초혈관의 용적 변화가 작아져 PPG신호의 크기 변화가 작아지거나 PTT가 길어진다.When cholesterol accumulates in normal blood vessels, the elasticity of the blood vessels decreases and the elastic modulus (the degree of resistance to pressure) is lowered. When cholesterol accumulates, vascular elasticity decreases and the change in the volume of peripheral blood vessels becomes smaller compared to the reference PCG signal, resulting in a smaller change in the size of the PPG signal or a longer PTT.
상기 콜레스테롤 측정은 중성지방(Triglyceride: TG), HDL-C(high-density lipoprotein-cholesterol), LDL-C(low-density lipoprotein-L cholesterol)값을 예측하는 것이 선호된다.For the cholesterol measurement, it is preferred to predict triglyceride (TG), high-density lipoprotein-cholesterol (HDL-C), and low-density lipoprotein-L cholesterol (LDL-C) values.
본 발명의 혈당 측정은, 스마트 리모컨 상에 설치된 복수개의 혈당 센서에 의해 터치 스크린상에 터치된 사용자의 손가락 부위로부터 글루코스 광 반사 신호를 얻고, 여러 등급의 혈당 수치에 의해 표지된 글루코스 광 반사 신호 와 개인 신체 정보로 학습된 인공지능 신경망을 이용하여 피 측정자의 혈당 수치를 예측하는 것이 선호된다.In the blood glucose measurement of the present invention, a glucose light reflection signal is obtained from a finger part of a user touched on a touch screen by a plurality of blood glucose sensors installed on a smart remote control, and a glucose light reflection signal marked by various grades of blood glucose level and It is preferred to predict the blood sugar level of the subject using an artificial intelligence neural network learned from personal body information.
혈중의 글루코스 농도와 근적외선의 파장에 따라, 근 적외선의 흡광도가 다르기 때문에, 서로 다른 근적외선의 파장을 출력하는 복수개의 혈당센서를 동시에 사용하여, 손가락 부위에서 반사한 근 적외선 신호들(글루코스 광 반사 신호 정보)을 상기 인공지능 신경망에 적용하면 사용자의 혈당을 계측할 수 있다. Since the absorbance of near-infrared rays is different depending on the blood glucose concentration and the wavelength of the near-infrared rays, a plurality of blood glucose sensors that output different near-infrared wavelengths are used simultaneously and the near-infrared signals reflected from the finger (glucose light reflection signal) information) is applied to the artificial intelligence neural network to measure the user's blood sugar.
본 발명의 사용자 신분 인증은 지문 인증, 정맥인증, 얼굴인식 중 선택된 어느 하나 이상 인식 방법을 사용하여 수행하는 것을 특징으로 한다. The user identity authentication of the present invention is characterized in that it is performed using any one or more recognition methods selected from fingerprint authentication, vein authentication, and face recognition.
상기 지문 인증은 스마트 리모컨을 쥐고 있는 사용자의 엄지 손가락의 지문을 인식하여 인증하는 것이 선호된다. The fingerprint authentication is preferably performed by recognizing the fingerprint of the user's thumb holding the smart remote control.
상기 정맥(vein) 인증은 스마트 리모컨을 쥐고 있는 사용자의 손가락들의 정맥 혈관 패턴 (finger vein pattern)을 인식하여 인증하는 것으로 적외선을 사용하여 혈관을 투시한 후 반사된 정맥 이미지를 템플릿 이미지와 비교하여 인증하는 것이 선호된다.The vein authentication is authentication by recognizing and authenticating the vein pattern of the user's fingers holding the smart remote control. It is preferred to do
또한, 본 발명의 음성인식 단말기는 바이오센서 및 의료기기에 의한 주기적 검사에 따른 의료 데이터의 변화 추이를 관찰하여 환자에게 위험도를 알려주거나 집중케어 검사가 필요한 항목을 알려주거나 다음 검사 일정을 환자에게 알려주는 건강 추적 관리부를 구비할 수 있다.In addition, the voice recognition terminal of the present invention observes the change in medical data according to the periodic examination by the biosensor and medical device to inform the patient of the degree of risk, inform the patient of items requiring intensive care examination, or inform the patient of the next examination schedule may include a health tracking management unit.
본 발명의 생체 신호 수집부는 스마트 리모컨상에 설치되며, 피측정자가 생활 속에서 스마트 리모컨을 사용동안 스마트 리모컨상의 바이오 센서에 의해 수집된 생체 신호 중 사용자 인증이 완료된 유효한 생체신호 (ECG신호, PPG신호, 산소포화도, 글루코스 광 반사 신호 정보, PCG신호, 체지방, 체온)만을 수집하여 의료 데이터 저장부에 저장한다. 이들 수집된 의료데이터는 인공지능 신경망, 전문가 시스템 과 건강 추적 관리부에 의해 피 측정자의 건강을 분석하고 관리하도록 하는 데에 활용된다. The bio-signal collection unit of the present invention is installed on the smart remote control, and among the bio-signals collected by the bio-sensor on the smart remote control while the subject uses the smart remote control in daily life, valid bio-signals (ECG signal, PPG signal) , oxygen saturation, glucose light reflection signal information, PCG signal, body fat, body temperature) are collected and stored in the medical data storage unit. These collected medical data are used to analyze and manage the health of the subject by the artificial intelligence neural network, expert system, and health tracking management unit.
스마트 리모컨의 각 부분(터치 스크린, 바이오 센서, 생체신호 수집부, 의료 데이터 저장부, 무선 통신 연결 수단)을 제어하는 스마트 리모컨 제어부는 특정 건강관리 항목에 대해 유효한 생체신호의 수집이 잘 안되어 부족한 경우, 이를 집중케어 항목에 추가 등록시켜 터치스크린의 건강관리 모드를 통해 사용자에게 자가 검사하도록 유도하는 생체신호 측정 요구 수단을 구비할 수 있다.The smart remote control controller, which controls each part of the smart remote control (touch screen, biosensor, biosignal collection unit, medical data storage unit, wireless communication connection means) , it may be provided with a bio-signal measurement request means for inducing the user to self-exam through the health management mode of the touch screen by additionally registering it in the intensive care item.
예를 들면, 사용자에게 생체신호 측정을 요구하기 위해 터치 스크린 화면을 건강관리 항목으로 강제 전환한 후, 집중케어 항목을 점멸하여 사용자에게 생체신호 측정을 요구할 수 있다.For example, after the touch screen screen is forcibly switched to a health care item in order to request the user to measure the biosignal, the intensive care item may be flickered to request the user to measure the biosignal.
예컨대, 식후에 혈당 체크를 요구하기 위해 터치 스크린 화면을 건강관리 항목으로 강제 전환 후, 혈당 항목을 점멸하여 사용자에게 혈당 체크를 요구할 수 있다.For example, in order to request a blood sugar check after a meal, the touch screen screen may be forcibly switched to a health care item, and then the blood sugar item may be blinked to request the user to check the blood sugar.
생체신호 측정 요구 수단의 또 다른 예를 들면 식후 혈당 체크를 위해, 주기적인 혈당 검사에 필요한 글루코스 광 반사 신호 정보가 확보가 안된 경우, 생체신호 측정 요구 수단은 사용자의 최근 식사 시간(또는 최근 식후 혈당 체크 시간)을 고려하여 식후에 즈음하여 식후 혈당 체크를 요구하는 문자나 음성 메시지를 사용자에게 전달할 수 있다. As another example of the biosignal measurement requesting means, for postprandial blood glucose check, when glucose light reflection signal information required for a periodic blood glucose test is not secured, the biosignal measurement requesting means may be used to determine the user's recent meal time (or recent postprandial blood glucose). In consideration of the check time), a text or voice message requesting to check blood glucose after a meal may be delivered to the user around the time of the meal.
생체신호 측정 요구 수단의 또 다른 측면은 음성인식부와 음성 재생부를 구비하여, 대화형으로 사용자에게 생체 신호 측정을 요구하거나 측정 방법을 안내할 수 있다.Another aspect of the biosignal measurement request means includes a voice recognition unit and a voice reproducing unit, so that the user can interactively request the biosignal measurement or guide the measurement method.
예컨대, 식후 혈당 체크 요구가 필요할 시 생체신호 측정 요구 수단은, 스마트 리모컨: “주인님 지금 식후 혈당 체크 시간입니다”. 사용자: “10분후에 다시 알려줘”. 스마트 리모컨: “알겠습니다. 주인님! 10분후에 뵙겠습니다.” 와 같이 대화형으로 사용자에게 생체 신호 측정을 요구할 수 있다.For example, when a postprandial blood glucose check request is required, the biosignal measurement request means is: “Master, it is time to check blood glucose after a meal”. User: “Tell me back in 10 minutes”. Smart Remote: “Okay. master! See you in ten minutes.” As such, it is possible to interactively request the user to measure biosignals.
또한 본 발명의 스마트 리모컨 제어부는 의료 데이터 저장부에 의해 수집된 의료 데이터를 터치스크린 화면상에 표시해 주거나 음성인식 단말기의 건강 추적 관리부로부터 제공된 부가 정보(예컨대 집중 케어 항목 및 다음 검사 일정)를 터치스크린 화면상에 표시해 줄 수 있다. In addition, the smart remote control control unit of the present invention displays the medical data collected by the medical data storage unit on the touch screen screen or displays additional information (eg, intensive care items and the next examination schedule) provided from the health tracking management unit of the voice recognition terminal on the touch screen. It can be displayed on the screen.
즉, 스마트 리모컨에 의해 측정되거나 얻어진 생체신호 정보(ECG신호, 심장맥박, PPG신호, 산소포화도, PCG신호, 체지방, 체온, 혈압, 혈당, 콜레스테롤 정보, 개인 신체 정보 등) 및 부가정보(집중 케어 항목 및 다음 검사 일정)는 스마트 리모컨의 터치 스크린상에 표시될 수 있다. That is, bio-signal information measured or obtained by the smart remote control (ECG signal, heart pulse, PPG signal, oxygen saturation, PCG signal, body fat, body temperature, blood pressure, blood sugar, cholesterol information, personal body information, etc.) and additional information (intensive care) item and next inspection schedule) can be displayed on the touch screen of the smart remote control.
본 발명의 심장 맥박 측정은 ECG신호의 피크(R 지점)을 이용하여 분당 Beats수(beats/min)로 산정할 수 있다, 또한 HRV(Heart Rate Variability)는 심장 맥박의 변동률을 말하며, 상기의 심장맥박으로 부터 산정할 수 있다.The cardiac pulse measurement of the present invention can be calculated as the number of beats per minute (beats/min) using the peak (point R) of the ECG signal. Also, HRV (Heart Rate Variability) refers to the rate of change of the heart pulse, and the heart It can be calculated from the pulse.
본 발명의 인공지능 신경망은 서버(server)상에 구축된 서비스 응용 프로그램이 될수 있으며, 이 경우, 음성인식 단말기가 스마트 리모컨 내지 의료기기에 의해 측정된 의료 데이터를 인터넷망을 통해 서버(server)에 전송하였을때, 상기 서버상의 인공지능 신경망은 음성 단말기에게 상기 의료 데이터의 분석한 결과를 피드백 해주는 서비스를 제공한다.The artificial intelligence neural network of the present invention may be a service application built on a server. In this case, the voice recognition terminal transmits medical data measured by a smart remote control or medical device to a server through the Internet network. When transmitted, the artificial intelligence neural network on the server provides a service that feeds back the analysis result of the medical data to the voice terminal.
본 발명의 인공지능 신경망의 또 다른 측면은 스마트 리모컨상의 바이오 센서에 의해 측정된 의료 데이터를 인터넷망을 통해 서버(server)에 전송하고, 이후 서버상의 인공지능 신경망에 의해 얻어진 상기 의료 데이터의 분석한 결과를 스마트 리모컨이 피드백 받아 터치 스크린을 통해 사용자에게 제공할 수 있다.Another aspect of the artificial intelligence neural network of the present invention is to transmit medical data measured by the biosensor on the smart remote control to a server through the Internet network, and then analyze the medical data obtained by the artificial intelligence neural network on the server. The result can be provided to the user through the touch screen by receiving feedback from the smart remote control.
본 발명의 인공지능 신경망의 또 다른 측면은 스마트 리모컨상의 바이오 센서에 의해 측정된 의료 데이터를 핸드폰에 전송하고, 이후 핸드폰상에 어플(application software)로서 설치된 인공지능 신경망 앱(app)에 의해 얻어진 상기 의료 데이터의 분석한 결과를 사용자에게 제공할 수 있다.Another aspect of the artificial intelligence neural network of the present invention is to transmit medical data measured by the biosensor on the smart remote control to the mobile phone, and then to the mobile phone obtained by the artificial intelligence neural network app installed as application software on the mobile phone. An analysis result of the medical data may be provided to the user.
본 발명의 스마트 리모컨은 스마트 리모컨의 움직임 신호 내지 인터럽트(interrupt) 신호에 의해 스마트 리모컨의 제어부 및 제반 전기회로가 수면 모드(sleep)에서 깨어나 정상적인 동작을 하며, 등록된 사용자의 유효한 생체 정보 획득후에는 다시 배터리 전원 절약하기 위한 수면 모드로 복귀하는 것이 선호된다. In the smart remote control of the present invention, the control unit of the smart remote control and all electrical circuits wake up from sleep mode and operate normally by the motion signal or interrupt signal of the smart remote control, and after obtaining valid biometric information of the registered user, It is preferred to return to sleep mode again to conserve battery power.
본 발명의 스마트 리모컨은 스마트 리모컨의 움직임(흔들림) 신호를 감지하기 위해 스마트 리모컨 내부에는 자이로 센서, 가속도 센서 또는 틸트 센서(tilt sensor)를 구비할 수 있다. The smart remote control of the present invention may include a gyro sensor, an acceleration sensor, or a tilt sensor inside the smart remote control to detect a motion (shake) signal of the smart remote control.
본 발명의 스마트 리모컨의 다른 측면은 스마트 리모컨의 바이오 센서가 스마트 폰 상에 설치되어 생활 속에서 스마트 폰 사용자의 생체 신호 수집을 수행할 수 있도록 개조될 수 있다.Another aspect of the smart remote control of the present invention may be modified so that the bio-sensor of the smart remote control is installed on the smart phone to collect the bio-signals of the smart phone user in daily life.
*이하에서는 상기에 자세히 설명된 내용을 기반으로, 본원의 동작 흐름을 간단히 살펴보기로 한다.* Hereinafter, based on the details described above, the operation flow of the present application will be briefly reviewed.
도면에 도시하진 않았으나 스마트 리모컨(400)을 이용한 자가 검사방법은 앞서 설명된 음성 인식 단말기(17) 및 음성인식 단말기(17)를 내장한 디지털 TV(300)에 의하여 수행될 수 있다. 따라서, 이하 생략된 내용이라고 하더라도 음성 인식 단말기(17) 및 디지털 TV(300)에 대하여 설명된 내용은 스마트 리모컨을 이용한 자가 검사방법에 대한 설명에도 동일하게 적용될 수 있다. 이하에서는 설명의 편의를 위해 스마트 리모컨 장치(미도시)에 의해 수행되는 것으로 설명하고자 한다. Although not shown in the drawings, the self-testing method using the smart remote control 400 may be performed by the above-described voice recognition terminal 17 and the digital TV 300 having the voice recognition terminal 17 embedded therein. Therefore, even if omitted below, the description of the voice recognition terminal 17 and the digital TV 300 may be equally applied to the description of the self-test method using the smart remote control. Hereinafter, for convenience of description, it will be described as being performed by a smart remote control device (not shown).
단계 S101에서, 스마트 리모컨(400)에 포함된 바이오 센서로부터 사용자의 생체 신호를 수집할 수 있다. In step S101, the user's bio-signals may be collected from the bio-sensors included in the smart remote control 400 .
단계 S102에서, 생체 신호 유효 판단 수단을 이용하여 수집된 생체 신호 중 사용자 인증이 완료된 유효한 생체신호를 판별할 수 있다.In step S102, a valid biosignal for which user authentication has been completed may be determined from among the collected biosignals by using the biosignal validity determining means.
단계 S103에서, 의료기기들에 의해 측정된 의료 데이터를 근거리 무선 통신 연결에 의해 수신한 의료데이터 및 생체 신호 수집부를 통해 획득된 의료데이터를 저장할 수 있다. In step S103, the medical data received through the short-range wireless communication connection with the medical data measured by the medical devices and the medical data obtained through the biosignal collecting unit may be stored.
단계 S104에서, 무선 통신 연결 수단을 통해 저장된 의료 데이터를 인터넷 망을 통해 서버에 전송하고, 서버 상의 인공지능 신경망 및 전문가 시스템에 의해 의료 데이터의 분석 결과를 피드백 받을 수 있다. In step S104, the medical data stored through the wireless communication connection means may be transmitted to the server through the Internet network, and the analysis result of the medical data may be fed back by the artificial intelligence neural network and the expert system on the server.
단계 S105에서, 의료 데이터의 분석 결과를 터치 스크린을 통해 사용자에게 제공할 수 있다. In step S105, the analysis result of the medical data may be provided to the user through the touch screen.
상술한 설명에서, 단계 S101 내지 S108은 본원의 구현예에 따라서, 추가적인 단계들로 더 분할되거나, 더 적은 단계들로 조합될 수 있다. 또한, 일부 단계는 필요에 따라 생략될 수도 있고, 단계 간의 순서가 변경될 수도 있다.In the above description, steps S101 to S108 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present application. In addition, some steps may be omitted if necessary, and the order between the steps may be changed.
본원의 일 실시 예에 따른 스마트 리모컨(400)을 이용한 자가 검사방법은 다양한 컴퓨터 수단을 통하여 수행될 수 있는 프로그램 명령 형태로 구현되어 컴퓨터 판독 가능 매체에 기록될 수 있다. 상기 컴퓨터 판독 가능 매체는 프로그램 명령, 데이터 파일, 데이터 구조 등을 단독으로 또는 조합하여 포함할 수 있다. 상기 매체에 기록되는 프로그램 명령은 본 발명을 위하여 특별히 설계되고 구성된 것들이거나 컴퓨터 소프트웨어 당업자에게 공지되어 사용 가능한 것일 수도 있다. 컴퓨터 판독 가능 기록 매체의 예에는 하드 디스크, 플로피 디스크 및 자기 테이프와 같은 자기 매체(magnetic media), CD-ROM, DVD와 같은 광기록 매체(optical media), 플롭티컬 디스크(floptical disk)와 같은 자기-광 매체(magneto-optical media), 및 롬(ROM), 램(RAM), 플래시 메모리 등과 같은 프로그램 명령을 저장하고 수행하도록 특별히 구성된 하드웨어 장치가 포함된다. 프로그램 명령의 예에는 컴파일러에 의해 만들어지는 것과 같은 기계어 코드뿐만 아니라 인터프리터 등을 사용해서 컴퓨터에 의해서 실행될 수 있는 고급 언어 코드를 포함한다. 상기된 하드웨어 장치는 본 발명의 동작을 수행하기 위해 하나 이상의 소프트웨어 모듈로서 작동하도록 구성될 수 있으며, 그 역도 마찬가지이다. 또한 본발명의 디지털 TV는 스마트 미러 내지 컴퓨터 모니터로 대체 가능하다.The self-test method using the smart remote control 400 according to an embodiment of the present application may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the present invention, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa. In addition, the digital TV of the present invention can be replaced with a smart mirror or computer monitor.
또한, 전술한 스마트 리모컨(400)을 이용한 자가 검사방법은 기록 매체에 저장되는 컴퓨터에 의해 실행되는 컴퓨터 프로그램 또는 애플리케이션의 형태로도 구현될 수 있다.In addition, the self-test method using the above-described smart remote control 400 may be implemented in the form of a computer program or application executed by a computer stored in a recording medium.
전술한 본원의 설명은 예시를 위한 것이며, 본원이 속하는 기술분야의 통상의 지식을 가진 자는 본원의 기술적 사상이나 필수적인 특징을 변경하지 않고서 다른 구체적인 형태로 쉽게 변형이 가능하다는 것을 이해할 수 있을 것이다. 그러므로 이상에서 기술한 실시예들은 모든 면에서 예시적인 것이며 한정적이 아닌 것으로 이해해야만 한다. 예를 들어, 단일형으로 설명되어 있는 각 구성 요소는 분산되어 실시될 수도 있으며, 마찬가지로 분산된 것으로 설명되어 있는 구성 요소들도 결합된 형태로 실시될 수 있다.The foregoing description of the present application is for illustration, and those of ordinary skill in the art to which the present application pertains will understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present application. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and likewise components described as distributed may also be implemented in a combined form.
본원의 범위는 상기 상세한 설명보다는 후술하는 특허청구범위에 의하여 나타내어지며, 특허청구범위의 의미 및 범위 그리고 그 균등 개념으로부터 도출되는 모든 변경 또는 변형된 형태가 본원의 범위에 포함되는 것으로 해석되어야 한다.The scope of the present application is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present application.
8: 디지털 통신 모듈8: digital communication module
11: 건강 추적 관리부11: Health Tracking Department
12: 의료 데이터 수신부12: medical data receiving unit
13: 서버13: Server
15: 의료 데이터 저장부15: medical data storage
16: 인공지능 신경망16: Artificial Intelligence Neural Network
17: 음성인식 단말기17: voice recognition terminal
18: 전문가 시스템18: Expert System
19: 체열 진단부19: body heat diagnosis unit
20a: 혈압측정용 인공지능 신경망20a: artificial intelligence neural network for blood pressure measurement
20b: 콜레스테롤 측정용 인공지능 신경망 20b: Artificial Intelligence Neural Network for Cholesterol Measurement
20c: 혈당 측정용 인공지능 신경망20c: Artificial Intelligence Neural Network for Blood Sugar Measurement
22: 카메라22: camera
23: 마이크23: microphone
24: 지문 인식부24: fingerprint recognition unit
24a: 손 전극24a: hand electrode
28: 스피커28: speaker
27a: 음성 인식부27a: voice recognition unit
28a: 음성 재생부28a: voice playback unit
29: 적외선 온도 센서29: infrared temperature sensor
30: 제어부30: control unit
37: 무선 통신 연결 수단37: wireless communication connection means
38a: 스피커부38a: speaker unit
38b: 마이크부38b: microphone unit
39: 리모컨 송신부39: remote control transmitter
40: Sine 파 발생기40: Sine wave generator
41: RMS 전압 검출기41: RMS voltage detector
44: 청진기44: stethoscope
44a: 박막 올림판44a: thin film lifting plate
46: AD 변환기46: AD converter
47: 인체 피부 표면47: human skin surface
48: ECG 센서48: ECG sensor
48a: 절연막 48a: insulating film
48b: ECG 전극면48b: ECG electrode surface
48c: ECG 전극48c: ECG electrode
50: 광 검출기50: photo detector
51a: 적색광 LED51a: red light LED
51b: 적외선 LED51b: Infrared LED
52a: 근 적외선 LED52a: near infrared LED
53: 스마트 리모콘 제어부53: smart remote control control
54a: 정맥 센서부54a: vein sensor unit
61: 차동 증폭기61: differential amplifier
62: HPF62: HPF
63:BRF63:BRF
68a: DC 성분68a: DC component
68b: AC 성분68b: AC component
69a: ECG 신호69a: ECG signal
69b: PPG 신호 69b: PPG signal
69c: SpO269c: SpO2
69d:PTT69d:PTT
69e: 글루코스 광 반사 정보69e: glucose light reflection information
69f: 개인 신체 정보69f: Personal Physical Information
70: 체지방 측정부70: body fat measurement unit
71: 심전도 측정부71: electrocardiogram measurement unit
73: 청진음 측정부73: auscultation sound measurement unit
81: 필터81: filter
84: 차동 증폭기 84: differential amplifier
140: TX 스캔부140: TX scan unit
142: RX 스캔부142: RX scan unit
172: 신호 증폭부 172: signal amplification unit
173: 노이즈 제거 필터173: noise removal filter
174: 주파수 필터174: frequency filter
176: ADC176: ADC
202: 인터넷202: Internet
300: 디지털 TV, 스마트 미러300: digital TV, smart mirror
400: 리모컨400: remote control
401 : 터치 스크린401: touch screen
420: 생체 신호 수집부420: bio-signal collection unit
420a : 생체신호 유효 판별 수단420a: means for determining the validity of a biosignal

Claims (27)

  1. 디지털 TV의 기능을 제어하기 위한 버튼 및 생체 신호 측정과 상기 생체 신호 측정 동안 사용자 인증을 동시에 수행하는 바이오 센서를 포함하는 스마트 리모컨; 및 a smart remote control comprising a button for controlling a function of a digital TV and a biosensor for simultaneously measuring biosignals and performing user authentication during biosignal measurement; and
    상기 디지털 TV를 음성 명령에 의해 제어하고, 상기 바이오 센서에 의해 측정된 생체 신호에 대한 의료 데이터를 수신하여 인공지능 신경망을 통해 분석하고, 의사와의 인터넷 통신 연결을 통해 원격 의료 진단 서비스를 제공하는 음성인식 단말기,Controlling the digital TV by voice commands, receiving medical data for biosignals measured by the biosensor, analyzing it through an artificial intelligence neural network, and providing a remote medical diagnosis service through an Internet communication connection with a doctor voice recognition terminal,
    를 포함하는, 인공지능형 스마트 리모컨 장치.Including, artificial intelligence smart remote control device.
  2. 제 1 항에 있어서,The method of claim 1,
    상기 스마트 리모컨은,The smart remote control,
    상기 바이오 센서로부터 생체 신호를 수집하는 생체 신호 수집부; 및a biosignal collecting unit for collecting biosignals from the biosensor; and
    상기 수집된 생체 신호 중 사용자 인증이 완료된 유효한 생체신호를 수집하기 위한 생체신호 유효 판별 수단을 포함하고,and bio-signal validity determining means for collecting valid bio-signals for which user authentication has been completed among the collected bio-signals;
    상기 음성인식 단말기는, 근거리 무선 통신 연결에 의해 복수의 의료기기로부터 측정된 의료 데이터 내지 상기 스마트 리모컨의 바이오 센서에 의해 측정된 의료 데이터를 수신하는 의료 데이터 수신부를 포함하는, 인공지능형 스마트 리모컨 장치.The voice recognition terminal comprises a medical data receiver for receiving medical data measured from a plurality of medical devices through a short-range wireless communication connection or medical data measured by a biosensor of the smart remote controller.
  3. 제 1 항에 있어서, The method of claim 1,
    상기 음성인식 단말기는, The voice recognition terminal,
    상기 의료 데이터를 인터넷 망을 통해 서버에 전송하고, 상기 서버 상의 인공지능 신경망 내지 전문가 시스템에 의해 분석된 상기 의료 데이터의 분석 결과를 피드백 받아 사용자에게 질병 유무와 질병의 위험도를 상기 디지털 TV를 통해 통지하는, 인공지능형 스마트 리모컨 장치.The medical data is transmitted to the server through the Internet network, and the result of the analysis of the medical data analyzed by the artificial intelligence neural network or expert system on the server is fed back, and the presence or absence of disease and the risk of disease are notified to the user through the digital TV. An artificial intelligence smart remote control device.
  4. 제 1 항에 있어서, The method of claim 1,
    상기 인공지능 신경망은 상기 음성인식 단말기에 설치되는 인공지능 신경망 앱(app)인 것인, 인공지능형 스마트 리모컨 장치.The artificial intelligence neural network is an artificial intelligence neural network app (app) installed in the voice recognition terminal, artificial intelligence smart remote control device.
  5. 생체 신호 측정과 상기 생체 신호 측정 동안 사용자 인증을 동시에 수행하는 바이오 센서;a bio-sensor that simultaneously performs bio-signal measurement and user authentication during the bio-signal measurement;
    상기 바이오 센서로부터 생체 신호를 수집하기 위한 생체 신호 수집부;a biosignal collecting unit for collecting biosignals from the biosensor;
    상기 생체 신호 수집부에 의해 수집된 생체 신호 중 사용자 인증이 완료된 생체 신호 성분 중 유효한 생체신호 부분을 판별하는 생체신호 유효 판별 수단; a biosignal validity determining means for discriminating a valid biosignal part among biosignal components for which user authentication has been completed among biosignals collected by the biosignal collecting unit;
    의료기기들에 의해 측정된 의료 데이터를 근거리 무선 통신 연결에 의해 수신한 의료데이터 내지 상기 생체 신호 수집부를 통해 얻어진 의료데이터를 저장하기 위한 의료 데이터 저장부;a medical data storage unit for storing medical data obtained by receiving medical data measured by medical devices through a short-range wireless communication connection or medical data obtained through the bio-signal collecting unit;
    상기 의료 데이터를 인터넷망을 통해 서버에 전송하고, 이후 상기 서버 상의 인공지능 신경망 내지 전문가 시스템에 의해 분석된 상기 의료 데이터의 분석 결과를 피드백 받기 위한 무선 통신 연결 수단; 및 a wireless communication connection means for transmitting the medical data to a server through the Internet network and receiving feedback on the analysis result of the medical data analyzed by an artificial intelligence neural network or an expert system on the server thereafter; and
    상기 바이오 센서, 생체 신호 수집부, 의료데이터 저장부, 무선 통신 연결 수단을 제어하고 상기 의료 데이터의 분석한 결과를 터치 스크린을 통해 사용자에게 제공하는 스마트 리모컨 제어부A smart remote controller that controls the biosensor, the biosignal collection unit, the medical data storage unit, and the wireless communication connection means, and provides the analysis result of the medical data to the user through the touch screen
    를 포함하는 스마트 리모컨을 포함하는 인공지능형 스마트 리모컨 장치.An artificial intelligence smart remote control device comprising a smart remote control comprising a.
  6. 사용자의 지문을 인증하는 지문 인증부;a fingerprint authentication unit for authenticating the user's fingerprint;
    상기 사용자가 상기 지문 인증부 접촉시, 상기 지문 인증부와의 동시 접촉을 제공하는 상기 지문 인증부의 외주 테두리를 감싸는 손 전극; a hand electrode surrounding the outer periphery of the fingerprint authentication unit providing simultaneous contact with the fingerprint authentication unit when the user contacts the fingerprint authentication unit;
    상기 손 전극으로부터 측정된 생체 신호 중 사용자 인증이 완료된 생체 신호 성분 중 유효한 생체신호를 판별하는 생체신호 유효 판별 수단; bio-signal validity determining means for discriminating valid bio-signals from among bio-signal components for which user authentication has been completed among bio-signals measured from the hand electrodes;
    상기 생체 신호를 분석하는 인공 지능 신경망 앱(app); 및an artificial intelligence neural network app that analyzes the biosignal; and
    상기 손 전극 및 생체신호 유효 판별 수단을 제어하고 상기 인공 지능 신경망 앱(app)에 의해 분석한 결과를 터치 스크린을 통해 사용자에게 제공하는 스마트 리모컨 제어부,A smart remote control control unit that controls the hand electrode and the biosignal validity determination means and provides the result of analysis by the artificial intelligence neural network app to the user through a touch screen;
    를 포함하는 스마트 리모컨을 포함하는 인공지능형 스마트 리모컨 장치.An artificial intelligence smart remote control device comprising a smart remote control comprising a.
  7. 제 5 항 또는 제 6 항에 있어서,7. The method according to claim 5 or 6,
    상기 터치 스크린은, The touch screen is
    리모컨 모드와 건강관리 모드 화면을 전환할 수 있는 메뉴버튼을 포함하고,Includes a menu button to switch between remote control mode and health care mode screen,
    상기 리모컨 모드 동안에는 디지털 TV의 기능을 제어하기 위한 리모컨 선택 버튼들이 상기 터치 스크린 화면상에 표시되고, During the remote control mode, remote control selection buttons for controlling functions of the digital TV are displayed on the touch screen screen,
    상기 건강관리 모드 동안에는 건강 관리 항목 중 하나를 선택하기 위한 메뉴 선택 버튼들이 상기 터치 스크린 화면상에 표시되는 것인, 인공지능형 스마트 리모컨 장치.During the health care mode, menu selection buttons for selecting one of health care items are displayed on the touch screen screen, artificial intelligent smart remote control device.
  8. 제 5 항 또는 제 6 항에 있어서, 7. The method according to claim 5 or 6,
    상기 스마트 리모컨 제어부는,The smart remote control control unit,
    특정 건강관리 항목에 대해 유효한 생체 신호가 미리 설정된 기준 값 이하로 수집되는 경우, 상기 특정 건강관리 항목의 생체신호 측정을 요청하는 문자나 음성 메시지를 사용자에게 발송하거나 상기 특정 건강관리 항목을 집중 케어 항목에 등록시켜 사용자에게 자가 검사하도록 유도하거나 대화형으로 사용자에게 생체 신호 측정을 요구하는 생체신호 측정 요구 수단을 포함하는 인공지능형 스마트 리모컨 장치.When a valid biosignal for a specific health care item is collected below a preset reference value, a text or voice message requesting measurement of a biosignal of the specific health care item is sent to the user, or the specific health care item is sent to the intensive care item An artificial intelligence smart remote control device including a bio-signal measurement request means for registering a bio-signal measurement request to induce a user to self-exam or interactively request the user to measure a bio-signal.
  9. 제 1 항, 제 5 항 또는 제 6 항에 있어서, 7. The method of claim 1, 5 or 6,
    상기 스마트 리모컨의 좌우 측면에는 반사형 혈관 광센서 어레이 모듈을 포함하고,A reflective blood vessel optical sensor array module is included on the left and right sides of the smart remote control,
    상기 반사형 혈관 광센서 어레이 모듈은 복수개의 SpO2센서 또는 복수개의 정맥 센서부를 포함하고,The reflective blood vessel optical sensor array module includes a plurality of SpO2 sensors or a plurality of vein sensors,
    사용자가 스마트 리모컨의 측면을 손가락으로 잡았을 때, 광 검출기에 의해 손가락의 내부에 있는 정맥의 혈관 패턴에 대한 정맥 이미지를 얻어 정맥 인증에 사용하거나 손가락의 동맥으로부터 산소 포화도와 PPG신호를 측정하는, 인공지능형 스마트 리모컨 장치.When the user holds the side of the smart remote control with a finger, a photodetector obtains a vein image of the vein pattern of the vein inside the finger and uses it for vein authentication or measures oxygen saturation and PPG signal from the finger's artery. Intelligent smart remote control device.
  10. 제 1 항 또는 제 5 항에 있어서, 6. The method of claim 1 or 5,
    상기 스마트 리모컨의 상하 좌우측면에 설치된 지문 인증부; 및Fingerprint authentication unit installed on the top, bottom, left and right sides of the smart remote control; and
    상기 지문 인증부 접촉시, 상기 지문 인증부와의 동시 접촉을 제공하는 상기 지문 인증부의 외주 테두리를 감싸는 복수의 손 전극,a plurality of hand electrodes surrounding the outer periphery of the fingerprint authentication unit providing simultaneous contact with the fingerprint authentication unit when the fingerprint authentication unit is in contact;
    을 포함하고, including,
    상기 지문 인증부는 사용자가 스마트 리모컨을 손으로 잡았을 때 사용자의 엄지손가락의 지문에 의해 지문 인증을 수행하고, 상기 복수의 손 전극은 체지방 측정을 위한 구동 전극과 검출 전극으로서 동작하거나 ECG센서의 제1전극을 형성하는 것을 특징으로 하는 인공지능형 스마트 리모컨 장치.The fingerprint authentication unit performs fingerprint authentication by the fingerprint of the user's thumb when the user holds the smart remote control by hand, and the plurality of hand electrodes operate as driving electrodes and detection electrodes for body fat measurement or the first of the ECG sensors. An artificial intelligence smart remote control device, characterized in that it forms an electrode.
  11. 제 10 항에 있어서, 11. The method of claim 10,
    상기 ECG 센서는 상기 스마트 리모컨의 후면부에 설치되고, 심전도 측정은 상기 제1전극과 제2전극간에 형성된 전압 신호를 증폭하여 측정하되, The ECG sensor is installed on the rear side of the smart remote control, and the electrocardiogram is measured by amplifying the voltage signal formed between the first electrode and the second electrode,
    상기 제2전극은 상기 ECG 센서가 피부에 접촉되었을 때 형성되는 것인,인공지능형 스마트 리모컨 장치.The second electrode will be formed when the ECG sensor is in contact with the skin, Artificial intelligence smart remote control device.
  12. 제 10 항에 있어서, 11. The method of claim 10,
    심전도 측정은 상기 제1전극과 제2전극 간에 형성된 전압 신호를 증폭하여 측정하되, 상기 제2전극은 상기 손의 반대편 손가락이 터치형 ECG 센서에 접촉되었을 때 형성되는 것인, 인공지능형 스마트 리모컨 장치. The electrocardiogram measurement is performed by amplifying the voltage signal formed between the first electrode and the second electrode, and the second electrode is formed when the opposite finger of the hand is in contact with the touch-type ECG sensor, artificial intelligent smart remote control device .
  13. 제 12 항에 있어서, 13. The method of claim 12,
    상기 터치형 ECG 센서는The touch-type ECG sensor is
    디스플레이 패널;display panel;
    상기 디스플레이 패널에 적층되는 하층 투명필름; a lower transparent film laminated on the display panel;
    상기 하층 투명필름상에 적층되는 상층 투명필름;an upper transparent film laminated on the lower transparent film;
    상기 상층 투명필름 상에 적층되는 ECG 센서; an ECG sensor laminated on the upper transparent film;
    상기 상층 투명필름과 하층 투명 필름을 분리하는 스페이서;a spacer separating the upper transparent film and the lower transparent film;
    상기 하층 투명필름의 윗면에 형성되어, 터치시 X축의 좌표를 제공하는 제1 저항막 패턴;a first resistive film pattern formed on the upper surface of the lower transparent film and providing X-axis coordinates when touched;
    상기 상층 투명필름의 하단에 형성되며 상기 제1 저항막 패턴과는 서로 수직으로 교차하도록 배열하여, 터치시 Y축의 좌표를 제공하는 제2 저항막 패턴;a second resistive film pattern formed at a lower end of the upper transparent film and arranged to cross perpendicularly to the first resistive film pattern to provide a Y-axis coordinate when touched;
    상기 제1 저항막 패턴의 말단에 설치되는 복수의 X축 전극; a plurality of X-axis electrodes installed at the ends of the first resistive layer pattern;
    상기 제2 저항막 패턴의 말단에 설치되는 복수의 Y축 전극; 및a plurality of Y-axis electrodes installed at the ends of the second resistive layer pattern; and
    상기 ECG 센서의 한쪽 단말에 설치되는 ECG전극, ECG electrode installed on one terminal of the ECG sensor,
    을 포함하는 인공지능형 스마트 리모컨 장치.Artificial intelligent smart remote control device comprising a.
  14. 제 1 항, 제 5 항 또는 제 6 항에 있어서, 7. The method of claim 1, 5 or 6,
    상기 스마트 리모컨은The smart remote control
    상기 스마트 리모컨의 좌우 측면에는 반사형 혈관 광센서 어레이 모듈을 포함하고,A reflective blood vessel optical sensor array module is included on the left and right sides of the smart remote control,
    상기 반사형 혈관 광센서 어레이 모듈은 복수개의 SpO2센서로 구성되어, 사용자가 상기 스마트 리모컨의 측면을 손가락으로 잡았을 때 지문 인증과 동시에, 광 검출기에 의해 손가락의 동맥으로부터 산소 포화도와 PPG신호를 측정하는, 인공지능형 스마트 리모컨 장치.The reflective blood vessel optical sensor array module is composed of a plurality of SpO2 sensors, and at the same time as fingerprint authentication when the user holds the side of the smart remote control with a finger, the photodetector measures oxygen saturation and PPG signal from the finger's artery. , artificial intelligence smart remote control device.
  15. 제 5 항 또는 제 6 항에 있어서, 7. The method according to claim 5 or 6,
    상기 스마트 리모컨은 후면부에 설치되는 청진기를 포함하고, The smart remote control includes a stethoscope installed on the rear part,
    사용자가 상기 스마트 리모컨의 측면을 손가락으로 잡아 상기 청진기를 심장 부위에 접촉시, 상기 청진기에 의해 심장 및 폐의 내부에서 나오는 PCG신호를 계측과 동시에 상기 손가락에 의해 지문 인증이 이루어지는 것인, 인공지능형 스마트 리모컨 장치. When the user holds the side of the smart remote control with a finger and touches the stethoscope to the heart, the PCG signal emitted from the inside of the heart and lungs is measured by the stethoscope and fingerprint authentication is performed by the finger at the same time. Smart remote control device.
  16. 제 15 항에 있어서, 16. The method of claim 15,
    상기 청진기는 디지털 TV를 음성 명령으로 제어하기 위한 마이크 입력으로 사용되고,The stethoscope is used as a microphone input for controlling the digital TV with voice commands,
    상기 스마트 리모컨 제어부는, 상기 청진기 사용 기간 동안 디지털 TV의 볼륨이 줄어 들도록 제어하고, 청진기 사용 종료시에는 사용 전의 디지털 TV의 볼륨 크기로 자동 복귀되도록 제어하는, 인공지능형 스마트 리모컨 장치.The smart remote control control unit controls the volume of the digital TV to decrease during the period of use of the stethoscope, and controls to automatically return to the volume level of the digital TV before use when the use of the stethoscope is terminated.
  17. 제 15 항에 있어서, 16. The method of claim 15,
    상기 청진기의 박막 울림판 표면을 도전성 재료로 코팅하고, 상기 도전성 재료의 상측에 절연막을 코팅하여 ECG 전극면을 형성함으로써, 상기 청진기를 ECG 센서로 겸용하는 것인, 인공지능형 스마트 리모컨 장치.By coating the surface of the thin-film sounding board of the stethoscope with a conductive material, and coating an insulating film on the upper side of the conductive material to form an ECG electrode surface, the stethoscope is also used as an ECG sensor, an artificial intelligence smart remote control device.
  18. 제 1 항 또는 제 5 항에 있어서, 6. The method of claim 1 or 5,
    상기 바이오 센서는 적외선 온도 센서, 카메라, 심전도를 측정하기 위한 ECG센서, 산소 포화도를 측정하기 위한 SpO2센서, 광전용적맥파(PPG, Photoplethysmography) 신호를 얻기 위한 PPG센서, 글루코스 광반사 신호들을 얻기 위한 혈당 센서, PCG 신호를 얻기 위한 청진기, 체지방 측정용 손 전극들 중 선택된 어느 하나 이상을 구비한 것인, 인공지능형 스마트 리모컨 장치.The biosensor includes an infrared temperature sensor, a camera, an ECG sensor for measuring an electrocardiogram, an SpO2 sensor for measuring oxygen saturation, a PPG sensor for obtaining a photoplethysmography (PPG) signal, and a blood sugar for obtaining glucose light reflection signals A sensor, a stethoscope for obtaining a PCG signal, and any one or more selected from hand electrodes for measuring body fat, an artificial intelligent smart remote control device.
  19. 제 1 항, 제 5 항 및 제 6 항 중 적어도 어느 한 항에 있어서, 7. The method of at least one of claims 1, 5 and 6,
    상기 인공지능 신경망은 상기 생체신호 및 개인 신체 정보를 입력으로 하여,혈압 측정용 인공지능 신경망, 콜레스테롤 측정용 인공지능 신경망, 혈당 측정용 인공지능 신경망, 심장 질환 측정부 중 선택된 어느 하나 이상을 포함하는, 인공지능형 스마트 리모컨 장치.The artificial intelligence neural network receives the bio-signals and personal body information as inputs, and includes at least one selected from an artificial intelligence neural network for blood pressure measurement, an artificial intelligence neural network for cholesterol measurement, an artificial intelligence neural network for blood sugar measurement, and a heart disease measurement unit. , artificial intelligence smart remote control device.
  20. 제 19 항에 있어서, 20. The method of claim 19,
    상기 개인 신체 정보는 체지방 정보 및 표준 혈압치를 포함하고,The personal body information includes body fat information and a standard blood pressure value,
    상기 표준 혈압치는 [수학식 1]에 적용하여 계산되고,The standard blood pressure value is calculated by applying [Equation 1],
    상기 인공지능 신경망은 생체 신호의 특징 벡터를 추출하는 딥 런닝 신경 회로망 및 LSTM(long Short term Memory)를 포함하고,The artificial intelligence neural network includes a deep running neural network for extracting a feature vector of a biosignal and a long short term memory (LSTM),
    [수학식 1]은[Equation 1] is
    Figure PCTKR2022003938-appb-I000004
    Figure PCTKR2022003938-appb-I000004
    인 것인, 인공지능형 스마트 리모컨 장치.An artificial intelligence smart remote control device.
  21. 제 19 항에 있어서, 20. The method of claim 19,
    상기 인공 지능 신경망의 입력은 ECG 신호의 R지점을 기준으로 동기화된 생체 신호를 사용하는 것인,인공지능형 스마트 리모컨 장치.The input of the artificial intelligence neural network is to use the synchronized biosignal based on the R point of the ECG signal, Artificial intelligence smart remote control device.
  22. 제 19 항에 있어서, 20. The method of claim 19,
    상기 생체신호는The biosignal is
    ECG신호, PPG신호, PCG신호, SpO2, PTT, 글루코스 광 반사 정보를 포함하는 것인,인공지능형 스마트 리모컨 장치.ECG signal, PPG signal, PCG signal, SpO2, PTT, which will include glucose light reflection information, artificial intelligence smart remote control device.
  23. 제 18 항에 있어서, 19. The method of claim 18,
    상기 혈당 센서는 The blood glucose sensor
    터치 스크린상에 표시되는 가상의 손가락 패턴; a virtual finger pattern displayed on the touch screen;
    상기 가상의 손가락 패턴 상에 정렬된 사용자의 각각의 손가락 부위에 서로 다른 파장의 근 적외선을 조사하는 복수개의 근 적외선 발광소자; 및 a plurality of near-infrared light emitting devices for irradiating near-infrared rays of different wavelengths to each finger part of the user aligned on the virtual finger pattern; and
    상기 손가락 부위로부터 반사한 근 적외선을 수광하여 전기적 신호로 변환하는 광 센서;an optical sensor that receives near-infrared light reflected from the finger and converts it into an electrical signal;
    를 포함하고, including,
    상기 광 센서로부터 글루코스 광 반사 신호 정보를 획득하는, 인공지능형 스마트 리모컨 장치.An artificial intelligence smart remote control device that acquires glucose light reflection signal information from the optical sensor.
  24. 제 1 항 에 있어서, According to claim 1,
    상기 음성 인식 단말기는 The voice recognition terminal
    상기 생체 신호 및 의료 데이터의 변화 추이를 관찰하여 사용자에게 위험도를 알려주거나 집중케어 검사가 필요한 항목을 알려주거나 다음 검사 일정을 사용자에게 알려주는 건강 추적 관리부를 포함하는, 인공지능형 스마트 리모컨 장치.An artificial intelligence smart remote control device comprising a health tracking management unit for observing changes in the bio-signals and medical data to inform the user of the level of risk, inform the item requiring intensive care examination, or inform the user of the next examination schedule.
  25. 제 1 항에 따른 인공지능형 스마트 리모컨 장치에 의해 수행되는 자가 검사 방법에 있어서,In the self-testing method performed by the artificial intelligent smart remote control device according to claim 1,
    상기 스마트 리모컨에 포함된 바이오 센서로부터 사용자의 생체 신호를 수집하는 단계;collecting a user's bio-signals from a bio-sensor included in the smart remote control;
    상기 생체 신호 중 사용자 인증이 완료된 유효한 생체신호를 판별하는 단계;determining a valid bio-signal for which user authentication has been completed among the bio-signals;
    복수의 의료기기들에 의해 측정된 의료 데이터를 근거리 무선 통신 연결에 의해 수신한 의료 데이터 및 생체 신호 수집부를 통해 획득된 의료데이터를 저장하는 단계; Storing the medical data obtained by receiving medical data measured by a plurality of medical devices through a short-range wireless communication connection and medical data obtained through a bio-signal collecting unit;
    무선 통신 연결 수단을 통해 저장된 상기 의료 데이터를 인터넷 망을 통해 서버에 전송하고, 서버 상의 인공지능 신경망 및 전문가 시스템에 의해 의료 데이터의 분석 결과를 피드백 받는 단계; 및transmitting the medical data stored through a wireless communication connection means to a server through an Internet network, and receiving a feedback result of analysis of the medical data by an artificial intelligence neural network and an expert system on the server; and
    피드백 받은 상기 의료 데이터의 분석 결과를 터치 스크린을 통해 사용자에게 제공하는 단계,providing an analysis result of the received medical data to a user through a touch screen;
    를 포함하는, 자가 검사 방법.A self-test method comprising a.
  26. 제 5 항 또는 제 6 항에 있어서, 7. The method according to claim 5 or 6,
    상기 스마트 리모컨 제어부는 The smart remote control
    상기 생체 신호 및 의료 데이터의 변화 추이를 관찰하여 사용자에게 위험도를 알려주거나 집중케어 검사가 필요한 항목을 알려주거나 다음 검사 일정을 사용자에게 알려주는 건강 추적 관리부를 포함하는, 인공지능형 스마트 리모컨 장치.An artificial intelligence smart remote control device comprising a health tracking management unit for observing changes in the bio-signals and medical data to inform the user of the level of risk, inform the item requiring intensive care examination, or inform the user of the next examination schedule.
  27. 제 19 항에 따른 인공지능형 스마트 리모컨 장치에 의해 수행되는 자가 검사 방법에 있어서,In the self-testing method performed by the artificial intelligent smart remote control device according to claim 19,
    상기 스마트 리모컨에 포함된 바이오 센서로부터 사용자의 생체 신호를 수집하는 단계;collecting a user's bio-signals from a bio-sensor included in the smart remote control;
    상기 생체 신호 중 사용자 인증이 완료된 유효한 생체신호를 판별하는 단계;determining a valid bio-signal for which user authentication has been completed among the bio-signals;
    복수의 의료기기들에 의해 측정된 의료 데이터를 근거리 무선 통신 연결에 의해 수신한 의료 데이터 및 생체 신호 수집부를 통해 획득된 의료데이터를 저장하는 단계; Storing the medical data obtained by receiving medical data measured by a plurality of medical devices through a short-range wireless communication connection and medical data obtained through a bio-signal collecting unit;
    무선 통신 연결 수단을 통해 저장된 상기 의료 데이터를 인터넷 망을 통해 서버에 전송하고, 서버 상의 인공지능 신경망 및 전문가 시스템에 의해 의료 데이터의 분석 결과를 피드백 받는 단계; 및transmitting the medical data stored through a wireless communication connection means to a server through an Internet network, and receiving a feedback result of analysis of the medical data by an artificial intelligence neural network and an expert system on the server; and
    피드백 받은 상기 의료 데이터의 분석 결과를 터치 스크린을 통해 사용자에게 제공하는 단계,providing an analysis result of the received medical data to a user through a touch screen;
    를 포함하는, 자가 검사 방법.A self-test method comprising a.
PCT/KR2022/003938 2021-04-27 2022-03-22 Artificial intelligence smart remote control device and self-test method using same WO2022231132A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020210054095A KR102313630B1 (en) 2021-04-27 2021-04-27 Ai smart remote controller apparatus and self-exam method using the same
KR10-2021-0054095 2021-04-27

Publications (1)

Publication Number Publication Date
WO2022231132A1 true WO2022231132A1 (en) 2022-11-03

Family

ID=78150940

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2022/003938 WO2022231132A1 (en) 2021-04-27 2022-03-22 Artificial intelligence smart remote control device and self-test method using same

Country Status (2)

Country Link
KR (1) KR102313630B1 (en)
WO (1) WO2022231132A1 (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102313630B1 (en) * 2021-04-27 2021-10-15 성균관대학교산학협력단 Ai smart remote controller apparatus and self-exam method using the same
KR20230148019A (en) * 2022-04-15 2023-10-24 서울대학교병원 Smart terminal device for obtaining medical infomration of electrocardiogram and hearing
KR102555157B1 (en) * 2022-07-04 2023-07-14 한요한 Integrated care system and method using heatlcare device
KR102509538B1 (en) * 2022-08-19 2023-03-16 주식회사 마크헬츠 Portable non-face-to-face diagnostic device equipped with sensor

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20140107945A (en) * 2013-02-28 2014-09-05 주식회사 펀진 healthcare System and method
KR20150099430A (en) * 2014-02-21 2015-08-31 삼성전자주식회사 Electronic device
KR20160028329A (en) * 2014-09-03 2016-03-11 삼성전자주식회사 Electronic device and method for measuring vital information
KR101706612B1 (en) * 2015-09-15 2017-02-15 금오공과대학교 산학협력단 Intelligent Remote Controller for Smart Healthcare
KR102199020B1 (en) * 2020-05-08 2021-01-06 성균관대학교산학협력단 Ceiling aihealth monitoring apparatusand remote medical-diagnosis method using the same
KR102313630B1 (en) * 2021-04-27 2021-10-15 성균관대학교산학협력단 Ai smart remote controller apparatus and self-exam method using the same

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20140107945A (en) * 2013-02-28 2014-09-05 주식회사 펀진 healthcare System and method
KR20150099430A (en) * 2014-02-21 2015-08-31 삼성전자주식회사 Electronic device
KR20160028329A (en) * 2014-09-03 2016-03-11 삼성전자주식회사 Electronic device and method for measuring vital information
KR101706612B1 (en) * 2015-09-15 2017-02-15 금오공과대학교 산학협력단 Intelligent Remote Controller for Smart Healthcare
KR102199020B1 (en) * 2020-05-08 2021-01-06 성균관대학교산학협력단 Ceiling aihealth monitoring apparatusand remote medical-diagnosis method using the same
KR102313630B1 (en) * 2021-04-27 2021-10-15 성균관대학교산학협력단 Ai smart remote controller apparatus and self-exam method using the same

Also Published As

Publication number Publication date
KR102313630B1 (en) 2021-10-15

Similar Documents

Publication Publication Date Title
WO2022231132A1 (en) Artificial intelligence smart remote control device and self-test method using same
Sun et al. Photoplethysmography revisited: from contact to noncontact, from point to imaging
WO2017146524A1 (en) Aparatus and method for assessing heart failure
CN101264011B (en) Method and apparatus for cufflessly and non-invasively measuring wrist blood pressure
CN112040846A (en) Method for estimating blood pressure and degree of arteriosclerosis based on photoplethysmography (PPG) signal
US20230099079A1 (en) Scale-based user-physiological heuristic systems
WO2018093136A1 (en) Device for subject image monitoring, method therefor, and system for image monitoring
WO2010038993A2 (en) Cardiovascular analysis system
WO2018093131A1 (en) Device for measuring sleep apnea and method therefor
WO2013141419A1 (en) Biometric system using both hands for evaluation of blood vessel and cardio pulmonary function
WO2014157896A1 (en) Film-type device for measuring biomedical signal, and blood pressure measurement device, cardiopulmonary endurance estimation device, and individual certification device using same
EP3288447A1 (en) Apparatus and method for extracting cardiovascular characteristic
WO2017147609A1 (en) Systems and methods for modified pulse transit time measurement
WO2017146519A1 (en) Sensor-based detection of changes in health and ventilation threshold
WO2017099426A1 (en) Electronic apparatus and control method thereof
JP2002052022A (en) Stethoscope mouth
Kao et al. Towards maximizing the sensing accuracy of an cuffless, optical blood pressure sensor using a high-order front-end filter
KR101407049B1 (en) An apparatus and method of extracting body signals from signals including noise signals and detected by a stethoscope
Mehrgardt et al. Deep learning fused wearable pressure and PPG data for accurate heart rate monitoring
Pflugradt et al. Multi-modal signal acquisition using a synchronized wireless body sensor network in geriatric patients
WO2020005027A1 (en) Electrocardiogram measurement method and system using wearable device
WO2021100994A1 (en) Non-contact method for measuring biological index
Phan et al. Noninvasive, wearable multi biosensors for continuous, long-term monitoring of blood pressure via internet of things applications
WO2023075253A1 (en) Electronic device for sleep monitoring and operation method thereof
EP3337391A1 (en) Electronic apparatus and control method thereof

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22795966

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE