KR20200104758A - Method and apparatus for determining a dangerous situation and managing the safety of the user - Google Patents

Abstract

According to one disclosure, provided is a method for determining a dangerous situation of a user through first danger determination which has analyzed a bio-signal of a user and second danger determination which has analyzed surrounding environment information, and managing the safety. The method comprises the steps of: sensing a bio-signal of a user by using at least one sensor every predetermined period; matching a surrounding environment of the user with any one of a plurality of state groups classified in accordance with the activity attribute of the user, when the change in the bio-signal of the user exceeds a threshold value; determining a danger level for the surrounding environment based on setting information of the matched state group and the bio-signal of the user; and transmitting a notification message to a terminal of a guardian in accordance with the determined danger level.

Description

METHOD AND APPARATUS FOR DETERMINING A DANGEROUS SITUATION AND MANAGING THE SAFETY OF THE USER}

According to the start of the day, a method of analyzing the user's biosignal and the user's current surrounding environment to determine whether the user is in a dangerous situation, and transmitting a notification to the guardian's terminal when the user's state is determined to be a dangerous situation; and The device is started.

In recent years, child abuse and mental abuse cases frequently occur in childcare facilities, leading to greater injuries to children and their parents. In particular, children who have been damaged by assault and abuse cases are suffering from post-traumatic stress disorder and need psychological treatment.

However, since most parents who entrust their children to child care facilities are double-income, difficulties may arise in areas such as taking a separate time to visit a psychological counseling center with the child or continuously receiving psychological treatment from experts.

In addition, if parents are not directly aware of the child's condition, the child is left as it is and may cause a greater psychological disorder in the future.Therefore, it is possible to predict child abuse by grasping the child's physical condition and emotional condition easily and freely in time and space. You need a system that can cope with it.

In addition, advances in medical technology are leading to an aging society with an extended life expectancy. Accordingly, the importance of the elderly's health is increasing. With the advent of an aging society, the number of elderly living alone living alone is increasing. As such, in the case of an emergency that causes abnormal health problems, the elderly living alone cannot get help from someone, so a monitoring system is being developed for their safety.

However, in the existing system, the risk situation can be notified to the guardian only through a direct request of the user, or there is a limit to determining the risk situation using only the user's biometric characteristics. There is an increasing need for a risk situation management system that can accurately judge the situation and notify the guardian.

Korean Patent Publication No. 10-2016-0161073 (published on June 8, 2018) Korean Registered Patent Publication No. 10-1677642 (announced on November 18, 2016)

The technical task according to the start of the day is to determine whether the user is in a dangerous situation based on a change in the user's bio-signal, determine the current state of the user in consideration of the user's activity attributes, and whether the user is currently in a dangerous situation, The objective is to provide a method and apparatus for determining a user's dangerous situation, which is notified to a guardian's terminal by determining the level of risk, if in a dangerous situation, and managing safety.

A method of determining a risk situation and managing a user's safety according to the first embodiment includes sensing a user's biosignal using at least one sensor every predetermined period, and the change of the user's biosignal determines a threshold value. In case of deviation, the step of acquiring at least one image of the user's surrounding environment by driving the camera, analyzing the at least one image, and any one of a plurality of state groups classified according to the user's activity attribute Matching to a state group of, determining a risk level for the surrounding environment included in at least one image based on the setting information of the matched state group and the user's biosignal, and to the guardian's terminal according to the determined risk level. It may include transmitting a notification message.

An apparatus for determining a dangerous situation and managing user safety according to the second embodiment includes a camera for photographing a user and a user's surrounding environment, a processor, and a memory storing instructions executable by the processor, and the processor is , By executing commands, the user's bio-signal is sensed using at least one sensor every predetermined period, and when the change in the user's bio-signal exceeds the threshold value, the camera is driven to at least one of the user's surroundings. And the user's surrounding environment is matched to any one of a plurality of state groups classified according to the user's activity attribute by acquiring an image of and analyzing at least one image, and setting information of the matched state group and the user Based on the biosignal of, the risk level for the surrounding environment included in at least one image may be determined, and a notification message may be transmitted to the terminal of the guardian according to the determined risk level.

According to one disclosure, the present invention can prevent abuse of infants or the elderly by determining whether physical or mental abuse of infants or seniors with low judgment ability using big data.

By initiation, it is possible to accurately determine a dangerous situation that cannot be confirmed only by a change in the user's bio-signal according to the secondary risk situation determination.

By the start of the day, by first using the image of the user and the user's surroundings acquired from the user's device, and by providing the CCTV image to the guardian through the IoT service according to the level of the danger situation, the guardian can make the danger situation more clear To be able to know. In addition, there is an effect that the guardian can use the acquired images and images as evidence.

In addition, by monitoring the user's usual behavior pattern and updating the setting information of the state group, it is possible to determine whether a risk situation is suitable for the user's activity.

1 is a view for explaining the operation of a dangerous situation analysis apparatus according to an embodiment.
2 is a flowchart illustrating a method of determining a user's risk situation and managing safety according to an exemplary embodiment.
3 is a diagram for explaining a method of informing a user of a dangerous situation to another device in a dangerous situation analysis apparatus according to an exemplary embodiment.
4 is a diagram illustrating a main configuration of an apparatus for analyzing a dangerous situation according to an exemplary embodiment.
FIG. 5 is a diagram for explaining contents of transmitting a notification message about a dangerous situation from a risk situation analysis device in the form of a wearable device to a guardian terminal according to an embodiment.
FIG. 6 is a diagram for explaining content of detecting a user's bio-signal and analyzing a dangerous situation in the apparatus for analyzing a dangerous situation in the form of a mobile terminal according to an exemplary embodiment.
7 is a view for explaining a characteristic of determining a user's risk situation using a sensing unit mounted on a clothing according to an exemplary embodiment.
8 is a view for explaining content of determining a user's risk situation by using a sensing unit mounted on a socks according to an embodiment.
9 is a diagram for explaining content of analyzing a user's bio-signal according to an exemplary embodiment.
10 is a diagram for explaining contents of acquiring an image of a user's surrounding environment by using a CCTV device around a user according to an exemplary embodiment.
11 is a diagram for describing a detailed configuration of an apparatus for analyzing a dangerous situation according to an exemplary embodiment.

The terms used in the present specification will be briefly described, and an embodiment of the present invention will be described in detail.

Terms used in the present specification have selected general terms that are currently widely used as possible while considering functions in the present invention, but this may vary depending on the intention or precedent of a technician working in the field, the emergence of new technologies, and the like. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning of the terms will be described in detail in the description of the corresponding invention. Therefore, the terms used in the present specification should be defined based on the meaning of the term and the overall contents of the present invention, not a simple name of the term.

When a part of the specification "includes" a certain component, it means that other components may be further included rather than excluding other components unless specifically stated to the contrary. In addition, terms such as "... unit" and "module" described in the specification mean units that process at least one function or operation, which may be implemented as hardware or software, or as a combination of hardware and software. .

In this specification, the “device 100 that determines a dangerous situation and manages the safety of a user” will be described as the device 100 for determining a dangerous situation.

In the present specification, “user” refers to a person whose ability to judge a dangerous situation is low, such as an infant, an elderly person, a child, or a disabled person, and further, may refer to anyone who uses the risk situation analysis device.

In the present specification, "guardian" refers to a person who manages the safety of the user, and means a user registered in advance so as to receive a notification message about a dangerous situation from the user's risk situation determination device. Furthermore, it may include public institutions such as police, fire departments, hospitals, and the like.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art may easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and similar reference numerals are assigned to similar parts throughout the specification.

1 is a view for explaining the operation of a dangerous situation analysis apparatus according to an embodiment.

By initiation, the apparatus 100 for analyzing a dangerous situation may periodically detect a user's bio-signal and perform detailed analysis on a current user's state according to a change in the user's bio-signal. Furthermore, when a sudden change in the user's bio-signal occurs, the dangerous situation analysis apparatus 100 determines what the current user's activity is, and then determines whether the current user is in a dangerous situation through analysis of the surrounding situation. I can. In addition, the danger situation analysis apparatus 100 may determine the user's risk level by analyzing the user's bio-signal and the context of the surrounding situation. In addition, the dangerous situation analysis apparatus 100 notifies the user's dangerous situation to the guardian's terminal 200 so that the guardian can more clearly manage the user's safety.

For example, looking at (a) of FIG. 1, a child, a user, is severely scolded by a teacher and is under serious mental stress. The dangerous situation analysis apparatus 100 may determine that the user is currently in a severe stress state by acquiring bio-signals such as the user's heart rate, blood pressure change amount, and sweat discharge. According to an initiation, the apparatus 100 for analyzing a dangerous situation may acquire information about a user's surrounding environment by driving a camera or a recording device. The dangerous situation analysis apparatus 100 may determine that the current user is a daily activity, and may determine that the user is being beaten, abused, or is receiving abusive speech from another person included in the image. Next, the dangerous situation analysis apparatus 100 may determine that the risk level in the user's daily activities is 3, and transmit it to the guardian's terminal 200 to cause the guardian to take action. In addition, by transmitting the image 201 of the surrounding environment together with a notification message to the guardian's terminal 200, the user's situation can be more clearly shared.

The user's risk level may be automatically determined using information such as age, lesion, and gender, and may be set by the user or guardian. It will be described in detail below.

Looking at (b) of FIG. 1, it can be seen that the user 10 fell in bed while sleeping. The apparatus 100 for analyzing a dangerous situation may determine that the user fell from bed while sleeping by using the change in the user's bio-signal and information on the surrounding environment. In this case, the danger situation analysis apparatus 100 may determine the risk level of the user according to time by continuously analyzing the next action of the user 10 and the change in the biosignal. For example, if the biosignal of the user 10 is stably changed again, or if the user's moaning sound is not heard in the image of the surrounding environment, it may be determined that the risk level of the user 10 is not high. The danger situation analysis apparatus 100 manages the safety status of the user 10 by transmitting a notification message 202 about a danger situation and a message 203 about the real time status of the user 10 to the guardian's terminal 200 can do.

2 is a flowchart illustrating a method of determining a user's risk situation and managing safety according to an exemplary embodiment.

In block 201 according to the start of the day, the apparatus 100 for analyzing a dangerous situation may sense a biosignal of a user by using at least one sensor every predetermined period.

By initiation, a predetermined period may be directly determined by the user, and when a dangerous situation occurs, a period for sensing a bio-signal may be set to be shorter.

By initiation, the user's biosignal may be used as an index value for determining the user's physical and mental state. By initiation, the biosignal may include body temperature information, sweat emission information, skin characteristic information, heart rate information, blood pressure information, etc. of the user, and may include all biosignals that can be measured using a sensing device. .

In block 202 according to the start of the day, when the change in the user's bio-signal exceeds the threshold value, the apparatus 100 may drive the camera to obtain at least one image of the user's surrounding environment.

By initiation, the apparatus 100 for analyzing a dangerous situation may generate an average value of the biosignal according to the user's activity type by using the periodically acquired biosignal of the user. Alternatively, the danger situation analysis apparatus 100 may store a normal range and a threshold value of a biosignal input by a user or a guardian.

When the user's biometric signal exceeds a predetermined threshold, the dangerous situation analysis apparatus 100 may make a primary determination that the user has encountered a dangerous situation. That is, the danger situation analysis apparatus 100 may primarily determine whether the user is in danger by analyzing only the user's biosignal without the user's intervention.

The apparatus 100 for analyzing a dangerous situation may acquire at least one image of the user's surrounding environment by driving a camera in order to determine the user's state more clearly. In addition, it is natural that the apparatus 100 for analyzing a dangerous situation may acquire an image of a surrounding environment using a camera.

In block 203 according to the start of the day, the apparatus 100 for analyzing a dangerous situation may analyze at least one image and match the user's surrounding environment to any one of the plurality of state groups classified according to the user's activity attribute. have.

At the beginning, the apparatus 100 for analyzing a dangerous situation may analyze what kind of activity the user is currently performing by analyzing an image photographed of the user and the user's surrounding environment. At the start of the day, the apparatus 100 for analyzing a dangerous situation may determine a user's activity by analyzing a user's image through artificial intelligence learning using big data.

The dangerous situation analysis apparatus 100 may classify the user's activity attribute into a plurality of state groups in advance. For example, the state group may include exercise, sleep, daily life, learning, and the like, and the state group is not limited.

According to one disclosure, the state group may be classified according to profile information of an object obtained by observing a plurality of objects. According to one disclosure, the state group may be classified based on any one profile information among a plurality of object profile information. According to one disclosure, groups may be classified according to criteria obtained by combining a plurality of profile information.

For example, the state group may be formed into a plurality of groups based on the age of 10 based on the age group. For example, 0 to 10 years old can be defined as an infant group, and 71 to 90 years old can be defined as an elderly group. In addition, as another example, groups may be classified according to a value calculated by assigning weights to a plurality of profile information. For example, a “group of elderly men with a waist disc” may be determined based on object profile characteristics such as 70s, men, and a curved waist.

More specifically, when the user is a child aged 0 to 10 years, the plurality of state groups may include an exercise state group, a sleep state group, a meal state group, a rest state group, and a learning state group. The setting information of the state group may be determined based on data for monitoring the activity of the child.

In addition, when the user is an elderly person aged 65 years or older, the plurality of state groups include an exercise state group, a sleep state group, a meal state group, a rest state group, and a learning state group, and the setting information of the plurality of state groups is It may be determined based on data learned based on age and medical history information.

A method of determining a plurality of state groups according to one disclosure is not limited. According to one initiation, a plurality of state groups may be classified according to a learning model by artificial intelligence. According to an initiation, a plurality of state groups may be classified according to a classification criterion input by a user.

The dangerous situation analysis apparatus 100 may create a state group using profile information for a plurality of users having similar activity attributes, and determine setting information for determining the risk level of the surrounding environment matching the state group. have. At this time, the profile information may include gender, age, lesion, usual activity amount, age-specific language use information, walking speed, activity pattern, stride length, whether or not an assistive device is installed, stress index, mental empathy ability, emotional expression ability. In addition, the setting information may mean average information of profile information for a plurality of users used to create a state group.

In block 204 according to the start, the apparatus 100 for analyzing a dangerous situation may determine a risk level for the surrounding environment included in the at least one image based on setting information of the matched state group and a biosignal of the user.

Accordingly, the danger situation analysis apparatus 100 has an effect that it is possible to accurately monitor whether or not the user is in danger in the light of the user's bio-signal, the user's current activity, and the usual activity habit. That is, after determining whether there is a risk using the first user's biometric signal, it is possible to determine whether the user is in a dangerous state through secondary analysis of the surrounding environment.

The dangerous situation analysis apparatus 100 may analyze at least one image included in the surrounding environment to obtain information on a change in movement of a person and an object included in the surrounding environment over time. The risk level for the surrounding environment may be determined by analyzing the change in the user's bio-signal corresponding to the information on the change in movement of the user.

In block 205 according to an initiation, the apparatus 100 for analyzing a danger situation may transmit a notification message to the terminal of the guardian according to the determined danger level.

In this case, the guardian's terminal may be a pre-registered user who has been authorized to check the user's status. In addition, the guardian's terminal may include a server of a national center, a police station, a fire station, a hospital, and other public institutions related to the user.

3 is a diagram for explaining a method of informing a user of a dangerous situation to another device in a dangerous situation analysis apparatus according to an exemplary embodiment.

The apparatus 100 for analyzing a danger situation according to an initiation may mean an electronic device attached to or close to a user's body. For example, the danger situation analysis apparatus 100 may include a wearable device. For example, the risk situation analysis device 100 includes an accessory type device (eg, a watch, a ring, a bracelet, an anklet, a necklace, glasses, contact lenses), a head-mounted-device (HMD), and a fabric. Or, it may include at least one of a clothing-integrated device (eg, electronic clothing), a body-attached device (eg, a skin pad), or a living body implantable device (eg, an implantable circuit), but is limited thereto. no.

Also, a user may wear a plurality of wearable devices, and the plurality of wearable devices may directly measure the user's biometric information. In addition, the danger situation analysis apparatus 100 may obtain information on a current state of a user from a plurality of wearable devices.

The dangerous situation analysis apparatus 100 may communicate with the guardian's terminal 200 and other servers 300. For example, the danger situation analysis apparatus 100 may transmit a notification message about the user's status to at least one of the guardian's terminal 200 and another server 300 according to the level of the user's risk situation.

The guardian's terminal 200 may be a device previously registered by the user or may be another user's device. By the beginning, the guardian's terminal 200 includes a mobile phone, a smart phone, a notebook computer, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), a camera, a navigation device, and a tablet. It may include a computer (tablet computer), an e-book (e-book) terminal, a smart watch (Smart watch), and the like.

The server 300 may include a server such as a nursing institution, a hospital, a nursing home, a kindergarten, a police station, a fire station, etc. that manages the user's state, and further includes a server for managing the user's state and analyzing the risk situation. I can.

4 is a diagram illustrating a main configuration of an apparatus for analyzing a dangerous situation according to an exemplary embodiment.

The dangerous situation analysis apparatus 100 may include a camera 1610, a memory 1700, a communication unit 1500, and a control unit 1300.

At the beginning, the camera 1610 may capture images of the user and the user's surrounding environment. The camera 1610 may obtain an image frame such as a still image or a video through an image sensor in a video call mode or a photographing mode. The image captured through the image sensor may be processed by the controller 1300 or a separate image processing unit (not shown). For example, an image captured by the camera 1610 may be used as information for determining whether an event has occurred.

According to one disclosure, a program for processing and controlling the memory 1700 and the controller 1300 may be stored, and data input to or output from the device of the present invention may be stored. Programs stored in the memory 1700 may be classified into a plurality of modules according to their functions. Here, the plurality of modules are software, not hardware, and functionally operated modules.

According to an initiation, the communication unit 1500 may transmit/receive data between the danger situation analysis apparatus 100 and the guardian's terminal or other devices. The communication unit 1500 may include a short range communication unit 1510, a mobile communication unit 1520, and a broadcast reception unit 1530. The other device may be a computing device or a sensing device, but is not limited thereto.

According to one disclosure, the controller 1300 may be described as being mixed with a processor, and has the same meaning as a processor. At the start of the day, the controller 1300 senses the user's biosignal using at least one sensor at a predetermined period by executing commands, and when the change of the user's biosignal exceeds the threshold, drives the camera. Acquires at least one image of the user's surrounding environment, analyzes at least one image, matches the user's surrounding environment to any one of a plurality of state groups classified according to the user's activity attribute, and matches A risk level for the surrounding environment included in at least one image may be determined based on the setting information of the set status group and the user's biosignal, and a notification message may be transmitted to the guardian's terminal according to the determined risk level.

FIG. 5 is a diagram for explaining contents of transmitting a notification message about a dangerous situation from a risk situation analysis device in the form of a wearable device to a guardian terminal according to an embodiment.

According to one disclosure, the apparatus 100 for analyzing a dangerous situation may be in the form of a wearable band mounted on a user's wrist. As an example, the apparatus 100 for analyzing a dangerous situation may include other types of electronic devices.

The apparatus 100 for analyzing a dangerous situation may acquire a user's biosignal using a sensor. For example, a temperature sensor that measures the user's body temperature, a humidity sensor that measures the user's sweat output, a reflective optical sensor that measures the user's skin characteristics, a heart rate sensor that measures the user's heart rate, and the user's blood pressure. The biosignal of the user may be sensed using at least one of a blood pressure sensor, an impedance sensor that measures an impedance signal in the user's body, and a motion sensor that measures a user's activity.

The dangerous situation analysis device 100 may generate the user's biosignal information in real time by considering at least one of the user's body temperature information, sweat emission information, skin characteristic information, heart rate information, blood pressure information, impedance information, and activity information. I can.

According to an initiation, the apparatus 100 for analyzing a dangerous situation may determine the user's activity attribute by acquiring information about the user's surrounding environment when a sudden change in information of the user's bio-signal occurs. For example, the risk situation analysis apparatus 100 may determine that the user is in an exercise state, and may confirm that the heart rate during exercise at night time has rapidly increased.

The apparatus 100 for analyzing a risk situation may determine a risk level of the user through information about a user's heart rate and surrounding environment such as a temperature at night and a late time. By initiation, the apparatus 100 for analyzing a dangerous situation may transmit a notification message according to the user's risk level to the terminal 200 of the guardian. Alternatively, it can be transmitted to a public institution server, a police station server, a hospital server, a fire station server, or the like.

In addition, the apparatus 100 for analyzing a dangerous situation may transmit a different type of notification message to the guardian's terminal according to the determined risk level.

At the beginning of the day, the apparatus 100 for analyzing a dangerous situation may analyze at least one image included in the surrounding environment, and obtain information on a change in movement of a person and an object included in the surrounding environment over time. . In addition, a risk level for a surrounding environment may be determined by analyzing a change in a user's bio-signal corresponding to information on a change in movement of a person and an object.

Furthermore, the apparatus 100 for analyzing risk situations may include user's body temperature information, sweat emission information, skin characteristic information, heart rate information, blood pressure information, and heart rate of activity information according to changes in movements of people and objects included in the surrounding environment. If is faster than the threshold value, the risk level for the surrounding environment can be adjusted upward. Alternatively, the risk level for the surrounding environment may be lowered according to the movement of people and objects included in the surrounding environment, and biosignal information of the user.

By way of initiation, determining the risk level for the surrounding environment may be performed by a learning network model driven by an artificial intelligence (AI) engine. The artificial intelligence engine may determine the risk level for the surrounding environment in consideration of the surrounding environment, user identification and biometric information, and state group information according to activity attributes.

For example, when the danger level is 1, the danger situation analysis apparatus 100 may transmit first notification information including information about the danger situation. In this case, the information on the dangerous situation includes biometric information of a user in which a sudden change has occurred, and information on a current activity of the user, and the first notification information may be text information.

When the danger level is 2, the first notification information and the second notification information including an image of the surrounding environment for the dangerous situation may be transmitted. For example, the danger situation analysis apparatus 100 may transmit text information and second notification information including the current user's surrounding environment and the current user's image.

When the risk level is 3, the second notification information and third notification information including an image of a dangerous situation may be transmitted to the guardian's terminal. For example, in addition to the second notification information, the danger situation analysis apparatus 100 may transmit an image around the current user in real time. Alternatively, an image at a time point at which the user's bio-signal rapidly changes may be transmitted.

When the risk level is 4, a video call may be attempted to the guardian's terminal while transmitting the third notification information. The danger situation analysis apparatus 100 may attempt to make a phone call to a contact information such as a public institution, a police station, or a hospital in addition to the guardian's terminal.

FIG. 6 is a diagram for explaining content of detecting a user's bio-signal and analyzing a dangerous situation in the apparatus for analyzing a dangerous situation in the form of a mobile terminal according to an embodiment.

According to one disclosure, the danger situation analysis apparatus 100-b may be in the form of a mobile terminal, and may acquire a biosignal of the user 100 at a location close to the user. In addition, the danger situation analysis apparatus 100-b may acquire a biosignal of the user 100 by using a separate sensor unit directly attached to the user 100.

For example, the danger situation analysis device 100-b is a temperature sensor that measures the user's body temperature, a humidity sensor that measures the user's sweat discharge, a reflective optical sensor that measures the user's skin characteristics, and measures the user's heart rate. The biosignal of the user may be obtained using a heart rate sensor that measures the user's blood pressure, a blood pressure sensor that measures the user's blood pressure, an impedance sensor that measures an impedance signal in the user's body, and a motion sensor that measures the user's activity.

In addition, the danger situation analysis apparatus 100-b includes a separate sensor unit attached to the user's body, for example, a temperature sensor unit that measures the temperature of the user's skin, a motion sensor unit that measures the user's vibration, and the user's heart rate. It is possible to receive a bio-signal from a heart rate sensor unit or the like that detects.

7 is a view for explaining a characteristic of determining a user's risk situation using a sensing unit mounted on a clothing according to an exemplary embodiment.

For example, the sensing unit 701 may be mounted on clothing mainly worn by the user. The sensing unit 701 may include a plurality of sensors for measuring the user's heart rate, and the plurality of sensors may be arranged to be located near the user's heart. The sensing unit 701 has an advantage of being able to sense an accurate heart rate since it can periodically measure the user's heart rate at the nearest position. The sensing unit 701 may transmit the measured value of the user's heart rate to the risk situation analysis apparatus 100.

8 is a view for explaining content of determining a user's risk situation by using a sensing unit mounted on a socks according to an embodiment.

For example, the sensing unit 801 may be mounted on a user's socks and sense a user's motion, skin temperature, pressure, humidity, and the like. The sensing unit 801 may include a plurality of sensors 802, and the plurality of sensors 802 may be signals for sensing different signals, respectively. For example, the sensing unit 801 may measure a force pressing the ground according to the user's walking and transmit it to the danger situation analysis apparatus 100. The danger situation analysis apparatus 100 may analyze the acquired sensing signal of the user.

9 is a diagram for explaining content of analyzing a user's bio-signal according to an exemplary embodiment.

By initiation, the apparatus 100 for analyzing a dangerous situation may extract a feature of the user's current state by using a bio-signal. A typical device for extracting features from a bio-signal searches for a local minimum point in the second derivative signal of the bio-signal, and the time corresponding to the local minimum point (T ₁ , T ₂ , T ₃ ), and The amplitudes (P1, P2, P3) of the entire bio-signal corresponding to the time (T ₁ , T ₂ , T ₃ ) may be obtained as a characteristic point.

In addition, the danger situation analysis apparatus 100 may extract a feature for detecting biometric information by using the acquired feature points. In this case, the local minimum point refers to a point in which the signal decreases when a partial section of the second derivative signal is observed and then increases again around a specific point, that is, a point having a downward convex shape.

10 is a diagram for explaining contents of acquiring an image of a user's surrounding environment by using a CCTV device around a user according to an exemplary embodiment.

The dangerous situation analysis apparatus 100 may acquire at least one CCTV image including the user's appearance from the CCTV 1001 where the user is located according to the risk level of the current state of the user determined according to the procedure described above. . In this case, the CCTV 1001 may have acquired authority to communicate with the dangerous situation analysis apparatus 100 in advance. Alternatively, the dangerous situation analysis apparatus 100 may obtain an image including the user from the CCTV in which the user is located by requesting a server such as a public institution or a building.

The dangerous situation analysis apparatus 100 may analyze at least one CCTV image and correct a risk level for the surrounding environment. For example, the apparatus 100 for analyzing a dangerous situation may determine whether the user is in a safe state or a dangerous state from an image captured at the same time of the user and the surrounding environment, and may change the risk level.

Meanwhile, according to an embodiment, the apparatus 1000 for analyzing a dangerous situation may compensate for the shape or size of an object included in the still image before analyzing the still image using the deep learning model. For example, the danger situation analysis apparatus 1000 compensates for the shape or size of an object included in at least one image, in consideration of at least one of the angle and position in which the CCTV 1001 is installed, and the shape or size is compensated. You can compare the posture of the object with a predefined posture.

For example, when an object image including an object in a posture related to a fall is detected, the danger situation analysis apparatus 1000 may include a series of images (for example, in order to accurately determine whether the state of the user 10 is a fall state). Frames accumulated for a certain period of time) may be analyzed to obtain information on a first motion change of the object. For example, when an image of the user 10 lying on the floor away from the bed (eg, the nth frame) is detected, the danger situation analysis apparatus 1000 may use the accumulated images (eg, n-10th frame). Frame ~ n+10th frame) can be analyzed to detect a change in movement of the user 10 falling from the bed to the floor. In addition, the danger situation analysis apparatus 1000 may detect a change in movement of an object after the body touches the floor.

The apparatus 1000 for analyzing a dangerous situation according to an initiation may determine whether the situation of the object included in the object image is a dangerous situation based on the setting information of the matched group. The apparatus 1000 for analyzing a dangerous situation may determine whether the state of the object is a fall state, based on the setting information of the group to which the object is matched and the information on the movement change of the object.

According to an embodiment, when the similarity between a change in motion of an object and a change in motion indicating a previously stored fall is greater than a threshold value (eg, 90%), the risk situation analysis apparatus 1000 determines the state of the object as a fall state. I can.

According to an embodiment, the “movement change indicating a previously stored fall” may be set according to setting information of a group to which an object is matched. According to an embodiment, the motion indicating the fall may be determined according to the user's profile information. For example, movement changes such as falling speed, falling shape, and shape of limbs indicating a fall of an elderly person in their 80s may be set differently from the movement change of a teenager.

On the other hand, when the similarity between the movement change of the object and the movement change indicating a previously stored fall is less than or equal to a threshold value (eg, 90%), the danger situation analysis apparatus 1000 may determine that the state of the object is not a fall state.

For example, if the change in movement of the user 10 is similar to a movement pattern according to a fall (for example, when the user 10 is no longer moving after falling from the bed to the floor), the risk situation analysis device 1000 The state of the user 10 may be determined as a fall state. However, when the movement change of the user 10 is different from the movement pattern according to the fall (e.g., when the object slides down from the bed and touches the buttocks and palms on the floor, but immediately stands up and walks), the risk situation analysis device 1000 ) May determine that the state of the user 10 is not a fall state.

Therefore, by transmitting the image of the user more objectively captured to the guardian's terminal, it is possible to help the guardian to clearly determine whether the user is safe. Furthermore, the captured image can be used as evidence later.

11 is a diagram illustrating a detailed configuration of an apparatus for analyzing a dangerous situation according to an exemplary embodiment.

As shown in FIG. 11, the apparatus 100 for analyzing a dangerous situation according to an embodiment may include a control unit 1300, a communication unit 1500, a camera 1610, and a memory 1700. However, not all of the components shown in FIG. 18 are essential components of the apparatus 100 for analyzing a dangerous situation. The danger situation analysis apparatus 100 may be implemented by more components than the components shown in FIG. 11, or the danger situation analysis apparatus 100 may be implemented by fewer components than the components shown in FIG. 11. have.

For example, as shown in FIG. 11, the apparatus 100 for analyzing a dangerous situation according to an embodiment includes a user input unit 1100, an output unit in addition to the control unit 1300, the communication unit 1500, and the memory 1700. It may further include 1200, a sensing unit 1400, and an A/V input unit 1600.

For example, the apparatus 100 for analyzing a dangerous situation according to an embodiment may basically include a control unit 1300, a user input unit 1100, and a memory 1700, and the apparatus for analyzing a dangerous situation according to an embodiment ( 100) further includes at least one of a communication unit 1500, an output unit 1200, a sensing unit 1400, and an A/V input unit 1600 in addition to the control unit 1300, the communication unit 1500, and the memory 1700. It may be a dangerous situation analysis device 100.

The user input unit 1100 refers to a means for a user to input data for controlling the dangerous situation analysis apparatus 100. For example, the user input unit 1100 includes a key pad, a dome switch, a touch pad (contact type capacitance method, pressure type resistive film method, infrared detection method, surface ultrasonic conduction method, integral type). Tension measurement method, piezo effect method, etc.), a jog wheel, a jog switch, and the like, but are not limited thereto.

The user input unit 1100 may receive a user input for controlling an operation of a module installed in the dangerous situation analysis apparatus 100.

The output unit 1200 may output an audio signal, a video signal, or a vibration signal, and the output unit 1200 may include a display unit 1210, an audio output unit 1220, and a vibration motor 1230. have. In an embodiment, the output unit 1200 may output a notification message in the form of audio, video, and/or vibration.

The display 1210 displays and outputs information processed by the dangerous situation analysis apparatus 100.

The sound output unit 1220 outputs audio data received from the communication unit 1500 or stored in the memory 1700. In addition, the sound output unit 1220 outputs sound signals related to functions (eg, a call signal reception sound, a message reception sound, and a notification sound) performed by the dangerous situation analysis apparatus 100. For example, the sound output unit 1220 may output an alarm sound for notifying that an event has occurred.

The controller 1300 generally controls the overall operation of the apparatus 100 for analyzing a dangerous situation. According to one disclosure, the control unit is also referred to as a processor. For example, by executing programs stored in the memory 1700, the control unit 1300, the user input unit 1100, the output unit 1200, the sensing unit 1400, the communication unit 1500, the A/V input unit 1600 ) And so on. In addition, the controller 1300 may perform a function of the dangerous situation analysis apparatus 100 by executing programs stored in the memory 1700. The control unit 1300 may include at least one processor. The controller 1300 may include a plurality of processors or may include one integrated processor according to its function and role. In an embodiment, the controller 1300 may include at least one processor to provide a notification message by executing at least one program stored in the memory 1700.

The sensing unit 1400 may detect a state of the dangerous situation analysis apparatus 100, a user's state, or a state around the dangerous situation analysis apparatus 100, and transmit the detected information to the control unit 1300.

The sensing unit 1400 includes a magnetic sensor 1410, an acceleration sensor 1420, a temperature/humidity sensor 1430, an infrared sensor 1440, a gyroscope sensor 1450, and a position sensor. (For example, a GPS) 1460, an atmospheric pressure sensor 1470, a proximity sensor 1480, and an RGB sensor 1490 may be included, but the present invention is not limited thereto. Since the function of each sensor can be intuitively inferred by a person skilled in the art from its name, detailed description will be omitted.

The communication unit 1500 may include one or more components that allow the dangerous situation analysis device 100 to communicate with the other device 100-2 and the other server 300. For example, the communication unit 1500 may include a short range communication unit 1510, a mobile communication unit 1520, and a broadcast reception unit 1530. The other user device may be a computing device or a sensing device, but is not limited thereto.

Short-range wireless communication unit 1500 (short-range wireless communication unit) 1510, Bluetooth communication unit 1500, BLE (Bluetooth Low Energy) communication unit 1500, short-range wireless communication unit 1500 (Near Field Communication unit), WLAN ( Wi-Fi) communication unit 1500, Zigbee communication unit 1500, infrared data association (IrDA) communication unit 1500, Wi-Fi Direct (WFD) communication unit 1500, ultra wideband (UWB) communication unit 1500 , Ant+ communication unit 1500, etc. may be included, but is not limited thereto.

The mobile communication units 1500 and 1520 transmit and receive wireless signals with at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include a voice call signal, a video call signal, or various types of data according to transmission/reception of text/multimedia messages.

The broadcast receiver 1530 receives a broadcast signal and/or broadcast-related information from outside through a broadcast channel. Broadcast channels may include satellite channels and terrestrial channels. Depending on implementation examples, the apparatus 100 for analyzing a dangerous situation may not include the broadcast receiving unit 1530.

In an embodiment, the communication unit 1500 may receive a video sequence including a plurality of frames from a camera installed in the dangerous situation analysis apparatus 100. In an embodiment, the communication unit 1500 may transmit a command for controlling an operation of a module installed in the apparatus 100 for analyzing a dangerous situation to a module installed in the apparatus 100 for analyzing a dangerous situation.

The A/V (Audio/Video) input unit 1600 is for inputting an audio signal or a video signal, and may include a camera 1610 and a microphone 1620. The camera 1610 may obtain an image frame such as a still image or a video through an image sensor in a video call mode or a photographing mode. The image captured through the image sensor may be processed by the controller 1300 or a separate image processing unit (not shown). For example, an image captured by the camera 1610 may be used as information for determining whether an event has occurred.

The microphone 1620 receives an external sound signal and processes it as electrical voice data. For example, the microphone 1620 may receive an acoustic signal from an external device or a user. The microphone 1620 may use various noise removal algorithms for removing noise generated in a process of receiving an external sound signal.

The memory 1700 may store a program for processing and control of the controller 1300, and may store data input to the dangerous situation analysis apparatus 100 or output from the dangerous situation analysis apparatus 100.

The memory 1700 is a flash memory 1700 type (flash memory type), a hard disk type (hard disk type), a multimedia card micro type (multimedia card micro type), a card type memory 1700 (for example, SD or XD memory (1700), RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read- Only Memory), a magnetic memory 1700, a magnetic disk, and an optical disk.

Programs stored in the memory 1700 can be classified into a plurality of modules according to their functions. Here, the plurality of modules are software, not hardware, meaning modules that operate functionally. For example, a UI module ( 1710), a touch screen module 1720, a notification module 1730, and the like.

The UI module 1710 may provide a specialized UI, a GUI, etc. linked with the apparatus 100 for analyzing a dangerous situation for each application. The touch screen module 1720 may detect a user's touch gesture on the touch screen and transmit information on the touch gesture to the controller 1300. The touch screen module 1720 according to an embodiment may recognize and analyze a touch code. The touch screen module 1720 may be configured as separate hardware including a controller.

The notification module 1730 may generate a signal for notifying the occurrence of an event. The notification module 1730 may output a notification signal in the form of a video signal through the display unit 1210, may output a notification signal in the form of an audio signal through the sound output unit 1220, and the vibration motor 1230 It is also possible to output a notification signal in the form of a vibration signal through.

The method according to an embodiment of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the present invention, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler but also high-level language codes that can be executed by a computer using an interpreter or the like.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements by those skilled in the art using the basic concept of the present invention defined in the following claims are also provided. It belongs to the scope of rights.

Claims (11)

Sensing a user's bio-signal using at least one sensor every predetermined period;
Matching the surrounding environment of the user with any one of a plurality of state groups classified according to the user's activity attribute when the change in the user's bio-signal exceeds the threshold value;
Determining a risk level for the surrounding environment based on the matched state group setting information and a user's bio-signal; And
Including the step of transmitting a notification message to the terminal of the guardian according to the determined risk level,
How to determine risk situations and manage user safety. The method of claim 1,
The step of determining the risk level for the surrounding environment,
Driving a camera to obtain at least one image of the user's surrounding environment;
Analyzing at least one image included in the surrounding environment and acquiring information on a change in movement of a person and an object included in the surrounding environment over time; And
Including the step of determining a risk level for the surrounding environment by analyzing a change in the user's bio-signal corresponding to the information on the change in the movement of the person and the object,
How to determine risk situations and manage user safety. The method of claim 2,
When the heart rate of the user's body temperature information, sweat emission information, skin characteristic information, heart rate information, blood pressure information, and activity information changes faster than a threshold value according to a change in movement of people and objects included in the surrounding environment , Comprising the step of increasing the risk level for the surrounding environment,
How to determine risk situations and manage user safety. The method of claim 1,
Creating a state group by using profile information for a plurality of users having similar activity attributes; And
Determining setting information for determining a risk level of the surrounding environment matching the state group; and
The profile information includes gender, age, lesion, usual activity amount, age-specific language use information, walking speed, activity pattern, stride length, whether or not an assistive device is installed, stress index, mental empathy ability, emotional expression ability,
The setting information is average information of profile information for a plurality of users used to create the state group,
How to determine risk situations and manage user safety. The method of claim 1,
Acquiring at least one CCTV image including the appearance of the user from the CCTV in which the user is located according to the determined risk level; And
Analyzing the at least one CCTV image, further comprising the step of correcting the risk level for the surrounding environment,
How to determine risk situations and manage user safety. The method of claim 1,
The step of sensing the user's bio-signal,
A temperature sensor that measures the user's body temperature, a humidity sensor that measures the user's sweat discharge, a reflective optical sensor that measures the user's skin characteristics, a heart rate sensor that measures the user's heart rate, and the user's blood pressure. Sensing a biosignal of a user using at least one of a blood pressure sensor, an impedance sensor measuring an impedance signal in the user's body, and a motion sensor measuring an activity of the user; And
In consideration of at least one of the user's body temperature information, sweat emission information, skin characteristic information, heart rate information, blood pressure information, impedance information, and activity information, generating bio-signal information of the user in real time,
How to determine risk situations and manage user safety. The method of claim 1,
Transmitting a notification message to the guardian's terminal according to the determined risk level,
When the risk level is 1, first notification information including information on a dangerous situation is transmitted,
When the danger level is 2, transmitting the first notification information and second notification information including an image of the surrounding environment for the dangerous situation,
When the risk level is 3, transmitting second notification information and third notification information including an image of a dangerous situation to the guardian's terminal,
When the risk level is 4, transmitting the third notification information and attempting a video call to the guardian's terminal at the same time,
How to determine risk situations and manage user safety. The method of claim 1,
The step of determining the risk level for the surrounding environment,
Performed by a learning network model driven by an artificial intelligence (AI) engine,
How to determine risk situations and manage user safety. The method of claim 1,
The user is a child aged 0 to 10 years,
The plurality of state groups include an exercise state group, a sleep state group, a meal state group, a rest state group, and a learning state group,
The setting information of the plurality of state groups is determined based on data for monitoring the activity of the child,
How to determine risk situations and manage user safety. The method of claim 1,
The user is an elderly person over 65 years of age,
The plurality of state groups include an exercise state group, a sleep state group, a meal state group, a rest state group, and a learning state group,
The setting information of the plurality of state groups is determined based on data learned based on the age and medical history information of the elderly,
How to determine risk situations and manage user safety. Processor; And
A memory storing instructions executable by the processor; Including, the processor,
By executing the above commands,
Sensing the user's bio-signal using at least one sensor every predetermined period,
When the change in the user's biometric signal exceeds the threshold value, the user's surrounding environment is matched to any one of a plurality of state groups classified according to the user's activity attribute,
Based on the setting information of the matched state group and the user's biosignal, a risk level for the surrounding environment is determined,
Transmitting a notification message to the guardian's terminal according to the determined risk level,
A device that judges dangerous situations and manages the safety of users.

Images