KR102546045B1 - monitering system with LiDAR for a body - Google Patents

Abstract

The present invention is to extract the boundary value of the human body or the lines following the human body from the data collected through the lidar sensor, thereby significantly reducing the amount of processed data to quickly detect an emergency situation while protecting the privacy of the individual. Specific features include a LiDAR (Light Detection And Ranging) sensor for emitting a laser to detect a human body and receiving a reflected laser; 3D image information obtained by scanning the location information of the reflected laser beam collected from the LIDAR sensor is obtained, and the image information is processed and based on the processed image information, the patient needs help by an artificial intelligence method. a monitoring controller that determines whether an emergency has occurred and propagates the emergency to the outside; It is connected to the monitoring controller through a wired or wireless communication network and includes a management server that propagates the situation to related organizations.

Description

Human body monitoring device using LiDAR {monitoring system with LiDAR for a body}

The present invention is used by a large number of people, such as a public toilet, a public bathhouse, an interior of an autonomous vehicle, or a living space for elderly patients, so that a quick response is possible when an emergency occurs without invading the privacy of a person in need of protection. It relates to a human body monitoring device that monitors the human body by installing LiDAR (Light Detection and Ranging) in space.

Today, security devices such as closed-circuit TVs and cameras are installed in countless places to make it easy to monitor or take action in areas where accidents occur. However, assaults, theft, falls, or myocardial infarction or stroke that can occur in places where security equipment such as closed-circuit TVs or cameras cannot be installed due to invasion of privacy, even though many people use it, such as shared toilets, public baths, and shared changing rooms. It may occur to patients or the elderly who need protection with symptoms such as those, or to those who are riding alone in an upcoming autonomous vehicle, and may require quick response to emergency situations along with privacy protection.

In this case, LiDAR is a collection of points (Point Cloud) that has x, y, z data values, and it can be modeled as a stereoscopic image in real time by using it, so you can get immediate results at a level that does not directly expose privacy like a camera. situation can be grasped. However, it cannot be checked directly with the eyes like a camera, and it is difficult for a person to directly check a lot of data, so when a specific movement is detected through artificial intelligence deep learning using big data, it is possible to send a text message or call to confirm to the person, or to an emergency center. Methods are being devised so that measures can be taken by contacting the police department or other related organizations.

Registration No. 10-1956009 (name of invention: patient location tracking system using LiDAR) (hereinafter referred to as 'prior art') is also a device for protecting the privacy of a patient while monitoring the patient's movement using lidar .

1 is a block diagram for explaining the overall configuration of the prior art.

The monitoring system shown in FIG. 1 acquires three-dimensional depth information by attaching one or more rotating LIDAR sensors 100 to a ceiling in a hospital. By tracking a moving patient with the acquired 3D depth information, estimating the patient's position and current condition and displaying them on the monitor, unlike conventional image management systems using cameras, without invading the patient's privacy or privacy. It also has a feature that can easily manage patients.

Here, LIDAR (Light Detection And Ranging) refers to recognizing the surroundings in 3D using 3D laser light. That is, the lidar sensor irradiates laser light to the patient's surroundings instead of radio waves to recognize 3D information about the patient and accurate direction and distance. Because these lidars are more accurate than sound waves and use the rotation of mirrors, they can scan in multiple directions at once.

In addition, the LiDAR sensor 100 is a non-contact method using a mirror and image processing, and is a means for measuring the three-dimensional shape and dimension of the patient, Coordinates of the shape of the patient can be acquired, and color information (texture) can be simultaneously acquired according to characteristics. While a commonly used scanner is a machine that copies two-dimensional pictures or documents, lidar refers to a device that can measure a patient's three-dimensional shape and dimension.

Here, the color information (Texture) is texture mapping (Texture Mapping), which is a term mainly used in 3D programs. Usually, when modeling in 3D at first, it only looks gray, but it refers to adding color or texture to modeling.

The scan station (Scan Station, 200) is a means for scanning and matching the shape information according to the positional movement of the patient received through the lidar sensor 100 into three-dimensional coordinate value data, and the lidar sensor 100 ), when a laser beam is emitted toward the patient, the beam is reflected from the patient and returns to the lidar sensor 100 again. The LiDAR sensor 100 measures the distance between the scanner and the patient by calculating the time when the beam is emitted and returns. The measured area of lidar sensors is generally 100 m or more, and the measurement speed is 1,000,000 point/sec or more. In addition, the accuracy of the measurement is less than 3mm at a distance of 50m, and the scan angle is more than 0° vertically and 0° 360° horizontally. And since the lidar sensor fires laser beams about 50,000 times per second, lasers can be fired on the entire patient's surface in a matter of minutes, and the patient's shape can be obtained as 3D data. However, since the laser has straightness, it cannot measure the invisible surface. Accordingly, the entire data can be obtained by scanning the patient through mirrors in various directions according to prior planning and necessity. In the registration of the scan data, data scanned at each point where the lidar sensor 100 is installed is acquired in the form of point cloud data. Depending on the range of the scanning target, a process of combining each acquired data into one is required, and this process is registration.

In addition, the filter 300 is a means for removing interference and noise included in the three-dimensional coordinate values, which are shape information according to the movement of the patient's position scanned through the scan station 200.

In addition, the location tracking device 400 obtains the strength of the reflected signal for each wavelength and the 3D location coordinate information of the measurement point for the 3D coordinate value, which is shape information according to the patient's location movement from which interference and noise have been removed through the filter. Decide. In addition, it is a means of accurately recognizing an object by classifying the object and the index, and tracking the exact location according to the movement of the object. In addition, data tracking the location of the patient is transmitted to the monitor of the management center 500 .

According to this prior art, the 3D image processed by the data detected by the lidar sensor cannot be called a real image like the image of a camera, but because it is very close to the real image, not only the movement of the patient but also the actual appearance of the patient In addition to being captured, the patient's real-life movements are exposed, so it can be said that the patient's privacy is very insufficient.

In addition, in the prior art in which data processing for 3D images containing color information is performed, a large amount of data to be processed by the system increases and the processing speed is delayed, which can cause problems in real-time monitoring of patients. However, there is a problem that expensive software must be installed.

The present invention has been devised in view of these problems, and the solution of the present invention is to extract the boundary value of the human body or the lines following the human body from the data collected through the LIDAR sensor, thereby significantly reducing the amount of processed data. This is to ensure that the privacy of individuals is protected while promptly detecting them.

The solution to solve the above problem is in a human body monitoring device using LiDAR that accurately detects the human body and allows privacy protection: emits a LiDAR signal that detects the human body, and the reflected LiDAR LiDAR (Light Detection And Ranging) sensor for receiving a signal; 3D image information obtained by scanning the positional information of the reflected lidar beam collected from the lidar sensor is acquired, and the processed image information is analyzed by processing the image information to be used in an emergency situation in which the patient needs help. a monitoring controller that determines whether it is missing and propagates an emergency to the outside; It includes a management server connected to the monitoring controller by a wired/wireless communication network and propagating the situation to related organizations, and the monitoring controller controls the connected control target and performs calculation processing to take charge of the OS (Operating System) of the system. control module; The scanning image of each reflection point in the room received by the lidar sensor is acquired frame by frame in real time, and a human body image part as a moving object is extracted by comparing the scanning images before and after the frames, and an edge image is obtained from the human body image part an edge image extraction unit for extraction; a center line extraction unit extracting a center line of the human body from the edge image extracted by the edge image extraction unit; and an emergency determination module for comparing a specific movement of the center line extracted by the center line extraction unit with a preset movement to determine whether it is an emergency situation, wherein the center line extraction unit determines the middle between edge lines in the width direction. The connecting line is extracted as the center line.

delete

delete

delete

According to the present invention having the above problem and the above solution, by determining the motion of the human body based on the image signal input by the lidar sensor, the privacy of the individual is protected, and the scanning image formed by the lidar sensor An edge line is formed from the edge line, and the emergency situation of the human body is determined based on the minimized data by extracting the center line, thereby making a quick emergency judgment, so that the production cost and detection speed are greatly improved.

1 is a block diagram for explaining the overall configuration of the prior art.
2 is a block diagram for explaining the overall configuration of the present invention.
3 is a three-dimensional point map of location data scanned by the laser beam of the present invention.
FIG. 4 is a map obtained by extracting the edge line of FIG. 3 .
FIG. 5 is a map obtained by extracting the center line of the human body of FIG. 4 .

The present invention collects data (Point Cloud) of a number of points on the human body and uses the collected data as big data to prevent emergency situations such as fights, theft, accidents, and falls with AI (Artificial Intelligence). It recognizes and checks safety through speakers and microphones connected to lidar equipment, and when a problem occurs, it is to contact the police station or the family emergency center so that necessary measures can be taken. It can be used in various fields, For convenience of description, application for monitoring a patient requiring protection will be described as an example, but the application scope of the present invention is not limited thereto.

Figure 2 is a block diagram for explaining the overall configuration of the present invention, Figure 3 is a three-dimensional point map of the location data scanned by the laser beam of the present invention, Figure 4 is a map extracted from the edge line of Figure 3, FIG. 5 is a map obtained by extracting the center line of the human body of FIG. 4 .

The patient monitoring system of the embodiment shown in FIG. 2 includes a LIDAR (Light Detection And Ranging) sensor 11 that irradiates a laser beam into a room and collects a laser beam reflected from a patient, and a LIDAR sensor 11 Acquire 3D image information obtained by scanning the location information of the reflected laser beam collected from It consists of a monitoring controller 20 that determines whether or not and propagates an emergency situation to the outside, and a management server 30 that propagates the situation to related organizations by being connected to the monitoring controller 20 and wired or wireless communication networks.

The monitoring controller 20 controls the connected control object and performs calculation processing to control the control module 201 in charge of the operating system (OS) of the system, and the scanning image of each reflection point in the room received by the lidar sensor 11 An edge image extraction unit 202 that acquires in real time frame by frame, compares the before and after scanning images of the frames to extract a human body image part as shown in Figure 3, which is a moving object, and extracts an edge image for the human image part. , The center line extractor 203 that forms the center line of the human body from the edge image extracted from the edge image extractor 202, and the specific movement of the center line extracted from the center line extractor 203 is artificial intelligence learning It includes an emergency situation judgment module 204 that determines whether it is an emergency situation by the module, and when it is determined that it is an emergency situation in the emergency situation judgment module 204, the emergency situation is propagated to the external environment including the management server 201. .

The edge image extraction unit 202 applies a generally used Sobel filter to detect an edge to the image of FIG. 3 and detects a pixel whose change rate is greater than or equal to a set change rate to form an edge line as shown in FIG. 4 .

Also, the center line extraction unit 203 extracts a line connecting the middle between edge lines as a center line in the width direction.

In addition, the center line extraction unit 203 is the head (A), hands (B), (C), feet (D), (E) of the human body from the edge line of FIG. 4 by big data and artificial intelligence (A.I) techniques. It can be schematized as shown in FIG. 5 by determining the location of and defining a line connecting them as a center line.

In addition, the emergency situation determination module 204 detects the movement of the central line of the human body in FIG. 5 and determines whether the movement pattern matches a preset emergency situation pattern to determine an emergency situation.

Although various emergency situations can be extracted by the movement of the center line, illustratively, the first state in which the head (A), feet (D), and (E) of the target object (assumed to be a patient) is standing at a height of 150 cm or more. When the time difference between the head (A), the feet (D), and (E) is converted to the second state in which the head (A), feet (D), and (E) lie down by 20 cm or less occurs within a preset time difference, the emergency judgment module 204 determines that the patient is mentally ill. Losing and falling can be judged as an emergency.

The emergency situation determination module 204 determines that the change rate of the movements of the hands (B) and (C) exceeds a predetermined range in this emergency situation, as a mild emergency, and the movements of the hands (B) and (C) are determined. When moving within range, it can be subdivided into more severe cases.

In this way, the emergency judgment module 204 can be designed in various ways according to the designer's intention, and it is preferable to configure it so that it can be learned by artificial intelligence using big data.

Although the embodiments of the present invention are described using a patient as a model, it is natural that the present invention can be applied to various fields such as detection of crimes that may occur in public toilets, etc., or remote monitoring of elderly people living alone, as described above. and the scope of protection of the present invention should be determined by the claims described below.

11: lidar sensor
20: watchdog controller
201: control module
202: edge image extraction unit
203: center line extraction unit
204: emergency judgment module

Claims (4)

In the human body monitoring device using LiDAR that accurately detects the human body and enables privacy protection:
A LiDAR (Light Detection And Ranging) sensor for emitting a lidar signal for detecting a human body and receiving a reflected lidar signal;
3D image information obtained by scanning the location information of the reflected lidar beam collected from the lidar sensor is acquired, and the processed image information is analyzed by processing the image information to be used in an emergency situation in which the patient needs help. a monitoring controller that determines whether it is missing and propagates an emergency to the outside;
It includes a management server that is connected to the monitoring controller through a wired and wireless communication network and propagates the situation to related organizations,
The monitoring controller
A control module that controls the connected control object and performs calculation processing to be in charge of the OS (Operating System) of the system;
Acquire the scanning image of each reflection point in the room received by the lidar sensor frame by frame in real time, compare the before and after scanning images of the frames to extract the human body image part, which is a moving object, and obtain an edge image from the human body image part an edge image extraction unit for extraction;
a center line extraction unit extracting a center line of the human body from the edge image extracted by the edge image extraction unit;
An emergency judgment module for comparing the specific motion of the center line extracted from the center line extractor with a preset motion to determine whether it is an emergency situation,
The center line extraction part
In the width direction, a human body monitoring device using LiDAR, characterized in that a line connecting the middle between the edge lines is extracted as a center line. delete delete delete

Images