KR102052883B1 - Prediction system using a thermal imagery camera and fall prediction method using a thermal imagery camera - Google Patents

Abstract

According to an embodiment, there is provided a method of controlling a fall prediction system using a thermal imaging camera, the method including: photographing a person with a thermal imaging camera; Analyzing each pixel data constituting the photographed image; Obtaining x-axis histogram and y-axis histogram data of the photographed image based on the pixel data; Determining whether a person is in a stable state or a fall risk state based on the x-axis histogram and the y-axis histogram data; And operating the notification device based on the state of the person.

Description

Fall prediction system using a thermal imaging camera and fall prediction method using a thermal imaging camera {FREDICTION SYSTEM USING A THERMAL IMAGERY CAMERA AND FALL PREDICTION METHOD USING A THERMAL IMAGERY CAMERA}

The following description relates to a fall prediction system using a thermal imaging camera and a fall prediction method using a thermal imaging camera.

Falling does not mean that the patient suddenly falls down due to stun or the like and falls down by external force, but suddenly falls to a lower position due to unintentional change of position regardless of the patient's intention. Falls in hospitalized patients lead to medical accidents in serious rooms. If a nurse or patient can detect a fall in advance, the accident can be prevented.

Kim, Emily Ang Neo, et al. "Evaluation of three fall-risk assessment tools in an acute care setting." According to the Journal of advanced nursing 60.4 (2007): 427-435., 38% of the risks to health care in the US are attributed to falls.

Kim, Chul-Gyu, and Moon-Ja Suh. "An analysis of fall incidence rate and its related factors of fall in inpatients." According to Quality Improvement in Health Care 9.2 (2002): 210-228., It can be seen that 30% of Korean hospital case reports are caused by falls. In addition, as a result of the fall rate survey, 72.4% of the falls occurred in the ward, and 63% of the falls occurred in the bed.

There are many scenarios in which falls can occur. For example, when a patient climbs up and down a bed, when he lowers the auxiliary railing and up and down a bed, when he leans on his body without knowing that the railing is down, when he goes over the bed without knowing that the railing is up, when he is sleeping, When slipping on the floor, falls can occur.

Patient fall prediction systems using conventional cameras have also observed patients through RGB cameras. However, the RGB camera has a problem in that it may cause a privacy problem of the patient because the image of the patient as it is displayed in high resolution.

An object of one embodiment is to provide a patient fall prediction system that can predict the patient's fall with high performance as well as protect the privacy of the patient.

According to one embodiment, a fall prediction system using a thermal imaging camera, a thermal imaging camera for photographing a person; A controller which analyzes each pixel data constituting the image photographed by the thermal imaging camera and determines whether a person is at risk of falling; And a display configured to receive the information on the risk of falling from the controller and display the information externally, wherein the controller obtains the x-axis histogram and the y-axis histogram data of the photographed image based on the pixel data. The fall risk may be determined based on the x-axis histogram and the y-axis histogram data.

The pixel data may include pixel brightness information.

The controller may store reference data for distinguishing between a stable state and a fall risk state of a person and compare pixel data of the photographed image with the reference data to determine whether a person is at risk of falling.

The controller may determine whether a person is at risk of falling by comparing pixel data of the photographed image with the reference data using a support vector machine (SVM).

According to an embodiment of the present disclosure, a fall prediction method using a thermal imaging camera may include photographing a person with a thermal imaging camera; Analyzing each pixel data constituting the photographed image; Obtaining x-axis histogram and y-axis histogram data of the photographed image based on the pixel data; Determining whether a person is in a stable state or a fall risk state based on the x-axis histogram and the y-axis histogram data; And operating the notification device based on the state of the person.

The pixel data may include pixel brightness information.

The fall prediction method using the thermal imaging camera may further include providing a criterion for distinguishing between a stable state and a fall risk state of a person.

The preparing of the reference may include photographing a reference image with a thermal imaging camera; Obtaining brightness data of each pixel constituting the reference image; And analyzing the x-axis histogram and the y-axis histogram of the reference image.

Determining whether the person is in a stable state or a fall risk state may be performed by a support vector machine (SVM).

According to an embodiment, the method of controlling the fall prediction system using the thermal imaging camera may protect the privacy of the patient using the thermal imaging camera and prevent the fall of the patient with high performance.

In addition, the fall prediction system control method using the thermal imaging camera does not need to attach a separate sensor to the patient body, it can be conveniently used.

1 is a diagram schematically illustrating a state in which a falls prediction system is used, according to an exemplary embodiment.
FIG. 2 is a diagram illustrating an experimental space in which an environment similar to a hospital room environment is constructed in order to acquire data using a thermal imaging camera.
3 is a block diagram of a fall prediction system using a thermal imaging camera, according to an exemplary embodiment.
4 is a flowchart of a fall prediction method using a thermal imaging camera according to an exemplary embodiment.
FIG. 5 is a detailed flowchart of a step of providing a criterion for distinguishing between a stable state and a risk of falling in a fall in a fall prediction method using a thermal imaging camera according to an exemplary embodiment.
6 is a diagram illustrating six scenarios in which a person sleeps safely.
7 is a view showing an image taken with a thermal imaging camera of a patient lying in a stable state.
8 is a view showing an image taken with a thermal imaging camera of a patient lying in a risk of falling.
9 is a diagram illustrating a process of obtaining an x-axis histogram and a y-axis histogram after normalization of an image photographed by a thermal imaging camera to a predetermined size.
10 is a diagram illustrating the accuracy of the fall prediction system.

Hereinafter, some embodiments of the present invention will be described in detail with reference to the accompanying drawings. In adding reference numerals to the components of each drawing, it should be noted that the same reference numerals are assigned to the same components as much as possible even though they are shown in different drawings. In addition, in describing the embodiments of the present invention, when it is determined that a detailed description of a related well-known configuration or function interferes with the understanding of the embodiments of the present invention, the detailed description thereof will be omitted.

In addition, in describing the components of the embodiment of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are only for distinguishing the components from other components, and the nature, order or order of the components are not limited by the terms. If a component is described as being "connected", "coupled" or "connected" to another component, that component may be directly connected or connected to that other component, but between components It will be understood that may be "connected", "coupled" or "connected".

FIG. 1 is a diagram schematically illustrating a state in which a fall prediction system is used according to an embodiment, FIG. 2 is a diagram illustrating an experimental space in which a sickroom environment is constructed to acquire data using a thermal imaging camera, and FIG. 3. Is a block diagram of a fall prediction system using a thermal imaging camera according to an exemplary embodiment.

1 to 3, a fall prediction system using a thermal imaging camera may include a thermal imaging camera 11, a controller 12, and a display 10.

The thermal imaging camera 11 may be installed in a room R in which a person P is lying. For example, the thermal imaging camera 11 may be installed on the ceiling of the room R. The person P may be lying in the bed B provided in the room R, and the thermal imaging camera 11 may photograph the person P on the bed B. FIG. The image photographed from the thermal imaging camera 11 may be transferred to the controller 12.

The controller 12 may determine whether or not a person P falls by analyzing each pixel data constituting the image photographed by the thermal imaging camera. The controller 12 may distinguish whether the person P is in a stable state or a fall risk state. The controller 12 may transmit information on whether the fall is dangerous to the display 10.

The display 10 may receive information about whether a fall is dangerous from the controller 12 and display the information externally. For example, the fall prediction system may detect the fall risk of the patient in advance, and notify the nurse or the patient if the patient is in a fall risk state. The display 10 may be installed at a position that can be always checked by a nurse or a hospital staff. Nurses or hospital staff can locate patients at risk of falling and help them avoid falling. In addition, the display 10 may be installed at a position that can be confirmed by the patient. If the patient confirms via display 10 that he or she is at risk of falling, the patient may call a nursing or hospital staff. For example, the fall prediction system may include an alarm siren as an audible notification means. The siren can make a sound when the patient is at risk of falling. Sirens can be installed where nurses or hospital staff can hear the sirens at all times.

In the room (R) shown in FIG. 2, 1920 images of a fall risk state and 1260 images of a safe state were photographed, and based on this, the accuracy of the fall prediction system was determined. In the room R, the left side of the person P may be blocked by a wall. A person can be in danger of falling to the right.

4 is a flowchart illustrating a fall prediction method using a thermal imaging camera according to an embodiment, and FIG. 5 is a diagram illustrating a fall prediction method using a thermal imaging camera according to an embodiment. A detailed flow chart of the steps for establishing the criteria.

4 and 5, the fall prediction method using the thermal imaging camera includes preparing a criterion for distinguishing between a stable state and a danger of falling of a person (S100), and photographing a person's state with a thermal imager. (S300), analyzing the respective pixel data constituting the photographed image (S300), and obtaining x-axis histogram and y-axis histogram data of the photographed image based on the pixel data (S400). And determining, based on the x-axis histogram and the y-axis histogram data, whether the person is in a stable state or a fall risk state (S500), and operating the notification device based on the state of the person (S600). ) May be included.

In operation S100, the controller may prepare a criterion for distinguishing between a stable state and a fall risk state of a person. For example, the user may input reference data in advance to the controller. Alternatively, the reference data may be input through an artificial intelligence method in real time. The method of preparing a reference may include taking a reference image with a thermal imaging camera, obtaining brightness data of each pixel constituting the reference image, and analyzing an x-axis histogram and a y-axis histogram of the reference image. It may include. In this manner, the controller may store data about a state in which a person lies in a normal state in advance.

In operation S200, the thermal imaging camera may photograph a person's figure with the thermal imaging camera.

In operation S300, the controller may analyze each pixel data constituting the photographed image. The pixel data may include pixel brightness information.

In operation S400, the controller may obtain x-axis histogram and y-axis histogram data of the captured image based on the pixel data.

In operation S500, the controller may determine whether a person is in a stable state or a fall risk state based on the x-axis histogram and the y-axis histogram data. The control unit may determine whether the person is in a stable state or a fall risk state with the support vector machine (SVM).

In operation S600, the controller may operate the notification device based on the state of the person.

6 is a diagram illustrating six postures in which a person sleeps safely.

Referring to FIG. 6, there may be six positions in which a person sleeps safely. There may be a fetus, a log, a yearner, a soldier, a freefaller, and a starfish.

7 is a view showing an image of a patient lying in a stable state with a thermal imaging camera.

Referring to FIG. 7, the fall prediction system may classify a stable state of a patient into nine, and store an image for each state. The patient's steady state may be a condition in which the patient is safe from falls on the bed. For example, the patient may be in a stable state lying in the center of the bed.

The nine postures are as follows. First, there are six postures: fetal type, log type, craving type, soldier type, free fall type and starfish type. In addition, fetal type, log type and craving type are added to the posture that the person is lying in the opposite direction, respectively, three postures are added. Therefore, there are 9 postures in total.

Referring to FIG. 7, the fetal type is shown in S1 and S2, the log type is shown in S3 and S4, the craving type is shown in S5 and S6, and the soldier type, free-fall type and starfish type are S7 and S8, respectively. And S9.

The fall prediction system may store an image for each of the nine steady states. For example, the fall prediction system may capture 20 images for each of the seven people in a stable state, thereby obtaining a total of 1260 images of a fall risk state.

8 is a view showing an image taken with a thermal imaging camera of a patient lying in the danger of falling.

Referring to FIG. 8, the fall prediction system may classify a patient's fall risk state into 14 and store an image of each state. The patient's risk of falling may be a condition where the patient can fall on a bed. For example, if there is a risk of falling to the right side of the bed, the patient lying on the right side of the bed may be at risk of falling.

The 14 postures are as follows. First, there are six postures: fetal type, log type, craving type, soldier type, free fall type and starfish type. In addition, fetal type, log type and craving type are added to the posture that the person is lying in the opposite direction, respectively, three postures are added. In addition, one posture in which the patient's leg falls out of bed is added. In addition, one posture with arms out of bed in a soldier and one posture with legs out of bed are added. In addition, in the free-fall type, one posture in which the legs fall out of bed is added. In addition, one posture in which a leg falls out of bed in a starfish type is added. Thus, there are a total of 14 postures for the patient's fall risk state. Craving, free-falling and starfish types were excluded because the patient would naturally fall out of bed if the patient was lying at the end of the bed.

The fall prediction system may store an image for each of the 14 fall risk states. For example, the fall prediction system may capture 20 images for each fall risk state for 7 people, thereby obtaining a total of 1920 images of fall risk states.

9 is a diagram illustrating a process of obtaining an x-axis histogram and a y-axis histogram after normalizing the image photographed by the thermal imaging camera to a predetermined size.

Referring to FIG. 9, the fall prediction system may analyze an image obtained through a thermal imaging camera.

First, the fall prediction system may cut the acquired image to an appropriate size. For example, one side of the bed may be walled so that there is no risk of falling, and the other side of the bed may be walled, so there may be a risk of falling. For example, the right side of the patient may be walled on FIG. 6. In this case, the fall prediction system can prevent unnecessary data processing by cutting off the wall portion.

Next, the fall prediction system may obtain brightness data for each pixel constituting the image. For example, if the resolution of the image is 640 * 480, the fall prediction system may obtain brightness data of 30, 7,200 pixels.

Next, the fall prediction system may obtain the x-axis histogram and the y-axis histogram of the image based on the brightness data of the pixel. Each pixel constituting the image captured by the thermal imaging camera may have a different brightness depending on the temperature. The pixel may have a value from 0 to 255 depending on the brightness. By adding brightness data of all pixels having the same x-coordinate, the value of the x-coordinate may be determined, and an x-axis histogram may be obtained based on the value. In a similar manner, brightness data of all pixels having the same y-coordinate may be added to determine the value of the y-coordinate and a y-axis histogram may be obtained based on the value.

Next, the fall prediction system may normalize the x-axis histogram and the y-axis histogram. By normalizing each histogram, the data processing speed of the fall prediction system can be improved. For example, in the case of 640 * 480, the x-axis histogram may include 640 data. For example, the fall prediction system may set a minimum value of 0 and a maximum value of 1 among 640 x-axis data to normalize all data values to values between 0 and 1.

Next, the fall prediction system may perform a support vector machine (SVM) based on the normalized x-axis histogram and y-axis histogram data. The SVM enables the SVM to learn the criteria for dividing a patient's steady state and fall risk state. The fall prediction system can distinguish, based on the learned criteria, whether the patient is in a stable state or a fall risk state.

For example, the patient's appearance may be taken by a thermal imaging camera in real time. The captured image can be cropped to a suitable size. Thereafter, brightness data of each pixel constituting the image can be obtained. Then, based on the brightness data of the pixel, the x-axis histogram and the y-axis histogram of the image can be obtained. The x-axis histogram and the y-axis histogram can be normalized. The fall prediction system can distinguish between a patient's stable state and a fall risk state based on normalized data.

10 is a diagram illustrating the accuracy of the fall prediction system.

Figure 10 is a table showing the results of one experiment, the experiment was carried out in the following manner.

Fall prediction accuracy experiments were performed using only the x-axis histogram. Through the SVM, the fall prediction system was trained on the x-axis histogram of a randomly selected 70% of the 1920 falling images and 1260 stable images. The x-axis histogram was then analyzed for the remaining 30% of the image.

Next, the fall prediction accuracy experiment was performed using only the y-axis histogram. Through the SVM, the fall prediction system was trained on the y-axis histogram for a randomly selected 70% of the 1920 falling images and 1260 stable images. The y-axis histogram was then analyzed for the remaining 30% of the image.

Finally, the fall prediction accuracy experiment was performed using the x-axis histogram and the y-axis histogram. Through the SVM, the fall prediction system was trained on the x-axis histogram and the y-axis histogram for a randomly selected 70% of the 1920 falling images and 1260 stable images. Then, the x-axis histogram and the y-axis histogram were analyzed for the remaining 30% of the image.

As a result, the accuracy is 99.41% when the fall prediction is performed with the x-axis histogram, and the accuracy is 74.89% when the fall prediction is performed with the y-axis histogram. It was found to be 99.70%.

The embodiments described above are merely to describe the preferred embodiments of the present invention, the scope of the present invention is not limited to the described embodiments, the technical spirit and patents of the present invention by those skilled in the art Various modifications, variations, or substitutions may be made within the scope of the claims, and such embodiments should be considered to be within the scope of the present invention.

Claims (9)

In the fall prediction system, which predicts a danger that a person lying on a bed falls out of bed,
Thermal imaging cameras for photographing people;
A controller which analyzes each pixel data constituting the image photographed by the thermal imaging camera and determines whether a person is at risk of falling; And
It includes a display for receiving the information on the risk of falling from the control unit to display to the outside,
The pixel data includes pixel brightness information,
The control unit,
Based on the pixel data, the brightness data of all pixels having the same x-coordinate is added to determine the value of the corresponding x-coordinate, to obtain an x-axis histogram based on the determined values of the x-coordinate, and to determine the values of all the pixels having the same y-coordinate. The thermal imaging camera determines the value of the y-coordinate by adding brightness data, obtains a y-axis histogram based on the determined y-coordinate values, and determines whether the fall risk is based on the x-axis histogram and the y-axis histogram data. Fall prediction system used.
delete The method of claim 1,
And the controller is configured to store reference data for distinguishing between a stable state and a fall risk state of a person, and to determine whether or not a person is at risk of falling by comparing pixel data of the photographed image with the reference data.
The method of claim 3, wherein
The control unit, by using a support vector machine (SVM) to compare the pixel data of the photographed image and the reference data to determine whether or not the risk of human falls.
In the fall prediction method of predicting the danger that a person lying on the bed falls out of the bed,
Photographing a person's figure with a thermal imaging camera;
Analyzing each pixel data constituting the photographed image, the pixel data including pixel brightness information;
Adding brightness data of all pixels having the same x coordinate to determine a value of the corresponding x coordinate, and obtaining an x-axis histogram based on the determined values of the x coordinate;
Adding brightness data of all pixels having the same y-coordinate to determine a value of the y-coordinate and obtaining a y-axis histogram based on the determined y-coordinate values;
Determining whether a person is in a stable state or a fall risk state based on the x-axis histogram and the y-axis histogram data; And
Operating the notification device based on the state of the person;
Fall prediction method using a thermal imaging camera comprising a.
delete The method of claim 5,
Establishing a criterion for distinguishing between a person's steady state and a fall risk state;
Fall prediction method using a thermal imaging camera further comprising.
The method of claim 7, wherein
Preparing the criteria,
Photographing a reference image with a thermal imaging camera;
Obtaining brightness data of each pixel constituting the reference image; And
Analyzing an x-axis histogram and a y-axis histogram of the reference image;
Fall prediction method using a thermal imaging camera comprising a.
The method of claim 5,
The determining of whether the person is in a stable state or a fall risk state, the fall prediction method using a thermal imaging camera performed by a support vector machine (SVM).

Images