JP2017038777A - Motion recognition apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a motion recognition apparatus capable of determining a condition of a person with a high degree of precision even by a CPU which is relatively low in calculation capability.SOLUTION: A motion recognition apparatus includes: a three-dimensional data input unit 21 for creating a gray scale image and a distance image from three-dimensional information acquired from a camera 1; a bed area grasping unit 22 for grasping a bed area based on position data on four corners of a bed; a head detection/head direction estimation unit 23 for detecting centers of the head and two shoulders of a person on the bed from the distance image and the gray scale image, and estimating a head direction viewed from the center of the two shoulders; a coordinate conversion unit 24 for converting the six points of the acquired center of the head, center of the two shoulders, and four corners of the bed to a world coordinate system; a motion determination unit 25 for determining that the motion is in a specific motion state to be notified of if relationships between the position and direction of the head, and the position of the bed satisfies a specific condition based on the data converted to the world coordinate system, and the direction of the head; and a result output unit 26 for notifying that the motion is in a specific motion state.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a motion recognition apparatus that captures an image with a camera in order to watch a person on a bed and detects a person's rising and getting out of the bed from the captured image.

In order to prevent a person on the bed from falling off the bed or accidents caused by dredging, a system has been developed to watch the person using various sensors and notify the parties concerned when an abnormality is detected. In such a system, a technique has been proposed in which three-dimensional data relating to a person to be watched is obtained in a non-contact manner and the occurrence of a person leaving the bed is prevented based on this data.
For example, Patent Document 1 discloses a configuration in which an infrared laser sensor is used to grasp the operation of a cared person on a bed to detect rising or getting out of bed. Patent Document 2 describes a technique that uses a distance image sensor that generates an image having distance data to a patient to be imaged as a pixel value and discriminates a patient's movement in comparison with a bed position detected together. Yes.

JP 2014-149731 A JP 2012-30042 A

  The technique of the above-mentioned patent document that obtains the three-dimensional data related to the person to be watched and discriminates the movement of the person can be discriminated with higher accuracy than the technique that judges from the two-dimensional data so far. However, in any technique, a sensor device that acquires three-dimensional data is installed obliquely above the bed. In such a configuration, in order to accurately grasp the person to be watched from the sensor device, it is necessary to convert the distance image obtained from the sensor device into data such as a point cloud that has been rearranged in the world coordinate system. For this reason, the calculation amount is large and the burden on the CPU is large. Further, when performing subsequent recognition processing, since processing using data such as a point cloud is performed, the amount of calculation is large and a high performance CPU is required.

  Therefore, in view of such problems, the present invention provides a motion recognition device that can determine the state of a person with high accuracy even with a CPU having a relatively low calculation capability by reducing the data amount itself to be converted into the world coordinate system. The purpose is to do.

In order to solve the above-mentioned problem, the invention of claim 1 is characterized in that information acquisition means for capturing image information and distance information by imaging the entire bed from above, and motion recognition for determining the motion of a person on the bed from the acquired information. A motion recognition device comprising: a three-dimensional data input unit for generating image data and distance data from the acquired information; and a bed area based on position data at the four corners of the image data bed. Head detection that detects the center of the head and both shoulders of the person on the bed from the bed area grasping unit to grasp and the image data and distance data, and further estimates the head direction seen from the center of both detected shoulders -Estimated head direction estimation unit, coordinate conversion unit that converts the obtained head center, both shoulder centers, and the four corners of the bed to the world coordinate system, and data converted to the world coordinate system Based on head direction And an operation determining unit that outputs a specific operation detection signal when the relationship between the position and direction of the head and the bed position satisfies a specific condition, and a result output unit that performs a notification operation upon receiving the specific operation detection signal. Features.
According to this configuration, since the motion of the person on the bed is determined based on the image data and the distance data, it is possible to determine the rising motion and the leaving motion with high accuracy. On the other hand, since the data amount to be converted into the world coordinate system is as small as 6 points, the calculation amount can be reduced and detection can be performed at high speed.

According to a second aspect of the present invention, in the configuration according to the first aspect, when the estimated head direction is upward and the position of the head exceeds a predetermined height, the motion determination unit It is characterized in that it is determined that has risen.
According to this configuration, since the rising is determined from the orientation of the patient's head and the height of the head, the rising can be determined with high accuracy.

According to a third aspect of the present invention, in the configuration according to the first or second aspect, the motion determination unit is configured such that the estimated head direction faces outward from the bed area and the head center is a constant distance from the bed area in the lateral direction. It is characterized that it is determined that the bed has left when it leaves.
According to this configuration, since getting out is determined from the orientation of the patient's head and the distance from the bed area, it is possible to determine getting out with high accuracy.

  According to the present invention, since the movement of the person on the bed is determined based on the data of both the distance image and the image information, it is possible to determine the rising motion and the leaving motion with high accuracy. On the other hand, since the data amount to be converted into the world coordinate system is as small as 6 points, the calculation amount can be reduced and detection can be performed at high speed.

It is function explanatory drawing which shows an example of the action recognition apparatus which concerns on this invention. It is explanatory drawing which shows the installation position of an information acquisition means. It is explanatory drawing of a gray scale image. It is explanatory drawing of a distance image. It is explanatory drawing which displayed the recognized bed area on the distance image. It is explanatory drawing which added the center point of both shoulders and the center point of the head to the image of FIG. It is explanatory drawing which converts 6 points into a world coordinate system. It is explanatory drawing of rising determination, (a) is the state which fell down on the bed, (b) has each shown the state of determination of rising. It is explanatory drawing of a bed leaving determination, (a) has shown the state immediately before a bed leaving operation | movement, (b) has each shown the state determined to be a bed leaving. It is another explanatory drawing of bed leaving determination. (A) has shown the state just before a bed leaving operation | movement, (b) has each shown the state determined to be a bed leaving.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments embodying the present invention will be described below in detail with reference to the drawings. FIG. 1 is a functional explanatory diagram showing an example of a motion recognition apparatus according to the present invention. 1 is a camera as an information acquisition means for capturing an image of a watching target and obtaining image information and distance information, and 2 is determining the operation of the watching target. Motion recognition means.
As the camera 1, for example, a stereo camera, a TOF (Time of Flight) camera, or an RGB-D camera (a camera that can acquire a distance image in addition to a color image) can be used. The motion recognition unit 2 includes a three-dimensional data input unit 21, a bed area grasping unit 22, a head detection / head direction estimation unit 23, a data conversion unit 24, a motion determination unit 25, and a result output unit 26.

  FIG. 2 is an explanatory view of the installation position of the camera 1. As shown in FIG. 2, the bed B is imaged from above and is installed above the head of the patient K lying on the bed. . As described above, the case where the motion recognition device is installed in the hospital and the inpatient is watched is described here.

  The three-dimensional data input unit 21 generates and outputs a grayscale image as image data and a distance image as distance data based on the three-dimensional image information acquired from the camera 1.

The bed area grasping unit 22 grasps the bed area by acquiring the positions of the four corners of the bed. The positions of the four corners of the bed can be acquired from the acquired gray scale image and the distance image by image recognition technology such as template matching. good.
The four corners of the bed may be input by the user. For example, a grayscale image may be displayed on a monitor, and the four corners of the bed may be input by a coordinate operation or the like.

The head detection / head direction estimation unit 23 first determines the center of the head (center point) and the center of both shoulders (medium size points) from the grayscale image and the distance image output from the three-dimensional data input unit 21. To detect. As a detection method, a method combining a local feature amount such as HOG (Histogram of Oriented Gradients) and learning such as Random Forests may be used, or a method such as deep learning may be used.
When the center of the head and the center of both shoulders are detected in this way, the direction of the head center (head direction) viewed from the center of both shoulders is estimated based on the data.

  The data conversion unit 24 arranges (converts) the detected head center, the centers of both shoulders, and the four corner points of the bed in the world coordinate system. By this conversion, the center of the bed, the patient's head, and both shoulders is projected onto the three-dimensional space, so that the head direction estimated by the head detection / head direction estimation unit 23 can also be projected at the same time. Three-dimensional information for performing necessary determination can be obtained.

  The motion determination unit 25 determines the patient's motion based on the data converted into the world coordinate system and the estimated head direction information, and outputs a signal to the result output unit 26 when determining that the patient is in a specific motion state. . For example, if the head changes its direction from the sideways direction to the ceiling direction and moves away from the bed to a higher position, it is determined that the patient has stood up. Further, if the head direction is outside the bed and the center of the head is away from the bed area by a certain distance, it is determined that the person is getting out of bed.

The result output unit 26 receives the signal output from the operation determination unit 25 and performs a notification operation. For example, it is notified by a sound report of a voice message or a display on a display so that a nurse resident at a nurse station recognizes the occurrence of an abnormality.
The bed area grasping unit 22, the head detecting / head direction estimating unit 23, the data converting unit 24, and the operation determining unit 25 can be realized by causing a CPU or DSP to execute a predetermined program.

Hereinafter, the flow of motion detection will be described with reference to an image diagram. 3 is a grayscale image output from the three-dimensional data input unit 21 based on information acquired from the camera 1, and FIG. 4 is a distance image similarly output from the three-dimensional data input unit 21.
From the gray scale image, it is possible to determine the patient's condition and bed area, but it is not possible to grasp the position of each point in three dimensions. On the other hand, in the distance image of FIG. 4, the distance is displayed by the density of the image. Specifically, the closer to the camera 1, the darker color (black) is displayed, and the farther away, the lighter color (white) is displayed, so that the relative distance can be grasped.

  FIG. 5 is an image diagram in which the bed area B1 acquired by the bed area grasping unit 22 is displayed on the distance image acquired in this way, and FIG. 6 shows the head center detected by the head detection / head direction estimation unit 23. The image figure which further displayed M1 and the center M2 of both shoulders is shown.

The four corners of the bed, the head center M1, and the shoulder centers M2 thus obtained are converted into the world coordinate system by the data converter 24. FIG. 7 is an explanatory diagram showing a state where six points of the four corners of the bed, the head center M1, and the shoulder centers M2 are converted into the world coordinate system.
In FIG. 7, the direction of the head center M1 (not shown) viewed from the center M2 of both shoulders is indicated by an arrow P1. This head direction coincides with the head direction estimated by the head detection / head direction estimation unit 23.

8 to 10 are explanatory diagrams of patient motion determination, and motion determination based on coordinate-converted data and head direction data will be specifically described with reference to this figure. FIGS. 8A and 8B are explanatory diagrams of the determination of getting up. FIG. 8A shows the state of lying down on the bed, and FIG. 8B shows the state of determination of getting up, respectively. The operation determination such as getting up can be easily determined by changing the viewpoint as shown in FIG.
In a state where the patient is lying on the bed B, as shown in FIG. 8A, the head direction P1 is oriented in the horizontal direction (more precisely, the bed longitudinal direction in the bed). As shown in (b), the head direction P1 is directed upward (for example, the angle A1 is 45 ° or more), and the head B is separated from the bed B by a certain distance H1 (for example, 50 cm). When this state is detected, it is determined that the rising has occurred, and a rising detection signal (specific operation detection signal) is output. The determination of the distance can be made by assuming that the length L1 in the longitudinal direction of the bed B shown in FIG. 8A is about 2 meters, for example.
Upon receiving this rising detection signal, the result output unit 26 notifies that the patient to be watched has risen.

The bed leaving operation can be determined from the operations shown in FIGS. Both are converted to the state seen from the horizontal direction, but unlike FIG. 8, they are displayed in the horizontal direction seen from the pillow side. 9A shows a state immediately before leaving the bed, and FIG. 9B shows a state in which it is determined that the user has left the bed. As shown in FIG. 9, the position of the head is separated from the bed B by a certain distance L2 (for example, 80 cm) or more. When it is determined that the user has left the bed, a bed generation signal (specific operation detection signal) is output.
FIG. 10A shows a state immediately before leaving the bed, and FIG. 10B shows a state in which it is determined that the user has left the bed. As shown in FIG. 10, the head is out of the bed B and the head direction P1 Even when the extension line PL of the line goes out of the area of the bed B, it is determined that the person has left the bed and outputs a bed generation signal.
And the result output part 26 which received the bed leaving generation signal alert | reports that the patient to watch over got out of bed.

As described above, since the movement of the person on the bed B is determined based on the image data and the distance data, it is possible to determine the rising movement and the leaving movement with high accuracy. On the other hand, since the data amount to be converted into the world coordinate system is as small as 6 points, the calculation amount can be reduced and detection can be performed at high speed.
Then, since the rising is determined from the patient's head direction P1 and the height of the head, the rising can be determined with high accuracy, and the bed leaving is determined from the orientation of the patient's head and the distance L2 from the bed area B1. Therefore, it is possible to determine bed leaving with high accuracy.

In the above-described embodiment, the rising operation and the detection of getting out of bed have been described. However, other operations such as falling from the bed can also be detected. For example, if the detected head (head center M1) deviates from the bed area B1 and stays at a position lower than the bed B, it can be estimated that a fall has occurred.
In the three-dimensional data input unit 21, the captured image of the camera 1 is converted into a grayscale image. However, the captured image of the camera 1 is output simply by converting it into digital data (image data) as a color image. It is possible to detect the head later.

  In the above embodiment, the case where the patient who is admitted to the hospital is watched is described. It can be used when watching.

  1 .. Camera (information acquisition means) 2 .. Motion recognition means 21.. 3D data input unit 22. Bed area grasping unit 23 23 Head detection / head direction estimation unit 24 Data conversion unit (coordinate conversion unit), 25 ... Motion determination unit, 26 ... Result output unit, P1, head direction, B ... bed.

Claims (3)

Information acquisition means for capturing image information and distance information by imaging the entire bed from above;
A motion recognition device comprising motion recognition means for determining the motion of a person on the bed from the acquired information,
The motion recognition means includes a three-dimensional data input unit that generates image data and distance data from the acquired information,
A bed area grasping unit for grasping a bed area based on position data of the four corners of the bed of the image data;
A head detection / head direction estimation unit that detects the center of the head and both shoulders of the person on the bed from the image data and the distance data, and further estimates the head direction viewed from the detected center of both shoulders When,
A coordinate conversion unit that converts the obtained center of the head, the center of both shoulders, and the six points of the four corners of the bed into a world coordinate system;
Based on the data converted into the world coordinate system and the estimated head direction, a motion determination unit that outputs a specific motion detection signal when the relationship between the head position and direction and the bed position satisfies a specific condition; ,
A motion recognition device comprising: a result output unit that performs a notification operation upon receiving the specific motion detection signal.   The operation determining unit determines that the person on the bed has risen when the estimated head direction is upward and the position of the head exceeds a predetermined height. Motion recognition device.   The motion determination unit determines that the bed is out of position when the estimated head direction faces outward from the bed area and the head center is separated from the bed area by a certain distance in the lateral direction. The motion recognition apparatus according to claim 1 or 2.

Images