JP2007072964A - Bed-leaving prediction automatic sensing and notification method, and its automatic sensing and notification system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a bed-leaving prediction automatic sensing and notification system capable of preventing a secondary accident which may occur when a patient or the like of dementia leaves a bed, eliminating fine setting and only requiring the setting of the direction of a device so that the range in which the patient or the like moves is covered in the view angle of a lens of a camera, easy to handle, and capable of accurately sensing a start state of a bed-leaving motion of the patient or the like. <P>SOLUTION: An image obtained by imaging movement on the bed of an observed person at predetermined intervals is converted into a silhouette image or mosaic image. Whether the observed person is awake or sleeping is identified by using image data. Whether the observed person leaves the bed or not is determined by using a neural network. When bed-leaving of the observed person is sensed, it is notified automatically to a nurse or the like who is previously specified. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  In the present invention, in various facilities such as hospitals, nursing homes and nursing homes (hereinafter simply referred to as hospitals or facilities), for example, patients with cognitive impairment or elderly people (hereinafter referred to as patients) wake up. Premature bed prediction automatic detection and notification method to prevent falling accidents such as patients from the bed by predicting and detecting the so-called bed leaving behavior that is about to leave the bed (bedding) just before the start. And related to the automatic sensing and reporting system.

  Currently, there are many secondary accidents in elderly care patients such as elderly people who have cognitive impairments, etc., get up from the bed in the facility, fall down and cause fractures or bruises. Under such circumstances, at present, a pressure sensitive mat (contact type sensor) is used which is arranged beside the bed and senses the load pressure when a patient or the like gets on and switches on. A far-infrared sensing device (non-contact type sensor) that always irradiates a patient on the bed with near-infrared light and determines that the patient has taken a bed leaving action when the patient is out of the far-infrared radiation range. Has been developed and used. All sensors are connected to a nurse call by wire, and when these sensors sense a bed leaving action of a patient or the like, they are put into practical use as a notification system that automatically sounds a nurse call.

  By the way, as a prior art related to this type of automatic notification system, detection means for detecting that the bed user has left the bed and outputting a bed leaving signal, and receiving the bed leaving signal from the detection means and outputting a caution signal to the outside. A bed reporting device comprising an external display means for displaying, a pressure sensor type detection means for outputting an electrical signal as a bed leaving signal when the detection means is installed outside the bed and senses pressure, or a bed installed outside the bed There has been proposed an apparatus including touch sensor type detection means that detects that the user has touched and outputs the floor signal as an electrical signal (see, for example, Patent Document 1).

  However, the detection accuracy is not constant, and the current situation is that there is no definitive countermeasure and its system, such as the patient consciously passing through the sensor detection area and dropping out to get up from the bed.

In other prior art, a pressure body that receives the load of the bedding, a sensor board that is provided with a strain sensitive resistor on at least one surface and is pressed by the pressure body, a sensor case that supports the sensor board, A load sensor composed of an electric circuit that outputs an electric signal corresponding to a resistance value between the terminals of the strain sensitive resistor, and a control that inputs the electric signal from the load sensor, processes it, and outputs it to the monitoring system A filter circuit that inputs an electrical signal from a load sensor and selects only the necessary signal, and a determination means that inputs a signal from the filter circuit and determines occupancy information; There is proposed an in-bed detection apparatus provided with display means for inputting and displaying a signal from the determination means, and output means for inputting a signal from the determination means and outputting it to a monitoring system (example) If, see Patent Document 2).

And a step of detecting the fall of the person to be observed by using the image data of the person to be observed, photographing the person to be observed to obtain image data, and using the image data, A step of obtaining a motion vector of the person, a step of storing a predetermined threshold value indicating the fall of the person to be observed, and the motion vector and the threshold value are compared to determine whether or not the person has fallen. A detection method using a neural network has been proposed for the determination step (see, for example, Patent Document 3).
Japanese Patent Laid-Open No. 10-94525 (paragraphs 0007 to 0028 and FIGS. 1 and 4) JP 2000-105884 A (paragraphs 0078 to 0100 and FIGS. 1 and 2) JP 2002-232870 A (paragraphs 0020 to 0046 and FIGS. 1 and 7)

  However, each of the above-described conventional devices performs contact detection using a pressure-sensitive mat and non-contact detection using an infrared beam, both of which are intended for notification after an accident. Pressure sensitive mats cannot be detected when the patient leaves the bed avoiding the mat. On the other hand, the infrared beam cannot be detected when a patient or the like leaves the bed through the infrared beam. In addition, since the infrared beam is a linear beam that senses the movement of the patient, the setting for ensuring the capture of the patient on the bed is careful and takes time. In addition, when the setting is shifted during use, or when the patient is not set accurately, the detection accuracy is low and the detection is insufficient.

  Furthermore, each of the detection methods or detection devices described in the above publications detects a state in which an operation such as a patient falling is completed, for example, detects an operation immediately before a fall and prevents an accident in advance. It is not a thing.

  The present invention has been made to solve the problems in the conventional devices described above, and in particular prevents secondary accidents that may occur when a patient with cognitive impairment or the like tries to get out of bed. It is possible to set the orientation of the device so that the range of movement of the patient is within the viewing angle of the camera lens. It is possible to detect the start state including prediction, and to automatically notify the movement of the patient etc. in advance, that the device is small and can be manufactured at low cost, that the image information while using the device does not infringe on personal information, the patient It is an object of the present invention to provide an automatic detection notification method and an automatic detection notification system for predicting getting out of bed, which have advantages such as being less likely to cause discomfort as monitored.

  In order to achieve the above object, a method for automatically detecting a bed leaving prediction according to claim 1 is characterized in that a silhouette is obtained from an image obtained by shooting a bed leaving motion including a wake-up / sleeping motion of a large number of other observers on a bed in advance. Image or mosaic image is converted, and the converted image data is input to a neural network to be registered for learning, and a silhouette of an image taken by the camera, including getting up and going to sleep on the bed, is captured. An image or a mosaic image is converted, and the image data is input to a neural network to identify whether the person is in a wake-up state or a sleep state. It is characterized by automatically reporting to a nurse or the like specified in advance.

According to the method for automatically detecting a bed leaving prediction according to claim 1 having the above-described configuration, if an image of a patient or the like (a person to be observed) including a bed is taken, the movement of the patient or the like is within the range of the viewing angle of the camera. Because it is included, subtle settings are not required compared with conventional non-contact sensors using infrared beams, handling is easy, and learning and identification using a neural network, which is a nonlinear identification means, can be used for patients and the like. Because it detects and reports the movement of the bed, especially in learning and registration, it is possible to show the wake-up pattern and bed-getting pattern of various patients in advance in the actual movement image, resulting in a flexible identification system that matches the actual situation and noise. In addition, it is possible to accurately determine whether or not you are getting out of bed, and since a neural network is used as the determination means, you can wake up and enter new patients, etc. Operation can be automatically registered in the field of image data captured by converting processing by the camera ambulation pattern comprising, a better determination accuracy than the determination only by previously prepared image data is exhibited. Furthermore, the movement of the patient or the like is photographed by the camera, but the calculation of the input value to the neural network is targeted for the converted mosaic image or silhouette image, so that personal information of the patient and the like can be protected and leakage thereof can be prevented.

  As described in claim 2, the determination of the start state of the bed leaving movement is performed by photographing a series of movements of the person on the bed at a predetermined time within a predetermined time by the camera a plurality of times. It is desirable to determine the start of the bed leaving operation from the operation a plurality of times using a neural network, and to finally determine whether or not the bed leaving operation is based on the majority rule.

  In this way, it is possible to accurately determine the movement of the patient such as getting out of bed, and the accuracy is improved. In addition, since a notification is made by a nurse call etc. immediately before the patient leaves the bed, an accident can be prevented in advance. it can.

In order to achieve the above object, a bed prediction automatic detection notification system according to claim 3 is based on image data composed of a silhouette image or a mosaic image of a bed pattern including a plurality of various wake-up patterns and bedtime patterns previously captured. A system for automatically detecting a bed leaving prediction that determines a start state of a bed moving operation of the person to be observed from an image obtained by photographing an action on the bed of the person to be observed;
Image capturing means for capturing the movement of the person on the bed at regular intervals, image conversion means for converting an image captured by the image capturing means into a silhouette image or a mosaic image, and the converted image data A determination means using a neural network for identifying the movement pattern of the person to be observed and determining whether or not the person to be observed starts leaving the bed, including prediction, and start of the bed movement of the person to be observed And a reporting means for automatically reporting when the state is determined.

  According to the system for automatically detecting a bed leaving prediction according to claim 3 having the above-described configuration, in addition to the same effect as the above-described operation effect relating to claim 1, the image conversion means and the determination means can be a PC board or a DSP (digital signage). Just by placing a small device built in the board and equipped with a photographing means such as a digital camera near the bed, the notification means can use a nurse call that has been installed in advance. It is small and can be manufactured at low cost, and it is difficult to get in the way by placing it beside the bed. In particular, if the determination result is transmitted wirelessly to the nurse call device as in the embodiment, the sensing device can be easily arranged without affecting the complicated environment as compared with the wired case. be able to.

  5. The image converted by the image conversion unit is a mosaic image according to claim 4, and the mosaic image is obtained by adding all pixels in each mesh obtained by dividing the image captured by the imaging unit into a predetermined number. Then, the average value is calculated and obtained, and the obtained average value can be input to the neural network.

  By doing so, it is difficult to identify the image data handled in hospitals even if the image data is touched by a third party, so that personal information is protected and the patient or other person is monitored. It is difficult to give pleasure.

  6. A silhouette image or mosaic image of a leaving pattern including a plurality of various wake-up patterns / sleeping patterns captured in advance and image data of a silhouette image or a mosaic image obtained by photographing and converting the image of the person to be observed. It is preferable to determine whether or not the person to be observed gets out of bed, including prediction, by inputting image data consisting of

  In this way, it is possible to register not only various types of bed pattern image data imaged in advance, but also image data related to the individual bed pattern of the individual to be observed in the field, so that a bed system specialized for individuals can be realized. , It can demonstrate detection (judgment) accuracy that exceeds the performance created in advance. In addition, since the identification error can be reduced as the number of registered image data regarding the bed pattern of the person to be observed increases, a more accurate sensing (determination) capability can be expected.

  The bed prediction automatic detection notification method and the automatic detection notification system according to the present invention have the following excellent effects.

  -Accidents when getting out of bed can be prevented by detecting the bed leaving action of the patient immediately before the start and automatically notifying the nurse.

  ・ Protection and leakage of personal information can be prevented.

  -A flexible determination (identification) system specialized for individuals can be constructed, and correct identification (determination) can be made as to whether or not the person is getting out of bed.

  ・ Easy to set up and install the equipment, and the installation location is small.

  Hereinafter, embodiments of the bed prediction automatic detection notification system of the present invention will be described based on the drawings, and a bed prediction automatic detection notification method will also be described.

  FIG. 1 (a) is a front view showing a sensing device as a main part of a bed leaving prediction automatic sensing notification system according to an embodiment of the present invention, and FIG. 1 (b) is a diagram in which an opening / closing lid 14 is removed from a base portion of the sensing device. FIG. 1C is a perspective view of the electronic device built in the pedestal portion of the sensing device, as viewed from above.

  As shown in FIG. 1, the sensing device 1 is a stand type having a pedestal 2 at the lower end, and the digital camera (imaging means) 3 sets the vertical position and the direction of the camera lens at the upper end of the telescopic support column 4 of the stand. It is attached so that it can be adjusted. The sensing device 1 is arranged on the foot side of the bed B with the lens of the digital camera 3 facing in a direction in which the motion (posture) of the subject A who is a patient on the bed B can be photographed as a whole. In the pedestal 2, as shown in FIG. 1 (c), a rechargeable battery 5 with an AC / DC converter, a nurse call calling transmitter 6, an identification result communication PHS, a DSP board or a PC board (such as learning / identification) 7 (including software related to the neural network) is provided, and the digital camera 3 is connected to the DSP board or the PC board 7 by the USB camera cable 8. Further, an LED pilot (drive) lamp 9 and a liquid crystal display panel 10 with an operation switch 13 are provided in the opening / closing lid 14 near the center of the outer surface of the pedestal 2. Further, a power switch 11 and an AC cable insertion opening 12 are provided on the side peripheral surface of the base portion 2.

  In the neural network in the DSP board or PC board 7, a large number of patients or the like in advance sleep on the bed (sleeping pattern), wake-up state (wake-up pattern), operation state (pattern) that shifts from the sleep state to the wake-up state, wake-up Various image data such as an operation state (pattern) for shifting from a state to a sleeping state and an operation state (pattern) for getting up and getting out of bed are input and registered for learning.

However, even when hundreds of image data are registered, when using the sensing device 1 for a new patient, the bed leaving pattern of a patient or the like that has been registered in advance differs depending on the individual. Since there is a point, each pattern of a new patient's wake-up state, sleeping state, bed-exit state, etc. is photographed on the site, processed into a mosaic image, etc., and then automatically registered in the neural network. As a result, as compared with the case where the bed leaving pattern is determined using only image data that has been learned and registered in advance, the determination error is extremely reduced, and the sensing accuracy is greatly improved. In the case of this example, since it is assumed to be used in a hospital or a care facility, the report destination is a nurse, and when the sensing device 1 determines that it is a bed leaving pattern, it is transmitted to a nurse call and an alarm sound is generated wirelessly. ing.

  On the other hand, when the power switch 11 of the sensing device 1 is turned on, the LED lamp 9 is turned on, and a moving image of the patient is displayed on the liquid crystal display panel 10, so that the orientation of the digital camera 3 can be confirmed. Further, in response to a command from the DSP board or PC board 7, the state of the person on the bed B is photographed at regular intervals (for example, at intervals of 30 seconds) by the digital camera 3. The captured image is sent to the DSP board or PC board 7 and converted into a mosaic image (image combining white, black and gray square tiles: see Fig. 5) or silhouette image (black shadow image with white background). The Then, the converted image data is sent to a neural network (hereinafter referred to as NN).

  In this example, as shown in FIG. 2, the captured image is divided into 7 × 7 meshes, and the addition average value of all the pixels in each divided mesh is defined as the NN slab value (input value). As a specific calculation method, all the pixels in each mesh are added, and the average value is calculated.

The input to NN is the average value of Slab1 to Slab49, except that Slab50 is 1.
The NN operates with a configuration of 50 input layers, 35 hidden layers (intermediate layers), and 2 output layers.

  That means

こうして、各メッシュに算出した平均値が４９、また５０番目が１として入力層に５０の入力値が入力される。

In this way, the average value calculated for each mesh is 49, the 50th is 1, and 50 input values are input to the input layer.

  As for the mosaic image of the basic bed leaving pattern, as described above, the mosaic image data obtained by imaging the bed patterns of a plurality of patients (as many as possible) is input to the NN for learning and registration. The pattern is different. Therefore, when using the sensing device 1 of the bed leaving prediction automatic detection notification system of this example, as many new bed leaving patterns (including a wake-up pattern and a sleeping pattern) as possible are photographed on the spot when used for a new patient. By registering with the NN, it is possible to sense the bed leaving movement more reliably using image data specialized for each patient, and when the NN determines that it is a bed leaving movement, a nurse call is automatically notified to the nurse. can do.

3 (a) to 3 (e) are images showing a sleep pattern of an observer when a simulation system performance confirmation experiment is performed on one observer, and FIGS. 4 (a) to 4 (e) are the same. It is an image which shows a to-be-observed person's wake-up pattern. FIG. 5 is the mosaic image of FIG. 4C used during the simulation. FIG. 6 is a schematic front view showing the entire simulation system GUI (graphic user interface) at the end of learning when it is input to NN and registered for learning, and it is any of the two patterns of sleeping and waking up. It represents the experimental results that have been identified. FIG. 7A is an enlarged view of a part of FIG. 6, a graph showing the relationship between the strength of identification and the number of learnings for the sleeping pattern, FIG. 7B is a graph showing the relationship between the error and the number of learnings, FIG. 7 (c) is a graph showing the relationship between the strength of identification and the number of learning times for the wake-up pattern, FIG. 7 (d) is a graph showing the relationship between the error and the number of learning times, and FIG. 8 (a) is a part of FIG. In the error curve, the error is approximately 0.0001, and FIG. 8B is a graph showing the learning gradient.

  As a result of the above experiment, learning converges with an error as low as 0.0001, an operation recognition rate of 90% is obtained, and the learning gradient is 0.7. These are shown as an example.

  In the case of the above experiment, the hierarchical calculation unit is composed of three layers: an input layer (50 neuro elements), a hidden layer (35 neuro elements), and an output layer (2 neuro elements). A neuro element is provided so as to correspond one-to-one to the type of pattern based on the slab value (average value calculated by adding all pixels of each mesh) calculated for each mask mesh. Input to the corresponding neuro element. In this example, the hidden layer is composed of 35 neuro element layers, and plays a role of separating and transmitting information of the input layer to the output layer. As the number of hidden layers increases, the calculation can be performed separately for each pattern invariably with respect to the variation of each neuro element in the input layer. In the output layer, two neuro elements are provided so as to correspond to the wake-up pattern and the sleeping pattern to be identified on a one-to-one basis. Then, the output neuro element values based on the weighting coefficient between the neuro elements completed by learning are calculated for several output neuro elements.

  By the way, about the bed prediction automatic detection notification system according to the embodiment of the present invention, the mosaic image data such as a sleeping pattern, a wake-up pattern, and a bed pattern of a specific patient is obtained from a number of existing bed patterns (just before bed leaving). (Including the pattern) and the mosaic image data. Then, when actually applied to a specific patient or the like, the motion of the specific patient on the bed is captured by the digital camera 3 of the sensing device 1, for example, at intervals of several minutes, and the image is converted into mosaic image data. An input pattern (slab value) is input to the NN input layer. This input pattern is transmitted and processed from the input layer → the hidden layer → the output layer, and an output value (output pattern) is output from the output layer. This output value is based on the weighting coefficient obtained by learning so far, and in the case of this example, it is output whether or not it is a bed leaving pattern (including a pattern just before bed leaving).

  On the other hand, when a correct output value is given to the NN regarding whether or not it is a bed leaving pattern, it is transmitted and processed in the order of output layer → hidden layer → input layer, and the weighting coefficient between each layer is learned. This learning of the weighting factor is to change and converge the strength of coupling between the neuro elements of the input neuro element, hidden neuro element and output neuro element so that the difference between the actual output value and the correct output value is reduced. It is.

  Then, the floor pattern of the target viewer is compared with the floor patterns of a large number of patients that have been registered in advance, and the various motion patterns of the target viewer are generally grasped and learned. , It is possible to more accurately determine whether or not it is a bed leaving pattern. When the input corresponding to the floor pattern of the observer is made, the maximum response value of the neuro element in the output layer corresponding to the floor pattern (closely 1) is obtained, and it is determined as “determination from the floor pattern”. Is done.

  However, the notification by the nurse call as a result of such determination is not performed only by one determination by the NN, but the wake-up pattern on the patient's bed, the sleeping pattern, By monitoring movements such as the pattern immediately before getting up and the pattern immediately before leaving, the judgment by the NN (determination of whether or not to get out of bed (including the action immediately before leaving)) is performed a plurality of times (for example, 9 times) before leaving the patient. It is judged comprehensively by the majority rule. In other words, if more than half (for example, 6 times or more) is determined to be a bed leaving action, it is determined that the patient is moving to a bed leaving action (a bed leaving start state) or an action immediately before leaving the bed, and the patient leaves the bed. Just call the nurse call and notify the nurse.

  Although it is repeated, it is not determined only by the image (including a plurality of images) that is taken once, but the image that is taken repeatedly at regular time intervals (1). It is determined whether or not it is a bed leaving pattern for not only a sheet but also a plurality of continuous images), and this determination is performed a plurality of times within a predetermined time, and the majority rule based on the determination results of the plurality of times It is judged comprehensively that it is a bed leaving pattern. If it is determined only once, the probability of a determination error is high, but since the determination is finally made based on a plurality of determinations, the probability of a determination error occurring is extremely small.

  Therefore, in the present example of the automatic detection and notification system for getting out of bed, the start state of bed leaving is sensed immediately before the patient leaves, and the nurse is notified, so when the nurse who received the report comes to bed, for example, the patient It is a state where the person gets up on the bed and is about to get out of bed, and an accident at the time of getting out of bed can be prevented without fail.

FIG. 1 (a) is a front view showing a sensing device as a main part of a predicted bed leaving detection system according to an embodiment of the present invention, and FIG. 1 (b) is an enlarged view showing a pedestal portion of the sensing device. FIG. 1C is a perspective view seen from above, showing an enlarged view of the electronic device built in the pedestal portion of the sensing device. It is explanatory drawing which shows the calculation method which divides the image | photographed image into a 7x7 mesh, and converts to a mosaic image by calculating it as the addition average value of all the pixels in each divided | segmented mesh. FIGS. 3A to 3E are images showing a sleep pattern of a person to be observed when a performance confirmation experiment of the simulation system is performed for one person to be observed. 4A to 4E are images showing the wake-up pattern of the person to be observed. It is the mosaic image of FIG.4 (c) used at the time of simulation. It is a schematic front view showing the entire simulation system GUI (graphic user interface) at the end of learning when it is entered into the neural network and registered for learning, and it identifies which pattern is associated with the two patterns of sleeping and waking up Represents the experimental results. FIG. 7A is an enlarged view of a part of FIG. 6, a graph showing the relationship between the strength of identification and the number of learnings for the sleeping pattern, FIG. 7B is a graph showing the relationship between the error and the number of learnings, FIG. 7C is a graph showing the relationship between the strength of identification and the number of learning times for the wake-up pattern, and FIG. 7D is a graph showing the relationship between the error and the number of learning times. FIG. 8A is an error curve with an error of approximately 0.0001, and FIG. 8B is a graph showing a learning gradient of 0.7.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Sensing apparatus 2 Base part 3 Digital camera (imaging means)
4 Telescopic shaft 5 Rechargeable battery with AC / DC converter 6 Nurse call calling transmitter 7 PC board (including software related to neural network such as learning and identification)
8 USB camera cable 9 LED pilot (drive) lamp 10 LCD panel 11 Power switch 12 AC cable outlet 13 Operation switch

Claims (5)

Converting a bed movement including a wake-up / sleeping movement of many other observers on a bed from an image captured in advance by a camera into a silhouette image or a mosaic image, and inputting the converted image data into a neural network Let me register
An image obtained by taking a camera-awake operation including waking up and going to sleep on the bed is converted into a silhouette image or a mosaic image, and the image data is input to a neural network.
Identifying whether the subject is in a wake-up state or a sleeping state, and further, when determined to be in a state to start getting out of bed, automatically notifies a nurse or the like specified in advance by a nurse call or the like A method to automatically detect getting out of bed.   The determination of the start state of the bed leaving movement is performed by photographing a series of movements of the person on the bed at predetermined intervals within a predetermined time by the camera a plurality of times, and the start of the bed leaving movement is determined from the series of actions by a neural 2. The automatic sensing and reporting method according to claim 1, wherein a determination is made a plurality of times by a network, and a final determination is made as to whether or not the bed leaving operation is based on a majority rule. Based on image data consisting of a silhouette image or a mosaic image of a leaving pattern including a plurality of various wake-up patterns / sleeping patterns that have been photographed in advance, the subject's bed is taken from the image obtained by photographing the motion of the person on the bed. It is a system for automatically detecting a bed leaving prediction that determines a start state of an operation and reports at the same time as the determination,
Image capturing means for capturing an operation of the person on the bed at regular intervals, image conversion means for converting an image captured by the image capturing means into a silhouette image or a mosaic image, and the converted image data Using a neural network to identify the movement pattern of the person to be observed, and to determine whether or not the person to start getting out of bed includes prediction,
A bed leaving prediction automatic sensing and reporting system, comprising: reporting means for automatically reporting when it is determined that the observation subject's bed leaving operation has started.   The image converted by the image conversion means is a mosaic image, and the mosaic image adds all the pixels in each mesh obtained by dividing the image captured by the imaging means into a predetermined number, and calculates an average value thereof. 4. The bed leaving prediction automatic sensing notification system according to claim 3, wherein the calculated average value is input to a neural network.   Image data of the silhouette image or mosaic image obtained by photographing the subject and converted, and the image data consisting of the silhouette image or mosaic image of the leaving pattern including a plurality of various wake-up patterns / sleeping patterns previously captured are input to the neural network. 4. The bed leaving prediction automatic detection notification system according to claim 3, further comprising: determining whether or not the person to get out of the bed includes prediction.

Images