JP7458813B2 - Monitoring device, monitoring method, monitoring program - Google Patents

Description

The present invention relates to a monitoring device, a monitoring method, and a monitoring program that determine the operation of a monitored object and enable monitoring.

In an aging society with longevity, there is a demand to contribute to extending healthy lifespans, but among medical accidents, injuries caused by falls during treatment are on the rise.

To prevent falls, pressure and infrared sensors are sometimes used as a monitoring function to detect and respond to patients at high risk of falling as they get out of bed.

The pressure sensor is installed on the floor or bed rails and is configured to notify when weight is applied. The infrared sensor is installed on the bed rails or outside the bed and is configured to notify when a patient enters the detection range.

However, the monitoring function above notifies the patient after their feet touch the floor or just before they do, so by the time a nurse or other staff member arrives, the patient may already be out of bed and walking, putting them at risk of falling.

Patent Document 1 describes that a fall state is determined based on a difference in silhouette lengths reflected on a camera. According to Patent Document 1, a state transition from a state of lying on a bed to a state of standing can also be determined.

Patent Document 2 describes that an image captured by a camera is subjected to calculations using a neural network to determine whether a person is sitting up. According to Patent Document 2, after the camera is installed, the camera performs a rising motion multiple times to perform learning (teaching data creation).

Patent document 3 describes how the bed is photographed from the side, a range is set, and if the area of the silhouette within the set range is less than a specified amount, it is determined that the behavior is a wake-up behavior.

Patent document 4 describes how an image is captured from behind of a person to be monitored while lying in bed, and the condition is determined from the range and position of the silhouette.

For reference, Patent Document 5 describes setting the boundaries of areas using IR (infrared) reflective tape.

For reference, Patent Document 6 describes how comforters and pillows are made from infrared-reflective fibers and a person's head is recognized using image processing.

For reference, Patent Document 7 describes attaching infrared-sensitive markers to the shoulders of work hats or work clothes, allowing the air volume of air conditioners to be controlled according to individual preferences.

Special Publication No. 2006-522959 JP2007-072964A Japanese Patent Application Publication No. 2011-005171 Japanese Patent Application Publication No. 2012-170483 Japanese Patent Application Publication No. 2001-243472 JP2017-021718A JP 2011-033220 A

In Patent Document 1, movements on the bed (turning over the futon, getting up, turning over, etc.) cannot be determined, and signs of becoming in the edge sitting position (also referred to as edge sitting position, hereinafter referred to as ``edge sitting position'') cannot be detected. .

In Patent Document 2, learning by a neural network is required each time a camera is installed or a person to be monitored changes, which poses a problem with machine resources. Also, the results are likely to change depending on the lighting.

In Patent Document 3, the image information is processed as is, and the number of pixels required to calculate the area is large, creating a problem with the control burden.

Furthermore, the silhouette of the surroundings of the person to be monitored, such as a futon, is not taken into account, and accurate recognition may not be possible.

In Patent Document 4, the camera is positioned close to the subject, so there is a risk that the subject may remove it or shift the optical axis.

Furthermore, the silhouette of the surroundings of the person to be monitored, such as a futon, is not taken into account, and accurate recognition may not be possible.

An object of the present invention is to obtain a monitoring device, a monitoring method, and a monitoring program that can remotely monitor the operation of a monitoring target whose movement by one person is restricted.

In addition to the above purpose, if a monitored subject who is restricted from walking alone satisfies certain conditions in the movement pattern from when he or she goes to bed until he leaves the bed, a notification can be sent to him before he leaves the bed. The purpose is to obtain monitoring equipment, monitoring methods, and monitoring programs.

The imaging device according to the present invention is made of a material that reflects or absorbs invisible light, or a material that has luminous properties, and is attached to an object of interest that deforms according to the movement of a monitored object, and is attached to an object of interest that deforms according to the movement of the object to be monitored. a marker part that is displaced according to the object of interest; an imaging part that captures an image with an angle of view that includes the object of interest and outputs an image signal; and a marker part that is identified from the image signal output from the imaging part. , and a monitoring unit that monitors the operation of the monitored object.

According to the present invention, the target to be monitored is monitored by the displacement of the marker part that is imaged by the imaging part.

At this time, the marker is difficult to recognize with the human eye at least in a bright place, and can be identified and imaged as an output signal of the imaging unit in a dark place.

As a result, the sign does not impair the design of the target object, and the distinctiveness of the sign is maintained, allowing monitoring of the target object to continue.

For example, if the object to which the sign is attached is cloth, paint that reflects or absorbs non-visible light (typically infrared light) or has luminous properties may be applied directly or applied to adhesive materials such as stickers. There is no particular limitation on the attachment method, such as attaching the cloth material or directly dyeing the raw material (thread, etc.) of the cloth material.

The imaging device according to the present invention includes a marker section that is attached to an object of interest that deforms according to the movement of the object of interest, and that is displaceable in accordance with the deformation of the object of interest, and that captures an image of an angle of view that includes the object of interest. an imaging unit that captures an image and outputs an image signal; and a monitoring unit that monitors the operation of the monitoring target by identifying the marker from the image signal output from the imaging unit, the monitoring unit , an analysis unit that analyzes the displacement of the position of the marker unit in a time series of images captured by the imaging unit; and a prediction unit that predicts the movement of the monitoring target from the current state based on the analysis result by the analysis unit. , a notification unit that notifies at least a prediction result to an observer when a change in movement predicted by the prediction unit satisfies a certain condition, and there are a plurality of objects of interest to which the sign unit is attached; The marking portions are attached with different shapes depending on the type of the object of interest and on the front and back sides .

According to the present invention, the imaging unit images the marker.

The captured image is analyzed by the analysis section of the monitoring section to determine the displacement of the position of the marker over time in the image.

The prediction unit predicts the movement of the monitored object from the current state based on the analysis results.

The notification unit notifies at least the observer of the prediction result when the predicted change in movement meets certain conditions.

Thereby, it is possible to monitor the movement of the object to be monitored and the prediction of the movement after the current state.

In the present invention, the label portion is made of a material that reflects or absorbs at least invisible light or has luminous properties.

The marker portion is difficult to recognize with the human eye at least in a bright place, and can be identified and imaged by an imaging unit in a dark place.

Therefore, the marking part can maintain the distinctiveness of the marking part without impairing the design of the object of interest.

For example, if the object of interest to which the sign is attached is a cloth material, invisible light (typically infrared light) or phosphorescent paint may be applied directly, or it may be applied to an adhesive material such as a sticker and pasted. Alternatively, there is no particular limitation on the attachment method, such as directly dyeing the raw material (thread, etc.) of the cloth material.

In the present invention, the marking portion is attached in a known pattern in advance.

The pattern change in the sign makes it possible to monitor the displacement of the object of interest.

The type of object of interest can be identified by the shape of the marker. Furthermore, when an object of interest has front and back sides, by attaching markings of different shapes to the front and back sides, it is possible to identify the front and back sides of the object of interest.

In the present invention, the monitoring target is a person who is sleeping in bed, and is a person with limited mobility who is restricted from leaving the bed and walking alone from the bed.

By displacing the marker, it is possible to monitor the movement of the person with restricted mobility, for example, from the state of being in bed until the person leaves the bed.

The monitoring method according to the present invention is a monitoring method that includes a marker unit made of a material that reflects invisible light or a material that absorbs or has phosphorescence, and that is attached to bedding required when going to bed, and an imaging unit that captures an image with an angle of view that includes the bedding, and detects getting out of bed from the movements of the monitored subject while in bed, and is characterized in that the method analyzes the output signal captured and output by the imaging unit to identify the displacement of the marker unit, determines the state of the monitored subject while in bed from the displacement state of the marker unit, and notifies the monitor when the change in the state of the monitored subject satisfies certain conditions.

According to the present invention, for example, a hospitalized patient may be monitored, but the monitor may not be able to monitor the patient 24 hours a day and may need to temporarily restrain the patient.

By using the monitoring device of the present invention, a monitor is notified when a change in the movement of a monitored object satisfies a certain condition.

As a result, the supervisor can rush in before the subject to be monitored leaves the bed and starts walking, making it possible to relieve restraint on the patient.

In the present invention, as the bedding, the marking portions of different shapes are attached to at least two or more places of the pajamas, the front and back of the comforter, and the pillow, and the monitoring target is determined according to the displacement state of each marking portion of the different shape. Determine the state of.

The bedding can be identified by the shape of the marker.

The monitoring program according to the present invention is characterized in that a computer operates as the monitoring unit of the monitoring device.

As described above, according to the present invention, it is possible to remotely monitor the operation of a monitoring target whose movement is restricted by one person.

In addition to the above effects, if a monitored subject who is restricted from walking alone satisfies certain conditions in their movement pattern from when they are in bed until they leave the bed, it is possible to notify them before they leave the bed. .

(A) is a plan view of bedding (pajamas, comforters, pillows) as objects of interest, each with different markers that are invisible to the human eye, and (B) is a plan view showing the state of (A) on the imaging unit. (For example, an IR camera or the like) is a front view of an image taken (bedding cannot be recognized, but markers can be identified). (A) is a plan view of the monitored target (patient, etc.) in bed (stable state), (B) is a plan view of the monitored target in the edge sitting position (pre-movement 1 for getting out of bed), ( C) is a plan view of a state in which the subject to be monitored has flipped up the futon and put out his/her feet (preliminary movement 2 for getting out of bed). FIG. 1 is a system configuration diagram of a monitoring device (an imaging system, a server system, and a client system cooperate with each other via a network). 4 is a functional block diagram of each system in FIG. 3. FIG. 7 is a flowchart showing an imaging control routine in the imaging system according to the present embodiment. 3 is a flowchart showing a monitoring control routine in the server system according to the present embodiment. 3 is a flowchart showing a notification control routine in the client system according to the present embodiment.

(Configuration of the sign part “marker”)

FIG. 1 shows bedding 12 (pajamas 12A, futon 12B, pillow 12C) as objects of interest to be monitored in place of the objects to be monitored by the monitoring system 100 (see FIG. 3) according to the present embodiment. .

The bedding 12, which is the object of interest, is a member that the monitoring object whose motion is to be monitored wears or directly touches when sleeping.

The subjects to be monitored are mainly those who are at risk of falling when walking alone. In addition to people who are at risk of falling when walking alone, there are other people to be monitored who may be at risk of falling when walking alone, such as people who would be in trouble if they wandered around or running away. Explain as someone who is afraid.

A plurality of markers 14A, 14B, and 14C are attached to each of the pajamas 12A, the futon 12B, and the pillow 12C as marking portions having different shapes. Although they are distinguished in the drawings, in the detailed description, when explaining the common functions of the markers 14A, 14B, and 14C, they may be collectively referred to as markers 14.

The markers 14 are attached directly to the bedding 12 using a material that reflects or absorbs invisible light, or has phosphorescent properties. In the following, an IR reflective dye is used as an example, but the markers may also be invisible materials that target the UV (ultraviolet) range, phosphorescent materials, etc.

As shown in FIG. 1(A), the bedding 12 (pajamas 12A, futon 12B, pillow 12C) is in a visible state (see the solid line in FIG. 1(A)), but the marker 14 is in an invisible state (see the solid line in FIG. 1(A)). ).

As a result, the marker 14 does not alter the original design of the bedding 12.

Furthermore, the marker 14 may be applied directly to the bedding 12 with an IR reflective dye using a touch pen, brush, etc., or may be attached via a mount (adhesive sticker, etc.). Moreover, an IR reflective dye may be applied in advance to the raw material of the thread material when producing the bedding 12.

On the other hand, in FIG. 1(B), for example, when the bedding 12 is photographed from above with an infrared camera, the marker 14, which is an IR reflective dye, is photographed (see the solid line in FIG. 1(B)), but the bedding 12 is not photographed (see the solid line in FIG. 1(B)). (See dotted line in Figure 1(B)).

The markers 14A attached to the pajamas 12A have a round shape and are attached to the abdomen, back (not shown), sleeves, and shoulders of the upper garment of the pajamas 12A, and to the hip and ankle regions of the lower garment of the pajamas 12A. has been done.

The markers 14B attached to the futon 12B are provided in a matrix (for example, a 3×5 array) on each of the front and back sides of the futon 12B (see FIGS. 2(B) and (C)). In this embodiment, the marker 14B (FR) on the front side of the futon 12B has a star shape, and the marker 14B (BK) on the back side has a triangular shape, as shown in FIGS. 2(B) and (C).

The markers 14C attached to the pillow 12C have a rectangular shape, and are placed in three horizontal rows on the pillow.

Note that the arrangement of the markers 14A, 14B, and 14C on each bedding 12 is not particularly limited, and if the arrangement in the normal state when the monitored subject sleeps is recognized as the initial arrangement state (home position), each subsequent The displacement of markers 14A, 14B, and 14C may be individually monitored. However, if a geometric arrangement pattern is used as shown in FIG. 1, it is possible to quickly recognize that there is movement in the monitored object by the observer's visual judgment of the infrared image.

Here, by attaching the markers 14A, 14B, and 14C of this embodiment to the bedding 12, the subject of monitoring can monitor the movements of the subject using images captured by the infrared sensor, without the subject having to wear the markers 14A, 14B, and 14C, etc., directly.

Figure 2 shows the infrared image conditions according to the movement of the monitored object ((A) to (C)).

Figure 2 (A) shows a person who may fall if walking alone (here, a hospitalized patient 16) immediately after going to sleep in bed 20, with the patient 16's head placed in the center of pillow 12C and the duvet 12B draped over his or her shoulders.

The requirements for estimating the state in FIG. 2(A) are shown below.

(Requirement 1) The position of the patient's 16 head can be estimated because the marker 14C at the center of the pillow 12C is hidden.

(Requirement 2) The markers 14A on the pajamas 12A are completely invisible, and the star-shaped markers 14B (FR) on the front side of the futon 12B are in the initial arrangement, so it can be inferred that the patient 16 is wearing the futon 12B up to his shoulders.

In FIG. 2(B), the patient 16 is in an awake state with the upper body raised on the bed 20.

The conditions that allow the state of Figure 2 (B) to be inferred are as follows:

(Requirement 3) By being able to see all of the markers 14C on the pillow 12C, it can be estimated that the patient's 16 head is in a position away from the pillow 12C.

(Requirement 4) By recognizing the pattern of the plural markers 14A of the pajamas 12A, it can be estimated that the upper body is raised and one arm is placed on the futon 12B. As an example, the fact that a portion of the marker 14A is a perspective image (elliptical) is also one of the estimation factors.

(Requirement 5) Both the star-shaped marker 14B (FR) on the front side of the futon 12B and the triangular marker 14B (BK) on the back side are visible, and from the ratio of their numbers, etc., a part of the futon 12B is turned over. It can be estimated that

FIG. 2C shows a state in which the patient 16 has left the bed 20 and is about to walk.

The requirements for estimating the state in FIG. 2(C) are shown below.

(Requirement 6) By seeing all the markers 14C on the pillow 12C, it can be estimated that the head of the patient 16 is located away from the pillow 12C.

(Requirement 7) By pattern recognition of multiple markers 14A on the pajamas 12A, it can be estimated that the person is sitting up with one arm resting on the futon 12B. As an example, the fact that some of the markers 14A are oblique images (oval shapes) is also one of the estimation factors.

(Requirement 8) Both the star-shaped marker 14B (FR) on the front side of the duvet 12B and the triangular marker 14B (BK) on the back side are visible, and based on the ratio of their numbers, etc., it can be inferred that the duvet 12B has been turned over significantly.

(Requirement 9) Since some of the markers 14A of the pajamas 12A are located outside the initial arrangement pattern of the markers 14B of the futon 12B, it is estimated that the patient 16 is about to take his feet out of the bed 20. be able to.

As described above, by using the markers 14A, 14B, and 14C of this embodiment, there is no need to directly attach markers to the monitoring target, and the movement of the monitoring target is monitored based on images taken with an infrared camera or the like. can do.

In addition, since the markers 14A, 14B, and 14C use IR reflective dye, the design of the bedding 12 (pajamas 12A, futon 12B, and pillow 12C) is not changed, and the markers 14A, 14B, and 14C can be used as necessary. It can be attached at a certain position.

In this embodiment, an invisible member that reflects infrared rays, such as an IR reflective dye, is used as a means of not changing the design of the markers 14A, 14B, and 14C. By providing a function that can extract the color, the specific color can be mixed with different surrounding colors without any discomfort, and if a pattern defined by the specific color (pattern with design) is formed, it can be invisible. It doesn't have to be.

In addition, the subjects to be monitored who are restricted from walking alone are not limited to patients in hospitals16, but may also include caregivers at home, infants, and pets. It can be used not only for predicting signs, but also for understanding posture when walking, position in a room, etc.

(Monitoring system using markers)

Figure 3 shows a monitoring system 100 for monitoring a patient 16 using markers 14A, 14B, and 14C.

The monitoring system 100 according to this embodiment is used to monitor a patient 16 in a hospital.

A patient 16 to be monitored is restricted from walking alone and is lying on a bed 20 in a hospital room 18.

An imaging device 104 configuring a part of the imaging system 102 is installed on the ceiling surface 18A of the hospital room 18, and images an area including at least the bed 20. The imaging device 104 is selected according to the type of marker 14 used. In this embodiment, since the marker 14 is coated with an IR reflective dye, a camera capable of detecting infrared rays is used.

The imaging device 104 is connected to a control unit 106 that together with the imaging device constitutes the imaging system 102. Details of the control unit 106 will be described later.

The control unit 106 is communicably connected to a server system 110 via a network 108 such as a LAN (local area network) or a WAN (wide area network). The server system 110 is installed at a nurse station 21 or the like installed in a hospital. Images from imaging devices 104 installed in a plurality of hospital rooms 18 are aggregated. Details of the server system 110 will be described later.

A communication terminal device 114 is connected to the network 108, and communication is possible between the server system 110 and the communication terminal device 114 as a client system. For example, the communication terminal device 114 may be portable by the nurse 22, and may be installed at the nurse station 21. That is, the communication terminal device 114 is used as a tool for supporting the nurse 22 who has received a notification from the server system 110 to quickly go to the patient 16.

Here, in the monitoring system 100 of the present embodiment, when the patient 16 lying on the bed 20 in the hospital room 18 (who is restricted from walking alone) tries to get out of the bed 20, the imaging device 104 captures an image. Based on the analysis result of the image, it is predicted that the patient 16 will leave the bed, and the communication terminal device 114 will be notified.

FIG. 4 is a functional block diagram showing details of each part for realizing monitoring of the patient 16 by the imaging system 102, the server system 110, and the communication terminal device 114, which is a client system, of the monitoring system 100.

(Imaging system 102)

The imaging device 104 is controlled by the imaging control unit 116 of the imaging system 102, and is controlled to specifications (sensitivity, exposure, etc.) appropriate for the conditions of the installation location (hospital room 18), and the set imaging time period (24 hours is preferable in this embodiment, but it may also be set to detect a person and start operation (start imaging)). The imaging control unit 116 also takes in image data captured by the imaging device 104 and sends it to the calculation unit 118.

The calculation unit 118 analyzes (calculates) the arrangement pattern of the markers 14 based on the image data, and determines the condition of the patient 16.

The calculation unit 118 is connected to the holding/comparing unit 120 and sends the discrimination information (the condition of the patient 16) to the holding/comparing unit 120.

The storage and comparison unit 120 accumulates and stores the received discrimination information, and compares the discrimination information stored in chronological order with the previously stored discrimination information to determine whether there has been a change.

The holding/comparing unit 120 notifies the first transmitting/receiving unit 122 when there is a change in the discrimination information. The first transmitting/receiving unit 122 sends the image data before and after the change along the time axis to the second transmitting/receiving unit 124 of the server system 110. That is, the imaging system 102 notifies the server system 110 of the identification information of the patient 16 and the image data (log data) before and after the movement occurs only when there is movement of the sleeping patient 16. There is.

(Server system 110)

The server system 110 includes an image recording unit 126, and records in chronological order the identification information of the patient 16 and the image data before and after the change (log data) received by the second transmitting/receiving unit 124.

The image recording section 126 is connected to a change amount comparison section 128. The change amount comparison unit 128 analyzes the image data before and after the change in the patient 16, out of the log data recorded in chronological order in the image recording unit 126, and determines whether or not a characteristic change has occurred. .

The characteristic change refers to a change in movement when the patient 16 gets out of bed and tries to walk.

The change amount comparison unit 128 is connected to the third transmission/reception unit 130, and if it is determined that a characteristic change has occurred, it outputs an emergency notification to the image recording unit 126 and outputs bed exit indication information to the third transmission/reception unit 130.

The image recording unit 126 that has received the emergency notification sends the log data that has been determined to have a characteristic change to the third transmitting/receiving unit 130.

The third transmitting/receiving section 130 transmits the bed leaving sign information received from the change amount comparing section 128 and the log data as a set to the fourth transmitting/receiving section 132 of the communication terminal device 114 .

(Communication terminal device 114 “client system”)

The communication terminal device 114 as a client system has a user interface 134. The user interface 134 is configured with a touch panel, and upon receiving the bed leaving sign information and log data received by the fourth transmitting/receiving unit 132, it outputs an alarm sound as necessary, and displays the bed leaving sign information and log data on the touch panel. Is displayed.

Triggered by the display on the user interface 134 (touch panel) and the output of the alarm sound, the nurse 22 in possession of the communication terminal device 114 identifies the patient 16 from the log data and rushes to the hospital room 18. This makes it possible to extremely increase the probability that the patient 16 will arrive at the hospital room 18 before walking out.

The operation of this embodiment will be explained below with reference to the flowcharts in Figures 5 to 7.

5 to 7 explain the processing flow of the normal monitoring mode, which is the main mode, among the plurality of monitoring modes that can be executed using the monitoring system 100 of this embodiment.

(Operation control of imaging system 102)

FIG. 5 is a flowchart showing an imaging control routine in the imaging system 102.

In step 150, image data is captured from the imaging device 104, and then the process proceeds to step 152, where the arrangement pattern of the markers 14, etc., is analyzed (calculated) based on the image data to determine the condition of the patient 16.

In the next step 154, the discrimination information is stored, and then the process proceeds to step 156 to determine whether the current condition of the patient 16 has changed from the previously stored condition.

If an affirmative determination is made in step 156, the process proceeds to step 158, where the log data is output to the server system 110, and the process proceeds to step 160.

Further, if a negative determination is made in step 156, the process moves to step 160.

In step 160, it is determined whether or not to continue monitoring. If the determination in step 160 is negative, the process returns to step 150 and the above process is repeated. If the determination in step 160 is positive, the routine ends and the function of the monitoring system 100 is stopped.

Note that in the case of 24-hour monitoring in the monitoring system 100, the determination in step 160 is always "No," but in the case where a monitoring time period is set in advance or when monitoring is performed at regular intervals, an affirmative determination is made. may be done.

(Operation control of server system 110)

FIG. 6 is a flowchart showing a monitoring control routine in the server system 110.

In step 170, it is determined whether or not log data has been received from the imaging system 102. If an affirmative determination is made, the process proceeds to step 172, where the log data is recorded in chronological order, and the process proceeds to step 174.

In step 174, the change comparison unit 128 analyzes the image data of the patient 16 before and after the change.

In the next step 176, it is determined whether the analysis in step 174 has revealed any characteristic changes (such as signs that the person is about to start walking on their own).

If a negative determination is made in step 176, it is determined that the patient 16 will not leave the bed 20, and the process returns to step 170. Further, if an affirmative determination is made in step 176, it is determined that the patient 16 is making a movement that indicates that the patient 16 will leave the bed 20, and the process proceeds to step 178.

In step 178, the change comparison unit 128 issues an emergency notification to the image recording unit 126, and then the process proceeds to step 180, where related log data is read from the image recording unit 126, and the process proceeds to step 182.

In step 182, log data and bed leaving sign information are output to the communication terminal device 114, and the process returns to step 170 to repeat the above steps.

(Operation control of communication terminal device 114 (client system))

FIG. 7 is a flowchart showing a notification control routine in the communication terminal device 114.

In step 190, it is determined whether log data and bed leaving sign information have been received. If an affirmative determination is made in step 190, the process moves to step 192, and an alarm is output from the output device (speaker, etc.) of the communication terminal device 114. Note that the output device is not limited to an auditory signal from a speaker or the like, but may also be a tactile signal from a vibrator or the like, or a visual signal from a lighting or blinking LED light source. Other five senses (smell, taste) may be used as needed. Furthermore, the alarm is not limited to a specific communication terminal device 114 (the communication terminal device 114 owned by the nurse 22 in charge of the patient 16), but can be sent to a plurality of communication terminal devices 114 or to the management of the nurse station 21. It may also be notified to a PC or the like.

In the next step 194, the log data and bed exit indicator information are displayed on the communication terminal device 114, and then the process proceeds to step 196 to identify the patient 16, and then to step 198.

In step 198, it is determined whether an operation such as an information confirmation reset has been performed, and if the determination is negative, the alarm output in step 192 and the information display in step 194 are continued.

Further, if an affirmative determination is made in step 198, the process proceeds to step 199, where the alarm output and information display are reset, and the process returns to step 190.

As described above, in this embodiment, invisible markers 14, such as IR reflective dyes, are attached to the bedding 12, and an image of the patient 16 lying in bed 20 is captured by an imaging device (infrared camera, etc.) 104, so that the position of the markers 14 can be reliably captured in an unlit hospital room 18. Based on changes in the arrangement pattern of the markers 14, the system is able to determine the signs that the patient 16 is about to get out of bed 20 and start walking, and notify the nurse 22. This makes it possible to send a nurse 22 to the bed before the patient 16, who has restrictions on walking alone, gets out of bed.

(Modified example)

In this embodiment, the flow of processing in the normal monitoring mode among the monitoring modes that can be executed using the monitoring system 100 has been described. As the overall flow of the monitoring system 100, the processes are executed in the order shown in a1 to a11 in FIGS. 5 to 7, as shown in (a) below.

(a) Normal monitoring mode

The image captured by the imaging device 104 is sent to the calculation unit 118 via the imaging control unit 116 for analysis (a1), the calculation result is stored in the storage/comparison unit 120 (a2), and sent to the first transmission/reception unit 122 (a3), which then transmits it to the second transmission/reception unit 124 of the server system 110 (a4).

The second transmitting/receiving unit 124 sends the received analysis image to the image recording unit 126 (a5), and predicts bed leaving in cooperation with the change amount comparing unit 128 (a6).

If there is a possibility of getting out of bed, the change amount comparison unit 128 notifies the image recording unit 126 and the third transmitting/receiving unit 130 (a7), and the change amount comparison unit 128 sends a reply to the image recording unit 126 to record the image. The log data is output from the recording unit 126 to the third transmitting/receiving unit 130 (a8, a9), and is transmitted to the fourth transmitting/receiving unit 132 of the communication terminal device 114 (a10).

The fourth transmitting/receiving unit 132 sends the received analysis image and notification to the user interface 134, and notifies the nurse 22 or the like who has the communication terminal device 114 that there is a possibility of leaving the bed.

On the other hand, in a modified example, the image monitoring mode and the arbitrary monitoring mode can be executed as parallel processing with the normal monitoring mode or in place of the normal monitoring mode.

Each mode is shown below as a modified example.

“Modification 1: Image monitoring mode”

In addition to the normal monitoring mode, the image monitoring mode is a mode in which images captured by an imaging device are transmitted to a communication terminal device owned by the nurse when there is a possibility of the nurse leaving the bed.

In the image monitoring mode according to the first modification, when the change amount comparing unit 128 of the server system 110 determines that there is a characteristic change, the bed leaving sign information is output to the second transmitting/receiving unit 124 (see FIG. 4). (See dotted line b1).

The second transmitter/receiver 124 outputs the bed leaving sign information to the first transmitter/receiver 122 of the imaging system 102 (see dotted line b2 in FIG. 4).

Upon receiving this bed exit indication information, the first transceiver 124 acquires image data captured in real time from the imaging control unit 116 (see b3 and b4 in FIG. 4) and returns the data to the second transceiver 124 of the server system 110 (see b5 in FIG. 4).

The second transmitting/receiving section 124 outputs the image data to the fourth transmitting/receiving section 132 of the communication terminal device 114 (see b7 in FIG. 4) via the third transmitting/receiving section 130 (see b6 in FIG. 4), and outputs the image data to the user interface 134. The condition of the patient 16 is displayed in real time (see b8 in FIG. 4).

"Variation 2: Optional monitoring mode"

The optional monitoring mode is a mode in which images captured by an imaging device are simply viewed on the communication terminal device 114 based on the judgment of the nurse 22.

When the nurse 22 operates the user interface 134 of the communication terminal device 114 to instruct a request for image data, the request is notified to the third transmitting/receiving section 130 of the server system 110 via the fourth transmitting/receiving section 132. (See c1 and c2 in Figure 4).

The third transmitter/receiver 130 transmits the request notification to the first transmitter/receiver 122 of the imaging system 102 via the second transmitter/receiver 124 (see c3 and c4 in FIG. 4).

In response to this image data request instruction, the first transmitting/receiving unit 124 acquires the image data being captured in real time from the imaging control unit 116 (see c5 and c6 in FIG. 4), and the second transmitting/receiving unit 124 of the server system 110 124 (see c7 in FIG. 4).

The second transmitter/receiver 124 outputs the image data to the fourth transmitter/receiver 132 of the communication terminal device 114 (see c9 in FIG. 4) via the third transmitter/receiver 130 (see c8 in FIG. 4), and outputs the image data to the user interface 134. The condition of the patient 16 is displayed in real time (see c10 in FIG. 4).

Note that the monitoring system of the present invention can be applied to various situations other than those described in the embodiment (including modifications). Further, depending on the application situation, effects other than the main purpose and effect can be obtained. Below, examples of its application and effects are listed.

(1) It is assumed that the system will notify the nurse 22 when it detects when the patient 16 is about to leave the bed. However, it may also be possible to notify the patient himself/herself when he/she is about to leave the bed while in a state of drowsiness, such as when he/she is completely drunk. Once he/she regains consciousness, he/she will be able to walk without falling.

(2) Monitored subjects who are restricted from walking alone are not limited to patients 16 admitted to hospitals, but may also be people receiving care at home, infants, and pets. It can be used not only to predict signs of getting out of bed, but also to understand posture when walking, position in the room, etc.

(3) In addition to comparing the order of a series of movements and the amount of change with a reference pattern, the definition of "when a change in pattern satisfies certain conditions..." includes the displacement of bedding due to breathing and the number of tossing and turning. etc., may be compared with past big data, etc., to learn the pattern up to when an individual leaves the bed, and to predict the individual's time to leave the bed.

(4) As an additional effect, for example, in nursing care facilities, the movement of patients may be restricted as a last-ditch measure to prevent them from leaving the bed, and if the monitoring system of the present invention can predict the patient's departure from the bed, it will reduce the burden on the patient. can be alleviated.

(5) In order to prevent the marker 14 from impairing the design of the bedding 12, an invisible member that reflects infrared rays is used, but the imaging side should have a function that allows extraction of only a specific color. However, if a pattern (a pattern with good design) defined by the specific color is formed by mixing the specific color with different surrounding colors without any discomfort, it does not need to be invisible.

12 Bedding 12A Pajamas 12B Futon 12C Pillow 14 (14A, 14B, 14C) Marker (marker)
16 Patient 18 Hospital room 18A Ceiling surface 20 Bed 21 Nurse station 22 Nurse 100 Monitoring system (monitoring department)
102 Imaging system (imaging unit)
104 Imaging device 106 Control unit 108 Network 110 Server system 114 Communication terminal device 116 Imaging control unit 118 Arithmetic unit 120 Holding/comparison unit (analysis unit)
122 First transmitting/receiving section 124 Second transmitting/receiving section 126 Image recording section 128 Change amount comparison section (prediction section)
130 Third transmitting/receiving section 132 Fourth transmitting/receiving section 134 User interface (notification section)

Claims (7)

a marker unit that is attached to an object of interest that deforms in accordance with the movement of the object to be monitored, and that is displaced in accordance with the deformation of the object of interest;
an imaging unit that captures an image with an angle of view that includes the object of interest and outputs an image signal;
a monitoring unit that monitors the operation of the monitoring target by identifying the marker unit from the image signal output from the imaging unit;
The monitoring unit,
an analysis unit that analyzes displacement of the position of the marker unit in time series of images captured by the imaging unit;
a prediction unit that predicts the movement of the monitoring target from the current state based on the analysis result by the analysis unit;
and a notification unit that notifies at least a prediction result to a monitor when the change in movement predicted by the prediction unit satisfies a certain condition,
There are multiple objects of interest to which the sign portion is attached,
The marking portions are attached with different shapes depending on the type of the object of interest and on the front and back sides,
monitoring equipment. The monitoring device according to claim 1, wherein the marker portion is made of a material that reflects or absorbs at least invisible light or has luminous properties . The monitoring device according to claim 1 or 2, wherein the marker portion is attached in a known pattern in advance . Any one of claims 1 to 3, wherein the monitoring target is a person who is in bed and is a person with restricted movement who is restricted from leaving the bed and walking alone from the bed. The monitoring device according to item 1. A sign part made of a material that reflects invisible light, a material that absorbs invisible light, or a material that has luminous properties and is provided on bedding necessary for going to bed, and an imaging part that takes an image of an angle of view that includes the bedding. Prepare,
A monitoring method for detecting leaving bed from the behavior of a monitored object while sleeping, the method comprising:
Analyzing the output signal captured and output by the imaging unit to identify the displacement of the marking unit,
Determining the state of the monitoring target while sleeping from the displacement state of the indicator part,
Notifying a monitor when a change in the state of the monitored object satisfies a certain condition;
Monitoring methods. The monitoring method according to claim 5, wherein the bedding includes at least two or more of pajamas, a front and back of a duvet, and a pillow, and the state of the monitored object is determined based on the displacement state of each of the differently shaped marking parts. computer,
operating as the monitoring unit of the monitoring device according to any one of claims 1 to 4;
monitoring program.