JP7483486B2 - Monitoring system, monitoring device, monitoring method, and program - Google Patents

Description

This disclosure relates to a monitoring system, a monitoring device, a monitoring method, and a program.

In a field such as a nursing home, for example, a device has been proposed for monitoring the behavior of a monitored person, such as a person requiring nursing care or long-term care. For example, Patent Document 1 discloses a monitored person monitoring system that detects a predetermined behavior of the monitored person based on an image obtained by an imaging device. Patent Document 2 discloses a fall prevention system that prevents a subject from falling while walking by attaching a detection device to the subject's foot. Patent Document 3 discloses a monitoring support device that determines the body position of the human body by detecting temperature distribution. Furthermore, Cited Document 4 discloses a medical system for monitoring geriatric mental patients at home or in a nursing home or care facility.

JP 2017-91552 A JP 2017-221502 A JP 2014-106636 A JP 2003-91790 A

It would be beneficial if monitoring the monitored subject could contribute to their safety.

The objective of this disclosure is to provide a monitoring system, monitoring device, monitoring method, and program that can ensure the safety of monitored subjects.

A monitoring system according to an embodiment includes:
An imaging unit that images an object to be monitored;
an extraction unit that extracts a movement of a feature point of the monitored object from an image captured by the imaging unit;
a controller that outputs a predetermined warning signal before the monitored subject completes a standing-up motion, based on machine learning data obtained by machine learning of the movement of the feature points when the monitored subject stands up and the feature points extracted by the extraction unit;
a warning unit that issues a predetermined warning based on the predetermined warning signal output from the controller;
Equipped with.
The controller outputs the predetermined warning signal when the monitored object does not complete a standing up motion within a predetermined time.

A monitoring device according to an embodiment includes:
an extraction unit that extracts a movement of a feature point of a monitored object from an image of the monitored object;
a controller that outputs a predetermined warning signal before the monitored subject completes a standing-up motion, based on machine learning data obtained by machine learning of the movement of the feature points when the monitored subject stands up and the feature points extracted by the extraction unit;
Equipped with.
The controller outputs the predetermined warning signal when the monitored object does not complete the standing up action within a predetermined time.

A monitoring method according to an embodiment includes the steps of:
extracting a movement of a feature point of a monitored object from an image of the monitored object;
a step of outputting a predetermined warning signal before the monitored subject completes a standing-up motion based on machine learning data obtained by machine learning the movement of the feature points when the monitored subject stands up and the feature points extracted in the extracting step;
Including,
If the monitored object does not complete the standing up action within a predetermined time, the predetermined warning signal is output.

A program according to an embodiment includes:
On the computer,
extracting a movement of a feature point of a monitored object from an image of the monitored object;
a step of outputting a predetermined warning signal before the monitored subject completes a standing-up motion based on machine learning data obtained by machine learning the movement of the feature points when the monitored subject stands up and the feature points extracted in the extracting step;
Run the command ,
If the monitored object does not complete the standing up action within a predetermined time, the predetermined warning signal is output.

According to one embodiment, it is possible to provide a monitoring system, a monitoring device, a monitoring method, and a program that can ensure the safety of the monitored subject.

1 is a functional block diagram showing a schematic configuration of a monitoring system according to an embodiment. 10 is a flowchart illustrating an example of an operation of the monitoring system according to an embodiment. 11 is a flowchart illustrating an example of an operation of the monitoring system according to an embodiment. 1 is a diagram illustrating an example of feature points extracted in a surveillance system according to an embodiment. FIG. 1 is a diagram illustrating an example of feature points extracted in a surveillance system according to an embodiment. FIG. 1 is a diagram illustrating an example of feature points extracted in a surveillance system according to an embodiment. FIG. 1 is a diagram illustrating an example of feature points extracted in a surveillance system according to an embodiment. FIG. 1 is a diagram illustrating an example of feature points extracted in a surveillance system according to an embodiment. FIG. 1 is a diagram illustrating an example of feature points extracted in a surveillance system according to an embodiment. FIG. 1 is a diagram illustrating an example of feature points extracted in a surveillance system according to an embodiment. FIG. 1 is a diagram illustrating an example of feature points extracted in a surveillance system according to an embodiment. FIG. 1 is a diagram illustrating an example of feature points extracted in a surveillance system according to an embodiment. FIG. 1 is a diagram illustrating an example of feature points extracted in a surveillance system according to an embodiment. FIG. 1 is a diagram illustrating an example of feature points extracted in a surveillance system according to an embodiment. FIG.

In the present disclosure, a "monitoring device" may be a device that is powered by electricity. Furthermore, a "monitoring system" may include a device that is powered by electricity. Furthermore, a "user" may be a person (typically a human) that uses a monitoring system and/or a monitoring device according to an embodiment. A user may include a person who monitors a monitored subject by using a monitoring system and/or a monitoring device according to an embodiment. Furthermore, a "monitored subject" may be a person (e.g., a human or an animal) that is the subject of monitoring by a monitoring system and/or a monitoring device according to an embodiment.

The monitoring system according to one embodiment is assumed to be used in specific facilities used by people engaged in social activities, such as companies, hospitals, nursing homes, schools, sports gyms, and care facilities. For example, in a company, it is extremely important to understand and/or manage the health status of employees. Similarly, in a hospital, it is extremely important to understand and/or manage the health status of patients and medical professionals, and in a nursing home, it is extremely important to understand and/or manage the health status of residents and staff. The monitoring system according to one embodiment is not limited to the above-mentioned facilities such as companies, hospitals, and nursing homes, but may be any facility where it is desired to understand and/or manage the health status of a monitored subject. Any facility may also include non-commercial facilities such as a user's home. The monitoring system according to one embodiment may also be used in, for example, inside moving objects such as trains, buses, and airplanes, as well as at stations and boarding points.

The monitoring system according to one embodiment may be used, for example, in a care facility or the like to monitor the behavior of a monitored subject, such as a person requiring nursing care or care. The monitoring system according to one embodiment may monitor the standing up motion of a monitored subject, such as a person requiring nursing care or care. Here, the standing up motion of the monitored subject may be, for example, the motion of the monitored subject standing up from a sitting or lying position.

In particular, the monitoring system according to one embodiment can issue a predetermined warning before a monitored person, such as a person requiring nursing care or care, stands up, for example, before completing the standing up motion. Therefore, according to the monitoring system according to one embodiment, staff at a care facility, for example, can know that a monitored person, such as a person requiring nursing care or care, is about to stand up before completing the standing up motion.

The monitoring system according to one embodiment can also issue a predetermined warning before a monitored person, such as a person requiring nursing care or care, makes a dangerous standing-up motion, for example, before completing the dangerous standing-up motion. The monitoring system according to one embodiment can also issue a predetermined warning before a specific monitored person makes a standing-up motion, for example, before completing the standing-up motion.

The monitoring system according to one embodiment will be described in detail below with reference to the drawings.

FIG. 1 is a diagram showing a schematic configuration of a monitoring system according to one embodiment. As shown in FIG. 1, the monitoring system 1 according to one embodiment may be configured to include a monitoring device 10 and an imaging unit 20. The monitoring device 10 and the imaging unit 20 may be connected by wire or wirelessly, or by a combination of wire and wireless. The monitoring system 1 according to one embodiment may not include some of the functional units shown in FIG. 1, or may include functional units other than those shown in FIG. 1. For example, the monitoring system 1 according to one embodiment may not include at least one of the warning unit 17 and the communication unit 19.

The imaging unit 20 shown in FIG. 1 may be configured to include an image sensor that electronically captures an image, such as a digital camera. The imaging unit 20 may be configured to include an imaging element that performs photoelectric conversion, such as a CCD (Charge Coupled Device Image Sensor) or a CMOS (Complementary Metal Oxide Semiconductor) sensor. The imaging unit 20 may capture an image of the monitored object T, for example, as shown in FIG. 1. Here, the monitored object T may be, for example, a human being. The imaging unit 20 may convert the captured image into a signal and transmit it to the monitoring device 10. For example, the imaging unit 20 may transmit a signal based on the captured image to the extraction unit 11, the storage unit 13, and/or the controller 15 of the monitoring device 10. The imaging unit 20 is not limited to an imaging device such as a digital camera, and may be any device that captures an image of the monitored object T.

In one embodiment, the imaging unit 20 may capture, for example, still images of the monitored object T at predetermined time intervals (for example, every 0.5 seconds). Also, in one embodiment, the imaging unit 20 may capture, for example, continuous video of the monitored object T.

As shown in FIG. 1, the monitoring device 10 according to one embodiment may include an extraction unit 11, a memory unit 13, a controller 15, a warning unit 17, and a communication unit 19. The monitoring device 10 according to one embodiment may not include some of the functional units shown in FIG. 1, or may include functional units other than those shown in FIG. 1. For example, the monitoring device 10 according to one embodiment may include machine learning data 132 (described below) stored in the memory unit 13. For example, the monitoring device 10 according to one embodiment may have at least a portion of the machine learning data 132 (described below) stored in an external device such as an external server.

The extraction unit 11 may have a function of extracting a predetermined feature point from an image captured by the imaging unit 20. For example, the extraction unit 11 may extract the movement of the feature point of the monitored target T from the image of the monitored target T captured by the imaging unit 20. Here, the feature point will be described further below. In one embodiment, the extraction unit 11 may extract the movement of each part of the monitored target T, such as the head, trunk, limbs, and/or each joint, from the image of the monitored target T captured by the imaging unit 20. The extraction unit 11 may be configured as dedicated hardware, may be configured to include software at least in part, or may be configured entirely by software. In this way, the extraction unit 11 may extract the movement of the feature point of the monitored target T from the image captured by the imaging unit 20.

The storage unit 13 may function as a memory that stores various types of information. The storage unit 13 may store, for example, a program executed in the controller 15 and the results of processing executed in the controller 15. The storage unit 13 may also function as a work memory for the controller 15. The storage unit 13 may be configured, for example, by a semiconductor memory or the like, but is not limited to this and may be any storage device. For example, the storage unit 13 may be a storage medium such as a memory card inserted into the monitoring device 10 according to one embodiment. The storage unit 13 may also be an internal memory of a CPU used as the controller 15 described below, or may be connected to the controller 15 as a separate unit.

As shown in FIG. 1, the storage unit 13 may store, for example, machine learning data 132. Here, the machine learning data 132 may be data generated by machine learning. Machine learning may be an AI (Artificial Intelligence) technology that enables a specific task to be executed by training. More specifically, machine learning may be a technology in which an information processing device such as a computer learns a large amount of data and automatically constructs an algorithm or model that performs tasks such as classification and/or prediction. In this specification, machine learning may be included as a part of AI (Artificial Intelligence). In this specification, machine learning may include supervised learning that learns the characteristics or rules of input data based on correct answer data, unsupervised learning that learns the characteristics or rules of input data in the absence of correct answer data, and reinforcement learning that learns the characteristics or rules of input data by giving rewards or punishments. In this specification, machine learning may be any combination of supervised learning, unsupervised learning, and reinforcement learning. The concept of the machine learning data 132 in this embodiment may include an algorithm that outputs a predetermined inference result using an algorithm learned for input data. In this embodiment, as the algorithm, for example, linear regression that predicts the relationship between dependent variables and independent variables, a neural network (NN) that mathematically models the neurons of the human nervous system, the least squares method that calculates by squaring the error, a decision tree that creates a tree structure for problem solving, regularization that transforms data in a predetermined manner, and other appropriate algorithms can be used. This embodiment may use deep learning, which is a type of neural network. Deep learning is a type of neural network, and a neural network with a deep network hierarchy is called deep learning.

In particular, in one embodiment, the machine learning data 132 may be data that has been machine-learned on the movement of characteristic points when the monitored target T stands up. The machine learning data 132 may also be data that has been machine-learned on the movement of characteristic points when the monitored target T makes a dangerous standing up motion. Furthermore, the machine learning data 132 may be data that has been machine-learned on the movement of characteristic points when a specific person (e.g., monitored target T1) as the monitored target T stands up. The machine learning data 132 according to one embodiment will be described further below.

The controller 15 controls and/or manages the entire monitoring device 10, including each functional unit constituting the monitoring device 10. The controller 15 may include at least one processor, such as a CPU (Central Processing Unit), to provide control and processing power for executing various functions. The controller 15 may be realized as a single processor, as a number of processors, or as individual processors. The processor may be realized as a single integrated circuit. An integrated circuit is also called an IC (Integrated Circuit). The processor may be realized as multiple communicatively connected integrated circuits and discrete circuits. The processor may be realized based on various other known technologies.

In one embodiment, the controller 15 may be configured as, for example, a CPU and a program executed by the CPU. The program executed in the controller 15 and the results of the processing executed in the controller 15 may be stored in, for example, the storage unit 13. The controller 15 may include memory necessary for the operation of the controller 15 as appropriate. The operation of the controller 15 of the monitoring device 10 according to one embodiment will be described further below.

The warning unit 17 may issue a predetermined warning to alert the user of the monitoring system 1 or the monitoring device 10, based on a predetermined warning signal output from the controller 15. The warning unit 17 may be any functional unit that stimulates at least one of the user's hearing, vision, and touch, such as sound, voice, light, text, video, and vibration, as the predetermined warning. Specifically, the warning unit 17 may be at least one of a sound output unit such as a buzzer or speaker, a light-emitting unit such as an LED, a display unit such as an LCD, and a tactile sensation providing unit such as a vibrator. In this way, the warning unit 17 may issue a predetermined warning based on a predetermined warning signal output from the controller 15. In one embodiment, the warning unit 17 may issue the predetermined alarm as information that acts on at least one of hearing, vision, and touch.

In one embodiment, the warning unit 17 may issue a warning that the monitored object T is going to stand up before the monitored object T stands up. In one embodiment, the warning unit 17 may issue a warning that the monitored object T is going to stand up before the monitored object T stands up. In one embodiment, the warning unit 17 may issue a warning that a specific person is going to stand up. For example, in one embodiment, the warning unit 17 that outputs visual information may warn the user of the fact that the monitored object T is going to stand up by emitting light or a predetermined display when it is detected that the monitored object T is going to stand up. In one embodiment, the warning unit 17 that outputs audio information may warn the user of the fact that the monitored object T is going to stand up by a predetermined sound or voice when it is detected that the monitored object T is going to stand up. In this embodiment, the warning may be a combination of light emission or a predetermined display and a predetermined sound or voice.

The monitoring device 10 shown in FIG. 1 has a built-in warning unit 17. However, in one embodiment of the monitoring system 1, the warning unit 17 may be provided outside the monitoring device 10. In this case, the warning unit 17 and the monitoring device 10 may be connected by wire or wirelessly, or by a combination of wire and wireless.

The communication unit 19 has an interface function for wired or wireless communication. The communication method performed by the communication unit 19 in one embodiment may be a wireless communication standard. For example, the wireless communication standard includes cellular phone communication standards such as 2G, 3G, 4G, and 5G. For example, the cellular phone communication standard includes LTE (Long Term Evolution), W-CDMA (Wideband Code Division Multiple Access), CDMA2000, PDC (Personal Digital Cellular), GSM (registered trademark) (Global System for Mobile communications), and PHS (Personal Handy-phone System). For example, the wireless communication standard includes WiMAX (Worldwide Interoperability for Microwave Access), IEEE 802.11, WiFi, Bluetooth (registered trademark), IrDA (Infrared Data Association), and NFC (Near Field Communication). The communication unit 19 can support one or more of the above communication standards. The communication unit 19 may be configured to include, for example, an antenna for transmitting and receiving radio waves and an appropriate RF unit. The communication unit 19 may also be configured as an interface such as a connector for wired connection to the outside. The communication unit 19 can be configured using known technology for wireless communication, so a detailed description of the hardware will be omitted.

The various information received by the communication unit 19 may be supplied to, for example, the storage unit 13 and/or the controller 15. The various information received by the communication unit 19 may be stored in, for example, a memory built into the storage unit 13 and/or the controller 15. In addition, the communication unit 19 may transmit, for example, the processing results by the controller 15, the extraction results by the extraction unit 11, and/or information stored in the storage unit 13 to the outside.

As shown in FIG. 1, at least a portion of the functional units constituting the monitoring device 10 according to one embodiment may be constituted by specific means in which software and hardware resources work together.

Next, we will explain the operation of the monitoring system 1 according to one embodiment.

FIG. 2 is a flowchart showing an example of the operation of the monitoring system 1 according to an embodiment. The operation shown in FIG. 2 may be started when monitoring the standing up motion of the monitored subject T in, for example, a hospital or a care facility. For example, in a care facility or hospital, there may be elderly or injured people whose legs are weak and who are at high risk of falling if they try to stand up on their own. In addition, for example, patients with dementia may be at risk of wandering or getting lost if they stand up on their own. By monitoring such a monitored subject T with the monitoring system 1, staff at, for example, a care facility or hospital can recognize that the monitored subject T is trying to stand up on his or her own before the monitored subject T actually stands up.

2 starts, the imaging unit 20 captures an image of the monitored object T in the monitoring system 1 according to one embodiment (step S1). In step S1, the imaging unit 20 may capture an image of the monitored object T. In step S1, the imaging unit 20 may start capturing an image of the monitored object T at this point, or may continue capturing an image of the monitored object T that has already started. In addition, in step S1, the imaging unit 20 may capture an image of the monitored object T as a still image, for example, at predetermined intervals (for example, every 0.5 seconds). In addition, in step S1, the imaging unit 20 may capture an image of the monitored object T as a continuous video. The image of the monitored object T captured by the imaging unit 20 in step S1 is supplied to the extraction unit 11 of the monitoring device 10. In addition, the imaging unit 20 may capture an image of the monitored object T, for example, as a combination of a still image at predetermined intervals (for example, every 0.5 seconds) and a continuous video. In this embodiment, the capacity of the image data may be reduced by shifting from a video capture mode to a still image mode. In this embodiment, when there is little movement of the monitored subject T, such as at night, the volume of image data may be reduced by switching from video capture mode to still image mode.

When the monitored object T is imaged in step S1, the extraction unit 11 of the monitoring device 10 extracts the movement of the characteristic points of the monitored object T from the image captured by the imaging unit 20 (step S2).

Figures 4 and onward are diagrams that show examples of feature points of the monitored target T extracted by the extraction unit 11 based on the image of the monitored target T captured by the imaging unit 20. Figure 4 shows an example of feature points extracted from an image of the monitored target T sitting in a chair C1 captured from the left side.

In one embodiment, the feature points of the monitored target T extracted by the extraction unit 11 may be the following feature points, for example, as shown in FIG. 4 and subsequent figures.
The characteristic point P1 may be, for example, the head of the monitored object T.
The characteristic point P2 may be, for example, the connection between the head and neck of the monitored subject T.
The characteristic point P3 may be, for example, the connection between the neck and torso of the monitored subject T.
The characteristic point P4 may be, for example, the connection between the trunk and the legs of the monitored subject T.
The characteristic point P5 may be, for example, the shoulder of the monitored object T.
The characteristic point P6 may be, for example, the elbow of the monitored object T.
The characteristic point P7 may be, for example, the hand of the monitored subject T.
The characteristic point P8 may be, for example, the knee of the monitored object T.
The characteristic point P9 may be, for example, the foot of the monitored object T.

Returning to the explanation of FIG. 2, in step S2, the extraction unit 11 may extract feature points using existing technology such as image recognition. Also, in step S2, the extraction unit 11 may extract feature points using technology based on, for example, AI.

In step S2, the extraction unit 11 may extract the movement of the characteristic points from a plurality of successive still images of the monitored object T captured by the imaging unit 20. Also, in step S2, the extraction unit 11 may extract the movement of the characteristic points from a video of the monitored object T captured by the imaging unit 20.

Once the movement of the feature points is extracted in step S2, the controller 15 of the monitoring device 10 acquires the machine learning data 132 (step S3). In step S3, the controller 15 may read the machine learning data 132 from the memory unit 13. Furthermore, if the necessary machine learning data 132 is not stored in the memory unit 13, the controller 15 may acquire the necessary machine learning data 132, for example, from an external server. In this case, the controller 15 may receive the necessary machine learning data 132 via the communication unit 19. The machine learning data 132 received in this manner may be stored, for example, in the memory unit 13 or an internal memory of the controller 15. The machine learning data 132 will be further described below.

As described above, the monitoring device 10 according to one embodiment may determine that the monitored subject T will stand up before the monitored subject T stands up, based on the movement of the characteristic points of the monitored subject T. Before the monitored subject T stands up may be, for example, before the monitored subject T completes the standing up movement. For this reason, in the monitoring device 10 according to one embodiment, the controller 15 may monitor the movement of the characteristic points extracted from the captured image of the monitored subject T. Specifically, the controller 15 may monitor the positions of the multiple characteristic points extracted from the image of the monitored subject T at a predetermined timing. The controller 15 may also monitor the timing at which the multiple characteristic points extracted from the image of the monitored subject T arrive at a predetermined position.

In one embodiment, machine learning data 132 may be used as an object to be compared with the movement of the feature points extracted from the image of the monitored object T. This machine learning data 132 may also include an algorithm for performing machine learning. Here, in one embodiment, data obtained by machine learning the movement of the feature points when the monitored object T stands up may be used as the machine learning data 132. For example, the machine learning data 132 may include data based on the movement of the feature points extracted from the stand-up movement actually performed by the monitored object T in the past. In particular, the machine learning data 132 may include, for example, data based on the movement of the feature points extracted from the successful examples of the stand-up movement actually performed by the monitored object T in the past. Such machine learning data 132 may include, for example, data based on the movement of the feature points when the monitored object T stands up multiple times. By comparing such machine learning data 132 with the movement of the feature points extracted by the extraction unit 11, the controller 15 can determine whether the monitored object T stands up. In addition, the machine learning data 132 of this embodiment is not limited to data based on the movement of the feature points extracted from the successful examples of the stand-up movement actually performed by the monitored object T in the past. That is, the machine learning data 132 may be data based on the movement of feature points extracted from actions other than the standing up action actually performed by the monitored subject T up to now. For example, the machine learning data 132 may include data based on the movement of feature points extracted from actions other than the standing up action, such as the action of the monitored subject T picking up a cup or the action of turning toward a conversation partner.

As described below, the standing up motion performed by the monitored subject T includes at least one or more partial motions in a series of motions from a sitting state to the completion of the standing up motion. Therefore, if it can be determined that at least a part of at least one or more partial motions included in the series of motions from a sitting state to standing up has been performed, it can be estimated that the monitored subject T is performing (or is about to perform) a standing up motion. Such an estimation can be made before the monitored subject T completes the standing up motion. Therefore, according to one embodiment of the monitoring device 10, it is possible to detect that the monitored subject T is performing a standing up motion before the monitored subject T completes the standing up motion and issue a predetermined warning. In other words, based on the machine learning data 132, the monitoring device 10 can issue a predetermined warning before the monitored subject T performs a standing up motion.

Also, for example, the machine learning data 132 is not limited to data based on the movement of characteristic points extracted from successful examples of the standing up movement actually performed by the monitored subject T. In other words, the machine learning data 132 may be data based on the movement of characteristic points extracted from failed examples of the standing up movement actually performed by the monitored subject T. For example, the machine learning data 132 may be data based on the movement of characteristic points extracted from the movement of the monitored subject T when the monitored subject T attempts to stand up but fails (e.g. falls) or is on the verge of failing. Based on such machine learning data 132, the monitoring device 10 may not issue a predetermined warning when the probability that the monitored subject T's standing up movement will be successful (i.e. will not fail) is equal to or higher than a predetermined value. On the other hand, based on such machine learning data 132, the monitoring device 10 may issue a predetermined warning when the probability that the monitored subject T's standing up movement will not be successful (i.e. will not fail) is equal to or higher than a predetermined value.

For example, the machine learning data 132 may be supervised learning data that uses data labeled by a human to train an algorithm that determines whether the standing up motion of the monitored subject T is successful or not. Here, in the machine learning data 132, cases where the standing up motion of the monitored subject T is successful and cases where it fails may be labeled differently. In addition, in the machine learning data 132, different labels may be attached not only to cases where the standing up motion of the monitored subject T is simply successful or unsuccessful, but also to each standing up motion. For example, when the standing up motion of the monitored subject T is successful but not smooth, a label different from the case where the standing up motion is successful may be attached.

On the other hand, the machine learning data 132 may be unlabeled data provided as unsupervised learning data, allowing the algorithm that determines whether the monitored subject T's rising action is successful to find patterns.

Furthermore, the machine learning data 132 may be data that is machine-learned about the movement of characteristic points when a specific person (e.g., monitored subject T1) as the monitored subject T stands up. Based on such machine learning data 132, the monitoring device 10 can issue a predetermined warning before a specific person such as monitored subject T1 stands up.

Also, for example, the machine learning data 132 is not limited to data based on the movement of feature points extracted from successful examples of a standing up movement actually performed by a specific person such as the monitored subject T1. In other words, the machine learning data 132 may include data based on the movement of feature points extracted from failed examples of a standing up movement actually performed by a specific person such as the monitored subject T1. Based on such machine learning data 132, the monitoring device 10 may not issue a predetermined warning when the probability that a specific person such as the monitored subject T1 will succeed in standing up (i.e., will not fail) is equal to or higher than a predetermined value. On the other hand, based on such machine learning data 132, the monitoring device 10 may issue a predetermined warning when the probability that a specific person such as the monitored subject T1 will not succeed in standing up (i.e., will fail) is equal to or higher than a predetermined value.

When the machine learning data 132 is acquired in step S3, the controller 15 may determine whether or not the monitored subject T will stand up based on the feature points extracted by the extraction unit 11 and the machine learning data 132 (step S4). In step S4, the controller 15 may determine whether or not the monitored subject T will stand up based on a machine learning algorithm using AI technology that uses the machine learning data 132.

If it is determined in step S4 that the monitored subject T does not stand up, the controller 15 ends the operation shown in FIG. 2. After ending the operation shown in FIG. 2, the controller 15 may start the operation shown in FIG. 2 again continuously or at predetermined time intervals.

On the other hand, if it is determined in step S4 that the monitored subject T is about to stand up, the controller 15 may output a predetermined warning signal before the monitored subject T stands up (step S5). In step S5, the controller 15 may output the predetermined warning signal to the warning unit 17. Based on the predetermined warning signal output from the controller 15 in step S5, the warning unit 17 may issue a predetermined warning to the user.

In this way, the controller 15 may output a predetermined warning signal before the monitored subject T stands up, based on the feature points extracted by the extraction unit 11 and the machine learning data 132.

In step S4 shown in FIG. 2, the controller 15 determines whether the monitored subject T stands up. This allows the monitoring device 10 to issue a predetermined warning before the monitored subject T stands up, for example, before the monitored subject T completes the standing up motion.

In response to this, as described above, the controller 15 may determine whether the monitored subject T will perform a standing-up action that is relatively likely to be successful (i.e., safe) or a standing-up action that is relatively likely to fail (i.e., dangerous). In this case, the machine learning data 132 may be data based on the movement of characteristic points extracted from successful examples of standing-up actions actually performed by a specific person such as the monitored subject T1. In addition, in this case, the machine learning data 132 may be data based on the movement of characteristic points extracted from unsuccessful examples of standing-up actions actually performed by a specific person such as the monitored subject T1.

Figure 3 is a flowchart showing an example of the operation of the monitoring system 1 for monitoring dangerous rises as described above.

In FIG. 3, step S4 in FIG. 2 has been changed to step S14. That is, in step S4 in FIG. 2, the controller 15 judges whether the monitored object T will perform a standing up motion. In contrast, in step S14 in FIG. 3, the controller 15 may judge whether the monitored object T will perform a dangerous standing up motion (i.e., a motion with a relatively high probability of failure). Operations other than the operation in step S14 can be performed in the same manner as described in FIG. 2, and therefore further detailed description will be omitted.

In this way, the controller 15 may output a predetermined warning signal before the monitored subject T performs a dangerous standing-up motion, based on the feature points extracted by the extraction unit 11 and the machine learning data 132. Here, the machine learning data 132 may be data that is machine-learned on the movement of the feature points when the monitored subject T performs a dangerous standing-up motion.

As described above, the monitoring device 10 may also issue a predetermined warning before a specific person, such as the monitored subject T1, stands up. In this way, the controller 15 may output a predetermined warning signal before the specific person stands up, based on the feature points extracted by the extraction unit 11 and the machine learning data 132. Here, the machine learning data 132 may be data that is machine-learned on the movement of the feature points when a specific person (monitored subject T1), for example, as the monitored subject T, stands up.

Below, we further explain some of the findings gained from the demonstration experiment when generating machine learning data 132 based on the feature points of the monitored object T extracted by the extraction unit 11.

(When standing up from a chair without armrests)
4 to 7 are diagrams for explaining an example of extracting feature points of the motion of the monitored target T standing up from a chair without armrests. Fig. 4 to Fig. 7 show the movement of feature points when the monitored target T sitting on a chair C1 without armrests gradually stands up. Fig. 4 to Fig. 7 show schematic examples of feature points extracted from an image of the monitored target T captured from the left side. Here, the chair C1 may be a chair without armrests or armrests as shown in Fig. 4 to Fig. 7.

As shown in Figures 4 to 7, the series of movements in which the monitored subject T stands up from a position where he is sitting in chair C1 includes several partial movements. For example, Figure 4 shows a schematic example of feature points extracted from an image of the monitored subject T sitting in chair C1 captured from the left side, as described above.

A partial movement included in an example of a standing-up movement performed by the monitored target T from the state shown in FIG. 4 to the state shown in FIG. 5 may be, for example, the following movement.
(1) Movement of pulling back the leg (movement of feature point P9 (distance d1 and angle a1 shown in FIG. 5))
(2) Movement of taking a forward leaning posture (movement of the characteristic point P1, P2, or P3 (angle a2 shown in FIG. 5))
(3) Movement of putting the hand forward (movement of feature points P7 and/or P6 (distance d2 shown in FIG. 5))

Here, in a normal (safe) standing-up motion of the monitored subject T, the distance d1 in (1) above tends to be about 10 cm and the angle a1 to be about 70°. On the other hand, in a dangerous standing-up motion of the monitored subject T, the distance d1 in (1) above tends to be about 20 cm and the angle a1 to be about 50°. Hereinafter, a normal (safe) standing-up motion may be, for example, a motion where the probability of the standing-up motion being successful (i.e. not failing) is greater than or equal to a predetermined value. Also, a dangerous standing-up motion may be, for example, a motion where the probability of the standing-up motion not being successful (i.e. failing) is greater than or equal to a predetermined value, such as falling.

In addition, in a normal standing motion of the monitored subject T, the angle a2 in (2) above tends to be approximately 40°. On the other hand, in a dangerous standing motion of the monitored subject T, the angle a2 in (2) above tends to be approximately 70°.

In addition, the distance d2 in (3) above tends to be about 10 cm, both in normal and dangerous standing-up movements of the monitored subject T.

Next, partial movements included in an example of a standing up movement performed by the monitored target T from the state shown in Fig. 5 through the state shown in Fig. 6 to the state shown in Fig. 7 may be, for example, the following movements. Figs. 5 and 6 show examples of characteristic points extracted from an image of an example of a standing up movement performed by the monitored target T in the middle of the standing up movement. Fig. 7 shows an example of characteristic points extracted from an image of an example of a standing up movement completed by the monitored target T.
(4) Movement of lifting the buttocks (movement of the feature point P4 (distance d3 shown in FIG. 6))
(5) Knee straightening (movement of feature points P4 and/or P9 (angle a3 shown in FIG. 6))
(6) Head lifting motion (movement of feature point P1 (distance d4 shown in FIG. 6))
(7) Arm stretching motion (movement of feature point P7 (angle a4 shown in FIG. 6))
(8) Movement of straightening the back (movement of the characteristic point P1, P2, or P3 (angle a5 shown in FIG. 6))

Here, in a normal standing motion of the monitored subject T, the angle a3 in (5) above tends to change from about 70° to about 180°. On the other hand, in a dangerous standing motion of the monitored subject T, the angle a3 in (5) above tends to change from about 50° to about 150°.

Furthermore, in a normal standing-up motion of the monitored subject T, the distance d4 in (6) above tends to change from the position P1 shown in FIG. 6 to the position P1 shown in FIG. 7. On the other hand, in a dangerous standing-up motion of the monitored subject T, the distance d4 in (6) above tends to change from the position P1 shown in FIG. 6 to a position that is approximately 20 cm short of the distance to the position P1 shown in FIG. 7.

In addition, in a normal standing-up motion of the monitored subject T, the angle a4 in (7) above tends to change from approximately 90° to approximately 170°. On the other hand, in a dangerous standing-up motion of the monitored subject T, the angle a4 in (7) above tends to change from approximately 90° to approximately 160°.

Furthermore, in a normal standing-up motion of the monitored subject T, the angle a5 in (8) above tends to be approximately 0°. On the other hand, in a dangerous standing-up motion of the monitored subject T, the angle a5 in (8) above tends to be approximately 20°.

So far, we have explained the movement of the characteristic points when the monitored subject T stands up. Below, we will further explain the timing of the movement of the characteristic points when the monitored subject T stands up.

The results of the demonstration experiment showed that in the normal standing-up motion of the monitored subject T, the above steps (1) to (3) tend to be performed within approximately 0.6 seconds. Also, in the normal standing-up motion of the monitored subject T, the above steps (4) to (8) tend to be performed within approximately 1.2 seconds. In other words, the normal standing-up motion of the monitored subject T tends to be performed overall within approximately 2 seconds.

On the other hand, in the dangerous standing-up movement of the monitored subject T, the above actions (1) to (3) tend to occur over a period of about 3 seconds. Also, in the dangerous standing-up movement of the monitored subject T, the above actions (4) to (8) tend to occur over a period of about 3 seconds. In other words, the dangerous standing-up movement of the monitored subject T tends not to occur over a period of about 2 seconds, as is the case with the normal standing-up movement of the monitored subject T.

From the above, for example, if the monitored subject T has not completed the standing up motion two seconds after starting the standing up motion, the controller 15 of the monitoring device 10 may output a warning signal indicating that the standing up motion is dangerous. In this way, the monitoring device 10 can issue a warning to the user that the monitored subject T is performing a dangerous standing up motion before the monitored subject T completes the dangerous standing up motion.

The above actions (1) to (8) are included in a series of rising actions performed by the monitored object T. Therefore, for example, it can be assumed that when the above actions (1) to (3) are performed, the above actions (4) to (8) are subsequently performed. Therefore, for example, the controller 15 may output a warning signal indicating that the monitored object T is performing a dangerous rising action when it takes more than three seconds for the monitored object T to perform the above actions (1) to (3). In this way, the monitoring device 10 can issue a warning to the user that the monitored object T is performing a dangerous rising action a predetermined time (e.g., three seconds) before the monitored object T completes the dangerous rising action. In this way, the controller 15 may output a predetermined warning signal a predetermined time before the monitored object T completes the rising action.

Furthermore, in this embodiment, in the dangerous standing-up motion of the monitored object T, the controller 15 may output a warning signal indicating that the monitored object T is standing up if the monitored object T has not completed the above steps (1) to (3) within one second after starting the standing-up motion. Furthermore, in the dangerous standing-up motion of the monitored object T, the controller 15 may not output a warning signal indicating that the monitored object T is standing up if the monitored object T has completed the above steps (1) to (3) within one second after starting the standing-up motion. In this way, the monitoring device 10 can issue a warning to the user that the monitored object T is standing up dangerously a predetermined time before (for example, one second before) the monitored object T completes the dangerous standing-up motion. In this case, in this embodiment, whether or not to issue a warning is determined based on whether or not it took one second to complete the above steps (1) to (3), so that a warning is not issued for a person who is capable of standing up normally, but a warning is issued for a person who stands up dangerously. Therefore, in this embodiment, the reliability and usefulness of the warning can be improved. In addition, in this embodiment, the elapsed time for determining whether or not to output a predetermined warning signal is not limited to one second as described above. That is, when the motion time of a normal standing-up motion is T1 and the motion time of a dangerous standing-up motion is T2, the elapsed time Tth for determining whether to output a predetermined warning signal may be T2≧Tth≧T1. When the elapsed time Tth is T1T, this embodiment can output a warning quickly. When the elapsed time Tth is T2, this embodiment can reduce the probability of misrecognizing a normal standing-up motion as a dangerous standing-up motion. When the elapsed time Tth is T2>Tth>T1, this embodiment can set a good balance between the promptness of warning output and the reduction of misrecognition. In addition, in this embodiment, the motion time T1 of a normal standing-up motion is 0.6 seconds, and the motion time T2 of a dangerous standing-up motion is 3 seconds, but in this embodiment, other values may be set as appropriate based on the age, physique, health condition, mental and physical disability state, time, and situations such as after exercise of the monitored subject T.

The above operations may be performed during the normal standing-up motion of the monitored subject T. For example, during the normal standing-up motion of the monitored subject T, it can be assumed that the above operations (1) to (3) are performed within 0.6 seconds, and then the above operations (4) to (8) are performed within 1.2 seconds.

Therefore, the controller 15 may output a warning signal indicating that the monitored object T is performing a standing-up motion when the monitored object T completes the above-mentioned actions (1) to (3) within one second after starting the standing-up motion. In this embodiment, the elapsed time for determining whether or not to output a predetermined warning signal may be one second. In this way, the monitoring device 10 can issue a warning to the user that the monitored object T is starting a standing-up motion a predetermined time before the monitored object T completes a normal standing-up motion (for example, about one second before). In this way, the controller 15 may output a predetermined warning signal a predetermined time before the monitored object T performs a standing-up motion. Here, in this embodiment, the elapsed time for determining whether or not to output a predetermined warning signal is not limited to one second as described above. In other words, when the normal standing-up motion motion time T1 and the dangerous standing-up motion motion time T2 are taken as the motion time, the elapsed time Tth for determining whether or not to output a predetermined warning signal may be T2≧Tth≧T1. When the elapsed time Tth is taken as T1, this embodiment can quickly output a warning. When the elapsed time Tth is T2, this embodiment can reduce the probability of misrecognizing a normal standing-up motion as a dangerous standing-up motion. When the elapsed time Tth is T2>Tth>T1, this embodiment can set a good balance between the promptness of the warning output and the reduction of the false recognition. In addition, in this embodiment, the motion time T1 of the normal standing-up motion is 0.6 seconds, and the motion time T2 of the dangerous standing-up motion is 3 seconds, but in this embodiment, other values may be set as appropriate for T1 and T2 based on the age, physique, health condition, mental and physical disability state, time, and the situation, such as after exercise, of the monitored subject T.

In either case, the monitoring device 10 may issue a warning to the user that the monitored subject T has started to stand up before the monitored subject T has completed the standing up motion. In this manner, the controller 15 may output a predetermined warning signal before the monitored subject T completes the standing up motion. As described above, in one embodiment, the controller 15 may output a predetermined warning signal before the monitored subject T stands up based on the feature points extracted by the extraction unit 11 and the machine learning data 132.

According to the monitoring system 1 of one embodiment, when the monitored subject T is about to perform a dangerous standing up motion, for example, a warning can be issued to the user before the dangerous standing up motion is performed. Therefore, the monitoring system 1 of one embodiment can contribute to the safety of the monitored subject.

(When standing up from a chair with armrests)
8 to 10 are diagrams for explaining an example of extracting feature points of the motion of the monitored target T standing up from a chair with armrests. FIG. 8 to FIG. 10 show the movement of feature points when the monitored target T, sitting on a chair C2 with armrests, stands up. FIG. 8 to FIG. 10 show a schematic example of feature points extracted from an image of the monitored target T captured from the left side. Here, the chair C2 may be a chair with an armrest or armrests, as shown in FIG. 8 to FIG. 10. Below, the points that are different from the above-mentioned "when standing up from a chair without armrests" will be mainly explained.

As in FIG. 4, FIG. 8 shows a schematic example of feature points extracted from an image of the monitored subject T sitting in chair C2 captured from the left side.

As described above, partial movements included in an example of a normal standing-up movement performed by the monitored subject T from the state shown in FIG. 8 to the state shown in FIG. 9 may be, for example, the following movements.
(1) Movement of pulling back the leg (movement of feature point P9)
(2) Movement of leaning forward (movement of feature points P1, P2, or P3)
(3) Movement of putting the hand forward (movement of feature points P7 and/or P6 (distance d5 shown in FIG. 9))

Here, in the normal standing-up motion of the monitored subject T, the distance d5 in (3) above tends to be about 10 cm, as in the case shown in Figure 5.

On the other hand, in the dangerous standing-up movement of the monitored subject T, the above-mentioned "(3) movement of putting the hand forward" tends to be replaced by "(3') movement of pulling the arm backward (movement of feature points P7 and P6)" as shown in FIG. 10. "(3') movement of pulling the arm backward" may include movement of feature point P7 (distance d6 shown in FIG. 10) and movement of feature point P6 (distance d7 shown in FIG. 10), as shown in FIG. 10. Here, in the dangerous standing-up movement of the monitored subject T, the above-mentioned distance d6 tends to be about 10 cm, and distance d7 tends to be about 10 cm.

In other words, when extracting the movement of the characteristic points of the monitored subject T, if the characteristic points are as shown in FIG. 9 when the monitored subject T completes the above actions (1) to (3), the monitored subject T will tend to make a normal standing-up movement. On the other hand, when extracting the movement of the characteristic points of the monitored subject T, if the characteristic points are as shown in FIG. 10 when the monitored subject T completes the above actions (1) to (3), the monitored subject T will tend to make a dangerous standing-up movement.

(When standing up from the chair you were using with your desk)
11 and 12 are diagrams for explaining an example of extracting feature points of the motion of the monitored object T standing up from a chair used together with the desk. FIG. 11 and FIG. 12 show the movement of feature points when the monitored object T, sitting on a chair C1 used together with the desk D1, stands up. FIG. 11 and FIG. 12 show a schematic example of feature points extracted from an image of the monitored object T captured from the left side. Here, the chair C1 may be a chair without armrests or armrests, as shown in FIG. 4 to FIG. 7. Below, differences from the above description will be mainly explained.

Figure 11, similar to Figure 5, shows a schematic example of feature points extracted from an image captured from the left side of a monitored subject T who is about to stand up from a seated position in a chair C1.

As described above, partial movements included in an example of a normal standing-up movement performed by the monitored subject T up to the state shown in FIG. 5 may be, for example, the following movements.
(1) Movement of pulling back the leg (movement of feature point P9)
(2) Movement of leaning forward (movement of feature point P1, P2, or P3)
(3) Movement of putting the hand forward (movement of feature points P7 and/or P6)

In contrast, among the partial movements included in the example of normal standing-up movement performed by the monitored subject T up to the state shown in FIG. 11, the above (1) and (2) may be considered to be included in the same manner as the state shown in FIG. 5. On the other hand, the above "(3) movement of putting the hand forward" tends to be replaced by "(3") movement of pushing the desk with the hand and/or pulling the chair (movement of feature points P7 and P6)" in the state shown in FIG. 11. "(3") movement of pushing the desk with the hand and/or pulling the chair" may include movement of feature point P7 and/or feature point P6 (angle a6 shown in FIG. 12) as shown in FIG. 12. The angle a6 in the above (3") movement tends to change from about 70° to about 170° in both normal standing-up movement and dangerous standing-up movement of the monitored subject T.

As described above, when the monitored subject T stands up, the movement of the extracted feature points may differ depending on the type of chair and/or the presence or absence of a desk. Therefore, by performing machine learning based on the feature points extracted in each situation, the monitoring system 1 can issue a predetermined warning before the monitored subject T stands up depending on each situation.

In addition, when the monitored subject T stands up, the movements of the extracted feature points may differ depending on the type of chair and/or the presence or absence of a desk, but there may also be movements of common feature points that are extracted. Therefore, by performing machine learning based on feature points that are commonly extracted in each of the different situations, the monitoring system 1 can issue a predetermined warning before the monitored subject T stands up, regardless of each of the different situations.

(When a monitored subject with hemiplegia stands up)
13 and 14 are diagrams for explaining an example of extracting feature points of the motion of a monitored target T with hemiplegia standing up from a chair. FIG. 13 and FIG. 14 show the movement of feature points when a monitored target T sitting on a chair C1 stands up. FIG. 13 and FIG. 14 show an example of feature points extracted from an image of a monitored target T captured from the front. Here, the chair C1 may be a chair with no armrest or armrest, as shown in FIG. 4 to FIG. 7. Hereinafter, differences from the above description will be mainly explained. Here, FIG. 13 and FIG. 14 are diagrams of a monitored target T with left hemiplegia. FIG. 14 shows an example of paralysis on the side where the shoulder of the monitored target T is lowered. For example, as a sequela of a stroke, there is a symptom of hemiplegia (hemiplegia). Hemiplegia can be a case of paralysis of the upper and lower halves of the left body, or a case of paralysis of the upper half of the left body. The monitored subject T with hemiplegia in Fig. 14 may be considered as a case of paralysis of the upper half of the left body. Since the paralyzed left half of the body cannot exert any strength, the upper half of the left side of the body tends to drop down. In this case, the feet are in contact with the ground, so the lower half of the left side of the body does not drop down. Therefore, in this case, the posture is the same as when the lower half of the left side of the body is not paralyzed.

FIG. 13, similar to FIG. 4, shows an example of feature points extracted from an image of the monitored subject T sitting in chair C1. FIG. 4 shows an example of feature points extracted from an image of the monitored subject T sitting in chair C1 taken from the left side. In contrast, FIG. 13 shows an example of feature points extracted from an image of the monitored subject T sitting in chair C1 taken from the front.

As described above, partial movements included in an example of a normal standing-up movement performed by the monitored subject T up to the state shown in FIG. 5 may be, for example, the following movements.
(1) Movement of pulling back the leg (movement of feature point P9)
(2) Movement of leaning forward (movement of feature points P1, P2, or P3)
(3) Movement of putting the hand forward (movement of feature points P7 and/or P6)

Here, as shown in FIG. 14, among the partial movements included in an example of a standing-up movement performed by a monitored subject T with hemiplegia, the above-mentioned "(2) movement of leaning forward" tends to be replaced by a movement different from the state shown in FIG. 5. Therefore, in this case, the above-mentioned "(2) movement of leaning forward" may be replaced by "(2') movement of leaning forward" including an asymmetric movement. In this case, in the "(2') movement of leaning forward", the movement of the above-mentioned characteristic points P1, P2, or P3 may be the same as in the case of "(2) movement of leaning forward" described in FIG. 5. On the other hand, the "(2') movement of leaning forward" may include, for example, the movement of characteristic points P5L and/or P5R (angle a7 shown in FIG. 14) as an asymmetric movement.

As described above, when the monitored subject T stands up, the movement of the extracted feature points may differ depending on the physical characteristics and/or disabilities of the monitored subject T. Therefore, by performing machine learning based on the feature points extracted from each monitored subject T, the monitoring system 1 can issue a specified warning before each monitored subject T stands up.

In addition, when the monitored subject T stands up, the extracted feature points may vary depending on the physical characteristics and/or disabilities of the monitored subject T, but there may also be common feature points. Therefore, by performing machine learning based on feature points commonly extracted for each of the different monitored subjects T, the monitoring system 1 can issue a predetermined warning before the monitored subject T stands up, without depending on each of the different monitored subjects T.

Although the embodiments of the present disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art can easily make various modifications or corrections based on the present disclosure. Therefore, it should be noted that these modifications or corrections are included in the scope of the present disclosure. For example, the functions included in each component or each step can be rearranged so as not to cause logical inconsistencies, and multiple components or steps can be combined into one or divided. Although the embodiments of the present disclosure have been described mainly with respect to the device, the embodiments of the present disclosure can also be realized as a method including steps executed by each component of the device. The embodiments of the present disclosure can also be realized as a method, a program, or a storage medium on which a program is recorded, executed by a processor provided in the device. It should be understood that these are also included in the scope of the present disclosure.

The above-described embodiment is not limited to implementation as a monitoring system 1. For example, the above-described embodiment may be implemented as a monitoring device 10 included in the monitoring system 1. The above-described embodiment may also be implemented as a monitoring method using equipment such as the monitoring device 10. Furthermore, the above-described embodiment may also be implemented as a program executed by equipment such as the monitoring device 10 or an information processing device (e.g., a computer).

REFERENCE SIGNS LIST 1 monitoring system 10 monitoring device 11 extraction unit 13 storage unit 132 machine learning data 15 controller 17 warning unit 19 communication unit 20 imaging unit

Claims (8)

An imaging unit that images an object to be monitored;
an extraction unit that extracts a movement of a feature point of the monitored object from an image captured by the imaging unit;
a controller that outputs a predetermined warning signal before the monitored subject completes a standing-up motion, based on machine learning data obtained by machine learning of the movement of the feature points when the monitored subject stands up and the feature points extracted by the extraction unit;
a warning unit that issues a predetermined warning based on the predetermined warning signal output from the controller;
Equipped with
The controller outputs the predetermined warning signal when the monitored object does not complete a stand-up action within a predetermined time. The monitoring system of claim 1, wherein the controller outputs the predetermined warning signal a predetermined time before the monitored subject completes a standing-up motion. The machine learning data is data obtained by machine learning of the movement of the feature points when the monitored subject makes a dangerous standing-up motion,
The monitoring system according to claim 1 or 2, wherein the controller outputs a predetermined warning signal before the monitored subject completes a dangerous standing-up motion based on the feature points extracted by the extraction unit and the machine learning data. The machine learning data is data obtained by machine learning of the movement of the feature points when a specific person as the monitored subject stands up,
The monitoring system according to claim 1 , wherein the controller outputs a predetermined warning signal before the specific person completes a standing-up motion based on the feature points extracted by the extraction unit and the machine learning data. The monitoring system according to any one of claims 1 to 4, wherein the warning unit issues the predetermined warning as information that affects at least one of the senses of hearing, vision, and touch. an extraction unit that extracts a movement of a feature point of a monitored object from an image of the monitored object;
a controller that outputs a predetermined warning signal before the monitored subject completes a standing-up motion, based on machine learning data obtained by machine learning of the movement of the feature points when the monitored subject stands up and the feature points extracted by the extraction unit;
Equipped with
The controller outputs the predetermined warning signal when the monitored object does not complete a standing-up action within a predetermined time. extracting a movement of a feature point of a monitored object from an image of the monitored object;
a step of outputting a predetermined warning signal before the monitored subject completes a standing-up motion based on machine learning data obtained by machine learning the movement of the feature points when the monitored subject stands up and the feature points extracted in the extracting step;
Including,
a monitoring method, the predetermined warning signal being outputted if the monitored object does not complete a standing-up action within a predetermined time. On the computer,
extracting a movement of a feature point of a monitored object from an image of the monitored object;
a step of outputting a predetermined warning signal before the monitored subject completes a standing-up motion based on machine learning data obtained by machine learning the movement of the feature points when the monitored subject stands up and the feature points extracted in the extracting step;
Run the command ,
a program that outputs the predetermined warning signal when the monitored object does not complete a standing-up action within a predetermined time.

Images