JP6849494B2 - Notification device, notification method and program - Google Patents

Description

The present invention relates to a notification device, a notification method and a program.

In recent years, the aging of the population has progressed, and the number of elderly households living alone is increasing rapidly. In an elderly household living alone, when a fall occurs around the bed, it is desired that an outsider recognize the fall and immediately go to rescue.

Therefore, a technique for watching a person represented by an elderly person, a person requiring long-term care, and an infant from a remote place where it is difficult to visually recognize the person has been proposed (see, for example, Patent Document 1). Patent Document 1 describes that an abnormality of a person to be watched over is detected by using a plurality of infrared light receiving elements that detect the movement of the human body. According to this technology, a person can be watched from a remote place.

Japanese Unexamined Patent Publication No. 2001-76272

During the daytime, there are many cases where the person being watched over and the observer share a living space, or the person being watched over is cared for at a specialized facility. For this reason, there is a demand for recognizing an abnormality of a person to be watched only during the nighttime when the person is normally sleeping. Since the technique described in Patent Document 1 detects an abnormality based only on the movement of the human body, there is a possibility that this activity may be erroneously detected as an abnormality even when the person being watched over is awake and active.

Therefore, it is conceivable to limit the time zone in which the technique described in Patent Document 1 is applied to nighttime. However, the life rhythm of the subject to be watched differs depending on the subject, and the life rhythm of a specific subject is not always constant. Therefore, even if the technique described in Patent Document 1 is applied only at night. , It is difficult to accurately recognize the abnormality of the person being watched over.

The present invention has been made under the above circumstances, and an object of the present invention is to accurately notify an abnormality that occurs during the time when a person to be watched sleeps.

In order to achieve the above object, the notification device of the present invention is a notification device connected to a photographing means for repeatedly capturing an image of an indoor space, and is an acquisition means for acquiring image data related to an image captured by the photographing means. , When the length of time that the person stays outside the sleep-on area in the indoor space where the person falls asleep exceeds the first threshold after the condition for the person to be photographed by the photographing means is satisfied. A notification means for notifying that an abnormality has occurred in a person is provided , the image includes an infrared image, the condition is satisfied when the body temperature of the person falls below the second threshold value, and the notification means is an indoor space. The second threshold value is changed according to the operation mode of the air conditioning means for harmonizing the air .

According to the present invention, it is possible to accurately notify an abnormality that occurs during the time when the person to be watched sleeps.

Diagram showing the outline of the notification system The first figure for explaining an indoor space The second figure for explaining the indoor space Diagram showing the hardware configuration of the notification device The figure which shows the functional configuration of the notification device Diagram showing an example of a coordinate table The figure which shows an example of the body temperature conversion table The figure which shows an example of the body temperature transition table Diagram to explain the transition of body temperature in a day Diagram showing an example of a sleep state table Diagram for explaining the estimation of the sleep-onset area Diagram showing an example of the sleep onset area table Diagram showing an example of an abnormal behavior table First flow chart showing notification processing Second flow diagram showing notification processing Diagram showing a notification system configured using a server installed outside the building

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

Embodiment 1.
FIG. 1 shows an outline of the notification system 1000 according to the present embodiment. The notification system 1000 is a system that notifies the remote observer U2 that an abnormality has occurred in the person U1 in the indoor space A1 in the building BD.

A building BD is a house or a house. However, the building BD is not limited to this, and may be an accommodation facility for the person U1 to sleep. Further, the indoor space A1 is usually a bedroom. However, the indoor space A1 is not limited to this, and may be any space having a sleep-onset area A2 in which the person U1 falls asleep. The person U1 according to the present embodiment is an elderly person in a single household. The abnormalities that occur in the person U1 include a fall of the person U1, a wandering outside the indoor space A1, and a wandering inside the indoor space A1. The observer U2 is, for example, a relative or a caregiver of the person U1. Further, FIG. 1 shows a state in which the observer U2 is outside the building BD, but the present invention is not limited to this, and the observer U2 may be inside the building BD.

The notification system 1000 includes an air conditioner 20 including a photographing unit 21, a notification device 100 for notifying an abnormality of the person U1, and a terminal 30 carried by the observer U2. The air conditioner 20 and the notification device 100 are connected so as to be able to communicate with each other. Further, the notification device 100 and the terminal 30 are connected via a network NW so as to be able to communicate with each other, and are well known represented by Wi-Fi (registered trademark), Wi-SUN (registered trademark) and a wired LAN. Communicate according to the communication standard.

The air conditioner 20 and the notification device 100 are connected via, for example, a wired or wireless LAN (Local Area Network) provided in the building BD, and communicate according to ECHONET Lite (registered trademark). Communicate with the protocol. Further, the network NW may be a wide area communication network represented by the Internet. The communication mode of the air conditioner 20, the notification device 100, and the terminal 30 is not limited to that shown in FIG. 1, and the air conditioning device 20, the notification device 100, and the terminal 30 may all be connected to the network NW.

The air conditioner 20 includes an outdoor unit, an indoor unit, and a dedicated remote controller (not shown) in addition to the photographing unit 21, and has any operation mode of cooling operation, heating operation, or dehumidifying operation in the space where the indoor unit is installed. Operates in. In the present embodiment, the indoor unit of the air conditioner 20 is installed so as to be hung on the wall of the indoor space A1.

The person U1 operates a remote controller to input commands for starting or stopping cooling, heating, and dehumidification, and specifies a set temperature as a target value of the temperature of the indoor space A1 and an air volume. The communication interface of the air conditioner 20 may be provided in at least one of the indoor unit, the outdoor unit, and the remote controller. Further, the air conditioner 20 may be connected to the notification device 100 via an external communication adapter. This communication adapter is electrically connected to the indoor unit of the air conditioner 20 by an interface conforming to a standardized serial communication standard.

The photographing unit 21 is a thermal image sensor or an infrared sensor, and captures a thermal image by measuring the temperature within the detection range. The photographing unit 21 transmits the ID of the air conditioner 20 and the image data related to the thermal image to the notification device 100 in response to the request from the notification device 100. The image data relating to the thermal image may be data showing the thermal image or may be data obtained by processing the thermal image.

Here, the relationship between the indoor space A1 and the photographing unit 21 will be described with reference to FIGS. 2 and 3. FIG. 2 is a top view of an example of the indoor space A1, and FIG. 3 is a bird's-eye view of an example of the indoor space A1. The indoor space A1 is a space partitioned from the outside by a wall, a door, or a window, and the person U1 in the indoor space A1 is always photographed by the photographing unit 21. In other words, when the person U1 moves to the external space A3 outside the indoor space A1, the person U1 is not photographed by the photographing unit 21. The external space A3 is, for example, a corridor or a common passage.

Returning to FIG. 1, the terminal 30 is a mobile device typified by a smartphone and a tablet terminal. The terminal 30 includes an input device typified by a push button, a touch panel, a touch pad and a microphone, an output device typified by an organic EL (Electro-Luminescence) display, a liquid crystal display and a speaker, and a communication interface. Will be done.

The terminal 30 receives the notification data indicating the notification content transmitted from the notification device 100, displays the notification content, and transmits the notification content to the observer U2 by an acoustic signal or an audio signal. In this way, the terminal 30 is used as a user interface for the observer U2.

The notification device 100 is a controller that is installed at an appropriate place in the building BD and manages a plurality of devices installed in the building BD in an integrated manner. The notification device 100 may be an information device typified by a PC (Personal Computer), a smartphone, or a tablet terminal, or may be a dedicated controller for managing a plurality of devices. The notification device 100 notifies the observer U2 of an abnormality that has occurred in the person U1 by generating notification data based on the image data obtained from the photographing unit 21 of the air conditioner 20 and transmitting the notification data to the terminal 30.

Further, the notification device 100 controls the air conditioning device 20 by giving a command to the air conditioning device 20 to start or stop cooling, heating or dehumidification, or by designating a set temperature and an air volume. Further, the notification device 100 acquires the operation information indicating the current operating state of the air conditioner 20 from the air conditioner 20 by requesting the air conditioner 20. The operation information includes, for example, an ID of the air conditioner 20, data indicating an operation mode and whether or not the air conditioner is operating, and data indicating a designated set temperature and air volume.

FIG. 4 shows the hardware configuration of the notification device 100. As shown in FIG. 4, the notification device 100 is a computer having a processor 11, a main storage unit 12, a ROM (Read-Only Memory) 13, an auxiliary storage unit 14, an input unit 15, an output unit 16, and a communication unit 17. It is configured. The main storage unit 12, the ROM 13, the auxiliary storage unit 14, the input unit 15, the output unit 16, and the communication unit 17 are all connected to the processor 11 via the internal bus 18.

The processor 11 includes a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The processor 11 comprehensively controls the components of the notification device 100. Then, the processor 11 executes the process described later by executing the program 19 stored in the auxiliary storage unit 14.

The main storage unit 12 includes a RAM (Random Access Memory). The program 19 is loaded into the main storage unit 12 from the auxiliary storage unit 14. Then, the main storage unit 12 is used as a work area of the processor 11.

The ROM 13 stores a plurality of firmwares and data used when executing these firmwares. The auxiliary storage unit 14 includes an EEPROM (Electrically Erasable Programmable Read-Only Memory) and a readable and writable non-volatile semiconductor memory represented by a flash memory, or a magnetic memory represented by an HDD (Hard Disk Drive). To. In addition to the program 19, the auxiliary storage unit 14 stores various data used in the processing of the processor 11. The auxiliary storage unit 14 supplies the data used by the processor 11 to the processor 11 according to the instruction of the processor 11, and stores the data supplied from the processor 11.

The input unit 15 includes an input device. The input unit 15 acquires the information input by the administrator of the notification device 100, and notifies the processor 11 of the acquired information. The output unit 16 is configured to include an output device, and presents various information to the administrator according to the instructions of the processor 11.

The communication unit 17 includes a network card for communicating with the air conditioner 20 and a network card for communicating with the terminal 30. The communication unit 17 receives a signal from the air conditioner 20 and the terminal 30, and outputs the data contained in the signal to the processor 11. Further, the communication unit 17 transmits a signal indicating the data output from the processor 11 to the air conditioner 20 and the terminal 30.

The notification device 100 exhibits various functions in cooperation with the above-mentioned hardware configurations. FIG. 5 shows the functional configuration of the notification device 100. As shown in FIG. 5, as its functions, a storage unit 110 that stores various data, an acquisition unit 120 that acquires image data, and a person who determines the coordinates of the person U1 in the captured image. The coordinate determination unit 130, the body temperature calculation unit 140 for calculating the body temperature of the person U1, the sleep determination unit 150 for determining whether or not the person U1 is in a sleep state, and the sleep onset area estimation unit 160 for estimating the sleep onset area A2. It has an abnormal behavior determining unit 170 that determines the presence or absence of abnormal behavior of the person U1 and a notification unit 180 that notifies the observer U2 when an abnormal behavior is detected.

The storage unit 110 is mainly realized by the auxiliary storage unit 14. As shown in FIG. 5, the storage unit 110 records a coordinate table 111 showing the coordinates where the person U1 is located, a body temperature conversion table 112 for calculating the body temperature of the person U1, and a transition of the body temperature of the person U1. The body temperature transition table 113 for the purpose, the sleep state table 114 indicating whether or not the person U1 is in the sleeping state, the sleeping area table 115 indicating the sleeping area A2, and the result of determining the presence or absence of abnormal behavior of the person U1 are shown. It has an abnormal behavior table 116 and.

The acquisition unit 120 acquires image data from the photographing unit 21. This image data is data in which the temperature information indicating the temperature of the subject reflected in the image is associated with the coordinate information indicating the coordinates of the image. Specifically, the acquisition unit 120 acquires image data transmitted from the air conditioner 20 at a predetermined cycle, and outputs the acquired image data to the human coordinate determination unit 130 in association with the current time. This cycle is, for example, 1 minute or 5 minutes. The acquisition unit 120 may acquire time data indicating the current time from the outside, if necessary.

The human coordinate determination unit 130 determines the coordinates where the person U1 is present based on the temperature information and the coordinate information included in the image data. Various methods for detecting the coordinates can be considered, but in the present embodiment, the coordinates of the point having the highest temperature among the image data are detected as the coordinates of a person. Generally, since the temperature near the human face is the highest, the detected coordinates correspond to the position of the human face. Since the coordinates in the image correspond one-to-one with the coordinates on the floor surface in the indoor space A1, the surface on the sleep area A2, and the wall surface, the coordinates in the image are substantially the coordinates in the indoor space A1. be equivalent to.

Further, when the indoor space A1 has a heat source other than the person U1, the human coordinate determination unit 130 erroneously recognizes this heat source as the position of the human face. In order to avoid such erroneous recognition, when the detected temperature is equal to or higher than a preset predetermined value, it is not necessary to determine the coordinates having the highest temperature as the coordinates of a person. This default value is, for example, 40 ° C.

Further, when there is no person in the indoor space A1, the human coordinate determination unit 130 erroneously recognizes the place where the temperature of the indoor space A1 is the highest as the position of the human face. In order to avoid such erroneous recognition, the human coordinate determination unit 130 does not have to determine the coordinate having the highest temperature as the human coordinate when the detected temperature is equal to or less than a predetermined predetermined value. This specified value is, for example, 25 ° C.

Then, the human coordinate determination unit 130 adds the determined coordinates to the coordinate table 111 stored in the storage unit 110 in association with the temperature of the coordinates and the time when the image data is acquired. When the human coordinate determination unit 130 determines that there is no person in the indoor space A1, the coordinate information (0,0) is added to the coordinate table 111.

FIG. 6 shows an example of the coordinate table 111. In the example shown in FIG. 6, at 23:30, the person U1 is located at the coordinate point (200,100), and the temperature at this coordinate point is shown to be 34 ° C.

The body temperature calculation unit 140 converts the temperature of the face of the person U1 included in the coordinate table 111 into the body temperature of the person U1. Specifically, the body temperature calculation unit 140 calculates the body temperature of the person U1 by converting the output value of the photographing unit 21 with reference to the body temperature conversion table 112 stored in the storage unit 110. FIG. 7 shows an example of the body temperature conversion table 112. In the example shown in FIG. 7, it is shown that the body temperature is estimated to be 36.5 ° C. when the temperature detected by the photographing unit 21 is 34 ° C.

Then, the body temperature calculation unit 140 adds the time extracted from the coordinate table 111 and the body temperature converted from the temperature extracted from the coordinate table 111 to the body temperature transition table 113 stored in the storage unit 110 in association with each other. FIG. 8 shows an example of the body temperature transition table 113. In the example shown in FIG. 8, it is shown that the estimated body temperature was 36.3 ° C. at 1:00.

The sleep determination unit 150 determines whether or not the person U1 is in a sleep state based on the body temperature transition table 113. Here, the determination of sleep according to the present embodiment will be described with reference to FIG. FIG. 9 shows when the patient does not sleep at all, when he / she sleeps at night and is not active during the day, and when he / she is lying in bed continuously, and when he / she sleeps at night, he / she is active as usual during the day. And, for each, the transition of the average body temperature of a person is shown. As can be seen from FIG. 9, the body temperature decreases when sleeping at night regardless of the presence or absence of activity during the day. Therefore, a decrease in body temperature is a condition for falling asleep. The sleep determination unit 150 determines whether or not the person U1 is in a sleeping state by determining whether or not the body temperature has decreased. Then, the sleep determination unit 150 records the determination result in the sleep state table 114. As shown in FIG. 10, the sleep state table 114 according to the present embodiment is a sleep flag indicating whether or not the person U1 is currently sleeping. If the value of the sleep flag is 1, it indicates that the person U1 is sleeping, and if the value is zero, it indicates that the person U1 is not sleeping.

The sleep onset area estimation unit 160 estimates the sleep onset area A2 by acquiring the coordinate data of the corresponding coordinate table 111 when the value of the sleep flag is 1 and learning from the coordinate data collected in a certain period of time. .. The fixed period is, for example, one month. If the person U1 wakes up during bedtime and goes to the bathroom during this fixed period, it is considered that the estimated range of the sleep-onset area A2 becomes wider. Taking such variations into consideration, the estimated range of the sleep-onset region A2 is a coordinate range of the distribution obtained from the flow line of the head of the person U1 that falls within 2σ with the variance as σ, as shown in FIG. And. In FIG. 11, the sleep onset area A21 is shown as a range in which the actual sleep onset area A2 is estimated. If there are coordinates indicating the person U1 in the sleep-onset area A21, the person U1 is in the sleep-onset area A2.

In FIG. 11, the thin dotted line indicates the flow line of the head of the person U1. The coordinate range within 2σ from the distribution of the coordinate points constituting the flow line on the horizontal axis (x axis) and the vertical axis (y axis) is defined as the sleep onset region A21.

Then, the sleep-onset area estimation unit 160 stores the data indicating the estimated sleep-onset area A21 as the sleep-onset area table 115 of the storage unit 110. FIG. 12 shows an example of the sleep onset area table 115. In the example shown in FIG. 12, the sleep-onset area A21 is a portion surrounded by the coordinates (180,80), the coordinates (180,150), the coordinates (250,80), and the coordinates (250,150).

The abnormal behavior determination unit 170 refers to the coordinate table 111 and the sleep onset area table 115 to determine the presence or absence of abnormal behavior represented by falling and wandering of the person U1 during bedtime. Then, the abnormal behavior determination unit 170 stores the result of determining the abnormal behavior in the abnormal behavior table 116.

FIG. 13 shows an example of the abnormal behavior table 116. The abnormal behavior table 116 is table data in which the type of abnormal behavior is associated with the time when the abnormal behavior is detected. In the example shown in FIG. 13, it is shown that wandering was detected as an abnormal behavior at 4:00.

When the abnormal behavior determination unit 170 detects the abnormal behavior, the notification unit 180 notifies the observer U2 of the abnormality by transmitting a message indicating that the abnormality has occurred in the person U1 to the terminal 30.

Subsequently, the notification process executed by the notification device 100 will be described with reference to FIGS. 14 to 15. The notification process is executed at predetermined time intervals when the automatic execution of the program 19 is set to on in advance by the administrator of the notification device 100. This time interval is, for example, 1 minute or 5 minutes.

In the notification process, first, the acquisition unit 120 acquires the image data, and the human coordinate determination unit 130 determines the coordinates of the person U1 (step S1). Then, the body temperature calculation unit calculates the body temperature of the person U1 (step S2).

Next, the sleep determination unit 150 determines whether or not the current time is included in the predetermined bedtime (step S3). This bedtime is, for example, from 22:00 to 7:00. When it is determined that the current time is not included in the bedtime (step S3; No), the sleep determination unit 150 determines that the person U1 is not in the sleeping state and sets the sleep flag to zero (step S4). After that, the notification process proceeds to step S21 shown in FIG.

On the other hand, when it is determined that the current time is included in the bedtime (step S3; Yes), the sleep determination unit 150 determines whether or not the body temperature has risen during the bedtime with reference to the body temperature transition table 113. (Step S5). In addition, in order to exclude an instantaneous increase in body temperature, when the average body temperature for a certain length of time rises by a predetermined value or more, the sleep determination unit 150 determines that the body temperature has risen. .. The time of a certain length is, for example, 30 minutes, and the specified value is, for example, 0.2 ° C.

When it is determined that the body temperature has risen (step S5; Yes), the sleep determination unit 150 executes step S4. On the other hand, when it is determined that the body temperature has not risen (step S5; No), the sleep determination unit 150 determines whether or not the air conditioner 20 is currently in the heating operation (step S6).

When it is determined that the air conditioner 20 is in the heating operation (step S6; Yes), the sleep determination unit 150 adds 0.5 to the predetermined body temperature threshold value to obtain a new body temperature threshold value (step S7). .. This body temperature threshold value is a body temperature threshold value for determining the sleep state, and is predetermined as, for example, 36.5 ° C. from the graph shown in FIG. After that, the process executed by the sleep determination unit 150 shifts to step S10.

On the other hand, when it is determined that the air conditioner 20 is not in the heating operation (step S6; No), the sleep determination unit 150 determines whether or not the air conditioner 20 is currently in the cooling operation (step S8). When it is determined that the air conditioner 20 is not in the cooling operation (step S8; No), the process executed by the sleep determination unit 150 shifts to step S10.

On the other hand, when it is determined that the air conditioner 20 is in the cooling operation (step S8; Yes), the sleep determination unit 150 subtracts 0.5 from the body temperature threshold value to obtain a new body temperature threshold value (step S9).

Next, the sleep determination unit 150 determines whether or not the sleep condition, which is the condition for the person U1 to be in the sleep state, is satisfied (step S10). That is, the sleep determination unit 150 determines whether or not the current body temperature is lower than the body temperature threshold value.

When it is determined that the sleep condition is satisfied (step S10; Yes), the sleep determination unit 150 shifts the process to step S12. On the other hand, when it is determined that the sleep condition is not satisfied (step S10; No), the sleep determination unit 150 determines whether the length of time that the person U1 stays outside the sleep-onset area A2 exceeds a predetermined threshold value. It is determined whether or not (step S11). Specifically, the sleep determination unit 150 determines whether or not the length of time that the coordinate point of the person U1 continues outside the sleep onset area A21 exceeds the threshold value.

When the determination in step S11 is denied (step S11; No), the sleep determination unit 150 shifts the process to step S4. On the other hand, when the determination in step S11 is affirmed (step S11; Yes), the sleep determination unit 150 determines that the person U1 is in the sleeping state and sets the sleep flag to 1 (step S12).

When step S4 or step S12 is executed, as shown in FIG. 15, the sleep determination unit 150 determines whether or not the person U1 is sleeping with reference to the sleep state table 114 (step S21). ). That is, the sleep determination unit 150 determines whether or not the sleep flag is 1. When it is determined that the person U1 is not sleeping (step S21; No), the notification process ends.

On the other hand, when it is determined that the person U1 is sleeping (step S21; Yes), the person coordinate determination unit 130 determines whether or not the person U1 is in the indoor space A1 with reference to the coordinate table 111. (Step S22). When it is determined that the person U1 is not in the indoor space A1 (step S22; No), the abnormal behavior determination unit 170 refers to the coordinate table 111 and is the first after the person U1 is no longer in the indoor space A1. It is determined whether or not the time has passed (step S23). That is, the abnormal behavior determination unit 170 determines whether or not the length of time elapsed since the person U1 goes out of the indoor space A1 exceeds the threshold value. This threshold is, for example, 30 minutes.

If it is determined that the first time has not elapsed (step S23; No), the processes after step S22 are repeated. On the other hand, when it is determined that the first time has passed (step S23; Yes), the abnormal behavior determination unit 170 detects that the person U1 has wandered outside the indoor space A1 as an abnormality that has occurred in the person U1. The notification unit 180 notifies the observer U2 that it has detected the wandering to the outside (step S24). After that, the notification process ends.

When it is determined in step S22 that the person U1 is in the indoor space A1 (step S22; Yes), the abnormal behavior determination unit 170 refers to the coordinate table 111 and the sleep onset area table 115, and the person U1 falls asleep. It is determined whether or not the person is outside the area A2 (step S26). When it is determined that the person U1 is not outside the sleep-onset area A2 (step S26; No), that is, when the person U1 continues to sleep in the sleep-onset area A2, the notification process ends.

On the other hand, when it is determined that the person U1 is outside the sleep-onset area A2 (step S26; Yes), the abnormal behavior determination unit 170 refers to the coordinate table 111 and the sleep-onset area table 115, and the person U1 is in the sleep-onset area A2. It is determined whether or not the second time has passed since going out (step S27). That is, the abnormal behavior determination unit 170 determines whether or not the length of time elapsed since the person U1 went out of the sleep-onset area A2 exceeds the threshold value. This threshold is, for example, 1 minute.

If it is determined that the second time has not elapsed (step S27; No), the processes after step S22 are repeated. On the other hand, when it is determined that the second time has elapsed (step S27; Yes), the abnormal behavior determination unit 170 refers to the coordinate table 111 and determines whether or not the person U1 is moving (step). S28).

When it is determined that the person U1 is not moving (step S28; No), the abnormal behavior determination unit 170 determines that the person U1 has detected that the person U1 has fallen in the indoor space A1 as an abnormality that has occurred in the person U1. The notification unit 180 notifies the observer U2 that the fall has been detected (step S29). After that, the notification process ends.

On the other hand, when it is determined that the person U1 has not moved (step S28; Yes), the abnormal behavior determination unit 170 refers to the coordinate table 111 and the sleep-onset area table 115, and the person U1 goes out of the sleep-onset area A2. It is determined whether or not the third time has elapsed since then (step S30). That is, the abnormal behavior determination unit 170 determines whether or not the length of time elapsed since the person U1 went out of the sleep-onset area A2 exceeds the threshold value. This threshold is, for example, 3 minutes.

If it is determined that the third time has not elapsed (step S30; No), the processes after step S22 are repeated. On the other hand, when it is determined that the third time has elapsed (step S30; Yes), the abnormal behavior determination unit 170 detects that the person U1 is wandering in the indoor space A1 as an abnormality that has occurred in the person U1. The notification unit 180 notifies the observer U2 that the wandering in the room has been detected (step S31). After that, the notification process ends.

As described above, the notification device 100 acquires image data from the photographing unit 21 mounted on the air conditioner 20, estimates the sleep-onset area A2, and determines that there is an abnormal behavior from the behavior of the person U1. In this case, the observer U2 was notified that an abnormality had occurred in the person U1. Therefore, the air-conditioning device 20 on which the photographing unit 21 is mounted can be used to monitor the person U1 who sleeps in the indoor space A2 during the bedtime. Therefore, if the air conditioner 20 is installed, the notification system 1000 can be easily configured at a low cost without newly introducing a sensor for detecting the behavior of the person U1.

Further, the notification device 100 notifies the abnormality when the length of time that the person U1 stays outside the sleep-onset area A2 exceeds the threshold value after the sleep condition is satisfied (steps S29 and S31 in FIG. 15). .. As a result, the notification device 100 performs notification after the condition that the person U1 goes to sleep is once satisfied. Therefore, it is possible to avoid erroneous notification when the person U1 is awake and active without going to bed. That is, the notification device 100 can accurately notify the abnormality that occurs during the time when the person U1 sleeps.

Further, the notification device 100 has determined that the sleep condition is satisfied when the body temperature of the person U1 becomes equal to or lower than the threshold value. As shown in FIG. 9, since the body temperature is an effective index for determining falling asleep, the notification device 100 can accurately notify the abnormality that occurred during the time zone when the person U1 sleeps.

Further, the notification device 100 changed the body temperature threshold value for determining the sleep condition according to the operation mode of the air conditioner 20. As a result, the notification device 100 can accurately notify the abnormality that has occurred in the person U1 by taking into account the body temperature of the person U1 that has moved up and down according to the operation mode of the air conditioner 20.

Further, in the notification device 100, when the length of time that the person U1 stays outside the sleep-onset area A2 exceeds the threshold value, the person U1 has fallen while the person U1 is not moving and is stationary. (Step S29 in FIG. 15), and when the person U1 is moving, the wandering of the person U1 in the room is notified (step S31 in FIG. 15). This allows the observer U2 to recognize the details of the abnormal behavior.

Further, the notification device 100 estimated the sleep-onset area A2 from the history of the position coordinates of the person U1. For example, even if the observer U2 presets the location of the bedding of the person U1, if the person U1 goes to bed in a place different from the bedding, accurate notification cannot be given. On the other hand, the notification device 100 according to the present embodiment can accurately recognize the place where the person U1 actually goes to bed and notify the abnormality with high accuracy.

Further, the notification device 100 notifies the abnormality when the length of time that the person U1 continues to stay outside the indoor space A1 exceeds the threshold value after the sleep condition is satisfied (step S24 in FIG. 15). ). As a result, the notification device 100 can notify the abnormality even when the person U1 is not photographed by the photographing unit 21.

Embodiment 2.
Subsequently, the second embodiment will be described focusing on the differences from the first embodiment described above. For the same or equivalent configuration as that of the first embodiment, the same reference numerals are used, and the description thereof will be omitted or abbreviated. The notification system 1000 according to the present embodiment is different from that according to the first embodiment in that the photographing unit 21 captures an image in the visible light region instead of the thermal image.

The image data according to the present embodiment is data including coordinate information and color information indicating a color corresponding to each coordinate without including temperature information. The person coordinate determination unit 130 according to the present embodiment identifies a person reflected in the image data by image recognition, and stores a coordinate table in which the coordinates indicating the position of the person and the time when the image data is acquired are associated with each other. It is stored in the unit 110. For example, the human coordinate determination unit 130 detects the face of the person U1 by using a general face detection technique represented by the Viola-Jones method, and sets the center point of the face as the position of the face of the person U1 together with the time information. Add to the coordinate table.

Subsequently, since the temperature information is not used in the notification process according to the present embodiment, steps S2 and S5-S9 in FIG. 14 are omitted. Further, in the present embodiment, the determination in step S11 in FIG. 14 corresponds to the determination of whether or not the sleep condition is satisfied.

As described above, according to the notification device 100 according to the present embodiment, the air conditioning device 20 provided with a camera for capturing a visible light image is used instead of the photographing unit 21 for capturing a thermal image. The same effect as that of the first embodiment can be obtained.

The present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present invention.

For example, the notification device 100 may be configured by omitting the input unit 15 and the output unit 16 from the hardware configuration of the notification device 100 (see FIG. 4).

In each of the above embodiments, the case where the notification device 100 is installed in the building BD has been described, but a device having the same function as the notification device 100 may be installed outside the building BD.

FIG. 16 shows an example in which a server 100a having the same yesterday as the notification device 100 is installed outside the building BD. In the notification system 1000a in this example, a router 100b is installed in the building BD instead of the notification device 100. On the other hand, outside the building BD, a server 100a that is communicably connected to the router 100b via the network NW is installed. In this case, the router 100b and the server 100a cooperate to exert the same function as the notification device 100.

In each of the above embodiments, each function of the notification device 100 is realized by executing the program 19 stored in the auxiliary storage unit 14 by the processor 11. However, all or part of the notification device 100 functions may be realized by dedicated hardware. As the dedicated hardware, for example, a single circuit, a composite circuit, a programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof can be adopted. ..

In each of the above embodiments, the program 19 includes a CD-ROM (Compact Disc Read Only Memory), a DVD (Digital Versatile Disc), a magneto-optical disc (Magneto-Optical Disc), a USB (Universal Serial Bus) memory, and a memory card. , HDD or the like may be stored in a computer-readable recording medium and distributed. Then, by installing the distributed program 19 on a specific or general-purpose computer, the computer may function as the notification device 100 in each of the above embodiments.

Further, the program 19 may be stored in a disk device included in the server device on the Internet so that the program 19 can be downloaded from the server device to the computer.

The present invention is capable of various embodiments and modifications without departing from the broad spirit and scope. Moreover, the above-described embodiment is for explaining the present invention, and does not limit the scope of the present invention. That is, the scope of the present invention is indicated not by the embodiment but by the claims. Then, various modifications made within the scope of the claims and within the equivalent meaning of the invention are considered to be within the scope of the present invention.

1000, 1000a notification system, 100 notification device, 100a server, 100b router, 11 processor, 12 main storage, 13 ROM, 14 auxiliary storage, 15 input, 16 output, 17 communication, 18 internal bus, 19 programs , 110 memory unit, 111 coordinate table, 112 body temperature conversion table, 113 body temperature transition table, 114 sleep state table, 115 sleep area table, 116 abnormal behavior table, 120 acquisition unit, 130 person coordinate determination unit, 140 body temperature calculation unit, 150 Sleep determination unit, 160 sleep area estimation unit, 170 abnormal behavior determination unit, 180 notification unit, 20 air conditioning equipment, 21 imaging unit, 30 terminals, A1 indoor space, A2, A21 sleep area, A3 external space, BD building, NW network , U1 person, U2 observer.

Claims (10)

A notification device connected to a photographing means that repeatedly captures an image of an indoor space.
Obtaining means for obtaining image data concerning the image photographed by the photographing means,
After the condition for the person to be photographed by the photographing means to be in a sleeping state is satisfied, the length of time that the person is outside the sleep-onset area in the indoor space where the person falls asleep exceeds the first threshold value. In some cases, a notification means for notifying that an abnormality has occurred in the person, and
Equipped with a,
The image includes an infrared image and
The condition is satisfied when the body temperature of the person falls below the second threshold value.
The notification means is a notification device that changes the second threshold value according to the operation mode of the air conditioning means that harmonizes the air in the indoor space. The condition is satisfied when the person has been in the sleep zone for more than a predetermined time.
The notification device according to claim 1. In the notification means, when the length of time that the person stays outside the sleep-onset area exceeds the first threshold value after the conditions are satisfied, the person photographed by the photographing means is stationary. Notifying that the person has fallen while doing
The notification device according to claim 1 or 2. A notification device connected to a photographing means that repeatedly captures an image of an indoor space.
Acquiring means for acquiring image data related to the image captured by the photographing means, and
After the condition for the person to be photographed by the photographing means to be in a sleeping state is satisfied, the length of time that the person is outside the sleep-onset area in the indoor space where the person falls asleep exceeds the first threshold value. In some cases, a notification means for notifying that an abnormality has occurred in the person, and
With
In the notification means, when the length of time that the person stays outside the sleep-onset area exceeds the first threshold value after the conditions are satisfied, the person photographed by the photographing means moves. when it is to inform the wandering of the person, multicast known device. An estimation means for estimating the sleep area from the history of the position of the person is further provided.
When the length of time that the person stays outside the sleep-onset area estimated by the estimation means exceeds the first threshold value after the condition is satisfied, the notification means causes an abnormality in the person. Notify that it has happened,
The notification device according to any one of claims 1 to 4. The notification means notifies that an abnormality has occurred in the person when the length of time that the person stays outside the indoor space exceeds the third threshold value after the conditions are satisfied.
The notification device according to any one of claims 1 to 5. Notifying means, the person to be repeatedly photographed after filled condition to be asleep, among the length of the person is time out of sleep onset region where the person is falling asleep in the indoor space exceeds a first threshold value In this case, it is a notification method for notifying that an abnormality has occurred in the person.
The above condition is satisfied when the body temperature of the person reflected in the infrared image repeatedly photographed becomes equal to or less than the second threshold value.
The notification means is a notification method that changes the second threshold value according to the operation mode of the air conditioning means that harmonizes the air in the indoor space. When the notification means exceeds the threshold value for the length of time that the person stays outside the sleep-on area in the indoor space where the person falls asleep after the condition for the person to be repeatedly photographed to fall asleep is satisfied. , A notification method for notifying the wandering of the person as an abnormality that has occurred in the person when the person being photographed is moving. On the computer
Acquire image data showing images of the indoor space that are repeatedly taken,
When the length of time that the person stays outside the sleep-onset area in the room space where the person falls asleep exceeds the first threshold value after the condition that the person in the indoor space falls asleep is satisfied. , Notifies that an abnormality has occurred in the person,
Let me do that
The image includes an infrared image and
The condition is satisfied when the body temperature of the person falls below the second threshold value.
The second threshold value is a program that is changed according to the operation mode of the air conditioning means that harmonizes the air in the indoor space. On the computer
Acquire image data showing images of the indoor space that are repeatedly taken,
After the condition that the person in the indoor space is in a sleeping state is satisfied, when the length of time that the person is outside the sleep-onset area in the indoor space where the person falls asleep exceeds the threshold value, the photograph is taken. When the person is moving, the wandering of the person is notified as an abnormality that has occurred in the person.
A program to do things.

Images