JP2016157170A - Abnormal condition notification system, abnormal condition notification program, abnormal condition notification method, and abnormal condition notification equipment - Google Patents

Abstract

CONSTITUTION: An abnormal state notification system 100 includes plural distance image sensors 12 disposed on the ceiling of a nursing care room. Information representing a shape of a human being is calculated from pieces of sensor information sent from the distance image sensors 12, and a feature quantity representing the condition of the human being is extracted from the information. The feature quantity is inputted to a discrimination model which is created using machine learning and used to discriminate an abnormal condition. A sensor information acquisition time is associated with a result of discrimination performed using the discrimination model, and the result of discrimination is stored in a buffer of a memory (84). Based on results of discrimination performed over a predetermined time, whether the condition of the human being is the abnormal condition is recognized. If the human being is recognized as being in the abnormal condition, a nursing care person is notified of a result of recognition.EFFECT: Various abnormal conditions can be recognized by utilizing a feature quantity representing a condition of a human being. Therefore, a nursing care person or the like can be properly notified of the fact that the condition of a human being is an abnormal condition.SELECTED DRAWING: Figure 2

Description

  TECHNICAL FIELD The present invention relates to an abnormal state notification system, an abnormal state notification program, an abnormal state notification method, and an abnormal state notification device, and in particular, for example, an abnormal state notification system, an abnormal state notification program for notifying that a human state is an abnormal state, The present invention relates to an abnormal state notification method and an abnormal state notification device.

  An example of background art is disclosed in Patent Document 1. In the on-bed and indoor monitoring system in this patent document, the bed area, the bedside area, and the floor area are compared with the height at which the subject is photographed and the comparison reference height, and the area based on the comparison result in each area. The ratio calculation is performed. Based on the result, it is immediately recognized whether the subject is getting up on the bed, has left the bed, or is in a critical state of falling from the bed. The recognized result is then reported to a nurse, helper or family member.

JP 2014-38667 A [G08B 21/02, G08B 25/04]

  In addition to the subject falling down from the bed, the subject may be in a state where he / she is in physical condition and gently bends on the spot, leans against the wall or furniture, and becomes a middle waist or cramped state Possible state. However, in the watching system of the cited document 1, there is a possibility that a critical state other than the fall where the height of the subject does not change so much is not recognized as described above. For this reason, in the watching system of the cited document 1, it is conceivable that the state is not recognized depending on the crisis state that has occurred and the state is not reported to a nurse or the like.

  Therefore, a main object of the present invention is to provide a novel abnormal state notification system, abnormal state notification program, abnormal state notification method, and abnormal state notification device.

  Another object of the present invention is to provide an abnormal state notification system, an abnormal state notification program, an abnormal state notification method, and an abnormal state notification device that can appropriately notify that a human state is an abnormal state. is there.

  The present invention employs the following configuration in order to solve the above problems. The reference numerals in parentheses, supplementary explanations, and the like indicate the corresponding relationship with the embodiments described in order to help understanding of the present invention, and do not limit the present invention.

  The first invention is a detecting means for detecting the shape of a person in the space, an extracting means for extracting a feature quantity indicating the state of the person from the shape of the person detected by the detecting means, and a feature extracted by the extracting means. Based on the amount, the determination means for determining whether the person's state is abnormal, the storage means for storing the determination result by the determination means, the person's state is abnormal based on the determination result for a predetermined time stored by the storage means An abnormal condition notification system comprising: a recognition means for recognizing whether a person is in a state; and a notification means for notifying that a person's state is an abnormal state when the recognition means recognizes that the person's state is an abnormal state. is there.

  In the first invention, the detection means (12, 80) of the abnormal state notification system (100: reference numerals exemplifying corresponding parts in the embodiment, hereinafter the same) includes, for example, a distance image sensor (12), For example, information indicating the shape of a person is calculated from sensor information output from a distance image sensor. For example, the extraction means (80, S45-S51) extracts the position of each part of the body, the contact area with the floor and the wall, the posture of the person, and the like from the information indicating the shape of the person and makes it one feature quantity. . For example, a determination model for determining an abnormal state using machine learning is created in advance, and the determination means (80, S53) inputs a feature amount into the determination model to determine whether the human state is an abnormal state. judge. The storage means (80, S55, S75) stores the determination result by the determination means in, for example, the buffer (334, 336) of the memory (84). The recognizing means (80, S97) recognizes whether or not the human state is abnormal based on, for example, a determination result for a predetermined time (observation time) stored in the buffer. The notifying means (80, S99) notifies the caregiver or the like that the person is in an abnormal state when the person is recognized to be in an abnormal state.

  According to the first invention, it is possible to recognize various abnormal states by using the feature amount indicating the human state. Therefore, it is possible to appropriately notify that the human state is an abnormal state.

  The second invention is dependent on the first invention, and further comprises a calculating means for calculating a ratio in which the human state is determined to be abnormal within a predetermined time from the determination result for a predetermined time, and a recognition means Recognizes that the person is in an abnormal state when the ratio calculated by the calculating means is larger than the threshold value.

  In the second invention, the calculating means (80, S93) calculates, for example, the ratio at which the human state is determined to be abnormal within a predetermined time from the determination results for a predetermined time stored in the buffer. And when the calculated ratio is larger than a threshold value, it is recognized that a person's state is an abnormal state.

  According to the second invention, it is possible to improve the reliability of recognition by recognizing whether a person's state is abnormal using the determination result for a predetermined time.

  A third invention is dependent on the first invention or the second invention, wherein the feature amount includes a positional relationship between a person's head and waist, and the extracting means extracts the person's head and waist from the shape of the person. First extraction means for extracting the positional relationship is included.

  In the third invention, the first extraction means (80, S45) extracts, for example, the positions of the head and waist included in the positions of the respective parts of the person.

  A fourth invention is according to any one of the first to third inventions, wherein the space is partitioned by a floor and a wall, the feature amount further includes a contact area with the floor and the wall, and the extraction means includes And an area calculating means for calculating a contact area with the floor and the wall from the shape of the person.

  In the fourth invention, the space is a nursing room, for example, and is partitioned by a floor and a wall. The area calculating means (80, S47) calculates, for example, the area obtained from the shape of the cross section of the body at a fixed distance (about 5 cm) from the floor or wall as the contact area with the floor and wall.

  A fifth invention is according to any one of the first to fourth inventions, wherein the feature amount further includes a human posture, and the extracting means extracts a second human posture from the human shape. Means are further included.

  In the fifth invention, for example, the information indicating the shape of the person includes the posture of the person indicating the directions of the head and the torso. Then, the second extracting means (80, S49) extracts the directions of the head and the torso from information indicating the shape of a person, for example.

  According to the third invention, the fourth invention, and the fifth invention, recognizing whether the human condition is abnormal using the positional relationship between the head and the waist, the contact area, the posture of the person, etc. Thus, the possibility of being erroneously recognized can be reduced.

  A sixth invention is according to any one of the first to fifth inventions, further comprising a period determining means for determining whether a period during which the detecting means detects a human shape is shorter than a predetermined value, and a recognizing means When it is determined that the period for detecting a human shape is shorter than a predetermined value, the human body state is recognized based on the determination result for a predetermined time stored by the storage means.

  In the sixth invention, the period determining means (80, S9) determines whether the period in which the human shape is detected is shorter than a predetermined value (for example, 1 second). For example, when it is determined that the period in which the human shape is detected by the period determination unit is shorter than a predetermined value, it is considered that no abnormality has occurred in the detection of the human shape by the detection unit. Therefore, if it is determined that the period in which the human shape is detected is shorter than the predetermined value, the determination unit recognizes whether the human state is an abnormal state.

  A seventh invention is according to the sixth invention, and further comprises a detection abnormality notification means for notifying the abnormality of the detection means when the period determination means determines that the period for detecting the human shape is not shorter than a predetermined value. Prepare.

  In the seventh invention, if it is determined that the time during which the human shape is detected is not shorter than the predetermined value, it is considered that an abnormality has occurred in the detection of the human shape by the detecting means. Therefore, if it is determined that the time during which the human shape is detected is not shorter than the predetermined value, the detection abnormality notification means (80, S17) notifies the abnormality of the detection means to a human caregiver or the like.

  According to the sixth invention and the seventh invention, for example, a human caregiver or the like can notice that the person's condition cannot be properly recognized.

  According to an eighth aspect of the present invention, there is provided a processor (80) of an abnormal condition notification system (100) having detection means (12, 80) for detecting the shape of a person in the space based on the shape of the person detected by the detection means. Extracting means (S45-S51) for extracting a feature amount indicating a person's state, determination means (S53) for determining whether the person's state is abnormal from the feature amount extracted by the extracting means, Storage means for storing determination results (S55, S75), recognition means for recognizing whether the human state is abnormal based on the determination results for a predetermined time (observation time) stored by the storage means (S97) When the recognition unit recognizes that the person's state is abnormal, the abnormal state notification program is made to function as notification means (S99) for notifying that the person's state is abnormal. It is.

  A ninth aspect of the invention is an abnormal state notification method in the abnormal state notification system (100) having detection means (12, 80) for detecting the shape of a person in the space, the processor (80 of the abnormal state notification system) ) Is an extraction step (S45-S51) for extracting a feature quantity indicating the person's state from the shape of the person detected by the detection means, and the person's state is abnormal from the feature quantity extracted by the extraction step. Based on the determination step (S53) for determining whether or not, the storage step (S55, S75) for storing the determination result by the determination step, and the determination result for the predetermined time (observation time) stored by the storage step, A recognition step (S97) for recognizing whether the person is in an abnormal state, and when the person's state is recognized as an abnormal state by the recognition step, the state of the person Performing a notification step (S99) for notifying an abnormal state, the abnormal state notification method.

  A tenth aspect of the invention is a detection means (12, 80) for detecting the shape of a person in the space, and an extraction means (80, 80) for extracting a feature quantity indicating the state of the person from the shape of the person detected by the detection means. S45-S51), determination means (80, S53) for determining whether the human state is abnormal from the feature quantity extracted by the extraction means, and storage means (80, S55, for storing the determination result by the determination means) S75), recognizing means (80, S97) for recognizing whether the state of the person is abnormal based on the determination result for the predetermined time (observation time) stored by the storing means, and the state of the person by the recognizing means It is an abnormal condition notification device provided with a notification means (80, S99) for notifying that a person is in an abnormal condition when is recognized as an abnormal condition.

  In the eighth to tenth inventions, the same effect as in the first invention can be obtained.

  According to the present invention, it is possible to appropriately notify that a person's condition is abnormal.

  The above object, other objects, features, and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

FIG. 1 is an illustrative view showing an outline of an abnormal state notification system according to an embodiment of the present invention. FIG. 2 is an illustrative view showing one example of a configuration of the abnormal state notification system shown in FIG. FIG. 3 is a block diagram showing an example of an electrical configuration of the distance image sensor shown in FIG. FIG. 4 is a block diagram showing an example of the electrical configuration of the central controller shown in FIG. FIG. 5 is an illustrative view showing an example of an abnormal state recognized by the abnormal state notification system shown in FIG. 2, and FIG. 5 (A) shows an example of a state in which a person leans against furniture and gently crawls, FIG. 5B shows an example of a state in which a person leans against a wall and gently crawls. FIG. 6 is an illustrative view showing one example of a feature amount used when an abnormal state is recognized by the abnormal state notification system shown in FIG. FIG. 7 is an illustrative view showing another example of an abnormal state recognized by the abnormal state notification system shown in FIG. FIG. 8 is an illustrative view showing one example of a memory map of a memory of the central control unit shown in FIG. FIG. 9 is a flowchart showing an example of the abnormal state notification processing of the processor of the central controller shown in FIG. FIG. 10 is a flowchart showing an example of the storage process of the processor of the central controller shown in FIG. FIG. 11 is a flowchart showing an example of a holding process of the processor of the central controller shown in FIG. FIG. 12 is a flowchart showing an example of recognition processing of the processor of the central control unit shown in FIG.

  With reference to FIG. 1 and FIG. 2, the abnormal state notification system 100 of this embodiment is used in a space (environment) such as a nursing room in a nursing facility such as a nursing home. Furniture such as a bed is placed in the space, and a person such as a caregiver or a cared person can freely move in the space. In addition, the space is partitioned by a floor and a wall, and people can move from the space to other care rooms or hallways by passing through doorways and other doors provided on the walls of the space, or from the hallways ( It is possible to move to a nursing room. Further, a plurality of distance image sensors 12 (12a, 12b,...) Connected to the central control device 10 are provided on the ceiling in the space. When the abnormal state notification system 100 recognizes that the state of the person in the space is an abnormal state, the abnormal state notification system 100 notifies the caregiver or the like of the recognition result.

  The central control device 10 of the abnormal state notification system 100 is also referred to as an abnormal state notification device, and is connected to a distance image sensor 12 or the like. The central control device 10 acquires sensor information output by the distance image sensor 12 at regular intervals (for example, 0.1 seconds). The central control device 10 stores a history of information on the position of the person obtained from the sensor information of the distance image sensor 12 and the shape of the person indicated by the sensor information.

  A person ID is assigned to a person whose position is detected in the space. Person ID is given in the order detected.

  In addition, although the space of a present Example is a nursing care room of a nursing care facility, the abnormal condition notification system 100 can be used also in spaces, such as a hospital hospital room or a bedroom such as a house.

  Further, in FIG. 1, only one person is shown for simplicity, but there may be more people in the space.

  Further, when it is not necessary to distinguish the distance image sensors 12a, 12b,..., They are simply referred to as “distance image sensor 12”.

  In the present embodiment, the period in which the distance image sensor 12 outputs sensor information may be referred to as “elapsed time”.

  FIG. 3 is a block diagram showing an electrical configuration of the distance image sensor 12. Referring to FIG. 3, distance image sensor 12 includes a control IC 60 and the like. An A / D converter 64, a camera 68, a depth sensor 70, a depth camera 72, an I / O 74, and the like are connected to the control IC 60 via a bus 62.

  The control IC 60 includes a cache memory and the like, and controls the operation of the distance image sensor 12. For example, the control IC 60 operates according to a command from the central controller 10 and transmits the detected result to the central controller 10.

  A microphone 66 is connected to the A / D converter 64, and an audio signal from the microphone 66 is converted into digital audio information by the A / D converter 64 and input to the control IC 60.

  The camera 68 is a camera for photographing RGB information of the space in which the distance image sensor 12 is installed, that is, a color image. In addition, the camera 68 is provided in the distance image sensor 12 so that it can capture a space substantially the same as a space captured by a depth camera 72 described later.

  The depth sensor 70 is, for example, an infrared projector, and the depth camera 72 is, for example, an infrared camera. The depth sensor 70 irradiates the front surface of the distance image sensor 12 with, for example, infrared laser light. A special pattern is drawn in the space by the irradiated laser light, and the depth camera 72 captures the drawn pattern. The captured image is input to the control IC 60, and the control IC 60 analyzes the image to measure the depth information of the space irradiated with the laser light.

  The I / O 74 is a digital port capable of input / output control, and sensor information including voice information, RGB information, depth information, and the like is output from the output port and provided to the central controller 10. On the other hand, a control signal is output from the central controller 10 and applied to the input port.

  The distance image sensor 12 outputs RGB information and depth information and is sometimes called an RGB-D sensor.

  The distance image sensor 12 of this embodiment employs a product called a Kinect (registered trademark) sensor manufactured by Microsoft (registered trademark). However, in other embodiments, Xtion manufactured by ASUS (registered trademark), D-IMAGEr (registered trademark), which is a three-dimensional distance image sensor manufactured by Panasonic (registered trademark), and the like are employed as the distance image sensor 12. Also good.

  FIG. 4 is a block diagram showing an electrical configuration of the central controller 10. Referring to FIG. 4, central controller 10 includes a distance image sensor 12 and a processor 80. The processor 80 is sometimes called a microcomputer or CPU. A plurality of distance image sensors 12, a memory 84, an output device 86, an input device 88, and the like are connected to the processor 80 via a bus 82.

  The distance image sensor 12 outputs depth information as described above. This depth information includes the shape of the person in the space and the distance to the person. For example, when a person is sensed by the distance image sensor 12 provided on the ceiling, the shape of the head and both shoulders when the person is viewed from above and the distance to the head and both shoulders are obtained as depth information. .

In addition, 35 distance image sensors 12 are installed at predetermined positions (known) on the ceiling of the space, and the processor 80 acquires sensor information including depth information from each, and indicates information indicating the shape of the person. Calculate For example, the information indicating the shape of the person includes the position of the person in the space (world coordinate system) (for example, the position coordinates (x, y, z) of the feature point such as the center of gravity), the position of each part of the body (head, shoulder , Elbows, hands, chest, waist, knees, feet, etc.), human postures (for example, head orientation (θ _H ) and shoulder (torso) orientation (θ _B )).

  Here, the position of each part of the body is obtained by applying a three-dimensional humanoid model to the position of the person based on the position of the person's head and shoulders, the posture of the person, and the like. However, in other embodiments, the position of each part of the body may be calculated using other methods.

  In another embodiment, the position and posture of a person may be detected using not a distance image sensor 12 but a three-dimensional LRF. The distance image sensor 12 and the processor 80 may be collectively referred to as detection means.

  The processor 80 includes an RTC that controls the operation of the central controller 10 and outputs date and time information. The memory 84 includes a ROM, an HDD, and a RAM. In the ROM and the HDD, a control program for controlling the operation of the central controller 10 is stored in advance. The RAM is used as a work memory or a buffer memory for the processor 80.

  The output device 86 is, for example, a display, and the input device 88 is, for example, a mouse or a keyboard. For example, the administrator of the central control device 10 can check the status of this system using the output device 86 and the input device 88.

  FIG. 5A and FIG. 5B are illustrations showing an example of a fall that gently crouches. FIG. 5A shows a state in which, for example, a person leans against furniture (bed) and gently swells. FIG. 5B shows a state in which, for example, a person leans against a wall and gently crawls. For example, when a person develops anemia, there are a case where the person falls as it is, and a case where the person leans on the surroundings and slowly crawls, that is, falls slowly. Then, in the abnormal state notification system 100 of the present embodiment, an abnormal state such as a tumbling that gently crawls is recognized and notified to a caregiver or the like.

  Here, with reference to FIG. 6, the feature-value used in order to recognize an abnormal state is demonstrated. This feature amount indicates the state of the person and is extracted from the information indicating the shape of the person described above.

  First, as one of the feature amounts, the positional relationship of each part of the body included in the information indicating the shape of the person is extracted. For example, in the present embodiment, the distance between the head and the waist, the distance between the head and the feet, and the like are calculated as the positional relationship of each part of the body. However, in another embodiment, the position of each part of the human body may be one of the feature quantities.

  Next, as one of the feature amounts, an area where the person is in contact with the floor and an area where the person is in contact with the wall are calculated from information indicating the shape of the person. In this embodiment, when a human body exists within a certain distance (about 5 cm) from the floor or wall, it is possible to acquire the shape of the cross section of the body at a certain distance from the floor or wall. And in an Example, an area is calculated | required from the shape of the cross section of the body in a fixed distance from a floor or a wall, and the area is made into the area where a person is in contact with the floor, or the area where a person is in contact with the wall. However, the area is set to “0” when the body is not present at a certain distance from the floor or wall. The two areas may be collectively referred to as a contact area.

  Then, the orientation of the head and the orientation of the trunk included in the information indicating the shape of the person are extracted as one of the feature values. In addition, the direction of the head and the direction of the torso are sometimes referred to as “human posture”.

  Thus, the feature quantity indicating the state of the person includes the positional relationship (1) of each part of the body described above, the area where the person is in contact with the floor (2), the area where the person is in contact with the wall (3), Includes head orientation (4) and torso orientation (5). The feature amount is extracted every time sensor information is output from the distance image sensor 12.

  Subsequently, a plurality of feature quantities are prepared when the person's state becomes an abnormal state, the plurality of feature quantities are used as teacher data, and the teacher data is learned by a machine learning method such as SVM (Support Vector Machine). Thereby, it is possible to create a determination model for determining an abnormal state such as a fall that gently vortexes. When an unknown feature amount is input to this determination model, a determination result as to whether the human state is abnormal is output.

  When a determination result is output from the determination model, it is stored (accumulated) in the memory 84 of the central controller 10 in association with the time. In the present embodiment, information indicating a person's shape is calculated from sensor information, a feature amount is extracted from the information indicating the person's shape, and whether the person's state is abnormal each time the feature amount is extracted. Is determined. That is, every time sensor information output by the distance image sensor 12 is acquired, a determination result is output from the determination model. Therefore, each determination result is associated with a time when the sensor information is acquired (hereinafter referred to as an acquisition time). The determination result may be referred to as “queue”.

  Subsequently, when the determination results for a predetermined time are accumulated, the ratio of the determination results for the predetermined time determined as an abnormal state (hereinafter referred to as an abnormality determination ratio) is obtained. When the abnormality determination ratio is larger than a threshold value (for example, 80%), it is recognized that the human state is an abnormal state. And if it recognizes as an abnormal state, a caregiver etc. will be notified that a person's state is an abnormal state. Thus, the recognition reliability can be improved by recognizing whether the human state is an abnormal state using the determination result for a predetermined time. Further, in this embodiment, by calculating the abnormality determination ratio, it is recognized that the human state is an abnormal state even if the determination results determined to be abnormal are not continuous.

  For example, when the determination result determined to be an abnormal state is continuous for a predetermined time, if the human state is recognized as being in an abnormal state, it is determined even if it is not an abnormal state within the predetermined time If it is done, even if the state of the person is an abnormal state, it may not be recognized as an abnormal state. Therefore, the above-described problem is avoided in advance by obtaining the abnormality determination ratio to recognize the abnormal state.

  From the above, it is possible to recognize various abnormal states by using the feature amount indicating the human state. Therefore, it is possible to appropriately notify a caregiver or the like that the person is in an abnormal state.

  Further, by recognizing whether the human state is abnormal using the positional relationship between the head and the waist, the contact area, the posture of the person, and the like, the possibility of erroneous recognition can be reduced.

  Further, the period (elapsed time) at which the distance image sensor 12 outputs sensor information may be delayed due to a failure of the distance image sensor 12 or the like. When the elapsed time is delayed, recognition of the abnormal state is delayed, and it is impossible to appropriately notify that the abnormal state is recognized. Therefore, in this embodiment, it is determined whether the elapsed time is delayed every time sensor information is acquired. If the elapsed time is delayed, a sensor abnormality is notified to a caregiver or the like. As a result, a caregiver or the like can notice that the person's state is not properly recognized as being in an abnormal state.

  Note that the predetermined time for obtaining the abnormality determination ratio is sometimes referred to as observation time. Further, the abnormal state notification system 100 according to the present embodiment can appropriately recognize not only an overturn that gently swells but also an abnormal state such as a lying overturn as shown in FIG.

  The features of the present embodiment have been outlined above. Hereinafter, the present embodiment will be described in detail using the memory map of the memory 84 shown in FIG. 8 and the flowcharts shown in FIGS.

  FIG. 8 is an illustrative view showing one example of a memory map of the memory 84 of the central controller 10 shown in FIG. As shown in FIG. 8, the memory 84 includes a program storage area 302 and a data storage area 304. In the program storage area 302, as a program for operating the central control apparatus 10, an abnormal state notification program 310 for recognizing and notifying an abnormal state of a person, a storage program 312 for storing a determination result based on a determination model. A holding program 314 for holding determination results for a predetermined time, a recognition program 316 for recognizing whether a person is in an abnormal state, and the like are stored. The storage program 312, the holding program 314, and the recognition program 316 are executed as subroutines of the abnormal state notification program 310.

  In addition, although illustration is abbreviate | omitted, the program for operating the central control apparatus 10 includes the program for detecting a person's position, etc.

  In the data storage area 304, a time buffer 330, a sensor buffer 332, a determination result buffer 334, a holding buffer 336, and the like are provided.

  In the time buffer 330, time information output by the RTC included in the processor 80 is temporarily stored. Sensor information output from the distance image sensor 12 is temporarily stored in the sensor buffer 332. Each sensor information stored in the sensor buffer 332 is associated with time information. The determination result buffer 334 stores (accumulates) the determination result determined by the determination model. The holding buffer 336 temporarily stores (holds) determination results for a predetermined time.

  Although not shown, the data storage area 304 is also provided with a buffer for temporarily storing various calculation results, and other counters and flags necessary for the operation of the central controller 10.

  The processor 80 of the central control device 10 performs an abnormal state notification process shown in FIG. 9, a storage process shown in FIG. 10, a holding process shown in FIG. A plurality of tasks including the recognition process shown in FIG.

  FIG. 9 is a flowchart of the abnormal state notification process. When sensor information output from the distance image sensor 12 is received and the sensor information is stored in the sensor buffer 332, an abnormal state notification process is executed.

  When the abnormal state notification process is executed, the processor 80 acquires sensor information in step S1. That is, the sensor information stored in the sensor buffer 332 is acquired. Subsequently, in step S3, the processor 80 determines whether or not sensor information has been acquired for the first time. That is, it is determined whether a plurality of pieces of sensor information are stored in the sensor buffer 332. If “YES” in the step S3, that is, the sensor information stored in the sensor buffer 332 is one, and when the first sensor information is acquired for the first time, the processor 80 performs the storing process in a step S5. Run. That is, the determination result of the abnormal state determined based on the sensor information is associated with the acquisition time of the sensor information, and the determination result is stored in the determination result buffer 334. Since this storage process will be described later, a detailed description thereof is omitted here. Then, when the process of step S5 ends, the processor 80 ends the abnormal state notification process.

  If “NO” in the step S3, that is, if a plurality of pieces of sensor information are stored in the sensor buffer 332 and the acquired sensor information is not the first, the processor 80 sets the elapsed time in a step S7. calculate. That is, the period at which the distance image sensor 12 outputs the sensor information is calculated. Specifically, the time associated with the latest sensor information stored in the sensor buffer 332 and the previous sensor information. The elapsed time (cycle) is calculated from the time associated with. Subsequently, in step S9, the processor 80 determines whether or not the elapsed time is shorter than a predetermined value. That is, it is determined whether an abnormality has occurred in the distance image sensor 12 by using an elapsed time indicating a cycle in which the distance image sensor 12 outputs sensor information.

  In the present embodiment, the predetermined value is, for example, “1 second”. The processor 80 that executes the process of step S9 functions as a time determination unit.

  If “YES” in the step S9, that is, if no abnormality has occurred in the distance image sensor 12, the processor 80 executes a storage process in a step S11 as in the case of the step S5. Subsequently, the processor 80 executes a holding process in step S13, and executes a recognition process in step S15. That is, in the holding process, the determination result for a predetermined time is held by the holding buffer 336, and in the recognition process, it is recognized whether the person's state is abnormal using the held determination result. Then, when the process of step S15 ends, the processor 80 ends the abnormal state notification process. Since the holding process and the recognition process will be described later, a detailed description thereof will be omitted here.

  If “NO” in the step S9, for example, if the elapsed time is longer than a predetermined value and there is a possibility that the distance image sensor 12 has an abnormality, the processor 80 notifies the sensor abnormality in a step S17. To do. For example, the output device 86 of the central control device 10 outputs a message that informs that there is a possibility that the distance image sensor 12 is abnormal. The processor 80 that executes the process of step S17 functions as a detection abnormality notification unit.

  Subsequently, the processor 80 executes a storage process in step S19, and executes a holding process in step S21. That is, even when it is considered that an abnormality has occurred in the distance image sensor 12, the determination result is stored in the determination result buffer 334, and the determination result for a predetermined time is held in the holding buffer 336. Then, when the process of step S19 ends, the processor 80 ends the abnormality notification process.

  FIG. 10 is a flowchart of the storage process. For example, in the abnormal state notification process shown in FIG. 9, when any of step S5, step S11, or step S19 is executed, the storage process is executed.

  When the storage process is executed, the processor 80 reads the acquisition time in step S41. That is, the time information stored in the time buffer 330 is read as the acquisition time.

  Subsequently, in step S43, the processor 80 calculates the shape of the person from the sensor information. For example, information indicating the shape of the person including the position of the person, the position of each part of the body, the posture of the person, and the like is calculated from the depth information included in the sensor information.

  Subsequently, in step S45, the processor 80 extracts the positional relationship of each part of the body from the shape of the person. For example, the distance between the head and the waist, the distance between the head and the feet, and the like are calculated. The processor 80 that executes the process of step S45 functions as a first extraction unit.

  Subsequently, in step S47, the processor 80 calculates the contact area with the floor and the wall from the shape of the person. For example, the area where the person is in contact with the floor and the area where the person is in contact with the wall are calculated from the shape of the cross section of the body at a certain distance from the floor or wall. The processor 80 that executes the process of step S47 functions as an area calculating unit.

  Subsequently, in step S49, the processor 80 extracts a human posture from the human shape. For example, in this embodiment, the head orientation and the trunk orientation included in the information indicating the shape of the person are extracted. The processor 80 that executes the process of step S49 functions as a second extraction unit.

  Subsequently, in step S51, the processor 80 creates a feature amount. For example, feature quantities including the positional relationship of each part of the body, the area where the person is in contact with the floor, the area where the person is in contact with the wall, the head direction, and the body direction are created. Note that the processor 80 that executes the processes of steps S45 to S51 functions as an extraction unit.

  Subsequently, in step S53, the processor 80 determines whether or not there is an abnormal state from the feature amount. For example, a feature amount is input to a determination model that determines that the state is abnormal. The processor 80 that executes the process of step S53 functions as a determination unit.

  Subsequently, in step S55, the processor 80 associates the determination result with the acquisition time and stores the determination result in the determination result buffer 334. For example, the acquisition time acquired in step S41 is associated with the determination result of the abnormal state output from the determination model. Then, the determination result associated with the acquisition time is stored in the determination result buffer 334. The processor 80 that executes the process of step S55 functions as a storage unit.

  Then, when the process of step S55 ends, the processor 80 ends the storage process and returns to the abnormal state notification process.

  FIG. 11 is a flowchart of the holding process. For example, in the abnormal state notification process shown in FIG. 9, when step S13 or step S21 is executed, the holding process is executed.

  When the holding process is executed, the processor 80 calculates the accumulation time of the determination result in step S71. That is, the time when the determination result is accumulated in the determination result buffer 334 is calculated. Specifically, the difference between the acquisition time of the oldest determination result stored in the determination result buffer 334 and the acquisition time of the latest determination result is calculated as the accumulation time.

  Subsequently, in step S73, the processor 80 determines whether or not the accumulation time is longer than the observation time. For example, if the observation time is “1 minute”, it is determined whether the accumulation time for accumulating the determination result in the determination result buffer 334 is longer than “1 minute (observation time)”. If “YES” in the step S73, that is, if the accumulation time for accumulating the determination result is longer than the observation time, the processor 80 stores the determination result of the determination result buffer 334 in the holding buffer 336 in a step S75. That is, determination results for the observation time (predetermined time) stored in the determination result buffer 334 are stored (held) by the holding buffer 336. The processor 80 that executes the process of step S75 functions as a storage unit.

  Subsequently, in step S77, the processor 80 deletes the oldest determination result. That is, the oldest determination result among the determination results stored in the determination result buffer 334 is deleted.

  Here, by executing the processing of step S75, it is possible to hold the determination results for a predetermined time necessary for recognition. Moreover, unnecessary determination results can be deleted by executing the processing of step S77. The processor 80 that executes the process of step S75 functions as a holding unit, and the processor 80 that executes the process of step S77 functions as a deleting unit.

  Then, if the process of step S77 ends or if “NO” in step S73, that is, if the accumulation time is shorter than the observation time, the processor 80 ends the holding process and returns to the abnormal state notification process.

  FIG. 12 is a flowchart of recognition processing. For example, in the abnormal state notification process shown in FIG. 9, when step S15 is executed, a recognition process is executed.

  When the recognition process is executed, the processor 80 determines whether or not the determination result is held in the holding buffer 336 in step S91. That is, it is determined whether the determination result is stored in the holding buffer 336. If “NO” in the step S91, that is, if the determination result is not held by the holding buffer 336, the processor 80 ends the recognition process and returns to the abnormal state notification process.

  On the other hand, if “YES” in the step S91, that is, if the determination result is held in the holding buffer 336, the processor 80 calculates an abnormality determination ratio from the determination result held in the holding buffer 336 in a step S93. . That is, of the determination results stored in the holding buffer 336, the ratio determined as an abnormal state is calculated. The processor 80 that executes the process of step S93 functions as a calculation unit.

  Subsequently, in step S95, the processor 80 determines whether the abnormality determination ratio is greater than a threshold value. That is, it is determined whether to recognize that the state is abnormal. If “NO” in the step S95, that is, if the calculated abnormality determination ratio is smaller than the threshold value, the processor 80 proceeds to the process of the step S101.

  On the other hand, if “YES” in the step S95, that is, if the abnormality determination ratio is larger than the threshold value, the processor 80 recognizes the abnormal state in a step S97, and notifies the abnormal state in a step S99. That is, it is recognized whether the person is in an abnormal state, and a caregiver or the like is notified that the person is recognized as an abnormal state. For example, when the caregiver has a portable terminal, it is notified that the person is in an abnormal state by e-mail or the like. In addition, the caregiver may be notified of the abnormal state of the person not only by mail but also by voice, light, vibration, and a combination thereof. The processor 80 that executes the process of step S97 functions as a recognition unit, and the processor 80 that executes the process of step S99 functions as a notification unit.

  Subsequently, in step S101, the processor 80 initializes the holding buffer 336. That is, since the determination result held by the holding buffer 336 is used to recognize whether the state is abnormal, the determination result held (stored) by the holding buffer 336 is deleted.

  Then, when the process of step S101 ends, the processor 80 ends the recognition process and returns to the abnormal state notification process.

  In another embodiment, it may be recognized whether the human state is abnormal every predetermined time. In other embodiments, the contact area with the furniture may be calculated as one of the feature amounts.

  In still other embodiments, the determination model may be created by a machine learning method using multiple regression analysis, a neural network, or an algorithm such as C4.5.

  In the above-described embodiments, words such as “greater than” are used for the threshold (predetermined value) and the like, but “greater than the threshold” includes the meaning of “greater than or equal to the threshold”. Similarly, “smaller than a threshold” includes the meanings “below the threshold” and “below the threshold”.

  The plurality of programs described in the present embodiment may be stored in the HDD of the data distribution server and distributed to the system or the central control apparatus 10 having the same configuration as that of the present embodiment via a network. In addition, storage programs such as CDs, DVDs, and BDs (Blu-ray (registered trademark) Disc) are sold or distributed with these programs stored in storage media such as USB memory and memory card. May be. When the plurality of programs downloaded through the server or storage medium described above are applied to a system having the same configuration as that of the present embodiment or the central control apparatus 10, the same effects as those of the present embodiment can be obtained. .

  The specific numerical values given in this specification are merely examples, and can be appropriately changed according to a change in product specifications.

DESCRIPTION OF SYMBOLS 10 ... Central control device 12 ... Distance image sensor 80 ... Processor 84 ... Memory 100 ... Abnormal state notification system

Claims (10)

Detection means for detecting the shape of a person in space;
Extraction means for extracting a feature quantity indicating the state of the person from the shape of the person detected by the detection means;
Determination means for determining whether the person's condition is abnormal from the feature quantity extracted by the extraction means;
Storage means for storing a determination result by the determination means;
Recognizing means for recognizing whether the state of the person is an abnormal state based on a determination result for a predetermined time stored by the storage unit; and the recognizing unit recognizes that the state of the person is an abnormal state. An abnormal state notification system comprising notification means for notifying that the state of the person is an abnormal state. From the determination result for the predetermined time, further comprising a calculation means for calculating a ratio determined that the person's state is abnormal within the predetermined time,
The abnormal state notification system according to claim 1, wherein the recognition unit recognizes that the state of the person is an abnormal state when the ratio calculated by the calculation unit is larger than a threshold value. The feature amount includes a positional relationship between the person's head and waist.
The abnormal condition notification system according to claim 1, wherein the extraction unit includes a first extraction unit that extracts a positional relationship between the person's head and waist from the shape of the person. The space is partitioned by a floor and a wall;
The feature amount further includes a contact area between the floor and the wall,
The abnormal condition notification system according to claim 1, wherein the extraction unit further includes an area calculation unit that calculates a contact area between the floor and the wall from the shape of the person. The feature amount further includes a posture of the person,
The abnormal condition notification system according to claim 1, wherein the extraction unit further includes a second extraction unit that extracts the posture of the person from the shape of the person. A cycle determination unit that determines whether a cycle in which the detection unit detects the shape of the person is shorter than a predetermined value;
When it is determined that the period for detecting the shape of the person is shorter than a predetermined value, the recognizing unit determines that the human state is abnormal based on the determination result for the predetermined time stored by the storage unit. The abnormal condition notification system according to claim 1, wherein the abnormal condition notification system recognizes whether or not there is an abnormality.   The abnormal state notification system according to claim 6, further comprising a detection abnormality notification unit that notifies the abnormality of the detection unit when it is determined by the cycle determination unit that the cycle for detecting the shape of the person is not shorter than a predetermined value. . A processor of an abnormal condition notification system having detection means for detecting the shape of a person in space;
Extraction means for extracting a feature quantity indicating the state of the person from the shape of the person detected by the detection means;
Determination means for determining whether the person's condition is abnormal from the feature quantity extracted by the extraction means;
Storage means for storing a determination result by the determination means;
Recognizing means for recognizing whether the state of the person is an abnormal state based on a determination result for a predetermined time stored by the storage unit; and the recognizing unit recognizes that the state of the person is an abnormal state. An abnormal state notification program that functions as notification means for notifying that the state of the person is an abnormal state. An abnormal state notification method in an abnormal state notification system, comprising a detection means for detecting the shape of a person in space, wherein the processor of the abnormal state notification system comprises:
An extraction step of extracting a feature amount indicating the state of the person from the shape of the person detected by the detection means;
A determination step of determining whether the state of the person is an abnormal state from the feature amount extracted by the extraction step;
A storage step for storing a determination result by the determination step;
A recognition step for recognizing whether the person's state is an abnormal state based on the determination result for a predetermined time stored in the storage step; and the recognition step recognizes that the person's state is an abnormal state. An abnormal state notification method that executes a notification step of notifying that the state of the person is an abnormal state. Detection means for detecting the shape of a person in space;
Extraction means for extracting a feature quantity indicating the state of the person from the shape of the person detected by the detection means;
Determination means for determining whether the person's condition is abnormal from the feature quantity extracted by the extraction means;
Storage means for storing a determination result by the determination means;
Recognizing means for recognizing whether the state of the person is an abnormal state based on a determination result for a predetermined time stored by the storage unit; and the recognizing unit recognizes that the state of the person is an abnormal state. An abnormal state notification device comprising notification means for notifying that the state of the person is an abnormal state.

Images