JP7366820B2 - Behavior recognition server and behavior recognition method - Google Patents

Description

The present invention relates to a behavior recognition server and a behavior recognition method.

In recent years, high-performance sensors connected to the Internet have become popular as IoT (Internet of Things) devices. Attempts are being made to provide monitoring services for people being observed by analyzing a large amount of sensor information from a large number of sensors installed in environments such as homes.

For example, Patent Document 1 describes a system that estimates daily activities by combining a presence detection sensor installed in a home and a motion detection sensor. The degree of abnormality of the estimated current lifestyle behavior is determined by comparing it with a lifestyle behavior pattern database prepared in advance.

Japanese Patent Application Publication No. 2002-352352

When providing a monitoring service for an observed person, it is necessary to recognize with high precision whether the observed person's current behavior is normal or abnormal. This is because the quality of the monitoring service will deteriorate if a warning is issued when the behavior is normal or if abnormal behavior is overlooked. Therefore, in order to improve recognition accuracy, it is important to have a phase in which the behavior of the observed person is uniquely identified from multiple types of sensor information measured at the same time.

Note that when individual sensor information is used alone, only basic information such as "the person to be observed is in the living room" or "the person to be observed is sitting" can be obtained. Therefore, it is necessary to combine multiple pieces of sensor information to identify the detailed behavior of the observed person, such as "eating in the living room" or "watching TV in the living room."

In Patent Document 1, it is necessary to prepare a daily activity database in advance as a process for identifying daily activities from sensor information, and the effort of preparing it becomes a burden on the administrator. For example, in order to construct a lifestyle behavior database, it is necessary to prepare statistical parameters and question answer data from the observed person in advance, which is a burden.

Therefore, the main objective of the present invention is to recognize the behavior of an observed person with high precision and at low cost.

In order to solve the above problems, the behavior recognition server of the present invention has the following features.
The present invention includes a sensor information acquisition unit that acquires sensor information indicating a detection result for each sensor from a set of sensors that detect an observed person;
In addition to the arbitrary range of values based on the values of each sensor information, the arbitrary range of values based on the time information of the sensor information can also be applied to the area of each sensor information in the image defined by the layout data. an image conversion unit that converts each sensor information into an image by writing;
By inputting the image of the sensor information created by the image conversion unit into an inference model in which correspondence data between the image of the sensor information and the behavior of the observed person is prepared in advance, A behavior inference unit that determines the behavior of the observed person to be output.
Other means will be described later.

According to the present invention, the behavior of an observed person can be recognized with high precision and at low cost.

1 is a configuration diagram of an action recognition system according to an embodiment of the present invention. 1 is a hardware configuration diagram of an action recognition system according to an embodiment of the present invention. FIG. 2 is a configuration diagram showing details of a behavior recognition server according to an embodiment of the present invention. It is a flowchart which shows the processing of the behavior recognition server regarding one embodiment of the present invention. FIG. 3 is an explanatory diagram showing an example of layout data used for imaging processing according to an embodiment of the present invention. 6 is an explanatory table of the layout data of FIG. 5 related to one embodiment of the present invention. FIG. 7 is an explanatory diagram of image data resulting from writing the "value" of FIG. 6 into the layout data of FIG. 5 according to an embodiment of the present invention. FIG. 7 is an explanatory diagram of image data resulting from writing actions different from those in FIG. 7 on the layout data of FIG. 5 according to an embodiment of the present invention. FIG. 2 is an explanatory diagram showing an example of a normal behavior pattern stored in a database according to an embodiment of the present invention. FIG. 3 is an explanatory diagram showing an example of a current behavior pattern stored in a current behavior storage unit according to an embodiment of the present invention. FIG. 3 is a screen diagram showing a display screen when the current behavior pattern is normal according to an embodiment of the present invention. FIG. 3 is a screen diagram showing a display screen when the current behavior pattern is abnormal according to an embodiment of the present invention.

Hereinafter, one embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is a configuration diagram of an action recognition system.
The behavior recognition system is configured such that an observer 3u remotely monitors the behavior pattern of an observed person 2u living at home 2h using an observer terminal 3. The "behavior pattern" is a series of patterns in which the "behavior" of the observed person 2u at each time is connected in chronological order.
The behavior recognition server 1 recognizes the behavior pattern of the observed person 2u based on the sensor information acquired from the various sensors 2, and notifies the observer terminal 3 of the recognition result. Thereby, the observer 3u who looks at the display screen of the observer terminal 3 can grasp the behavioral pattern of the observed person 2u.
The observed person 2u is, for example, a care recipient, and the observer 3u is, for example, a family member of the care recipient. Alternatively, the behavior recognition system may be introduced at a hospital or nursing care facility instead of at home 2h, in which case the observer 3u becomes a doctor or care manager.

In the home 2h, various sensors 2 for monitoring the behavior of the person to be observed 2u are connected to a network. The sensor 2 may be, for example, a sensor built into a home appliance such as a refrigerator 2a or an autonomously moving vacuum cleaner 2b, or may be a single sensor such as a human sensor 2c.
Note that it is desirable that the sensor 2 such as the human sensor 2c is installed in a direction where its measurement area does not face the entrance of the room. With this installation, it is possible to prevent the human sensor 2c from erroneously detecting a person different from the observed person 2u passing by in the hallway outside the room.

FIG. 2 is a hardware configuration diagram of the behavior recognition system.
The sensor 2 includes a communication unit 121 that notifies other devices of sensor information detected by the detection unit 122, a detection unit 122 that detects the observed person 2u, and a communication unit 122 that notifies the observed person 2u of messages etc. from the observer 3u. It has a notification section 123 that provides notification. Note that the notification unit 123 may be incorporated into the sensor 2, or may be configured as a separate device from the sensor 2 (a display, a speaker, etc.).
The behavior recognition server 1 includes a communication unit 111 that receives sensor information from the sensor 2 and notifies the observer terminal 3 of the recognition result from the sensor information, a control unit 112 that recognizes the behavior pattern of the observed person 2u, and a control unit 111 that receives sensor information from the sensor 2 and notifies the observer terminal 3 of the recognition result from the sensor information. The storage unit 113 stores data used in the processing of the unit 112.
The observer terminal 3 has a communication unit 131 that receives the recognition result of the observed person 2u, a notification unit 132 that notifies the observer 3u of the recognition result of the observed person 2u, and inputs messages etc. from the observed person 2u. It has an input section 133 that allows

The behavior recognition server 1 is configured as a computer having a CPU (Central Processing Unit) as an arithmetic unit (control unit 112), a memory as a main storage device, and a hard disk as an external storage device (storage unit 113).
In this computer, the CPU operates a control section (control means) constituted by each processing section by executing a program (also called an application or an abbreviated application) read into the memory.

FIG. 3 is a configuration diagram showing details of the behavior recognition server 1.
The control unit 112 (FIG. 2) of the behavior recognition server 1 includes a sensor information acquisition unit 11, a time information acquisition unit 12, an image conversion unit 13, a behavior inference unit 14, a current behavior identification unit 15, and a current behavior accumulation unit. 16 and a change detection section 18.
The storage unit 113 (FIG. 2) of the behavior recognition server 1 stores layout data 13L, an inference model 14m, and a database 17.
The details of the components shown in FIG. 3 will be explained below along with the flowchart shown in FIG. 4.

FIG. 4 is a flowchart showing the processing of the behavior recognition server 1.
The sensor information acquisition unit 11 acquires sensor information from the sensors 2 (refrigerator 2a, vacuum cleaner 2b, human sensor 2c) installed in the home 2h (S101). The sensor information may have a different data format depending on the type of sensor 2.
The time information acquisition unit 12 acquires time information indicating the measurement time of the sensor information acquired in S101 (S102). The time information acquisition unit 12 acquires the time when the sensor 2 includes a time stamp in the sensor information, and acquires the reception time of the sensor information when there is no time stamp.

The image conversion unit 13 converts the sensor information acquired in S101 and the time information acquired in S102 into one image (S103).
Note that the layout data 13L that the image conversion unit 13 refers to when converting into an image defines in advance information regarding the layout within the image, such as where the sensor information of which sensor 2 is to be placed in which part of the image ( Figure 5, Figure 6).

The behavior inference unit 14 infers the behavior of the observed person 2u based on the time information of the sensor information by inputting image data indicating the sensor information into the inference model 14m (S104). For this inference process, an inference model 14m that outputs a corresponding action when image data is input in advance is prepared in advance. The inference model 14m may be trained by, for example, a machine learning algorithm such as deep learning, or the equation of the inference model 14m may be established from the estimation contents illustrated below.
・If all the motion sensors in the house do not respond, it is assumed that the person is out.
・If the motion sensor responds in the bedroom at night and the illuminance is low, it is assumed that the person is sleeping.
The current behavior identification unit 15 stores in the current behavior storage unit 16 a “current behavior pattern” in which the “behaviors” of the observed person 2u at each time inferred by the behavior inference unit 14 are connected in chronological order (S105).

When the change detection unit 18 detects a change (deviation) in the current behavior pattern from the normal behavior pattern registered in advance in the database 17 (S111, Yes), it determines that an abnormality has occurred in the behavior pattern of the observed person 2u. Therefore, the change (abnormality) is notified to the observer 3u via the observer terminal 3 (S112). Further, the output destination of S112 is not limited to the customer environment (observer terminal 3), but may be output to other systems such as a database system or a cloud system.
On the other hand, when a change (deviation) from the normal behavior pattern is not detected (S111, No), the change detection unit 18 registers the current behavior pattern as it is in the database 17 as a normal behavior pattern (S113), and returns the process to S101. return.

A specific example of the process (S103) of the image conversion unit 13 will be described below with reference to FIGS. 5 to 8.
FIG. 5 is an explanatory diagram showing an example of layout data 13L used by the image conversion unit 13 for imaging processing. In the layout data 13L, the data content to be written at each position in the square image data of 12 squares in the vertical direction and 12 squares in the horizontal direction is arranged as symbols in the figure such as "T" and "ACC1". Note that a "cell" is the smallest unit that subdivides an area within an image into which data content is written, and at least one area is allocated to sensor information and time information. Further, the 12×12 squares in the image data are just an example, and the number of squares may be smaller or larger than that.

FIG. 6 is an explanatory table of the layout data 13L in FIG. For example, the top "T" in FIG. 5 corresponds to the symbol "T" in the first line "Time" in FIG. Note that the image data placed at the top “T” in FIG. 5 is time data acquired by the time information acquisition unit 12. In other words, one image shown in Figure 6 is a visualization of a collection of sensor information measured at the same measurement time (time data of "T") from sensors 2 placed at each location. This is the result.

Layout data 13L in FIG. 6 is an example of arranging the same type of sensor information close to each other in an image. On the other hand, pieces of sensor information having the same sensor installation location (room) may be placed together in close proximity within the image.
The types of sensors 2 include, for example, those that detect the movement of the observed person 2u such as an acceleration sensor and a (door) opening/closing sensor, those that detect the presence of the observed person 2u such as a human sensor, and those that detect the presence of the observed person 2u such as a human sensor, and humidity sensors. Examples include sensors that measure the environment at home 2 hours.

The third column "number of squares" in the explanation table indicates the size of the area. Note that if the amount of data to be written is smaller than the amount of data that can be expressed by the area, the area will be left over. At that time, the image conversion unit 13 fills up the number of squares in the image by copying and writing the same data content in multiple locations.

Note that the number of cells in the layout data 13L indicates the weight between pieces of information to be written, and the larger the number of cells allocated, the greater the influence on behavior. This distribution of the number of squares is determined, for example, by the following policy P1.
・Humans are accustomed to taking actions based on the time of day, such as going out during the day and sleeping at night, so time information "T" is allocated a larger number of squares (24 squares) than other sensor information. .
- Since the actions that humans can take are limited to a certain extent depending on where they are, the sensor information (location information) of human sensors "HM1 to HM5" is allocated a larger number of squares (12 squares) than other sensor information.
・Humans have the habit of taking the same actions depending on the day of the week, such as going to work on weekdays and resting at home on holidays, so day of the week information "DoW" has a larger number of squares than sensor information that measures the environment 2 hours at home. Allocate (12 squares).
・As sensor information for detecting human movements, acceleration sensors "ACC1 to ACC4" and opening/closing sensors "OC1 to OC3" are allocated a larger number of squares (4 squares) than sensor information for measuring the environment of the home 2 hours. .
The above-exemplified policy P1 is an example in which a motion sensor is attached to each room to mainly read "where people are = what they are doing" in a floor plan with a plurality of rooms.

On the other hand, the policy (required sensors and their importance) may be changed as appropriate depending on the behavior to be detected and the type of floor plan.
For example, although the importance of environmental sensors was low in policy P1, a policy P2 may be used in which the importance of environmental sensors is increased, as described below.
・If you increase the sensitivity of the temperature sensor, you can read the ON/OFF status of the air conditioner from the temperature sensor.
- If you increase the sensitivity of the humidity sensor near the bathroom, you can detect the opening and closing of the bathroom door based on changes in humidity.
・If behavior and illuminance are linked (such as turning off the light when going to bed at night or opening the curtains in the morning), you can derive behavior from illuminance.

As another example, although the importance of the human sensor is high in the policy P1, a policy P3 may be used in which the position of the person is determined from a sensor other than the human sensor, as described below.
- If a large number of motion sensors cannot be installed due to the floor plan of a studio, etc., the operation information of home appliances may be able to read the location of the person in more detail than the motion sensors.
・While the refrigerator door is being operated, information can be obtained that there is a person in front of the refrigerator.

In this way, the layout data 13L is defined according to the policies P1 to P3, etc. For example, in policy P1, in descending order of the area within the image, sensor information time information "T", sensor information "HM1 to HM5" indicating the location of the observed person, and sensor information indicating the observed person's movement "ACC1 to ACC4". ”, the size of the area defined as sensor information “TMP” that measures the living environment of the person being observed is defined.
Thereby, by reflecting the empirical rule of the degree of influence of each sensor information on behavior as the size of the area in the input data of the inference model 14m, the accuracy of behavior recognition can be improved. This time, we used an empirical rule that time information and location information have a greater influence on behavior recognition than other information.

The fourth column "value" of the explanation table indicates the data content to be written into the area. For example, when the color depth of image data is 8 bits of gray scale, the amount of data that can represent an area is 2 to the 8th power = 256 different numerical values.
Of course, the color depth of the image data is not limited to grayscale, and colors that can express values in more detailed stages may be used. Since the color depth of image data can be set arbitrarily, the amount of data that can be expressed is not limited to 256 ways. Therefore, for example, in 8-bit gray scale and 16-bit color, the same sensor response value may be converted to different values and precision.

In this embodiment, the range from 0.00 to 1.00 is described with an accuracy of 0.01.
For example, the value "0.31" of time "T" indicates 7:40 a.m. when 0:00 is the value "0.00" and 23:59 is the value "1.00". On the other hand, the day of the week is selected from seven types, where Monday is the value "0.00" and Sunday is the value "1.00".
Note that the above-mentioned "value" refers to a value in an arbitrary range based on the value of each sensor information. As mentioned above, in addition to the case where the color corresponds to the value of each sensor information, it also includes the case where the value of each sensor information is referred to as it is. In this way, even if the range of possible values varies depending on the type of sensor, it is normalized as a value within an arbitrary range.

In addition, in the "Humidity" row "HUM" in Figure 6, the number of squares "5 (=1x5)" means that the area size of the humidity sensor is one square for one sensor, and the number of sensors is 5. This means a total of 5 squares. Also, the "humidity" values "0.66, 0.57, 0.64, 0.58, 0.7" are, in order from the left, the value of the first humidity sensor "0.66", the value of the second humidity sensor "0.57", ..., the value of the fifth humidity sensor Indicates "0.7".
Furthermore, although the number of human sensors "HM1 to HM5" is five in the example, the type and number of sensors may change, increase or decrease depending on the layout of the room and the behavior that is desired to be recognized.

FIG. 7 is an explanatory diagram of image data resulting from writing the "value" of FIG. 6 into the layout data 13L of FIG. 5. In FIG. 7, symbols in the diagram such as "T" and "ACC1" are also shown to make the explanation easier to understand, but in reality, the symbols in the diagram are omitted from the image.
For example, the image conversion unit 13 writes black, which indicates the value "0", in the "ACC1" area as a color corresponding to the value of the sensor information. On the other hand, the image conversion unit 13 writes white, which indicates the value "1", in the "HM4" area as a color corresponding to the value of the sensor information. In other words, the larger the written value, the closer the color becomes to white.
Furthermore, the inference model 14m is defined by associating the image data created by the image conversion unit 13 with the behavior "going home" of the observed person 2u, which indicates the situation represented by the image data.

By referring to the inference models 14m registered in the past, the behavior inference unit 14 uses the inference model when image data that matches or is similar to the image data of the inference model 14m is detected from the current observed person 2u. At 14 m, the corresponding action "go home" is output as the inference result (S104).
Note that it is preferable that a human such as the observer 3u act as a teacher and associate behavior labels such as "going home" and "rest" with the image data of the inference model 14m.

FIG. 8 is an explanatory diagram of image data as a result of writing actions different from those in FIG. 7 into the layout data 13L of FIG. 5. In the inference model 14m of FIG. 8, since the human sensors "HM1 to HM5" are all white (the observed person 2u is absent from each room), the behavior "going out" of the observed person 2u is associated with them.
Hereinafter, with reference to FIGS. 9 to 12, specific examples of the process of comparing the current behavior pattern and the normal behavior pattern (S111) and the process of notifying abnormalities in the current behavior pattern extracted through the comparison process (S112) Explain.

FIG. 9 is an explanatory diagram showing an example of normal behavior patterns stored in the database 17. The normal behavior pattern is a behavior pattern that connects the normal behavior of the observed person 2u, starting from "going to bed" at 5 a.m., "washing the bathroom" at 6 a.m., and ending at "going to bed" at 11 p.m.
FIG. 10 is an explanatory diagram showing an example of the current behavior pattern stored in the current behavior storage unit 16. The current behavior pattern is the behavior that connects the observed person 2u's current behavior (on the day of monitoring), starting from "going to bed" at 5:00 a.m., "meal" at 9:00 a.m., and ending with "going to bed" at 11:00 p.m. It's a pattern.

Here, the change detection unit 18 detects abnormalities in the current behavior pattern of the observed person 2u by the process of comparing the current behavior pattern in FIG. 9 and the normal behavior pattern in FIG. 10 for each behavior (S111). Extract behavior.
・I don't wake up until 9 o'clock and don't move around indoors.
・Meal times are different from usual and the meal times are long.
・I came home at an unexpected time (3:00 p.m.) while on my way out.
・The bathing times are different and the bathing time is long.

In this way, the change detection unit 18 detects abnormalities of the observed person 2u as exemplified below.
(1) The normal behavior defined in the normal behavior pattern was not currently being performed in the behavior pattern.
(2) An unnatural behavior that is not in the normal behavior pattern was performed in the current behavior pattern.
(3) The normal action defined in the normal action pattern was performed in the current action pattern, but the time was different from the time defined in the normal action pattern.
(4) The normal behavior defined in the normal behavior pattern was performed in the current behavior pattern, but the length was different from the length defined in the normal behavior pattern.
(5) The behavioral speed of the observed person 2u in the current behavioral pattern deviated from (was significantly lower than) the behavioral speed defined as the normal behavioral pattern.

Then, in S112, the change detection unit 18 notifies the observer 3u of an emergency signal via the observer terminal 3 in the event of a serious abnormality (1) or (2), thereby making arrangements for an ambulance, etc. Urge observer 3u to take urgent action.
On the other hand, the change detection unit 18 transmits an observation required signal to the observer 3u via the observer terminal 3 for the mild abnormalities (3) to (5), thereby detecting the illness of the observed person 2u. Notify the observer 3u of the sign. Signs of illness include, for example, a state on the verge of requiring nursing care (frailty) and mild cognitive impairment (MCI).

In particular, the mild abnormalities (3) to (5) are difficult to detect from single sensor information. On the other hand, since the image conversion unit 13 collectively converts a plurality of pieces of sensor information into an image, it is possible to extract the interval between actions, so detection becomes easy in this embodiment.
Note that the process of acquiring the behavior speed of the observed person 2u from the current behavior pattern in (5) may be performed using the following methods.
・Measure the speed of movement between the washroom and kitchen based on the operation interval of the washing machine and refrigerator 2a (range, IH).
- Measure reaction speed from changes in behavior regarding the door of the refrigerator 2a (opening/closing speed, opening/closing interval, opening/closing time, etc.).
・Measure the speed of movement from the interval between opening/closing the front door and turning on/off the lights.
-Measure the walking speed during cleaning from the acceleration of the vacuum cleaner 2b.

FIG. 11 is a screen diagram showing the display screen of the observer terminal 3 when the current behavior pattern is normal (S111, No).
The change detection unit 18 notifies the observer 3u of the healthy state of the observed person 2u by displaying a display screen including the following display contents on the observer terminal 3.
・Message column 221 showing an overview of the current behavior pattern
・Current behavior pattern time chart column 222
・Statistical data column 223 of this week's lifestyle pattern
・Statistical data column 224 for the difference between this week and last week
・Statistical data column 225 for differences with previous history

FIG. 12 is a screen diagram showing the display screen (S112) of the observer terminal 3 when the current behavior pattern is abnormal (S111, Yes).
In the floor plan column 211 at the center of the screen, the names of rooms such as bedrooms and living rooms for each room in the home 2h, and sensor information (temperature, humidity, illuminance) measured by the environmental sensor for each room are displayed.
Here, the estimated position 212 of the observed person 2u and a warning message 213 (emergency signal) indicating abnormal behavior of the observed person 2u are also displayed in the floor plan column 211. Thereby, the observer 3u can easily grasp the details of an abnormality of the observed person 2u, such as a person falling down.

In the present embodiment described above, the image conversion unit 13 converts many types of sensor information, such as sensor information of connected home appliances such as the refrigerator 2a and the vacuum cleaner 2b, and sensor information of the human sensor 2c, into one image. (Figure 7, Figure 8).
This makes it possible to detect large changes in the behavior of the observed person 2u, such as a fall, which could not be detected using single sensor information alone.

Here, this embodiment will be compared with a method of constructing a lifestyle behavior database such as that disclosed in Patent Document 1. In the conventional technology such as Patent Document 1, rules regarding which sensor information influences behavior and to what degree are manually inputted, which takes time and effort.
On the other hand, in the method of machine learning based on imaged sensor information according to this embodiment, the rules regarding which sensor information influences behavior and to what degree are automatically discovered by machine learning, so humans can manually learn the rules. It saves you the trouble of inputting data at work. Additionally, machine learning can automatically discover rules that humans are unaware of, improving the accuracy of behavior recognition.

Note that the present invention is not limited to the embodiments described above, and includes various modifications. For example, the embodiments described above are described in detail to explain the present invention in an easy-to-understand manner, and the present invention is not necessarily limited to having all the configurations described.
Furthermore, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment.
Further, it is possible to add, delete, or replace a part of the configuration of each embodiment with other configurations. Further, each of the above-mentioned configurations, functions, processing units, processing means, etc. may be partially or entirely realized in hardware by, for example, designing an integrated circuit.
Further, each of the configurations, functions, etc. described above may be realized by software by a processor interpreting and executing programs for realizing the respective functions.

Information such as programs, tables, and files that realize each function can be stored in memory, recording devices such as hard disks, SSDs (Solid State Drives), IC (Integrated Circuit) cards, SD cards, DVDs (Digital Versatile Discs), etc. can be stored on a recording medium.
Further, the control lines and information lines are shown to be necessary for explanation purposes, and not all control lines and information lines are necessarily shown in the product. In reality, almost all configurations may be considered to be interconnected.
Furthermore, the communication means for connecting each device is not limited to wireless LAN, but may be changed to wired LAN or other communication means.

2 Sensor 2u Observed person 3 Observer terminal 3u Observer 1 Behavior recognition server 11 Sensor information acquisition unit 12 Time information acquisition unit 13 Image conversion unit 13L Layout data 14 Behavior inference unit 14m Inference model 15 Current behavior identification unit 16 Current behavior accumulation Section 17 Database 18 Change detection section

Claims (7)

a sensor information acquisition unit that acquires sensor information indicating a detection result for each sensor from a set of sensors that detect an observed person;
In addition to the arbitrary range of values based on the values of each sensor information, the arbitrary range of values based on the time information of the sensor information can also be applied to the area of each sensor information in the image defined by the layout data. an image conversion unit that converts each sensor information into an image by writing;
By inputting the image of the sensor information created by the image conversion unit into an inference model in which correspondence data between the image of the sensor information and the behavior of the observed person is prepared in advance, A behavior recognition server comprising: a behavior inference unit that determines the behavior of the observed person to be output. The layout data defines the size of the area in the image according to the type of each sensor information,
The image conversion unit writes values in an arbitrary range based on the values of each sensor information according to the size of the area defined in the layout data.
The behavior recognition server according to claim 1 . The layout data includes, in descending order of the area in the image, time information of the sensor information, sensor information indicating the location of the observed person, sensor information indicating the movement of the observed person, and the observed person. The size of the area used as the sensor information for measuring the living environment of the person is defined.
The behavior recognition server according to claim 2 . a sensor information acquisition unit that acquires sensor information indicating a detection result for each sensor from a set of sensors that detect an observed person;
an image conversion unit that converts each sensor information into an image by writing a value in an arbitrary range based on the value of each sensor information into a region of each sensor information within an image defined by layout data;
By inputting the image of the sensor information created by the image conversion unit into an inference model in which correspondence data between the image of the sensor information and the behavior of the observed person is prepared in advance, a behavior inference unit that calculates the behavior of the observed person to be output ;
a current behavior identification unit that identifies a current behavior pattern in which the observed person's behaviors determined by the behavior inference unit are connected in chronological order;
A database that stores normal behavior patterns that connect normal behaviors in chronological order,
a change detection unit that detects the current behavior pattern that deviates from the normal behavior pattern as abnormal behavior of the observed person by comparing the current behavior pattern and the normal behavior pattern ,
The change detection unit acquires a behavioral speed of the observed person from an interval of behaviors included in the current behavioral pattern, and determines whether the acquired behavioral speed of the observed person deviates from the behavioral speed defined as the normal behavioral pattern. An action recognition server, characterized in that, when the observed person does so, an observation required signal indicating a slight abnormality of the observed person is notified . The change detection section includes:
If the normal behavior defined in the normal behavior pattern is not performed in the current behavior pattern, or if an action that is not in the normal behavior pattern is performed in the current behavior pattern, the observed person's Alerts an emergency signal indicating a serious abnormality,
If the normal action defined in the normal action pattern is also performed in the current action pattern, but at a different time from the time defined in the normal action pattern, or the normal action defined in the normal action pattern is performed in the current action pattern. If the length is different from the length defined in the normal behavior pattern, a signal requiring observation indicating a slight abnormality of the observed person is notified.
The behavior recognition server according to claim 4 . The behavior recognition server includes a sensor information acquisition unit, an image conversion unit, and a behavior inference unit,
The sensor information acquisition unit acquires sensor information indicating a detection result for each sensor from a set of sensors that detect the observed person,
The image converting unit converts the values in an arbitrary range based on the time information of the sensor information in addition to the values in an arbitrary range based on the values of the respective sensor information into the values in the image defined by the layout data. Converting each sensor information into an image by writing in the area of each sensor information,
The behavior inference unit inputs the image of the sensor information created by the image conversion unit to an inference model in which correspondence data between the image of the sensor information and the behavior of the observed person is prepared in advance. A behavior recognition method, characterized in that the behavior of the observed person output from the inference model is determined by: The behavior recognition server includes a sensor information acquisition unit, an image conversion unit, a behavior inference unit, a current behavior identification unit, a database, and a change detection unit ,
The sensor information acquisition unit acquires sensor information indicating a detection result for each sensor from a set of sensors that detect the observed person,
The image conversion unit converts each sensor information into an image by writing a value in an arbitrary range based on the value of each sensor information into a region of each sensor information in an image defined by layout data,
The behavior inference unit inputs the image of the sensor information created by the image conversion unit to an inference model in which correspondence data between the image of the sensor information and the behavior of the observed person is prepared in advance. determine the behavior of the observed person output from the inference model ,
The current behavior identification unit identifies a current behavior pattern that connects the observed person's behaviors determined by the behavior inference unit in chronological order,
The database stores normal behavior patterns in which normal behaviors are connected in chronological order.
The change detection unit detects the current behavior pattern that deviates from the normal behavior pattern as abnormal behavior of the observed person by comparing the current behavior pattern with the normal behavior pattern. The observed person's behavioral speed is obtained from the behavioral interval included in the behavioral pattern, and when the obtained behavioral speed of the observed person deviates from the behavioral speed defined as the normal behavioral pattern, the observed person's behavioral speed is An action recognition method characterized by notifying a signal requiring observation indicating a slight abnormality in a person .

Images