JP2020181257A - Method and system for generating teacher data of learning model estimating mental and physical state of person watched over - Google Patents

Abstract

To provide a method for generating teacher data of a learning model that estimates a mental and physical state of a person who is watched over.SOLUTION: A method includes the steps of: acquiring operation logs from a plurality of IoT apparatuses 40 installed in a living space of a person who is watched over; extracting a plurality of feature quantities from the acquired operation logs; generating a feature tensor having the extracted plurality of feature quantities as elements; controlling a light emission mode of corresponding light-emitting elements on the basis of values of the feature quantities extracted from the operation logs in a light-emitting display device including the plurality of light-emitting elements associated with the respective feature quantities; and labeling the feature tensor as abnormal or normal on the basis of a judgement by a watcher who has observed the light-emitting display device whose light emission is controlled.SELECTED DRAWING: Figure 5

Description

The present invention relates to a method of generating teacher data, and more particularly to a method of generating teacher data necessary for generating a learning model for estimating the mental and physical state of a watching subject.

In Japan, which has entered a super-aging society, the proportion of elderly people living alone is increasing year by year as the nuclear family grows. In response to this social situation, various monitoring systems for watching over the elderly have been proposed in recent years.

Most of the conventional monitoring systems try to detect an abnormality by comparing the output of a motion sensor installed in the living space of an elderly person or the usage status of home appliances with a predetermined rule (for example, Patent Document 1, 2) In reality, it is extremely difficult to create rules that are practical, and the current situation is that satisfactory results have not been obtained, such as many false positives and oversights.

On the other hand, families living apart from the elderly are constantly concerned about their physical and mental condition and want to be aware of changes in their physical and mental condition as soon as possible.

However, although the conventional rule-based approach can detect obvious abnormal conditions such as "falling down in the bathroom" or "not returning while going out", it can detect changes in the physical and mental condition of the elderly. It could not be detected.

JP-A-2017-116994 Japanese Unexamined Patent Publication No. 2014-222298

In view of the above problems, the present inventor got the idea of constructing a watching system for estimating the mental and physical condition of the elderly by a machine learning approach, but in the process of embodying this idea, a learning model was generated. I ran into the question of how to generate the teacher data needed to do this. The present invention has been made to solve this problem, and an object of the present invention is to provide a method for generating teacher data of a learning model for estimating the mental and physical condition of a watching subject represented by an elderly person living alone. ..

In generating teacher data for a learning model that estimates the physical and mental state of the person being watched over, firstly, what is the input data is an issue, and secondly, who and how to label the input data. The problem is whether to attach (abnormal / normal).

Regarding the first problem, the present inventor, based on the knowledge that the mental and physical condition of the consumer is reflected in the lifestyle pattern of the consumer, obtained through many years of research on the improvement of the quality of life of the consumer, is the subject of the watch. I got the idea of using the features of life behavior patterns as input data.

Regarding the second problem, the present inventor pays attention to the excellent visual ability of human beings who sense the change of light with high sensitivity, expresses the feature amount of the life behavior pattern of the person to be watched by light, and the person to watch over it. I got the idea of presenting it to a watcher who has an attachment to the watch and letting the watcher judge the physical and mental condition of the watcher.

As a result of diligent studies based on the above-mentioned idea, the present inventor came up with the following configuration and arrived at the present invention.

That is, according to the present invention, it is a method of generating teacher data of a learning model for estimating the mental and physical state of the person being watched over, and acquiring operation logs from a plurality of IoT devices installed in the living space of the person being watched over. It is associated with each of a step, a step of extracting a plurality of feature quantities from the acquired operation log, a step of generating a feature tensor having the extracted plurality of feature quantities as elements, and a plurality of feature quantities. The step of controlling the light emission of the light emitting display device composed of a plurality of light emitting elements, the step of controlling the light emitting mode of the corresponding light emitting element based on the value of the feature amount extracted from the operation log, and the light emitting A method is provided that includes a step of labeling the feature tensor as abnormal or normal based on a determination by a watcher who has observed the controlled light emission display device.

As described above, according to the present invention, there is provided a method of generating teacher data of a learning model for estimating the mental and physical state of a watching subject.

The figure which shows the structure of the monitoring system of this embodiment. The figure which shows the light emission display device of this embodiment. The conceptual diagram for demonstrating the light emission display device of this embodiment. The figure which shows the functional structure of the monitoring system of this embodiment. The sequence diagram of the process executed by the monitoring system of this embodiment. The figure which shows the table used by the monitoring system of this embodiment. The figure which shows the table used by the monitoring system of this embodiment. The sequence diagram of the process executed by the monitoring system of this embodiment. The figure which shows the UI screen displayed on the watcher's PC. The conceptual diagram for demonstrating the correspondence between a light emitting element and a feature amount.

Hereinafter, the present invention will be described with reference to the embodiments shown in the drawings, but the present invention is not limited to the embodiments shown in the drawings. In each of the figures referred to below, the same reference numerals are used for common elements, and the description thereof will be omitted as appropriate.

FIG. 1 shows the system configuration of the monitoring system 1000 according to the embodiment of the present invention. The monitoring system 1000 of the present embodiment includes a centralized management server 100 that manages the entire system, a plurality of IoT devices 40, a personal computer 200, and a light emitting display device 300.

The plurality of IoT devices 40 are electronic devices installed in the living space 400 of the person to be watched (for example, an elderly person living alone), and each IoT device 40 is a centralized management server via a wireless access point 42 and a network 50. It is possible to communicate with 100.

In the present embodiment, the "IoT device" means all network-compatible electronic devices capable of performing data communication via a network such as the Internet. Here, examples of the IoT device 40 include so-called "smart home appliances" and various state sensors, and examples of the smart home appliances include refrigerators, televisions, air conditioners, washing machines, indoor lighting fixtures, microwave ovens, and HI. Cooking heaters, rice cookers, electric pots, etc. can be exemplified, and the status sensors include open / close sensors installed on doors and windows, human sensor installed in each room, illuminance sensor, temperature / humidity sensor, and odor sensor. Etc. can be exemplified.

The light emitting display device 300 is a device that expresses the life behavior pattern of the person being watched over by light. As shown enlarged in FIG. 2, the light emitting display device 300 includes a square support base 32, four support frames 34a, 34b, 34c, 34d, 80 light emitting elements 30, and a cubic transparent cover. Four support frames 34a, 34b, 34c, 34d, each of which is composed of 36 and has 20 light emitting elements 30 arranged in a two-dimensional matrix, are erected on the support base 32 at equal intervals. As a result, the structure is such that 80 light emitting elements 30 are arranged in a three-dimensional matrix. In the present embodiment, as the light emitting element 30, a full-color LED capable of controlling the light emitting intensity and the light emitting color is adopted, and hereinafter, the light emitting element 30 is referred to as an LED element 30.

In the present embodiment, unique addresses are assigned to each of the 80 LED elements 30 constituting the light emitting display device 300. Specifically, as shown in FIG. 3, addresses a1 to a20 are assigned to the 20 LED elements 30 arranged on the support frame 34a, and 20 LED elements arranged on the support frame 34b. Addresses b1 to b20 are assigned to 30 and addresses c1 to c20 are assigned to 20 LED elements 30 arranged on the support frame 34c, and 20 LED elements are assigned to the support frame 34d. Addresses d1 to d20 are assigned to 30 so that the address of each LED element 30 can be specified and the light emission intensity, light emission color, and light emission timing thereof can be individually controlled.

The personal computer 200 (hereinafter referred to as a user PC 200) is a computer used by a watcher (for example, a family living away from an elderly person living alone) who watches over the person to be watched, and is connected to the centralized management server 100 via the network 50. Mutual communication is possible. In the present embodiment, the user PC 200 is connected to the light emitting display device 300 via wire or wirelessly, controls the light emission of the light emitting display device 300, and inputs the determination result of the mental and physical condition of the person to be watched by the watcher. Takes the role of accepting. In FIG. 1, a notebook PC is illustrated as the user PC 200, but the user PC 200 is not limited to this, and may be a desktop PC, a tablet PC, or a smartphone.

The system configuration of the monitoring system 1000 of the present embodiment has been described above, but subsequently, the functional configurations of the user PC 200 and the centralized management server 100 will be described based on the functional block diagram shown in FIG.

In the present embodiment, a dedicated application for using this system is pre-installed in the user PC 200, and the user PC 200 functions as a UI unit 202 and a light emission control unit 203 by executing the dedicated application.

The UI unit 202 is a means for displaying a predetermined UI screen on the display of the user PC 200 and receiving an operation input from a watcher via the UI screen.

The light emission control unit 203 is a means for controlling the light emission of the light emission display device 300 based on the light emission control information (described later) acquired from the centralized management server 100.

On the other hand, the centralized management server 100 includes an operation log collection unit 101, a feature amount extraction unit 102, a feature vector generation unit 103, a light emission control information generation unit 104, a light emission control information provision unit 105, and a teacher data generation unit 106. , A machine learning unit 107, and a watching execution unit 108.

The operation log collecting unit 101 is a means for periodically collecting operation logs from a plurality of IoT devices 40 installed in the living space 400 of the person to be watched over.

The feature amount extraction unit 102 is a means for extracting a plurality of feature amounts from the operation log of each device collected by the operation log collection unit 101. The feature amount extraction unit 102 includes an appropriate inference engine, and extracts a predetermined feature amount from the operation log by using the knowledge base stored in the storage area 109.

The feature vector generation unit 103 is a means for generating a feature vector having a plurality of feature quantities extracted by the feature quantity extraction unit 102 as elements.

The light emission control information generation unit 104 is a means for generating light emission control information necessary for controlling light emission of the light emission display device 300.

The light emission control information providing unit 105 is a means for providing light emission control information generated by the light emission control information generation unit 104 in response to a request from the user PC 200.

The teacher data generation unit 106 is a means for generating teacher data of a learning model that estimates the mental and physical state of the person being watched over by labeling the feature vector generated by the feature vector generation unit 103.

The machine learning unit 107 is a means for generating a learning model that estimates the mental and physical state of the watching target person using the teacher data generated by the teacher data generation unit 106.

The watching execution unit 108 is a means for estimating the mental and physical state of the watching target person using the learning model generated by the machine learning unit 107 and giving necessary notification to the watching person.

In the present embodiment, the computer constituting the centralized management server 100 functions as each of the above-mentioned means by executing a predetermined program.

The functional configurations of the user PC 200 and the centralized management server 100 have been described above. Next, the contents of the processing executed by the monitoring system 1000 in the "teacher data generation mode" will be described. In the following, a case where 20 IoT devices 40 are installed in the living space 400 of the person to be watched over will be described as an example.

First, the contents of the processes continuously executed by the monitoring system 1000 of the present embodiment on a daily basis will be described with reference to the sequence diagram shown in FIG.

In the present embodiment, the operation log collecting unit 101 collects operation logs from 20 IoT devices 40 installed in the living space 400 of the person to be watched at a predetermined timing (S1). For example, the operation log collection unit 101 collects operation logs for one day from each IoT device 40 at the end of the day (24:00), and saves the collected 20 operation logs in the storage area 109. To do. The operation log referred to here includes, for example, the opening / closing time of the smart refrigerator door, the power on / off time of the smart TV, the on / off time of the toilet lighting, the opening / closing time of the entrance door, the detection values of various status sensors, and the like. Can be exemplified.

After collecting the operation logs of the 20 IoT devices 40, the feature amount extraction unit 102 extracts a plurality of feature amounts from the collected 20 operation logs based on a predetermined algorithm (S2). Here, what kind of feature amount is extracted from the operation log can be arbitrarily designed, and the present invention does not limit the content of the feature amount, but here, for convenience of explanation, the following The description will be given assuming that the features described in the above are extracted.

That is, in the present embodiment, the day is divided into four time zones (0-6, 6-12, 12-18, 18-24) every 6 hours, and the operation log (of the door) of each time zone of the smart refrigerator. The number of times the refrigerator is used in each time zone is extracted as a feature amount from the opening / closing time), and the number of times the TV viewing time in each time zone is used is characterized from the operation log (power on / off time) of each time zone of the smart TV. Each IoT device 40 is extracted as an amount, and the number of times the entrance door is opened / closed in each time zone is extracted as a feature quantity from the operation log (door opening / closing time) of the sensor provided on the entrance door in each time zone. Extract one feature amount corresponding to the time zone from the operation log of. In this case, a total of 80 features will be extracted from the operation logs of 20 IoT devices 40 for one day.

The feature amount extraction unit 102 stores the feature amount value extracted in S2 in the table 500 stored in the storage area 109. FIG. 6 shows the table 500. As shown in FIG. 6, in the table 500, 80 feature quantities extracted from the operation log for one day by the feature quantity extraction unit 102 and 80 LED elements 30 constituting the light emitting display device 300 are one-to-one. It is a table to be associated, and the column 501 for storing the ID of the feature amount, the column 502 for storing the contents of the feature amount, the column 503 for storing the value of the feature amount, and the LED element 30 associated with the feature amount. It has a column 504 for storing an address, a column 505 for storing the emission color of the LED element 30, and a column 506 for storing the emission intensity of the LED element 30. The feature amount extraction unit 102 stores the values of the 80 feature amounts extracted in S2 in the corresponding fields of the column 503.

Here, in the present embodiment, a unique emission color is assigned in advance to each LED element 30 constituting the emission display device 300, and the emission color of each LED element 30 is designated in each field of the column 505. RGB information for the purpose is stored in advance. Further, in the present embodiment, the light emission intensity of the LED element 30 can be controlled in four levels, and level information for designating the light emission intensity level of each LED element 30 in each field of the column 506. (0,1,2,3) is stored.

The explanation will be continued by returning to FIG. 5 again.

When the feature amount extraction is completed, the feature vector generation unit 103 subsequently generates a feature vector having the extracted feature amount as an element (S3). Specifically, 80 feature quantities stored in column 503 of the table 500 are read out, and 80-dimensional vectors (X ₁ , X ₂ , X ₃ , ... X ₈₀ ) having these as elements are generated.

Here, the feature vectors (X ₁ , X ₂ , X ₃ , ... X ₈₀ ) whose elements are the feature quantities extracted from the operation log of the IoT device 40 installed in the living space 400 of the watch target person are the watch targets. It can be seen that it represents the characteristics of a person's life behavior pattern, and in light of the knowledge that the mental and physical condition of a person is reflected in that person's life behavior pattern, after all, the feature vector generated by S3. Can be seen as representing the characteristics of the physical and mental condition of the person being watched over.

The feature vector generation unit 103 stores the feature vector generated in S3 in the table 600 stored in the storage area 109. FIG. 7 shows the table 600. As shown in FIG. 7, the table 600 includes a column 601 for storing the date when the operation log is extracted, a column 602 for storing the generated feature vector, and a column 603 for storing the light emission control information. The feature vector generation unit 103 uses the feature vector (X ₁ , X ₂ , X ₃ , ... X ₈₀ ) generated in S3 as a column 602 corresponding to the date when the operation log from which the feature vector is collected is collected. Store in the field of.

When the generation of the feature vector is completed, the light emission control information generation unit 104 subsequently generates light emission control information based on the feature amount extracted in S2 (S4). Here, the "light emission control information" means a data set for designating the light emission mode of each LED element 30 constituting the light emission display device 300, and the light emission control information generation unit 104 emits light by the following procedure. Generate control information.

First, based on the value of each feature amount extracted in S2, the level of the light emission intensity of the LED element 30 is determined according to a predetermined rule, and the level information (0, 1, 2, 3) for designating the determined level. Is stored in the field of column 506 corresponding to the feature quantity of interest in table 500. The present invention does not particularly limit the rule for determining the level of emission intensity, but here, the emission intensity is based on the rule that the level of emission intensity is gradually increased as the value of the feature amount increases. Shall determine the level of.

In response to the fact that 80 level information is stored in the column 506 of the table 500, the light emission control information generation unit 104 receives the address of the LED element 30, the light emission color (RGB information), and the light emission intensity from each record of the table 500. Three pieces of information (level information) are extracted, and 80 sets of data sets including these three pieces of information are generated as light emission control information.

Finally, the light emission control information (or its storage destination path) generated in S4 is stored in the field of column 603 of the table 600 corresponding to the date when the operation log was collected in S1. As a result, the date when the operation log is collected, the feature vector whose elements are the feature amount extracted from the operation log, and the light emission control information generated based on the feature amount extracted from the operation log are stored in association with each other. Is done (S5).

In the present embodiment, the series of processes described above is executed every day, and the table 600 is updated each time.

Subsequently, the content of the processing executed when the monitoring system 1000 of the present embodiment generates the teacher data will be described with reference to the sequence diagram shown in FIG.

When the watcher operates his / her own user PC200 to start a dedicated application for using this system (S1), in response to this, the UI unit 202 displays the observation date designation screen shown in FIG. 9A. The 700 is displayed on the display (S2). At this time, a calendar for designating the date is displayed on the observation date designation screen 700, and the watcher specifies the date to be observed on the displayed calendar and clicks the OK button (S3).

In response to the click of the OK button, the UI unit 202 requests the centralized management server 100 for light emission control information corresponding to the date specified by the watcher (S4). In response to this, the light emission control information providing unit 105 of the centralized management server 100 reads the light emission control information associated with the date specified by the watcher from the table 600 (S5), and transmits the read light emission control information to the user PC 200. (S6).

In response to this, the light emission control unit 203 of the user PC 200 controls the light emission mode of the 80 LED elements 30 constituting the light emission display device 300 based on the light emission control information received from the centralized management server 100 (S7). Specifically, the RGB information and the level information associated with the address of each LED element 30 are read from the received light emission control information, and the light emission of the LED element 30 at the address is based on the read RGB information and the level information. Individually control color and emission intensity.

At this time, the lump of light presented by the light emitting display device 300 visualizes the mental and physical condition of the person to be watched on the day specified by the watcher, and the watcher observes this lump of light and gives an impression from it. Based on this, the physical and mental condition of the person being watched over is determined.

When the light emission control unit 203 finishes the light emission control of the light emission display device 300, the UI unit 202 displays the determination result input screen 800 shown in FIG. 9B on the display (S8). At this time, the answer button (yes / no) is displayed on the determination result input screen 800 together with the message "Did you feel something abnormal?". The watcher clicks the "Yes" button when he / she feels something abnormal from the mass of light presented by the light emitting display device 300, and clicks the "No" button when he / she feels nothing in particular. The determination result is input (S9).

What is expected at this time is the intuition of the watcher who has an attachment to the watcher. In general, it is empirically known that a loved one can intuitively perceive potential changes that cannot be noticed by others in red from the casual expressions and gestures of a loved one. The present inventor considers that the same intuition can be exhibited by a watcher who observes a mass of light expressing the physical and mental state of the watcher.

In response to the click of the answer button on the determination result input screen 800, the UI unit 202 transmits the determination result input by the watcher to the centralized management server 100 (S10). Specifically, when the "Yes" button is clicked, the date of the observation date specified by the watcher and the judgment result including the "abnormal" flag are sent to the centralized management server 100, and the "No" button is clicked. If this is the case, the determination result including the date of the observation date specified by the watcher and the "normal" flag is transmitted to the centralized management server 100.

Upon receiving the determination result from the user PC 200, the teacher data generation unit 106 of the centralized management server 100 obtains a feature vector linked to the date of the observation date included in the received determination result from the table 600 shown in FIG. Read and label the read feature vector. Specifically, when the received determination result includes the "abnormal" flag, the read feature vector is labeled as "abnormal" (S11), and the received determination result is "normal". If the flag is included, the read feature vector is labeled as "normal" (S12).

The teacher data generation unit 106 stores in the storage area 109 the pair of the feature vector and the label obtained in the above procedure as the teacher data of the learning model for estimating the mental and physical state of the watching target person (S13).

In the present embodiment, the above-mentioned series of processes is executed in response to the request of the watcher, and one new teacher data is accumulated in the storage area 109 each time.

The "teacher data generation mode" of the watching system 1000 of the present embodiment has been described above, but subsequently, the "learning mode" and the "watching execution mode" of the watching system 1000 will be described.

In the "learning mode", the machine learning unit 107 of the centralized management server 100 monitors the physical and mental state of the subject by performing supervised learning using a plurality of teacher data accumulated in the storage area 109 through the above procedure. A learning model to be estimated is generated, and the generated learning model is stored in the storage area 109.

On the other hand, in the "watching execution mode", the centralized management server 100 first executes the processes S1 to S3 described with reference to FIG. That is, the operation log collecting unit 101 collects the operation log of each device from the 20 IoT devices 40 installed in the living space 400 of the person to be watched at a predetermined timing (S1), and the feature amount extracting unit 102 , 80 feature quantities are extracted from the collected 20 operation logs (S2), and the feature vector generation unit 103 generates an 80-dimensional feature vector having the extracted 80 feature quantities as elements (S3).

Subsequently, the watching execution unit 108 estimates the mental and physical state of the watching target person using the learning model stored in the storage area 109. Specifically, the 80-dimensional feature vector generated by the feature vector generation unit 103 is given as an input to the learning model, and the mental and physical state (abnormal / normal) of the watch target person is obtained as an output. As a result, when an "abnormality" in the mental and physical condition of the person to be watched is presumed, the watching execution unit 108 notifies the user PC 200 to that effect by an appropriate method such as e-mail.

As described above, according to the present embodiment, it is possible to generate the teacher data necessary for generating the learning model for estimating the mental and physical state of the person to be watched over, and the learning model in which the teacher data is learned can be generated. By using it, it is possible to estimate the physical and mental condition of the person being watched over.

In the case of elderly people living together, the family can intuitively feel small changes such as feeling unwell or feeling sick from their facial expressions, gestures, and language, so use gentle words to comfort them. While appropriate measures such as taking them to a hospital or taking them to a hospital can be taken, elderly people living far away may inevitably be late in noticing the change, and when they do, it may be too late. In this regard, according to the present embodiment, since the family can remotely detect a small change in the mental and physical condition of the elderly living apart, it is possible to take appropriate measures before it becomes important.

Although the present invention has been described above, the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention. This will be specifically described below.

In the present invention, the feature amount can be assigned to any light emitting element. For example, in the above-described embodiment, as shown in FIG. 10A, the feature quantities extracted from the operation logs of the four time zones (0-6, 6-12, 12-18, 18-24) are assigned. The light emitting elements are arranged in the Z direction in chronological order, but in another embodiment, as shown in FIG. 10B, four time zones (0-6, 6-12, 12-18) are shown. , 18-24) The light emitting elements to which the features assigned are assigned are arranged in the X direction in chronological order, and are extracted from the operation logs of the four rooms (living room, kitchen, guest room, bedroom). It is also possible to allocate the light emitting elements to which the feature amounts are assigned so as to be arranged in the Z direction together for each room. Needless to say, various variations are possible in the method of assigning the feature amount.

In the above-described embodiment, one feature amount is assigned to one light emitting element, but in another embodiment, one feature amount may be assigned to two or more light emitting elements. However, two or more feature quantities may be assigned to one light emitting element. As a case where two or more features are assigned to one light emitting element, from the operation logs of four time zones (0-6, 6-12, 12-18, 18-24) for one light emitting element. An example can be illustrated of a case in which the four extracted features are assigned. In this case, it is necessary to assign each period (for example, 15 seconds) obtained by dividing a predetermined light emission period (for example, 1 minute) into four to the feature amount of each time zone. In that case, the above-mentioned light emission control information includes In addition to the emission color and emission intensity of each light emitting element, information for designating the emission timing of each light emitting element is included.

In the above-described embodiment, a light emitting display device in which a plurality of light emitting elements are arranged in a three-dimensional matrix is shown, but the structure of the light emitting display device is not limited to this, and a plurality of light emitting elements can be arranged in any embodiment ( It may be arranged in two dimensions or three dimensions).

In the above-described embodiment, an example in which a full-color LED is adopted as the light emitting element constituting the light emitting display device is shown, but the light emitting element constituting the light emitting display device is not limited to this, and at least one of the light emitting intensity and the light emitting color can be used. Any light emitting element that can be controlled may be used.

In the above-described embodiment, an example in which the emission intensity of the corresponding light emitting element is changed according to the magnitude of the feature amount value is shown, but in another embodiment, the emission intensity is changed according to the magnitude of the feature amount value. The emission color of the corresponding light emitting element may be changed.

In another embodiment, the above-mentioned light emitting display device can be replaced with an arbitrary display device, in which case each of the plurality of pixel regions defined on the display screen of the display device is used as the above-mentioned light emitting element. It may be configured to work. Further, in that case, if the display of the watcher's PC is made to function as a light emitting display device, it is not necessary to additionally prepare a light emitting display device.

In the above-described embodiment, an example of generating a feature vector using the same feature amount as the feature amount used for generating the light emission control information is shown, but in another embodiment, N pieces extracted from the operation log for one day are shown. While generating light emission control information based on the first feature amount of, M-dimensional feature vector using M second feature amounts (feature amounts different from the first feature amount) extracted from the same operation log. May be generated. In this case, if the second feature amount is designed so that M <N, the number of dimensions of the data to be learned becomes small, so that learning efficiency can be expected. As such an example, while extracting the above-mentioned 80 features as the first feature from the operation log for one day, five rooms (living room, guest room, bedroom, kitchen, bath, etc.) are extracted from the same operation log. It is possible to estimate the staying time of the toilet) and extract five estimated values as the second feature amount.

In the above-described embodiment, an example in which the characteristics of the living behavior pattern of the watching target person are expressed by a vector (tensor on the first floor) is shown, but in another embodiment, the characteristics of the living behavior pattern of the watching target person are expressed on the Nth floor. It may be expressed by a tensor (N is an integer of 2 or more). In this case, the description of the above-described embodiment should be understood by replacing "feature vector" with "feature tensor".

In the above-described embodiment, the description has been made assuming an elderly person living alone as a person to be watched over, but the present invention does not particularly limit the attributes of the person to be watched over, and the present invention assumes it. The people to be watched over include, for example, young children who are away from home and pets (dogs and cats) that are kept free in the room.

In addition to the above, it is included in the scope of the present invention as long as the action and effect of the present invention are exhibited within the scope of embodiments that can be conceived by those skilled in the art.

30 ... light emitting element (LED element), 32 ... support base, 34a ... support frame, 34 ... support frame, 36 ... transparent cover, 40 ... IoT device, 42 ... wireless access point, 50 ... network, 100 ... centralized management server, 101 ... Operation log collection unit, 102 ... Feature amount extraction unit 103 ... Feature vector generation unit, 104 ... Light emission control information generation unit, 105 ... Light emission control information provision unit, 106 ... Teacher data generation unit, 107 ... Machine learning unit, 108 ... Watching execution unit, 109 ... Storage area, 200 ... User PC (personal computer), 202 ... UI unit, 203 ... Light emission control unit, 300 ... Light emission display device, 400 ... Living space, 500 ... Table, 501,502, 503, 504, 505, 506 ... Column, 600 ... Table, 601, 602, 603 ... Column, 700 ... Observation date designation screen, 800 ... Judgment result input screen, 1000 ... System

Claims (6)

It is a method of generating teacher data of a learning model that estimates the mental and physical condition of the person being watched over.
Steps to acquire operation logs from multiple electronic devices installed in the living space of the person being watched over,
A step of extracting a plurality of features from the acquired operation log, and
Steps to generate a feature tensor with multiple extracted features as elements,
It is a step of controlling the light emission of a light emitting display device composed of a plurality of light emitting elements associated with each of the plurality of feature amounts, and corresponds to the corresponding light emission based on the value of the feature amount extracted from the operation log. Steps to control the light emission mode of the element,
A method including a step of labeling the feature tensor as abnormal or normal based on a determination by a watcher who observes the light emission display device whose light emission is controlled. The step of extracting the plurality of features is
It includes a step of extracting a plurality of first feature amounts from the acquired operation log and a step of extracting a plurality of second feature amounts smaller than the number of the first feature amounts from the operation log.
The step of generating the feature tensor is
This is a step of generating a feature tensor having a plurality of extracted second feature quantities as elements.
The step of controlling the light emission mode is
This is a step of controlling the light emitting mode of the corresponding light emitting element based on the extracted value of the first feature amount.
The method according to claim 1. The step of controlling the light emission mode is
A step of controlling at least one of the emission intensity and the emission color of the light emitting element is included.
The method according to claim 1 or 2. The step of controlling the light emission mode is
A step of controlling the light emission timing of the light emitting element is included.
The method according to claim 3. The plurality of light emitting elements constituting the light emission display device are three-dimensionally arranged.
The method according to any one of claims 1 to 4. It is a system that generates teacher data of a learning model that estimates the mental and physical condition of the person being watched over.
A means of acquiring operation logs from multiple electronic devices installed in the living space of the person being watched over,
A means for extracting a plurality of features from the acquired operation log, and
A means for generating a feature tensor whose elements are a plurality of extracted features,
It is a means for controlling the light emission of a light emitting display device composed of a plurality of light emitting elements associated with each of the plurality of feature amounts, and corresponds to the corresponding light emission based on the value of the feature amount extracted from the operation log. Means for controlling the light emitting mode of the element and
A system including means for labeling the feature tensor as abnormal or normal based on a determination by a watcher who observes the light emission display device whose light emission is controlled.