WO2020196636A1

WO2020196636A1 - Storage device for house

Info

Publication number: WO2020196636A1
Application number: PCT/JP2020/013369
Authority: WO
Inventors: 信孝井出
Original assignee: 株式会社ワコム
Priority date: 2019-03-26
Filing date: 2020-03-25
Publication date: 2020-10-01

Abstract

The present invention implements a storage device for a house, which detects a change in the house and can reproduce the change so as not to excessively interfere with the context of the change and so as to impart, to members of the house, a sense of belonging to the house. The storage device for a house: accumulates, as reminiscent handwriting data, handwriting data obtained from autograph performed in the house of family residence in association with context data indicating autograph context; detects a change in a physical quantity pertaining to light, sound, or air in the house, or the handwriting, or a change in an estimation quantity pertaining to feelings of family members; randomly selects, from among a plurality of sets of the accumulated context data, weak correlation context data having relatively small relevance with the change by taking, as an opportunity, the detection of the change; and instructs a playback device provided in the house to project one piece of the reminiscent handwriting data associated with the weak correlation context data.

Description

Home storage

The present invention detects a change in the house, does not over-interfere with the context of the change, and gives the members of the house a sense of belonging to the house. Regarding the device.

The digital ink generated from the electronic pen is data used to reproduce the trajectory of the electronic pen, just as a conventional pen leaves a handwriting on paper. As digital ink, Patent Document 1 discloses an example of an object-oriented digital ink as a data model inside a computer, and Patent Document 2 discloses an example of a serialized format of digital ink.

Furthermore, digital ink data is known that makes it possible to record "when, who, where, under what circumstances" as a trajectory of human behavior, beyond the framework of data for reproducing mere handwriting in digital ink. ing. For example, Patent Document 3 contains digital ink as digital ink that makes it possible to identify who wrote stroke data indicating a locus, and Patent Document 4 contains author as context data when stroke data is input. Digital ink that acquires information such as pen ID, time information, and regional information acquired by GPS (Global Positioning System) and makes it possible to record them as metadata is disclosed.

In recent years, the use of digital ink has been considered to support the estimation of "what kind of thoughts or feelings you wrote". For example, Patent Document 5 discloses a method and an apparatus for grasping and determining the mental, psychological, and physiological states of a writer by quantifying the spatiotemporal information of handwritten characters and pictures and extracting feature quantities. .. Patent Document 6 discloses an emotion estimation system that associates a writing state such as writing pressure with biological information corresponding to an emotion and derives biological information corresponding to the emotion only from the writing state.

Such digital ink that indicates "when, where, and what kind of handwriting was done" can be said to be the history data of the human trajectory, that is, the behavior or emotion. By combining such digital ink with personalized AI (Artificial Intelligence), it is expected that a more advanced life will be realized.

For example, Patent Document 7 describes a system using AI used in a house. In particular, a smart home system that proposes or executes a house policy according to prior criteria based on the results (data before and after changes and changes) sensed using sensors in the home environment is disclosed. ..

AI is learned in the direction of minimizing the error (loss function value, error) between the past data and the predicted data based on the learning data obtained in the past. Therefore, there is a high probability that the proposal made by AI in the house will be compatible with the environment of the house, the data before and after making the proposal, or the context data (context) which is the environmental data.

On the other hand, learning using AI is known to have the problem of overfitting, in which learning that is locally correct and does not reach a global solution or that overfits environmental data. We believe that this problem of overfitting may apply to the behavioral patterns of people who follow AI's suggestions. Continuing to adopt proposals that are overly adapted to the context of behavior at home may, in the big picture, limit the choices of those who make up the home. Think.

In other words, traditional AI, which pursues optimization of rationality and convenience, does not develop the creativity of people created by contingency or irrationality, but the lives and societies of human beings or those who make up their families. There is a possibility that it will be suppressed to a tasteless and dry product centered on minimizing errors (Non-Patent Document 1). In order to solve them, there are several known methods of applying a genetic algorithm incorporating mutations to the learning of neural network-based AI (Non-Patent Document 2, etc.).

U.S. Pat. No. 07158675 U.S. Pat. No. 07397949 Japanese Patent No. 5886487 U.S. Patent Application Publication No. 2016/0224239 Japanese Unexamined Patent Publication No. 2010-131280 International Publication No. 2018/043061 Pamphlet U.S. Patent Application Publication No. 2016/0259308

However, while bringing in processing using random numbers may be useful in the learning process of AI, if AI produces random output at the practical stage, it will not be familiar to the home environment at all and will be visually or audibly. It becomes noise and cannot withstand long-term use in a home environment.

The first storage device of the house according to the present invention is a storage unit that stores handwritten handwriting data made in a house, which is a family residence, as recollection handwriting data in association with context data indicating the handwritten context. A sensor that detects changes in physical quantities related to light, sound, air, or handwriting in the house, or changes in estimated quantities related to the emotions of members of the family, and that the sensors have detected the changes. An analysis unit that randomly selects weakly correlated context data having a relatively small relevance to the change from a plurality of sets of context data accumulated by the storage unit, and no analysis unit. A control unit that reads one recollection stroke data associated with the weakly correlated context data randomly selected from the storage unit and instructs a reproduction device provided in the house to project the recollection stroke data. To be equipped with.

The second home storage device of the present invention stores handwritten handwriting data made in the house, which is the residence of the family, as recollection handwriting data in association with the event type for classifying the handwriting context. The storage unit, a sensor that detects changes in physical quantities related to light, sound, air, or handwriting in the house, or changes in estimated quantities related to the emotions of members of the family, and the sensors detect the changes. Taking this opportunity, an analysis unit that randomly selects a weakly correlated event type that has a relatively small relevance to the change from a plurality of event types accumulated by the storage unit, and the analysis unit. A control unit that reads one recollection handwriting data associated with a weakly correlated event type randomly selected by the method from the storage unit and instructs a reproduction device provided in the house to project the recollection handwriting data. And.

It is a device used in the home environment, and because it outputs out of the context of changes in the house, it does not excessively interfere with human creativity and judgment, and handwriting or sounds recorded in the house in the past, etc. It is expected to provide a sense of belonging to the environment of the house because it outputs.

It is a schematic diagram of the storage device of a house in one Embodiment of this invention. It is a flowchart about the operation of the system shown in FIG. It is a figure which shows an example of the handwriting detected by the handwriting sensor group. It is a figure which shows an example of the event data and context data which are accumulated. It is a figure which shows an example of the data structure which a context data has. It is the first detailed flowchart about the selection of contextual data. It is a figure which shows an example of the classification method of a feature space. It is a second detailed flowchart regarding the selection of contextual data. It is a figure which shows an example of the structure of the learner for acquiring the feature vector about an event type.

The home storage device in the present invention will be described with reference to the attached drawings. In order to facilitate understanding of the description, the same components and steps may be designated as the same reference numerals as possible in each drawing, and duplicate description may be omitted. It should be noted that the present invention is not limited to the embodiments shown below, and of course, the present invention can be freely changed without departing from the gist of the present invention. Alternatively, each configuration may be arbitrarily combined as long as there is no technical contradiction.

<System configuration and operation>
FIG. 1 is a schematic view of a home storage device according to an embodiment of the present invention. FIG. 2 is a flowchart relating to the operation of the storage device of the house shown in FIG. This device is realized by a sensing system 1, a control device 2, a reproduction device 3, and a network 4. The sensing system 1 and the reproduction device 3 are provided in the house. Here, the "house" means a place where a person who is a member of a family lives, and is a broad concept including not only a residential building but also a garden, a parking lot, a storeroom, and the like.

The sensing system 1 shown in FIG. 1 acquires a measured amount in a house or an estimated amount calculated by using this measured amount, and also includes an event (or simply "change") related to the measured amount or a change in the estimated amount. Is detected (step S111 in FIG. 2). Here, the "measured quantity" is a feature quantity indicating the measurement result of physical quantities such as light, sound, air, and handwriting in the house. The "estimated amount" is a feature amount indicating the emotions of family members. Further, when the sensing system 1 detects the above-mentioned event, not only the data indicating the change in the measured amount or the estimated amount at the time of detection (hereinafter referred to as event data S) but also the data indicating the occurrence status of the event (hereinafter referred to as event data S). , Contextual data SC) is transmitted (step S112 in FIG. 2).

The sensing system 1 includes an IoT (Internet of Things) sensor group 11 capable of acquiring a measured amount in the house and a handwriting sensor group 12 for detecting the handwriting of a member. The IoT sensor group 11 includes an optical sensor including an imaging sensor 111, a sound collecting sensor 112 including a microphone, and an air sensor including a temperature sensor. The handwriting sensor group 12 is an aggregate of contact type or non-contact type handwriting sensors. As a result, the sensing system 1 is configured to be able to supply various data generated by itself to the control device 2 via the network 4. The handwriting sensor group 12 is not limited to the handwriting written by the members inside the house, and may be configured to be able to detect the handwriting written outside the house.

The image sensor 111 is provided, for example, at the position of the mirror on the washbasin (hereinafter referred to as position 1). The image pickup sensor 111 has a temporary memory that holds the image pickup data for a predetermined time. For example, when an event in which a person approaches to wash the face is detected, the image sensor 111 reads out the video data sd1 which is one aspect of the event data S from the memory and extracts it. In this case, the context data SC1 includes metadata smc1 such as the time of occurrence of the video data sd1 and the spatial position, and video data (hereinafter, adjacent video data sdc1) that is temporally or spatially adjacent at the time of the occurrence of the event. Consists of.

The sound collecting sensor 112 is provided, for example, at a position in the dining room or garden (hereinafter, position 2). The sound collecting sensor 112 has a temporary memory that holds sound data for a predetermined time. For example, when an event that emits a louder voice than usual, an event that the tableware falls and cracks, or an event that steps on a fallen leaf is detected, the sound collecting sensor 112 uses the sound data sd2, which is one aspect of the event data S. Read from memory and extract. In this case, the context data SC2 includes metadata smc2 such as the time of occurrence of sound data sd2 and spatial position, and sound data (hereinafter referred to as adjacent sound data sdc2) that are temporally or spatially adjacent to each other at the time of event occurrence. Consists of.

The first sensor 121 is, for example, a capacitance type touch sensor that detects a position indicated by an electronic pen, and is embedded in the back surface of a mirror (position 1) of a washbasin. The second sensor 122 is, for example, an image scanner that captures an image of the writing surface, and is installed around the dining table (position 2). For example, when an event in which a family member writes is detected, the first sensor 121 or the second sensor 122 reads the handwriting data sd3, which is one aspect of the event data S, from the memory and extracts it. In this case, the context data SC3 is the sound data (hereinafter, adjacent handwriting data sdc3) that is temporally or spatially adjacent to the metadata scc3 of the handwriting data sd3 at the time of writing, the writing position, the writer, etc. And are included. The writer may be identified from the pen ID included in the transmission signal of the electronic pen, or from the owner of the computer provided with the image sensor or the login information.

FIG. 3 is a diagram showing an example of handwriting detected by the handwriting sensor group 12. The first handwriting 301 shows a small shopping memo and is a visualization of the handwriting data sd31. The second handwriting 302 shows a scribbled message, and is a visualization of the handwriting data sd32. The third handwriting 303 shows a sketch that has little meaning, and is a visualization of the handwriting data sd33. The fourth handwriting 304 shows a thank-you letter and is a visualization of the handwriting data sd34. In addition to this, a portrait of a family who continued to draw innocently, a notebook of learning, an old diary, a sketch of the future, etc. can be recorded as handwriting data sd3. In other words, it can be said that the handwriting data sd3 is the data created by the family members in the house, leaving innumerable numbers in the long-term change of at least one year, and by the various thoughts of the members. Hereinafter, the event data S accumulated before a predetermined period (for example, 1 year, 5 years, 10 years, etc.) retroactively from the present is also referred to as "recollection event data".

Next, the control device 2 will be described. The control device 2 stores a memory for a long period of time, such as from the birth to the death of a member of the house or family. The elements that make up a house include not only physical elements such as the house, furniture, smell, sound, light, air, and the people who live in the house, but also the relationships between the people who generate those physical quantities and their emotions. Estimators are included. These elements change over time, and it is thought that the accumulation of aging forms the history of the house, and the continuity of the changes makes the house unique and unique. Therefore, the control device 2 performs various processes related to "house memory" as a series of such transitions.

This control device 2 is realized by one or more computers connected to the network 4. Each computer is configured to include a memory, a processor, and a communication interface. As shown in FIG. 1, the control device 2 functionally includes a storage unit 21 (corresponding to a “storage means”), an analysis unit 22, and a control unit 23.

The storage unit 21 stores the event data S transmitted from the sensing system 1 in association with the context data SC (step S210 in FIG. 2). Hereinafter, in order to clearly indicate that the data is the accumulated past data, the event data may be referred to as "Spst" and the context data may be referred to as "SCpst".

FIG. 4 is a diagram showing an example of the accumulated event data S and context data SC. S11, S12, ... Show the correspondence between the event data S1 acquired by the image pickup sensor 111 and the context data SC1. S21, S22, ... Show the correspondence between the event data S2 acquired by the sound collecting sensor 112 and the context data SC2. S31, S32, ... Show the correspondence between the event data S3 acquired by the handwriting sensor group 12 and the context data SC3.

FIG. 5 is a diagram showing an example of the data structure of the context data SC. The context data SC has N context elements (E1 to EN). The context element E1 indicates the position where the event occurs (for example, position 1, position 2, etc.). The context element E2 indicates the time when the event occurs (for example, hour / minute / second, time zone, etc.). The context element E3 indicates the elapsed time from the occurrence of the event (for example, the number of days, the number of months, the number of years, etc.). The context element E4 indicates the subject of the event (for example, person / object classification, name, name, etc.). The context element E5 indicates the recording mode of the event data S (for example, video, sound, handwriting, etc.).

The type of context element is not limited to the example shown in FIG. 5, and may be various elements having some relation to the event occurrence status. For example, when the recording modes of the event data match, the analogy of the frequency analysis between the adjacent sound data sdc2, the similarity between the adjacent handwriting data sdc3, and the like may be used.

The analysis unit 22 performs various analysis processes on the data accumulated by the storage unit 21 (step S22 in FIG. 2). This step S22 is composed of three sub-steps S221 to S223.

First, the analysis unit 22 determines whether or not the detected event satisfies the operating condition (step S221). This operating condition may be set variously according to the type of event. For example, regarding an event related to sound generation, the volume level of the sound data sd21 is continuously generated in the adjacent sound data sdc21. May be good. Hereinafter, the event that has occurred at present may be referred to as a "first event", and the data corresponding to the first event may be referred to as "first event data Scur" and "first context data SCcur".

Next, the analysis unit 22 selects the event data S belonging to the second event, which has a relatively low relevance to the first event, from the plurality of sets of event data Spst accumulated by the storage unit 21 ( Step S222 in FIG. 2). Hereinafter, this event data may be referred to as "second event data Srep", and the context data associated therewith may be referred to as "second context data SCrep". The method of selecting the second event data Rep will be described in detail later.

Finally, the analysis unit 22 identifies a playback device located near the sensor installation location (step S223 in FIG. 2). This is a process for executing reproduction at a position close to the occurrence point of the event that triggered the event, based on the prediction that it is highly probable that the family members are moving before and after the occurrence of the event.

The control unit 23 controls the sensing system 1 or the reproduction device 3 through the transmission of the instruction signal CT. The control unit 23 transmits a reproduction instruction signal CT1 including the second event data Thread to the reproduction device 3 specified by the analysis unit 22 (step S231 in FIG. 2). The control unit 23 transmits an acquisition instruction signal CT2 to the effect that the measured amount or the estimated amount in the house is acquired to the sensing system 1 in which the first event is detected (step S232 in FIG. 2). Here, the control unit 23 may perform timing control so that the reproduction period P1 by the reproduction device 3 is included in the acquisition period P2 by the sensing system 1.

The reproduction device 3 reproduces the second event having a relatively low relevance to the first event based on the second event data Thread included in the reproduction instruction signal C1T (step S300 in FIG. 2). In the example of FIG. 1, the reproduction device 3 includes a display device 31 provided at the position 1 and a speaker 32 provided at the position 2. The display device 31 is configured to be able to project or display the handwriting left in the past based on the handwriting data which is one aspect of the second event data Thread. The speaker 32 is configured to be able to reproduce the sound left in the past based on the sound data which is one aspect of the second event data Sep. The sensing system 1 and the reproducing device 3 may be incorporated as a single device such as a smart speaker.

The configuration and operation of the home storage system in this embodiment are as described above. The effects expected of the system configured in this way will be described using a scenario example.

For example, on the morning of 7:00 am, person 1 looks at the mirror provided at position 1 when washing his face. The image sensor 111 provided on the mirror measures the facial expression as a physical quantity and detects a change in the state of a person, which is an estimated quantity predicted from the physical quantity. When the image sensor 111 detects a "washing face" event, the control device 2 displays handwriting data sd31 indicating a "shopping memo" having a low relevance to "washing the face" at position 1. It is transmitted to the device 31. The handwriting data sd31 is data obtained by detecting the handwriting left by the person 2 on the dining table at 6 pm by the second sensor 122 provided at the position 2.

For example, suppose that the plate (object 3) broke while eating in the dining room (position 2) on the night of 7:00 pm. The sound collecting sensor 112 provided at the position 2 detects a change in the surrounding sound after emitting a loud sound that the dish cracks. When the "dish" event is detected by satisfying the operation condition that the silent state continues for a certain period of time, the control device 2 provides the sound data sd23 regarding the "fallen leaves", which is less related to the "dish", at the position 2. In addition to transmitting to the speaker 32, the handwriting data sd33 relating to the "meaningless sketch" having a low relevance to the "dish" is transmitted to the display device 31 provided at the position 1. This recreates the sound of someone stepping on a fallen leaf in the yard of the house and the handwriting showing a less meaningful sketch.

<First selection operation>
Subsequently, the first selection operation (step S222 in FIG. 2) by the analysis unit 22 will be described with reference to FIGS. 6 and 7.

In step S601 of FIG. 6, the analysis unit 22 calculates the feature vectors related to the first context data SCcur and the past context data SCpst. For example, this feature vector consists of N vector components corresponding to N context elements. This vector component is a feature quantity that quantitatively expresses a context element, and may be either a discrete value or a continuous value. Hereinafter, it is assumed that each feature vector consists of N vector components normalized in the range of [0,1].

In addition, various methods may be used for the calculation method of the feature vector. For example, this feature vector may be calculated through machine learning using a neural network, as will be described later in FIG. In the example of FIG. 9, the number of dimensions is reduced from N to P (1 <P <N) by using the output value of the intermediate layer 93 (first part 93A) with respect to the feature vector input from the first input layer 91. The feature vector is obtained.

In step S602, the analysis unit 22 classifies the feature amount space 70 using the feature vector acquired in step S601. Here, it should be noted that the strength of the relationship between the context data SCs is evaluated by the norm (distance D) on the feature space 70.

FIG. 7 is a diagram showing an example of a method for classifying the feature space 70. For convenience of illustration, only two components (i, j) are shown. The reference point 71 shows a feature vector for the first contextual data SCcur. The entire region forming the feature space 70 includes [1] first strongly correlated region 72, [2] second strongly correlated region 73, and [3] weakly correlated region 74, depending on the distance D from the reference point 71. It is classified into three areas. The first strongly correlated region 72 corresponds to a region having a strong positive correlation (0 ≦ D ≦ D1). The second strongly correlated region 73 corresponds to a region having a strong negative correlation (D ≧ D2). The weakly correlated region 74 corresponds to the remaining region (D1 <D <D2).

The number of groups to be classified is not limited to the example (3) in FIG. 7, and may be two or four or more. Further, the classification method is not limited to the above-mentioned distance-based method, and various clustering methods including the K-means method may be applied.

In step S603, the analysis unit 22 randomly selects one set from the plurality of sets of context data SCps belonging to the weak correlation region 74 classified in step S602 to select the second context data SCrep to be reproduced. get. Instead, the analysis unit 22 may acquire the second context data SCrep from the context data SCpst belonging to the second strongly correlated region 73. Alternatively, the analysis unit 22 may select the second contextual data SCrep accumulated before a predetermined period (for example, one year) retroactively from the time of occurrence of the first event.

In step S604, the analysis unit 22 reads out the second event data Srep corresponding to the second context data SCrep selected in step S604 from the storage unit 21 and acquires it. In this way, the first selection operation by the analysis unit 22 is completed.

<Second selection operation>
Subsequently, the second selection operation (step S222 in FIG. 2) by the analysis unit 22 will be described with reference to FIGS. 8 and 9.

In step S801 of FIG. 8, the analysis unit 22 calculates the feature vectors related to M event types including the first event. For example, this feature vector consists of Q (1 <Q <M) vector components and is calculated through machine learning using a neural network. Specifically, a P-dimensional feature vector can be obtained by extracting the row components corresponding to the event types from the combined weight matrix {W2} of the M × Q matrix determined by machine learning.

FIG. 9 is a diagram showing an example of the structure of the learner 90 for acquiring the feature vector related to the event type. The learner 90 is a hierarchical neural network in which the first input layer 91, the second input layer 92, the intermediate layer 93, and the output layer 94 are sequentially connected.

The first input layer 91 is composed of N arithmetic units for inputting feature vectors having N feature quantities. Each feature quantity is a value that quantifies the features related to the contextual data SC before the occurrence of the event. The second input layer 92 is composed of M arithmetic units for inputting classification vectors having M labels. This classification vector is a one-hot vector that expresses an event type using two values of applicable (1) and non-applicable (0).

The intermediate layer 93 is composed of (P + Q) arithmetic units. The first portion 93A (P arithmetic units) of the intermediate layer 93 is connected to the first input layer 91 by full coupling. The second portion 93B (Q arithmetic units) of the intermediate layer 93 is connected to the second input layer 92 by full coupling. Here, {W1} and {W2} indicate the coupling weight matrix between the first input layer 91 and the first portion 93A, and between the second input layer 92 and the second portion 93B, respectively.

The output layer 94 is composed of N arithmetic units for outputting a feature vector having N feature quantities. Each feature quantity is a value that quantifies the features related to the contextual data SC after the occurrence of the event. The output layer 94 is connected to the intermediate layer 93 by full coupling. Here, {W3} indicates a coupling weight matrix between the intermediate layer 93 and the output layer 94.

Here, the connection weight matrix {W2} {W3} is updated / optimized through machine learning in which the event type and the feature vector before the occurrence of the event are used as input values and the feature vector after the event occurs is used as the correct answer value. .. On the other hand, the connection weight matrix {W1} is pre-optimized through pre-learning and is fixed at the time of actual learning. This pre-learning corresponds to unsupervised learning by a self-encoder using a partial neural network consisting of a first input layer 91, a first portion 93A, and an output layer 94.

In step S802 of FIG. 8, the analysis unit 22 classifies the feature amount space 70 using the feature vector acquired in step S801. The analysis unit 22 may perform classification using the method already described in step S602 of FIG. Here, as shown in FIG. 7, it is assumed that the feature space 70 is classified into three regions, a first strongly correlated region 72, a second strongly correlated region 73, and a weakly correlated region 74.

In step S803, the analysis unit 22 acquires the event type to be reproduced by randomly selecting one of the plurality of event types belonging to the weak correlation region 74 classified in step S802.

In step S804, the analysis unit 22 acquires the second context data SCrep to be reproduced by randomly selecting one of the plurality of event data Spsts corresponding to the event types selected in step S803. .. Alternatively, the analysis unit 22 may select the second contextual data SCrep accumulated before a predetermined period (for example, one year) retroactively from the time of occurrence of the first event.

<Summary of Embodiment>
As described above, the home storage device in this embodiment uses handwritten handwriting data (corresponding to "event data S") made in the house, which is the residence of the family, as context data SC indicating the context of handwriting. A storage unit 21 that is associated and accumulated as recollection handwriting data, and a sensor that detects a change in the physical quantity of light, sound, air, or handwriting in the house, or a change in the estimated amount of emotions of family members. When a change (corresponding to the "first event") is detected, a weak correlation context having a relatively small relevance to the change is selected from a plurality of sets of context data SCs accumulated by the storage unit 21. Recollection writing data (second event data) associated with analysis unit 22 that randomly selects data (corresponding to "second context data SCrep") and weakly correlated context data randomly selected by analysis unit 22. It is provided with a control unit 23 that reads one (corresponding to “Srep”) from the storage unit 21 and instructs the reproduction device 3 provided in the house to project the recollection stroke data. Here, the storage device of the house may further include one or a plurality of reproduction devices 3 provided in the house and projecting the recollection writing data.

Further, the analysis unit 22 may select weak correlation context data accumulated before a predetermined period from the time when the change is detected. Further, the analysis unit 22 analyzes the relationship between the accumulated plurality of sets of context data SCs, classifies the plurality of sets of context data into two or more groups, and changes context data indicating the context of change (“No. 1”. Weakly correlated context data may be selected from a group different from the group to which (corresponding to 1 context data SCcur) belongs. Further, the analysis unit 22 may generate feature vectors for each context data SC through machine learning, and classify a plurality of sets of context data SCs into two or more groups based on the relationships between the feature vectors. Further, when a plurality of reproduction devices 3 are provided in the house, the control unit 23 may specify the reproduction device 3 for projecting the recollection handwriting data based on the position where the change is detected. Further, the control unit 23 may instruct the sensor to detect a change over the acquisition period (P2) including the reproduction period (P1) of the recollection written data.

The home storage device in this embodiment associates handwritten handwriting data (corresponding to "event data S") made in the house, which is the residence of the family, with the event type for classifying the handwritten context. A storage unit 21 that accumulates as recollection handwriting data, a sensor that detects changes in physical quantities related to light, sound, air, or handwriting in the house, or changes in estimated quantities related to the emotions of family members, and changes ("" When (corresponding to the "first event") is detected, a weakly correlated event type ("second event") that has a relatively small relevance to the change from among the plurality of event types accumulated by the storage unit 21 One analysis unit 22 that randomly selects (corresponding to "event type") and one recollection handwriting data (corresponding to the second event data "Srep") associated with the weakly correlated event type selected by the analysis unit 22. It is provided with a control unit 23 that reads from the storage unit 21 and instructs a reproduction device 3 provided in the house to project the recollection handwriting data. Here, the storage device of the house may further include one or more reproduction devices 3 provided in the house and projecting the recollection writing data.

Further, the analysis unit 22 may select weak correlation context data accumulated before a predetermined period from the time when the change is detected. In addition, the analysis unit 22 analyzes the relationship between the accumulated event types, classifies the plurality of event types into two or more groups, and is in a group different from the group to which the event type of change belongs. Weakly correlated contextual data may be selected from. Further, the analysis unit 22 may generate a feature vector for each event type through machine learning, and classify the plurality of event types into two or more groups based on the relationship between the feature vectors. Further, when a plurality of reproduction devices 3 are provided in the house, the control unit 23 may specify the reproduction device 3 for projecting the recollection handwriting data based on the position where the change is detected. Further, the control unit 23 may instruct the sensor to detect a change over the acquisition period (P2) including the reproduction period (P1) of the recollection written data.

<Explanation of symbols>
1) Sensing system, 11 ... IoT sensor group, 12 ... Handwriting sensor group, 2 ... Control device, 21 ... Storage unit, 22 ... Analysis unit, 23 ... Control unit, 3 ... Reproduction device, 31 ... Display device, 32 ... Speaker Device, S ... event data, Scur ... first event data, Srep ... second event data, SC ... context data, SCcur ... first context data, SCrep ... second context data

Claims

A storage unit that stores handwritten handwriting data made in the house, which is the residence of the family, as recollection handwriting data in association with the context data indicating the context of the handwriting.
A sensor that detects changes in physical quantities related to light, sound, air, or handwriting in the house, or changes in estimated quantities related to the emotions of members of the family.
When the change is detected by the sensor, weakly correlated context data having a relatively small relevance to the change is randomly selected from a plurality of sets of context data accumulated by the storage unit. The analysis department to select and
One recollection handwriting data associated with the weakly correlated context data randomly selected by the analysis unit is read from the storage unit, and the projection of the recollection handwriting data is instructed to a reproduction device provided in the house. Control unit and
Home storage with.
The analysis unit selects the weakly correlated contextual data accumulated prior to a predetermined period retroactively from the time of detection of the change.
The home storage device according to claim 1.
The analysis unit
The relationship between the plurality of sets of contextual data accumulated by the storage unit is analyzed, and the plurality of sets of contextual data are classified into two or more groups.
The weakly correlated context data is selected from a group different from the group to which the change context data indicating the context of the change belongs.
The home storage device according to claim 1.
The analysis unit generates feature vectors for each context data through machine learning, and classifies the plurality of sets of context data into two or more groups based on the relationships between the feature vectors.
The home storage device according to claim 3.
When a plurality of reproduction devices are provided in the house, the control unit identifies a reproduction device that projects the recollection handwriting data based on the position where the change is detected.
The home storage device according to claim 1.
The control unit instructs the sensor to detect the change over the acquisition period including the reproduction period of the recollection written data.
The home storage device according to claim 1.
Further equipped with one or more playback devices provided in the house to project the recollection written data.
The home storage device according to claim 1.
A storage unit that stores handwritten handwriting data made in the house, which is the residence of the family, as recollection handwriting data in association with the event type for classifying the handwriting context.
A sensor that detects changes in physical quantities related to light, sound, air, or handwriting in the house, or changes in estimated quantities related to the emotions of members of the family.
When the change is detected by the sensor, a weakly correlated event type having a relatively small relevance to the change is randomly selected from a plurality of event types accumulated by the storage unit. Analysis department and
One recollection handwriting data associated with a weakly correlated event type randomly selected by the analysis unit is read from the storage unit, and the projection of the recollection handwriting data is instructed to a reproduction device provided in the house. Control unit and
Home storage with.
The analysis unit selects the weakly correlated contextual data accumulated prior to a predetermined period retroactively from the time of detection of the change.
The home storage device according to claim 8.
The analysis unit
The relationship between the plurality of event types accumulated by the storage unit is analyzed, and the plurality of event types are classified into two or more groups.
Select the weakly correlated event type from a group different from the group to which the event type of the change belongs.
The home storage device according to claim 8.
The analysis unit generates a feature vector for each event type through machine learning, and classifies the plurality of event types into two or more groups based on the relationship between the feature vectors.
The home storage device according to claim 10.
When a plurality of reproduction devices are provided in the house, the control unit identifies a reproduction device that projects the recollection handwriting data based on the position where the change is detected.
The home storage device according to claim 8.
The control unit instructs the sensor to detect the change over the acquisition period including the reproduction period of the recollection written data.
The home storage device according to claim 8.
Further equipped with one or more playback devices provided in the house and projecting the recollection written data.
The home storage device according to claim 8.