WO2023062250A1 - Method and system for taking decisions and executing unattended actions in a limited environment - Google Patents

Method and system for taking decisions and executing unattended actions in a limited environment Download PDF

Info

Publication number
WO2023062250A1
WO2023062250A1 PCT/ES2021/070744 ES2021070744W WO2023062250A1 WO 2023062250 A1 WO2023062250 A1 WO 2023062250A1 ES 2021070744 W ES2021070744 W ES 2021070744W WO 2023062250 A1 WO2023062250 A1 WO 2023062250A1
Authority
WO
WIPO (PCT)
Prior art keywords
inf
layer
information
environment
home
Prior art date
Application number
PCT/ES2021/070744
Other languages
Spanish (es)
French (fr)
Inventor
Pablo LÓPEZ COYA
Original Assignee
Telefonica Digital España, S.L.U.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Telefonica Digital España, S.L.U. filed Critical Telefonica Digital España, S.L.U.
Priority to PCT/ES2021/070744 priority Critical patent/WO2023062250A1/en
Publication of WO2023062250A1 publication Critical patent/WO2023062250A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/22—Microcontrol or microprogram arrangements
    • G06F9/28—Enhancement of operational speed, e.g. by using several microcontrol devices operating in parallel
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning

Definitions

  • the present invention has its application in the field of Artificial Intelligence (AI) technologies and, more specifically, applied to the domestic environment for the construction of intelligent homes (“Smart Home”, in English). More particularly, the present invention relates to a system and method for unattended automatic decision making using machine learning techniques.
  • AI Artificial Intelligence
  • Smart Home Today, the solutions presented in the field of what is called Smart Home (“Smart Home”) continue to be perceived as a resolution of isolated problems and, furthermore, reduced to the field of information collected by a data source. captive or limited by the commercial scope of the device itself.
  • a device, system or method that claims to be intelligent, at least in the home environment, must include at least two types of capabilities:
  • the present invention serves to solve the problem mentioned above, by means of a method for unattended decision-making and execution of actions in real time within a bounded environment, understanding the bounded environment as a closed ecosystem whose contextualization is not complex, that is, the events that can occur in said environment are explainable, measurable and parameterizable.
  • the method described here is capable of interpreting situations of various kinds that take place in said bounded environment by collecting data generated by a base of hardware devices belonging to the environment to be explored and applying algorithms based on automated learning techniques (ML: Machine Learning, in English).
  • An essential element of the present invention is an inference engine that, through a layered information association model, generates insights relevant to the understanding of the events that take place in the environment. bounded environment.
  • the result of this process is a dynamic context that allows the system itself to make unattended decisions.
  • the present invention achieves this objective due to its ability to associate sources of understanding obtained by different inference processes around semantically complex themes.
  • This capability is provided by the topmost layer of a layered model, here called the Discovery Stack, which is a machine learning model (ML model) where each layer or level provides more refined information to the higher level.
  • the Pile of Discovery defined here is a model that comprises three layers, ordered here from lowest to highest level:
  • Layer 1 Data Generation, made up of all the devices belonging to the environment where the discovery of events is to be carried out, each device generating a series of data contribution sources.
  • Layer 2 Pattern Recognition, to adapt the raw data from all the contribution sources of the previous layer 1 and obtain inference elements with relevant information for the next level or layer.
  • Layer 2 is made up of preprocessing modules that apply techniques to identify recognition patterns of different kinds using specific ML algorithms depending on the nature of the data source.
  • Layer 3 Information Association, formed by different information association modules to relate elements inferred by the levels and generate at least one element of understanding (or knowledge or perception) (“insight”) relevant (semantically adequate) for detection of a specific event in the bounded environment.
  • insight element of understanding (or knowledge or perception)
  • an element of understanding is defined as the minimum unit of information discovered in the delimited environment and understandable by a system capable of carrying out a semantically appropriate action for the reference ontology of the environment. action (enclosed environment).
  • One aspect of the invention refers to a method for unattended decision-making and execution of actions in a bounded environment, comprising the steps defined in claim 1.
  • Another additional aspect of the present invention refers to a system for unattended decision-making and unattended execution of actions in a bounded environment, the system comprising the following components that carry out the steps of the method described above.
  • the system of the invention is made up of different modules (SW: software), distributed in the three logical layers defined above and which are connected in cascade.
  • a module is constituted by the technical means that carry out a minimum functional unit belonging to a layer of the system.
  • the different modules are structure and interconnect according to a (logical) model.
  • each layer is a logical association of modules that carry out related tasks within the system.
  • the different functionalities or functions indicated in the previous method are performed by electronic programmable devices (for example, a server or a client node) or set of electronic programmable devices, which in turn can be co-located or distributed in different locations. and communicated by any type of wired or wireless communication network.
  • the component (hardware or software) of that electronic device that performs a certain functionality is what is known as a module.
  • the different functionalities or functions that are indicated in the previous method can be implemented as one or more components.
  • Each functionality can be carried out on a different device (or set of them) or the same device (or set of them) implement vahas or all of the indicated functionalities; that is to say, the same device (for example, a server) can have several modules, each of them carrying out one of the functionalities or these modules can be distributed in different devices.
  • These components and associated functionality may be used by client, server, distributed, or peer-to-peer computing systems.
  • These components may be written in a computer language corresponding to one or more programming languages such as functional, declarative, procedural, object-oriented languages, and the like. They can be linked to other components through various application programming interfaces and implemented in a server application or a client application. Alternatively, the components can be implemented in both server and client applications.
  • Another last aspect of the invention relates to a computer program, which contains instructions or computer code (stored on a non-transient computer-readable medium) to cause processing means (of a computer processor) to perform the steps of the described method. previously for unattended decision making in a bounded environment.
  • the present invention provides a dynamic context with information on the most relevant events that occur within the bounded environment. This allows the environment to constitute an intelligent and conscious entity that solves non-obvious problems, without requiring any action from the user and being able to anticipate the user's action.
  • the present invention allows the environment to not require any action by the user to carry out an authentication process, since it has sufficient contextual information about the environment, and does not need to know sensitive user data either.
  • the present invention allows the environment to function as an autonomous entity that guarantees the privacy of the information handled, for example, information belonging to a home environment and, also in this case, the home environment itself thus achieves autonomy to create parental control. .
  • FIGURE 1. Shows a generic functional model with the three layers that make up the discovery stack of an intelligent system for unattended decision-making, according to a preferred embodiment of the invention.
  • FIGURE 2. Shows a block diagram of the system architecture for decision-making in a limited environment, according to a preferred embodiment of the invention.
  • FIGURE 3. Shows a detail of the generation, storage and analysis blocks of snapshots used by the system for decision making.
  • FIGURE 4. Shows a diagram of the logical unit used by the decision-making system, according to a possible embodiment of the invention.
  • FIGURE 5. Shows an example of a relationship matrix between detected events and inferred logical sentences in the bounded environment.
  • FIGURE 6.- Shows another example of a relationship matrix between events detected and logical sentences inferred in the delimited environment to authenticate a user who owns a mobile telephone line at the household level.
  • FIGURE 7.- Shows another example of a relationship matrix between events detected and logical sentences inferred in the delimited environment to carry out parental control at the household level.
  • FIGURE 8.- Shows an inference map that associates privacy rings according to the level of belonging to the nuclear family in a household.
  • a preferred embodiment of the invention refers to a system for unattended automatic decision-making in a bounded environment.
  • Figure 1 schematically shows the components of the proposed system which, in its most generic form, consists of a layered model, where each layer or level provides more refined information to the higher level, as described below.
  • a first layer 110 for data generation is made up of all the devices DISP_1, DISP_2, DISP_i, belonging to the environment where it is desired to perform event discovery.
  • Each DISP_i device, DISP_i being the ith device generates a number of data sources.
  • a DISP_i device can give rise to one or several data sources, each data source is denoted, for the ith DISP_i device, as FD_ ⁇ 1, .. FD_ij.
  • Each data source is classified by the nature of the sensitive element that collects the data (the subscript j is used to denote the j-th data source);
  • the data sources can be audio, video or even telemetry sources related to some temperature, pressure or humidity sensor, or also more sophisticated data sources, such as the position of a mobile terminal, the level Wi-Fi router signal strength, internal telemetry from a video set-top box, or even the battery level of a wireless speaker.
  • the preprocessing modules appear.
  • the PREP_p preprocessing module may adapt data from one or more of the data sources FD_1j,...,FD_ij.
  • Each PREP_p preprocessing module accommodates one or more sources to a given recognition technique TR_k. All these recognition techniques TR_1, TR_2,...TR_k have the mission of identifying recognition patterns of different kinds using specific machine learning (ML) algorithms depending on the nature of the data source.
  • ML machine learning
  • Several sources can contribute to nurture one or several recognition techniques (the subscript k is used to denote the kth pattern recognition technique.
  • the result is an inference element INF_1, INF _2,... INF _k with relevant information to the next level or layer
  • An inference element is provided by a recognition technique.
  • This third layer 130 is made up of different information association modules.
  • Each of these modules employs specific information association techniques TAI_1, TAI_2,..., TAI_I (the subscript I is used to denote the lth information association technique) to relate inference elements INF_1, INF_2,. .. INF _k by the level below this last level.
  • An information association technique can be nourished by various elements of inference.
  • the criterion of association followed in each information association module is directed in a certain line depending on the nature of the discovery process that is intended to be addressed.
  • the final result is the generation of an understanding element, INS_1, INS_2..., INS_I or "insight" relevant to the detection of a specific event in the environment.
  • the element of understanding is the minimum unit of information discovered in the bounded environment of action, that is, the bricks on which relevant use cases are built for a given task.
  • An element of understanding (“Insight”) is provided by an information association technique.
  • Table 1 shows some examples of ML algorithm strategies, the applicable state-of-the-art ML modeling techniques (SOTA techniques: State-of-the-art, in English) and their scope within the field of application. layer model described above, as well as the data source(s) that contribute to generating a discovery base.
  • SOTA techniques State-of-the-art, in English
  • the translation of the previous model of layers that make up the Discovery Stack in the home environment is, according to an example:
  • the first layer 110 for data generation is the ecosystem of home devices, made up of the most common devices that make up the ecosystem of a standard home.
  • the goal of this layer is to determine which data sources from each device can actively contribute to event discovery; For example, an IP camera contributes a video source, a router contributes WiFi signal strength level data and information about the devices that are currently connected to the home network, etc.
  • the second layer 120 for pattern recognition focuses on pattern recognition of specific activities related to the day-to-day life of a home, relying on pattern recognition techniques in a video signal, audio signal, the WiFi signal from the router or even pattern recognition techniques related to traffic carried within the home network. home.
  • the third layer 130 for the association of information combines the simple patterns of the previous level by means of ML algorithms that allow more complex patterns to be inferred. Specifically, for the casuistry of the home, association algorithms can be used around the understanding of human activities (Human Activity Recognition) and the understanding of scenes (Scenario Understanding). All this in order to define the most relevant fields that allow the generation of relevant elements of understanding or “insights”, in this case, related to events discovered in the home environment.
  • Figure 2 shows a summary of the architecture proposed to shape the unattended decision-making system for a given environment.
  • the key piece of the system is the inference engine 240, which is responsible for generating an instantaneous capture 250 or "snapshot" of what is happening in the home at all times (ie, a "snapshot” collects all the "insights") , generating a "snapshot” every certain period of time, for example, every x seconds.
  • the proposed system is capable of storing the "snapshots" generated in a time line as a history, in such a way that with a medium-long term analysis of stored "snapshots" a discovery of patterns 260 is made, which become to be injected as part of the conclusions of the next snapshot 250 or "snapshot".
  • automata 270 which is the module that provides the system with the capacity for unattended decision-making, for which It is based on two fundamental concepts: i) the activation signals or "triggers", TRIG_1, TRIG_2,... TRIG_y, defined as the set of logical signals coming from the inferred conclusions and collected in the snapshot 250, which allows triggering a sequence of actions unattended; and ii) the actions ACT_1, ACT_2, ACT_n, that the system is capable of executing on the bounded environment in question and that an actuator 280 generates from the activation signals TRIG_1, TRIG_2,... TRIG_y received, defining the actions ACT_1, ACT_2, ACT_n as a set of operations that are carried out on different electronic devices 290 in the home, for example, telephones, alarms, computers,...
  • the snapshot 250 or "snapshot” can be understood as a detailed report of all the events that are taking place in the exploration environment, both in the physical and in the virtual plane, understanding the virtual plane as everything that happens as a consequence of a digital transaction of information.
  • An important characteristic of the snapshot is that it allows the feedback of elements of understanding or "insights", that is, apart from including conclusions in real time, the snapshots SN_1, SN_2, SN_x are stored, generated in a time line as a history of instantaneous captures 255, in such a way that a posteriori processing extracts the patterns 260 of behavior in the medium term.
  • FIG. 3 shows a particular case of information processing for the extraction of medium and long-term patterns, which are then combined with the insights extracted in real time from the environment, thus feeding back the inference engine to provide greater understanding, that is, a user "customs detector” would thus be constituted.
  • the inference engine 240 a key part of the system shown in Figure 2, is made up of different logical parts that must be detailed in order to understand its operation.
  • the inference engine 240 comprises an inference map, which is an algorithm element for combining information extracted from the processed from different sources of information and thereby deduce the elements of understanding or "insights", which emerge as a new predictive link is added to the chain.
  • each link in the chain is organized as a node within a decision tree and is called a logical unit UL_r.
  • the subscript r is used to denote the rth logical unit, logical question, base event, and the subscript s for the sth inference model per logical unit.
  • FIG. 4 schematically collects the diagram of a standard logical unit.
  • Each logical unit UL_r is made up of the following elements:
  • Logical question CUE_r This is a simple logical question to be answered by processing information from the environment.
  • Base event EVENT_r defines the base event on which the recognition pattern is built and, therefore, the classification scheme followed by all the inference models, MOD_r1, MOD_rs, which contribute to answering the logical question CUE_r posed.
  • the model combination approach follows a multi-modal pattern, where each input inference model, MOD_r1, MOD_rs, solves a decision problem in different data domains, for example, by combining models trained with video sources, from audio, etc, that are relevant to answering the logical question contained in the logical question CUE_r.
  • Logical Sentence SL_r The result of the inference problem is expressed in terms of numerical probabilities, therefore, a translation to the language domain understood by a human is necessary. In this sense, Logical Sentence SL_r is defined as the maximum expression of affirmation on the logical question CUE_r that in terms of information can be inferred from the base event EVENTO_r.
  • the error Err (cuE_r) made in each prediction is limited to the definition of the initial question CUE_r and the level of difficulty in the prediction of the base event EVENT_ r, which delimits the amount of information that can be extracted from the environment and therefore, with which the initial question CUE_r can be answered.
  • the semantic error committed in the inference process and which is given by Err() measured on the logical question that is intended to be answered is equal to the semantic difference between the information extracted from the initial question and the logical sentence inferred by the logical drive. And, in any case, this semantic error is bounded in that a less ambitious logical question about the environment can always be found and a better base event can always be chosen for the modeling of the predictive schema.
  • the power of this approach is achieved by combining various logical units in a tree or graph-type decision scheme, where the last leaves or extreme nodes reach considerable levels of predictive difficulty, without affecting the complexity of the system architecture. .
  • the proposed architecture guarantees the scalability of the solution.
  • the home ecosystem inference map is a decision tree that tries to answer simple logical questions related to Who, What, Where and How.
  • An example of the beginning of the decision tree designed for a standard home environment might begin with scanning the environment by asking about who is on the scene and what is wanted. know if there is movement, if there is human presence, etc. For example, with motion detector devices it is determined if there are moving bodies in the home. If so, human presence detectors intervene to determine if there are people in the home. And, if not, it can be determined if there are pets in the home through animal detectors, to conclude if any pet is moving around the home.
  • any home automation device has made a movement or the movement has been from an accidentally moved object or a false alarm, using mobile device detectors.
  • the motion detectors detected people moving, using group detectors it can be determined if there are several people or just one person in the home.
  • the decision tree starting from simple premises, gradually deduces more complex situations.
  • the nodes of the tree which are the UL_r logical units, further away feed on the patterns inferred from the previous level to reach more sophisticated conclusions about the environment.
  • the system is capable of determining whether the person belongs to the family and whether or not they have the mobile with them through the information obtained from home wireless WLAN network.
  • the system deduces that the person is alone and without the mobile. If, in addition, the WLAN network does not detect a previously registered device either, it is concluded that that person is not a member of the family. But if the WLAN network finds a new connected device that was also previously registered, it is concluded that the person who is alone is a family member.
  • the system When the system has discovered in the previous stage that there is a group of people at home and the WLAN network detects a new connected device that In addition, it was previously registered, the system deduces that someone in the group is a member of the family. If the WLAN network does not detect any new connected device, the system determines that someone in the group is not a member of the family or may be but does not have the mobile with them; if, in addition, the previously registered device is not found, it is concluded that someone in the group is not part of the family.
  • the system is capable of distinguishing if that foreign person is a child or an adult using recognition techniques. age and determine if the person (child or adult) who is in the group (accompanied) outside the family is male or female, using gender classification techniques such as those in Table 1 above.
  • the system is able to distinguish if the person in the family who is in the group is a child (and, therefore, according to their gender, for example, conclude if it is the son or the daughter) or an adult (man or woman, husband or wife) who, therefore, is accompanied at home.
  • the gender classification and age recognition techniques discriminating between those over 18 years of age or under allow the system to equally distinguish if the person who is alone is a child (the son or the daughter) or an adult (the husband or the wife) or, if he is older than a certain age (for example, using a second range of discrimination set at 70 years), determine if an older person in the family (for example, the grandfather or the grandmother) is the one who is in the home without company.
  • the system uses gender classification and age recognition techniques, discriminating between those over 18 years of age or under, to determine whether the person who, in the previous stage of discovery of family members in the home, it has been concluded that she is alone without being part of the family.
  • Figure 5 finally shows in matrix form, starting from the logical aspect or question of Who, the relationship between base events that have been discovered through prediction techniques and the non-trivial logical sentences inferred in relation to the described home environment.
  • the matrix in the example shown has the following base detected events arranged in columns of the relationship matrix and the following logical statements in rows:
  • Event e1 There is a male minor from outside the family at home accompanied
  • Event e2 There is a minor outside the family at home accompanied
  • Event e3 There is an adult male from outside the family at home accompanied
  • Event e4 There is an adult woman from outside the family at home accompanied
  • Event e5 My son is at home accompanied
  • Event e6 My daughter is at home accompanied
  • Event e7 My husband is at home accompanied
  • Event e8 My wife is at home accompanied
  • Sentence s1 My children are home alone
  • Sentence s2 Mother is at home with non-family adults
  • Judgment s4 My son is at home with minors from outside the family
  • Sentence s5 My son is at home with a minor outside the family
  • Sentence s6 My son is at home with a minor outside the family
  • Sentence s7 My daughter is at home with minors outside the family
  • Sentence s8 My daughter is at home with a minor outside the family
  • Sentence s9 My daughter is at home with a minor outside the family
  • Sentence s10 There is a woman adult at home outside the family with my children
  • Sentence s11 There is an adult woman at home outside the family with my son
  • Sentence s12 There is an adult woman at home outside the family with my daughter
  • Sentence s13 There is an adult man at home outside the family with my children
  • Sentence s14 There is an adult male at home outside the family with my son
  • Sentence s15 There is an adult male outside the family at home with my daughter
  • Sentence s16 Mother and children are home alone
  • Sentence s17 Mother and son are home alone Sentence s18: Mother and daughter are home alone Sentence s19: Father and sons are home alone Sentence s20: Father and son are home alone Sentence s21: Father and daughter are home alone Sentence s22: Father and Mother are home alone
  • the Discovery Stack launches a battery of predetermined predictive models based on deep learning (DL: Deep Learning, in English) that, combined with additional logic, allow for the construction of a dynamic context with information on the most relevant events that have occurred in the environment in question.
  • the composition of the context is fed by vectors of information of a different nature, providing elements of inference related to the physical world and the virtual world.
  • the system presents information feedback mechanisms based on the history of previously inferred material.
  • the home is perceived outwardly as an active entity that abstracts from the need to specify the destination communication device or mechanism.
  • the home as an autonomous entity aware of its environment is capable of delivering the right message, to the right person, in the right place or channel and at the right time.
  • it is no longer necessary to specify a telephone number, an IP address, a domain or a specific device as the destination of the communication, it would only be necessary to specify a specific home as the recipient of the information. From here, the home, aware of its environment, selects the ideal mechanism to guarantee the delivery of the message and the attention of the recipient.
  • the owner of the telephone line is located in different scenarios where the delivery of the notification must be articulated through different channels: 1) The owner of the line is in the living room of his house watching television. The mobile is charging in the bedroom.
  • the notification would be delivered to the phone.
  • the delivery channel is clearly not the right one, since the owner would see the message when he returned to the bedroom to pick up his phone.
  • the layered model of the invention is applied, by means of a simple combination of logical sentences deduced from the environment, which would be "my husband is home alone” and “my husband is in the living room", together with the power on/ turning off the decoder as part of the logic extracted from the environment, the system is capable of enabling television as the ideal channel for delivery.
  • the holder of the line is cooking.
  • There is a display-enabled media device eg tablet, smart speaker, home assistant, etc, Certainly installed in the kitchen.
  • the mobile is in the bedroom charging again. The children are watching television in the living room.
  • the system concludes that it is "My children" who are watching TV.
  • My children who are watching TV.
  • the system arrives at the fact that there is a minor and a minor, who are members of the family, therefore, the son and daughter, at home and are accompanied by an adult. It is also detected that they are in the living room and the decoder's on/off status is obtained, with which the system finally infers that "my children are in the living room watching TV",
  • the final addressee is the owner of the line
  • television is not considered a valid channel to deliver the information.
  • the system concludes that "My father” (the owner of the line in this example) is in the kitchen carrying out a certain activity (for example, it is determined that he is preparing food for the on/off state of the kitchen), starting from the base event that "my husband is at home accompanied".
  • the device located in the kitchen is, therefore, selected by the Home as the message delivery channel.
  • the owner of the line is away from home
  • the initial question or question is whether or not there is human presence in the home and, using the inference capability of the system, the tree node provides this information as received by the detectors. home presence.
  • the message is delivered to the mobile terminal of the line owner (the mobile terminal is notified in the first instance because it is the appropriate delivery option and not the default option, as in the two previous cases). . b) Authentication at the Home level
  • authentication processes tend to move between two red lines: one related to the usefulness of the method for the user and the other related to the intrusion of the identification algorithm.
  • the differential value introduced by the home as an autonomous entity can be summarized in two things: i. the home does not need to bother the user with an authentication process that implies any action on their part, since it has enough contextual information from the environment to recognize at all times the roles that are participating in the scene and what action is intended to be completed.
  • the household needs to solve a simpler problem than that of biometric authentication, where it only needs to locate each individual in the ring belonging to the appropriate family nucleus.
  • a guest In terms of interaction with the home, a guest is not the same as the owner of the line or an intruder, in the same way that a small child is not the same as an adult or an elderly person.
  • the inference engine goes through a decision tree answering a series of key questions with information collected from the environment through the set of DL techniques.
  • Figure 6 shows an example of how the authentication of the holder of the line is carried out (assuming that it is the father who is holding this role), relying on a relationship matrix, similar to the one in Figure 5, to find the logical sentences Which give way to this unattended authentication at the home level:
  • Sentence s20 Father and son are home alone
  • Sentence s21 Father and daughter are home alone
  • Sentence s22 Father and Mother are home alone Sentence s3: Father is home with non-family adults based on the following events:
  • Event e5 My son is at home accompanied
  • Event e6 My daughter is at home accompanied
  • Event e7 My husband is at home accompanied
  • Event e8 My wife is at home accompanied
  • Event e9 There is an adult man at home outside the family and accompanied Event e10: There is an adult woman at home outside the family and accompanied Event e11: My husband is at home alone
  • the differential value resides precisely in the fact that the home as an autonomous entity is capable of creating parental control at a holistic level, making joint decisions on the physical level (for example, blocking potentially dangerous electrical appliances for a child) and the virtual level ( disabling certain gaming activities or excessive consumption of content). All this taking advantage of the contextual information collected from the environment where the application of certain profiles is carried out ad-hoc to the scene in question in real time and without the need for default configurations. All this represents a substantial improvement compared to the current scenario where each service contracted at home has its own parental control. Specifically, Figure 7 shows how to find the conditions that favor activation of parental control at the household level in the system, that is, all those scenarios in which the household perceives minors without the presence of adults:
  • Event e5 My son is at home accompanied
  • Event e6 My daughter is at home accompanied
  • Event e13 My daughter is at home alone to conclude if
  • Sentence s1 My children are home alone d) Fluency in experience
  • the home as an autonomous entity allows to organize, synchronize and switch the user's attention to different states in a smooth and fluid way. These states do not fight each other, but rather complement and adapt to the true interests of the home user.
  • the contextual information that the Home makes use of as an autonomous entity makes it possible to safeguard the user's privacy, the rate of information consumption and the adaptation of the environment to the specific circumstances of the moment.
  • the inference engine reaches a series of conclusions reflected in the map in Figure 8, where the behavior of the household is associated with a privacy ring depending on the level of belonging to the family nucleus of the components on the scene.
  • the inference map we start from two basic events: e81 : “there is a group of several people at home” e82: “there is only one person at home”
  • the logical question that arises is if someone belongs to the family nucleus: c1 : “Is there someone from the family in the group?”
  • c2 “is the person a family member?”
  • n1 there is any new device connected in the family WLAN network
  • n2 there is any new device connected in the WLAN_GUEST network
  • n3 there is no new device connected in the family WLAN network
  • the system is capable of inferring the following sentences shown on the map:
  • the system obtains more refined information, finding out if: d1 : the connected wireless device was previously registered d2 : no previously registered wireless device is detected
  • the system is capable of making inferences beyond the previous statements, as shown in the top level of the map:
  • a Cloud service (“cloud”) that displays multimedia content through a screen (Tablet, TV, ...) is taken as an actuator.
  • the home is capable of articulating the application programming interface (API) of the Cloud, showing only appropriate content based on the privacy ring reported by the inference engine.
  • API application programming interface
  • the system is capable of inferring what action/s are being carried out in the delimited environment (Home). Specifically, applying techniques based on Human Activity Recognition, Scene Understanding and a time stamp, the system is capable of classifying day-to-day activities and ordering them on a time line. Subsequently, a model based on time series can predict that every afternoon, for example at 8:15 p.m., the owner of the line waters the plants in the garden. Therefore, if the activity in question is not detected when the time comes, the appropriate user is notified that the plants have not been watered today.
  • the system can identify the usual work hours of the smallest members of the household and remind them that it is time to do homework. Likewise, a notification could also be sent to the parents in case the school tasks are not being carried out.
  • An example of the logic followed is: from the event "My children are alone at home” it is inferred that "My children are alone in the classroom” but, when it is time to do their homework, the activity in the classroom is not detected “ read/study” and, therefore, a reminder must be sent to "Children” or "Mother” or “Father” (line holder).
  • the inference engine would recognize an elderly person (grandparent) who regularly takes a medication at one hour. The day that no record of this activity is found at that time, the Home sends a notification to this person (grandparent) reminding them to take the medication.

Abstract

The invention relates to a method and system for taking decisions and executing actions in a limited environment, based on a model comprising three layers (110, 120, 130) that uses machine learning, wherein each layer provides more refined information to the layer above: a data generation layer (110) comprising data sources (FD_11,..., FD_1j, FD_21,...FD_2j,.., FD_i1,...FD_ij) of devices (DISP_1, DISP_2,..., DISP_i) in the environment; a pattern recognition layer (120), wherein preprocessing modules (PREP_1, PREP_2,... PREP_p) associate the data sources (FD_11,..., FD_1j, FD_21,...FD_2j,.., FD_i1,...FD_ij) with a pattern recognition technology (TR_1, TR_2,..., Tr_k) to obtain with each pattern an inference element (INF_1, INF_2,..., INF_k) with information regarding the limited environment; and an information association layer (130) for associating information from each inference (INF_1, INF_2,..., INF_k), thereby generating elements of understanding; and generating, from the associated information, activation signals (TRIG_1, TRIG_2,... TRIG_y) regarding actions (ACT_1, ACT_2,..., ACT_n) to be executed on devices in the environment.

Description

MÉTODO Y SISTEMA PARA LA TOMA DE DECISIONES Y EJECUCIÓN DE ACCIONES DESATENDIDAS EN UN ENTORNO ACOTADO METHOD AND SYSTEM FOR DECISION-MAKING AND EXECUTION OF NEGLECTED ACTIONS IN A LIMITED ENVIRONMENT
DESCRIPCIÓN DESCRIPTION
OBJETO DE LA INVENCIÓN OBJECT OF THE INVENTION
La presente invención tiene su aplicación en el sector de las tecnologías de Inteligencia Artificial (IA) y, más concretamente, aplicadas al entorno doméstico para la construcción de hogares inteligentes (“Smart Home”, en inglés). Más particularmente, la presente invención se refiere a un sistema y método para la toma automática de decisiones de forma desatendida usando técnicas de aprendizaje automatizado (“Machine Learning”, en inglés). The present invention has its application in the field of Artificial Intelligence (AI) technologies and, more specifically, applied to the domestic environment for the construction of intelligent homes (“Smart Home”, in English). More particularly, the present invention relates to a system and method for unattended automatic decision making using machine learning techniques.
ANTECEDENTES DE LA INVENCIÓN BACKGROUND OF THE INVENTION
Actualmente existen estrategias y soluciones basadas en el procesado de datos que permiten tomar decisiones relevantes sobre una actividad determinada. Normalmente estas soluciones pertenecen al campo de la Inteligencia Empresarial (“Business Intelligence”, en inglés) y se materializan en una amplia gama de productos. Dichas soluciones permiten tomar decisiones a posteriori una vez analizadas por un experto en la disciplina de la Ciencia de Datos (“Data Science”, en inglés). Por otro lado, la creciente apertura de los ecosistemas de Inteligencia Artificial (IA), la especialización de arquitecturas predictivas basadas en redes neuronales, aumento de la calidad del dato, la creciente capacidad de potencia de cálculo y la fuerte inversión por parte de los gobiernos y la industria privada, ha generado una proliferación exponencial de algoritmos de inferencia dedicados a la resolución de problemas concretos. Currently there are strategies and solutions based on data processing that allow relevant decisions to be made about a given activity. Normally these solutions belong to the field of Business Intelligence (“Business Intelligence”) and materialize in a wide range of products. These solutions allow decisions to be made a posteriori once analyzed by an expert in the discipline of Data Science (“Data Science”, in English). On the other hand, the growing opening of Artificial Intelligence (AI) ecosystems, the specialization of predictive architectures based on neural networks, the increase in data quality, the growing capacity for computing power and the strong investment by governments and private industry, has generated an exponential proliferation of inference algorithms dedicated to solving specific problems.
El problema de todos estos enfoques es su baja generalidad para entender problemas del entorno más complejos que requieren de una relación semántica entre fuentes de datos no obvia. Todo ello unido a la incapacidad de los diferentes sistemas de tomar decisiones de forma autónoma, hacen que exista un espacio a explorar en torno a cómo las predicciones de esos modelos aislados se pueden relacionar generando como resultado un elemento de entendimiento o percepción (“insight”, en inglés) de mayor complejidad semántica, de forma continuada y adecuadas a los acontecimientos de un entorno cambiante. The problem with all these approaches is their low generality to understand more complex environmental problems that require a non-obvious semantic relationship between data sources. All this, together with the inability of the different systems to make decisions autonomously, means that there is a space to explore around how the predictions of these isolated models can be related, generating as a result an element of understanding or perception. (“insight”, in English) of greater semantic complexity, continuously and appropriate to the events of a changing environment.
Esta descripción del problema que se pretende resolver, siendo suficiente, resulta algo abstracta y difícil de visualizar a simple vista. Esto lleva irremediablemente a aplicar la ¡dea de asociación de información acotada a un entorno sobre un escenario real: el hogar. This description of the problem that is intended to be solved, being sufficient, is somewhat abstract and difficult to visualize with the naked eye. This inevitably leads to applying the idea of delimited information association to an environment on a real stage: the home.
Hoy en día, las soluciones presentadas en el ámbito de lo que se denomina Hogar Inteligente (“Smart Home”, en inglés) siguen percibiéndose como una resolución de problemas aislados y, además, reducidos al ámbito de la información recogida por una fuente de datos cautiva o limitada por el ámbito comercial del propio dispositivo. Today, the solutions presented in the field of what is called Smart Home (“Smart Home”) continue to be perceived as a resolution of isolated problems and, furthermore, reduced to the field of information collected by a data source. captive or limited by the commercial scope of the device itself.
El principal problema de calificar cualquier producto o servicio como Inteligente (“Smart”, en inglés) es que la percepción como algo inteligente es algo totalmente subjetivo. Un dispositivo, sistema o método que pretenda ser inteligente por lo menos en el ámbito del hogar, debe contemplar al menos dos tipos de capacidades:The main problem with qualifying any product or service as "Smart" is that the perception as something intelligent is totally subjective. A device, system or method that claims to be intelligent, at least in the home environment, must include at least two types of capabilities:
1.- Sensibilidad para poder percibir de forma inequívoca lo que está sucediendo en cada momento dentro del hogar. 1.- Sensitivity to be able to perceive unequivocally what is happening at all times inside the home.
2.- Capacidad de apertura para poder incorporar nuevas fuentes de información a medida que éstas se van haciendo hueco en el ámbito cotidiano del hogar. 2.- Ability to open up to be able to incorporate new sources of information as they make their way into the daily environment of the home.
Todo ello además aderezado con una serie de líneas rojas aplicadas a la base de inferencia donde la privacidad del usuario debe ser respetada en todo momento. All this is also seasoned with a series of red lines applied to the inference base where the user's privacy must be respected at all times.
Prácticamente la totalidad de las soluciones del estado de la técnica se basan una capa de procesado y agregación de datos en crudo y una pequeña capa de aprendizaje automatizado (ML: Machine Learning, en inglés) para tratar de obtener una serie de percepciones (“insights”) aisladas y con un fin muy concreto. Posteriormente un experto debe realizar una lectura e interpretar dichas percepciones en una dirección determinada. Esto implica la acción humana, lo cual imposibilita la toma de decisiones en tiempo real de forma desatendida. El problema técnico objetivo que se presenta es permitir en un entorno acotado, como puede ser un hogar inteligente, una toma de decisiones y ejecución de acciones en tiempo real de forma desatendida, sin la intervención humana, para obtener un entorno inteligente que puede aprender dinámicamente y hacerse cada vez más sensible/consciente sobre lo que sucede en su interior. Virtually all of the state-of-the-art solutions are based on a layer of raw data processing and aggregation and a small layer of automated learning (ML: Machine Learning, in English) to try to obtain a series of perceptions (“insights”). ”) isolated and with a very specific purpose. Subsequently, an expert must carry out a reading and interpret these perceptions in a certain direction. This implies human action, which makes unattended real-time decision-making impossible. The objective technical problem that is presented is to allow, in a limited environment, such as a smart home, unattended decision-making and execution of actions in real time, without human intervention, to obtain an intelligent environment that can learn dynamically. and become increasingly sensitive/aware of what is going on inside.
DESCRIPCIÓN DE LA INVENCIÓN DESCRIPTION OF THE INVENTION
La presente invención sirve para solucionar el problema mencionado anteriormente, mediante un método para la toma de decisiones y ejecución de acciones de forma desatendida y en tiempo real en el ámbito de un entorno acotado, entendiendo entorno acotado como un ecosistema cerrado cuya contextual ización no es compleja, es decir, los eventos que se pueden dar en dicho entorno son explicables, medibles y parametrizables. El método que aquí se describe es capaz de interpretar situaciones de diversa índole que tienen lugar en dicho entorno acotado mediante la recolección de datos generados por una base de dispositivos hardware perteneciente al entorno a explorar y aplicando algoritmia basada en técnicas de aprendizaje automatizado (ML: Machine Learning, en inglés). The present invention serves to solve the problem mentioned above, by means of a method for unattended decision-making and execution of actions in real time within a bounded environment, understanding the bounded environment as a closed ecosystem whose contextualization is not complex, that is, the events that can occur in said environment are explainable, measurable and parameterizable. The method described here is capable of interpreting situations of various kinds that take place in said bounded environment by collecting data generated by a base of hardware devices belonging to the environment to be explored and applying algorithms based on automated learning techniques (ML: Machine Learning, in English).
Un elemento esencial de la presente invención es un motor de inferencia que, a través de un modelo de asociación de información por capas, genera elementos de entendimiento (“insights”, en inglés) relevantes para la comprensión de los acontecimientos que tienen lugar en el entorno acotado. El resultado de este proceso es un contexto dinámico que permite al propio sistema tomar decisiones de forma desatendida. An essential element of the present invention is an inference engine that, through a layered information association model, generates insights relevant to the understanding of the events that take place in the environment. bounded environment. The result of this process is a dynamic context that allows the system itself to make unattended decisions.
La inclusión de modelos de asociación de información en un entorno acotado, como es el entorno del hogar, viene a derribar la limitación técnica entre silos de información que conviven en el mismo ecosistema. The inclusion of information association models in a limited environment, such as the home environment, brings down the technical limitation between information silos that coexist in the same ecosystem.
La presente invención consigue tal objetivo por su capacidad para asociar fuentes de entendimiento obtenidas por distintos procesos de inferencia en torno a temáticas semánticamente complejas. Esta capacidad es proporcionada por la capa más alta de un modelo de capas, que aquí se denomina Pila de Descubrimiento, que es un modelo de aprendizaje automatizado (modelo de ML) donde cada capa o nivel proporciona una información más refinada al nivel superior. La Pila de Descubrimiento que aquí se define es un modelo que comprende tres capas, ordenadas aquí de menor a mayor nivel: The present invention achieves this objective due to its ability to associate sources of understanding obtained by different inference processes around semantically complex themes. This capability is provided by the topmost layer of a layered model, here called the Discovery Stack, which is a machine learning model (ML model) where each layer or level provides more refined information to the higher level. The Pile of Discovery defined here is a model that comprises three layers, ordered here from lowest to highest level:
- Capa 1 : Generación de Datos, formada por todos los dispositivos pertenecientes al entorno donde se desea realizar el descubrimiento de eventos, cada dispositivo generando una serie de fuentes de contribución de datos. - Layer 1: Data Generation, made up of all the devices belonging to the environment where the discovery of events is to be carried out, each device generating a series of data contribution sources.
- Capa 2: Reconocimiento de Patrones, para adaptar los datos en crudo procedentes de todas las fuentes de contribución de la capa 1 anterior y obtener elementos de inferencia con información relevante para el siguiente nivel o capa. La capa 2 está formada por módulos de preprocesado que aplican técnicas para identificar patrones de reconocimiento de distinta índole empleando algoritmos específicos de ML según la naturaleza de la fuente de datos. - Layer 2: Pattern Recognition, to adapt the raw data from all the contribution sources of the previous layer 1 and obtain inference elements with relevant information for the next level or layer. Layer 2 is made up of preprocessing modules that apply techniques to identify recognition patterns of different kinds using specific ML algorithms depending on the nature of the data source.
- Capa 3: Asociación de Información, formada por diferentes módulos de asociación de información para relacionar elementos inferidos por los niveles y generar al menos un elemento de entendimiento (o conocimiento o percepción) (“insight”) relevante (adecuado semánticamente) para la detección de un evento concreto del entorno acotado. - Layer 3: Information Association, formed by different information association modules to relate elements inferred by the levels and generate at least one element of understanding (or knowledge or perception) (“insight”) relevant (semantically adequate) for detection of a specific event in the bounded environment.
En el contexto de la invención, se define elemento de entendimiento (“insight”) como la unidad mínima de información descubierta en el entorno acotado y entendible por un sistema capaz de llevar a cabo una acción adecuada semánticamente para la ontología de referencia del entorno de acción (entorno acotado). In the context of the invention, an element of understanding (“insight”) is defined as the minimum unit of information discovered in the delimited environment and understandable by a system capable of carrying out a semantically appropriate action for the reference ontology of the environment. action (enclosed environment).
Un aspecto de la invención se refiere a un método para la toma de decisiones y ejecución de acciones desatendida en un entorno acotado, el comprendiendo los pasos definidos en la reivindicación 1. One aspect of the invention refers to a method for unattended decision-making and execution of actions in a bounded environment, comprising the steps defined in claim 1.
Otro aspecto adicional de la presente invención se refiere a un sistema de toma desatendida de decisiones y ejecución desatendida de acciones en un entorno acotado, el sistema comprendiendo los siguientes componentes que realizan los pasos del método descrito anteriormente. Another additional aspect of the present invention refers to a system for unattended decision-making and unattended execution of actions in a bounded environment, the system comprising the following components that carry out the steps of the method described above.
El sistema de la invención está compuesto por distintos módulos (SW: software), distribuidos en las tres capas lógicas definidas anteriormente y que se conectan en cascada. Un módulo se constituye por los medios técnicos que realizan una unidad funcional mínima perteneciente a una capa del sistema. Los distintos módulos se estructuran e interconectan según un modelo (lógico). Y cada capa es una asociación lógica de módulos que llevan a cabo las tareas relacionadas dentro del sistema. The system of the invention is made up of different modules (SW: software), distributed in the three logical layers defined above and which are connected in cascade. A module is constituted by the technical means that carry out a minimum functional unit belonging to a layer of the system. The different modules are structure and interconnect according to a (logical) model. And each layer is a logical association of modules that carry out related tasks within the system.
Las distintas funcionalidades o funciones que se indican en el método anterior están realizadas por dispositivos programadles electrónicos (por ejemplo, un servidor o un nodo cliente) o conjunto de dispositivos programadles electrónicos, que a su vez pueden estar co-localizados o distribuidos en distintas ubicaciones y comunicados por cualquier tipo de red de comunicación cableada o sin cables. El componente (hardware o software) de ese dispositivo electrónico que realiza una determinada funcionalidad es lo que se conoce como módulo. Las distintas funcionalidades o funciones que se indican en el método anterior pueden implementarse como uno o más componentes. Cada funcionalidad puede estar realizada en un dispositivo (o conjunto de ellos) distinto o el mismo dispositivo (o conjunto de ellos) implementar vahas o todas las funcionalidades indicadas; es decir el mismo dispositivo (por ejemplo, un servidor) puede tener vahos módulos, cada uno de ellos realizando una de las funcionalidades o estar dichos módulos distribuidos en distintos dispositivos. Estos componentes y la funcionalidad asociada pueden ser utilizados por sistemas informáticos de cliente, servidor, distribuidos o una red de pares. Estos componentes pueden estar escritos en un lenguaje informático correspondiente a uno o más lenguajes de programación tales como lenguajes funcionales, declarativos, procedimentales, orientados a objetos y similares. Pueden estar vinculados a otros componentes a través de vahas interfaces de programación de aplicaciones e implementarse en una aplicación de servidor o una aplicación de cliente. Alternativamente, los componentes se pueden implementar en aplicaciones tanto de servidor como de cliente. The different functionalities or functions indicated in the previous method are performed by electronic programmable devices (for example, a server or a client node) or set of electronic programmable devices, which in turn can be co-located or distributed in different locations. and communicated by any type of wired or wireless communication network. The component (hardware or software) of that electronic device that performs a certain functionality is what is known as a module. The different functionalities or functions that are indicated in the previous method can be implemented as one or more components. Each functionality can be carried out on a different device (or set of them) or the same device (or set of them) implement vahas or all of the indicated functionalities; that is to say, the same device (for example, a server) can have several modules, each of them carrying out one of the functionalities or these modules can be distributed in different devices. These components and associated functionality may be used by client, server, distributed, or peer-to-peer computing systems. These components may be written in a computer language corresponding to one or more programming languages such as functional, declarative, procedural, object-oriented languages, and the like. They can be linked to other components through various application programming interfaces and implemented in a server application or a client application. Alternatively, the components can be implemented in both server and client applications.
Otro último aspecto de la invención se refiere a un programa de ordenador, que contiene instrucciones o código informático (almacenado en un medio legible por ordenador no transitorio) para hacer que unos medios de procesamiento (de un procesador informático) realicen los pasos del método descrito anteriormente para la toma desatendida de decisiones en un entorno acotado. Another last aspect of the invention relates to a computer program, which contains instructions or computer code (stored on a non-transient computer-readable medium) to cause processing means (of a computer processor) to perform the steps of the described method. previously for unattended decision making in a bounded environment.
Las ventajas de la presente invención frente al estado de la técnica anterior y en relación a los sistemas existentes son fundamentalmente: The advantages of the present invention compared to the prior state of the art and in relation to existing systems are fundamentally:
La presente invención proporciona un contexto dinámico con información de los eventos más relevantes que suceden dentro del entorno acotado. Esto permite que el entorno constituya una entidad inteligente y consciente que resuelve problemas no obvios, sin necesitar ninguna acción del usuario y pudiéndose anticipar a que el usuario actúe. The present invention provides a dynamic context with information on the most relevant events that occur within the bounded environment. This allows the environment to constitute an intelligent and conscious entity that solves non-obvious problems, without requiring any action from the user and being able to anticipate the user's action.
La presente invención permite que el entorno no requiera acción alguna por parte del usuario para realizar un proceso de autenticación, ya que dispone de suficiente información contextual del entorno, y tampoco necesita conocer datos sensibles del usuario. The present invention allows the environment to not require any action by the user to carry out an authentication process, since it has sufficient contextual information about the environment, and does not need to know sensitive user data either.
La presente invención permite que el entorno funcione como una entidad autónoma que garantiza la privacidad de la información manejada, por ejemplo, información perteneciente a un entorno doméstico y, también en este caso, el propio entorno del hogar consigue así autonomía para crear un control parental. The present invention allows the environment to function as an autonomous entity that guarantees the privacy of the information handled, for example, information belonging to a home environment and, also in this case, the home environment itself thus achieves autonomy to create parental control. .
Éstas y otras ventajas se desprenden de la descripción detallada de la invención que se hace a continuación. These and other advantages are apparent from the detailed description of the invention that follows.
BREVE DESCRIPCIÓN DE LAS FIGURAS BRIEF DESCRIPTION OF THE FIGURES
A continuación, se pasa a describir de manera muy breve una serie de dibujos que ayudan a comprender mejor la invención y que se relacionan expresamente con una realización de dicha invención que se presenta como un ejemplo no limitativo de ésta. Next, a series of drawings that help to better understand the invention and that are expressly related to an embodiment of said invention that is presented as a non-limiting example thereof, is briefly described.
FIGURA 1.- Muestra un modelo funcional genérico con las tres capas que conforman la pila del descubrimiento de un sistema inteligente para la toma desatendida de decisiones, según una realización preferente de la invención. FIGURE 1.- Shows a generic functional model with the three layers that make up the discovery stack of an intelligent system for unattended decision-making, according to a preferred embodiment of the invention.
FIGURA 2.- Muestra un diagrama de bloques de la arquitectura del sistema para la toma de decisiones en un entorno acotado, según una realización preferente de la invención. FIGURE 2.- Shows a block diagram of the system architecture for decision-making in a limited environment, according to a preferred embodiment of the invention.
FIGURA 3.- Muestra un detalle de los bloques de generación, almacenamiento y análisis de capturas instantáneas usadas por el sistema para la toma de decisiones. FIGURA 4.- Muestra un diagrama de la unidad lógica usada por el sistema de toma de decisiones, según una posible realización de la invención. FIGURE 3.- Shows a detail of the generation, storage and analysis blocks of snapshots used by the system for decision making. FIGURE 4.- Shows a diagram of the logical unit used by the decision-making system, according to a possible embodiment of the invention.
FIGURA 5.- Muestra un ejemplo de una matriz de relación entre eventos detectados y sentencias lógicas inferidas en el entorno acotado. FIGURE 5.- Shows an example of a relationship matrix between detected events and inferred logical sentences in the bounded environment.
FIGURA 6.- Muestra otro ejemplo de una matriz de relación entre eventos detectados y sentencias lógicas inferidas en el entorno acotado para autenticar a nivel de hogar a un usuario titular de una línea de teléfono móvil. FIGURE 6.- Shows another example of a relationship matrix between events detected and logical sentences inferred in the delimited environment to authenticate a user who owns a mobile telephone line at the household level.
FIGURA 7.- Muestra otro ejemplo de una matriz de relación entre eventos detectados y sentencias lógicas inferidas en el entorno acotado para realizar a nivel de hogar un control parental. FIGURE 7.- Shows another example of a relationship matrix between events detected and logical sentences inferred in the delimited environment to carry out parental control at the household level.
FIGURA 8.- Muestra un mapa de inferencia que asocia anillos de privacidad según nivel de pertenencia al núcleo familiar en un hogar. FIGURE 8.- Shows an inference map that associates privacy rings according to the level of belonging to the nuclear family in a household.
REALIZACIÓN PREFERENTE DE LA INVENCIÓN PREFERRED EMBODIMENT OF THE INVENTION
Una realización preferida de la invención se refiere a un sistema para la toma automática de decisiones de manera desatendida en un entorno acotado. A preferred embodiment of the invention refers to a system for unattended automatic decision-making in a bounded environment.
La Figura 1 muestra esquemáticamente los componentes del sistema propuesto que, en su forma más genérica, consiste en un modelo de capas, donde cada capa o nivel proporciona una información más refinada al nivel superior, según se describe seguidamente. Figure 1 schematically shows the components of the proposed system which, in its most generic form, consists of a layered model, where each layer or level provides more refined information to the higher level, as described below.
- Una primera capa 110 para la generación de datos, en el nivel más bajo o inferior: Esta primera capa 110 está formada por todos los dispositivos DISP_1, DISP_2, DISP_i, pertenecientes al entorno donde se desea realizar el descubrimiento de eventos. Cada dispositivo DISP_i, siendo DISP_i el i-ésimo dispositivo , genera una serie de fuentes de datos. Un dispositivo DISP_i puede dar lugar a una o vahas fuentes de datos cada fuente de datos queda denotada, para el i-ésimo dispositivo DISP_i, como FD_¡1 , .. FD_ij. Cada fuente de datos se clasifica por la naturaleza del elemento sensible que recoge los datos (el subíndice j se usa para denotar la j-esima fuente de datos); por ejemplo, típicamente, las fuentes de datos pueden ser fuentes de audio, de vídeo o incluso telemetría relacionada con algún sensor de temperatura, presión o humedad, o también fuentes de datos más sofisticadas, tales como la posición de un terminal móvil, el nivel de intensidad de la señal del enrutador WiFi, la telemetría interna de un decodificador de señal de vídeo o incluso el nivel de batería de un altavoz inalámbrico. - A first layer 110 for data generation, at the lowest or lowest level: This first layer 110 is made up of all the devices DISP_1, DISP_2, DISP_i, belonging to the environment where it is desired to perform event discovery. Each DISP_i device, DISP_i being the ith device, generates a number of data sources. A DISP_i device can give rise to one or several data sources, each data source is denoted, for the ith DISP_i device, as FD_¡1, .. FD_ij. Each data source is classified by the nature of the sensitive element that collects the data (the subscript j is used to denote the j-th data source); For example, typically, the data sources can be audio, video or even telemetry sources related to some temperature, pressure or humidity sensor, or also more sophisticated data sources, such as the position of a mobile terminal, the level Wi-Fi router signal strength, internal telemetry from a video set-top box, or even the battery level of a wireless speaker.
- Una segunda capa 120 para el reconocimiento de patrones, en el nivel intermedio, entre la primera capa 110 y la última o tercera capa 130: - A second layer 120 for pattern recognition, at the intermediate level, between the first layer 110 and the last or third layer 130:
En esta segunda capa 120, con el objetivo de adaptar los datos en crudo procedentes de todas las fuentes de datos de contribución anteriormente citadas, aparecen los módulos de preprocesado. Hay un módulo de preprocesado por cada tipo de fuente de datos; por ejemplo, el módulo de preprocesado PREP_p puede adaptar los datos de una o más de las fuentes de datos FD_1j,...,FD_ij. Cada módulo de preprocesado PREP_p acomoda uno o varias fuentes a una técnica de reconocimiento TR_k determinada. Todas estas técnicas de reconocimiento TR_1, TR_2,...TR_k tienen la misión de identificar patrones de reconocimiento de distinta índole empleando algoritmos específicos de aprendizaje automatizado (ML) según la naturaleza de la fuente de datos. Varias fuentes pueden contribuir a nutrir una o varias técnicas de reconocimiento (el subíndice k se usa para denotar la k-esima técnica de reconocimiento de patrones. El resultado es un elemento de inferencia INF_1, INF _2,... INF _k con información relevante para el siguiente nivel o capa. Un elemento de inferencia es proporcionado por una técnica de reconocimiento. In this second layer 120, with the aim of adapting the raw data coming from all the aforementioned contribution data sources, the preprocessing modules appear. There is a preprocessing module for each type of data source; for example, the PREP_p preprocessing module may adapt data from one or more of the data sources FD_1j,...,FD_ij. Each PREP_p preprocessing module accommodates one or more sources to a given recognition technique TR_k. All these recognition techniques TR_1, TR_2,...TR_k have the mission of identifying recognition patterns of different kinds using specific machine learning (ML) algorithms depending on the nature of the data source. Several sources can contribute to nurture one or several recognition techniques (the subscript k is used to denote the kth pattern recognition technique. The result is an inference element INF_1, INF _2,... INF _k with relevant information to the next level or layer An inference element is provided by a recognition technique.
- Una tercera capa 130 para la asociación de información, en el nivel más alto o superior: - A third layer 130 for the association of information, at the highest or superior level:
Esta tercera capa 130, en esencia, está formada por diferentes módulos de asociación de información. Cada uno de estos módulos emplea técnicas de asociación de información TAI_1, TAI_2,..., TAI_I específicas (el subíndice I se usa para denotar la l-esima técnica de asociación de información) para relacionar elementos de inferencia INF_1, INF _2,... INF _k por el nivel inferior a este último nivel. Una técnica de asociación de información puede ser nutrida por varios elementos de inferencia. En este aspecto, el criterio de asociación seguido en cada módulo de asociación de información va dirigido en una línea determinada dependiendo de la naturaleza del proceso de descubrimiento que se pretenda abordar. El resultado final es la generación de un elemento de entendimiento, INS_1, INS_2..., INS_I o”insight” relevante para la detección de un evento concreto del entorno. El elemento de entendimiento es la unidad mínima de información entendióle descubierta en el entorno acotado de acción, es decir, los ladrillos sobre los que se construyen casos de uso relevantes para una determinada tarea. Un elemento de entendimiento (“Insight”) es proporcionado por una técnica de asociación de información. This third layer 130, in essence, is made up of different information association modules. Each of these modules employs specific information association techniques TAI_1, TAI_2,..., TAI_I (the subscript I is used to denote the lth information association technique) to relate inference elements INF_1, INF_2,. .. INF _k by the level below this last level. An information association technique can be nourished by various elements of inference. In this regard, the criterion of association followed in each information association module is directed in a certain line depending on the nature of the discovery process that is intended to be addressed. The final result is the generation of an understanding element, INS_1, INS_2..., INS_I or "insight" relevant to the detection of a specific event in the environment. The element of understanding is the minimum unit of information discovered in the bounded environment of action, that is, the bricks on which relevant use cases are built for a given task. An element of understanding (“Insight”) is provided by an information association technique.
La siguiente tabla, Tabla 1 , muestra algunos ejemplos de estrategias de algoritmos de ML, las técnicas de modelos ML del estado de la técnica aplicables (técnicas SOTA: State-of-the-art, en inglés) y su ámbito de aplicación dentro del modelo de capas anteriormente descrito, así como la fuente o fuentes de datos que contribuyen a generar una base de descubrimiento. The following table, Table 1 , shows some examples of ML algorithm strategies, the applicable state-of-the-art ML modeling techniques (SOTA techniques: State-of-the-art, in English) and their scope within the field of application. layer model described above, as well as the data source(s) that contribute to generating a discovery base.
Tabla 1 - Ejemplos de técnicas de ML y su ámbito de aplicación en el modelo de capas de la Pila de Descubrimiento
Figure imgf000012_0001
Table 1 - Examples of ML techniques and their scope in the layered model of the Discovery Stack
Figure imgf000012_0001
Como caso práctico, la traducción del modelo anterior de capas que forman la Pila de Descubrimiento en el entorno del hogar es, según un ejemplo: As a practical case, the translation of the previous model of layers that make up the Discovery Stack in the home environment is, according to an example:
La primera capa 110 para la generación de datos es el ecosistema de dispositivos del hogar, formada por los dispositivos más comunes que componen el ecosistema de un hogar estándar. El objetivo de esta capa es determinar qué fuentes de datos procedentes de cada dispositivo pueden contribuir activamente en el descubrimiento de acontecimientos; por ejemplo, una cámara IP contribuye con una fuente de vídeo, un enrutador contribuye con datos de nivel de intensidad de señal WiFi y con información acerca de los dispositivos que están conectados en un momento determinado a la red del hogar, etc. The first layer 110 for data generation is the ecosystem of home devices, made up of the most common devices that make up the ecosystem of a standard home. The goal of this layer is to determine which data sources from each device can actively contribute to event discovery; For example, an IP camera contributes a video source, a router contributes WiFi signal strength level data and information about the devices that are currently connected to the home network, etc.
La segunda capa 120 para el reconocimiento de patrones se centra en el reconocimiento de patrones de actividades concretas relacionadas con el día a día de un hogar, apoyándose para ello en técnicas de reconocimiento de patrones en una señal de vídeo, de audio, de la señal WiFi procedente del enrutador o incluso técnicas de reconocimiento de patrones relacionadas con el tráfico cursado dentro de la red del hogar. La tercera capa 130 para para la asociación de información combina los patrones simples del nivel anterior mediante algoritmos de ML que permiten inferir patrones más complejos. En concreto, para la casuística del hogar, se pueden usar algoritmos de asociación en torno al entendimiento de actividades humanas (Reconocimiento de Actividad Humana) y la comprensión de escenas (Comprensión de escenario). Todo ello con el fin de definir los campos más relevantes que permiten generar los elementos de entendimiento o “insights” relevantes, en este caso, relacionados con eventos descubiertos en el entorno del hogar. The second layer 120 for pattern recognition focuses on pattern recognition of specific activities related to the day-to-day life of a home, relying on pattern recognition techniques in a video signal, audio signal, the WiFi signal from the router or even pattern recognition techniques related to traffic carried within the home network. home. The third layer 130 for the association of information combines the simple patterns of the previous level by means of ML algorithms that allow more complex patterns to be inferred. Specifically, for the casuistry of the home, association algorithms can be used around the understanding of human activities (Human Activity Recognition) and the understanding of scenes (Scenario Understanding). All this in order to define the most relevant fields that allow the generation of relevant elements of understanding or “insights”, in this case, related to events discovered in the home environment.
La Figura 2 muestra un resumen de la arquitectura planteada para dar forma al sistema de toma desatendida de decisiones para un entorno determinado. Los primeros tres componentes, mapa de dispositivos 210, fuentes de datos 220 y módulos de preprocesado 230, implementan las dos primeras capas, 110 y 120, del modelo de la pila de descubrimiento mostrado en la Figura 1, cuya misión es: acomodar las diferentes fuentes de datos al formato de procesado correcto, extraer patrones sencillos empleando técnicas de ML y proporcionar al siguiente nivel, la tercera capa 130 de asociación de información, elementos de inferencia suficientes para poder construir “insights” entendióles y relevantes para el entorno a explorar. La pieza clave del sistema es el motor de inferencia 240, que es el encargado de generar una captura instantánea 250 o “snapshot” de lo que sucede en el hogar en cada momento (i.e., un “snapshot” recoge todos los “insights”), generándose un “snapshot” cada cierto período de tiempo, por ejemplo, cada x segundos. El sistema propuesto es capaz de almacenar los “snapshots” generados en una línea temporal a modo de histórico, de tal manera que con un análisis a medio-largo plazo de “snapshots” almacenados se realiza un descubrimiento de patrones 260, los cuales se vuelven a inyectar como parte de las conclusiones de la siguiente captura instantánea 250 o “snapshot”. Esta información de la captura instantánea 250 generada en un instante de tiempo pasa a un automáta 270, que es el módulo que aporta al sistema la capacidad de toma desatendida de decisiones, para lo que parte de dos conceptos fundamentales: i) las señales de activación o “triggers”, TRIG_1, TRIG_2,... TRIG_y, definidas como el conjunto de señales lógicas procedentes de las conclusiones inferidas y recogidas en el snapshot 250, que permite desencadenar una secuencia de acciones de forma desatendida; y ¡i) las acciones ACT_1, ACT_2, ACT_n, que el sistema es capaz de ejecutar sobre el entorno acotado en cuestión y que un actuador 280 genera a partir de las señales de activación TRIG_1, TRIG_2,... TRIG_y recibidas, definiendo las acciones ACT_1, ACT_2, ACT_n como un conjunto de operaciones que se realizan sobre diferentes dispositivos electrónicos 290 del hogar, por ejemplo, teléfonos, alarmas, ordenadores,... Figure 2 shows a summary of the architecture proposed to shape the unattended decision-making system for a given environment. The first three components, device map 210, data sources 220 and preprocessing modules 230, implement the first two layers, 110 and 120, of the discovery stack model shown in Figure 1, whose mission is: to accommodate the different data sources to the correct processing format, extract simple patterns using ML techniques and provide the next level, the third information association layer 130, with sufficient inference elements to be able to build "insights" that are understood and relevant to the environment to be explored. The key piece of the system is the inference engine 240, which is responsible for generating an instantaneous capture 250 or "snapshot" of what is happening in the home at all times (ie, a "snapshot" collects all the "insights") , generating a "snapshot" every certain period of time, for example, every x seconds. The proposed system is capable of storing the "snapshots" generated in a time line as a history, in such a way that with a medium-long term analysis of stored "snapshots" a discovery of patterns 260 is made, which become to be injected as part of the conclusions of the next snapshot 250 or "snapshot". This information from the instantaneous capture 250 generated at an instant of time passes to an automata 270, which is the module that provides the system with the capacity for unattended decision-making, for which It is based on two fundamental concepts: i) the activation signals or "triggers", TRIG_1, TRIG_2,... TRIG_y, defined as the set of logical signals coming from the inferred conclusions and collected in the snapshot 250, which allows triggering a sequence of actions unattended; and ii) the actions ACT_1, ACT_2, ACT_n, that the system is capable of executing on the bounded environment in question and that an actuator 280 generates from the activation signals TRIG_1, TRIG_2,... TRIG_y received, defining the actions ACT_1, ACT_2, ACT_n as a set of operations that are carried out on different electronic devices 290 in the home, for example, telephones, alarms, computers,...
La captura instantánea 250 o “snapshot” puede entenderse como un informe desglosado de todos los acontecimientos que están teniendo lugar en el entorno de exploración, tanto en el plano físico como en el virtual, entendiendo el plano virtual como todo lo que sucede como consecuencia de una transacción digital de información. Una característica importante del snapshot, es que permite la retroalimentación de elementos de entendimiento o “insights”, es decir, a parte de incluir conclusiones en tiempo real, se almacenan los snapshots SN_1, SN_2, SN_x, generados en una línea temporal a modo de histórico de capturas instantáneas 255, de tal manera que un procesado a posteriori extrae los patrones 260 de comportamiento en el medio plazo. Dichos patrones son incluidos nuevamente en la toma de decisiones para el entorno en cuestión volviéndolos a entrar como parte de las conclusiones de la captura instantánea a generar en el instante de tiempo siguiente, snapshot SN_x+1, como se representa en la retro- alimentación de información que muestra la Figura 3. La Figura 3 muestra un caso particular de tratamiento de información para la extracción de patrones de medio y largo plazo, que luego se combinan con los insights extraídos en tiempo real del entorno, realimentando así el motor de inferencia para proporcionar mayor entendimiento, es decir, se constituiría así un “detector de costumbres” de usuario. The snapshot 250 or "snapshot" can be understood as a detailed report of all the events that are taking place in the exploration environment, both in the physical and in the virtual plane, understanding the virtual plane as everything that happens as a consequence of a digital transaction of information. An important characteristic of the snapshot is that it allows the feedback of elements of understanding or "insights", that is, apart from including conclusions in real time, the snapshots SN_1, SN_2, SN_x are stored, generated in a time line as a history of instantaneous captures 255, in such a way that a posteriori processing extracts the patterns 260 of behavior in the medium term. These patterns are again included in the decision making for the environment in question by re-entering them as part of the snapshot conclusions to be generated at the next time instant, snapshot SN_x+1, as represented in the feedback from information shown in Figure 3. Figure 3 shows a particular case of information processing for the extraction of medium and long-term patterns, which are then combined with the insights extracted in real time from the environment, thus feeding back the inference engine to provide greater understanding, that is, a user "customs detector" would thus be constituted.
El motor de inferencia 240, pieza clave del sistema mostrado en la Figura 2, está compuesto por diferentes piezas lógicas que es preciso detallar para poder comprender su funcionamiento. El motor de inferencia 240 comprende un mapa de inferencia, que es un elemento de algoritmia para combinar información extraída del procesado de distintas fuentes de información y con ello deducir los elementos de entendimiento o “insights”, los cuales van emergiendo a medida que se añade un nuevo eslabón predictivo a la cadena. En su forma más abstracta, cada eslabón de la cadena se organiza como un nodo dentro de un árbol de decisión y recibe el nombre de unidad lógica UL_r. El subíndice r se usa para denotar la r-ésima unidad lógica, cuestión lógica, evento base y el subíndice s para el s-ésimo modelo de inferencia por unidad lógica. The inference engine 240, a key part of the system shown in Figure 2, is made up of different logical parts that must be detailed in order to understand its operation. The inference engine 240 comprises an inference map, which is an algorithm element for combining information extracted from the processed from different sources of information and thereby deduce the elements of understanding or "insights", which emerge as a new predictive link is added to the chain. In its most abstract form, each link in the chain is organized as a node within a decision tree and is called a logical unit UL_r. The subscript r is used to denote the rth logical unit, logical question, base event, and the subscript s for the sth inference model per logical unit.
La Figura 4 recoge esquemáticamente el diagrama de una unidad lógica estándar. Cada unidad lógica UL_r está formada por los siguientes elementos: Figure 4 schematically collects the diagram of a standard logical unit. Each logical unit UL_r is made up of the following elements:
Cuestión lógica CUE_r: se trata de una cuestión lógica simple a responder mediante el procesado de información procedente del entorno. Logical question CUE_r: This is a simple logical question to be answered by processing information from the environment.
Evento base EVENTO_r: define el evento base sobre el que se construye el patrón de reconocimiento y por tanto, el esquema de clasificación que siguen todos los modelos de inferencia, MOD_r1, MOD_rs, que contribuyen a responder la cuestión lógica CUE_r planteada. El enfoque que sigue la combinación de modelos sigue un patrón multi-modal, donde cada modelo de inferencia de entrada, MOD_r1, MOD_rs, resuelve un problema de decisión en dominios de datos diferentes, por ejemplo, combinando modelos entrenados con fuentes de vídeo, de audio, etc, que sean relevantes para responder la pregunta lógica contenida en la cuestión lógica CUE_r. Base event EVENT_r: defines the base event on which the recognition pattern is built and, therefore, the classification scheme followed by all the inference models, MOD_r1, MOD_rs, which contribute to answering the logical question CUE_r posed. The model combination approach follows a multi-modal pattern, where each input inference model, MOD_r1, MOD_rs, solves a decision problem in different data domains, for example, by combining models trained with video sources, from audio, etc, that are relevant to answering the logical question contained in the logical question CUE_r.
Sentencia Lógica SL_r: El resultado del problema de inferencia está expresado en términos de probabilidades numéricas, por tanto, es necesaria una traducción al dominio de lenguaje entendióle por un humano. En este sentido, se define Sentencia Lógica SL_r como la máxima expresión de afirmación sobre la cuestión lógica CUE_r que en términos de información puede llegarse a inferir del evento base EVENTO_r. Logical Sentence SL_r: The result of the inference problem is expressed in terms of numerical probabilities, therefore, a translation to the language domain understood by a human is necessary. In this sense, Logical Sentence SL_r is defined as the maximum expression of affirmation on the logical question CUE_r that in terms of information can be inferred from the base event EVENTO_r.
El error Err (cuE_r) que se comete en cada predicción está acotado a la definición de la cuestión inicial CUE_r y el nivel de dificultad en la predicción del evento base EVENTO_ r, el cual delimita la cantidad de información que se puede llegar a extraer del entorno y por tanto, con la que se puede contestar a la pregunta inicial CUE_r. Dicho formalmente, esto es la fórmula Err (cuE_r) = Inf (EVENTO^) - Inf (cuE_r), donde la función lnf() define el nivel de información o contenido semántico del elemento lógico del que se trate, cuestión CUE_r o evento EVENTO_r. The error Err (cuE_r) made in each prediction is limited to the definition of the initial question CUE_r and the level of difficulty in the prediction of the base event EVENT_ r, which delimits the amount of information that can be extracted from the environment and therefore, with which the initial question CUE_r can be answered. Formally stated, this is the formula Err (cuE_r) = Inf (EVENT^) - Inf (cuE_r), where the lnf() function defines the level of information or semantic content of the logical element in question, question CUE_r or event EVENT_r .
De hecho, otra manera de verlo es que la sentencia lógica SL_r que infiere la unidad lógica UL_r es la máxima cantidad de información que puede obtenerse del evento base EVENTO_r para tratar de responder la cuestión inicial CUE_r; i.e., lnf(EVENTo_r) = SL_r. In fact, another way of looking at it is that the logical statement SL_r that the logical unit UL_r infers is the maximum amount of information that can be obtained from the base event EVENT_r to try to answer the initial question CUE_r; i.e., lnf(EVENT_r) = SL_r.
Sustituyendo esa expresión en la fórmula anterior, queda: Err (cuE_r) = SL_r - Inf (cUE_r) Substituting that expression in the previous formula, it remains: Err (cuE_r) = SL_r - Inf (cUE_r)
Por tanto, el error semántico cometido en el proceso de inferencia y que viene dado por Err() medido sobre la cuestión lógica que se pretende responder es igual a la diferencia semántica entre la información extraída de la cuestión inicial y la sentencia lógica inferida por la unidad lógica. Y, en cualquier caso, este el error semántico está acotado en cuanto a que siempre puede encontrarse una cuestión lógica menos ambiciosa acerca del entorno y siempre puede escogerse un mejor evento base para el modelado del esquema predictivo. Therefore, the semantic error committed in the inference process and which is given by Err() measured on the logical question that is intended to be answered is equal to the semantic difference between the information extracted from the initial question and the logical sentence inferred by the logical drive. And, in any case, this semantic error is bounded in that a less ambitious logical question about the environment can always be found and a better base event can always be chosen for the modeling of the predictive schema.
La potencia de este planteamiento se alcanza mediante la combinación de vahas unidades lógicas en un esquema de decisión tipo árbol o grafo, donde las últimas hojas o nodos extremo alcanzan niveles de dificultad predictiva considerables, sin ver por ello afectada la complejidad de la arquitectura del sistema. En otras palabras, la arquitectura propuesta garantiza la escalabilidad de la solución. The power of this approach is achieved by combining various logical units in a tree or graph-type decision scheme, where the last leaves or extreme nodes reach considerable levels of predictive difficulty, without affecting the complexity of the system architecture. . In other words, the proposed architecture guarantees the scalability of the solution.
Para tratar de entender el modelo anterior, se explica aquí un caso práctico: en el entorno del hogar. El mapa de inferencia para el ecosistema del hogar es un árbol de decisión que trata de responder a cuestiones lógicas sencillas relacionadas con el Quién, Qué, Dónde y Cómo. Un ejemplo del comienzo del árbol de decisión diseñado para un entorno de hogar estándar puede empezar con la exploración del entorno preguntando acerca de quién se encuentra en escena y para ello se quiere saber si existe movimiento, si existe presencia humana, etc. Por ejemplo, con dispositivos detectores de movimiento se determina si hay cuerpos en movimientos en el hogar. Si es así, intervienen detectores de presencia humana para determinar si hay personas en el hogar. Y, si no, se puede determinar si hay mascotas en el hogar mediante detectores de animales, para concluir si alguna mascota se está moviendo por el hogar. Si existe domótica inteligente, además se puede identificar si algún dispositivo de domótica ha realizado un desplazamiento o bien el movimiento ha sido de un objeto movido accidentalmente o una falsa alarma, usando detectores de dispositivos móviles. A la vez, si los detectores de movimiento detectaron personas moviéndose, usando detectores de grupos se puede determinar si hay varias personas o solo una en el hogar. Así, el árbol de decisión, partiendo de premisas simples, va deduciendo situaciones de mayor complejidad. En resumidas cuentas, los nodos del árbol, que son las unidades lógicas UL_r, más alejados se van alimentando de los patrones inferidos del nivel anterior para llegar a conclusiones más sofisticadas acerca del entorno. To try to understand the previous model, a practical case is explained here: in the home environment. The home ecosystem inference map is a decision tree that tries to answer simple logical questions related to Who, What, Where and How. An example of the beginning of the decision tree designed for a standard home environment might begin with scanning the environment by asking about who is on the scene and what is wanted. know if there is movement, if there is human presence, etc. For example, with motion detector devices it is determined if there are moving bodies in the home. If so, human presence detectors intervene to determine if there are people in the home. And, if not, it can be determined if there are pets in the home through animal detectors, to conclude if any pet is moving around the home. If there is intelligent home automation, it can also be identified if any home automation device has made a movement or the movement has been from an accidentally moved object or a false alarm, using mobile device detectors. At the same time, if the motion detectors detected people moving, using group detectors it can be determined if there are several people or just one person in the home. Thus, the decision tree, starting from simple premises, gradually deduces more complex situations. In short, the nodes of the tree, which are the UL_r logical units, further away feed on the patterns inferred from the previous level to reach more sophisticated conclusions about the environment.
A continuación, se describen etapas más avanzadas del árbol de inferencia en su vertiente del Quién, terminando con una matriz de relación donde se plantean sentencias lógicas no triviales acerca del entorno. Next, more advanced stages of the inference tree are described in its Who side, ending with a relationship matrix where non-trivial logical statements about the environment are posed.
Etapa para descubrir en el hogar miembros de la familia: Stage to discover family members at home:
Si en la etapa anterior del mapa de inferencia se ha determinado que hay una o más personas en el hogar, el sistema es capaz de determinar si la persona pertenece a la familia y si lleva o no el móvil encima a través de la información obtenida de la red inalámbrica WLAN del hogar. If in the previous stage of the inference map it has been determined that there are one or more people in the home, the system is capable of determining whether the person belongs to the family and whether or not they have the mobile with them through the information obtained from home wireless WLAN network.
Si el sistema ha descubierto en la etapa anterior que hay una persona sola en el hogar y la red WLAN no detecta ningún dispositivo nuevo conectado, el sistema deduce que la persona está sola y sin el móvil. Si además la red WLAN tampoco detecta un dispositivo previamente registrado, se concluye que esa persona no es miembro de la familia. Pero si la red WLAN encuentra un nuevo dispositivo conectado que además estaba previamente registrado, se concluye que la persona que está sola es un miembro de la familia. If the system has discovered in the previous stage that there is a single person at home and the WLAN network does not detect any new connected device, the system deduces that the person is alone and without the mobile. If, in addition, the WLAN network does not detect a previously registered device either, it is concluded that that person is not a member of the family. But if the WLAN network finds a new connected device that was also previously registered, it is concluded that the person who is alone is a family member.
Cuando el sistema ha descubierto en la etapa anterior que hay un grupo de personas en el hogar y la red WLAN detecta un nuevo dispositivo conectado que además estaba previamente registrado, el sistema deduce que alguien del grupo es miembro de la familia. Si la red WLAN no detecta ningún dispositivo nuevo conectado, el sistema determina que alguien del grupo no es miembro de la familia o puede serlo pero no lleva el móvil encima; si además no se encuentra el dispositivo previamente registrado, se concluye que alguien del grupo es ajeno a la familia. When the system has discovered in the previous stage that there is a group of people at home and the WLAN network detects a new connected device that In addition, it was previously registered, the system deduces that someone in the group is a member of the family. If the WLAN network does not detect any new connected device, the system determines that someone in the group is not a member of the family or may be but does not have the mobile with them; if, in addition, the previously registered device is not found, it is concluded that someone in the group is not part of the family.
Etapa para descubrir la edad y género de las personas extrañas en el hogar: Stage to discover the age and gender of the strange people in the home:
Cuando en la etapa anterior del mapa de inferencia se ha determinado que hay un grupo de personas en el hogar donde alguna es ajena a la familia, el sistema es capaz de distinguir si esa persona ajena es un niño o un adulto usando técnicas de reconocimiento de edad y determinar si la persona (niño o adulto) que está en el grupo (acompañada) ajena a la familia es hombre o mujer, mediante técnicas de clasificación de género como las de la Tabla 1 anterior. When in the previous stage of the inference map it has been determined that there is a group of people in the home where some of them are foreign to the family, the system is capable of distinguishing if that foreign person is a child or an adult using recognition techniques. age and determine if the person (child or adult) who is in the group (accompanied) outside the family is male or female, using gender classification techniques such as those in Table 1 above.
Etapa para descubrir qué personas de la familia hay en el hogar: Stage to discover which family members are in the home:
Cuando en la etapa anterior de descubrimiento de miembros de la familia en el hogar se ha determinado que alguien del grupo es del núcleo familiar, usando técnicas de reconocimiento de edad y de clasificación de género como se ha mencionado anteriormente, el sistema es capaz de distinguir si la persona de la familia que está en el grupo es un niño (y, por tanto, según su género, por ejemplo, concluir si es el hijo o la hija) o un adulto (hombre o mujer, el marido o la esposa) que, por tanto, está en el hogar acompañado. When in the previous stage of discovery of family members in the household it has been determined that someone in the group is from the family nucleus, using age recognition and gender classification techniques as mentioned above, the system is able to distinguish if the person in the family who is in the group is a child (and, therefore, according to their gender, for example, conclude if it is the son or the daughter) or an adult (man or woman, husband or wife) who, therefore, is accompanied at home.
Etapa para descubrir qué persona de la familia está sin compañía en el hogar: Stage to discover which person in the family is unaccompanied at home:
Cuando en la etapa anterior de descubrimiento de miembros de la familia en el hogar se ha determinado que hay una sola persona en el hogar, las técnicas de clasificación de género y de reconocimiento de edad discriminando entre mayores de 18 años o menores permiten al sistema igualmente distinguir si la persona que está sola es un niño (el hijo o la hija) o un adulto (el marido o la esposa) o, si es mayor de cierta edad (por ejemplo, usando un segundo rango de discriminación establecido en 70 años), determinar si una persona mayor de la familia (por ejemplo, el abuelo o la abuela) es quien está en el hogar sin compañía. When in the previous stage of discovery of family members in the home it has been determined that there is only one person in the home, the gender classification and age recognition techniques discriminating between those over 18 years of age or under allow the system to equally distinguish if the person who is alone is a child (the son or the daughter) or an adult (the husband or the wife) or, if he is older than a certain age (for example, using a second range of discrimination set at 70 years), determine if an older person in the family (for example, the grandfather or the grandmother) is the one who is in the home without company.
Etapa para descubrir qué persona ajena a la familia está sola en el hogar: Stage to discover which person outside the family is alone in the home:
De forma similar, el sistema usa las técnicas de clasificación de género y de reconocimiento de edad discriminando entre mayores de 18 años o menores, para determinar si es un niño o niña o un adulto hombre o mujer la persona que, en la etapa anterior de descubrimiento de miembros de la familia en el hogar, se ha concluido que está sola sin ser de la familia. Similarly, the system uses gender classification and age recognition techniques, discriminating between those over 18 years of age or under, to determine whether the person who, in the previous stage of discovery of family members in the home, it has been concluded that she is alone without being part of the family.
La Figura 5 muestra finalmente en forma de matriz, partiendo de la vertiente o cuestión lógica de Quién, la relación entre eventos base que se han descubierto mediante las técnicas de predicción y las sentencias lógicas no triviales inferidas en relación al entorno del hogar descrito. La matriz en el ejemplo mostrado tiene los siguientes eventos detectados como base dispuestos en columnas de la matriz de relación y las siguientes sentencias lógicas en filas: Figure 5 finally shows in matrix form, starting from the logical aspect or question of Who, the relationship between base events that have been discovered through prediction techniques and the non-trivial logical sentences inferred in relation to the described home environment. The matrix in the example shown has the following base detected events arranged in columns of the relationship matrix and the following logical statements in rows:
Evento e1 : Hay un menor varón ajeno a la familia en casa acompañado Event e1: There is a male minor from outside the family at home accompanied
Evento e2: Hay una menor ajena a la familia en casa acompañada Event e2: There is a minor outside the family at home accompanied
Evento e3: Hay un hombre adulto ajeno a la familia en casa acompañadoEvent e3: There is an adult male from outside the family at home accompanied
Evento e4: Hay una mujer adulta ajena a la familia en casa acompañadaEvent e4: There is an adult woman from outside the family at home accompanied
Evento e5: Mi hijo está en casa acompañado Event e5: My son is at home accompanied
Evento e6: Mi hija está en casa acompañada Event e6: My daughter is at home accompanied
Evento e7: Mi marido está en casa acompañado Event e7: My husband is at home accompanied
Evento e8: Mi mujer está en casa acompañada Event e8: My wife is at home accompanied
Sentencia s1: Mis hijos están solos en casa Sentence s1: My children are home alone
Sentencia s2: Madre está en casa con adultos ajenos a la familia Sentence s2: Mother is at home with non-family adults
Sentencia s3: Padre está en casa con adultos ajenos a la familia Statement s3: Father is at home with non-family adults
Sentencia s4: Mi hijo está en casa con menores ajenos a la familia Judgment s4: My son is at home with minors from outside the family
Sentencia s5: Mi hijo está en casa con un menor ajeno a la familia Sentencia s6: Mi hijo está en casa con una menor ajena a la familia Sentencia s7: Mi hija está en casa con menores ajenos a la familia Sentencia s8: Mi hija está en casa con un menor ajeno a la familia Sentencia s9: Mi hija está en casa con una menor ajena a la familia Sentencia s10: Hay una mujer adulta en casa ajena a la familia con mis hijosSentence s5: My son is at home with a minor outside the family Sentence s6: My son is at home with a minor outside the family Sentence s7: My daughter is at home with minors outside the family Sentence s8: My daughter is at home with a minor outside the family Sentence s9: My daughter is at home with a minor outside the family Sentence s10: There is a woman adult at home outside the family with my children
Sentencia s11 : Hay una mujer adulta en casa ajena a la familia con mi hijoSentence s11: There is an adult woman at home outside the family with my son
Sentencia s12: Hay una mujer adulta en casa ajena a la familia con mi hijaSentence s12: There is an adult woman at home outside the family with my daughter
Sentencia s13: Hay un hombre adulto en casa ajeno a la familia con mis hijosSentence s13: There is an adult man at home outside the family with my children
Sentencia s14: Hay un hombre adulto en casa ajeno a la familia con mi hijoSentence s14: There is an adult male at home outside the family with my son
Sentencia s15: Hay un hombre adulto en casa ajeno a la familia con mi hijaSentence s15: There is an adult male outside the family at home with my daughter
Sentencia s16: Madre e hijos están solos en casa Sentence s16: Mother and children are home alone
Sentencia s17: Madre e hijo están solos en casa Sentencia s18: Madre e hija están solas en casa Sentencia s19: Padre e hijos están solos en casa Sentencia s20: Padre e hijo están solos en casa Sentencia s21 : Padre e hija están solos en casa Sentencia s22: Padre y Madre están solos en casa Sentence s17: Mother and son are home alone Sentence s18: Mother and daughter are home alone Sentence s19: Father and sons are home alone Sentence s20: Father and son are home alone Sentence s21: Father and daughter are home alone Sentence s22: Father and Mother are home alone
El resto de las vertientes del árbol o mapa de inferencia, vertientes de las cuestiones iniciales de Dónde, Qué y Cómo, se construyen de forma similar a lo explicado para vertiente del Quién. Y con todo, usando el contexto de los dispositivos, la detección de los objetos inferidos y un modelo supervisado, por ejemplo, se establecen los elementos de entendimiento o “insights” de los siguientes lugares del hogar: garaje, puerta principal de la casa, salón, sala de estar, cocina y jardín. The rest of the aspects of the tree or inference map, aspects of the initial questions of Where, What and How, are constructed in a similar way to that explained for the Who aspect. And with everything, using the context of the devices, the detection of the inferred objects and a supervised model, for example, the elements of understanding or "insights" of the following places in the home are established: garage, front door of the house, hall, living room, kitchen and garden.
La asociación de los “insights”, obtenidos a través de los distintos procesos de inferencia, proporcionada por la tercera capa 130 del modelo de capas de la Pila de Descubrimiento descrito anteriormente presenta numerosas ventajas diferenciales frente a las soluciones existentes en el entorno del hogar y que se muestras a continuación. The association of the "insights", obtained through the different inference processes, provided by the third layer 130 of the layered model of the Discovery Stack described above, presents numerous differential advantages compared to existing solutions in the home environment and which are shown below.
Información contextual dinámica en el entorno del hogar Dynamic contextual information in the home environment
La Pila de Descubrimiento, en cada nivel de inferencia, lanza una batería de modelos predictivos predeterminados basados en aprendizaje profundo (DL: Deep Learning, en inglés) que, combinados con una lógica adicional, permiten ir construyendo un contexto dinámico con información de los acontecimientos más relevantes sucedidos en el entorno en cuestión. La composición del contexto se alimenta de vectores de información de distinta naturaleza, proporcionando elementos de inferencia relacionados con el mundo físico y el mundo virtual. A parte de la generación de “insights” en tiempo real, el sistema presenta mecanismos de retroalimentación de información basado en el histórico del material inferido previamente. The Discovery Stack, at each level of inference, launches a battery of predetermined predictive models based on deep learning (DL: Deep Learning, in English) that, combined with additional logic, allow for the construction of a dynamic context with information on the most relevant events that have occurred in the environment in question. The composition of the context is fed by vectors of information of a different nature, providing elements of inference related to the physical world and the virtual world. Apart from the generation of "insights" in real time, the system presents information feedback mechanisms based on the history of previously inferred material.
El hogar como entidad autónoma The home as an autonomous entity
Toda esta capacidad contextual permite hablar del hogar como una entidad consciente que resuelve problemas no obvios, pudiendo anticiparse incluso a la propia acción del usuario. En concreto, aquí se enumeran algunas capacidades - a), b), c) y d)- que emergen inmediatamente aplicando el modelo de la invención: a) Notificaciones Inteligentes All this contextual capacity allows us to talk about the home as a conscious entity that solves non-obvious problems, even being able to anticipate the user's own action. Specifically, here are some capabilities - a), b), c) and d) - that immediately emerge by applying the model of the invention: a) Smart Notifications
El hogar se percibe de puertas hacia fuera como una entidad activa que abstrae de la necesidad de especificar el dispositivo o mecanismo de comunicación destino. Es decir, el hogar como entidad autónoma y consciente de su entorno es capaz de entregar el mensaje adecuado, a la persona adecuada, en el sitio o canal y en el momento adecuado. En otras palabras, ya no es necesario especificar como destino de la comunicación un número de teléfono, una dirección IP, un dominio o un dispositivo concreto, sólo sería necesario especificar un hogar determinado como receptor de la información. A partir de aquí, el hogar, conocedor de su entorno, selecciona el mecanismo idóneo para garantizar la entrega del mensaje y la atención del destinatario. The home is perceived outwardly as an active entity that abstracts from the need to specify the destination communication device or mechanism. In other words, the home as an autonomous entity aware of its environment is capable of delivering the right message, to the right person, in the right place or channel and at the right time. In other words, it is no longer necessary to specify a telephone number, an IP address, a domain or a specific device as the destination of the communication, it would only be necessary to specify a specific home as the recipient of the information. From here, the home, aware of its environment, selects the ideal mechanism to guarantee the delivery of the message and the attention of the recipient.
A modo de ejemplo, se detallan algunas situaciones donde la entrega de información a través de notificaciones se ve beneficiada por la capacidad de inferencia ofrecida por la solución. En concreto, se ubica al usuario titular de la línea telefónica en diferentes escenarios donde la entrega de la notificación debe articularse a través de distintos canales: 1) El titular de la línea está en el salón de su casa viendo la televisión. El móvil se está cargando en el dormitorio. By way of example, some situations are detailed where the delivery of information through notifications is benefited by the inference capacity offered by the solution. Specifically, the owner of the telephone line is located in different scenarios where the delivery of the notification must be articulated through different channels: 1) The owner of the line is in the living room of his house watching television. The mobile is charging in the bedroom.
En una situación normal la notificación se entregaría al teléfono. Sin embargo, claramente el canal de entrega no es el adecuado ya que el titular vería el mensaje cuando volviera al dormitorio a recoger su teléfono. Pero si se aplica el modelo de capas de la invención, mediante una combinación sencilla de sentencias lógicas deducidas del entorno, que serían “mi marido está solo en casa” y “mi marido está en el salón”, junto con el estado de encendido/apagado del descodificador como parte de la lógica extraída del entorno, el sistema es capaz de habilitar la televisión como canal idóneo para realizar la entrega. In a normal situation the notification would be delivered to the phone. However, the delivery channel is clearly not the right one, since the owner would see the message when he returned to the bedroom to pick up his phone. But if the layered model of the invention is applied, by means of a simple combination of logical sentences deduced from the environment, which would be "my husband is home alone" and "my husband is in the living room", together with the power on/ turning off the decoder as part of the logic extracted from the environment, the system is capable of enabling television as the ideal channel for delivery.
2) El titular de la línea se encuentra cocinando. Hay un dispositivo multimedia con una pantalla (“display”) habilitada (por ejemplo, una tableta, altavoz inteligente, asistente de hogar, etc, ...) instalado en la cocina. El móvil está en el dormitorio nuevamente cargándose. Están los hijos viendo la televisión en el salón. 2) The holder of the line is cooking. There is a display-enabled media device (eg tablet, smart speaker, home assistant, etc, ...) installed in the kitchen. The mobile is in the bedroom charging again. The children are watching television in the living room.
En este caso, nuevamente la notificación siguiendo una solución convencional llegaría al teléfono. Y de la misma manera que en el caso anterior, el canal de entrega no sería el adecuado al no llegar la información al receptor final en el momento preciso, si no al terminal móvil. In this case, again the notification following a conventional solution would reach the phone. And in the same way as in the previous case, the delivery channel would not be adequate since the information did not reach the final recipient at the precise moment, if not the mobile terminal.
Este escenario resultaría a priori más complicado de resolver puesto que hay varios individuos en diferentes escenas. No obstante, la capacidad de escala de la solución permite extraer inferencias concretas de las dos escenas que plantea el caso de uso: This scenario would be a priori more complicated to solve since there are several individuals in different scenes. However, the scalability of the solution allows us to draw concrete inferences from the two scenes posed by the use case:
2.1) Escena del salón 2.1) Living room scene
Mediante la combinación de las sentencias lógicas inferidas por el motor de análisis, el sistema a la conclusión de que son “Mis hijos” quienes están viendo la TV. Según se explicó anteriormente, usando el detector de presencia de personas, clasificadores de edad y de género, y la información de los dispositivos móviles registrados y conectados en la WLAN, el sistema llega a que hay un menor y una menor, que son miembros de la familia, por tanto, el hijo y la hija, en el hogar y están acompañados por un adulto. Se detecta además que están en el salón y se tiene el estado de encendido/apagado del descodificador, con lo que el sistema infiere finalmente que “mis hijos están en el salón viendo la TV”, By combining the logical sentences inferred by the analysis engine, the system concludes that it is "My children" who are watching TV. As explained above, using the presence of people detector, age and gender classifiers, and the information of the mobile devices registered and connected in the WLAN, the system arrives at the fact that there is a minor and a minor, who are members of the family, therefore, the son and daughter, at home and are accompanied by an adult. It is also detected that they are in the living room and the decoder's on/off status is obtained, with which the system finally infers that "my children are in the living room watching TV",
Por tanto, atendiendo a la especificación de la notificación (el destinatario final es el titular de la línea), en este caso la televisión no es considerado un canal válido para entregar la información. Therefore, according to the specification of the notification (the final addressee is the owner of the line), in this case television is not considered a valid channel to deliver the information.
2.2) Escena de la cocina 2.2) Kitchen scene
Por otro lado, de forma similar el sistema llega a la conclusión de que “Mi padre” (el titular de la línea en este ejemplo) se encuentra en la cocina realizando una actividad determinada (por ejemplo, se determina que está preparando la comida por el estado de encendido/apagado de la cocina), partiendo del evento base de que “mi marido está en casa acompañado”. En este caso el dispositivo ubicado en la cocina es, por tanto, seleccionado por el Hogar como canal de entrega del mensaje. ) El titular de la línea se encuentra fuera del hogar On the other hand, similarly, the system concludes that "My father" (the owner of the line in this example) is in the kitchen carrying out a certain activity (for example, it is determined that he is preparing food for the on/off state of the kitchen), starting from the base event that "my husband is at home accompanied". In this case, the device located in the kitchen is, therefore, selected by the Home as the message delivery channel. ) The owner of the line is away from home
Este caso probablemente es el más sencillo de resolver, porque la pregunta o cuestión inicial es si existe o no presencia humana en el hogar y, utilizando la capacidad de inferencia del sistema, el nodo del árbol proporciona esta información según lo recibido por los detectores de presencia del hogar. Como resultado, el mensaje, esta vez sí, es entregado al terminal móvil del titular de la línea (se notifica en primera instancia al terminal móvil porque es la opción de entrega adecuada y no la opción por defecto, como en los dos casos anteriores). b) Autenticación a nivel de Hogar This case is probably the easiest to solve, because the initial question or question is whether or not there is human presence in the home and, using the inference capability of the system, the tree node provides this information as received by the detectors. home presence. As a result, the message, this time, is delivered to the mobile terminal of the line owner (the mobile terminal is notified in the first instance because it is the appropriate delivery option and not the default option, as in the two previous cases). . b) Authentication at the Home level
En general, los procesos de autenticación suelen moverse entre dos líneas rojas: una relacionada con utilidad del método para el usuario y la otra relacionada con la intrusión del algoritmo de identificación. El valor diferencial que introduce el hogar como entidad autónoma se resume en dos cosas: i. el hogar no necesita molestar al usuario con un proceso de autenticación que implique acción alguna por su parte, ya que dispone de suficiente información contextual del entorno como para reconocer en cada momento los roles que están participando en la escena y qué acción se pretende completar. In general, authentication processes tend to move between two red lines: one related to the usefulness of the method for the user and the other related to the intrusion of the identification algorithm. The differential value introduced by the home as an autonomous entity can be summarized in two things: i. the home does not need to bother the user with an authentication process that implies any action on their part, since it has enough contextual information from the environment to recognize at all times the roles that are participating in the scene and what action is intended to be completed.
¡i. el hogar no necesita conocer datos sensibles del usuario (por ejemplo, datos biométricos). Nuevamente le vale con conocer la huella contextual que el usuario deja a diario sobre el entorno del hogar. Yo. the home does not need to know sensitive user data (for example, biometric data). Once again, it is worth knowing the contextual footprint that the user leaves daily on the home environment.
En concreto, el hogar necesita resolver un problema más simple que el de la autenticación biométrica, donde únicamente necesita ubicar a cada individuo en el anillo de pertenencia al núcleo familiar adecuado. En términos de interacción con el hogar, no es lo mismo un invitado, que el titular de la línea o que un intruso, de la misma manera que no es lo mismo un niño pequeño, que un adulto o que una persona en edad avanzada. Para ello, el motor de inferencia recorre un árbol de decisión respondiendo a una serie de preguntas clave con información recogida del entorno a través del conjunto de técnicas de DL. Specifically, the household needs to solve a simpler problem than that of biometric authentication, where it only needs to locate each individual in the ring belonging to the appropriate family nucleus. In terms of interaction with the home, a guest is not the same as the owner of the line or an intruder, in the same way that a small child is not the same as an adult or an elderly person. To do this, the inference engine goes through a decision tree answering a series of key questions with information collected from the environment through the set of DL techniques.
La Figura 6 muestra un ejemplo de cómo se realiza la autenticación del titular de la línea (suponiendo que es el padre quien está ostentando este rol), apoyándose en una matriz de relación, similar a la de la Figura 5, para encontrar las sentencias lógicas que dan paso a esta autenticación desatendida a nivel de hogar: Figure 6 shows an example of how the authentication of the holder of the line is carried out (assuming that it is the father who is holding this role), relying on a relationship matrix, similar to the one in Figure 5, to find the logical sentences Which give way to this unattended authentication at the home level:
Sentencia s19: Padre e hijos están solos en casa Sentence s19: Father and children are home alone
Sentencia s20: Padre e hijo están solos en casa Sentence s20: Father and son are home alone
Sentencia s21 : Padre e hija están solos en casa Sentence s21 : Father and daughter are home alone
Sentencia s22: Padre y Madre están solos en casa Sentencia s3: Padre está en casa con adultos ajenos a la familia en base a los siguientes eventos: Sentence s22: Father and Mother are home alone Sentence s3: Father is home with non-family adults based on the following events:
Evento e5: Mi hijo está en casa acompañado Event e5: My son is at home accompanied
Evento e6: Mi hija está en casa acompañada Event e6: My daughter is at home accompanied
Evento e7: Mi marido está en casa acompañado Event e7: My husband is at home accompanied
Evento e8: Mi mujer está en casa acompañada Event e8: My wife is at home accompanied
Evento e9: Hay un hombre adulto en casa ajeno a la familia y acompañado Evento e10: Hay una mujer adulta en casa ajena a la familia y acompañada Evento e11 : Mi marido está en casa solo Event e9: There is an adult man at home outside the family and accompanied Event e10: There is an adult woman at home outside the family and accompanied Event e11: My husband is at home alone
La autenticación de otros miembros o actores del entorno del hogar se puede obtener de forma similar. c) Control Parental a nivel de Hogar Authentication of other members or actors in the home environment can be obtained in a similar way. c) Parental Control at Home level
En esta ocasión el valor diferencial reside justamente en que el hogar como entidad autónoma es capaz de crear un control parental a nivel holístico, tomando decisiones conjuntas sobre el plano físico (por ejemplo, bloqueando electrodomésticos potencialmente peligrosos para un niño) y el plano virtual (deshabilitando ciertas actividades de juego o consumo excesivo de contenido). Todo ello aprovechando la información contextual recabada del entorno donde la aplicación de determinados perfiles se realiza ad-hoc a la escena en cuestión en tiempo real y sin necesidad de configuraciones predeterminadas. Todo esto supone una mejora sustancial frente al panorama actual donde cada servicio contratado en el hogar tiene su propio control parental. En concreto, la Figura 7 muestra cómo encontrar las condiciones que propician una activación del control parental a nivel de hogar en el sistema, es decir, todos aquellos escenarios en los que la casa perciba a menores sin presencia de adultos: On this occasion, the differential value resides precisely in the fact that the home as an autonomous entity is capable of creating parental control at a holistic level, making joint decisions on the physical level (for example, blocking potentially dangerous electrical appliances for a child) and the virtual level ( disabling certain gaming activities or excessive consumption of content). All this taking advantage of the contextual information collected from the environment where the application of certain profiles is carried out ad-hoc to the scene in question in real time and without the need for default configurations. All this represents a substantial improvement compared to the current scenario where each service contracted at home has its own parental control. Specifically, Figure 7 shows how to find the conditions that favor activation of parental control at the household level in the system, that is, all those scenarios in which the household perceives minors without the presence of adults:
Evento e5: Mi hijo está en casa acompañado Event e5: My son is at home accompanied
Evento e6: Mi hija está en casa acompañada Event e6: My daughter is at home accompanied
Evento e12: Mi hijo está en casa solo Event e12: My son is home alone
Evento e13: Mi hija está en casa sola para concluir si Event e13: My daughter is at home alone to conclude if
Sentencia s1: Mis hijos están solos en casa d) Fluidez en la experiencia Sentence s1: My children are home alone d) Fluency in experience
Hoy en día, debido a la segmentación de dispositivos y servicios en el hogar, todas las experiencias que percibe el usuario tienen un carácter aislado y generalista, y por supuesto, interrumpido por otro tipo de estímulos accesorios que luchan sin cuartel por la atención del usuario. El hogar como entidad autónoma permite organizar, sincronizar y cambiar la atención del usuario a diferentes estados de una manera suave y fluida. Dichos estados no luchan entre sí, sino que se complementan y adecúan a los intereses verdaderos del usuario del hogar. La información contextual de la que hace uso el Hogar como entidad autónoma permite salvaguardar la privacidad del usuario, el ritmo de consumo de información y la adecuación del entorno a las circunstancias concretas del momento. Se describen aquí unos ejemplos -d1) y d2)- concretos: d.1) Anillos de privacidad en el entorno del hogar Nowadays, due to the segmentation of devices and services in the home, all the experiences that the user perceives have an isolated and general character, and of course, interrupted by other types of accessory stimuli that fight mercilessly for the user's attention. . The home as an autonomous entity allows to organize, synchronize and switch the user's attention to different states in a smooth and fluid way. These states do not fight each other, but rather complement and adapt to the true interests of the home user. The contextual information that the Home makes use of as an autonomous entity makes it possible to safeguard the user's privacy, the rate of information consumption and the adaptation of the environment to the specific circumstances of the moment. Some specific examples -d1) and d2)- are described here: d.1) Privacy rings in the home environment
Este caso de uso refleja claramente cómo el hogar, aplicando el nivel de privacidad adecuado, es capaz de modular la información que proyecta al entorno en función de quién se encuentra en escena. De esta manera se distinguen tres niveles de privacidad en función del nivel de pertenencia al núcleo familiar: This use case clearly reflects how the home, applying the appropriate level of privacy, is capable of modulating the information it projects to the environment based on who is on the scene. In this way, three levels of privacy are distinguished depending on the level of belonging to the family nucleus:
(a) Escena privada: un único miembro perteneciente al núcleo familiar se encuentra solo en casa. (a) Private scene: a single member belonging to the family nucleus is alone at home.
(b) Escena familiar: vahos miembros pertenecientes al núcleo familiar se encuentran en la escena. (b) Family scene: several members belonging to the family nucleus are at the scene.
(c) Escena pública: a parte de algún miembro perteneciente al núcleo familiar se encuentra en escena una persona ajena al núcleo familiar. (c) Public scene: apart from a member belonging to the family nucleus, a person from outside the family nucleus is on stage.
En este caso el motor de inferencia llega a una serie de conclusiones reflejadas en el mapa de la Figura 8, donde se asocia el comportamiento del hogar con un anillo de privacidad dependiendo del nivel de pertenencia al núcleo familiar de los componentes en escena. En el ejemplo de mapa de inferencia se parte de dos eventos básicos: e81 : “hay un grupo de vahas personas en el hogar” e82: “hay una sola persona en el hogar” La cuestión lógica que se plantea es si alguien pertenece al núcleo familiar: c1 : “¿hay alguien de la familia en el grupo?” c2: “¿la persona es miembro de la familia?” In this case, the inference engine reaches a series of conclusions reflected in the map in Figure 8, where the behavior of the household is associated with a privacy ring depending on the level of belonging to the family nucleus of the components on the scene. In the example of the inference map, we start from two basic events: e81 : "there is a group of several people at home" e82: "there is only one person at home" The logical question that arises is if someone belongs to the family nucleus: c1 : “Is there someone from the family in the group?” c2: “is the person a family member?”
En el hogar hay una red de comunicaciones de área local inalámbrica WLAN familiar y otra WLAN de invitados, WLAN_GUEST. El sistema obtiene de esas redes información sobre si: n1 : hay algún dispositivo nuevo conectado en la red WLAN familiar n2: hay algún dispositivo nuevo conectado en la red WLAN_GUEST n3: no hay ningún dispositivo nuevo conectado en la red WLAN familiar At home there is a family WLAN wireless local area communication network and another guest WLAN, WLAN_GUEST. The system obtains from those networks information about whether: n1 : there is any new device connected in the family WLAN network n2 : there is any new device connected in the WLAN_GUEST network n3 : there is no new device connected in the family WLAN network
Así, el sistema es capaz de inferir las siguientes sentencias mostradas en el mapa: Thus, the system is capable of inferring the following sentences shown on the map:
S81: Alguna persona del grupo es ajena al núcleo familiar o no lleva el móvil encima S81: Someone in the group is not part of the family nucleus or does not have a mobile phone with them
S82: Alguna persona del grupo lleva el móvil encima y está conectada a la red WLAN_GUEST S82: Someone in the group has their mobile on them and is connected to the WLAN_GUEST network
S83: Alguna persona del grupo lleva el móvil encima y está conectada a la red WLAN familiar S83: Someone in the group has a mobile phone on them and is connected to the family WLAN network
S84: Hay una sola persona en el hogar, lleva el móvil encima y está conectada a la red WLAN familiar S84: There is only one person at home, they have their mobile phone with them and they are connected to the family WLAN network
S85: Hay una sola persona en el hogar, lleva el móvil encima y está conectada a la red WLAN_GUEST S85: There is only one person at home, they have their mobile phone with them and they are connected to the WLAN_GUEST network
S86: Hay una persona sola en el hogar y no lleva el móvil encimaS86: There is a single person at home and they do not have their mobile with them
En el siguiente nivel, el sistema obtiene información más depurada, descubriendo si: d1 : el dispositivo inalámbrico conectado estaba previamente registrado d2: no se detecta dispositivo inalámbrico previamente registradoIn the next level, the system obtains more refined information, finding out if: d1 : the connected wireless device was previously registered d2 : no previously registered wireless device is detected
Así, el sistema es capaz de hacer inferencias más allá de las anteriores sentencias, como se muestran en el nivel superior del mapa: Thus, the system is capable of making inferences beyond the previous statements, as shown in the top level of the map:
S811 : Alguna persona del grupo es ajena al núcleo familiar S821 : Alguna persona del grupo es invitada S811 : Some person in the group is foreign to the family nucleus S821 : Someone from the group is invited
S831 : Alguna persona del grupo pertenece al núcleo familiarS831 : Some person in the group belongs to the family nucleus
S812: Hay una persona sola en el hogar que es ajena al núcleo familiar S812: There is a single person in the home who is not part of the family nucleus
S822: Hay una persona sola en el hogar que es invitadaS822: There is a single person in the home who is invited
S833: Hay una persona sola en el hogar y pertenece al núcleo familiar S833: There is a single person at home and they belong to the nuclear family
Para visualizar este caso de uso, se toma por ejemplo como actuador un servicio de la Nube (“cloud”) que muestra contenido multimedia a través de una pantalla (Tableta, TV, ...). En este caso el hogar es capaz de articular la interfaz de programación de aplicaciones (API) de la Nube, mostrando únicamente contenidos adecuados en función del anillo de privacidad reportado por el motor de inferencia. Esto sin duda proporciona un valor diferencial al servicio de “cloud” que aprovecha los anillos de privacidad en el hogar para dotar de privacidad a la entrega de contenidos. d.2) El Hogar como extensión de memoria To visualize this use case, for example, a Cloud service (“cloud”) that displays multimedia content through a screen (Tablet, TV, ...) is taken as an actuator. In this case, the home is capable of articulating the application programming interface (API) of the Cloud, showing only appropriate content based on the privacy ring reported by the inference engine. This undoubtedly provides a differential value to the "cloud" service that takes advantage of the privacy rings in the home to provide privacy to content delivery. d.2) Home as an extension of memory
Actualmente se da un crecimiento progresivo del número de transacciones en las relaciones diarias. Esto conlleva indudablemente un deterioro en la capacidad de memorización de tareas y compromisos que finalmente quedan sin ejecutarse en el momento oportuno por una falta latente de atención, priorización o tiempo. Aquí es donde el hogar como entidad autónoma aprovecha su información contextual para recordar todas esas tareas relevantes que quedan sin hacer a sus usuarios. Y precisamente, por su capacidad para identificarlas como parte de su comprensión del entorno, el sistema es capaz de informar al usuario en el momento máximo de atención o incluso de operar autónomamente la acción adecuada para asegurar su consecución. Para visualizar esta capacidad del Hogar vamos a plantear algunos casos de uso: Currently there is a progressive growth in the number of transactions in daily relationships. This undoubtedly entails a deterioration in the ability to memorize tasks and commitments that ultimately remain unexecuted at the right time due to a latent lack of attention, prioritization or time. This is where the home as an autonomous entity takes advantage of its contextual information to remind its users of all those relevant tasks that remain undone. And precisely, due to its ability to identify them as part of its understanding of the environment, the system is capable of informing the user at the moment of maximum attention or even autonomously operating the appropriate action to ensure its achievement. To visualize this capacity of the Home we are going to propose some use cases:
• El Hogar recuerda regar las plantas • The Home remembers to water the plants
Aplicando técnicas de asociación de información el sistema es capaz de inferir qué acción/es se están llevando a cabo en el entorno acotado (el Hogar). En concreto, aplicando técnicas basadas en Reconocimiento de Activitidad Humana, Comprensión de la Escena y una marca de tiempo, el sistema es capaz de clasificar las actividades del día a día y ordenarlas en una línea de tiempos. Posteriormente, un modelo basado en series temporales puede llegar a predecir que todas las tardes, por ejemplo a las 20:15 horas, el titular de la línea riega las plantas del jardín. Por tanto, si no se detecta la actividad en cuestión cuando llega la hora, se notifica al usuario adecuado que las plantas no se han regado hoy. Un ejemplo de la lógica seguida es: del evento “Padre está solo en casa” se infiere que “Padre está solo en el jardín” pero, a las 20:15 horas, no se detecta en el jardín la actividad “regar las plantas” y, por tanto, hay que enviar recordatorio a “Padre” (titular de la línea). By applying information association techniques, the system is capable of inferring what action/s are being carried out in the delimited environment (Home). Specifically, applying techniques based on Human Activity Recognition, Scene Understanding and a time stamp, the system is capable of classifying day-to-day activities and ordering them on a time line. Subsequently, a model based on time series can predict that every afternoon, for example at 8:15 p.m., the owner of the line waters the plants in the garden. Therefore, if the activity in question is not detected when the time comes, the appropriate user is notified that the plants have not been watered today. An example of the logic followed is: from the event "Father is home alone" it is inferred that "Father is alone in the garden" but, at 8:15 p.m., the activity "watering the plants" is not detected in the garden and, therefore, a reminder must be sent to "Father" (line owner).
• El Hogar recuerda a mis hijos hacer los deberes • The Home reminds my children to do their homework
De la misma manera y aplicando técnicas similares, el sistema puede identificar el horario de trabajo habitual de los más pequeños del hogar y recordarles que es hora de hacer la tarea. Así mismo también se podría mandar una notificación a los padres en caso de que no se estén realizando las tareas del colegio. Un ejemplo de la lógica seguida es: del evento “Mis hijos están solos en casa” se infiere que “Mis hijos están solos en el salón” pero, cuando llega la hora de hacer la tarea, no se detecta en el salón la actividad “leer/estudiar” y, por tanto, hay que enviar recordatorio a “Hijos” o “Madre” o “Padre” (titular de la línea). In the same way and applying similar techniques, the system can identify the usual work hours of the smallest members of the household and remind them that it is time to do homework. Likewise, a notification could also be sent to the parents in case the school tasks are not being carried out. An example of the logic followed is: from the event "My children are alone at home" it is inferred that "My children are alone in the classroom" but, when it is time to do their homework, the activity in the classroom is not detected " read/study" and, therefore, a reminder must be sent to "Children" or "Mother" or "Father" (line holder).
• El Hogar recuerda a una persona mayor tomar la medicación• The Home reminds an elderly person to take the medication
En la misma línea, el motor de inferencia reconocería a una persona mayor (abuelo/a) que toma un medicamento regularmente a una hora. El día que no se encuentra registro de esta actividad a esa hora, el Hogar envía una notificación a esta persona (abuelo/a) recordándole la toma del medicamento. Along the same lines, the inference engine would recognize an elderly person (grandparent) who regularly takes a medication at one hour. The day that no record of this activity is found at that time, the Home sends a notification to this person (grandparent) reminding them to take the medication.
Los propósitos de esta propuesta es la creación de: una verdadera solución para que el usuario perciba el valor de sus datos y como éstos gestionados de forma adecuada le devuelven un valor sustancial; nuevos modelos de negocio basados en la conciencia (“Awareness”, en inglés) de acontecimientos; nuevas formas de construir experiencias diferenciadoras en el ámbito del hogar; nuevo valor entorno a productos y servicios de proveedores, los cuales ven potenciado y amplificado su ámbito de actuación aprovechando la base de capacidades que ofrece el hogar como una nueva entidad proactiva. The purposes of this proposal is the creation of: a true solution for the user to perceive the value of their data and how properly managed they return a substantial value; new business models based on awareness of events; new ways of building differentiating experiences in the home environment; new value around the products and services of suppliers, which see their scope of action strengthened and amplified by taking advantage of the base of capacities offered by the home as a new proactive entity.

Claims

29 29
REIVINDICACIONES Un método implementado por ordenador para la toma de decisiones y ejecución de acciones en un entorno acotado, caracterizado por que comprende los siguientes pasos: generar datos, en una primera capa (110) de un modelo de aprendizaje automatizado, por una pluralidad de fuentes de datos (FD_11,... ,FD_1j, FD_21,... FD_2j,.., FD_¡ 1 , ... FDJj) generadas por una pluralidad de dispositivos (DISP_1, DISP_2, ... , DISPJ) pertenecientes al entorno acotado; reconocer patrones, en una segunda capa (120) del modelo de aprendizaje automatizado a la que la primera capa (110) inyecta los datos generados, donde reconocer patrones comprende adaptar por una pluralidad de módulos de preprocesado (PREP_1, PREP_2,... PREP_p) de la segunda capa (120) los datos generados en la primera capa (110), cada módulo de preprocesado asociando al menos una de las fuentes de datos (FD_11 , ... , FD_1 j, FD_21 ,... FD_2j,.., FD_¡ 1 , ... FDJj) a una técnica de reconocimiento (TR_1, TR_2, ... , TR_k), identificar patrones de reconocimiento por cada técnica de reconocimiento (TR_1, TR_2, ... , TR_k) usando uno o más algoritmos de aprendizaje automatizado, y obtener un elemento de inferencia (INF_1, INF _2, ... , INF _k) de cada patrón identificado con información del entorno acotado; asociar la información de cada elemento de inferencia (INF_1, INF _2, ... , INF _k), en una tercera capa (130) del modelo de aprendizaje automatizado a la que la segunda capa (120) inyecta los patrones identificados, donde asociar la información comprende relacionar los elementos inferidos (INF_1, INF _2, ... , INF _k) mediante técnicas de asociación de información (TAI_1, TAI _2, ... , TAI J) ejecutadas por una pluralidad de módulos de asociación de información de la tercera capa (130), cada módulo de asociación de información generando al menos un elemento de entendimiento que es una unidad mínima de información entendióle por un humano descubierta en el entorno acotado; y 30 generar unas señales de activación (TRIG_1, TRIG_2, ... TRIG_y) a partir de la información asociada a cada elemento de inferencia (INF_1, INF _2, ... , INF _k) que un actuador (280) transforma en unas señales de acciones (ACT_1, ACT _2, ... , ACT _n) a ejecutar sobre dispositivos electrónicos (290) del entorno acotado. El método de acuerdo con la reivindicación 1, caracterizado por que además comprende generar periódicamente una captura instantánea (250) que recoge todos los elementos de entendimiento generados en un período de tiempo, donde la captura instantánea (250) es un informe desglosado de acontecimientos que ocurren en el entorno acotado. El método de acuerdo con la reivindicación 2, caracterizado por que además comprende almacenar todas las capturas instantáneas (250) generadas en una línea temporal a modo de histórico, donde las capturas instantáneas almacenadas (SN_1, SN_2, ... , SN_x) en el período de tiempo hasta un instante de tiempo, x, son procesadas para descubrir patrones de reconocimiento (260), y además comprende inyectar en la tercera capa (130) los patrones descubiertos usando las capturas instantáneas almacenadas (SN_1, SN_2, ... , SN_x) para generar una siguiente captura instantánea (SN_x+1) en el instante de tiempo siguiente, x+1. El método de acuerdo con cualquiera de las reivindicaciones anteriores, caracterizado por que generar los elementos de entendimiento comprende usar una unidad lógica (UL_r) que es un nodo de un árbol de decisión formado por los siguientes elementos: una cuestión lógica (CUE_r) a responder mediante el procesado de la información asociada por la tercera capa (130), un evento base (EVENTO_r) sobre el que el patrón de reconocimiento es construido por una combinación de modelos de inferencia (MOD_r1 , ... , MOD_rs) entrenados con una selección de las fuentes de datos de la primera capa (110) que contribuyen a responder la cuestión lógica (CUE_r), y una sentencia lógica (SL_r) definida como una cantidad máxima de información entendióle por un humano sobre la cuestión lógica (CUE_r) que se infiere del evento base (EVENTO_r). El método de acuerdo con la reivindicación 4, caracterizado por que además comprende acotar un error semántico cometido por los modelos de inferencia (MOD_r1, MOD_rs) calculado como la diferencia semántica entre la información extraída de la cuestión lógica (CUE_r) y la sentencia lógica (SL_r) inferida por la unidad lógica (UL_r). El método de acuerdo con cualquiera de las reivindicaciones anteriores, caracterizado por que se ejecuta en el entorno acotado de un hogar. Un sistema para la toma de decisiones y ejecución de acciones en un entorno acotado, caracterizado por que comprende: en una primera capa (110) de un modelo de aprendizaje automatizado para generar datos, la primera capa (110) comprendiendo una pluralidad de fuentes de datos (FD_11,... ,FD_1j, FD_21 ,... FD_2j,.., FD_¡1 , ... FDJj) generadas por una pluralidad de dispositivos (DISP_1 , DISP_2, ... , DISPJ) pertenecientes al entorno acotado; una segunda capa (120) del modelo de aprendizaje automatizado a la que la primera capa (110) inyecta los datos generados para reconocer patrones, la segunda capa (120) comprendiendo una pluralidad de módulos de preprocesado (PREP_1, PREP_2,... PREP_p) configurados para adaptar los datos generados en la primera capa (110), asociando al menos una de las fuentes de datos (FD_11,... ,FD_1j, FD_21,... FD_2j,.., FD_¡1 ,... FD_ij) a una técnica de reconocimiento (TR_1, TR_2, ... , TR_k), y para identificar patrones de reconocimiento por cada técnica de reconocimiento (TR_1, TR_2, ... , TR_k) usando uno o más algoritmos de aprendizaje automatizado, y obtener un elemento de inferencia (INF_1 , INF _2, ... , INF _k) de cada patrón identificado con información del entorno acotado; una tercera capa (130) del modelo de aprendizaje automatizado a la que la segunda capa (120) inyecta los patrones identificados para asociar la información de cada elemento de inferencia (INF_1, INF _2, ... , INF _k) por una pluralidad de módulos de asociación de información configurados para, mediante, técnicas de asociación de información (TAI_1, TAI _2, ... , TAI _l), generar al menos un elemento de entendimiento que es una unidad mínima de información entendióle por un humano descubierta en el entorno acotado; y un automáta (270) configurado para generar unas señales de activación (TRIG_1, TRIG_2, ... TRIG_y) a partir de la información asociada a cada elemento de inferencia (INF_1, INF _2, ... , INF _k) y un actuador (280) que transforma las señales de activación (TRIG_1, TRIG_2, ... TRIG_y) en unas señales de acciones (ACT_1, ACT _2, ... , ACT _n) a ejecutar sobre dispositivos electrónicos (290) del entorno acotado. El sistema de acuerdo con la reivindicación 7, caracterizado por que además comprende un motor de inferencia (240) configurado para generar periódicamente una captura instantánea (250) que recoge todos los elementos de entendimiento generados en un período de tiempo, donde la captura instantánea (250) es un informe desglosado de acontecimientos que ocurren en el entorno acotado. El sistema de acuerdo con la reivindicación 8, caracterizado por que además comprende medios de almacenamiento para almacenar todas las capturas instantáneas (250) generadas en una línea temporal a modo de histórico, donde las capturas instantáneas almacenadas (SN_1, SN_2, ... , SN_x) en el período de tiempo hasta un instante de tiempo, x, son usadas por el motor de inferencia (240) para generar una siguiente captura instantánea (SN_x+1) en el instante de tiempo siguiente, x+1. El sistema de acuerdo con cualquiera de las reivindicaciones 8-9, caracterizado por que el motor de inferencia (240) comprende un árbol de decisión formado por los siguientes elementos: una cuestión lógica (CUE_r) a responder mediante el procesado de la información asociada por la tercera capa (130), un evento base (EVENTO_r) sobre el que el patrón de reconocimiento es construido por una combinación de modelos de inferencia 33 CLAIMS A computer-implemented method for decision-making and action execution in a delimited environment, characterized in that it comprises the following steps: generate data, in a first layer (110) of an automated learning model, from a plurality of sources of data (FD_11,... ,FD_1j, FD_21,... FD_2j,.., FD_¡ 1 , ... FDJj) generated by a plurality of devices (DISP_1, DISP_2, ... , DISPJ) belonging to the environment bounded; recognizing patterns, in a second layer (120) of the machine learning model to which the first layer (110) injects the generated data, where recognizing patterns comprises adapting by a plurality of preprocessing modules (PREP_1, PREP_2, ... PREP_p ) of the second layer (120) the data generated in the first layer (110), each preprocessing module associating at least one of the data sources (FD_11,..., FD_1j, FD_21,...FD_2j,. ., FD_¡ 1 , ... FDJj) to a recognition technique (TR_1, TR_2, ... , TR_k), identify recognition patterns for each recognition technique (TR_1, TR_2, ... , TR_k) using one or more machine learning algorithms, and obtaining an inference element (INF_1, INF _2, ... , INF _k) of each identified pattern with information from the bounded environment; associate the information of each inference element (INF_1, INF _2, ... , INF _k), in a third layer (130) of the machine learning model to which the second layer (120) injects the identified patterns, where to associate the information comprises relating the inferred elements (INF_1, INF _2, ... , INF _k) by means of information association techniques (TAI_1, TAI _2, ... , TAI J) executed by a plurality of information association modules of the third layer (130), each information association module generating at least one element of understanding that is a minimum unit of information understood by a human discovered in the bounded environment; and generating activation signals (TRIG_1, TRIG_2, ... TRIG_y) from the information associated with each inference element (INF_1, INF _2, ... , INF _k) that an actuator (280) transforms into signals of actions (ACT_1, ACT _2, ... , ACT _n) to be executed on electronic devices (290) of the bounded environment. The method according to claim 1, characterized in that it also comprises periodically generating a snapshot (250) that includes all the elements of understanding generated in a period of time, where the snapshot (250) is a broken down report of events that occur in the bounded environment. The method according to claim 2, characterized in that it further comprises storing all the snapshots (250) generated in a time line as a history, where the snapshots stored (SN_1, SN_2,..., SN_x) in the period of time up to an instant of time, x, are processed to discover recognition patterns (260), and also comprises injecting into the third layer (130) the patterns discovered using the stored snapshots (SN_1, SN_2,..., SN_x) to generate a next snapshot (SN_x+1) at the next time instant, x+1. The method according to any of the previous claims, characterized in that generating the understanding elements comprises using a logical unit (UL_r) that is a node of a decision tree formed by the following elements: a logical question (CUE_r) to answer through the processing of the associated information by the third layer (130), a base event (EVENT_r) on which the recognition pattern is built by a combination of inference models (MOD_r1,..., MOD_rs) trained with a selection of the first layer data sources (110) that contribute to answering the logical question (CUE_r), and a logical sentence (SL_r) defined as a maximum amount of information understood by a human about the logical question (CUE_r) that is infers from the base event (EVENT_r). The method according to claim 4, characterized in that it also comprises delimiting a semantic error committed by the inference models (MOD_r1, MOD_rs) calculated as the semantic difference between the information extracted from the logical question (CUE_r) and the logical sentence ( SL_r) inferred by the logical unit (UL_r). The method according to any of the preceding claims, characterized in that it is executed in the enclosed environment of a home. A system for making decisions and executing actions in a bounded environment, characterized in that it comprises: in a first layer (110) of an automated learning model to generate data, the first layer (110) comprising a plurality of sources of data (FD_11,... ,FD_1j, FD_21 ,... FD_2j,.., FD_¡1 , ... FDJj) generated by a plurality of devices (DISP_1 , DISP_2, ... , DISPJ) belonging to the bounded environment ; a second layer (120) of the machine learning model to which the first layer (110) injects the data generated to recognize patterns, the second layer (120) comprising a plurality of preprocessing modules (PREP_1, PREP_2,... PREP_p ) configured to adapt the data generated in the first layer (110), associating at least one of the data sources (FD_11,... ,FD_1j, FD_21,... FD_2j,.., FD_¡1,... FD_ij) to a recognition technique (TR_1, TR_2, ... , TR_k), and to identify recognition patterns for each recognition technique (TR_1, TR_2, ... , TR_k) using one or more machine learning algorithms, and obtain an inference element (INF_1 , INF _2, ... , INF _k) of each pattern identified with information from the bounded environment; a third layer (130) of the machine learning model to which the second layer (120) injects the identified patterns to associate the information of each inference element (INF_1, INF_2,..., INF_k) by a plurality of association modules information configured to, through information association techniques (TAI_1, TAI _2, ... , TAI _l), generate at least one element of understanding that is a minimum unit of information understood by a human discovered in the delimited environment; and an automata (270) configured to generate activation signals (TRIG_1, TRIG_2, ... TRIG_y) from the information associated with each inference element (INF_1, INF _2, ... , INF _k) and an actuator (280) that transforms the activation signals (TRIG_1, TRIG_2, ... TRIG_y) into action signals (ACT_1, ACT _2, ..., ACT _n) to be executed on electronic devices (290) of the bounded environment. The system according to claim 7, characterized in that it also comprises an inference engine (240) configured to periodically generate a snapshot (250) that collects all the elements of understanding generated in a period of time, where the snapshot ( 250) is a disaggregated report of events that occur in the bounded environment. The system according to claim 8, characterized in that it also comprises storage means to store all the snapshots (250) generated in a time line as a history, where the snapshots stored (SN_1, SN_2,..., SN_x) in the time period up to a time instant, x, are used by the inference engine (240) to generate a next snapshot (SN_x+1) at the next time instant, x+1. The system according to any of claims 8-9, characterized in that the inference engine (240) comprises a decision tree made up of the following elements: a logical question (CUE_r) to be answered by processing the information associated with the third layer (130), a base event (EVENT_r) on which the recognition pattern is built by a combination of inference models 33
(M0D_r1, MOD_rs) entrenados con una selección de las fuentes de datos de la primera capa (110) que contribuyen a responder la cuestión lógica (CUE_r), y una sentencia lógica (SL_r) definida como una cantidad máxima de información entendióle por un humano sobre la cuestión lógica (CUE_r) que se infiere del evento base (EVENTO_r). El sistema de acuerdo con cualquiera de las reivindicaciones 7-10, caracterizado por que las fuentes de datos son fuentes de audio, fuentes de vídeo, fuentes de telemetría asociadas a sensores de temperatura, presión o humedad, fuentes de telemetría interna de decodificadores de vídeo, fuentes de información sobre terminales móviles, fuentes de consumo de batería de dispositivos. El sistema de acuerdo con cualquiera de las reivindicaciones 7-11 , caracterizado por que el entorno acotado es un hogar. Un programa de ordenador que implementa el método definido de acuerdo con las reivindicaciones 1-6. (M0D_r1, MOD_rs) trained with a selection of the first layer data sources (110) that contribute to answering the logical question (CUE_r), and a logical sentence (SL_r) defined as a maximum amount of information understood by a human on the logical question (CUE_r) that is inferred from the base event (EVENT_r). The system according to any of claims 7-10, characterized in that the data sources are audio sources, video sources, telemetry sources associated with temperature, pressure or humidity sensors, internal telemetry sources of video decoders , sources of information on mobile terminals, sources of device battery consumption. The system according to any of claims 7-11, characterized in that the bounded environment is a home. A computer program implementing the method defined according to claims 1-6.
PCT/ES2021/070744 2021-10-13 2021-10-13 Method and system for taking decisions and executing unattended actions in a limited environment WO2023062250A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/ES2021/070744 WO2023062250A1 (en) 2021-10-13 2021-10-13 Method and system for taking decisions and executing unattended actions in a limited environment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/ES2021/070744 WO2023062250A1 (en) 2021-10-13 2021-10-13 Method and system for taking decisions and executing unattended actions in a limited environment

Publications (1)

Publication Number Publication Date
WO2023062250A1 true WO2023062250A1 (en) 2023-04-20

Family

ID=85987387

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/ES2021/070744 WO2023062250A1 (en) 2021-10-13 2021-10-13 Method and system for taking decisions and executing unattended actions in a limited environment

Country Status (1)

Country Link
WO (1) WO2023062250A1 (en)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160098021A1 (en) * 2014-10-06 2016-04-07 Fisher-Rosemount Systems, Inc. Regional big data in process control systems

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160098021A1 (en) * 2014-10-06 2016-04-07 Fisher-Rosemount Systems, Inc. Regional big data in process control systems

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
"Applications of Uncertainty Formalisms", 1 January 2001, SPRINGER, DE, ISBN: 978-3-540-65312-7, article SIMON PARSONS, ANTHONY HUNTER : "A Review of Uncertainty Handling Formalisms", pages: 8 - 37, XP009545809, DOI: 10.1007/3-540-49426-X_2 *
"Sistemas expertos y modelos de redes probabilisticas", 31 December 1998, SPANISH ACADEMY OF ENGINEERING, ES, article E. CASTILLO, J.M. GUTIÉRREZ, A.S. HADI: "2.4.1 Coherencia de Reglas", pages: 52 - 54, XP009545806 *
ERAN BELINSKY ; NATALIA MARMASSE ; EYAL SONSINO ; VLADIMIR SOROKA ; MARIA EBLING ; WILLIAM JEROME ; ARCHAN MISRA ; DABY SOW ; SUNE: "PASTA: Deriving Rich Presence for Converged Telecommunications Network Applications", COMMUNICATION SYSTEMS SOFTWARE AND MIDDLEWARE, 2007. COMSWARE 2007. 2N D INTERNATIONAL CONFERENCE ON, IEEE, PI, 1 January 2007 (2007-01-01), Pi , pages 1 - 12, XP031113883, ISBN: 978-1-4244-0613-5 *

Similar Documents

Publication Publication Date Title
US10682097B2 (en) People monitoring and personal assistance system, in particular for elderly and people with special and cognitive needs
Aztiria et al. Learning patterns in ambient intelligence environments: a survey
Slone et al. Self‐generated variability in object images predicts vocabulary growth
CN110139598B (en) Monitoring and tracking system, method, article and apparatus
CN107065586B (en) Interactive intelligent home service system and method
US11583997B2 (en) Autonomous robot
US11040441B2 (en) Situation-aware robot
Novák et al. Anomaly detection in user daily patterns in smart-home environment
US10217350B2 (en) Adaptive exception handling in security system
Roy et al. Possibilistic activity recognition in smart homes for cognitively impaired people
CN107431649A (en) For the generation and realization of resident family's strategy of intelligent household
Guesgen et al. Using contextual information for recognising human behaviour
Liming et al. SHINESeniors: personalized services for active ageing-in-place
WO2023062250A1 (en) Method and system for taking decisions and executing unattended actions in a limited environment
Bouchard et al. Spatial recognition of activities for cognitive assistance: realistic scenarios using clinical data from Alzheimer’s patients
US11855932B2 (en) Method for adjusting a device behavior based on privacy classes
Demrozi et al. Shpia: a low-cost multi-purpose smart home platform for intelligent applications
Menicatti et al. A cloud-based scene recognition framework for in-home assistive robots
Acampora et al. Interoperable services based on activity monitoring in Ambient Assisted Living environments
Khaemba et al. Security and Privacy of IoT Devices for Aging in Place
Qolomany Efficacy of Deep Learning in Support of Smart Services
Singh et al. 15 Future Technology
Shanmugapriya The Sway of Artificial Intelligence and Internet of Things in Human’s Life Style: An Inspiring Perspective with AI and Human’s Common Sense.
Magnani Human action recognition and monitoring in ambient assisted living environments
COŞKUN et al. Internet of Things on the Axis of Design and Technology Interaction: IOT Products as a New Product Paradigm.

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21960505

Country of ref document: EP

Kind code of ref document: A1