DE102016124894A1 - Method and device for creating a user interface for operating a building automation system by means of a cleaning robot, cleaning robot and building automation system - Google Patents

Abstract

The invention relates to a method for creating a user interface for operating a building automation system (102) by means of a cleaning robot (100), wherein the cleaning robot (100) is designed to create a map of an environment associated with the building automation system (102) and using the map at least one destination point (106) to drive in the area. In the method, location information (114) is read in first, which is an action of at least one component (112) of the building automation system (102) detected during the approach of the destination point (106) by a sensor (108) of the cleaning robot (100) and / or at Approaching the target point (106) by means of a sensor (112) of the building automation system (102) represents detected action of the cleaning robot (100). The component (112) is then located in the map using the location information (114). Finally, depending on a result of the request, at least one control element for controlling the component (112) is inserted in the card to create the user interface.

Description

The invention relates to a method for creating a user interface for operating a building automation system by means of a cleaning robot, a corresponding device, a corresponding computer program, a cleaning robot and a building automation system.

Vacuum robots are known, which can autonomously build a map of their environment while vacuuming. For example, this map may be displayed on a mobile device such as a smartphone or tablet.

Furthermore, so-called smart home systems for building automation are known, the controls can be put together manually. In particular, the logical linking and grouping to rooms is typically done by the user.

Against this background, the present invention has the object to provide an improved method and an improved device for creating a user interface for operating a building automation system, an improved cleaning robot and an improved building automation system.

According to the invention this object is achieved by a method, a device, a cleaning robot and a building automation system with the features of the main claims. Advantageous embodiments and further developments of the invention will become apparent from the following subclaims.

• Einlesen einer Verortungsinformation, die eine beim Anfahren des Zielpunktes mittels eines Sensors des Reinigungsroboters erfasste Aktion zumindest einer Komponente des Gebäudeautomationssystems und/oder eine beim Anfahren des Zielpunktes mittels eines Sensors des Gebäudeautomationssystems erfasste Aktion des Reinigungsroboters repräsentiert;
• Verorten der Komponente in der Karte unter Verwendung der Verortungsinformation; und
• Einfügen zumindest eines Steuerungselements zum Steuern der Komponente in die Karte abhängig von einem Ergebnis des Verortens, um die Bedienoberfläche zu erstellen.

The invention relates to a method for, preferably autonomously, creating a user interface for operating a building automation system by means of a cleaning robot, wherein the cleaning robot is designed to create a map of an environment associated with the building automation system and to drive at least one destination point in the environment using the map, the method comprising the steps of:

• Reading a location information that represents an action of the cleaning robot detected by a sensor of the cleaning robot when approaching the destination point of at least one component of the building automation system and / or an action of the cleaning robot detected by the start of the destination point by means of a sensor of the building automation system;
• Locating the component in the map using the location information; and
• Inserting at least one control element for controlling the component into the map depending on a result of the request to create the user interface.

Under a user interface, for example, a control card with control icons for controlling individual components of the building automation system can be understood. For example, the user interface may be displayed on a touch-sensitive screen, which may be part of the cleaning robot, a smartphone, or a tablet, so that a user is able to control the building automation system by touching the operation icons. A building automation system, also called smart home or intelligent living, can be understood as a system for networking devices, lights and buttons in a building, in particular a living environment. The networked devices, also called components of the building automation system, may, for example, appliances of domestic technology, household appliances, such as shutters, radiators, stoves, refrigerators or washing machines, or devices of consumer electronics. The component can be either an actuator or a sensor. For example, the component may be remotely controllable. A cleaning robot may be understood to mean a floor care device that is designed to clean rooms autonomously. The cleaning robot can be designed to autonomously exit the environment of the building automation system, for example a living environment, by means of the map created by it. The card may, for example, have been created in a SLAM (Simultaneous Localization And Mapping) method by the cleaning robot. A destination point can be understood as a location to be approached by the cleaning robot in the map whose coordinates are known to the cleaning robot.

Processing information may be taken to mean an optical or acoustic signal detected by the sensor of the cleaning robot or by the sensor of the building automation system, which may represent a specific effect caused by the component, such as when a lamp is switched on or off or when it is turned on. and rolling a roller shutter arises.

The sensor of the building automation system can be, for example, the component of the building automation system to be located.

For example, a control element can be understood as an operating element which, depending on the embodiment, graphically simulates switches or buttons for controlling the component. The user interface can be created, for example, by the control element being displayed in the map of the cleaning robot, for example at a point on the map where the component was located by means of the location information.

The approach described here is based on the recognition that a map display of a cleaning robot, such as a robotic vacuum cleaner or other service robots, can be extended to interactive controls for a smart home system.

By using the map material of the cleaning robot, a cumbersome, possibly completely to be performed by hand device of the smart home system can be avoided by the user and thus the ease of use of the smart home system can be significantly increased. As a result, a clear control surface can be realized, which is particularly suitable for the respective living situation. Above all, a smart home interface represented as a card greatly accommodates the user's understanding.

The cleaning robot can automate the creation of such an interactive control card. In this case, the cleaning robot autonomously create control elements for its respective card, with which the smart home system can be controlled more intuitive. For example, the map material can already be created by the cleaning robot according to various algorithms that solve the problem of SLAM, for example, and provided by Wi-Fi for further processing. For example, the cleaning robot may be connected to the same network as a controller of the smart home system. In this case, the network can be provided locally or by the provider of the smart home system control via the Internet.

According to one embodiment, component information representing the component may be read in the step of reading. In the step of providing, the component can be located using the component information. Component information can be understood to mean information by which the cleaning robot can conclude that the component has a semantic meaning. For example, the component information may include information about a type, a name, a group name or a space of the component. This embodiment enables a reliable and accurate location of the component in the map of the cleaning robot.

According to another embodiment, in the step of insertion, the control element may be linked to a portion of the map in which the component has been located. For example, the control element can be linked to the card in such a way that the control element is automatically superimposed upon touching the user interface at a location assigned to the component. By this embodiment, a manual link between a location of the component in the map and the control element can be omitted.

It is advantageous if, in the step of making the component, the component is located by triangulation. This allows the component to be located accurately and efficiently.

The method may comprise, according to another embodiment, a step of receiving a presence signal representing a presence of the cleaning robot at the destination point. At this time, in a step of outputting, an activation signal for activating the component may be output by using the presence signal. This allows the component to be activated depending on the presence of the cleaning robot at the target point.

The cleaning robot may be configured to use the card to approach at least one further target point in the environment. In this case, it is advantageous if at least one further location information is read in the step of reading in, the further location information being an action of the component detected when the further destination is approached by the sensor of the cleaning robot or, additionally or alternatively, a starting point of the further destination represents the action of the cleaning robot detected by the sensor of the building automation system. Accordingly, in the step of providing, the component may be located using the further location information. This allows a particularly accurate location of the component in the map of the cleaning robot.

This method can be implemented, for example, in software or hardware or in a mixed form of software and hardware, for example in a device.

The approach presented here also creates a device that is designed to perform the steps of a variant of a method presented here in appropriate facilities to drive or implement. Also through this Embodiment of the invention in the form of a device, the object underlying the invention can be solved quickly and efficiently.

In the present case, a device can be understood as meaning an electrical device which processes sensor signals and outputs control and / or data signals in dependence thereon. The device may have an interface, which may be formed in hardware and / or software. In the case of a hardware-based embodiment, the interfaces can be part of a so-called system ASIC, for example, which contains a wide variety of functions of the device. However, it is also possible that the interfaces are their own integrated circuits or at least partially consist of discrete components. In a software training, the interfaces may be software modules that are present, for example, on a microcontroller in addition to other software modules.

Furthermore, the approach described here creates a cleaning robot with a device according to a preceding embodiment as well as a building automation system with a device according to a preceding embodiment.

Also of advantage is a computer program product or computer program with program code which can be stored on a machine-readable carrier or storage medium such as a semiconductor memory, a hard disk memory or an optical memory and for carrying out, implementing and / or controlling the steps of the method according to one of the embodiments described above is used, especially when the program product or program is executed on a computer or a device.

• 1 eine schematische Darstellung eines Reinigungsroboters gemäß einem Ausführungsbeispiel;
• 2 eine schematische Darstellung einer Vorrichtung gemäß einem Ausführungsbeispiel;
• 3 eine schematische Darstellung einer Bedienoberfläche gemäß einem Ausführungsbeispiel;
• 4 eine schematische Darstellung einer Bedienoberfläche gemäß einem Ausführungsbeispiel;
• 5 eine schematische Darstellung eines Verfahrens zum Erstellen einer Karte mittels eines Reinigungsroboters gemäß einem Ausführungsbeispiel; und
• 6 eine schematische Darstellung eines Verfahrens zum Erstellen einer Bedienoberfläche zum Bedienen eines Gebäudeautomationssystems mittels eines Reinigungsroboters gemäß einem Ausführungsbeispiel.

An embodiment of the invention is shown purely schematically in the drawings and will be described in more detail below. It shows

• 1 a schematic representation of a cleaning robot according to an embodiment;
• 2 a schematic representation of a device according to an embodiment;
• 3 a schematic representation of a user interface according to an embodiment;
• 4 a schematic representation of a user interface according to an embodiment;
• 5 a schematic representation of a method for creating a map by means of a cleaning robot according to an embodiment; and
• 6 a schematic representation of a method for creating a user interface for operating a building automation system by means of a cleaning robot according to an embodiment.

1 shows a schematic representation of a cleaning robot 100 according to an embodiment. The cleaning robot 100 is via a suitable interface, such as a Wi-Fi or Bluetooth connection, with a building automation system 102 connected. To a the building automation system 102 assigned environment, such as a residential environment, to be able to drive off, is the cleaning robot 100 able to create a map of the environment, such as by means of a below with reference to 3 described method. In this case, that of the cleaning robot 100 created map at least one destination point 106 to be controlled when driving off the environment. To capture the environment is the cleaning robot 100 with a sensor 108 , For example, an optical or acoustic sensor equipped.

The cleaning robot 100 has a device 110 for creating a user interface for operating the building automation system 102 on. When approaching the destination point 106, the sensor detects 108 an action of at least one component 112 of the building automation system 102 , such as a lamp or a roller shutter, and sends a location information representing the action 114 to the device 110 , This is designed to be the component 112 using the location information 114 in the map of the cleaning robot 100 to locate. Depending on the result of the location, the device adds 110 at least one control element, such as an operating icon, in the map and creates a graphical user interface for easy and comfortable operation of the building automation system 102 , The device links 110 the control element, for example, with that portion of the card in which the component 112 using the location information 114 was located.

Optional is the device 110 trained to the thus created user interface in the form of appropriate map data 116 to the building automation system 102 or to another with the cleaning robot 100 coupled device, such as a smartphone or tablet to transfer for further processing.

According to the in 1 In the exemplary embodiment shown, the building automation system 102 is designed to be a component 112 representative component information 118 to the cleaning robot 100 to send. The component information 118 Depending on the embodiment, for example, contains information about a type, a name, a group name or a space of the component 112 , The device 110 is trained to get the component information 118 and read this in addition to the location information 114 for locating the component 112 in the map of the cleaning robot 100 to use.

According to a further embodiment, the cleaning robot 100 trained to reach the destination point 106 a presence signal 120 that the presence of the cleaning robot 100 at the destination point 106 indicating to the building automation system 102 to send. The presence signal 120 is from the building automation system 102 used an activation signal 122 to activate the component 112 to the component 112 to send the component 112 is activated only when the cleaning robot 100 at the destination point 106 has arrived. For example, the building automation system 102 trained to the component 112 by means of the activation signal 122 in a certain, from the cleaning robot 100 by means of the sensor 108 clearly and easily recognizable pattern to move or turn on or off. Alternatively, the output of the activation signal takes place 122 through the cleaning robot 100 This will allow the reliability of the location of the component 112 by means of the cleaning robot 100 be further increased.

According to 1 is the cleaning robot 100 designed to at least one other destination 124 in the map created by him the building automation system 102 to approach the assigned environment. When approaching the further destination point 124 the sensor detects 108 again an action of the component 112 which, for example, in turn through the building automation system 102 is triggered as soon as the cleaning robot 100 the other target point 124 reached. Analogous to the destination 106 the sensor sends 108 a corresponding further location information 126 to the device 110 that this in addition to the location information 114 used to the component 112 in the map of the cleaning robot 100 to locate. In particular, the device uses 110 the two location information 114 . 126 to the exact location of the component 112 to determine by triangulation.

The cleaning robot 100 is optionally designed to at least one of the two target points when creating the card 106 . 124 interconnecting path 128 to investigate.

According to an alternative embodiment, the device is 110 as part of a control device of the building automation system 102 realized. Here is the device 110 For example, trained to the cleaning robot 100 by outputting corresponding control signals 130 via the interface to the individual destination points 106 . 124 to control the environment and actions of the cleaning robot 100 when reaching the target points 106 . 124 , such as a movement of the cleaning robot 100 or one from the cleaning robot 100 generated optical or acoustic signal, by means of a corresponding sensor, for example, by the component 112 may be formed, in the form of location information 114 . 126 to process. For example, the building automation system 102 able to perform the actions of the cleaning robot 100 by means of the control signals 130 depending on a location of the cleaning robot 100 specifically trigger in the environment. In addition, the device receives 110 over the interface, for example, that of the cleaning robot 100 created map and uses it together with the location information 114 . 126 to the component 112 to locate in the map and to create the user interface that of the component 112 to insert associated control element according to the location in the map.

2 shows a schematic representation of a device 110 according to an exemplary embodiment, for example a preceding with reference to 1 described device. The device 110 includes a read-in unit 210 for reading in the location information 114 , a localization unit 220 for locating the component of the building automation system in the map created by the cleaning robot using the location information representing the component 114 as well as a creation unit 230 for creating the user interface by inserting the control element associated with the component into the card corresponding to one of the locating unit 120 provided location result 232 , The creation unit 230 is also designed to the user interface in the form of map data 116 issue.

3 shows a schematic representation of a user interface 300 according to an embodiment. The user interface 300 was, for example, from a preceding on the basis of 1 and 2 described device created. According to 3 is the user interface 300 exemplary as a map with graphical representations of various smart home actuators of the building automation system, here of shutters 302 and lights 304 , designed.

4 shows a schematic representation of a user interface 300 according to one Embodiment. The user interface 300 the previous one is based on 3 described user interface, with the difference that in 4 In addition, two each associated with a component of the building automation control elements 400 402, which are used to control two actuators, here by way of example a shutter 302 and a lamp 304 , serve.

For example, after clicking on the respective smart home actuator in the map, for example by touching with a finger, an overlay window opens with the associated control elements. In the case of the roller shutter, the first control element comprises 400 for example, a first operating symbol for rolling up the roller shutter and a second operating symbol for unrolling the roller shutter. In the case of the luminaire, the second control element comprises 402 For example, a first control to increase the brightness and a second control to reduce the brightness. Depending on the embodiment, the control elements 400 . 402 in each case more than two and / or other types of operating symbols, such as sliding or rotary control, color picker, etc. include. Alternatively, the controls become 400 . 402 directly displayed, ie without first selecting the relevant actuator in the map. According to a further embodiment, the user interface is 300 designed such that, for example, the shutter can be pushed up or pulled down in the card directly with the finger or with the mouse by dragging and dropping.

5 shows a flowchart of a method 500 for creating a map by means of a cleaning robot according to an embodiment, such as a preceding with reference to 1 to 4 described cleaning robot. First, in one step 502 a map creation by a SLAM process. In a further step 504 a threshold calculation is performed. In one step 506 the generation of paths from the map as well as the calculation of distributed destination points takes place. Depending on the embodiment, the path generation takes place in one step 508 by means of skeleton or in one step 510 by Voronoi tessellation (after inversion). The calculation of the distributed target points takes place either in one step 512 by closing and opening the results from the path calculation (as an abbreviation for Ultimate Points) or in a step 514 by calculating Ultimate Points. In one step 516 the results of the path generation or the calculation of the target points are superimposed and further processed.

On the paths, the cleaning robot can navigate safely after a continuity test. The Skeleton method creates more paths with better map coverage. The target points are all close to the calculated paths. Here, the Ultimate Points method provides better coverage. A threshold calculation can help to reduce the number of goals.

Individual versions of the card at or after creation are indicated by squares 518 characterized.

6 shows a flowchart of a method 600 according to an embodiment. The procedure 600 for creating a user interface for operating a building automation system by means of a cleaning robot, for example, by a device, as described above with reference to 1 to 5 is described. It is in one step 610 the location information is read. In a further step 620 the component of the building automation system is located in the map of the cleaning robot using the location information. In one step 630 The user interface is created by inserting a corresponding control element for controlling the component depending on a result of the location in the map.

By such a method, the cleaning robot in a smart home, also called building automation system above, largely autonomously create an intuitive map as a controller for the smart home. Depending on the embodiment, the user can intervene in any way supportive, if desired.

According to one embodiment, the method comprises 600 following steps. The prerequisite for this is that the cleaning robot and the smart home have already been put into operation and that both have set up a network connection. If necessary, the cleaning robot has already built the map of the building or apartment.

First, the cleaning robot scans in an IP segment assigned to it, for example 192.168.1. *, All 255 possible IP addresses to known interfaces of smart home systems and networked devices. For example, when using OpenHAB (Open Home Automation Bus), the following calls are started, waiting for backflushes:
http://192.168.1.0:8080/rest/items ... http://192.168.1.254:8080/rest/items

At the same time other protocols (HTTP, TCP / IP, UDP, WebSocket, etc. of different manufacturers) can be queried in a similar way. These calls cause the system to get a complete list of all the smart home output known sensors and actuators and their logical grouping. There will probably be a usable answer on at least one address that can be further processed.

This scan continues with other smart home interfaces that may have a different polling structure. These calls can also be retrofitted to the cleaning robot after delivery, for example via an interface for wireless communication, by means of software updates. The user can also specify specific IP addresses and interfaces in order to shorten this process.

{

 "link":"http://192.168.1.100:8080/rest/items/Shutter_GF_Living",
 "state":"100",
 "type":"Rollershutterltem",
 "name":"Shutter_G F_Living",
 "label":"Livingroom",
 "tags":[],
 "groupNames":[
 "GF_Living",
 "Shutters"
 },
 ...

If the cleaning robot has successfully completed the scan of the systems, a list of available sensors and actuators is available. In the case of OpenHAB, for example, an actuator, such as a roller shutter, is described in the JSON format as follows:

 {

 "Link": "http://192.168.1.100:8080/rest/items/Shutter_GF_Living""State":"100""Type":"Rollershutterltem""name":"Shutter_GF_Living","Label":"LivingRoom""Tags": []
 "Group names": [
 "GF_Living""Shutters"
 },
 ... 

It can be seen from the example that each list element has a type with which the cleaning robot can already deduce the semantic meaning of the element. Optionally, a graph is created by evaluating the field "groupNames" in order to conclude that the elements belong to each other.

In order to locate the collected elements of the smart home system in the map of the robot cleaner, target points and paths between the targets should be calculated with which the robot cleaner can safely drive off and observe the entire home environment. A flowchart illustrating these calculations in simplified form is exemplary in FIG 5 shown. The cleaning robot now has a list of smart home elements, previously called components, and a list of destination points interconnected by a network of paths.

The list of destination points is approached point by point. At each target point, the list of the smart home elements observable by the robot with sensors is switched on or several times in order, for example, to observe the respective effect in a camera image or with a microphone. For each target point and each element, the effect size, possibly linked to a direction, is recorded and stored in a crosstab. From these data, it is now relatively precisely determined by triangulation at which point in the map of the cleaning robot the source of the effect is to be located.

Analogously, it is also possible to use sensors of the smart home system. If the cleaning robot is capable of motion detectors, light sensors, volume meters or the like. These can also be mapped in a corresponding manner.

According to one embodiment, the user has the opportunity to intervene at this point to have only parts of the element or target list processed by the robot, such as when the user is aware that no smart home elements are installed in a particular room, or the control of certain elements by the user are not desired.

For example, in an app of the cleaning robot suitable control elements for the respective smart home actuator and corresponding displays for sensors are displayed at these points. These now provide the user with intuitive monitoring and control. Here, too, the user has the option to edit and change the result according to his own ideas.

Depending on the embodiment, the user is partially or completely supported by the cleaning robot when setting up the smart home system. Thus, a complex manual device of the smart home system can be omitted, especially since such a device can be very difficult without technical knowledge. At the same time, the user receives an intuitive user interface.

Maps created by such a method and interaction possibilities are exemplary in the 3 and 4 shown.

Cleaning robots and smart home installations are not yet considered to be interchangeable systems on the market. Consequently, although the cleaning robot generates a card, but this can only be used in the robot context. Conversely, it often takes a long time to set up the smart home by hand until it makes sense and is controllable. However, an intuitive representation for the control is often missing here, which often only contains an abstract representation of the sensor values and switches. By means of the approach proposed here, a map display is now made possible by which the effort in the device can be reduced to a minimum. By using synergies between cleaning robot and smart home, this problem can largely be solved in software.

Depending on the embodiment, the implementation can take place on the part of the cleaning robot as well as on the side of the smart home system. The smart home system can use map material that it receives via the API of the cleaning robot or even several cleaning robots. The cleaning robot is controlled accordingly by the smart home system.

Claims (11)

A method (600) for creating a user interface (300) for operating a building automation system (102), wherein the preparation is performed by a cleaning robot (100), wherein the cleaning robot (100) is adapted to generate a map (518) of a building automation system (102 ) and using the card (518) to drive at least one target point (106) in the environment, the method (600) comprising the steps of: Reading in (610) location information (114), which is an action of at least one component (112) of the building automation system (102) detected by a sensor (108) of the cleaning robot (100) when approaching the destination point (106) and / or when starting the vehicle Target point (106) by means of a sensor (112) of the building automation system (102) detected action of the cleaning robot (100) represents; Locating (620) the component (112) in the map (518) using the location information (114); and Inserting (630) at least one control element (400, 402) for controlling the component (112) into the map (518) in response to a result of the request (620) to create the user interface (300). Method (600) according to Claim 1 in which, in the reading-in step (610), component information (118) representing the component (112) is read in, wherein in the step of providing (620) the component (112) is located using the component information (118). A method (600) according to any one of the preceding claims, wherein in the step of inserting (630) the control element (400, 402) is associated with a portion of the card (518) in which the component (112) has been located. Method (600) according to one of the preceding claims, wherein in the step of providing (620) the component (112) is located by triangulation. The method (600) according to one of the preceding claims, comprising a step of receiving a presence signal (120) representing a presence of the cleaning robot (100) at the destination point (106) and a step of outputting an activation signal (122) for activating the component (112). using the presence signal (120). Method (600) according to one of the preceding claims, in which the cleaning robot (100) is designed to use the map (518) to approach at least one further target point (124) in the environment, wherein in the read-in step (610) at least one Further location information (126) is read in, wherein the further location information (126) detects an action of the component (112) detected by the sensor (108) of the cleaning robot (100) when approaching the further destination (124) and / or a start of the further Target point (124) by the sensor (112) of the building automation system (102) detected action of the cleaning robot (100), wherein in the step of Verortens (620), the component (112) is located using the further location information (126). Apparatus (110) comprising units (210, 220, 230) configured to perform the method (600) according to any one of Claims 1 to 6 execute and / or control. Cleaning robot (100) with a device (110) according to Claim 7 , Building automation system (102) with a device (110) according to Claim 7 , A computer program configured to perform the method (600) according to any one of Claims 1 to 6 execute and / or control. Machine-readable storage medium on which the computer program is based Claim 10 is stored.

Images