DE102022114045B3 - Method for operating a smart home platform and a smart home platform - Google Patents

Abstract

The present invention relates to a method for operating a smart home platform (1) as an entertainment environment, wherein in the method: - an “immersive home” application (50) provided and started on one of the smart terminal devices (40) as a control center is located in the Smart home platform (1) for the smart devices (10, 20, 30, 40) connects existing smart home services that control the respective different modes and records several smart devices (10, 20, 30) selected as participants in the immersive entertainment environment; and - by starting an entertainment offer selected via the control center (40) via remote access to the entertainment database (70), the selected smart devices (10, 20, 30) are automatically integrated into the "Immersive Home" application (50). Entertainment offer are integrated and their respective modes are automatically changed synchronously with the expiration of the entertainment offer based on the respective context data assigned to the expiry of the entertainment offer. Furthermore, a corresponding smart home platform is provided.

Description

The invention relates to a method for operating a smart home platform as an entertainment environment, the smart home platform comprising several smart devices or devices, each smart device being able to adopt different modes, and a number of entertainment offers being provided by an entertainment database. The present invention further relates to a smart home platform.

Smart home platforms are state of the art in various designs and are usually used to automatically control a home's facilities. For this purpose, the house includes a smart home platform, which usually includes at least one sensor arranged in the house and a computer functionally connected to the at least one sensor and the devices. The smart home platform enables the parameter values of the facilities to be set automatically.

A smart home platform can network both building technology and household equipment, such as light sources, blinds, heating, but also stoves, refrigerators and washing machines, as well as consumer electronics components, such as central storage and video and/or audio devices for provision home-wide use of video and audio content. The devices are so-called intelligent or smart devices or devices. The networking of the intelligent devices can be centrally configured and controlled via software, for example using a smartphone that activates and/or implements and executes the software. Both the devices and the software that controls the devices, i.e. the respective smart home services or smart home services, are usually from the same manufacturer. To operate the smart home platforms, it is therefore usually necessary to provide a manufacturer-specific setup. Compatibility of devices and software or smart home services from different manufacturers has so far only rarely been achieved.

Solutions such as HomeKit™ from Apple or Alexa™ from Amazon are known, where a customer can control their various intelligent devices using a single application. However, these automations and controls are limited to basic functionalities, such as switching on and off or IFTTT automation (“If this, then that”). A sequence of control commands and/or actions takes place in a strictly linear manner. In the area of entertainment, such as when watching a film or operating a game, the devices networked with the film or game are controlled and operated in a strictly linear manner based on predefined data. With Hue Synch, for example, an HDMI color signal output is used to control or change the color of the networked lights, i.e. the Hue lamps.

In contrast, with the Hue & Spotify cooperation, an input signal is analyzed in real time in order to provide the best possible or appropriate output in the form of a light show based on metadata. However, this analysis is based on concrete properties of the input signal, such as, in the case of a piece of music, its beat or its temporal sequence, whereas, for example, a mood caused by the piece of music, such as cheerful or melancholic, has no influence on the light show provided. Furthermore, the application is limited to a so-called digital ecosystem or ecosystem, i.e. to a group of intelligent networked facilities from the same company or from close partner companies. The underlying ecosystem of connected smart devices is not “open”, i.e. H. It is not possible to add additional intelligent devices from other manufacturers.

CN 2 04 360 111 U describes a smart home system with an intelligent terminal system, a remote control, a local network with Internet access and a plurality of control devices and end devices to be controlled. With the help of the intelligent terminal system, seven different scene modes can be set, with the respective terminal devices being controlled or set accordingly for each mode.

The article by Ingo Fischer, Sven Hansen and Nico Jurran “Custom-made smart home, the c't guide through the smart home jungle” from c't, 2020, issue 8, describes various options for setting up smart home technology .

CN 1 01 815 159 A discloses an intelligent household TV control system including a personal terminal for a user, a network server and a TV master controller for controlling household electrical appliances, the TV master controller being connected to a display interface and an operation interface ; the TV main control is provided with a network interface and a bus interface; the TV main controller is connected to the network server through the network interface and to the household electrical appliances through the bus interface to control the household electrical appliances. According to the system, a system platform is built on the TV, which is called Main controller is used and can be operated directly to control switches of all household electrical appliances at home and regulate other digital electrical appliances.

It is therefore an object of the invention to propose a method for operating a smart home platform, which enables the integration of smart devices from various digital ecosystems and also realizes content-adaptive control of the networked smart devices when an entertainment offer is running, so that a user can also In terms of mood, you can completely immerse yourself in the entertainment offering. Another object of the invention is to provide a smart home platform.

• - eine von einer auf einem der smarten Endgeräte als Steuerzentrale via Download bereitgestellte und gestartete „Immersive Home“ Applikation sich mit in der Smarthome-Plattform für die smarten Endgeräte bestehenden und die jeweiligen verschiedenen Modi steuernden Smart Home Diensten, insbesondere Cloud-Diensten, verbindet und mehrere als Teilnehmer an der immersiven Entertainmentumgebung ausgewählte smarte Endgeräte aus den verschiedenen Ökosystemen erfasst;
• - mit Starten eines über die Steuerzentrale via Fernzugriff auf die Entertainment-Datenbank ausgewählten Entertainment-Angebots, mittels der „Immersive Home“ Applikation die ausgewählten smarten Endgeräte automatisch in das Entertainment-Angebot eingebunden werden und deren jeweiligen Modi synchron mit Ablauf des Entertainment-Angebots basierend auf den jeweiligen dem Ablauf des Entertainment-Angebots zugeordneten Kontext-Daten automatisch geändert werden.

An object of the invention is a method according to claim 1 for operating a smart home platform as an immersive entertainment environment, the smart home platform comprising several different digital ecosystems of smart devices, each smart device being able to adopt different modes, and a number of an entertainment database of entertainment offers are provided, each entertainment offer comprising respective context data assigned to a process of the respective entertainment offer, wherein in the method:

• - an “Immersive Home” application provided and started via download on one of the smart devices as a control center connects to smart home services, in particular cloud services, existing in the smart home platform for the smart devices and controlling the respective different modes, and several smart devices from the various ecosystems selected as participants in the immersive entertainment environment are recorded;
• - by starting an entertainment offer selected via the control center via remote access to the entertainment database, the selected smart devices are automatically integrated into the entertainment offer using the “Immersive Home” application and their respective modes are synchronously based on the execution of the entertainment offer to the respective context data assigned to the flow of the entertainment offer can be changed automatically.

In the context of the present invention, a digital ecosystem or ecosystem is a system of wireless, interconnected smart devices that collect, exchange information and can be controlled using, usually cloud-based, services (smart home services or smart home services).

The “Immersive Home” application or application provided according to the invention is made available to the user via download (by a user) on a selected smart device that subsequently functions as a central control device or control center and is started from there by the user. However, the “Immersive Home” application provided according to the invention is executed on an external server, which is located, for example, in a cloud.

After the user has been authenticated with smart home services that exist for the smart devices in the various ecosystems, usually loT cloud services, which are also usually managed and executed on an external server, possibly located in another cloud, the system connects "Immersive Home" application with the smart home services and the user is shown a list of smart devices from the different ecosystems on the control center, from which the user can now select the smart devices from the different ecosystems, which will subsequently become participants in the immersive entertainment environment. The user can be authenticated directly by means of or via the control center or with a link (deep link) provided on the control center on a user terminal, such as a smartphone. The deep link provided points directly to a very specific, “deeper” subpage of the “Immersive Home” application, which is usually implemented as a web-based application. This “deeper” subpage can be a specific file, an individual article or an open directory.

With or from the “Immersive Home” application, the smart devices from the various ecosystems selected as participants in the immersive entertainment environment are now recorded.

The entertainment offer is started by the user via the control center via remote access to the entertainment database, which is also located on the external server of the “Immersive Home” application or is at least communicatively connected to it. When the entertainment offer is started, according to the invention, the selected smart devices are now automatically integrated into the entertainment offer by means of or from the “Immersive Home” application by changing their respective modes, ie their respective action or operating modes or their respective action modes. or operating states are automatically changed synchronously with the course of the entertainment offer based on the respective context data assigned to the course of the entertainment offer. This Synchronous and context-based changes to the respective modes of the smart devices are implemented using the “Immersive Home” application and its connection to both the entertainment offering and the existing smart home services for the respective smart devices.

In a possible embodiment of the method according to the invention, it is provided, as described above, that the “Immersive Home” application provides a list comprising the smart devices of the several different digital ecosystems for selecting the smart devices participating in the immersive entertainment environment and for display on a display unit of the control center brings, whereby the smart devices selected as participants are recorded by the “Immersive Home” application via respective inputs via or into an input unit of the control center.

According to another embodiment of the method according to the invention, the entertainment offer is selected and started on the control center, whereupon a corresponding session or “session” for the selected and started entertainment offer is held on the external server or the cloud via a suitable protocol, such as e.g. MQTT (Message Queuing Telemetry Transport) is generated. MQTT works according to the publisher/subscriber principle and is operated via a central broker, here for example the “Immersive Home” application running on the external server or on the cloud. This means that the sender and receiver, i.e. entertainment offer and user, have no direct connection. Senders and recipients report their data via a publish and all recipients who are interested in certain messages receive the data because they have registered as a subscriber.

In yet another embodiment of the method according to the invention, it is provided that a user registers to participate in the selected entertainment offer by recording an identifier entered via the control center and/or a user terminal functionally connected to the control center on an identifier on the control center and/or The website of the selected entertainment offering is accessed and displayed using a deep link on the user device. As a result, the user who registers to participate in the selected entertainment offer is considered a subscriber. The identifier can be, for example, a user-specific QR code. But any other suitable identifier, such as a PIN, is also conceivable. The registration of a user to participate in the selected entertainment offer can also be done using a suitable protocol, such as the MQTT mentioned above.

After all users participating in the entertainment offer have logged in, the selected entertainment offer is started using the “Immersive Home” application on the external server or cloud and the process or the individual process sections of the entertainment offer are started via MQTT orchestrated for each participating user. In the case of a game as an entertainment offer, each player, as a participating user of the game, receives individual game instructions from the “Immersive Home” application depending on the role they play within the game, such as a specific set of cards in the case of a card game, such as UNO, or specific story clues in the case of, for example, Escape Room.

A number of predetermined process sections are assigned to the entertainment offer. The process of the entertainment offer is dynamically composed of process sections of the number of predetermined process sections. The “Immersive Home” application assigns one of the modes of the respective selected smart device to each process section of the entertainment offer for each selected smart device on a context-based basis and sets this respective mode for the respective selected smart device automatically and in real time when the respective process section occurs a.

Depending on the selected entertainment offer, the number of predetermined process sections can also include at least one process section caused by a user input, usually a plurality of process sections caused by respective user inputs, which interact interactively during the course of the entertainment offer, in particular in the case of a game . If a user input is made in real time, then according to the invention this is now automatically accompanied by the synchronous entry of the modes assigned to this respective user input or to this process section triggered by this user input for the selected smart terminal devices.

Depending on a user input made in real time, the “Immersive Home” application adapts the resulting current status of the entertainment offer process and makes this or the process section triggered by the user input visible on the control center. If the entertainment offer is a game, the “Immersive Home” application distributes based on this current status and depending on the ver The different roles of the participating users send further instructions or instructions to the different users, which they can receive via their respective user terminals assigned to them. Furthermore, as described above, each of the selected smart devices is context-based placed in the mode assigned to the current segment of the entertainment offer for the respective smart device, so that the spatial atmosphere created by the selected smart devices is tailored precisely to this current segment of the entertainment -Offer is coordinated.

In a further embodiment of the method according to the invention, the control center, as a display unit common to all users participating in the entertainment offer, dynamically provides data about a respective current status of the process of the entertainment offer.

In a further embodiment, the entertainment offer can also be selected from the group including films, e-concerts and sports events in addition to an online game. The entertainment offering is managed by the “Immersive Home” application and executed on the external server or in a first cloud on which the “Immersive Home” application itself is also executed.

In yet another embodiment of the invention, the smart terminal devices are selected from the group at least comprising lamps, speakers, heating elements, air conditioning units, fans, with the smart home services existing for the selected smart terminal devices being provided via a respective functional connection to or from a second cloud become. Depending on the type of smart device, the smart home services can be selected from well-known services such as Hue, Sono, etc.

In the embodiment of the method according to the invention, it is provided that when the selected entertainment offer is started using the “Immersive Home” application, the selected smart devices are automatically integrated into the entertainment offer by the “Immersive Home” application on the first Connects the cloud-based entertainment offering with the existing smart home services for the selected smart devices in the second cloud.

A further subject of the invention is a smart home platform according to claim 9 with several different digital ecosystems of smart devices, where each smart device can adopt different modes, an “Immersive Home” application and a computer implementing and executing the “Immersive Home” application, via which an entertainment database that provides a number of entertainment offers can be accessed, the smart home platform being designed to carry out a method according to the invention and described above.

A significant advantage of the method according to the invention and the smart home platform according to the invention is that the operating modes of devices or smart devices, taking into account a selected and running entertainment offer, are suitable in terms of content (i.e. suitable for the context of the entertainment offer) and synchronous with the process of the entertainment offering can be discontinued. This creates an immersive entertainment environment that matches the selected entertainment offering, which allows a user to immerse themselves in the selected entertainment offering with multiple senses, i.e. visually, acoustically and/or tactilely, thus providing the user with a special user experience. The context-sensitive “Immersive Home” application or application provided according to the invention makes use of contextual data or contextual data associated with the respective entertainment offer or its process and, based on this contextual data, controls the respective data provided for the selected smart devices Smart home services or services. The “Immersive Home” application is a context-sensitive application that uses a context of the entertainment offering or information about the process of the entertainment offering to control the selected smart devices via the smart home services provided for the selected smart devices to vote.

The context data assigned to the respective process sections of the entertainment offer are used by the “Immersive Home” application to adapt the behavior of the smart home services connected to the “Immersive Home” application of the individual selected smart devices and thus of the selected smart devices themselves . Context data can be defined as any information that can be used to characterize a respective section of the entertainment offering. This context data is previously known and assigned to a respective process section and stored as such in a retrievable manner. The process of the entertainment offering is dynamically made up of process sections, each of which is assigned context data. The process of an entertainment offer does not have to include all of the specified number of process sections Process sections include, as a rule, the process of the entertainment offer only includes a subgroup of all possible process sections. The sequence of the subgroup of process sections from the predetermined number of process sections results dynamically and, in the case of a game, depending on the respective user input.

It is understood that the features mentioned above and those to be explained below can be used not only in the combination specified in each case, but also in other combinations or alone, without departing from the scope of the present invention.

• 1 eine schematische Darstellung einer Smartphone-Plattform nach einer Ausführungsform der Erfindung.

The invention is shown schematically in the drawing using an exemplary embodiment and is described in detail below with reference to the drawing. It shows

• 1 a schematic representation of a smartphone platform according to an embodiment of the invention.

1 shows a smart home platform 1 with several different digital ecosystems of smart devices 10, 20, 30, 40, an “Immersive Home” application 50 and a computer 60 implementing and executing the “Immersive Home” application, via which access to an entertainment database 70, which provides a number of entertainment offers, can be accessed, the smart home platform 1 being designed to carry out an embodiment of the method according to the invention.

In the embodiment of the smart home platform 1 shown here, the “Immersive Home” application 50 is implemented and executed on a computer 60 in a first cloud 101. The entertainment database 70 can also be located in the first cloud 101 or at least be functionally connected to it, as indicated here by arrow 110. A functional connection enables the components connected to each other via this connection, such as here the entertainment database 70 and the first cloud or the computer 60, to transmit data to the other component or to receive data from the other component or between the components exchange.

Each smart device 10, 20, 30, 40 can adopt different modes. Furthermore, there are respective smart home services for the respective smart terminal devices 10, 20, 30, which are located in a second cloud 102 and which control the respective modes of the smart terminal devices 10, 20, 30. The smart terminal devices 10 are, for example, lights whose luminosity and/or color can be controlled and varied using at least one smart home service that exists for them. The smart terminal devices 20 are heating or air conditioning devices whose heating and/or cooling and/or ventilation function can be controlled and varied using at least one existing smart home service. The smart terminal devices 30 are loudspeakers whose output function can be controlled and varied using at least one existing smart home service.

One of the smart devices is an output and display device 40, here a smart TV device, which is selected as the control center among the smart devices and to which the “Immersive Home” application 50 is downloaded, so that a user can use the TV device 40 the “Immersive Home” application 50 can start. After or when the “Immersive Home” application 50 is started, the user or the TV set is then automatically connected to the first cloud 101. The user can now authenticate himself to the smart home services existing for the smart devices 10, 20, 30. The authentication can take place directly on or via the TV device 40 or the user can carry out the authentication using a deep link on the TV device 40 on a user terminal 80. The “Immersive Home” application 50 then automatically connects to these smart home services, in particular cloud services, which exist for the smart devices 10, 20, 30 and control the respective different modes and run in the cloud, as indicated by arrow 120.

The “Immersive Home” application 50 then provides a list comprising the smart devices of the several different digital ecosystems on a display unit 41 of the TV set 40 for the selection of the smart devices 10, 20, 30 participating in the immersive entertainment environment. The smart terminal devices 10, 20, 30 selected by the user as participants are then recorded by the “Immersive Home” application 50 via respective inputs from the user via or into an input unit 42 of the TV set 40. The display unit 41 and the input unit 42 can also be combined, for example, in the form of a touchscreen.

The user can now select and start an entertainment offer via the TV set 40 via remote access to the entertainment database 70. The further sequence of an embodiment of the method according to the invention is explained below using the example of an online game.

When the online game is started on the TV device 40, a game session is generated via MQTT in the first cloud 101. By means of The “Immersive Home” application 50 also automatically integrates the smart devices 10, 20, 30 previously selected via the list provided into the online game.

Other players can now register or subscribe to this game session via MQTT via a user terminal 80 assigned to them, such as a smartphone, a tablet, with a deep link, for example a QR code, which means that via the MQTT protocol a communicative network enabling data transmission between the game session running on the first cloud 101 and the respective user terminals 80 is generated. After these respective communicative connections have been established, the game is started by the “Immersive Home” application 50 on the first cloud 101 and the process sections or the game steps of the game are controlled and a respective status of the game is displayed to the respective players via MQTT on their respective user terminals 80 . Depending on their role in the game and depending on the current status of the game, i.e. the current section of the game, each player receives individual instructions, in the case of UNO, for example, a player-specific set of cards or, in the case of Escape Room, player-specific background information Story. Depending on the players' respective inputs, the “Immersive Home” application 50 adapts the subsequent section of the game, for example the storyline in the case of Escape Room, and displays this on the display unit 41 of the TV set 40. Furthermore, the “Immersive Home” application 50 gives logically resulting further instructions to the user terminals 80 of the different players depending on the updated status of the game, with the instructions being distributed among the different players depending on their respective role in the game are usually different.

Furthermore, by means of the “Immersive Home” application 50 and its connection to the respective smart home services controlling the selected smart devices 10, 20, 30, the respective modes of the selected smart devices 10, 20, 30 are synchronized with the course of the game based on the respective ones Context data associated with the course of the game is automatically changed. For example, a jungle sound is emitted by the speaker 30 when a player enters the jungle in an escape room. Furthermore, the lighting of the lights 10 and air conditioning can be adjusted accordingly by the air conditioning device 20. With each new process section of the game, the modes of the respective smart terminal devices 10, 20, 30 can change depending on the context data assigned to the new process section. This means that the environment adapts to the course of the game in real time. This enables a gaming experience in which the user can fully immerse themselves in the game and its gaming atmosphere.

REFERENCE SYMBOL LIST:

1

Smarthome platform

10

smart device, light

20

smart device, air conditioning device

30

smart device, loudspeaker

40

smart device, TV set

41

Display unit

42

Input unit

50

“Immersive Home” application

60

external computer

70

Entertainment database

80

User terminal

101

first cloud

102

second cloud

110

Arrow

120

Arrow

Claims (9)

Method for operating a smart home platform (1) as an immersive entertainment environment, the smart home platform (1) comprising several different digital ecosystems of smart devices (10, 20, 30, 40), each smart device being able to adopt different modes, and wherein a number of entertainment offers are provided by an entertainment database (70), each entertainment offer comprising respective context data assigned to a process of the respective entertainment offer, with each entertainment offer being assigned a number of predetermined process sections, the Context data is previously known, respective context data is assigned to a respective process section of the predetermined number of process sections of the respective entertainment offer and is stored as such in a retrievable manner, and the process of the respective entertainment offer is dynamically made up of process sections to which context data is assigned are, composed, whereby the process of the entertainment offer includes all or a subgroup of the predetermined number of process sections and the sequence of the process sections results dynamically, in particular depending on the respective user input, with the method: - one on one of the smart devices ( 40) as The “Immersive Home” application (50) provided and started by the control center connects to smart home services existing in the smart home platform (1) for the smart devices (10, 20, 30, 40) and controlling the respective different modes, and several as participants selected smart devices (10, 20, 30) from the various ecosystems are recorded in the immersive entertainment environment; and - by starting an entertainment offer selected via the control center (40) via remote access to the entertainment database (70), the selected smart devices (10, 20, 30) are automatically integrated into the "Immersive Home" application (50). Entertainment offer are integrated and their respective modes are automatically changed synchronously with the flow of the entertainment offer based on the respective context data assigned to the respective flow sections from which the flow of the entertainment offer is dynamically composed, whereby the "Immersive Home" Application assigns one of the modes of the respective selected smart device to each process section of the entertainment offer for each selected smart device in a context-based manner and sets this respective mode for the respective selected smart device automatically and in real time when the respective process section occurs. Procedure according to Claim 1 , in which the “Immersive Home” application (50) provides a list comprising the smart devices of the several different digital ecosystems for selecting the smart devices (10, 20, 30) participating in the immersive entertainment environment and on a display unit (41) of the control center (40) is displayed, with the smart terminals (10, 20, 30) selected as participants being recorded by the “Immersive Home” application (50) via respective inputs via or into an input unit (42) of the control center (40). . Procedure according to Claim 1 or 2 , in which a user registers to participate in the selected entertainment offer by recording an identifier entered via the control center (40) and / or via a user terminal (80) functionally connected to the control center (40) on an identifier on the control center (40 ) and/or the website of the selected entertainment offering accessed and displayed on the user terminal (80). Procedure according to one of the Claims 1 until 3 , in which the number of predetermined process sections includes at least one process section assigned to a user input. Procedure according to one of the Claims 1 until 4 , in which the control center (40), as a control center common to all users participating in the entertainment offer, dynamically provides data about a respective current status of the process of the entertainment offer to all users participating in the entertainment offer. Procedure according to one of the Claims 1 until 5 , in which the entertainment offering is selected from the group comprising online games, films, e-concerts, sports events, and the flow of the entertainment offering in a first cloud (101) in operational connection with the “Immersive Home” Application (50) is directed and executed. Procedure according to one of the Claims 1 until 6 , in which the smart terminal devices (10, 20, 30, 40) are selected from the group at least comprising lamps, loudspeakers, heating elements, air conditioning units, fans, display devices, the Smart. existing for the selected smart terminal devices (10, 20, 30). Home services are provided via a respective functional connection (120) of the “Immersive Home” application (50) to or from a second cloud (102). Procedure according to Claim 6 and 7 , in which when you start the selected entertainment offer using the “Immersive Home” application (50), the selected smart devices (10, 20, 30) are automatically integrated into the entertainment offer by using the “Immersive Home” application (50 ) connects the entertainment offer running on the first cloud (101) with the smart home services existing for the selected smart devices (10, 20, 30) in or from the second cloud (102). Smart home platform (1) with several different digital ecosystems of smart devices (10, 20, 30, 40), whereby each smart device can adopt different modes, an “Immersive Home” application (50) and an “Immersive Home” application (50) implementing and executing computer (60), via which an entertainment database (70), which provides a number of entertainment offers, can be accessed, the smart home platform (1) being designed, a method according to one of Claims 1 until 8th to carry out.

Images