DE102013223949B4 - Method for enabling remote control and/or remote monitoring of the operation of household appliances using a communication device and corresponding system - Google Patents

Abstract

Method for enabling remote control and/or remote monitoring of the operation of at least one household appliance (9) of a user using a communication device (4) that is separate from the household appliance (9) and communicates with an Internet server (2), with the steps: - establishing a communication connection from the communication device (4) to the Internet server (2),- the communication device (4) receiving an authentication input from the user, the authentication input being made by the user at an input device (5) of the communication device (4), and transmission of the authentication input from the communication device (4) to the internet server (2),- comparing the transmitted authentication input with data stored on the internet server (2) and authenticating the user depending on the comparison and- enabling remote control and/or remote monitoring after successful authentication, characterized that by means of a camera (12) arranged on and/or in a household appliance (9) of the user, at least one reference image (15) of an interior and/or an area surrounding the household appliance (9) is provided and at least one image region of the reference image (15) transmitted to the communication device (4) and displayed together with at least one further image (16) on a display (5) of the communication device (4), wherein the authentication input includes the selection of one of the displayed images (15, 16) by the user and the authentication depends on the selection.

Description

The invention relates to a method for enabling remote control and/or remote monitoring of the operation of at least one household appliance of a user using a communication device that is separate from the household appliance and communicates with an Internet server. A communication connection is set up from the communication device to the Internet server. The communication device receives an authentication input from the user, which is made by the user at an input device of the communication device. The authentication input is transmitted from the communication device to the internet server. The authentication input is then compared on the Internet server with data stored on the Internet server, and the user is authenticated as a function of the comparison. Remote control and/or remote monitoring is enabled after successful authentication. The invention also relates to a system which is designed to carry out such a method.

It is already state of the art to equip household appliances with a communication module via which the household appliances are connected to the Internet. The home appliances can then communicate with an Internet server, enabling remote monitoring and remote control of the operation of the home appliances. Data communication between household appliances on the one hand and an internet server on the other hand is made possible, for example, by the "Home Connect" system, which is offered by the applicant's company.

The user of the household appliances has the option of using a corresponding communication device - for example a mobile communication device (e.g. a mobile phone or a tablet PC) or a conventional PC - to access the internet server and, through the intermediary of the internet server, to control or to operate the household appliances monitor. In this case, for example, control commands can be transmitted to the household appliances, which are generated as a function of operator inputs by the user on his communication device. On the other hand, it is also possible to transmit status data from the household appliances to the Internet server and make it available to the user. This data can then be shown on a display of the user's communication device.

The user's personal data is usually assigned to a user account which is managed on the Internet server. It is therefore assumed that a corresponding user account has been created on the Internet server. A large number of different household appliances can then be assigned to this user account, which can be remotely controlled or remotely monitored from the user account. This means that several household appliances can be registered on each user account. Access to the data of each user account requires the user to enter their user name (identifier or identifier) and usually a password, and thus to authenticate themselves. As is known, this identification data of the user can be entered on a PC or on a portable communication terminal by using a conventional keyboard of the PC or the virtual keyboard of the portable communication terminal.

Purely knowledge-based mechanisms can therefore be used to secure user accounts, which are based on the exclusive knowledge of the user himself. Here, for example, the user must have his secret password, a PIN or an input pattern ready in order to be able to successfully authenticate himself in his user account. On the other hand, so-called multi-factor authentication mechanisms are also used to secure user accounts, which are based on the one hand on the user's exclusive knowledge (e.g. password) and on the other hand on additional possession, such as unique certificates, fingerprints and the like. Since abstract knowledge - such as cryptic passwords - are usually relatively difficult to remember, they are rarely or never changed. On the other hand, users also tend to use relatively simple passwords, which are relatively easy to remember but also insecure. If changes to the passwords are forced by the system, this can be viewed negatively by the users. Overall, the user accounts on the Internet server can only be insufficiently protected using passwords.

US 2010/0 170 289 A1 describes a system for providing a seamless image of an interior of a home appliance. From the US 2004/0016242 A1 a method for remote control of a refrigerator connected to the Internet is already known. The US 2013/0 214 935 A1 deals with an information management system for home appliances. US 2005/0 060 554 A1 discloses a method for authorizing the use of an electronic device using the identification of an image. US 2007/0 101 010 A1 addresses a method in a computer system for verifying an individual's identity. US 2004/0 250 138 A1 again busy with an events-based graphical password system.

It is the object of the invention to show a solution as to how the user can be authenticated on the Internet server in a method of the type mentioned at the outset in a particularly secure manner and without much effort for the user.

According to the invention, this object is achieved by a method and by a system having the features according to the respective independent patent claims. Advantageous embodiments of the invention are the subject matter of the dependent patent claims, the description and the figures.

A method according to the invention serves to enable remote control and/or remote monitoring of the operation of at least one household appliance of a user using a communication device which is separate from the household appliance and communicates with an Internet server. The remote control and/or remote monitoring thus takes place through the mediation of the internet server. A communication connection is set up from the communication device to the Internet server. The user then makes an authentication input on an input device of the communication device, with this authentication input being received by the communication device and transmitted to the Internet server. This authentication input is compared to data stored on the Internet server and the user is authenticated based on the comparison. After successful authentication, remote control and/or remote monitoring is then carried out. It is provided that at least one reference image of an interior and/or a surrounding area of the household appliance is provided by means of a camera arranged on and/or in a household appliance of the user and at least one image region of the reference image - in particular the entire reference image - is transmitted to the communication device and together is displayed with at least one additional image on a display of the communication device, wherein the authentication input includes the selection of one of the displayed images (i.e. the at least one image region of the reference image or the at least one additional image) by the user and the authentication takes place depending on the selection.

According to the invention it is thus provided that several images are displayed for selection on the display of the communication device, one of which corresponds to a digital reference image or a partial region thereof, which shows the interior and/or surrounding area of one of the registered household appliances of the user. This reference image is provided by a camera which is arranged on and/or in the user's household appliance. The communication device prompts the user to select one of the displayed images. The method according to the invention enables very simple authentication with just one "click", which is also carried out particularly securely. Namely, it is assumed that the contents of the interior of the household appliance and/or its surroundings change over time, which requires the user's ever-changing knowledge. This is because the user must know what is currently in the interior and/or surrounding area of his household appliance. The older the knowledge of this content, the lower the chance that the correct reference image can be identified. The user accounts can consequently be secured particularly reliably and effectively with the method according to the invention.

In the present case, a communication device is understood to mean, in particular, a (personal) computer or a portable communication terminal device, such as a mobile phone, which is connected to the Internet and can therefore communicate with the Internet server via the Internet. Using such a communication device, the user can log into the personal user account by entering his authentication data and thus authenticate himself. The authentication input can be entered, for example, via an Internet portal of the Internet server. To do this, the user can use an Internet browser of the communication device. In addition or as an alternative, it is also possible to install a corresponding application or application on the communication device, which then communicates with the Internet server. The authentication then takes place via this application.

In the present case, a household appliance is understood to mean, in particular, a device which is used for housekeeping. In particular, this can be a large household appliance, such as an appliance for washing items, a household appliance for preparing food, for example a cooking appliance, such as an oven or stove, a household appliance for storing and preserving food, such as a household refrigerator, for example a refrigerator or a freezer or a fridge-freezer combination, an air conditioner or a dishwasher. A device for the care of laundry items can be a washing machine, a tumble dryer or a washer-dryer.

In addition to the at least one image region of the reference image - in particular in addition to the entire reference image - an image of an interior and/or surrounding area of a household appliance of at least one other user or of at least one other user account can be displayed on the display of the communication device as the at least one further image. Provision is then preferably made for the successful authentication to take place only if the selected image corresponds to the reference image or an image region of the reference image. Remote control and/or remote monitoring is thus only made possible if the selected image corresponds to the reference image or its sub-region. In other words, access to the user's user account is only released if the user selects the correct reference image of his own household appliance.

In one embodiment, provision is also made for household appliances of the same type, in particular household appliances of the same series, to be depicted in the reference image and the at least one further image of the other user. This further increases the security of the authentication because it is not enough for the user to know their own household appliance. Rather, the user who wants to authenticate himself on the Internet server must also have knowledge of the contents of the interior of his household appliance and/or of the surrounding area of the household appliance. In the displayed images, the respective household appliances are preferably also shown from the same perspective or from the same point of view and with the same scaling. This also further improves the security of the authentication.

In addition or as an alternative to an image of another user, at least one image that was provided by the same camera at an earlier point in time than the reference image can also be displayed as the at least one further image. The reference image can be a current image of the household appliance, while the at least one other image can represent an “archive image” that was recorded at an earlier point in time using the same camera. Successful authentication can only take place here if the selected image corresponds to the (current) reference image or an image region of the reference image. This also increases the security of the authentication because the user must know the current content of his household appliance or his current environment in order to be able to successfully authenticate himself on the Internet server. "Old" knowledge is not enough here. In particular, a minimum age can be specified for the “archive images”, from which these images can be displayed for selection. This ensures that the content of the household appliance and/or the environment has changed sufficiently.

Furthermore, in addition or as an alternative, it can also be provided that at least one partial region of the same reference image is displayed as the at least one further image, so that several, in particular different, partial regions of the same reference image are displayed on the display of the communication device. In this case, successful authentication can only take place on the condition that the selected image corresponds to a predetermined reference sub-region of the reference image. For example, a partial region can be used as a reference partial region, in which a predetermined object—for example foodstuffs added last—is depicted. Here an even more detailed knowledge of the user with regard to the contents of the household appliance or its surroundings is assumed and security is thus further improved.

It proves to be particularly advantageous if an interior of a household appliance for storing and preserving food, in particular a household refrigeration appliance such as a refrigerator, a freezer or a fridge-freezer combination, is imaged in the reference image. In this embodiment, the user must therefore have knowledge of what food is currently in his household appliance. This embodiment takes advantage of the fact that the contents of a refrigerator or freezer change relatively frequently over time. Knowing the content a week ago is usually only partially sufficient to be able to reliably identify the current reference image. The security of the authentication can thus be brought to a maximum. In order to be able to capture the interior of a refrigeration device, the camera can be a so-called night vision camera, for example, or else a conventional digital camera, which also has a lighting device that is activated when the image is recorded.

The reference image is preferably first transmitted from the camera to the Internet server. If the communication connection is then established from the communication device to the Internet server, the reference image and the further images can be transmitted from the Internet server to the communication device. The images are thus managed by the Internet server. In this way, manipulation of the data by unauthorized third parties can be prevented.

It is preferred if before the transmission of the reference image to the communication device tion data are removed from the reference image, which allow an assignment to the household appliance of the user or the household appliance uniquely identified. The reference image provided by the camera is expediently provided with an identifier for the household appliance or the user before it is transmitted to the Internet server, so that an unambiguous assignment on the Internet server is possible. This identifier is then removed from the reference image before this reference image is transmitted from the Internet server to the communication device. This data can be, for example, meta/header data, such as Exif. This embodiment prevents manipulation of the reference image on the part of the communication device.

One embodiment provides that at least one reference object depicted in the reference image—for example a food item—is determined by the Internet server and/or the communication device and the user's authentication input also includes the user's selection of an object in the reference image. Successful authentication then only takes place on the condition that the selected object corresponds to the reference object. This can be implemented, for example, in such a way that one of the displayed images is first selected and, after the correct reference image has been selected, the user is also prompted to select an object depicted in the reference image.

However, the object can also be selected at the same time as the image is selected, so that selecting the object automatically also selects the image in which the selected object is depicted. If this selected object corresponds to the reference object, the user is successfully authenticated. The food last added and/or a favorite food can be determined as a reference object. Such authentication is then particularly secure.

The user can also specify an identifier for the user (identification) via the authentication input. This enables the authentication input to be uniquely associated with the user's user account on the Internet server.

In order to always make a current reference image available on the Internet server, provision can be made for a current reference image to be transmitted from the camera to the Internet server at predetermined time intervals. The current reference image can then be displayed on the display of the communication device. This also contributes to increasing security during authentication.

A system according to the invention includes an internet server and a communication device and is designed to carry out a method according to the invention.

Further features of the invention result from the claims, the figures and the description of the figures. All features and feature combinations mentioned above in the description and the features and feature combinations mentioned below in the description of the figures and/or shown alone in the figures can be used not only in the combination specified, but also in other combinations or alone.

The invention will now be explained in more detail using a preferred exemplary embodiment and with reference to the accompanying drawings.

• 1 in schematischer Darstellung ein System gemäß einer Ausführungsform der Erfindung;
• 2 ein Flussdiagramm eines Verfahrens gemäß einer Ausführungsform der Erfindung; und
• 3 in schematischer Darstellung eine graphische Benutzeroberfläche, die auf einem Display einer Kommunikationseinrichtung angezeigt wird und über welche eine Authentifizierungseingabe eines Benutzers vorgenommen wird.

Show it:

• 1 in a schematic representation a system according to an embodiment of the invention;
• 2 a flow chart of a method according to an embodiment of the invention; and
• 3 a schematic representation of a graphical user interface that is displayed on a display of a communication device and through which an authentication input of a user is made.

a in 1 The system 1 shown includes an internet server 2 which is connected to the internet 3 . A large number of user accounts are managed on the Internet server 2, each of which is used for remote control and/or remote monitoring of the operation of household appliances. These user accounts can also be used to provide additional functionalities related to household appliances, such as help in the event of a malfunction, help with using the household appliances and the like.

The system 1 also includes a communication device 4, which in the exemplary embodiment is designed as a portable communication terminal, such as a cell phone. The communication device 4 has a display 5 on which a graphical user interface 6 can be generated. This can be provided, for example, by a corresponding application that is installed on the communication device 4 . The display 5 can, for example Touch screen with a touch-sensitive surface 7 his. The communication device 4 is also connected to the Internet 3, for example via a cellular network. The communication device 4 can thus communicate with the internet server 2 .

A household 8 can contain a large number of household appliances, of which 1 only one household appliance 9 is shown as a representative. In the exemplary embodiment, the household appliance 9 is a fridge-freezer combination with a refrigerator 10 and a freezer 11. The household appliance 9 has a camera 12, by means of which images of an interior of the refrigerator 10 can be provided. The camera 12 can be a CCD camera or a CMOS camera, for example. The household appliance 9 also includes a communication unit 13, via which the household appliance 9 is connected to the Internet 3, for example via a local (wireless) network of the household 8.

The camera 12 captures reference images of the interior of the refrigerator 10 and transmits the reference images to the internet server 2 at predetermined time intervals. The reference images can be transmitted to the internet server 2 every hour or every two hours or once a day, for example.

The user of the household appliances 9, which are located in the household 8 and are registered on the Internet server 2, can use the communication device 4 to access his user account on the Internet server 2 and use this user account to control and/or monitor the operation of the household appliances 9. In order for the user to be able to access his user account on the Internet server 2, the user must be authenticated on the Internet server 2. A method according to an embodiment of the invention will now be described with reference to FIG 2 explained in more detail. 2 FIG. 1 shows a flowchart of the method, steps being carried out by the communication device 4 being shown on the left-hand side, while steps being carried out by the Internet server 2 are shown on the right-hand side.

The method begins in a first step S1, in which the user activates the named application, for example, which then generates the graphical user interface 6 on the display 5. In a further step S2, the user enters an identification or an identifier on the communication device 4--for example using the touch-sensitive surface 7--which is/are transmitted from the communication device 4 to the Internet server 2. The identifier is received by the internet server 2 according to step S3. The identifier can also already be stored in the application on the communication device 4, so that no explicit input by the user is required. The identifier does not have to be entered every time, so that a 1-click authentication is also possible.

According to step S4, the internet server 2 checks whether the identifier is correct or not. If the identifier is incorrect, for example because there is no user account for this identifier, access is rejected in accordance with step S5. If the identifier is correct, the method goes to a further step S6, in which several images are selected by the Internet server 2.

These images include, on the one hand, the current reference image of the interior of the user's refrigerator 10 and several other images in which the interior of an identical refrigerator belonging to other users is also shown. All of the images show the interior of refrigerators from the same series. In addition or as an alternative to the images of other users, the Internet server 2 can also select images that were provided earlier as the current reference image using the same camera 12, ie “archive images”. In addition or as an alternative, it is also possible to divide the reference image into a plurality of sub-regions and to define one of the sub-regions as a reference sub-region.

From the images selected by the Internet server 2, data are removed that enable a technical assignment of the images to the respective household appliances. This is, for example, meta/header data, such as Exif. The selected images are then transmitted to the communication device 4 in the form of a message according to step S7 and displayed on the display 5 of the communication device 4 in a further step S8.

An exemplary graphical user interface 6 with the displayed images is in 3 shown. How out 3 shows, the graphical user interface 6 includes an input field 14 through which the user can enter the above identifier. As mentioned, the identifier does not have to be entered every time, but can also be stored permanently in the application. The display 5 also shows the current reference image of the interior of the refrigerator 10, which 3 is denoted by 15. In addition to this reference image 15, other images 16 that come from other user accounts and/or represent “archive images” of the same household appliance 9 are also displayed len. All images 15, 16 each show the interior of a refrigerator of the same series and from the same perspective. The user is now prompted to select one of the images 15, 16 displayed. To do this, the user only needs to touch one of the displayed images 15, 16 via the touch-sensitive surface 7. Alternatively, it can also be provided that the selected image 15, 16 is first marked and the user then has to touch another button 17 in order to confirm this authentication input.

Referring again to 2 an authentication input is made by the user according to step S9, in which one of the displayed images 15, 16 is selected. The information about which image 15, 16 was selected is then transmitted to the Internet server 2, which checks according to step S10 whether the selected image corresponds to the reference image 15. If the selected image corresponds to the reference image 15, the method goes to a step S11 in which access to the user account is released. If the selected image does not correspond to the reference image 15, the method goes to step S5, in which access is denied.

Successful authentication therefore only takes place if the selected image corresponds to the current reference image 15 .

In addition, provision can also be made for a reference object—for example food—to be determined in the reference image 15 by the internet server 2 . After the user has selected the reference image 15 or simultaneously with the selection of the reference image 15 (with a single "click"), the user can also be prompted to select one of the objects depicted in the reference image 15 . If the selected object then corresponds to the reference object, access to the user account can be granted. The reference object can be, for example, the food that was last added and/or favorite food. In particular, this embodiment can be implemented when the entire reference image 15 is displayed without being divided into several separate sub-regions.

If, on the other hand, several partial regions of the reference image 15 are displayed separately and thus made available for selection independently of one another, successful authentication can only take place on the condition that the selected image corresponds to a predetermined reference partial region of the reference image 15 . A subregion in which a predetermined object—for example foodstuffs that were last added—is depicted can also be used as a reference subregion.

Subsequent steps, such as the temporary certification of client and user using a valid token or the like, can also be implemented as required.

Reference List

1

system

2

internet server

3

Internet

4

communication facility

5

screen

6

graphical user interface

7

touch-sensitive surface

8th

Household

9

household appliance

10

Refrigerator

11

freezer

12

camera

13

communication unit

14

input box

15

reference image

16

more pictures

17

button

S1, S2, S3, S4, S5, S6, S7, S8, S9, S10, S11

steps

Claims (12)

Method for enabling remote control and/or remote monitoring of the operation of at least one household appliance (9) of a user using a communication device (4) that is separate from the household appliance (9) and communicates with an Internet server (2), with the steps: - establishing a communication connection from the communication device (4) to the Internet server (2), - the communication device (4) receiving an authentication input from the user, the authentication input being made by the user at an input device (5) of the communication device (4), and transmission of the authentication input of the Communication device (4) to the internet server (2), - comparing the transmitted authentication input with data stored on the internet server (2) and authenticating the user depending on the comparison and - enabling remote control and/or remote monitoring after successful authentication, characterized in that at least one reference image (15) of an interior and/or a surrounding area of the household appliance (9) is provided by means of a camera (12) arranged on and/or in a household appliance (9) of the user and at least one image region of the reference image (15) to the Communication device (4) and is displayed together with at least one other image (16) on a display (5) of the communication device (4), wherein the authentication input includes the selection of one of the displayed images (15, 16) by the user and the authentication depends on the selection. procedure after claim 1 , characterized in that at least one image of an interior and/or surrounding area of a household appliance (9) of another user is displayed as the at least one further image (16) and successful authentication only takes place if the selected image corresponds to the reference image (15) or corresponds to an image region of the reference image (15). procedure after claim 2 , characterized in that household appliances (9) of the same type, in particular household appliances (9) of the same series, are displayed in the reference image (15) and the at least one image (16) of another user. Method according to one of the preceding claims, characterized in that as the at least one further image (16) at least one image is displayed which was provided by the same camera (12) at an earlier point in time than the reference image (15), the successful authentication only takes place if the selected image corresponds to the reference image (15) or an image region of the reference image (15). Method according to one of the preceding claims, characterized in that at least one partial region of the same reference image (15) is displayed as the at least one further image (16), so that several, in particular different, partial regions of the reference image (15) are displayed on the display (5). and the successful authentication only takes place if the selected image corresponds to a predetermined reference sub-region of the reference image (15). Method according to one of the preceding claims, characterized in that an interior of a household appliance (9) for storing and preserving food, in particular a household refrigerating appliance (10, 11) of the user is depicted in the reference image (15). Method according to one of the preceding claims, characterized in that the at least one image region of the reference image (15) is transmitted from the camera (12) to the internet server (2) and from the internet server (2) to the communication device (4). Method according to one of the preceding claims, characterized in that before the at least one image region of the reference image (15) is transmitted to the communication device (4), data are removed from the at least one image region of the reference image (15) which indicate an assignment to the household appliance ( 9) allow the user. Method according to one of the preceding claims, characterized in that at least one reference object depicted in the reference image (15) is determined by the Internet server (2) and/or the communication device (4) and the authentication input also involves the selection of an object in the reference image (15 ) by the user, whereby successful authentication only occurs if the selected object corresponds to the reference object. Method according to one of the preceding claims, characterized in that the authentication input additionally comprises an identifier (ID) of the user. Method according to one of the preceding claims, characterized in that at least one image region of a current reference image (15) is transmitted from the camera (12) to the Internet server (2) at predetermined time intervals and the display (5) of the communication device (4) the at least one image region of the current reference image (15) is displayed. System (1) with an Internet server (2) and a communication device (4), wherein the system (1) is designed to carry out a method according to one of the preceding claims.

Images