KR102236593B1 - Controlling and displaying method of electronic apparatus - Google Patents

Abstract

Methods, systems, and computer programs for how the first device can display a user interface on the second device are described. The first and second devices can communicate over the network. Changes are detected in the operating state of the first device. In response to the detected change, a message is sent from the first device. The message identifies computer-readable instructions stored on the first device. The message is received at the second device, and in response to receipt, computer-readable instructions are retrieved from the first device. Thereafter, the second device uses at least a portion of the user interface determined by the computer-readable instructions retrieved from the first device, the computer-readable instructions to present the user interface to the second device.

Description

Control method and display method of electronic devices. {CONTROLLING AND DISPLAYING METHOD OF ELECTRONIC APPARATUS}

The present invention relates to a method, an apparatus, and a system capable of displaying a user interface of another device on one device. In particular, the present invention can be used in a Plug and Play system used in a local network.

Increasing numbers of devices are being connected to each other via local networks in homes and offices. This connection can be used to allow one device to be controlled by a second device. This may be used to provide user control of one device to another device via a user interface. The universal plug and play (UPnP) standard is an example of a standard that allows devices to be interconnected.

Under the UPnP standard, compatible devices are configured to automatically connect to one network (such as a home network) and discover other devices on the network. These discoveries are being done using the Simple Service Discovery Protocol (SSDP), which allows devices to announce their presence on the network so that they can be discovered by other devices and discover these other devices in the same way.

A discovery (discovery) message consistent with the SSDP protocol may be sent by the first device. This discovery message includes a link to an Extensible Markup Language (XML) file containing more information about the first device. The second device can use this link to retrieve the XML file, and retrieve information about the first device.

However, this process is limited in the functionality that can be provided to connect devices.

At least one embodiment, method, apparatus, system, and software is provided for support or execution functions for one device to display a user interface on another device.

This is achieved by a combination of features mentioned in each independent claim. Accordingly, the dependent claims describe the implementation of various embodiments in more detail.

According to the first embodiment, a method for allowing a first device to display a user interface on a second device is provided, the first and second devices capable of communicating via a network, and the method comprises: : To detect changes in the operating state of the first device; In response to the detection, sending a message from the first device; The message may include a message identifying computer-readable instructions stored in the first device; Receiving a message on the second device; In response to receiving, retrieving computer-readable instructions from the first device; And using at least a portion of the user interface determined by the computer-readable instructions retrieved from the first device, the computer-readable instructions to present the user interface to the second device.

In conventional UPnP systems, the discovery process may be used to allow a second device to identify and learn about the first device. According to this discovery process, the second device may be configured to connect to the first device. You can provide a user interface for the first device, for example by downloading software from the Internet. However, such systems not only require the second device to be preconfigured, but also require the user to request the user interface of the second device before the two devices can be connected.

The inventor recognizes that there are benefits to be gained from providing a way. In this way, the first device can have the user interface appear on the second device without being preconfigured. And there is no need for users to request a user interface on the second device. This, for example, allows the first device to alert the second device without being preconfigured, which is not possible with a conventional UPnP system.

Thus, when a state change is determined at the first device, a message is sent. This message, when received by the second device, allows the second device to retrieve computer-readable instructions from the first device. These computer-readable instructions may eventually be used to represent the user interface.

Thus, the system provides many benefits. First, the first device can cause the user interface to appear on the second device without being preconfigured. Secondly, by having a message identify computer instructions on the first device, the message itself may be stored in a standard format, regardless of the type of first device sending the message, and thus a wide range. And ensure that no incompatible second device is accidentally caused.

In the context of the above invention, an example of a first device may be a mobile phone and a change in operating state may be related to an incoming call (an incoming call). The second device may be a TV. In this way, when an incoming call (an incoming call) is detected, the mobile phone may send a message to the TV. This message may cause the TV to retrieve computer-readable instructions from the phone. This may cause the TV to present a user interface that allows the user watching TV to be alerted on the phone. Importantly, such an alarm is possible even though the TV is not previously configured as an interface with, for example, a cell phone.

The message may identify a network location, from which computer-readable instructions may be retrieved by the second device. In order for computer-readable instructions to be used by the second device, the message may identify a network location, from which the second device may retrieve computer-readable instructions. This means that the second device can handle the received message in a standardized way. By retrieving computer-readable instructions and by presenting a user interface judged by those instructions. This makes it even more certain that the network traffic is kept low, because some device receiving the message disabling or causing the computer-readable instructions to use it may ignore the message and thus the computer-readable instructions. I will not search for items.

Computer-readable instructions may make up the application, but the application runs on the second device. To the advantage, computer-readable instructions may provide an application that may run on a second device. To make this possible, the second device may be configured with an application environment, in which such applications may run. As an application, computer instructions may be contrasted with, for example, a web page, where instructions may be executed on a second device without a second device having to repeatedly retrieve information from the first device.

That method may constitute use by the second device. Computer-readable instructions send at least one control message to the first device. The method may further include receiving a control message spoken at the first device, causing the operating state of the first device to change based on the received control message.

Computer-readable instructions may not only cause a user interface to be displayed, but may additionally enable a second device to send control messages to the first device. Thus, the user may use the user interface to control the first device. The control message may be detailed by computer-readable instructions, ie the format and content of the messages may be limited by the instructions. Continuing the example above, the control message may include, for example, an ignore command that causes the mobile phone to ignore an incoming call (incoming call), for example, converting the call to voicemail.

The method may involve constructing, by the first device, a message based on the operating state of the first device. Alternatively or additionally, the method may involve constructing, by the first device, computer-readable instructions based on the operating state of the first device.

In some embodiments, the first device may send information to the second device, so that the user interface accurately represents the state of the first device. In some embodiments, a message identifying computer-readable instructions may be configured to carry such information. This may be done by having the message contain control parameters that are entered into computer instructions. Alternatively, the message may identify a different set of computer instructions based on the operational state. In other embodiments, the message may not change and computer-readable instructions may be properly structured before being sent to the second device. The second and third options allow the first device to communicate efficiently, e.g. an alarm, only as instructions required to indicate special alarms that need to be sent as part of the computer-readable instructions. to be. As such, in terms of the examples above, the computer-readable instructions sent in connection with an incoming call (incoming call) may differ from those for incoming messages.

The first device may be configured to act as a server on the network, and computer-readable instructions cause the second device to act as a server as a client. In spite of the first device initiating communication, the result between the first device and the second device may be one thing, i.e. by sending a message to the second device that constitutes information identifying the application. The first device provides the functionality of the server, and the second device provides the functionality of the client.

The method may constitute multicasting of messages by the first device. The message can be sent using SSDP. The message may be multicast in the network. Specifically, the message may be multicast to all devices in the network, which may be done without prior knowledge of which devices are on the network. For example, by sending the message to a special address. In the UPnP protocol, messages sent to IP address 239.255.255.250 port 1900 will be multicasted to all devices in the network. Multicasting a message reduces the signaling required by the first device prior to sending the message. In some embodiments, the message may be sent using SSDP, that is, the same protocol used to send discovery messages in UPnP. Thus, a new message format is proposed.

The method includes performing a discovery process on the network to identify what is being spoken by the first device to the second device, and sending a message to the second device based on the discovery process. In some embodiments, the first device may identify a particular second device in the network using the discovery process. It may be used to send messages to compatible devices.

The method is to send a discovery message by the first device, the discovery message being sent before what is said to be a message. The discovery message may identify the network location of information describing the first device. The information describing the first device may constitute an XML-formatted document. In some embodiments, the first device may send a discovery message for what is said to be a message. This discovery message may conform to the UPnP specification, but may identify an XML file containing information describing the first device. As such, it will be clear that what is said to be a message is separate and distinct from a UPnP discovery message. Thus, the discovery message allows the first device to be identified in the UPnP system, but what is referred to as the message may cause the first device to present the user interface.

Sending a message by the first device may be performed independently of any messages received by the first device from the second device. Receipt of the message may be performed independently from any message sent by the second device to the first device. The method includes detecting many changes in the operating state of the first device; It involves sending what is said to be a message to a second device that responds to such detection. The message is sent by the first device, without any internally contained information requested by the second device. As such, the message is sent to the second device. This can be contrasted with prior art systems, in which the second device has to initiate communication with the first device, some requested data from the first device. This provides the advantages that the first device can provide, such as an alarm. Each of these multiple messages involved may be sent with a given variation described. Thus, the message may be collated with a discovery message, which will only be sent to a given second device.

The method comprises receiving at a second device what is said to be a first message; Indicating a prompt for user input upon receipt of a first message at the second device, the prompt requesting permission to retrieve computer-readable instructions; Based on user input providing permission, the first, and then any subsequent message may be configured to retrieve and use computer-readable instructions for being said.

When a first message is received identifying computer-readable instructions, the second device may prompt for permission to use the computer instructions. This may be useful for security purposes. However, with the given permission, the second device may thus automatically cause the user computer instructions to be identified by any subsequent messages. Thus, the above method can be fast and can easily present a subsequent user interface, e.g. providing an alert.

The method may constitute storing user input instructions in memory. The second device may store user input instructions in memory. Thus, the second device may be able to track the permissions associated with many other first devices.

At least part of the network may make up the home network.

According to the second embodiment,

A method is provided for allowing a first device to present a user interface to a remote device, the first device and the second device connected to the local network, the remote device remote from the local network, the method comprising: two Establishing a connection between the first device and the remote device; Detecting changes in the operating state of the first device; In response to the detection, sending a message from a first device, the message identifying computer-readable instructions stored on the first device; Receiving a message on the second device; In response to receiving, retrieving computer-readable instructions from the first device; Sending computer-readable instructions to a remote device; And at least a portion of the user interface determined by the computer-readable instructions retrieved from a first device, by a distal device, the computer-readable instructions to present the user interface to the remote device. To use.

The method may further comprise operating a service that allows forward messages to be sent between the first device and the remote device at the second device.

Instead of using computer-readable instructions on the second device, the instructions may be directed to a remote device. The remote device may thus receive and use the instructions. This has the advantage, that there is no need to establish a connection to the remote device to enable the first device to communicate with the remote device. This has special advantages when there are multiple first devices, such as when the second device only needs to establish its connection. To facilitate provision of the user interface, the second device may serve forward messages between the first device and the remote device.

According to a third embodiment, a method of allowing a first device to present a user interface of a second device is provided. The first and second devices that enable communication over the network, the method constitutes: detecting a change in the operating state of the first device; In response to the detection, sending a message from a first device, the message identifying computer-readable instructions stored on the first device; Make the identified computer-readable instructions retrieved from the first device, and the computer-readable instructions on the first device are determined by the computer-readable instructions retrieved from the first device. At least a portion of the user interface being used by the second device to present the user interface to the second device.

According to the fourth embodiment, the first?? A method is provided to allow a device to present a user interface to a second device. The first and second devices can communicate over a network, the method comprising: receiving a message on the second device, the message identifying computer-readable instructions stored on the first device. To do; Responding to the receipt, retrieving computer-readable instructions from a first device; And at least a portion of the user interface determined by the second device as the computer-readable instructions retrieved from the first device, the computer-readable instructions to present the user interface to the second device. Using them.

According to a fifth embodiment, a system is provided that allows a first device to appear in a user interface of a second device. The system consists of: a first device; Second device; A network through which the first and second devices can communicate, where the first device is equipped: detection of changes in the operating state of the first device; Responding to the detection, sending a message identifying computer-readable instructions stored on a first device, a second device being equipped with: receiving the message; Responding to the receipt, retrieving computer-readable instructions from a first device; Using the computer-readable instructions to represent a user interface, at least a portion of the user interface determined by the computer-readable instructions retrieved from a first device.

According to the sixth embodiment, a first device capable of presenting a user interface in a second device is provided, with first and second devices capable of communicating over a network, and the first device being equipped with: Detection of changes in the operating state; Responding to the detection, sending a message identifying computer-readable instructions stored on the first device; Enabling the identified computer-readable instructions to be retrieved from a first device, in the first device, the computer-readable instructions retrieved from the first device It is available by the second device to present the user interface to the second device, which is at least a portion of the user interface determined by the items.

According to the seventh embodiment, a second device capable of representing a user interface connected to the first device is provided, where the first and second devices can communicate via a network, and the second device is equipped: the first Receiving a message from a device, the message comprising: a message identifying computer-readable instructions stored on the first device; Responding to the receipt, retrieving computer-readable instructions from a first device; Using computer-readable instructions representing the user interface on the second device, which is at least a portion of the user interface determined by computer-readable instructions retrieved from the first device.

According to an eighth embodiment, at least one computer program product is provided. This product constitutes a computer-readable storage medium that stores long-lasting computer-readable instructions, and describes how the first device can present a user interface to the second device. The computer-readable instructions enforced by the first and second computerized devices, which make the first devices perform, the first and second devices capable of communicating over a network, the method comprising: It consists of: detecting changes in the operating state of the first device; Responding to the detection, sending a message from a first device, the message identifying computer-readable instructions stored on the first device; Receiving the message on a second device; Responding to the receipt, retrieving computer-readable instructions from a first device; At least a portion of the user interface determined by the computer-readable instructions retrieved from the first device by the first device, using computer-readable instructions to present the user interface to the second device.

According to a ninth embodiment, there is provided a computer program product constituting a long-lasting computer-readable storage medium having stored computer-readable instructions. Implemented by the first and second devices that can communicate over the network, the first computerized device that allows the first device to perform a method that allows the first device to present the user interface to the second device. Computer-readable instructions, the method comprises: detecting a change in the operating state of the first device; Responding to the detection, sending a message from a first device, the message identifying computer-readable instructions stored on the first device; Causing the identified computer-readable instructions to be retrieved from a first device, wherein the computer-readable instructions are at least of a user interface determined by computer-readable instructions retrieved from the first device. It is available by the second device to present the user interface in one part, the second device.

According to a tenth embodiment, there is provided a computer program product constituting a long-lasting computer-readable storage medium having stored computer-readable instructions. The computer-readable instructions are the first and second devices that can communicate over the network, and the first device twice?? It is executable by a second computerized device that causes a second device to perform a method of allowing the device to present a user interface, the method comprising: receiving a message at the second device, the message being the first Identifying computer-readable instructions stored on the first device; In response to the receipt, retrieving computer-readable instructions from the first device; And using, by the second device, computer-readable instructions representing the user interface on the second device, which is at least a portion of the user interface determined by the computer-readable instructions retrieved from the first device.

Further features and advantages will become apparent from the following description of the embodiment, which will be described with reference to the accompanying drawings.

1 is a diagram of a system according to an embodiment of the present invention.
2 is a diagram illustrating a configuration of a first device in a system according to an embodiment of the present invention.
3 is a diagram illustrating a configuration of a second device in a system according to an embodiment of the present invention.
4 is a diagram illustrating a message flow according to an embodiment of the present invention.
5 is a diagram illustrating an additional message flow according to an embodiment of the present invention.
6 is a diagram illustrating a message flow including a remote device according to an embodiment of the present invention.

Embodiments of the present invention provide systems, devices, and methods for enabling a first device to control a second device. Embodiments of the present invention may be better understood and the following embodiments will be described in the context of a typical home network (local network) connected to the Internet.

As shown in FIG. 1, the communication system 1 comprises a local or home network 2. In the local network 2 there is a network hub 4. The devices 6, 8, 10 are connected to the hub 4 via connections 12, 14, 16, respectively. The hub is also connected to the Internet 20 via a connection 18. The remote device 24 is connected to the Internet 20, which can communicate with the local network 2 and the devices therein via the Internet 20.

The hub 4 may correspond to a typical home network hub, sometimes known as a home hub, home router, modem router, Wi-Fi hub or the like. The hub can constitute a member known as the following functional elements. Modems, Dynamic Host Configuration Protocol (DCHP) servers, Network Address Translation (NAT) servers, wireless or Wi-Fi access points, and Ethernet switches. These elements, and their purpose, are known in the art and will not be described in detail. Nevertheless, the fact that the hub 4 will provide a network address to each device 6, 8, 10 within the network 2 and will allow each device to communicate with each other and with the Internet 20 Will be understood. The hub 4 is itself compatible with the UPnP standard and thus can be included in the signaling described below. As such, the hub 4 can also be described as the device 4 on the network 2.

Devices (6, 8, 10) are all connected to the hub (4) via connections (12, 14, 16), the hub (4), each device, the Internet (20) and the remote device (24) and the Internet. You can communicate via Some of all these connections may be wireless. As an embodiment, the connection 14, 16 connected to some devices 8, 10 may be wireless (i.e., via 802.11-WiFi), and the connection 12 connected to some devices 6 is May be wired (eg via Ethernet). However, this is only an example and suitable connections can be used.

Devices 6, 8, 10 may be any suitable network connection device. Nevertheless, in order to give some context to the devices in the description below, it will be assumed that the first device 6 corresponds to a home automation system connected to a network providing for example heating and ventilation; The second device 8 corresponds to the TV to which the network is connected; And the third device 10 corresponds to a mobile phone wirelessly connected to the local network. It is assumed that each device operates according to the UPnP standard.

The remote device 24 may be a mobile device such as a smart phone or a tablet PC. The use of these elements will be described in more detail below.

A more detailed view of the first and second devices 6 and 8 (e.g., heating system and TV respectively) will now be described with reference to FIGS. 2 and 3.

In Fig. 2 a conceptual diagram of the components of the first device 6 is shown. The first device 6 comprises a processing system 26 and a memory 28. Stored in memory are computer readable instructions, which are represented as firmware 30 and application 32. The network interface 34 allows the first device 6 to communicate via a connection 12. Finally, block 36 represents additional hardware, which can be controlled by processing system 26. In the present context, this may correspond, for example, to valves and relays required to make the heating system operate.

In general, the first device 6 may contain a communication module, which can send messages and receive messages from other devices, and one or more modules can be used for subsidiary functions of the device, such as heating and lighting control. Provides.

These modules may not be physically separated, in which case the modules contain separate physical elements. However, additionally, the modules may be functionally separated, and both modules have the functionality provided by the processing system.

The processing system 26 retrieves and executes the computer readable instructions of the firmware 30 from the memory 28. Following the computer instructions, the processing system 26 can control the additional hardware 36 to have the desired effect (ie, heating the home). In doing so, processing system 26 can send data to or receive data from additional hardware 36. A detailed description of how this is done will become apparent to those skilled in the art and will depend on the characteristics of the additional hardware 36. Such a thing will not be explained in detail. Processing system 26 may also send and receive data via interface 34 as described below.

3 shows a conceptual diagram of the second device 8. Similar to the first device 6, the second device 8 comprises a processing system 38 and a memory 40. Memory 40 contains computer readable instructions represented by firmware 42. The network interface 44 allows the second device 8 to communicate via a connection 14. The second device 8, which in this example is a TV, constitutes a display 46 and includes a user input device 48, such as a remote control device.

The processing system 38 retrieves and executes the computer readable instructions of the firmware 42 from the memory 40. Depending on computer instructions, the processing system 38 can control the display 46 and the user input device 48 to provide a user interface on the second device 8. Processing system 38 can operate to send or receive data via interface 44.

In particular the operation of the first and second devices 6 and 8 will now be described with reference to FIGS. 4 to 6. For brevity, in the following the device will be described as performing certain steps by itself. Something like sending messages or processing data. It will be appreciated that this refers to the processing systems 26 and 38 of the first and second devices 6 and 8, respectively, which perform the required steps according to computer instructions.

Figure 4 shows a signaling diagram representing the hub 4 and devices 6, 8, 10. This diagram shows the steps associated with the first device 6 connected to the network 2, and also shows the steps for allowing the second device 8 to present the user interface for the first device 6 . Before connecting the first device 6 to the network, it can be assumed that the second and third devices 8, 10 have already been connected to the network 2. As such, as shown by the dashed arrows 52 and 54, the second devices 8 and 10 communicate with the hub 4, respectively, and are assigned a network address, for example, on the local network 2.

In step 56, the first device 6 is connected to the network 2 by communication with the hub 4. This step may be related to the hub 4 that assigns a network address, such as an IP address, to the first device 6, as in steps 52 and 54 above.

In subsequent steps 58 to 64, the first device 6 finds UPnP devices in the network 2 or uses the UPnP discovery process so that it can be found. As such, in step 58, the first device 6 multicasts the discovery message (step 58) to all devices on the network via the SSDP. This discovery message (step 58) may contain data for the first device 6. The discovery message may include the network location, as described in the UPnP standard. That is, the URL of an XML description file that can be used by the second and third devices 8 and 10 to retrieve additional information on the first device 6.

In response to the discovery message (step 58), the hub 4 and the second and third devices 8, 10 may send a response message indicated by arrows 60, 62, 64. These response messages can be unicast to the sender of the discovery message (i.e. sent to the first device) and used by the first device 6 to retrieve additional information from the devices 4, 8, 10. It may include a network location of a description file. Here, it can be understood that the hub 4 serves as a UPnP device.

The discovery process is known in the UPnP standard, and can be selectively performed in addition to the steps below. However, it can be understood that these steps are not essential.

In step 65, the first device 6 determines a change in the operating state of the first device 6. In terms of the example system described above, this could be a change in the heating or lighting system. For example, detection of an error, or a scheduled switch on/off. Other examples may be expected, and these particular examples should not be considered limiting.

In step 66, in response to the detected change, the first device 6 sends a start message. With this implementation, the start message is multicasted to all devices 4, 8, 10 in the network 2. The start message contains, as described above, information identifying the application 32 stored in the memory 28 of at least the first device 6. The information identifying the application 32 may be a URL that identifies the application and allows it to be downloaded from the first device 6.

Each of the devices 4, 8, 10 receiving the start message sent in step 66 may respond to it. However, in this example, only the second device 8 will be described as responding. As in step 68, the second device 8 determines whether to respond to the start message sent in step 66. This can be done by the second device 8 displaying a prompt on the display 46 and requesting the user to agree to the response of the second device 8. The user can either provide user input to agree to the response or deny permission. Alternatively, the second device 8 can be set to respond automatically without a user prompt. Either way, it can be assumed in step 68 that the second device 8 decides to respond to the start message (step 66).

Accordingly, the second device 8 uses the application identification information included in the received start message to search for the identified application from the first device 6.

This is seen as a request sent from the second device 8 to the first device 6 in step 70, the response being sent to be received by the second device 8 from the first device 6 in step 72.

The application 32 contains computer readable instructions used by the second device 8 to present a user interface on behalf of the first device 6. As part of the use of computer readable instructions, the second device 8 may display a message or screen on the display 46 and may provide an audible alarm. Depending on the commands of the application 32, the second device 8 is capable of user interaction with the user interface, and the second device 8 is able to interact with the first device 6 based on any user interaction of the user interface. Can communicate. That is, data can be transmitted to the first device 6 and data can be received from the first device 6. This communication is shown by a double arrow 76. The second device 8 represents the user interface of the first device 6.

It can be appreciated that there are many possibilities for the functionality of the user interface provided by the application 32. For example, the application may contain computer instructions that cause the second device 8 to display an alert without requiring the user to respond. Alternatively, more complex user interfaces can be presented to allow the user to provide input. The application 32 may cause the second device 8 to send a control message to the first device 6. Since the application can be specific to the first device 6, the user interface can be tailored to the requirements of the first device 6.

To relate the above to the system described above, the home automation system in one example assumes an error. In this case, the application can cause the second device 8 to display a warning identifying this error. In this example, the application may be relatively simple, but it doesn't have a device for letting the user react to the alert (other than hiding it once it has been read). In a second example, the home automation system can alert the user to a scheduled stop of the heating system, and provide a device for the user to delay this stop. In this second example, the second device 8 may provide a control message to the first device 6 so that the first device 6 delays stopping the heating.

An additional method will be described with reference to FIG. 5. This additional method may be additional or alternative. In this way, the steps take place after the devices 6, 8 and 10 are connected to the network and the requested discovery process has been performed. In the description below, the second device 8 stores data in the memory 40. The configurations of the second device 8 can be shown individually in order to give this a visualization. That is, processing system 38 and interface 44 are being viewed separately from memory 40.

In step 77, the first device 6 determines a change in the operating state of the first device 6, and in step 78, the first device 6 sends a start message to the second device 8. This message may be unicast in contrast to FIG. 4, and the start message in FIG. 4 is multicast. In order to unicast the message to the appropriate devices, i.e. the second device 8, not the other devices 4, 10, the first device 6 will use the information about the devices that are discovered as part of the discovery process. I can. The start message (step 78) contains information identifying the application 32.

In this example, the start message (step 78) is the first start message received by the second device 8 from the first device 6. Thus, the second device 8 determines in step 80 whether to respond to the start message 78. This can be done by requesting user permission, as above, assuming permission is given.

In step 82, the second device stores an indication of this permission in memory 40. In addition, as shown in steps 84 and 86, the second device 8 requests and retrieves the application 32 from the first device 6.

As shown in step 88, similar to step 74 above, the application is used by the second device 8 to provide the user interface. During step 100, the second device 8 and the first device 6 may perform communication as shown by arrow 94. The execution of the application will eventually end. Continuing with the above example, the change in the operating state may be a scheduled outage in the heating system. And, through the user interface shown in step 88, the user can postpone this interruption. If you do this, you exit from the user interface and the application can be terminated.

Thus, in step 91, an additional change in the operating state for the first device may be determined. This may be because, in the above example, a delayed downtime has reached, or because the home automation system again warns the user to make a scheduled interruption. Thus, based on this determined change, the first device 6 can send a second start message to the second device 8, as shown in step 92.

Like the start message described above, the second start message sent in step 96 may include information identifying an application. When a message is received in step 94, the second device determines whether user permission has been given for the applications provided by the first device 6. This can be done by retrieving the stored data from memory 40.

The stored user criteria can be used to determine if the application can be executed without user prompting. As such, in steps 96 and 98, the second device retrieves the application 32 from the first device 6. In step 100 the application is again used by the second device 8 to present the user interface. During this time, additional signaling may be performed as in step 102.

Many of the benefits have been described above. First, by providing an identifier of an application executable at the recipient of the second device 8, the first device 6 can provide increased functionality through the application. Moreover, the first device 6 may allow the application to be started via a start message. A third advantage is that the first device 6 can send a warning or other similar notification without requiring prior configuration of the second device 8, as shown in step 96. A fourth advantage is to provide a user interface to the second device 8 for a device such as the first device 6, where only one user interaction is required to give the first permission. Once this is done, the first device 6 can provide a user interface to the second device 8 based on the events of the first device 6.

Additional embodiment

Related to signaling will be described with reference to FIG. 6. 6 shows a situation in which the user interface is displayed on the remote device 24. This allows users to access remotely and control devices in their homes. The remote device 24 may be a smartphone or a tablet PC.

In the first step 116, the remote device 24 is connected with the second device 8. There are many ways to make this happen. For example, the remote device 24 may be provided with the network address of the second device 8. Or, in particular where dynamic addressing is used, the second device 8 and the remote device 24 can communicate with a fixed server, such as a web server, to advertise their network addresses. As such, in order to communicate, each device can get information from the server to determine the network addresses of others. Other methods are known in the prior art and will be apparent to a person skilled in the art. Since the remote device 24 is connected to the second device 8, the second device 8 knows that the remote device will receive start requests.

In step 118, the first device 6 sends a start message identifying the application 32. The second device 8 can respond to these messages as described above. Alternatively, alternatively or additionally, the second device 8 may enable the remote device 24 to respond. As such, the second device 8 retrieves the application 32 from the first device 6. The second device requests the application in step 120 and the application is sent and received in step 122.

In step 124, the second device 8 sends a message to the remote device 124. The message may include information identifying the application. In step 126, with the received message (step 124), the remote device determines whether to take an action upon receipt of the message. As mentioned above, this is also possible through user input.

It is determined whether to respond to message 124, and the remote device retrieves the application 32 from the second device 8. The application 32 is requested in step 128 and received from the second device 8 in step 130. In step 132, the second device 8 also provides a service such as a proxy application (different from application 32). This allows the second device 8 to send or receive messages to the first device 6 and to the remote device 24.

In step 134, the remote device 24 uses the application 32 to provide a user interface. As part of this, the remote device 24 can send and receive data to the second device 8. The second device 8 running the proxy application can forward the data received to the first device 6, and can also forward the data received from the first device 6 to the remote device 24. have. The resulting communication between the first device 6 and the remote device 24 via the second device 8 is shown in steps 136 and 138.

This has an additional advantage, the first device 6 does not require special knowledge of the relationship between the remote device 24 and the second device 8. In addition, this means that it is not necessary to be able to communicate with devices outside the local network 2 without the first device 6 being configured for this. Instead, the configuration that makes it possible to communicate with the remote device is handled by the second device 8. This is also useful where there is more than one first device, like the first device 6, where everyone can communicate with the remote device 24 even though only the second device 8 is set up for such a connection. Because.

In the examples provided above, the start message is sent based on recognizing a change in the operating state of the first device. It can be appreciated that the change in operating state may be related to the change in the first device and will depend on the functionality provided by the first device. As mentioned above, the first device may be considered to have a communication module and at least one additional module, and the state change may be a state change associated with one or at least one additional module. It will also be understood that the state change is independent of the messages that may be received from the second device. That is, the start message is put to the second device and is not transmitted in response to the request by the second device.

The first device can pass specific data to the second device so that the user interface behaves the way it wants. For example, displaying accurate warnings, requesting user input as needed, and so on. To this end, the computer-readable instructions of the application 32 are kept intact, and when started, appropriate information is retrieved from the first device for the application. However, as an alternative, the first device may set a start message including parameters used by the application at startup. The second device can consequently provide parameters when the application is used. This has the advantage that a single application is written and simplifies the provision of applications, allows at least a portion of the application to be stored in memory, and reduces the amount of data transferred.

In some embodiments, different start messages that identify different applications may be used. For example, there may be different applications for different alerts, and the sent start message may identify the application that will present the desired alert. These complex applications can also be stored. In some embodiments, the first device may configure the application prior to transmission to the second device, in other words, the identifier provided in the start message is the same in all circumstances, but when the application is retrieved by the second device, the first device May deliver an application specifically tailored to the current operating state of the first device. This has the advantage of keeping the application transferred from the first device to the second device, which is relatively small, and only the portion required for the current state of the first device is requested.

Although the above has been described in the context of a home automation system that presents a user interface to a TV, many other scenarios are possible. For example, a user interface may alternatively or additionally be displayed on a mobile phone. In addition, the cell phone may detect state changes within it, for example an incoming call (an incoming call), an SMS text message receipt, an alert, or a calendar reminder. Upon receipt, the mobile phone may act as the first device described above, or may transmit a start message so that a warning can be displayed on the TV.

The second device 8 and another device capable of executing the application may provide an appropriate application engine to execute the application. Such an application engine can be standardized so that applications from a given device can run on many devices. As an example, there are currently many application engines deployed for use in TV, for example HbbTV, MHEG, and MHP. These application engines allow broadcasters to provide interactive TV applications. Although, these application engines serve as examples of suitable application engines that may be used in the embodiments.

The above description refers to the application 32, but any form of computer-readable instructions may be provided by the first device and may be used on the second device to provide a user interface. For example, the application may be presented as a web page containing markup data, and may also contain scripting instructions such as JavaScript. However, as an alternative, the computer-readable instructions could simply be a text for the displayed warning, as well as a parameter such as an expiration time for the warning, a file defining what the user response is acceptable.

At least one part of the user interface can be determined by the application environment of the second device, e.g., some characters to be displayed are defined by computer-readable instructions provided by the first device, while the font The appearance of the character, such as size and style, can be defined by a second device.

The above-described start message may be unicast or multicast. In embodiments, the start message may be sent as SSDP messages that are multicast to all connected devices according to the UPnP standard. Alternatively, in embodiments, the start message may be unicast to specific devices. In the second case, the discovery process related to UPnP can be used to determine which device is suitable or can receive a start message.

Any suitable device may send a startup message, but it's not just a home heating device. Other examples include appliances having use controlled by a home device or a connected device. Some examples of suitable devices are:

-A home security system capable of additionally providing a security scene via an application;

-Home automation systems that control heating, ventilation, lighting and the like. In these cases, the application not only reports the state of the system, but can also allow configuration changes of the system to be made. (For example, changing time or temperature)

-Home appliances such as cookers, microwave ovens, kettles, coffee makers or the like that can report status (e.g. job completion) via the application

-Mobile or home phone that provides alerts for incoming calls, messages, calendar appointments, reminders or the like via the application;

-Doorbells or intercoms that the user may be alerted to the visitor; And

-Monitoring device for babies

The start message may be briefly described as identifying the application, but the start message may include other data as follows.

A string identifying the request as a start request. Although all UPnP-compliant devices on the network may receive such requests, incompatible SSDP clients may discard messages based on their identifiers.

-IP address of the remote device

-URL identifying the location of the application to start

-A device with a ‘friendly name’ that can be used to make the device identify the user

-‘Friendly name’ application

-Save options and start with options, e.g. whether the application should be saved locally on the second device, and

-One or more control parameters that may be provided to the application when implemented. These can be used to provide a warning to the user without further messages.

In the above examples, the hub device 4 and the third device 10 did not respond to the start messages. The reasons for this may be many, for example, the hub device 4 may not have a user interface system, and thus the application will not be able to be used. The third device 10 (eg cell phone) may be able to handle the start message, but has the user set to reject such requests.

The second device 8, upon receipt of the start message, may request a user input to allow the user interface to appear. This can, for example, appear on one screen of the second device 8. By giving permission, the user can confirm that a user input is required before the application is executed again, and alternatively, the user can confirm that the application can be executed after receiving the start message. Thus, the second device 8 can start the application after receiving the start message, allowing the first device to provide an alert to the second device.

If there are a plurality of first devices such as the first device 6, they may appear in the network 2. Each of these first devices may have a different set of user-specified parameters stored in the second device 8. As such, one application related to one device can be started as soon as a start message is provided, while another application may require permission at each time before the application is started. Additional applications can always be blocked. One user interface can be provided to the second device 8, allowing the user to change settings for each known application. In some embodiments, the second device is configured such that all start requests received on the local network become an automatic indication of the user interface (i.e., without user permission) regardless of whether the start message is known or received from a new first device. It can bring results.

The second device 8 may provide functionality to allow the user to start the application 32 without a start message received from the first device. As such, a user interface may be provided on the second device 8 that allows the user to select and start an application related to the desired device. Referring to the examples already provided, this could allow the user to set the temperature via the TV.

The first device 6 is mentioned alone, but can be implemented in a series of distributed elements. Equally, the first device 6 may be a control system for a larger system (such as a heating system) and should not be considered as including all elements of such a system.

The network can be divided into many areas and zones. For example, in the context of a large house, office, or hotel, each room or floor could be served to a different zone, and messages could only be sent to devices within the same area. This prevents the first device from giving an unintended warning to the second device.

The hub 4 is referenced solely for brevity, but many functions of the hub can be distributed among many entities.

In some embodiments, the system may be used in a hotel, for example. Such a network may be provided to specific regions, and devices may be set up to communicate with specific devices or specific regions.

1: communication system 2: home network
4: network hub 6, 8, 10: device
24: remote device
26, 38: processing system 28, 40: memory
34, 44: interface 36: additional hardware
46: display 48: user input device

Claims (17)

In the display method of an electronic device displaying a user interface of an external device,
Receiving a first message;
Upon receiving the first message, displaying a prompt for receiving a user consent to retrieve a command;
Receiving a user consent input;
After the consent, receiving a message identifying a command stored in the external device from the external device according to the scheduled switch-off of the external device;
Searching for the command of the external device based on the received message;
Displaying at least a portion of a user interface of an external device corresponding to the searched command; And
When a user command for delaying the scheduled switch-off of the external device is input, transmitting a control signal corresponding to the user command to the external device, and releasing the display of the user interface; Way. The method of claim 1,
Wherein the message can identify a network location where the command can be retrieved by the electronic device. The method according to claim 1 or 2,
Wherein the command includes an application that can be executed on the electronic device. The method according to claim 1 or 2,
The command is used to transmit at least one control message to the external device. The method of claim 4,
The control message is a display method of an electronic device, characterized in that to change the operation state of the external device. delete delete delete delete delete delete delete delete delete delete delete A display method of a system for displaying a user interface of a first device in a second device, the method comprising:
Transmitting, by the first device, a first message;
Receiving, by the second device, the first message, displaying a prompt through which a user consent can be input to retrieve a command;
Receiving, by the second device, a user consent input for consent from a user;
Transmitting, by the first device, a message identifying a stored command by detecting a scheduled switch-off;
After the consent, the second device receiving the message and retrieving the command from the first device;
Displaying, by the second device, at least a portion of a user interface of the first device corresponding to the command retrieved from the first device; And
When a user command for delaying the scheduled switch-off of the first device is input, transmitting a control signal corresponding to the user command to the first device, and releasing the display of the user interface; Display method.

Images