KR20010080609A - Handheld retrieves UI from server for control of appartus via server - Google Patents

Abstract

Handheld remote 102 has a wireless modem 108 to send an identifier to server 110 over the Internet. The server has a look-up table 112 to link the identifier to the URL. The URL points to the CGI program 118 on another machine on the Internet.

Description

Handheld retrieves UI from server for control of appartus via server}

Remote control units are well known. One example is Fronto (TM) manufactured by Royal Philips Electronics. Fronto ™ is a so-called universal programmable remote control unit. It makes it possible to actually store the IR codes of any existing CE device. Fronto ™ has an LCD with touch screen functionality as a user interface (UI). The control functions of Fronto ™ can be fully customized by him / her as a user. Fronto (TM) communicates with CE equipment that uses near infrared radiation.

The invention relates in particular to remote control of one or more devices in a consumer electonics (CE) environment.

The invention is explained in more detail by way of examples and with reference to the accompanying drawings. here:

1 is a block diagram of a system in the present invention;

2 is an exemplary diagram of a configuration page.

The present invention Extend the concept. It is an object of the present invention to provide a user with complete remote control, regardless of the distance between the facility to be controlled and the user.

For this purpose, the present invention provides a method that enables an end-user to control the apparatus via a control device. The control device is, for example, a palmtop computer with a graphical user-interface (GUI), and the device is, for example, one CE equipment. The server may be able to receive an identifier sent by the controlling device via a data network such as the Internet. The server has a look-up table or database to determine the URL associated with this identifier. At that time, the server can connect with the software program as specified by the URL to control the device. A software program is, for example, a CGI application running on the server itself or another machine. Preferably, the user can specify or program an association between the identifier and the URL. For example, a user can program an association via a browser.

One aspect of the present invention is to enable a server to download a component for a UI to a control device to enable the control device to transmit an identifier on a user-activation component. It's about things.

Another aspect is that each of the multiple servers is each of multiple URLs representing each of the multiple software programs. It is possible to receive an identifier for determining. In this way, one identifier sent to the server may cause multiple control events to occur at one or more controlled devices, such as in a macro or script.

The present invention relates to an information processing system including a device connected to a server. The server uploads the UI to the device. The UI makes it possible for a user to cause a program to run on a server or another machine. The server or other machine is connected to the subsystem. The sub-system can be controlled via a program (or script). Preferably, the user-interface can be customized or user-programmable, for example, via a PC that connects to the server or becomes a functional part of the server. Preferably, the user will be able to specify which control-related events in the sub-system will cause some interaction with the UI.

In the practice of the present invention, the control device is a handheld device that connects to the Internet via a wireless modem. The device is retrieved from a server that includes labeled widgets. The UI should be as user programmable or customizable as possible. Activating the widget on the handheld device causes the label to be sent to the server. The label is associated at the server with a URL that allows the server to select resources on the Internet, in this case a CGI application on a more internet-enabled machine or on the server itself. The machine or its server runs a CGI application and communicates with a controlled device using a proprietary protocol.

The invention allows for example the transmission of a UI for a home control system to be carried to portable handheld internet connected devices. These devices provide a means for users to interact with the home network, regardless of where the user or home network is located. UI devices then In terms of resources, they need to interact with the home network. UI devices do not require any specific hardware to directly control the home network. Instead, they rely on servers (UI devices communicate with the server exclusively) to provide a means for controlling the home network.

Thus, the present invention provides several advantages, among which are as follows: The UI is stored in a server and can be used independently by various control devices of the same user, the UI being connected to the user rather than the control device; Storing operations as URL = s at the server allows the server to invoke programs on other machines; The present invention uses a powerful internet infrastructure; The control device can be a very thin client.

1 is a diagram of a system 100 in the present invention. In system 100, handheld device 102 uses wireless modem 108 to communicate with server 110, via wireless IP connection 106 to data network 104, such as, for example, the Internet. Is connected. Device 102 is a personal digital assistant (PDA) connected to the Internet via wireless modem 108, such as, for example, PlamIIIx manufactured by 3COM or Windows CE-based handheld. It includes. Modem 108 is, for example, Minstrel or Ricochet. MInstrel, traded by Novatel Wireless, is a two-way wireless modem for PDAs that allows users, among other things, to browse the web and receive mail. In a more general sense, a wireless modem, such as Minstrel or Ricochet, provides the handheld device with an IP address that can be used by any kind of application that uses the Internet for communication. Minstrel uses a technology called Cellular Digital Packet Data (CDPD) that is supported by cellular service providers.

On a valid connection, server 110 uploads code for a user interface (UI) to device 102. UI is a number of UI It includes. Terms Is used in various ways in the computer field. The widget provides information for the user to interact with the operating system and applications. Widgets display icons, pull-down menus, buttons, selection boxes, progress indicators, on-off checkmarks, scrollbars, windows, and window for resizing the window to display user interaction functions. Edges, toggle buttons, and so forth. In programming, widgets also mean small programs written to describe how a particular widget looks, how it behaves, and how it interacts in response to user actions. Most operating systems include a set of ready-to-tailor widgets that a programmer can incorporate into an application and specify how it behaves. New widgets can be created. Most application development languages, such as Java and Tcl, come with prebuilt libraries that programmers can embed and modify today. Using Microsoft's Visual Basic, the widget can be executed as part of or as an ActiveX control (see for example www.whatis.com). The UI does not need to know the capabilities of the particular device to be controlled, only the widgets to be provided. Widgets determine how the UI is organized.

Each widget has a label that communicates to the user of device 102 an action that will occur if the user interacts with the widget on device 102. Alphanumeric examples of graphical labels are text Another example is a graphical representation of a lit lamp. When the user interacts with the widget, the identifier for the widget is passed back to the server 110. For example, a UI comprising four momentary buttons on the touch screen of device 102 passes back through AButton1 ", AButton2", AButton3 "or AButton4" depending on which button was activated by the user.

When the server 110 receives the identifier, the server 110 examines what will happen after using the database or lookup table 112 to associate the identifier with the particular operation that was performed.

In order to arrange the labels for the widgets of each UI, ie to set up the desired action and the relationship between the labels, the user interacts via a configuration panel. The configuration panel becomes useful to the user, for example via a web page, as discussed in more detail below. The configuration panel may be accessible via a handheld-device or another facility 114, such as a PC or set top box, which makes the browser 116 useful. Web pages are created as dynamically as possible, and uniquely created for each individual user. 2 shows an example of such a configuration page 200 and is discussed below. In addition to the setting of the label via the web site, the user sets the action that should be taken when the user interacts with the labeled widget in the UI of the device 102 as a result of the associated identifier being sent to the server 110. Or specify it.

The action is specified in the form of a Unified Resource Locator (URL). URLs are the means by which resources on the Internet are selected. Resources are, for example, HTML documents, WAV (sound) files, image files, and the like. In the case of a URL provided for the widget = s operation, the URL as specified in the database 112 points to the Common Gateway Interface (CGI) application 118 as much as possible. CGI application 118 is a program that is started by and executed on a server, like any other program that may be running on a computer. In this case, the CGI program 118 is executed on the operation server 120. The operational server 120 is depicted here as being among the plurality of servers. Each of these servers has its own set of CGI = s and is used to control the added facility in a manner similar to that described here.

The information comes from the CGI requester (traditional user using a web browser, traditionally Netscape or Internet Explorer). Can be passed via a mechanism such as

The CGI application 118 executes whatever action the user requires. In the example case, the CGI 118 application communicates with a hardware actuator 122 connected with the server 120, which may turn on the back. The connection between the actuator 122 and the server 120 can be a cable connection or a wireless connection, or is performed using the network itself. The hardware actuator 122 may be a relay or a triac that is directly connected to the output pins of the processor (ie I / O port) of the server 120, or if it is an RS232 serial port. May be an X-10 module (such as PL513 available from http://www.x10.com) connected to the server 120. In this example, the actuator is an X-10 compliant device. Other actuators 124, 126, ..., 128 are for example IR protocol, HAVi protocol, or Use The latter example shows that the system in the present invention can be easily upgraded. Actuators 122-128 directly control the associated devices (although not visible) as indicated by outgoing arrows.

It is of no value that the provided URL can point to a CGI program on the machine anywhere on the Internet 104 (assuming that the path is not blocked by a firewall or the like). Alternatively, the URL can Look-up server 108, incorporated into the IP specification and referenced in FIG. 1 by reference symbol 124, via an address (see address A127.0.0.1 "in FIG. 2). It can direct itself.

When the UI is reconfigured, or if the handheld device 102 requests it, a description of the UI is sent from the server 110 to the handheld device 102. In this example, the UI is sent to the handheld device 102 by, for example, sending an XML document describing the widgets and their associated labels. Since the handheld device 102 relies on the server 110 to understand the pressed keys or other user input entered on the handheld 102 and invoke the appropriate action URL, the action taken by the server 120. Are never sent to the UI.

Storing the operations represented by the URL allows the server 110 to easily call a program in other machines, such as the server 120. Home control, automation, and monitoring applications seem to use multiple distributed computers to provide other components of services. By using the existing strong HTTP interface to invoke applications on various computers, the existing infrastructure of the Internet is leveraged by means including extensions for security / privacy, ie encryption, authentication and so on. Since the UI is stored on a server 110 on the Internet, the UI allows it to be associated with the user rather than with the device 102 that the user accesses the UI through. This means that if a user has multiple devices 102 that he / she can control his / her environment, he / she only needs to update the UI once and the change is all interaction It will affect devices 102. By using the Internet to store the UI, modifications can be made through a more convenient interface than would be useful on the handheld devices themselves. For example, the UI of a handheld device is typically a small, executed via numeric keypad (such as on a phone). Keyboard or Limited to interfaces For improvement, see US serial number 09 / 062,364 (Attorney docket PHA 23,287), filed Apr. 17, 1998 by Sung Choi and Jan van Ee for GRAPHICAL USER INTERFACE TOUCH SCREEN WITH AUTO ZOOM FEATURE. Virtual keyboards and hunt-and-peck access typically have more errors that are slower and easier to use than traditional keyboards with a full-fledged PC 114. Having a UI that is accessible and stored through the network 104 in the manner described, it can be transformed through any client having a suitable application that can interact with the UI. Web browser 116 is sufficient. The above extension can be made for new clients to use these new modes when a newer mode of interaction is developed, for example graphical commands or voice commands. In lieu of the more conventional method mentioned above, new interaction methods can be used to modify the UI. The handheld device 102 does not need to be changed in any way to allow this. Since the handheld device 102 interacts with the server 110, it allows controlling the devices such that the UI does not have a means of directly connecting to itself. For example, handheld device 102 will typically not be able to control an X-10 device without being connected to a home power network. However, using the system of the present invention, the handheld device 102 can communicate to the server 110, which in turn is a CGI application 118, on a machine 120 having such a connection to the home power network. Will communicate (via HTTP).

The present invention allows for example a UI for a home control system to be transmitted to portable handheld internet access devices. These devices provide a means for the user to interact with the home network, regardless of where they are located. The devices are then a term that indicates their most recent needs regarding the resources they need to interact with the home network. to be. They do not require specific hardware to directly control the home network, but instead rely on servers (they communicate exclusively) to provide a means to control the network.

2 shows an example of a configuration panel 200 present within the browser 116. The panel 200 has a header. Column 202 with headers Column 204 with header and Has a column 206 with. Terms Refers to a button widget on the UI as given to the user at device 102. In column 202, the buttons are Enumerated as: Each of these buttons is labeled as shown in column 204. The labels specify what happens when the user activates the button. Column 206 lists the URLs that address the associated CGI = s. Panel 200 allows the user to edit the UI and reprogram each individual one of the buttons.

The aggregated UI may be provided by the server 110 with widgets (eg, buttons, slides, etc.) corresponding to CGI = s at other operational servers. For example, at home, one operation server, for example server 120, controls the sprinkler system, another server connects to the set-top box (STB), and a third server controls the X-10 network. It will act as an internet gateway. The GUI on the device 102 is, for example, an on / off switch for spin buttons and sprinkler systems, an on / off switch, channel selector, volume control, mute for STB, and home code for an X-10 server house-code) selector and on / off switch. Operational servers do not need to be in close proximity. The UI is equally well suited to webcam info from Internet servers around the world with controls including “switch to another camera” and “zoom / pan / tilt”. Gather.

In addition to aggregate control, multiple look-up servers negotiate with each other as active code (e.g. Java applets) that provide new GUI elements or from a selection of common widgets to send parts of the UI. can do. Remote device 102 may control negotiation, or survey servers may be based on relative placement, sizing according to size, and static preference information retrieved from a remote device or previously stored by a user. A protocol may be followed to determine the priorities of the widgets. A common example in another technical field is aggregate meta text-search, display 5 originating from AltaVista, 2 from Lycos, 3 from HotBot do. A better example is the time-varying percentage of the display dedicated to the controls of the home server 120, stock-market graphs, traffic information, weather information, slashdot and CNET headlines, and other operational servers. And negotiated percentages dedicated to email from one or more accounts. Depending on the agreement, only three of the servers may provide UI elements once per day, while seven or more may later provide UI elements.

What has been described with respect to GUI = s, with the necessary changes, also applies to aspects of user-interaction that differ from pure graphical, such as voice control. US serial number, filed on December 16, 1999 by Willem Bulthuis et al, for example HAND-EAR USER INTERFACE FOR HAND-HELD DEVICE in this context. See 09 / 464,855 (attorney docket PHA 23,875). This document relates to a hand-held information processing device, such as a mobile phone or a remote control, with a thumb wheel that allows a user to scan a circular array of selections. Each of the choices is represented by a respective audio output that is played out when the wheel is raised or lowered. This allows the user to make choices with one hand and without having to look at the device. It also considers a form factor smaller than that of conventional mobile phones, since the keypad does not need to enter numbers to call from a personalized directory. This device is also It can be used as, where a menu of control choices can be scanned by the thumb wheel and audio output. Upon finding the desired selection as indicated by the audio output, the user authorizes the current selection, and the device sends an associated identifier to the server for further processing as described above on that selection.

Above, remote control via the Internet has been described with reference to some conceptually simple examples. It is apparent that other and more complex scenarios can be implemented using the present invention. For example, when the user realizes that he or she has forgotten or cannot set the device to record his / her favorite TV program, the VCR or TiVo (ie, hard disk drive (HDD)) And a recording device (such as a personal video recorder) capable of simultaneous recording and play-out of a TV broadcast, can be programmed from anywhere in the world. Within this context, U.S. Patent Application, filed on April 1, 1999 by Eugene Shteyn, for TIME-AND LOCATION-DRIVEN PERSONALIZED TV, is also available. See serial number 09 / 283,545 (attorney docket PHA 23,633). As another example, a user may turn on central heating, or turn on lights in the home and garden while driving home late at night. As another example, a user may program or activate a sprinkler device in the garden of a home while working or traveling.

In addition, what is described with respect to CGI scripts may be executed using other protocols in the preferred data networks to invoke a script or application program on the operational server.

In addition, the following patent documents are referenced:

U.S. Serial No. 09 / 427,821 filed on Oct. 27, 1999 by Joost Kemink and Richard Sagar for PDA HAS WIRELESS MODEM FOR REMOTE CONTROL VIA THE INTERNET (attorney docket PHA 23,786). This document relates to an information processing system having a handheld computing device such as, for example, a personal digital assistant (PDA) with a user interface, and a wireless modem connected with the handheld. The wireless modem makes it possible to communicate with a server via a data network such as the Internet. The control network is connected between the server and the controllable equipment. The handheld can now function as a wireless remote control device for the equipment via the Internet and servers. The system may include a video camera with hardware and software to produce a suitable formatted still image for display on the handheld device. The user can instruct the retrieval of still images from the server via the Internet. This application acts as a security system, for example, allowing a remote user to monitor his / her front door porch or to monitor children as a remote (or backed off) babysitter. The user-accessibility of the equipment is ensured by the ubiquity of the Internet, thus allowing to expand the scope of control and supervision capabilities for mobile users.

-U.S. filed December 11, 1998 by Joost Kemink and Eugene Shteyn for REMOTE CONTROL DEVICE WITH LOCATION DEPENDENT INTERFACE. Serial number 09 / 210,416 (attorney docket PHA 23,522). This document relates to a user control interface, which is a dependent location within the home environment. The context control parameters are associated with the location and the user control interface will be adapted to the context in which the device is operating. The control interface includes a presentation of a context sensitive to information and communication of that context sensitive to user commands via the interface. Positioning may employ any number of commonly useful techniques, such as direct entry for relative positioning, infrared detectors, and active badges, as well as customary absolute positioning devices such as LORAN and GPS. Use, it is effected. In a preferred embodiment, the device communicates with a remote information source that provides a context sensitive to control information. The remote information source may be a home network server, an internet server, a public service network, or other communication network.

U.S. Serial No. 09 / 434,155, filed January 1, 1999, by Martin Freeman and Bonghan cho for REMOTE INITIATES RETRIEVAL OF CONTROL CONFIGURATION (Attorney docket PHA 23,783). This document relates to a programmable remote control unit that can initiate retrieval of a control configuration to a unit from outside a storage device. The unit comprises a retrieved control configuration; A display for displaying icons representing the configuration; And a memory to store a touch screen for initiating selection based on the displayed icons. The storage device is a component of the CE device. By storing or backing-up the control configuration for a particular device on the device itself, the remote control unit is made truly universally programmable.

CUSTOMIZED UPGRADIN OF INTERNET-ENABLED DEVICESBASE ON USER-PROFILE, by Adrian Turner, Simon Pearce, David Eves, and Allan Timms. Attorney docket PHA, filed US serial number 09 / 160,490 (attorney docket PHA 23,500), and US serial number 09 / 189,535 filed on November 10, 1998 by Eugene Shteyn for UPGRADING OF SYNERGETIC ASPECTS OF HOME NETWOKES. 23,527), both of which relate to a server system capable of sustaining a user profile of a particular end-user of a home appliance network-capable device and a database of new technical features for this type of device. If there is a match between the profile and the new technical features, and the user indicates that he / she is willing to receive information about updates or sales offers, the user is informed via the network of choice to obtain the feature. . Synergy is detected between the components of the user's equipment to inform him / her of further possibilities of using or extending his / her equipment.

-US serial number 09 / 165,682 filed on 1 January 1998 by Eugene Shteyn for CONTROL PROPERITY IS MAPPED ONTO MODALLY COMLATIBLE GUI ELEMENT (Attorney docket PHA 23,484). This document relates to an information processing system having an electronic device and a controller for controlling the function of the device. An abstract representation of the function is provided by the controller. The controller makes it possible to control the function through abstract representations and interactions. The aspect controls the control of the function in conjunction with the formally compatible control capability of the controller. The revealed aspect is, for example, Can be.

-US Serial No. 09 / 519,546 filed on June 3, 2000, by Erik Ekkel et al. For PERSONALIZING CE EQUIPMENT CONFIGURATION AT SERVER VIA WEB-ENABLED DEVICE (attorney docket PH-US 000014) ). This document is about facilitating the configuration of CE facilities by consumers by delegating the configuration to application servers on the Internet. The consumer enters his / her preferences into a particular interactive web page through the appropriate user interface of an internet enabled device such as a PC or set top box or digital cell phone. The application server generates control data based on the entered symbols and downloads the control data to the CE equipment itself or to an internet enabled device.

The present invention relates in particular to remote control of one or more devices in a consumer electonics (CE) environment.

Claims (9)

A method of enabling control of apparatuses 122-128 via control device 102, Enabling the server 110 to receive an identifier sent by the control device; Enabling the server to determine a URL associated with the identifier; And -Enabling the server to contact a software program (118) as specified by a URL to control the device. 2. The apparatus of claim 1, wherein the server makes it possible to download a component for a UI to the control device to enable the control device to transmit the identifier in a user-activated state of the component. Controllable method comprising the step of. 2. The controllable method of claim 1 including enabling a user to specify an association between the identifier and the URL. 4. The controllable method of claim 3, wherein the user makes it possible to program the association via a browser (116). The method of claim 1, wherein each of the multiple servers makes it possible to receive the identifier for determining each of the multiple URLs representing each of the multiple software programs. 7. The method of claim 6, wherein each of the multiple software programs controls each of multiple devices (122-128). Look-up table 112 for storage at server 110 and for associating an identifier received from user 102 with a URL specifying CGI program 118. 8. A lookup table according to claim 7, wherein the user is programmable. 8. The lookup table of claim 7, wherein the lookup table is customized for an individual user.