KR20070020488A - Spatial interaction system - Google Patents

Abstract

The present invention relates to a method, a device and a computer program product for remote control of a plurality of consumer electronic devices. The remote control device includes a user interface and means for determining a direction of the remote control device relative to the at least one household appliance device. The remote control device further includes means for adjusting the user interface based on the orientation of the remote control device.

Remote control, remote control, home appliance, computer, interface

Description

Spatial Interaction System {SPATIAL INTERACTION SYSTEM}

The present invention relates to a method, a device and a computer program product for remote control of a plurality of consumer electronic devices.

As more household appliance devices are used in the home environment, a growing number of remote controls follow this household appliance device. Although there are programmable multifunction remote controls, most of these devices are quite inconvenient to program and use. Moreover, these devices often require a form of interaction that is memoryized. This is because programmed interaction keys do not logically correspond to assigned functions. More sophisticated remote controls, such as Philips's ProntoPro, offer multifunction remote control, which has a color display screen, programmable buttons and facilities to set up, change and personalize the user interface. .

Previous US patent application 2003 0 071 104 A1 relates to a network system for remote control of an object such as an electrical appliance. The system includes a video camera that detects only light in a particular wavelength range and monitors the object to be controlled. The remote controller includes an optical pointer in the specific wavelength range. A control unit is arranged to process the output signal from the video camera to detect one or more objects represented by this pointer. In addition, the control unit is arranged for receiving an operation signal from the remote controller, which signal relates to a remote control operation executed on the displayed object. In addition, the control unit is arranged to provide a control signal according to the operation signal via the network to the object indicated by this pointer, so that the object can be controlled remotely by a remote controller. In a particular embodiment, the projector is further provided for displaying an operating panel for the object to be controlled, so that the object can be controlled remotely by pointing on this operating panel.

Although this provides a system for selecting an object to be controlled by simply directing the remote control towards the object, the disadvantage of the system described in accordance with US patent application 2003 0 071 104 A1 is that if the system simultaneously covers a full home environment, If you offer a camera that offers the possibility of providing an operating panel close to each object under control, the system will be complex and expensive. In addition, the cameras and projectors are somewhat stationary in setting, so that the system according to US patent application 2003 0 071 104 A1 is not very suitable for environments containing non-fixed objects to be controlled.

It is an object of the present invention to overcome the conventional drawbacks and to provide a device for remote control of a plurality of household appliance devices.

According to the invention, this object comprises a means for the remote control device to determine a user interface, a direction of the remote control device in relation to the at least one household appliance device and means for adjusting the user interface based on the direction of the remote control device. It is realized in that point.

The user interface may include, for example, a regular display screen, a touch screen, a speaker, and / or one or more keys. Adjustment of the user interface may include, for example, changing the image displayed on the display screen. The means for adjusting the user interface may be implemented in software and / or hardware. The remote control device can control robots and moving devices such as remotely controllable cars, airplanes, ships. Other examples of consumer electronic devices that can be controlled by this remote control device include televisions, audio receivers, audio amplifiers, satellite receivers, cable decoders, terrestrial receivers, recording devices, game consoles, optical disc players, and the like. Can be. Thus, an envisioned object can be accomplished while providing a flexible method of controlling a plurality of consumer electronic devices in an active environment using an intuitive graphical user interface.

The remote control device further comprises means for identifying a particular one of the consumer electronics devices based on the orientation of the remote control device and providing control of the particular one household appliance device on a display screen of the user interface. For example, if a user points this remote control device to a television, The touch screen of the control device can display channel up / down buttons, mute buttons, teletext, on / off buttons, and the like.

The means for determining the direction of the remote control device relative to the at least one household appliance device can determine the location and reference direction of the remote control device, the location of the at least one household appliance device. Means for determining the direction may include, for example, a receiver for receiving a location of at least one household appliance device from at least one household appliance device. The accessory device or household device in proximity to the consumer electronics device may be equipped with a GPS-receiver that determines the location of the device and a wireless transmitter (eg IR or RF) that transmits the location of the device to the remote control device. Instead of using GPS, the consumer electronics device or accessory device may receive a signal from another criterion, such as a marker located in an area, for example to triangulate the location of the device. The remote control device can also determine the position of the device by triangulation. The remote control device may also include a compass to determine the stimulation criteria, or determine the criteria with respect to the marker located in the area.

The remote control device may further comprise means for providing and compiling an interactive map of the plurality of household appliances devices on the display screen of the user interface based on the direction of the remote control device. This may allow, for example, the user to draw a virtual line between the middle of the display screen of the home appliance and the remote control device the user wishes to control, so that the user can, for example, display a text label or The representation of the home appliance device, such as an image, can be quickly located.

The interactive screen may be further based on the location of the at least one household appliance device. This allows the user to pinpoint the location of the consumer electronics device on the virtual line if multiple devices are near or on the line. The location can be two or three dimensional. In addition, the interactive screen may be two or three-dimensional. At least one household appliance device may be represented by an image of the at least one household appliance device. The image may be, for example, a drawing or a photograph. This allows the user to control the consumer electronic device without knowing the label of the device.

The remote control device may comprise means for providing a graphical representation of a media content item at an initial position on the interactive screen, means for imparting the graphical representation of a media content item to a simulated sound relating to the graphical representation of a particular consumer electronic device; And means for manipulating the physical representation of the media content item with respect to the particular household appliance device. This allows for example to quickly reproduce this media content without the user having to select the consumer electronics device first.

 The remote control device may further comprise means for providing a user interface on the display screen for controlling the particular household appliance device when a media content item is received by the particular household appliance device.

Advantageously, the user interface also needs to include controls useful for controlling the reproduction of the media-content item. For example, a picture-in-picture button does not need to be displayed when a music file is being played.

The remote control device may further comprise means for returning the graphical representation of the media content item to the initial position when the media content item is rejected by the consumer electronics device. This allows the user to quickly find out whether a given consumer electronic device can be used to reproduce the selected media content item.

In another aspect of the invention, the computer program allows the programmable device to function as the remote control device of claim 1. A computer program product comprising this computer program can be loaded directly into the internal memory of a programmable device. The computer program product may be stored on a computer readable storage medium.

By reference, international application WO0124387, incorporated herein by reference, describes a programmable remote control unit capable of initiating a retrieval of a control configuration from a storage device external to the unit. The unit includes a memory for storing the retrieved control configuration; A display for displaying an icon representing this configuration; And a touch screen for entering a selection based on the displayed icon. This storage device becomes a component of the household appliance. By storing or backing up the control specification for a particular device within the device itself, the remote control unit is made universally programmable.

By reference, US Pat. No. 6,563,430, incorporated herein, describes a location dependent user control interface. The context control parameters are position related and the user control interface is tailored to the context within the device in which it operates. The control interface includes a representation of context-sensitive information and communication of corresponding context-sensitive user commands via the interface. Location is determined by using conventional absolute positioning devices such as LORAN and GPS, as well as any many commonly available technologies such as active badges for direct entry, infrared sensors and comparative location tracking. Preferably, the device communicates with a remote information source that provides context-sensitive control information. The remote information source may be a home network server, an internet server, a public server network or another communication network.

By reference, International Patent Application No. WO0123994, incorporated herein by reference, describes a handheld remote control having a wireless modem that transmits an identifier to a server on the Internet. The server has a lookup table associated with the identifier with the URL. The URL points to a CGI program on another machine on the Internet. This machine controls the device through the execution of a CGI program.

Other objects and features of the present invention will become apparent from the following detailed description with reference to the accompanying drawings. It should be noted, however, that the drawings are shown for purposes of illustration only and not to limit the scope of the invention. This is because the reference should be made to the accompanying drawings. It is also to be noted that the drawings do not necessarily need to be drawn to scale, and unless otherwise indicated, the drawings are merely illustrative of the structures and procedures described hereinafter.

In the drawings, like reference numerals refer to like elements throughout the several views.

1 shows the principle of operation of a device according to the invention;

2 illustrates an interactive screen of a graphical representation of a consumer electronic device within a control range.

1 schematically illustrates a device for remote control of a plurality of household appliance devices 4, 5, 6. The device comprises a handheld remote control 1 with a display screen 2. The remote control 1 further comprises means for user interaction, which means for example the display screen 2 of the remote control 1 providing a touch screen function and / or a programmable function key 3. It may include a means integrated into. Moreover, the remote control has a directional nature, which allows the user to aim and direct the remote control in a predetermined direction.

The device further comprises means for compiling the real-time three-dimensional position and orientation information from the household appliances devices 4, 5, 6 and the remote control 1 within the control range of the remote control 1. This is realized by providing real-time three-dimensional position and orientation information to the household appliance devices 4, 5, 6 controlled by the remote control 1 of the device. For location tracking, for example, a local indoor ultrasound system can be used. Direction can be established by using a digital compass or more sophisticated three-dimensional sensing technology. Therefore, the household appliance devices 4, 5, 6 are arranged to transmit the position and orientation of the household appliance devices to the handheld remote control 1. In this way, the handheld remote control 1 can generate the latest spatial three-dimensional model of the household appliance 4, 5, 6 within the control range of the remote control 1.

Furthermore, means are provided for establishing the position and orientation of the remote control 1 with respect to the household appliances devices 4, 5, 6. Also, any indication indicated by the remote control 1 substantially towards the home appliance device (eg, in the direction indicated by the dashed arrow 10 pointing to the home appliance device 5, as shown in FIG. 1). Means are provided for using the direction and location information to identify the household appliance devices 4, 5, 6. When a household appliance, such as household appliance 5, is identified, means for providing a user interface on display screen 2 is activated to allow control of the identified household appliance.

In a further embodiment, the device selectively provides and compiles means for interpreting position and orientation information, and an interactive screen of the graphical representation of the consumer electronics device within the control range on the display screen 2, as illustrated in FIG. 2. It further comprises means for.

In a further embodiment, the device comprises means for providing a map of the physical structure on the display screen 2, which screen overlays the interactive screens and is placed on the walls, ceilings and floors of the room ( 4,5,6). Furthermore, means are provided for the remote control 1 to distinguish between spaces, for example rooms, for example, which are bounded by physical structures. It is envisioned that this distinguishing information can be used for a suitable communication mode, for example, to allow control of the household appliance devices 4, 5, 6, which are completely or partially obscured by the physical structure.

In addition, as illustrated in FIG. 2, in another embodiment, the device comprises means for providing a graphical representation of the media content item 11 at an initial position on the interactive screen. Also, for example, as illustrated in FIG. 2, the simulated movement of the media content item 11 in relation to the graphical representation of the particular household appliance device along the path indicated by the dashed arrow 12 toward the household appliance device 6. Means are provided for imparting graphical representations.

Means for directing the physical representation of the media content item 11 towards a particular household appliance device 4, 5, 6 are arranged to be activated when the devices are given this simulation movement. Moreover, when the media content item 11 is received by a particular household appliance device 4, 5, 6, it provides a user interface on the display screen 2 to control the particular household appliance device 4, 5, 6. Means and means are provided for returning the graphical representation of the media content item 11 to its initial position when the media content item 11 is rejected by the consumer electronic devices 4, 5, 6.

In another embodiment, the means for user interaction comprises means integrated with the display screen 2 of the remote control 1 to provide touch screen functionality to the remote control.

A method for allowing remote control of a plurality of household appliance devices 4, 5, 6 using a handheld remote control 1 having a display screen 2 and means for user interaction and directional characteristics is the next step. Providing real-time three-dimensional position and orientation information to the consumer electronics devices 4, 5, 6 and the remote control 1; Ensuring communication of the position and direction information from the household appliance devices 4, 5, 6 within the control range of the remote control 1 to the remote control 1; Directing the remote control (1) towards one of the plurality of household appliances devices (4, 5, 6); Identifying one household appliance device displayed through the direction operation using location and direction information; Providing a user interface on the display screen 2 to allow control of the one household appliance identified.

Using the above method, the user can "tune" the consumer electronic device. If the handheld remote control 1 is directed towards the consumer electronic device, the graphical user interface controls (e.g. buttons and sliders) of this device are displayed on the display screen 2 of the handheld remote control 1, This establishes a link for interactive control of the household appliance device.

In another embodiment, the method further includes the following steps: interpreting position and orientation information; As illustrated in FIG. 2, providing an interactive screen on the display screen 2 for a graphical representation of the home appliance device 4, 5, 6 within a control range.

In another embodiment, the method also includes the following steps: providing a screen of physical structure on the display screen 2, which screen is located in a home appliance device such as an indoor wall, a ceiling and a floor. Providing a screen, accommodating 4,5,6 and superimposing the interactive screen; Distinguishing between spaces, for example rooms, limited in scope by these physical structures. Further steps, such as using this identification information, for example, to select a suitable communication mode to allow control of the home appliance devices 4, 5, 6, which are completely or partially obscured by this physical structure. Can be envisioned.

In another embodiment, the method includes the following steps: providing a graphical representation of the media content item 11 at an initial position on the interactive screen; Assigning a simulated movement with respect to the graphical representation of the particular household appliance device 4, 5, 6 to the graphical representation of the media content item 11; Directing the physical representation of the media content item towards a particular household appliance device 4, 5, 6; If the media content item is received by a particular household appliance device (4,5,6), providing a user interface (not shown) on the display screen (2) to control the particular household appliance device; If the media content item 11 has been rejected by the particular consumer device, returning the graphical representation of the media content item 11 to an initial position, for example, the position illustrated in FIG.

The addition of these last mentioned method steps allows the user interaction manner in which the content item 11 is "thrown" from the display screen 2 in the direction of the household appliance devices 4, 5, 6. If there is a consumer electronics device 4, 5, 6 on the path of the content item 11, the content item is “caught” and received by the consumer electronics device 4, 5, 6, for example, a video clip. Or play a music playlist. If the content item 11 is accepted, the media control will be accessible on the display screen 2 of the handheld remote control 1, for example to start or stop playing the video. If the consumer electronics devices 4, 5, 6 are not encountered, the content item 11 will be returned to the display screen 2, like a boomerang that is thrown back into place. To initiate this operation, the user can drag a graphical representation of the media content item 11 onto the display screen 2, for example a touch screen, and display a speed and a graphical representation of the media content item 11. Direction can be given. Thus, this speed and direction can be used to determine the virtual path by the user releasing the object.

It is envisioned that the above method may be implemented by a computer program suitable for carrying out the above method when stored on a computer readable storage medium and the program is executed on a digital computer.

It is further envisioned that the method can be loaded directly into the internal memory of a digital computer and, if the program is run on a computer, can be implemented by a computer program product comprising a computer program that executes the method.

As illustrated above, the present invention utilizes a plurality of consumer electronic devices 4, 5, 6 using a handheld remote control 1 with a display screen 2 as well as means for user interaction and directional characteristics. To a method, a device and a computer program product for remote control. Through the remote control 1 which detects the position and orientation of the device with respect to the household appliances 4, 5, 6 which are within the control range, the household appliance can be automatically identified and a suitable graphical user for the control of the household appliance An interface may be provided on the display screen 2 of the remote control 1 in response to the remote control directed towards a particular device.

The above-described remote control 1 may be any suitable for remote control application (e.g., an infrared optical communication technology such as a Bluetooth (registered trademark) communication technology or a wireless local area network (WLAN) communication technology, a radio frequency communication technology). Can be arranged to communicate with the consumer electronics devices 4, 5, 6 by means of the current or future communication technology. Moreover, the device discovery mechanism can be used to scan the (wireless) environment at predetermined intervals to detect and compile information about the home appliance devices 4, 5, 6 within the control range. Common techniques for achieving device discovery are Bluetooth device scanning and Universal Plug & Play (UPnP) over WiFi networks.

Although the basic novel features of the invention as applied to the preferred embodiments have been shown, described and clarified, those skilled in the art can alter, replace, omit, and modify the details and forms of the devices described above without departing from the scope of the invention. It will be clear that For example, it should be particularly noted that any combination of components and / or method steps that perform the same function in substantially the same way to achieve the same result is within the scope of the present invention. Moreover, the structures and / or components and / or method steps described and / or illustrated in connection with any other disclosed form or embodiment of the present invention may be adapted to any other disclosed or described or suggested design, depending upon general design choices. It should be noted that it may be incorporated into a form or embodiment. Accordingly, it should be understood that any reference numeral should not be construed as limiting the scope of the claims, but only as indicated by the scope of the appended claims.

As noted above, the present invention is applicable to methods, devices, and computer program products for remote control of a plurality of consumer electronic devices.

Claims (11)

User interface; Means for determining a direction of the remote control device relative to the at least one household appliance device; And Means for adjusting the user interface based on the orientation of the remote control device A remote control device for controlling a plurality of home appliance devices comprising a. The method of claim 1, Means for identifying a particular one of the home appliance devices based on the orientation of the remote control device and providing control of the particular one home appliance device on a display screen of the user interface Remote control device for controlling the. The method of claim 1, Means for determining a direction of the remote control device relative to the at least one household appliance device controls the location and reference direction of the remote control device and the location of the at least one household appliance device Remote control device. The method of claim 1, Means for compiling and providing an interactive map of the plurality of household appliances devices on a display screen of the user interface based on a direction of the remote control device. Control device. The method of claim 4, wherein And wherein the interactive map is further based on a location of at least one household appliance device. The method of claim 4, wherein At least one household appliance device is a remote control device for controlling a plurality of household appliances, characterized in that represented by an image of the at least one household appliance device. The method of claim 4, wherein Means for providing a graphical representation of a media content item at an initial location on the interactive map; Means for imparting the graphical representation of a media content item to a simulated tone relating to the graphical representation of a particular consumer electronic device; And And means for sending a physical representation of the media content item to the particular household appliance device. The method of claim 7, wherein And means for providing a user interface on the display screen for controlling the specific household appliance when the media content item is received by the particular household appliance. . The method of claim 7, wherein And means for returning the graphical representation of the media content item to the initial position when the media content item is rejected by the consumer electronics device. A computer program stored on a computer readable storage medium that allows the programmable device to function as the remote control device of claim 1. A computer program product comprising a computer program that can be loaded directly into an internal memory of a programmable device, the computer program allowing the programmable device to function as the remote control device of claim 1.

Images