EP3007146A1

EP3007146A1 - System for controlling an electronic device and head mounted unit for such a system

Info

Publication number: EP3007146A1
Application number: EP14306571.2A
Authority: EP
Inventors: Didier Doyen; Sylvain Thiebaud; Stéphane Szymanski
Original assignee: Thomson Licensing SAS
Current assignee: Thomson Licensing SAS
Priority date: 2014-10-07
Filing date: 2014-10-07
Publication date: 2016-04-13

Abstract

The invention relates to a system for controlling an electronic device (20) through a user interface associated with said electronic device. The system comprises a head mounted unit comprising a display device (10) for displaying said dedicated user interface and a camera (11) for capturing an image of the electronic device to be controlled, a video processing unit (12), and a remote control unit (13). The head mounted unit, the video processing unit, the remote control unit and the electronic device are configured to communicate with each other. The video processing unit (12) is used to identify the electronic device (20) to be controlled in the image captured by the camera, to transmit the user interface of the electronic device to the display device and to transmit a command instruction associated with a command selected by the remote control unit.

Description

1. Technical Field

The present invention relates generally to the field of the user interaction with devices connected together. More specifically, the present invention concerns a system for controlling an electronic device through a user interface displayed on a display device of a head mounted unit. It also concerns a head mounted unit for such a system.

2. Background Art

The present invention aims at improving the communication with any electronic devices present in the home. It is still a problem for people in their living room to be able to find the right remote control to communicate with a given electronic device. This problem becomes a nightmare when the number of electronic devices is high and when each of them is equipped with its own remote control. So there is a clear need of a universal remote control.
But to be universal, a remote control should be able to address any electronic device whatever the type of command to be applied. This is generally not feasible to have a simplified set of buttons so as to address every kind of electronic devices for any command.
Some universal remote control units exist. They are able to address different electronic devices thanks to a pre-setting of each command to a given button of the remote control unit. So the electronic devices controlled by the remote control unit belong to a predefined list. The addition of a device to this list requires pre-setting other buttons of the remote control. As buttons should be associated with a command in a comprehensive way, their number can be huge.
On the other hand, there are user interfaces available for any electronic device having a display (e.g. a TV) or any electronic devices directly connected to a display (e.g. a set-top-box). In these cases the remote control unit associated with the electronic device can be simpler. One can navigate inside a menu of said user interface displayed on this display thanks to a set of arrows representing the different directions and a validation button. It allows reducing significantly the number of buttons of the remote control. But this is not adapted to electronic devices without display, like for example an audio amplifier. There are a lot of electronic devices for which no universal remote control and no display can be associated with, although they could be connected to a home communication framework or, more generally, to a local network, with which the electronic devices can communicate.
A purpose of the present invention is to propose a remote control for any electronic devices connected to at least a local network.
There are also some remote control systems using see-through glasses for providing a user interface. In such a system, the see-through glasses display the user interface associated with the electronic device to be controlled. The selection of a command in the user interface can be performed by moving the head in such a way that the command to be performed is aligned with the device to be controlled. Such a system is for example disclosed in the patent application US 2003/0020707 . This system does not require any dedicated remote control unit but the action of alignment to be made for selecting a command is not so easy to be executed, especially if there are lots of electronic devices in the field of view of the user. So the selection of commands by head moving is not easy, especially if the user interface comprises a lot of buttons.
In other systems using see-through glasses, it is proposed to the user to make the selection of a command in the user interface displayed by the see-through glasses by using his fingers. Such a system is for example disclosed in the patent application US 2011/0234386. A command in the user interface displayed on the see-through glasses is selected by placing a finger on an icon associated with said command. This system does not also require any dedicated remote control unit but the placement of the finger on the icon of the command to be performed without going through other icons and without risking selecting wrong commands is not so easy to execute. So this type of system is not reliable.
A purpose of the present invention is to propose a system able to reliably select a command of a user interface displayed through a head mounted display.

3. Summary of Invention

The present invention concerns a system for controlling an electronic device through a user interface associated with said electronic device, said user interface comprising a plurality of commands. The system comprises a head mounted unit comprising a display device for displaying said dedicated user interface and a camera for capturing an image of the electronic device to be controlled, a video processing unit, and a remote control unit. The head mounted unit, the video processing unit, the remote control unit and the electronic device are configured to communicate with each other. In this system, the video processing unit is configured to identify the electronic device to be controlled in the image captured by the camera, to transmit the user interface associated with the identified electronic device to the display device, and to transmit to the electronic device a command instruction associated with the command selected by the remote control unit in said user interface displayed by the display device.
In one particular embodiment, he head mounted unit, the video processing unit, the remote control unit and the electronic device are configured to communicate via a communication framework. The communication framework designates a framework by which devices connected to this framework can exchange data. The data are exchanged on one or more communication networks. These communication networks may be wireless or not. And the data are exchanged according to one or more communication protocols. The communication framework is for example a home communication framework like QeO framework developed by Technicolor.
The invention exploits the connection of electronic devices of any type to a communication framework to allow exchanges between a remote control combined with a display for controlling a connected electronic device.
The remote control unit is any electronic device connected to the communication framework having means to navigate in the menus of the user interface.
In one particular embodiment, the remote control unit is a hand-held device. It can be a smartphone or any hand held electronic device like a portable media player.
In one particular embodiment, the head mounted device comprises see-through glasses, said see-through glasses being equipped with said camera.
In one particular embodiment, the video processing unit is carried by the head mounted unit and is connected directly to the display device and/or the camera.
In a variant, the video processing unit is included in the remote control unit.
In one particular embodiment, the video processing unit is configured to identify, in the image captured by the camera, the electronic device to be controlled by detecting, in said image, a visual code associated with said electronic device.
In one particular embodiment, the visual code is a barcode or a QR code.
In one particular embodiment, the video processing unit is configured to identify, in the image captured by the camera, the electronic device to be controlled by detecting, in said image, the picture of the front end of said electronic device.
The invention concerns also a head mounted unit comprising a display device for displaying a user interface, a camera for capturing an image of the electronic device to be controlled, and a video processing unit configured to identify the electronic device to be controlled in the image captured by the camera, to transmit the user interface associated with the identified electronic device to the display device, to receive a command from a remote control, and to transmit a command instruction associated with the selected command to the electronic device.
In one particular embodiment, the video processing unit receives the command from the remote control via a communication framework and transmits the command instruction associated with the selected command to the electronic device via said communication framework.
In one particular embodiment, the video processing unit is configured to identify, in the image captured by the camera, the electronic device to be controlled by detecting, in said image, a visual code associated with said electronic device.
In one particular embodiment, the visual code is a barcode or a QR code.
In one particular embodiment, the video processing unit is configured to identify, in the image captured by the camera, the electronic device to be controlled by detecting, in said image, the front end of said electronic device.

4. Brief description of the drawings

The invention can be better understood with reference to the following description and drawings, given by way of example and not limiting the scope of protection, and in which:

Figure 1 illustrates a system for controlling an electronic device according the principles of the invention;
Figure 2 is a schematic view of the global architecture of the system of Figure 1.
Figure 3 is a flow chart of the successive steps implemented by the system according to the invention; and
Figure 4 illustrates the capture of an image of an electronic device to be controlled according to the invention.

5. Description of embodiments

In reference to Figures 1 and 2, the system according to the invention comprises a see through glasses 10 having a display for displaying user interface of electronic devices 20 to be controlled, a camera 11 for capturing an image of the electronic devices 20 to be controlled, a video processing unit 12, and a remote control unit 13. The see-through glasses 10 are worn by a user. All these components are connected together and are able to communicate with each other.
In Figures 1 and 2, the see through glasses 10, the camera 11, the video processing unit 12, the remote control unit 13 and the electronic devices 20 communicate with each other via a communication framework 14.
The communication framework 14 is preferably a local framework like a home communication framework. For example, the components of the system are all connected to a local network, for example a wireless network (WiFi), and communicate together via a communication protocol like QeO framework developed by Technicolor.
The communication framework 14 can also be used through a global network like internet.
The remote control unit 13 is an electronic device connected to the network and using the communication framework and having means to navigate in the menus of a user interface. It can be a smartphone or any hand held electronic device like a portable media player. This device may comprise keys (tactile or not), like arrow keys and enter key, for navigating in menus of user interfaces.
The video processing unit 12 may be a separate unit as illustrated by Figure 1, may be integrated to the see-through glasses 10 or may be the processing means of a set top box. The video processing unit may also be processing means of the remote control unit, for example processing means of a smart phone.
If the video processing unit 12 is integrated to the see-through glasses 10, it is directly connected to the camera 11 and the display of the see-through glasses without using the communication framework 14.
The video processing unit 12 is configured to identify the electronic device 20 to be controlled in the image captured by the camera 11 and to transmit to the display device the user interface associated with the identified electronic device via the communication framework 14. This user interface is displayed by the see-through glasses 10 and the remote control unit 13 is used to select commands in this user interface.
Figure 3 shows the successive steps implemented by the system of Figure 1 and 2.
In a step S1, the camera 11 captures an image of the device 20 to be controlled. The camera is directed towards the device 20 by the user by looking at the device. An image of the front end of the device is captured by the camera 11 as shown in Figure 4.
In a step S2, the captured image is sent to the video processing unit 12 via the communication framework 14.
In a step S3, the image is processed by the video processing unit 12 to detect the device 20 to be controlled and to identify it. As mentioned before, the device 20 is detected and identified by detecting a visual code present in its front end, for example a QR code 21, or by comparing a picture of its front end to pre-stored front ends pictures. A user interface associated with the identified device 20 is selected.
In a step S4, the video processing unit 14 sends the user interface software associated with the identified device 20 to the display 10 of the see through glasses.
In a step S5, the display device 10 of the see through glasses displays the received user interface software and displays the user interface on its screen.
In a step S6, the user selects by the remote control unit 13 a command in the displayed user interface for controlling the electronic device 20. This selection is displayed on the see-through glasses. It can correspond to different menus that are reached using the remote control unit. So the video processing unit 12 receives a command representative of the command selected by the user in the user interface.
In step S7, the video processing unit 12 transmits a command instruction associated with the command selected by the user to the electronic device 20.
If several electronic devices (2 devices stacked together) are present in the captured image and detected by the video processing unit 12, the video processing unit displays a menu for proposing to the user to select the electronic device to be controlled. This menu is displayed by the display device 10 and the selection of the device to be controlled is made by the remote control unit 13. In a variant, if several electronic devices are detected in the captured image, one of the detected device's user interfaces is proposed by default to the user on the display of see-through glasses. If the displayed user interface is not the right one, the user can change the user interface by selecting a dedicated icon in the user interface or by moving his head in the direction of the other device he wants to select. In a more elaborated embodiment, the video processing unit 12 sends through the communication framework 14 a message to the selected electronic device 20 asking it to show that it has been selected by lighting a LED of the front end of the electronic device 20.
Thus, according to the invention, any electronic device 20 connected to the communication framework 14 and identifiable by a specific visual code or by identifying the front end picture can be controlled by the system. It is then easy to drive these electronic devices with a simple universal remote control 13 that can be a dedicated device or even a smartphone.
The great advantage is to simplify the way you drive your electronic devices at home, avoiding spending time to find the right remote control in your living room. The way to control the electronic device 20 is then as simple as with any electronic device having a display. At last, it should be noted that in one embodiment of the invention, the previously mentioned processing means and/or means to navigate can be implemented in hardware in a programmable FPGA ("Field Programmable Gate Array") component or ASIC ("Application-Specific Integrated Circuit") component.
In one embodiment of the invention, a device such as a head mounted unit or the video processing unit or a remote control unit, comprises a computing unit (for example a CPU, for "Central Processing Unit"), and one or several memory units (for example a RAM (for "Random Access Memory") block in which intermediate results can be stored temporarily during the execution of instructions a computer program, or a ROM block in which, among other things, computer programs are stored, or an EEPROM ("Electrically-Erasable Programmable Read-Only Memory") block, or a flash block). Computer programs are made of instructions that can be executed by the computing unit. Such device can also comprise a dedicated unit constituting an input-output interface to allow the device to communicate with other devices. In particular, this dedicated unit can be connected with an antenna (in order to perform communication without contacts), or with serial ports (to carry communications via physical contacts). These units can exchange data through buses for example together.

Claims

System for controlling an electronic device (20) through a user interface associated with said electronic device, said user interface comprising a plurality of commands, characterized in that the system comprises:
a head mounted unit comprising a display device (10) for displaying said dedicated user interface and a camera (11) for capturing an image of the electronic device to be controlled,

a video processing unit (12), and

a remote control unit (13),
and in that:
- the head mounted unit, the video processing unit, the remote control unit and the electronic device are configured to communicate with each other, and

- the video processing unit (12) is configured to identify the electronic device to be controlled in the image captured by the camera, to transmit the user interface associated with the identified electronic device to the display device, and to transmit to the electronic device a command instruction associated with command selected by the remote control unit (13) in said user interface displayed by the display device.
System according to claim 1, wherein the remote control unit (13) is a hand-held device.
System according to claim 1 or 2, wherein the head mounted device comprises see-through glasses, said see-through glasses being equipped with said camera (11).
System according to any one of claims 1 to 3, wherein the video processing unit (12) is carried by the head mounted unit and is connected to the display device (10) and/or the camera (11).
System according to any one of claims 1 to 4, wherein the video processing unit (12) is configured to identify, in the image captured by the camera, the electronic device to be controlled by detecting, in said image, a visual code associated with said electronic device.
System according to claim 5, wherein the visual code is a barcode or a QR code (21).
System according to any one of claims 1 to 4, wherein the video processing unit (12) is configured to identify, in the image captured by the camera, the electronic device to be controlled by detecting, in said image, the picture of the front end of said electronic device.
Head mounted unit comprising a display device (10) for displaying a user interface and a camera (11) for capturing an image of the electronic device to be controlled,
characterized in that it further comprises a video processing unit (12) configured to identify the electronic device to be controlled in the image captured by the camera, to transmit the user interface associated with the identified electronic device to the display device, to receive a command from a remote control unit (13), and to transmit a command instruction associated with the selected command to the electronic device.
Head mounted unit according to claim 8, wherein the video processing unit (12) is configured to identify, in the image captured by the camera, the electronic device to be controlled by detecting, in said image, a visual code associated with said electronic device.
Head mounted unit according to claim 9, wherein the visual code is a barcode or a QR code (21).
Head mounted unit according to claim 8, wherein the video processing unit (12) is configured to identify, in the image captured by the camera, the electronic device to be controlled by detecting, in said image, the picture of the front end of said electronic device.