EP2141674B1 - Assembly with device which can be controlled remotely - Google Patents

Assembly with device which can be controlled remotely Download PDF

Info

Publication number
EP2141674B1
EP2141674B1 EP08075599.4A EP08075599A EP2141674B1 EP 2141674 B1 EP2141674 B1 EP 2141674B1 EP 08075599 A EP08075599 A EP 08075599A EP 2141674 B1 EP2141674 B1 EP 2141674B1
Authority
EP
European Patent Office
Prior art keywords
appliance
remote
control device
inputs
control
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Not-in-force
Application number
EP08075599.4A
Other languages
German (de)
French (fr)
Other versions
EP2141674A1 (en
Inventor
Ralf Kirchherr
Joachim Stegmann
Bernd Lochschmidt
Fred Runge
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Deutsche Telekom AG
Original Assignee
Deutsche Telekom AG
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Deutsche Telekom AG filed Critical Deutsche Telekom AG
Priority to EP08075599.4A priority Critical patent/EP2141674B1/en
Publication of EP2141674A1 publication Critical patent/EP2141674A1/en
Application granted granted Critical
Publication of EP2141674B1 publication Critical patent/EP2141674B1/en
Not-in-force legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G08—SIGNALLING
    • G08C—TRANSMISSION SYSTEMS FOR MEASURED VALUES, CONTROL OR SIMILAR SIGNALS
    • G08C17/00—Arrangements for transmitting signals characterised by the use of a wireless electrical link
    • G08C17/02—Arrangements for transmitting signals characterised by the use of a wireless electrical link using a radio link
    • G—PHYSICS
    • G08—SIGNALLING
    • G08C—TRANSMISSION SYSTEMS FOR MEASURED VALUES, CONTROL OR SIMILAR SIGNALS
    • G08C23/00—Non-electrical signal transmission systems, e.g. optical systems
    • G08C23/04—Non-electrical signal transmission systems, e.g. optical systems using light waves, e.g. infrared
    • G—PHYSICS
    • G08—SIGNALLING
    • G08C—TRANSMISSION SYSTEMS FOR MEASURED VALUES, CONTROL OR SIMILAR SIGNALS
    • G08C2201/00—Transmission systems of control signals via wireless link
    • G08C2201/30—User interface
    • G08C2201/31—Voice input
    • G—PHYSICS
    • G08—SIGNALLING
    • G08C—TRANSMISSION SYSTEMS FOR MEASURED VALUES, CONTROL OR SIMILAR SIGNALS
    • G08C2201/00—Transmission systems of control signals via wireless link
    • G08C2201/30—User interface
    • G08C2201/32—Remote control based on movements, attitude of remote control device
    • G—PHYSICS
    • G08—SIGNALLING
    • G08C—TRANSMISSION SYSTEMS FOR MEASURED VALUES, CONTROL OR SIMILAR SIGNALS
    • G08C2201/00—Transmission systems of control signals via wireless link
    • G08C2201/50—Receiving or transmitting feedback, e.g. replies, status updates, acknowledgements, from the controlled devices

Definitions

  • the invention relates to an arrangement with a remote-controlled device, wherein the arrangement next to the device to be operated remotely, preferably a device of entertainment electronics or for playback of media content, a remote control device for wireless operation of the aforementioned device. It relates to a solution which allows multimodal operation of the remote device by means of the associated remote control device.
  • Wireless remote controls have long been known and widely used in particular for the operation of consumer electronics and home appliance technology. Most commonly found are remote controls in which the respective device functions of the device to be operated by means of which they are controlled by means of buttons arranged on the remote control. The keys assigned to the respective keys for operating the associated device are converted in the remote control into a digital code and transmitted by modulating a carrier signal to a receiver of the remote-controlled device designed for this purpose. For coding, for example, the RC-5 code or the RC-6 code used in the aforementioned remote controls, which were developed starting from keyboard-controlled remote controls. The wireless transmission to the device to be operated is generally carried out by modulation of an infrared or a radio carrier signal. When operating a device using a keyboard, however, the natural human communication channels are not used. The operation of devices by means of appropriate remote controls is therefore designed for the user generally not very intuitive and not infrequently even creates considerable difficulties.
  • a remote control device which works at least bimodal.
  • the remote control device comprises, within a device, an at least bimodal input module, a module with an associated database for interpreting, for example, by means of speech and by movements of the remote control device inputs and a module for generating control signals on the basis of recognized during the interpretation commands.
  • the entire speech recognition and interpretation takes place within the remote control device, which preferably contains corresponding command memory for this input mode and at least one further input mode. Due to the derivation of the device operating control signals directly in the remote control device but their flexibility in terms of adaptation to different types of devices and device types is relatively limited.
  • a return channel between the device and the remote control device is intended to be designed to be hardly context-sensitive. This means that it is only possible to a limited degree to adapt the respectively provided operating functions to the context or to the current events in connection with a content respectively reproduced by the device. In that regard, a largely intuitive operation in terms of standardized and unchanging, that is static device functions possible, the provision of operating options with respect to dynamically changing states of the device to be operated or on dynamically changing, consumed with the device media content but not possible.
  • From the EP 1 079 371 A1 is also a bimodal working arrangement with a remote control device for wireless remote control of a device, preferably a television set known, wherein a bidirectional data exchange between the remote control device and the device operated with this takes place.
  • the arrangement makes it possible, on the basis of an electronic program guide provided by the device to be operated, to select a program by means of the remote control device and a microphone arranged in this voice-controlled manner in a largely natural manner, with a corresponding program channel optionally also by means of a channel selection button of the remote control device to be selected. A more comprehensive, beyond the channel selection operation by means of the keyboard is not provided.
  • At least some of the units required for the analysis or interpretation of the speech inputs can be arranged in the device to be operated itself. It is disadvantageous in this solution, as well as in the US 2006/0077174 A1 described that voice inputs for the operation of media devices, if they are to be possible speaker independent, may not always be reliably detected or interpreted. Difficulties could arise when the operation mediale Applies to devices over which audio signals, in particular in the form of speech-based content, are output during the operation.
  • the invention is defined according to claim 1. Embodiments are defined in dependent claims.
  • the object of the invention is to provide an alternative arrangement, which allows a particularly intuitive and flexible remote control of device functions of an electronic device with extensive use of natural human communication channels, such as the language, in particular, a secure detection of linguistic operator input should be guaranteed.
  • the proposed arrangement for solving the problem consists of a fernbedienenden device and a remote control device for multimodal, wireless remote control of the aforementioned device.
  • the arrangement has at least means for wireless communication between the remote control device and the device to be operated with it, on or in the remote control device arranged input means for receiving inputs in several different input modes, at least one digitizing unit for digitizing received input, at least one interpretation unit for interpreting the in digital data converted inputs and at least one control unit for deriving control signals from in the course of the interpretation of inputs detected operating commands and to control corresponding actuators of the operated device.
  • the operating commands corresponding to the respective inputs are converted in the device in accordance with the operating commands received from them by means of the at least one interpretation unit and the control signals derived therefrom by means of the at least one control unit.
  • the input means arranged on or in the remote control device for realizing the various input modes of the arrangement is at least one microphone for receiving from voice inputs, a motion sensor for receiving input from a movement of the remote control device, and a keyboard.
  • the above-mentioned means for wireless communication may be, in particular, infrared transmitters and receivers or transmitters and receivers for radio-supported transmission of data, for example according to the Bluetooth standard.
  • the at least one interpretation unit for interpreting the inputs converted into digital data by the digitizing unit is arranged in the device to be operated itself.
  • the inputs such as voice or motion inputs, which are carried out by the user via input means, are merely digitized in the remote control device by means of the at least one digitizing unit arranged therein and the digital data obtained by a transmitter of the remote control device as a digital data stream to a receiver of the remote device. Only then does the interpretation of the inputs then take place, ie their conversion into operating commands first and then the derivation of the control signals from the operating commands with which the corresponding actuators in the device to be operated are activated.
  • the interpretation unit has a function group for interpreting keyboard inputs, a function group for interpreting movement inputs, and a function group for interpreting voice inputs with a speech recognizer and with at least one grammar defining a speech space for speech recognition.
  • the aforementioned actuators are, for example, functional groups for adjusting the volume or parameters of a screen, such as contrast or color saturation, or else functional groups for switching over a reproduced program.
  • the device to be remotely operated can be, for example, a television set, a stereo system, a satellite receiver, a set-top box or a DVD player.
  • the corresponding device according to the invention is equipped with a unit of interpretation, by means of which a spoken word representing Data stream or a movement of the remote control device representative data stream are interpreted and as far as possible, a corresponding with the data stream operator command is derived.
  • the optionally recognized operating command is transferred by the interpretation unit to the at least one control unit likewise arranged in the device to be operated for deriving the required control signals.
  • the at least one interpretation unit may be a common interpretation unit for all operating modes or one or more units.
  • the arrangement is further configured such that the input means arranged on or in the remote control device can be activated to receive voice input by means of a button provided for this purpose and arranged on the remote control device.
  • the corresponding input means are depending on the configuration for the duration of the actuation of the aforementioned button (hereinafter also push-to-talk button) or until activated again.
  • a muting of the acoustic signals which may possibly be reproduced by the operated device takes place.
  • the volume of the sound reproduction is reduced.
  • a further improvement of the reliability in the recognition of voice input is achieved by a special development of the inventive arrangement equipped with a voice input unit.
  • a two-stage operating concept for voice input is provided. Thereafter, distinction is made between a speech operation mode for controlling basic functions with a static instruction set and a speech operation mode with a dynamically changing instruction set for context-related operation of the device to be remotely operated.
  • the first mode for the use of basic functions such as turning on and off the device or the volume setting, is constantly or, if provided on the remote control device a push-to-talk button, each time you press the push to talk Button active.
  • the activation of the extended or dynamic operating mode requires additional operation, for example by pressing another key or a special voice command (for example, voice command "extended mode").
  • voice command "extended mode" for example, voice command "extended mode"
  • changing and context-dependent operating functions at shorter intervals.
  • An example of this are titles of media content that can be selected by language and that constantly change depending on the program offered.
  • two different speech grammars are provided in the device to be operated as part of the at least one interpretation unit for the interpretation of speech inputs.
  • the language space covered by a first grammar serving the interpretation of static operating functions are static
  • the interpretable with the help of the second grammar language inputs are dynamically adapt to the context and, for example, modified accordingly with the help of the Electronic Program Guide.
  • the invention is designed such that after activation of the extended operating mode and the execution of an operating function possible by means of this mode, the arrangement immediately switches back to the first static operating mode or deactivates the extended operating mode.
  • Another particularly advantageous embodiment of the invention is designed so that it allows a so-called voice-zapping.
  • the operator it is possible for the operator to address selectively selected program contents by linguistically indicating a particular genre and to cause the operated device to switch over to such a program content.
  • the voice command "Sport” could cause an inventively designed satellite receiver to switch to a sports program.
  • a user of the arrangement according to the invention would like to see on television a certain program known to him from the title, but he is not immediately aware of which station this is being broadcast.
  • the switch is made possible on the transmitter concerned, if the desired program is currently running and, if appropriate, within a period of time stored in the interpretation unit by means of an update before the start of the program.
  • the advantage of a given arrangement of the interpretation unit or units in the device to be operated and the associated higher operating flexibility is very clear.
  • a context-related or context-sensitive operation in the sense of the previously explained is only possible by the inventive arrangement of the at least one interpretation unit in the device to be operated or in the device served.
  • a further advantageous embodiment of the invention is given by the fact that the input mode for operation by a movement of the remote control device only during and for the duration of pressing a designated button (hereinafter also motion control) or until the second operation of the motion control Button is activated.
  • the operation of the associated device by the movement of the remote control device enabling arrangement can also be designed such that it allows a so-called "EasyOn / EasyOff".
  • the device to be operated is switched off by turning a side of the remote control device which is recognizable as the top side downwards or by placing the remote control device on the table with this side downwards. The turning of the remote control device on its back accordingly causes the switching of the device to be operated.
  • the interaction between the remote control device and the device to be operated is designed such that continuously adjustable device functions, such as the volume setting for the sound reproduction or the contrast adjustment for the image reproduction, can be controlled by a rotary movement of the remote control device.
  • the remote control device of the arrangement according to the invention is also preferably equipped with a display.
  • active operating modes of the remote control device can be displayed or else visual feedbacks with which the remote-controlled device acknowledges the actuation of selected functions can be reproduced.
  • the remote control device can also be equipped with means for acoustic feedback (loudspeaker) or haptic feedback (vibration unit).
  • the Fig. 1 shows a rough diagram or block diagram of a possible embodiment of the arrangement according to the invention.
  • the arrangement consists of the remote control device 1 and the device 2 to be remotely operated with it, for example a set-top box.
  • the arrangement comprises means for wireless communication 6, 7 between the remote control device 1 and the device 2 to be operated, namely an infrared transmitter 6 on the remote control device 1 and an infrared receiver 7 on the device 2, arranged in the remote control device 1 digitizing unit 11, a arranged according to the invention in the device to be operated 2 interpretation unit 3 for the interpretation of inputs of different input modes, a control unit 4 and a plurality of actuated by the control unit by means of appropriate control signals actuators 5.
  • the arrangement shown in the example has three input modes.
  • corresponding input means 8, 9, 10 are provided in or on the remote control device 1.
  • the remote control device 1 shown in the example has a keyboard 10 for receiving keyboard input, a microphone 8 for receiving voice input and an arranged in the remote control device 1 acceleration sensor 9 for receiving movements of the remote control device 1 movement inputs.
  • the inputs made by a user of the arrangement by operation or use of the input means 8, 9, 10 become a digital input via the infrared transmitter 6 on the remote control device 1 in the direction of the device 2 to be operated after being received by the remote control device 1 in the digitizer 11 radiated data stream converted.
  • the corresponding data stream is received by the infrared receiver 7 of the set-top box 2.
  • a function group 21 belonging to the interpretation unit 3, a signal converter and distributor the received digital data are separated according to their origin from the speech input means 8, the input means 8 for movement inputs and the keyboard 10 and to the actual interpretation of different function groups 12, 16, 17 supplied to the interpretation unit 3.
  • a first function group 16 the interpretation of movement inputs, that is, by horizontal or vertical movement or rotation of the remote control device 1 inputs made. The interpretation takes place in such a way that the signals received due to movements of the remote control device 1 are analyzed in the corresponding function group 16 and corresponding operating commands are determined, wherein they are checked for agreement with possible operating commands held in a command memory 18.
  • a function group 12 for the interpretation of speech signals.
  • they are analyzed by means of a speech recognizer 15 (ASR - Automatic Speech Recognition) on the basis of two different held in the function group grammars 13, 14 and derived from them, if possible, an operating command according to the specifications of the command memory 18.
  • the two aforementioned grammars 13, 14 are a first grammar 13 containing a vocabulary for activating static operator commands.
  • a second grammar 14 is provided, whose vocabulary used in speech analysis on the data resulting from the speech input is context-dependent and thus dynamic.
  • EPG 22 Electronic Program Guide
  • EPG 22 gets its data from the backend, that is an Internet server 23, a provider providing the content via the Internet 24 and / or by receiving corresponding data via a satellite system.
  • the interpretation of keyboard input is performed by these are mapped to 18 according to the specifications of the command memory possible operating commands.
  • the operating commands determined using the individual input modes are fed to the control unit 4, which derives control signals from them and with these control signals actuators 5 of the device, in the example a function group 5 'for volume control, a function group for program selection 5 "and / or a function group 5'. "to adjust screen parameters such as brightness and / or contrast.
  • a crucial aspect of the invention is the fact that the manner in which interpreted in the individual input modes operator inputs are interpreted and converted into control signals for the device to be operated by the respective operated device 2 itself, namely the at least one arranged in this Interpretation unit 3, is determined. In this case, the firmware of the corresponding interpretation unit 3 in the way of the update, for example by transmission via the satellite channel, to be changed and thus the functionality of the device 2 are adapted to different circumstances.
  • the means 6, 7 for wireless communication arranged on the remote control device 1, on the one hand, and the device 2, on the other hand, may optionally also be designed in each case as a combination of infrared transmitter and infrared receiver and thus bidirectionally. If, in addition, appropriate means, such as a display, a loudspeaker or a vibration unit are arranged in the remote control device, using this bidirectionality, a feedback or a response of the device to operator inputs, such as an acknowledgment of inputs or a warning in case of incorrect entries done ,
  • the Fig. 2 shows again a schematic representation of the remote control device 1.
  • This has, according to the example explained above, inter alia, via an input mode for the control of Device functions of the device 2 by movements of the remote device 1.
  • a basic device function controllable by this input mode is turning the device on and off.
  • This can be configured by using movement inputs in the manner as "EasyOn” and "EasyOff” that the placing of the remote control device 1 on its back or on the "back” (see a) causes the device 2, while the laying of the Remote control device 1 on the front or on the "belly” (see b) leads to turn off the device 2, which may also be actuated by the user in addition, the motion control button 20 during movement of the remote control device.
  • placing the remote control in an upright position (upside down with the head or acceleration sensor 9, respectively) may, for example, be used to activate the time-shift function of the set-top box, which stops the current media stream and allows playback at a later time, for example, as soon as the remote control again takes a different position from the upright position.
  • the gestures of the user are detected by means of a three-axis acceleration sensor 9, which detects the position of the remote control device 1 relative to the gravitational acceleration. This allows you to detect the deflection angles 'Pitch' (rotation around the horizontal X-axis) and 'Roll' (rotation around the horizontal Y-axis) (see Fig. 2 ).
  • a deflection of the remote control device for pitch and roll in the range of -180 to +180 degrees is detected.
  • the motion control key 20 can not only be used to navigate the on-screen cursor within the on-screen menu.
  • the selected menu items in the on-screen menu can also be activated with the motion control key 20, so that a separate confirmation key can also be dispensed with ("One Button_Navigation").
  • One Button_Navigation As a result, only one button is needed to control the on-screen menu and the user must be with
  • the fingers or thumbs also differ no different pressure directions as usually with conventional 5-way keys.
  • the distinction between navigation and confirmation is based on the time duration during which the motion control key 20 was pressed. A short click on the motion control key 20 is considered as confirmation, while a long press on the motion control key 20 activates the horizontal and vertical navigation of the on-screen cursor.
  • the motion control can also be used for stepless control of other system parameters, such as For example, the volume or brightness can be used.
  • the desired system size can be changed at any speed, similar to an analog rotary control.
  • a previously issued voice command eg 'Volume' or 'Brightness', 'Contrast'
  • the respective system parameters which will then be controlled by the motion control.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Selective Calling Equipment (AREA)

Description

Die Erfindung betrifft eine Anordnung mit einem fernbedienbaren Gerät, wobei die Anordnung neben dem fernzubedienenden Gerät, vorzugsweise einem Gerät der Unterhaltungselektronik beziehungsweise zur Wiedergabe medialer Inhalte, eine Fernbedienungseinrichtung zur drahtlosen Bedienung des vorgenannten Geräts umfasst. Sie bezieht sich auf eine Lösung, welche eine multimodale Bedienung des fernzubedienenden Geräts mittels der zugehörigen Fernbedienungseinrichtung ermöglicht.The invention relates to an arrangement with a remote-controlled device, wherein the arrangement next to the device to be operated remotely, preferably a device of entertainment electronics or for playback of media content, a remote control device for wireless operation of the aforementioned device. It relates to a solution which allows multimodal operation of the remote device by means of the associated remote control device.

Drahtlose Fernbedienungen sind insbesondere zur Bedienung von Geräten der Unterhaltungselektronik und der Hausgerätetechnik seit langem bekannt und sehr verbreitet im Einsatz. Am häufigsten anzutreffen sind dabei Fernbedienungen, bei welchen die jeweiligen Gerätefunktionen des mit ihrer Hilfe zu bedienenden Geräts mittels auf der Fernbedienung angeordneter Tasten gesteuert werden. Die den Tasten jeweils zugeordneten Kommandos zur Bedienung des zugehörigen Geräts werden in der Fernbedienung in einen digitalen Code umgesetzt und durch Modulierung eines Trägersignals an einen dafür ausgebildeten Empfänger des fernzubedienenden Geräts übertragen. Zur Codierung werden bei den vorgenannten Fernbedienungen beispielsweise der RC-5-Code oder der RC-6-Code verwendet, welche ausgehend von tastaturgesteuerten Fernbedienungen entwickelt wurden. Die drahtlose Übertragung an das zu bedienende Gerät erfolgt im Allgemeinen durch Modulation eines Infrarot- oder eines Funkträgersignals. Bei der Bedienung eines Geräts mittels einer Tastatur werden jedoch die natürlichen menschlichen Kommunikationswege nicht genutzt. Die Bedienung von Geräten mittels entsprechender Fernbedienungen gestaltet sich daher für den Nutzer im Allgemeinen wenig intuitiv und stellt ihn nicht selten sogar vor erhebliche Schwierigkeiten.Wireless remote controls have long been known and widely used in particular for the operation of consumer electronics and home appliance technology. Most commonly found are remote controls in which the respective device functions of the device to be operated by means of which they are controlled by means of buttons arranged on the remote control. The keys assigned to the respective keys for operating the associated device are converted in the remote control into a digital code and transmitted by modulating a carrier signal to a receiver of the remote-controlled device designed for this purpose. For coding, for example, the RC-5 code or the RC-6 code used in the aforementioned remote controls, which were developed starting from keyboard-controlled remote controls. The wireless transmission to the device to be operated is generally carried out by modulation of an infrared or a radio carrier signal. When operating a device using a keyboard, however, the natural human communication channels are not used. The operation of devices by means of appropriate remote controls is therefore designed for the user generally not very intuitive and not infrequently even creates considerable difficulties.

Daher ist es bereits bekannt geworden, elektronische Geräte sprachgesteuert zu bedienen. Entsprechende Einrichtungen sind beispielsweise für die Bedienung von Automaten oder von Telefon-Freisprecheinrichtungen in Kraftfahrzeugen bekannt. Auch wenn das zur Bedienung der Geräte bereitgestellte Vokabular häufig eingeschränkt ist, kommt jedoch diese Art der Bedienung dem intuitiven menschlichen Handeln besser entgegen. Neben der Möglichkeit, Geräte sprachgeführt zu bedienen, ist es darüber hinaus bekannt geworden, eine Bedienung elektronischer Geräte durch die Erfassung menschlicher auf die Fernbedienung wirkender Bewegungen zu realisieren. Im Allgemeinen sind jedoch Vorrichtungen und Konzepte zur Fernbedienung von elektronischen Geräten monomodal ausgelegt. Das heißt, die Bedienung ist in vielen Fällen nur nach einer der vorgenannten Möglichkeiten, also mittels Tastatur oder Sprache oder Bewegung möglich. Teil intuitiver menschlicher Handlungsweise ist es jedoch, sich in der Kommunikation entweder entsprechend der jeweiligen Situation oder häufig sogar gleichzeitig mehrerer Kommunikationskanäle zu bedienen. Vor diesem Hintergrund ist es ein Ziel, die Bedienung von Geräten mittels Fernbedienungen noch intuitiver zu gestalten.Therefore, it is already known to operate electronic devices voice-controlled. Corresponding devices are for example for the operation of machines or telephone hands-free devices in motor vehicles known. Although the vocabulary provided for operating the devices is often limited, this type of operation better accommodates intuitive human action. In addition to the ability to operate devices voice-guided, it has also become known to realize an operation of electronic devices by detecting human acting on the remote control movements. In general, however, devices and concepts for remote control of electronic devices are designed monomodally. That is, the operation is possible in many cases only one of the above possibilities, ie by keyboard or voice or movement. However, part of an intuitive human mode of action is to use communication either according to the particular situation or frequently even simultaneously to multiple communication channels. Against this background, it is a goal to make the operation of devices using remote controls even more intuitive.

Durch die US 2006/0077174 A1 ist in diesem Zusammenhang eine Fernbedienungseinrichtung bekannt geworden, welche mindestens bimodal arbeitet. Die Fernbedienungseinrichtung umfasst hierzu innerhalb eines Gerätes ein mindestens bimodales Eingabemodul, ein Modul mit einer zugehörigen Datenbank zur Interpretation von beispielsweise mittels Sprache und durch Bewegungen der Fernbedienungseinrichtung erfolgenden Eingaben und ein Modul zur Generierung von Steuersignalen auf der Grundalge der im Rahmen der Interpretation erkannten Kommandos. Insbesondere auch im Falle der Nutzung einer sprachgesteuerten Bedienung erfolgt dabei die gesamte Spracherkennung und Interpretation innerhalb der Fernbedienungseinrichtung, welche hierzu für diesen Eingabemodus und zumindest einen weiteren Eingabemodus vorzugsweise entsprechende Kommandospeicher enthält. Aufgrund der Ableitung der das Gerät bedienenden Steuersignale unmittelbar in der Fernbedienungseinrichtung ist jedoch deren Flexibilität im Hinblick auf die Anpassung an unterschiedliche Gerätearten und Gerätetypen vergleichsweise eingeschränkt. Zudem kann die Kommunikation mit dem jeweils zu bedienenden Gerät, trotzdem gemäß einer Ausgestaltung der in der Schrift beschriebenen Lösung ein Rückkanal zwischen dem Gerät und der Fernbedienungseinrichtung vorgesehen ist, kaum kontextsensitiv gestaltet werden. Das heißt, es ist nur in beschränktem Maße möglich, die jeweils bereitgestellten Bedienfunktionen an den Kontext beziehungsweise an das aktuelle Geschehen in Verbindung mit einem von dem Gerät jeweils wiedergegebenen Inhalt anzupassen. Insoweit ist eine weitgehend intuitive Bedienung im Hinblick auf standardisierte und unveränderliche, das heißt statische Gerätefunktionen möglich, die Bereitstellung von Bedienungsmöglichkeiten im Hinblick auf sich dynamisch ändernde Zustände des zu bedienenden Geräts oder auf sich dynamisch ändernde, mit dem Gerät konsumierte mediale Inhalte aber nicht möglich.By the US 2006/0077174 A1 In this context, a remote control device has become known, which works at least bimodal. For this purpose, the remote control device comprises, within a device, an at least bimodal input module, a module with an associated database for interpreting, for example, by means of speech and by movements of the remote control device inputs and a module for generating control signals on the basis of recognized during the interpretation commands. In particular, even in the case of the use of a voice-controlled operation, the entire speech recognition and interpretation takes place within the remote control device, which preferably contains corresponding command memory for this input mode and at least one further input mode. Due to the derivation of the device operating control signals directly in the remote control device but their flexibility in terms of adaptation to different types of devices and device types is relatively limited. In addition, the communication with the respective device to be operated, nevertheless according to an embodiment of the solution described in the document, a return channel between the device and the remote control device is intended to be designed to be hardly context-sensitive. This means that it is only possible to a limited degree to adapt the respectively provided operating functions to the context or to the current events in connection with a content respectively reproduced by the device. In that regard, a largely intuitive operation in terms of standardized and unchanging, that is static device functions possible, the provision of operating options with respect to dynamically changing states of the device to be operated or on dynamically changing, consumed with the device media content but not possible.

Aus der EP 1 079 371 A1 ist eine ebenfalls bimodal arbeitende Anordnung mit einer Fernbedienungseinrichtung zur drahtlosen Fernbedienung eines Geräts, vorzugsweise eines Fernsehgerätes bekannt, wobei ein bidirektionaler Datenaustausch zwischen der Fernbedienungseinrichtung und dem mit dieser bedienten Gerät erfolgt. Dabei ermöglicht es die Anordnung, auf der Grundlage eines von dem zu bedienenden Gerät zur Verfügung gestellten elektronischen Programmguides ein Programm mittels der Fernbedieneinrichtung und eines in dieser angeordneten Mikrophons sprachgesteuert in einer weitgehend natürlicher Sprechweise auszuwählen, wobei ein entsprechender Programmkanal gegebenenfalls auch mittels einer Kanalwahltaste der Fernbedienungseinrichtung ausgewählt werden. Eine umfassendere, über die Kanalwahl hinausgehende Bedienung mittels der Tastatur ist dabei nicht vorgesehen. Gemäß einer in der Schrift beschriebenen Ausbildungsform kann zumindest ein Teil der für die Analyse beziehungsweise Interpretation der Spracheingaben erforderlichen Einheiten, insbesondere dann, wenn diese aufgrund ihrer Komplexität etwas mehr Bauraum benötigen in dem zu bedienenden Gerät selbst angeordnet sein. Als nachteilig ist es bei dieser Lösung, ebenso wie bei der in der US 2006/0077174 A1 beschriebenen, anzusehen, dass Spracheingaben zur Bedienung medialer Geräte, sofern sie sprecherunabhängig möglich sein sollen, möglicherweise nicht immer sicher erkannt beziehungsweise interpretiert werden. Schwierigkeiten könnten dabei dann entstehen, wenn die Bedienung mediale Geräte betrifft, über welche während der Bedienung Audiosignale, insbesondere auch in Form sprachbasierten Contents, ausgegeben werden.From the EP 1 079 371 A1 is also a bimodal working arrangement with a remote control device for wireless remote control of a device, preferably a television set known, wherein a bidirectional data exchange between the remote control device and the device operated with this takes place. In this case, the arrangement makes it possible, on the basis of an electronic program guide provided by the device to be operated, to select a program by means of the remote control device and a microphone arranged in this voice-controlled manner in a largely natural manner, with a corresponding program channel optionally also by means of a channel selection button of the remote control device to be selected. A more comprehensive, beyond the channel selection operation by means of the keyboard is not provided. According to an embodiment described in the document, at least some of the units required for the analysis or interpretation of the speech inputs, in particular if they require a little more installation space due to their complexity, can be arranged in the device to be operated itself. It is disadvantageous in this solution, as well as in the US 2006/0077174 A1 described that voice inputs for the operation of media devices, if they are to be possible speaker independent, may not always be reliably detected or interpreted. Difficulties could arise when the operation mediale Applies to devices over which audio signals, in particular in the form of speech-based content, are output during the operation.

Die Erfindung ist gemäß Anspruch 1 definiert. Ausführungsformen sind in abhängigen Ansprüchen definiert. Aufgabe der Erfindung ist es, eine alternative Anordnung bereitzustellen, welche eine besonders intuitive und flexible Fernbedienung von Gerätefunktionen eines elektronischen Geräts unter umfassender Nutzung natürlicher menschlicher Kommunikationswege, wie der Sprache, ermöglicht, wobei insbesondere eine sichere Erkennung von sprachlichen Bedieneingaben gewährleistet sein soll.The invention is defined according to claim 1. Embodiments are defined in dependent claims. The object of the invention is to provide an alternative arrangement, which allows a particularly intuitive and flexible remote control of device functions of an electronic device with extensive use of natural human communication channels, such as the language, in particular, a secure detection of linguistic operator input should be guaranteed.

Die Aufgabe wird durch eine Anordnung mit den Merkmalen des Hautpanspruchs gelöst. Vorteilhafte Aus- beziehungsweise Weiterbildungen der Erfindung sind durch die Unteransprüche gegeben.The object is achieved by an arrangement with the features of Hautpanlaim. Advantageous embodiments or further developments of the invention are given by the subclaims.

Die zur Lösung der Aufgabe vorgeschlagene Anordnung besteht aus einem fernzubedienenden Gerät und einer Fernbedienungseinrichtung zur multimodalen, drahtlosen Fernbedienung des vorgenannten Gerätes. Dabei weist die Anordnung zumindest Mittel zur drahtlosen Kommunikation zwischen der Fernbedienungseinrichtung und dem mit ihr zu bedienenden Gerät, an oder in der Fernbedienungsanrichtung angeordnete Eingabemittel zur Entgegennahme von Eingaben in mehreren verschiedenen Eingabemodi, mindestens eine Digitalisierungseinheit zur Digitalisierung entgegengenommener Eingaben, mindestens eine Interpretationseinheit zur Interpretation der in digitale Daten umgewandelten Eingaben sowie mindestens eine Steuereinheit zur Ableitung von Steuersignalen aus im Zuge der Interpretation von Eingaben erkannten Bedienkommandos und zur Ansteuerung entsprechender Aktoren des bedienten Gerätes auf. Mit Hilfe der letztgenannten, in dem fernzubedienenden Gerät angeordneten Aktoren werden die mit den jeweiligen Eingaben korrespondierenden Bedienkommandos nach Maßgabe der aus ihnen mittels der mindestens einen Interpretationseinheit erhaltenen Bedienkommandos und der aus diesen mittels der mindestens einen Steuereinheit abgeleiteten Steuersignale in dem Gerät umgesetzt. Bei den an oder in der Fernbedienungseinrichtung zu Realisierung der verschiedenen Eingabemodi der Anordnung angeordneten Eingabemitteln handelt es sich um mindestens ein Mikrofon zur Entgegennahme von Spracheingaben, einen Bewegungssensor zur Entgegennahme durch eine Bewegung der Fernbedienungseinrichtung erfolgender Eingaben und um eine Tastatur. Bei den eingangs genannten Mitteln zur drahtlosen Kommunikation kann es sich insbesondere um Infrarotsender und Empfänger oder um Sender und Empfänger für eine funkgestützte Übertragung von Daten, beispielsweise nach dem Bluetooth-Standard, handeln.The proposed arrangement for solving the problem consists of a fernbedienenden device and a remote control device for multimodal, wireless remote control of the aforementioned device. In this case, the arrangement has at least means for wireless communication between the remote control device and the device to be operated with it, on or in the remote control device arranged input means for receiving inputs in several different input modes, at least one digitizing unit for digitizing received input, at least one interpretation unit for interpreting the in digital data converted inputs and at least one control unit for deriving control signals from in the course of the interpretation of inputs detected operating commands and to control corresponding actuators of the operated device. With the aid of the last-mentioned actuators arranged in the remote-controlled device, the operating commands corresponding to the respective inputs are converted in the device in accordance with the operating commands received from them by means of the at least one interpretation unit and the control signals derived therefrom by means of the at least one control unit. The input means arranged on or in the remote control device for realizing the various input modes of the arrangement is at least one microphone for receiving from voice inputs, a motion sensor for receiving input from a movement of the remote control device, and a keyboard. The above-mentioned means for wireless communication may be, in particular, infrared transmitters and receivers or transmitters and receivers for radio-supported transmission of data, for example according to the Bluetooth standard.

Die mindestens eine Interpretationseinheit zur Interpretation der durch die Digitalisierungseinheit in digitale Daten umgewandelten Eingaben ist in dem zu bedienenden Gerät selbst angeordnet. Die durch die Betätigung de Eingabemittel durch einen Nutzer ausgeführten Eingaben, wie Sprach- oder Bewegungseingaben, werden dabei in der Fernbedienungseinrichtung mittels der mindestens einen in ihr angeordneten Digitalisierungseinheit lediglich digitalisiert und die dabei erhaltenen digitalen Daten durch einen Sender der Fernbedienungseinrichtung als digitaler Datenstrom an einen Empfänger des fernzubedienenden Geräts übertragen. Erst hier erfolgt dann die Interpretation der Eingaben, das heißt ihre Umsetzung in zunächst Bedienkommandos und danach die Ableitung der Steuersignale aus den Bedienkommandos, mit welchen die entsprechenden Aktoren in dem zu bedienenden Gerät angesteuert werden. Dazu weist die Interpretationseinheit eine Funktionsgruppe zur Interpretation von Tastatureingaben, eine Funktionsgruppe zur Interpretation von Bewegungseingaben sowie eine Funktionsgruppe zur Interpretation von Spracheingaben mit einem Spracherkenner und mit mindestens einer, einen Sprachraum für die Spracherkennung festlegenden Grammatik auf.The at least one interpretation unit for interpreting the inputs converted into digital data by the digitizing unit is arranged in the device to be operated itself. The inputs, such as voice or motion inputs, which are carried out by the user via input means, are merely digitized in the remote control device by means of the at least one digitizing unit arranged therein and the digital data obtained by a transmitter of the remote control device as a digital data stream to a receiver of the remote device. Only then does the interpretation of the inputs then take place, ie their conversion into operating commands first and then the derivation of the control signals from the operating commands with which the corresponding actuators in the device to be operated are activated. For this purpose, the interpretation unit has a function group for interpreting keyboard inputs, a function group for interpreting movement inputs, and a function group for interpreting voice inputs with a speech recognizer and with at least one grammar defining a speech space for speech recognition.

Bei den vorgenannten Aktoren handelt es sich beispielsweise um Funktionsgruppen zur Einstellung der Lautstärke oder von Parametern eines Bildschirms, wie beispielsweise Kontrast oder Farbsättigung, oder aber auch um Funktionsgruppen zur Umschaltung eines wiedergegebenen Programms.The aforementioned actuators are, for example, functional groups for adjusting the volume or parameters of a screen, such as contrast or color saturation, or else functional groups for switching over a reproduced program.

Im Sinne der Erfindung kann es sich bei dem fernzubedienenden Gerät beispielsweise um ein Fernsehgerät, eine Stereoanlage, einen Sat-Receiver, eine Set-Top-Box oder einen DVD-Player handeln. In jedem Falle ist das entsprechende Gerät gemäß der Erfindung mit einer Interpretationseinheit ausgestattet, mittels welcher ein gesprochene Worte repräsentierender Datenstrom oder ein Bewegungen der Fernbedienungseinrichtung repräsentierender Datenstrom interpretiert werden und soweit möglich ein mit dem Datenstrom korrespondierendes Bedienkommando abgeleitet wird. Das gegebenenfalls erkannte Bedienkommando wird von der Interpretationseinheit an die mindestens eine, ebenfalls im zu bedienenden Gerät angeordnete Steuereinheit zur Ableitung der erforderlichen Steuersignale übergeben. Dadurch, dass die Interpretationseinheit in dem zu bedienenden Gerät angeordnet ist, ist eine hohe Flexibilität im Zusammenspiel zwischen der Fernbedienungseinrichtung und dem fernzubedienenden Gerät gegeben. Je nach Geräteart und Gerätetyp können dabei Sprach- oder Bewegungseingaben durch das jeweilige Gerät völlig unterschiedlich interpretiert werden. Hierdurch wird die Möglichkeit einer intuitiven Bedienung des zu bedienenden Geräts und der von ihm jeweils zur Verfügung gestellten Funktionen deutlich verbessert. Zudem sind hierdurch erweiterte Möglichkeiten für das Update der aus der Fernbedienungseinrichtung und dem fernzubedienenden Gerät bestehenden Anordnung gegeben. So ist es heute bereits bekannt, Soft- beziehungsweise Firmware-Updates für Receiver oder Set-Top-Boxen über Satellit oder Kabel an die Geräte zu übermitteln. Durch die Erfindung ist die Möglichkeit eröffnet, die entsprechenden Update-Funktionen auch auf das Zusammenspiel zwischen Fernbedienungseinrichtung und Gerät anzuwenden. Sofern im Rahmen eines Updates an dem Gerät neue Funktionen zur Verfügung gestellt werden, kann in diesem Zusammenhang auch ein Update der Interpretationseinheit erfolgen, so dass die entsprechenden Funktionen unmittelbar und auf gewohnt intuitive Weise auch über die Fernbedienungseinrichtung ansprechbar sind.For the purposes of the invention, the device to be remotely operated can be, for example, a television set, a stereo system, a satellite receiver, a set-top box or a DVD player. In any case, the corresponding device according to the invention is equipped with a unit of interpretation, by means of which a spoken word representing Data stream or a movement of the remote control device representative data stream are interpreted and as far as possible, a corresponding with the data stream operator command is derived. The optionally recognized operating command is transferred by the interpretation unit to the at least one control unit likewise arranged in the device to be operated for deriving the required control signals. The fact that the interpretation unit is arranged in the device to be operated, there is a high flexibility in the interaction between the remote control device and the device to be operated remotely. Depending on the type of device and the type of device, speech or motion input can be interpreted completely differently by the respective device. As a result, the possibility of intuitive operation of the device to be operated and the functions provided by him is significantly improved. In addition, this extended possibilities for updating existing from the remote control device and the remote device arrangement are given. So it is already known today to transmit software or firmware updates for receivers or set-top boxes via satellite or cable to the devices. The invention opens up the possibility of applying the corresponding update functions also to the interaction between the remote control device and the device. If new functions are made available as part of an update to the device, an update of the interpretation unit can be made in this context, so that the corresponding functions can be addressed directly and in the usual intuitive manner via the remote control device.

Bei der mindestens einen Interpretationseinheit kann es sich um eine für alle Bedienmodi gemeinsame Interpretationseinheit oder aber um jeweils eine oder mehrere Einheiten handeln. Die Anordnung ist ferner so ausgebildet, dass die an oder in der Fernbedienungseinrichtung angeordneten Eingabemittel zur Entgegennahme von Spracheingeben mittels einer dazu vorgesehenen, an der Fernbedienungseinrichtung angeordneten Taste aktivierbar sind. Die entsprechenden Eingabemittel sind dabei je nach Ausgestaltung für die Dauer der Betätigung der vorgenannten Taste (im Weiteren auch Push-To-Talk-Taste) oder bis zu deren erneuter Betätigung aktiviert. Erfindungsgemäß ist weiterhin vorgesehen, dass während der Betätigung der Push-to-talk-Taste ein Muting der von dem bedienten Gerät gegebenenfalls gerade wiedergegebenen akustischen Signale erfolgt. Um die Erkennungssicherheit der Spracheingabe zu erhöhen, wird demnach zum Beispiel bei einem Fernsehgerät oder einer Set-Top-Box für die Dauer der Betätigung der Push-to-Talk-Taste und damit für die Dauer der Aktivierung der Spracheingabemittel die Lautstärke der Tonwiedergabe reduziert.The at least one interpretation unit may be a common interpretation unit for all operating modes or one or more units. The arrangement is further configured such that the input means arranged on or in the remote control device can be activated to receive voice input by means of a button provided for this purpose and arranged on the remote control device. The corresponding input means are depending on the configuration for the duration of the actuation of the aforementioned button (hereinafter also push-to-talk button) or until activated again. According to the invention, it is further provided that, during the actuation of the push-to-talk key, a muting of the acoustic signals which may possibly be reproduced by the operated device takes place. In order to increase the recognition security of voice input, therefore, for example, in a TV set or a set-top box for the duration of the push-to-talk button and thus for the duration of activation of the voice input means the volume of the sound reproduction is reduced.

Eine weitergehende Verbesserung der Zuverlässigkeit bei der Erkennung von Spracheingaben wird durch eine spezielle Weiterbildung der mit einer Spracheingabeeinheit ausgestatteten erfindungsgemäßen Anordnung erreicht. Bei dieser ist ein zweistufiges Bedienkonzept für die Spracheingabe vorgesehen. Danach wird zwischen einem Sprachbedienmodus zur Steuerung von Basisfunktionen mit einem statischen Befehlssatz und einem Sprachbedienmodus mit einem sich dynamisch ändernden Befehlssatz zur kontextbezogenen Bedienung des fernzubedienenden Geräts unterschieden. Der erste Modus zur Nutzung von Basisfunktionen, wie beispielsweise das An- und Ausschalten des Geräts oder die Lautstärkeeinstellung, ist dabei ständig oder, sofern an der Fernbedienungseinrichtung eine Push-To-Talk-Taste vorgesehen ist, jeweils beim Betätigen der Push-to-talk-Taste aktiv. Hingegen erfordert die Aktivierung des erweiterten beziehungsweise dynamischen Bedienmodus eine zusätzliche Bedienung, durch etwa das Betätigen einer weiteren Taste oder ein spezielles Sprachkommando (zum Beispiel Sprachkommando "erweiterter Modus"). Im erweiterten Bedienmodus ist es dabei möglich, sich in kürzeren Abständen ändernde und kontextabhängige Bedienfunktionen zu nutzen. Ein Beispiel hierfür sind Titel von Medieninhalten, die per Sprache ausgewählt werden können und die sich in Abhängigkeit des jeweils angebotenen Programms ständig ändern. Hierfür sind in dem zu bedienenden Gerät als Bestandteil der mindestens einen Interpretationseinheit zur Interpretation von Spracheingaben vorzugsweise zwei verschiedenen Sprachgrammatiken vorgesehenen. Dabei ist der mittels einer ersten, der Interpretation statischer Bedienfunktionen dienenden Grammatik abgedeckte Sprachraum (Wortschatz und Phrasen) statisch, während die mit Hilfe der zweiten Grammatik interpretierbaren Spracheingaben sich dynamisch an den jeweiligen Kontext anpassen und beispielsweise mit Hilfe des Electronic-Program-Guide jeweils entsprechend modifiziert werden. Vorzugsweise ist die Erfindung dabei so ausgestaltet, dass nach der Aktivierung des erweiterten Bedienmodus und der Ausführung einer mittels dieses Modus möglichen Bedienfunktion die Anordnung unmittelbar wieder in den ersten statischen Bedienmodus zurückschaltet beziehungsweise den erweiterten Bedienmodus deaktiviert.A further improvement of the reliability in the recognition of voice input is achieved by a special development of the inventive arrangement equipped with a voice input unit. In this, a two-stage operating concept for voice input is provided. Thereafter, distinction is made between a speech operation mode for controlling basic functions with a static instruction set and a speech operation mode with a dynamically changing instruction set for context-related operation of the device to be remotely operated. The first mode for the use of basic functions, such as turning on and off the device or the volume setting, is constantly or, if provided on the remote control device a push-to-talk button, each time you press the push to talk Button active. By contrast, the activation of the extended or dynamic operating mode requires additional operation, for example by pressing another key or a special voice command (for example, voice command "extended mode"). In the extended operating mode, it is possible to use changing and context-dependent operating functions at shorter intervals. An example of this are titles of media content that can be selected by language and that constantly change depending on the program offered. For this purpose, preferably two different speech grammars are provided in the device to be operated as part of the at least one interpretation unit for the interpretation of speech inputs. In this case, the language space covered by a first grammar serving the interpretation of static operating functions (vocabulary and phrases) are static, while the interpretable with the help of the second grammar language inputs are dynamically adapt to the context and, for example, modified accordingly with the help of the Electronic Program Guide. Preferably, the invention is designed such that after activation of the extended operating mode and the execution of an operating function possible by means of this mode, the arrangement immediately switches back to the first static operating mode or deactivates the extended operating mode.

Eine weitere besonders vorteilhafte Ausbildungsform der Erfindung ist so gestaltet, dass sie ein so genanntes Voice-Zapping ermöglicht. Hierbei ist es dem Bediener möglich, durch sprachliche Angabe eines bestimmten Genres, selektiv ausgewählte Programminhalte anzusprechen und das bediente Gerät zum Umschalten auf einen solchen Programminhalt zu veranlassen. Beispielsweise könnte das Sprachkommando "Sport" einen erfindungsgemäß gestalteten Sat-Receiver dazu veranlassen, auf ein Sportprogramm umzuschalten. Denkbar ist es ferner, dass ein Nutzer der erfindungsgemäßen Anordnung im Fernsehen eine bestimmte, ihm vom Titel her bekannte Sendung sehen möchte, ihm aber nicht unmittelbar bekannt ist, von welchem Sender diese ausgestrahlt wird. In diesem Falle ist gemäß der zuvor beschriebenen Ausbildungsform beispielsweise die Möglichkeit gegeben, dass er mittels Sprachbedienung durch Nennung des Titels der Sendung auf den entsprechenden Sender beziehungsweise das entsprechende Programm umschaltet, wobei das Umschalten auf den betreffenden Sender ermöglicht ist, sofern die gewünschte Sendung gerade läuft und, gegebenenfalls darüber hinaus, innerhalb einer in der Interpretationseinheit im Wege eines Update hinterlegten Zeitspanne vor dem Beginn der Sendung. Anhand dieser Beispiele wird der Vorteil einer gemäß der Erfindung gegebenen Anordnung der Interpretationseinheit oder -einheiten in dem zu bedienenden Gerät und der damit verbundenen höheren Bedienungsflexibilität sehr deutlich. Eine kontextbezogene beziehungsweise kontextsensitive Bedienung im Sinne des zuvor Erläuterten ist erst durch die erfindungsgemäße Anordnung der mindestens einen Interpretationseinheit in dem zu bedienenden beziehungsweise in dem bedienten Gerät möglich.Another particularly advantageous embodiment of the invention is designed so that it allows a so-called voice-zapping. In this case, it is possible for the operator to address selectively selected program contents by linguistically indicating a particular genre and to cause the operated device to switch over to such a program content. For example, the voice command "Sport" could cause an inventively designed satellite receiver to switch to a sports program. It is also conceivable that a user of the arrangement according to the invention would like to see on television a certain program known to him from the title, but he is not immediately aware of which station this is being broadcast. In this case, according to the embodiment described above, for example, given the possibility that he switches by voice operation by naming the title of the program on the corresponding transmitter or the corresponding program, the switch is made possible on the transmitter concerned, if the desired program is currently running and, if appropriate, within a period of time stored in the interpretation unit by means of an update before the start of the program. Based on these examples, the advantage of a given arrangement of the interpretation unit or units in the device to be operated and the associated higher operating flexibility is very clear. A context-related or context-sensitive operation in the sense of the previously explained is only possible by the inventive arrangement of the at least one interpretation unit in the device to be operated or in the device served.

Eine weitere vorteilhafte Ausgestaltung der Erfindung ist dadurch gegeben, dass der Eingabemodus zur Bedienung durch eine Bewegung der Fernbedienungseinrichtung nur beim und für die Dauer des Drückens einer dafür vorgesehenen Taste (im Weiteren auch Motion-Control) oder bis zum nochmaligen Betätigen der Motion-Control-Taste aktiviert ist. Eine solche, die Bedienung des zugehörigen Geräts durch die Bewegung der Fernbedienungseinrichtung ermöglichende Anordnung kann zudem derart ausgestaltet sein, dass sie ein so genanntes "EasyOn/EasyOff" ermöglicht. Dabei wird beispielsweise das zu bedienende Gerät dadurch ausgeschaltet, dass eine als Oberseite erkennbare Seite der Fernbedienungseinrichtung nach unten gedreht beziehungsweise die Fernbedienungseinrichtung mit dieser Seite nach unten auf den Tisch gelegt wird. Das Drehen der Fernbedienungseinrichtung auf ihre Rückseite bewirkt dementsprechend das Einschalten des zu bedienenden Geräts.A further advantageous embodiment of the invention is given by the fact that the input mode for operation by a movement of the remote control device only during and for the duration of pressing a designated button (hereinafter also motion control) or until the second operation of the motion control Button is activated. Such, the operation of the associated device by the movement of the remote control device enabling arrangement can also be designed such that it allows a so-called "EasyOn / EasyOff". In this case, for example, the device to be operated is switched off by turning a side of the remote control device which is recognizable as the top side downwards or by placing the remote control device on the table with this side downwards. The turning of the remote control device on its back accordingly causes the switching of the device to be operated.

Entsprechend einer weiteren vorteilhaften Ausgestaltung ist die Interaktion zwischen der Fernbedienungseinrichtung und dem zu bedienenden Gerät derart gestaltet, dass stufenlos einstellbare Gerätefunktionen, wie beispielsweise die Lautstärkeeinstellung für die Tonwiedergabe oder die Kontrasteinstellung für die Bildwiedergabe, durch eine Drehbewegung der Fernbedienungseinrichtung gesteuert werden können.According to a further advantageous embodiment, the interaction between the remote control device and the device to be operated is designed such that continuously adjustable device functions, such as the volume setting for the sound reproduction or the contrast adjustment for the image reproduction, can be controlled by a rotary movement of the remote control device.

Die Fernbedienungseinrichtung der erfindungsgemäßen Anordnung ist zudem vorzugsweise mit einem Display ausgestattet. Hier können beispielsweise aktive Betriebsmodi der Fernbedienungseinrichtung angezeigt oder aber auch visuelle Feedbacks, mit welchen das fernbediente Gerät die Betätigung ausgewählter Funktionen quittiert, wiedergegeben werden. Die Fernbedienungseinrichtung kann zudem mit Mitteln für ein akustisches Feedback (Lautsprecher) oder ein haptisches Feedback (Vibrationseinheit) ausgestattet sein.The remote control device of the arrangement according to the invention is also preferably equipped with a display. Here, for example, active operating modes of the remote control device can be displayed or else visual feedbacks with which the remote-controlled device acknowledges the actuation of selected functions can be reproduced. The remote control device can also be equipped with means for acoustic feedback (loudspeaker) or haptic feedback (vibration unit).

Anhand eines Ausführungsbeispiels sollen nachfolgend nochmals einige Aspekte und Details der Erfindung erläutert werden. In den zugehörigen Zeichnungen zeigen:

Fig. 1:
ein grobes Blockschaltbild beziehungsweise Schema der erfindungsgemäßen Anordnung,
Fig. 2:
die Fernbedienungseinrichtung gemäß Fig. 1 in einem Modus für Bewegungseingaben.
With reference to an embodiment, some aspects and details of the invention will be explained again below. In the accompanying drawings show:
Fig. 1:
a rough block diagram or scheme of the arrangement according to the invention,
Fig. 2:
the remote control device according to Fig. 1 in a mode for motion input.

Die Fig. 1 zeigt ein grobes Schema beziehungsweise Blockschaltbild einer möglichen Ausbildungsform der erfindungsgemäßen Anordnung. Die Anordnung besteht aus der Fernbedienungseinrichtung 1 und dem mit ihr fernzubedienenden Gerät 2, beispielsweise einer Set-Top-Box. Als wesentliche Einheiten weist die Anordnung Mittel zur drahtlosen Kommunikation 6, 7 zwischen der Fernbedienungseinrichtung 1 und dem zu bedienendem Gerät 2, nämlich einen Infrarotsender 6 an der Fernbedienungseinrichtung 1 und einen Infrarotempfänger 7 an dem Gerät 2, eine in der Fernbedienungseinrichtung 1 angeordnete Digitalisierungseinheit 11, eine gemäß der Erfindung in dem zu bedienenden Gerät 2 angeordnete Interpretationseinheit 3 zur Interpretation von Eingaben unterschiedlicher Eingabemodi, eine Steuereinheit 4 und mehrere von der Steuereinheit mittels entsprechender Steuersignale betätigte Aktoren 5 auf. Die in dem Beispiel gezeigte Anordnung verfügt über drei Eingabemodi. Diese ermöglichen es, Funktionen des Geräts 2 mittels der Fernbedienungseinrichtung durch Tastatureingaben, Spracheingaben und Bewegungseingaben zu steuern. Hierfür sind in beziehungsweise an der Fernbedienungseinrichtung 1 entsprechende Eingabemittel 8, 9, 10 vorgesehen. Die im Beispiel gezeigte Fernbedienungseinrichtung 1 verfügt über eine Tastatur 10 zur Entgegennahme von Tastatureingaben, ein Mikrofon 8 zur Entgegennahme von Spracheingaben und einen in der Fernbedienungseinrichtung 1 angeordneten Beschleunigungssensor 9 zur Entgegennahme durch Bewegungen der Fernbedienungseinrichtung 1 getätigter Bewegungseingaben. Die durch Betätigung beziehungsweise Nutzung der Eingabemittel 8, 9, 10 von einem Nutzer der Anordnung ausgeführten Eingaben werden nach der Entgegennahme durch die Fernbedienungseinrichtung 1 in der Digitalisierungseinrichtung 11 in einen digitalen, über den Infrarotsender 6 an der Fernbedienungseinrichtung 1 in Richtung des zu bedienenden Geräts 2 abgestrahlten Datenstrom umgewandelt. Der entsprechende Datenstrom wird von dem Infrarotempfänger 7 der Set-Top-Box 2 empfangen. In einer zu der Interpretationseinheit 3 gehörenden Funktionsgruppe 21, einem Signalkonverter und -verzweiger, werden die empfangenen digitalen Daten entsprechend ihrer Herkunft von den Spracheingabemitteln 8, den Eingabemitteln 8 für Bewegungseingaben und der Tastatur 10 getrennt und zur eigentlichen Interpretation unterschiedlichen Funktionsgruppen 12, 16, 17 der Interpretationseinheit 3 zugeführt. In einer ersten Funktionsgruppe 16 erfolgt die Interpretation von Bewegungseingaben, das heißt durch horizontale oder vertikale Bewegung oder Drehung der Fernbedienungseinrichtung 1 getätigter Eingaben. Die Interpretation erfolgt in der Weise, dass die aufgrund von Bewegungen der Fernbedienungseinrichtung 1 eingegangenen Signale in der entsprechenden Funktionsgruppe 16 analysiert und mit ihnen korrespondierende Bedienkommandos ermittelt werden, wobei sie auf Übereinstimmung mit möglichen, in einem Kommandospeicher 18 gehaltenen Bedienkommandos geprüft werden. Mittels des Mikrofons 8 der Fernbedienungseinrichtung 1 entgegengenommene Bedieneingaben werden nach der Digitalisierung und der Übermittlung an den Empfänger 7 des zu bedienenden Geräts 2 durch den Signalkonverter beziehungsweise Signalverzweiger 21 einer Funktionsgruppe 12 zur Interpretation von Sprachsignalen zugeführt. Hier werden sie mittels eines Spracherkenners 15 (ASR - Automatic Speech Recognition) auf der Grundlage zweier verschiedener in der Funktionsgruppe gehaltener Grammatiken 13, 14 analysiert und aus ihnen, sofern möglich, ein Bedienkommando nach den Vorgaben des Kommandospeichers 18 abgeleitet. Bei den beiden zuvor erwähnten Grammatiken 13, 14 handelt es sich um eine erste Grammatik 13, welche ein Vokabular zur Aktivierung statischer Bedienkommandos enthält. Darüber hinaus ist eine zweite Grammatik 14 vorgesehen, deren bei der Sprachanalyse auf die aus der Spracheingabe resultierenden Daten angewandtes Vokabular kontextabhängig und somit dynamisch ist. Die Anpassung des Vokabulars an den Kontext, das heißt beispielsweise an die zu einem bestimmten Zeitpunkt mittels der Set-Top-Box empfangbaren Programme, geschieht mit Hilfe des EPG 22 (Electronic-Program-Guide). Der EPG 22 bezieht seine Daten beispielsweise von dem Backend, das heißt einem Internetserver 23, eines den Inhalt bereitstellenden Anbieters über das Internet 24 und/oder durch den Empfang entsprechender Daten über eine Satellitenanlage. In einer dritten Funktionsgruppe 17 erfolgt die Interpretation von Tastatureingaben, indem diese auf gemäß den Vorgaben des Kommandospeichers 18 mögliche Bedienkommandos abgebildet werden. Die bei Nutzung der einzelnen Eingabemodi ermittelten Bedienkommandos werden der Steuereinheit 4 zugeführt, welche aus ihnen Steuersignale ableitet und mit diesen Steuersignalen Aktoren 5 des Geräts, im Beispiel eine Funktionsgruppe 5' zur Lautstärkereglung, eine Funktionsgruppe zur Programmauswahl 5" und/oder eine Funktionsgruppe 5'" zur Einstellung von Bildschirmparametern, wie Helligkeit und/oder Kontrast, ansteuert. Ein entscheidender Aspekt der Erfindung ist die Tatsache, dass die Art und Weise, in welcher in den einzelnen Eingabemodi getätigte Bedieneingaben interpretiert und in Steuersignale für das zu bedienende Gerät umgesetzt werden, durch das jeweils bediente Gerät 2 selbst, nämlich die mindestens eine in diesem angeordnete Interpretationseinheit 3, bestimmt wird. Dabei kann die Firmware der entsprechenden Interpretationseinheit 3 im Wege des Updates, beispielsweise durch Übertragung über den Satellitenkanal, geändert werden und somit die Funktionalität des Gerätes 2 an unterschiedliche Gegebenheiten angepasst werden.The Fig. 1 shows a rough diagram or block diagram of a possible embodiment of the arrangement according to the invention. The arrangement consists of the remote control device 1 and the device 2 to be remotely operated with it, for example a set-top box. As essential units, the arrangement comprises means for wireless communication 6, 7 between the remote control device 1 and the device 2 to be operated, namely an infrared transmitter 6 on the remote control device 1 and an infrared receiver 7 on the device 2, arranged in the remote control device 1 digitizing unit 11, a arranged according to the invention in the device to be operated 2 interpretation unit 3 for the interpretation of inputs of different input modes, a control unit 4 and a plurality of actuated by the control unit by means of appropriate control signals actuators 5. The arrangement shown in the example has three input modes. These make it possible to control functions of the device 2 by means of the remote control device by means of keystrokes, voice inputs and movement inputs. For this purpose, corresponding input means 8, 9, 10 are provided in or on the remote control device 1. The remote control device 1 shown in the example has a keyboard 10 for receiving keyboard input, a microphone 8 for receiving voice input and an arranged in the remote control device 1 acceleration sensor 9 for receiving movements of the remote control device 1 movement inputs. The inputs made by a user of the arrangement by operation or use of the input means 8, 9, 10 become a digital input via the infrared transmitter 6 on the remote control device 1 in the direction of the device 2 to be operated after being received by the remote control device 1 in the digitizer 11 radiated data stream converted. The corresponding data stream is received by the infrared receiver 7 of the set-top box 2. In a function group 21 belonging to the interpretation unit 3, a signal converter and distributor, the received digital data are separated according to their origin from the speech input means 8, the input means 8 for movement inputs and the keyboard 10 and to the actual interpretation of different function groups 12, 16, 17 supplied to the interpretation unit 3. In a first function group 16, the interpretation of movement inputs, that is, by horizontal or vertical movement or rotation of the remote control device 1 inputs made. The interpretation takes place in such a way that the signals received due to movements of the remote control device 1 are analyzed in the corresponding function group 16 and corresponding operating commands are determined, wherein they are checked for agreement with possible operating commands held in a command memory 18. By means of the microphone 8 of the remote control device 1 received operating inputs are supplied to the digitizer and the transmission to the receiver 7 of the device to be operated 2 by the signal converter or signal splitter 21 a function group 12 for the interpretation of speech signals. Here they are analyzed by means of a speech recognizer 15 (ASR - Automatic Speech Recognition) on the basis of two different held in the function group grammars 13, 14 and derived from them, if possible, an operating command according to the specifications of the command memory 18. The two aforementioned grammars 13, 14 are a first grammar 13 containing a vocabulary for activating static operator commands. In addition, a second grammar 14 is provided, whose vocabulary used in speech analysis on the data resulting from the speech input is context-dependent and thus dynamic. The adaptation of the vocabulary to the context, that is to say, for example, to the programs which can be received at a certain time by means of the set-top box, takes place with the aid of the EPG 22 (Electronic Program Guide). For example, EPG 22 gets its data from the backend, that is an Internet server 23, a provider providing the content via the Internet 24 and / or by receiving corresponding data via a satellite system. In a third function group 17, the interpretation of keyboard input is performed by these are mapped to 18 according to the specifications of the command memory possible operating commands. The operating commands determined using the individual input modes are fed to the control unit 4, which derives control signals from them and with these control signals actuators 5 of the device, in the example a function group 5 'for volume control, a function group for program selection 5 "and / or a function group 5'. "to adjust screen parameters such as brightness and / or contrast. A crucial aspect of the invention is the fact that the manner in which interpreted in the individual input modes operator inputs are interpreted and converted into control signals for the device to be operated by the respective operated device 2 itself, namely the at least one arranged in this Interpretation unit 3, is determined. In this case, the firmware of the corresponding interpretation unit 3 in the way of the update, for example by transmission via the satellite channel, to be changed and thus the functionality of the device 2 are adapted to different circumstances.

Die an der Fernbedienungseinrichtung 1 einerseits und dem Gerät 2 andererseits angeordneten Mittel 6, 7 zur drahtlosen Kommunikation können gegebenenfalls auch jeweils als eine Kombination von Infrarotsender und Infrarotempfänger und somit bidirektional ausgebildet sein. Sofern darüber hinaus entsprechende Mittel, wie beispielsweise ein Display, ein Lautsprecher oder eine Vibrationseinheit in der Fernbedienungseinrichtung angeordnet sind, kann unter Nutzung dieser Bidirektionalität ein Feedback beziehungsweise eine Rückantwort des Geräts auf Bedieneingaben, so beispielsweise ein Quittierung von Eingaben oder eine Warnung bei Fehleingaben, erfolgen.The means 6, 7 for wireless communication arranged on the remote control device 1, on the one hand, and the device 2, on the other hand, may optionally also be designed in each case as a combination of infrared transmitter and infrared receiver and thus bidirectionally. If, in addition, appropriate means, such as a display, a loudspeaker or a vibration unit are arranged in the remote control device, using this bidirectionality, a feedback or a response of the device to operator inputs, such as an acknowledgment of inputs or a warning in case of incorrect entries done ,

Die Fig. 2 zeigt nochmals eine schematische Darstellung der Fernbedienungseinrichtung 1. Diese verfügt, gemäß dem zuvor erläuterten Beispiel, unter anderem über einen Eingabemodus zur Steuerung von Gerätefunktionen des Geräts 2 durch Bewegungen der Fernbedienungseinrichtung 1. Eine grundlegende, mittels dieses Eingabemodus steuerbare Gerätefunktion ist das Ein- und Ausschalten des Geräts. Dieses kann unter Nutzung von Bewegungseingaben in der Weise als "EasyOn" und "EasyOff" ausgestaltet sein, dass das Ablegen der Fernbedienungseinrichtung 1 auf ihre Rückseite beziehungsweise auf dem "Rücken" (siehe a) ein Einschalten des Geräts 2 bewirkt, während das Ablegen der Fernbedienungseinrichtung 1 auf die Vorderseite beziehungsweise auf dem "Bauch" (siehe b) zum Ausschalten des Geräts 2 führt, wobei gegebenenfalls durch den Nutzer zusätzlich die Motion-Control-Taste 20 während der Bewegung der Fernbedienungseinrichtung betätigt werden muss. Darüber hinaus kann das Hinstellen der Fernbedienung (siehe c) in eine aufrechte Position (mit dem Kopf beziehungsweise dem Beschleunigungssensor 9 nach unten - Upright-Position) zum Beispiel dazu verwendet werden, die Timeshift-Funktion der Set-Top-Box zu aktivieren, die den aktuellen Medienstream anhält und die Wiedergabe zu einem späteren Zeitpunkt ermöglicht, sobald beispielsweise die Fernbedienung wieder eine von der Upright-Position verschiedene Position einnimmt.The Fig. 2 shows again a schematic representation of the remote control device 1. This has, according to the example explained above, inter alia, via an input mode for the control of Device functions of the device 2 by movements of the remote device 1. A basic device function controllable by this input mode is turning the device on and off. This can be configured by using movement inputs in the manner as "EasyOn" and "EasyOff" that the placing of the remote control device 1 on its back or on the "back" (see a) causes the device 2, while the laying of the Remote control device 1 on the front or on the "belly" (see b) leads to turn off the device 2, which may also be actuated by the user in addition, the motion control button 20 during movement of the remote control device. Moreover, placing the remote control (see c) in an upright position (upside down with the head or acceleration sensor 9, respectively) may, for example, be used to activate the time-shift function of the set-top box, which stops the current media stream and allows playback at a later time, for example, as soon as the remote control again takes a different position from the upright position.

Weiterhin sind beispielsweise im Rahmen der Steuerung eines On-Screen-Cursors zur Bedienung eines On-Screen-Menüs der Set-Top-Box 2 folgende Funktionen und Zuordnungen dieser Funktionen zu Bewegungen der Fernbedienungseinrichtung 1 denkbar, wobei durch entsprechende Bewegungen der Fernbedienungseinrichtung 1 eine Steuerung des On-Screen-Cursors in alle vier Richtung ermöglicht ist.Furthermore, for example, in the context of controlling an on-screen cursor for operating an on-screen menu of the set-top box 2, the following functions and assignments of these functions to movements of the remote control device 1 are conceivable, with appropriate control of the remote control device 1 control of the on-screen cursor in all four directions.

Die Gesten des Anwenders werden dabei mit Hilfe eines dreiachsigen Beschleunigungssensors 9 erfasst, der die Lage der Fernbedienungseinrichtung 1 relativ zur Erdbeschleunigung erfasst. Damit lassen sich die Auslenkungswinkel 'Pitch' (Drehung um die horizontale X-Achse) und 'Roll' (Drehung um die horizontale Y-Achse) erfassen (siehe Fig. 2).The gestures of the user are detected by means of a three-axis acceleration sensor 9, which detects the position of the remote control device 1 relative to the gravitational acceleration. This allows you to detect the deflection angles 'Pitch' (rotation around the horizontal X-axis) and 'Roll' (rotation around the horizontal Y-axis) (see Fig. 2 ).

Es wird eine Auslenkung der Fernbedienungseinrichtung für Pitch und Roll im Bereich von jeweils -180 bis +180 Grad erfasst.A deflection of the remote control device for pitch and roll in the range of -180 to +180 degrees is detected.

Während die Motion-Control-Taste 20 gehalten wird, werden des Auslenkungen des Pitch und Roll in Steuerinformationen des On-Screen-Cursors umgesetzt. Sobald die Motion-Control-Taste 20 gedrückt wird, werden die augenblicklichen Werte für Pitch und Roll als Referenzwerte (PitchRef, RollRef) gespeichert. Solange die Motion-Control-Taste 20 gedrückt ist, werden nun alle Bewegungen relativ zu den gespeicherten Referenzwerten (PitchRef, RollRef) interpretiert. Jeder erneute Druck auf die Motion-Control-Taste 20 führt zu einer erneuten Festlegung der Referenzwerte für Pitch und Roll. Die Bewegungssteuerung funktioniert nun derart, dass die Änderungen des Pitch und Roll relativ zu deren Referenzwerten den vier Bewegungsrichtungen zugeordnet werden, solange die Motion-Control-Taste 20 gedrückt gehalten wird.While the motion control key 20 is held, the pitch and roll deflections are translated into on-screen cursor control information. As soon as the Motion Control button 20 is pressed, the current pitch and roll values are stored as reference values (PitchRef, RollRef). As long as the motion control key 20 is pressed, all movements relative to the stored reference values (PitchRef, RollRef) are now interpreted. Each new press on the motion control key 20 resets the reference values for pitch and roll. The motion control now functions such that the pitch and roll changes relative to their reference values are assigned to the four directions of motion as long as the motion control button 20 is held down.

Die Zuordnung zu den Bewegungsrichtungen des On-Screen-Cursors erfolgt dann beispielsweise wie folgt:

  1. 1. Es werden folgende Schwellwerte definiert:
    1. a. Minimale Änderung des Pitch, wenn dieser sich vergrößert (minPosPitch)
    2. b. Minimale Änderung des Pitch, wenn dieser sich verringert (minNegPitch)
    3. c. Minimale Änderung des Roll, wenn dieser sich vergrößert (minPosRoll)
    4. d. Minimale Änderung des Roll, wenn dieser sich verringert (minNegRoll)
    5. e. Maximale Änderung des Pitch, wenn dieser sich vergrößert (maxPosPitch)
    6. f. Maximale Änderung des Pitch, wenn dieser sich verringert (maxNegPitch)
    7. g. Maximale Änderung des Roll, wenn dieser sich vergrößert (maxPosRoll)
    8. h. Maximale Änderung des Roll, wenn dieser sich verringert (maxNegRoll)
  2. 2. Es wird die aktuelle Differenz des Pitch (PitchDiff) und Roll (RollDiff) zu deren Referenzwerten bestimmt.
  3. 3. Liegt PitchDiff und RollDiff unterhalb der in 1a.-1d. definierten Schwellwerten wird der entsprechende Wert ignoriert.
  4. 4. Liegt PitchDiff und RollDiff oberhalb der in 1e.-1h. definierten Schwellwerten wird PitchDiff und PitchRoll auf den entsprechenden Maximalwert begrenzt.
  5. 5. Der Auslenkung in beiden Achsen (Pitch, Roll) wird linear ein Intensitätswert zwischen 0,0 und 1,0 zugeordnet, der linear zwischen den in 1. definierten Minimal- (0,0) und Maximalwerten (1,0) liegt.
  6. 6. Je nachdem, ob der in 5. berechnete Intensitätswert für die Pitch- oder die Roll-Auslenkung größer ist, erfolgt die Bewegungssteuerung des On-Screen-Cursors ausschließlich in horizontaler oder in vertikaler Richtung. Ein größerer Intensitätswert für die Pitch-Auslenkung führt zu einer Bewegung des On-Screen-Cursors nach oben oder unten, je nachdem in welcher Richtung der Pitch-Wert verändert wurde. Ein größerer Intensitätswert für die Roll-Auslenkung führt zu einer Bewegung des On-Screen-Cursors nach links oder rechts, je nachdem in welcher Richtung der Roll-Wert verändert wurde.
  7. 7. Die Geschwindigkeit mit der der On-Screen-Cursor des On-Screen-Menüs bewegt wird, ist linear zu dem entsprechenden Intensitätswert.
  8. 8. Die Punkte 2. bis 7. werden in festen Abständen wiederholt, solange die Motion-Control-Taste 20 gedrückt ist. Das heißt, bis die Motion-Control-Taste 20 wieder losgelassen wird, wird der On-Screen-Cursor auch dann noch weiterbewegt, wenn sich der Auslenkungswinkel nicht mehr ändert, solange nur die Auslenkung oberhalb der Minimalschwelle liegt. Auf diese Weise kann über den Auslenkungswinkel die Geschwindigkeit gesteuert werden, mit der der On-Screen-Cursor im On-Screen-Menü fortbewegt wird.
The assignment to the directions of movement of the on-screen cursor then takes place, for example, as follows:
  1. 1. The following thresholds are defined:
    1. a. Minimum pitch change as it increases (minPosPitch)
    2. b. Minimum pitch change as it decreases (minNegPitch)
    3. c. Minimum change of roll as it grows (minPosRoll)
    4. d. Minimum roll change as it decreases (minNegRoll)
    5. e. Maximum pitch change as it grows (maxPosPitch)
    6. f. Maximum pitch change as it decreases (maxNegPitch)
    7. G. Maximum roll change as it grows (maxPosRoll)
    8. H. Maximum roll change if it decreases (maxNegRoll)
  2. 2. It determines the actual difference of the pitch (PitchDiff) and roll (RollDiff) to their reference values.
  3. 3. If PitchDiff and RollDiff are below those in 1a.-1d. defined thresholds, the corresponding value is ignored.
  4. 4. Is PitchDiff and RollDiff above in 1e.-1h. PitchDiff and PitchRoll are limited to the corresponding maximum value.
  5. 5. The deflection in both axes (pitch, roll) is linearly assigned an intensity value between 0.0 and 1.0, which lies linearly between the minimum (0.0) and maximum values (1.0) defined in 1. FIG.
  6. 6. Depending on whether the intensity value for the pitch or roll deflection calculated in 5 is greater, the motion control of the on-screen cursor takes place exclusively in the horizontal or in the vertical direction. A larger intensity value for the pitch deflection will cause the on-screen cursor to move up or down, depending on the direction in which the pitch value was changed. A larger intensity value for the roll deflection results in a movement of the on-screen cursor to the left or right, depending on the direction in which the roll value was changed.
  7. 7. The speed with which the on-screen cursor of the on-screen menu is moved is linear with the corresponding intensity value.
  8. 8. Points 2 to 7 are repeated at fixed intervals as long as the Motion Control button 20 is pressed. That is, until the motion control key 20 is released, the on-screen cursor is still moved even if the deflection angle no longer changes, as long as only the deflection is above the minimum threshold. In this way, the deflection angle can be used to control the speed at which the on-screen cursor moves in the on-screen menu.

Die Motion-Control-Taste 20 kann dabei nicht nur zur Navigation des On-Screen-Cursors innerhalb des On-Screen-Menüs verwendet werden. Mit der Motion-Control-Taste 20 können zusätzlich auch die selektierten Menüpunkte im On-Screen-Menü aktiviert werden, so dass auch auf eine separate Bestätigungstaste verzichtet werden kann ("One Button_Navigation"). Dadurch wird zur Steuerung des On-Screen-Menüs nur noch eine Taste benötigt und der Nutzer muss mit dem Finger oder Daumen auch keine unterschiedlichen Druckrichtungen unterscheiden wie sonst üblicherweise bei herkömmlichen 5-Wege-Keys. Die Unterscheidung zwischen Navigation und Bestätigung erfolgt anhand der zeitlichen Dauer, während der die Motion-Control-Taste 20 gedrückt wurde. Ein kurzer Klick auf die Motion-Control-Taste 20 wird als Bestätigung gewertet, während ein längerer Druck auf die Motion-Control-Taste 20 die horizontale und vertikale Navigation des On-Screen-Cursors aktiviert.The motion control key 20 can not only be used to navigate the on-screen cursor within the on-screen menu. In addition, the selected menu items in the on-screen menu can also be activated with the motion control key 20, so that a separate confirmation key can also be dispensed with ("One Button_Navigation"). As a result, only one button is needed to control the on-screen menu and the user must be with The fingers or thumbs also differ no different pressure directions as usually with conventional 5-way keys. The distinction between navigation and confirmation is based on the time duration during which the motion control key 20 was pressed. A short click on the motion control key 20 is considered as confirmation, while a long press on the motion control key 20 activates the horizontal and vertical navigation of the on-screen cursor.

Die Ablauflogik gestaltet sich beispielsweise wie folgt:

  1. 1. Im Ausgangszustand ist die Motion-Control-Taste 20 im Ruhezustand (nicht gedrückt).
  2. 2. Der Zeitpunkt, zu dem die Motion-Control-Taste 20 heruntergedrückt wird, wird vom System gespeichert (tMCbegin).
  3. 3. Solange die Motion-Control-Taste 20 noch heruntergedrückt ist, kann prinzipiell eine Navigationssteuerung wie im oberen Abschnitt beschrieben erfolgen.
  4. 4. Sobald die Motion-Control-Taste 20 wieder losgelassen wird, wird der Zeitpunkt des Loslassens der Motion-Control-Taste 20 (tMCend) mit dem vorhergehenden Zeitpunkt tMCbegin verglichen.
  5. 5. Ist die Differenz zwischen tMCend und tMCbegin kleiner als ein vorher festgelegter Schwellwert DiffMCclick, dann wird der Druck auf die Motion-Control-Taste 20 als Bestätigung gewertet und das zum Zeitpunkt tMCend durch den On-Screen-Cursor selektierte Element wird bestätigt/aktiviert. Andernfalls beschränkt sich die Verwendung der Motion-Control-Taste 20, wie in Punkt 3 beschrieben, auf die Navigationssteuerung.
Üblicherweise ist der Schwellwert DiffMCclick so klein zu wählen, dass der Nutzer während der kurzen Zeitspanne DiffMCclick praktisch keinerlei Navigation vornehmen kann. Typische Werte sind beispielsweise wenige hundert Millisekunden.The flow logic is, for example, as follows:
  1. 1. In the initial state, the motion control button 20 is in the idle state (not pressed).
  2. 2. The time at which the motion control key 20 is depressed is stored by the system (tMCbegin).
  3. 3. As long as the motion control button 20 is still depressed, in principle, a navigation control as described in the upper section.
  4. 4. As soon as the motion control key 20 is released, the time of release of the motion control key 20 (tMCend) is compared with the previous time tMCbegin .
  5. 5. If the difference between tMCend and tMCbegin is less than a predefined threshold DiffMCclick, then the pressure on the motion control key 20 is taken as an acknowledgment and the item selected at the time tMCend by the on-screen cursor is confirmed / activated , Otherwise, the use of the motion control button 20, as described in point 3, limited to the navigation control.
Usually, the DiffMCclick threshold is so small that the user can do virtually no navigation during the short DiffMCclick period. Typical values are, for example, a few hundred milliseconds.

Die Bewegungssteuerung kann über die Steuerung des On-Screen-Menüs hinaus auch zur stufenlosen Regelung anderer Systemparameter, wie beispielsweise die Lautstärke oder Helligkeit, verwendet werden. Durch einfache Drehbewegungen während die Motion-Control-Taste 20 gedrückt ist, kann so ähnlich wie bei einem analogen Drehregler in beliebiger Geschwindigkeit die gewünschte Systemgröße verändert werden. Mittels eines im Vorfeld erteilten Sprachbefehls (z.B. 'Lautstärke' oder 'Helligkeit', 'Kontrast') kann gegebenenfalls auch der jeweilige Systemparameter ausgewählt werden, der dann anschließend über die Bewegungssteuerung geregelt werden soll.In addition to the control of the on-screen menu, the motion control can also be used for stepless control of other system parameters, such as For example, the volume or brightness can be used. By simple rotary movements while the motion control button 20 is pressed, the desired system size can be changed at any speed, similar to an analog rotary control. By means of a previously issued voice command (eg 'Volume' or 'Brightness', 'Contrast') can also be selected if necessary, the respective system parameters, which will then be controlled by the motion control.

Claims (13)

  1. Arrangement having a remote-controllable appliance (2), consisting of the appliance (2) to be remote-controlled and a remote-control device (1) for multimodal wireless remote control of the appliance (2) and also having input means (8, 9, 10), arranged on or in the remote-control device (1), for receiving inputs in multiple input modes, a digitization unit (11), arranged in the remote-control device (1), for digitizing received inputs, at least one interpretation unit (3) for interpreting the inputs converted into digital data by the digitization unit (11), means (6, 7) for wireless communication between the remote-control device (1) and the appliance (2) to be controlled thereby and a control unit (4) for deriving control signals from operator control commands detected in the course of the interpretation of inputs and for actuating applicable actuators (5) of the controlled appliance (2) that implement the operator control commands corresponding to the respective inputs in the appliance (2), characterized in that
    - arranged on or in the remote-control device (1) as input means (8, 9, 10) are at least one microphone (8) for receiving voice inputs, at least one acceleration sensor (9) for receiving inputs in the form of movements of the remote-control device (1) and a keypad (10), and
    - the at least one interpretation unit (3), which has a group of functions (17) for interpreting keypad inputs, a group of functions (16) for interpreting movement inputs and a group of functions (12) for interpreting voice inputs using a voice recognizer (15) and using at least one grammar (13) defining a language area for the voice recognition, is arranged in the appliance (2) to be controlled, and
    - the at least one microphone (18) is activable by means of a key (19) for voice input that is arranged on the remote-control device (1), wherein the at least one microphone (18) is activated for the duration of operation of the key (19) or until said key is operated again, and when voice input is activated the volume of audible content reproduced by the appliance (2) is reduced to a minimum volume to improve reliability for the recognition of voice inputs.
  2. Arrangement according to Claim 1, characterized in that the group of functions (12) for voice recognition has a grammar (13) for defining a language area for the interpretation of operator control commands of a static command set for an operator control mode that allows the control of basic functions of the appliance (2) and a grammar (14) for defining a language area for the interpretation of operator control commands of a dynamically changing command set for an operator control mode that allows the control of expanded appliance functions.
  3. Arrangement according to Claim 2, characterized in that the operator control mode for the control of expanded appliance functions allows context-dependent control in view of media content just received and/or reproduced by the appliance (2).
  4. Arrangement according to Claim 2 or 3, characterized in that changeover between the operator control mode allowing the control of basic functions of the appliance (2) and the operator control mode allowing the control of expanded appliance functions is possible by a voice input.
  5. Arrangement according to one of Claims 1 to 4, characterized in that it takes an applicable interpretation of voice inputs in the at least one interpretation unit (3) as a basis for providing voice zapping, in which changeover to a particular program content is possible by naming a genre denoting this program content.
  6. Arrangement according to Claim 1, characterized in that the operator control commands interpretable by means of the at least one interpretation unit (3) arranged in the appliance (2) comprise at least operator control commands for switching the appliance (2) on and off, which is effected by changing the position of the remote-control device (1).
  7. Arrangement according to Claim 1, characterized in that the operator control commands interpretable by means of the at least one interpretation unit (3) arranged in the appliance (2) comprise at least operator control commands for steplessly controlling appliance functions by rotating the remote-control device (1) about at least one of its axes.
  8. Arrangement according to Claim 6 or 7, characterized in that the input means (9) arranged on or in the remote-control device (1) are activable to receive inputs in the form of movements of the remote-control device (1) by means of a key (20) provided for this purpose and arranged on the remote-control device (1) and are activated for the duration of operation of this key (20) or until this key (20) is operated again.
  9. Arrangement according to Claim 1, characterized in that the at least one interpretation unit (3) comprises a command memory (18) that stores the operator control commands obtainable as a possible result in the course of the interpretation of operator control inputs.
  10. Arrangement according to one of Claims 1 to 9, characterized in that the means (6, 7) for wireless communication between the remote-control device (1) and the appliance (2) to be controlled thereby are in bidirectional form.
  11. Arrangement according to Claim 10, characterized in that the remote-control device (1) has at least one display for visually outputting responses, generated by the control unit (4) of the appliance (2), to inputs made on the remote-control device (1) for controlling appliance functions of the appliance (2).
  12. Arrangement according to Claim 10 or 11, characterized in that the remote-control device (1) has at least one loudspeaker for audibly outputting responses, generated by the control unit (4) of the appliance (2), to inputs made on the remote-control device (1) for controlling appliance functions of the appliance (2).
  13. Arrangement according to one of Claims 10 to 12, characterized in that the remote-control device (1) has at least one vibration unit for haptically outputting responses, generated by the control unit (4) of the appliance (2), to inputs made on the remote-control device (1) for controlling appliance functions of the appliance (2).
EP08075599.4A 2008-07-01 2008-07-01 Assembly with device which can be controlled remotely Not-in-force EP2141674B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP08075599.4A EP2141674B1 (en) 2008-07-01 2008-07-01 Assembly with device which can be controlled remotely

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP08075599.4A EP2141674B1 (en) 2008-07-01 2008-07-01 Assembly with device which can be controlled remotely

Publications (2)

Publication Number Publication Date
EP2141674A1 EP2141674A1 (en) 2010-01-06
EP2141674B1 true EP2141674B1 (en) 2019-03-06

Family

ID=40342361

Family Applications (1)

Application Number Title Priority Date Filing Date
EP08075599.4A Not-in-force EP2141674B1 (en) 2008-07-01 2008-07-01 Assembly with device which can be controlled remotely

Country Status (1)

Country Link
EP (1) EP2141674B1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102016123547A1 (en) 2016-12-06 2018-06-07 Fm Marketing Gmbh Natural Language Dialog

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050027539A1 (en) * 2003-07-30 2005-02-03 Weber Dean C. Media center controller system and method

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6553345B1 (en) * 1999-08-26 2003-04-22 Matsushita Electric Industrial Co., Ltd. Universal remote control allowing natural language modality for television and multimedia searches and requests
US6990639B2 (en) * 2002-02-07 2006-01-24 Microsoft Corporation System and process for controlling electronic components in a ubiquitous computing environment using multimodal integration
KR100621593B1 (en) 2004-09-24 2006-09-19 삼성전자주식회사 Integrated remote control device using multimodal input and method of the same

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050027539A1 (en) * 2003-07-30 2005-02-03 Weber Dean C. Media center controller system and method

Also Published As

Publication number Publication date
EP2141674A1 (en) 2010-01-06

Similar Documents

Publication Publication Date Title
DE102008046493B4 (en) Multimedia arrangement with a programmable universal remote control
EP3040816B1 (en) Method and device for entering data with two types of input and haptic feedback, especially in a motor vehicle
DE2755633C2 (en) Remote control for controlling, switching on and switching between variable and fixed device functions and function variables in communications technology. devices
EP1118979B1 (en) Speech controled device
DE102011109932B4 (en) Method for controlling functional devices in a vehicle during voice command operation
CH690951A5 (en) Sound mixing.
DE69819690T2 (en) LANGUAGE RECOGNITION USING A COMMAND LIKE
DE69915668T2 (en) PORTABLE REMOTE CONTROL DEVICE
DE102014206160B4 (en) USER INTERFACE DEVICE USING A SPEAKER, CORRESPONDING METHOD AND COMPUTER READABLE MEDIUM
WO2010083820A1 (en) Method for executing an input using a virtual keyboard displayed on a screen
EP3096969A1 (en) User interface and method for controlling a volume by means of a touch-sensitive display unit
EP2212149B1 (en) Dialogue system for a motor vehicle
DE60015797T2 (en) UNIVERSAL REMOTE CONTROL PROCESS AND DEVICE
DE102016010920A1 (en) Smart touch
EP2141674B1 (en) Assembly with device which can be controlled remotely
DE202006019659U1 (en) Multifunction remote controller for use with a range of electronic equipment such as a television hi-fi system or garage doors
EP1040965A1 (en) Vehicle sound system and initialisation method
EP0467157B1 (en) Voice-operated device for entertainment electronic, especially for video recorder
WO2006103196A1 (en) Input forecasting method and a user input forecasting interface
DE19715101A1 (en) Method for controlling graphical systems such as printing machines
EP1544880A2 (en) Control device
EP1578016A1 (en) Programmable remote control, in particular infrared remote control and programming device for programming the remote control
DE4427444A1 (en) Voice operated controller for installation and appts., e.g. car radio
AT510873B1 (en) METHOD FOR OPERATING A REMOTE CONTROL OPERABLE ELECTRONIC DEVICE, AND AN ASSOCIATED ELECTRONIC DEVICE OR APPARATUS A RELATED REMOTE CONTROL
DE19920162A1 (en) Controlling audio-visual equipment by remote control with contact sensitive actuators has menu function that appears for touched actuator up to the elapse of a defined time interval

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA MK RS

17P Request for examination filed

Effective date: 20100702

17Q First examination report despatched

Effective date: 20100729

AKX Designation fees paid

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20180925

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

Ref country code: AT

Ref legal event code: REF

Ref document number: 1105594

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190315

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 502008016624

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20190306

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190606

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190606

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190607

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190706

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 502008016624

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190706

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

26N No opposition filed

Effective date: 20191209

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20190731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190701

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190731

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190731

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190701

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 1105594

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190701

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190701

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20080701

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20210721

Year of fee payment: 14

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20210722

Year of fee payment: 14

Ref country code: DE

Payment date: 20210721

Year of fee payment: 14

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 502008016624

Country of ref document: DE

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20220701

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220701

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230201