WO2001031424A2

WO2001031424A2 - Interactive system

Info

Publication number: WO2001031424A2
Application number: PCT/DE2000/003716
Authority: WO
Inventors: Adolf Bernds; Christoph Maggioni; Hans RÖTTGER; Jürgen SIMMERER
Original assignee: Siemens Aktiengesellschaft
Priority date: 1999-10-25
Filing date: 2000-10-20
Publication date: 2001-05-03
Also published as: DE19951322A1; WO2001031424A3

Abstract

The invention relates to an interactive system that comprises a surface unit (121) that is separate from a module box (111), said module box comprising the following components: a projector (127), a camera (113) and a computer (112) that controls the projector and that evaluates the images recorded by the camera in such a manner that the recorded image of the displacement or the stationary position of an interactive component on or in front of the surface unit can be interpreted as an input cursor.

Description

description

Interaction arrangement

The invention relates to an arrangement for interaction

Such an arrangement is known from [1]. In particular, the object to be recognized is a pointer unit, e.g. a hand or a pointing stick. With the pointing device, a user can use a graphic

Interact with the interface by triggering the appropriate actions by gesture control. Such an arrangement is also referred to as a gesture computer or a "virtual touch screen".

The way in which the virtual touch screen works is that the interface is projected onto a predetermined area, the projection preferably having virtual buttons, the actuation of which triggers predetermined actions. A trigger mechanism is the pointing of the user with his hand as a pointer unit to one of the buttons. After a predetermined period of time, the action associated with the button is triggered. The position of the hand of the user is recorded by a camera, the picture is digitized and a computer determines the position of the hand in relation to the projection. In this way it can be determined whether a virtual button has been operated or not. This release mechanism can be compared in its effect with the operation of a computer mouse as an input medium for a graphic interface. The mouse pointer is moved by hand on the virtual touch screen, instead of double-clicking, the hand is held over a selected button for a predetermined minimum period.

To recognize the pointing object on the user interface or to differentiate the projection from the Pointing object, m [2] the possibility of additional lighting of the user interface with waves in the invisible spectral range is presented. The projection can be appropriately differentiated from the pointer unit by illuminating the user interface with waves in the invisible spectral range. When illuminated with infrared light, the user interface reflects the infrared light more strongly than the pointing device (hand). The hand therefore has a higher absorption than the user interface. According to [2], this effect can be enhanced by the fact that the control surface is additionally provided with a reflective layer, so that the absorption of the infrared light on the control surface is as low as possible. The hand appears dark in front of the user interface.

One advantage of the virtual touch screen is the robustness of its construction. In particular, vandalism can be a simple and effective protection m the physical separation of the user interface and camera, computer and

Projector can be reached. However, if input components, for example, are also to be designed on the user interface, then these and in particular their lines to the actuating unit are potentially exposed to vandalism.

The object of the invention is to provide an arrangement for interaction which also has components on a surface unit, the electrical connections of which are protected against vandalism.

This object is achieved according to the features of the independent claim. Further developments of the invention also result from the dependent claims.

To solve the problem, an arrangement for interaction is specified in which a surface unit is provided, which is carried out separately from a module box. The module box has the following components in particular:

(1) A projector based on which

Surface unit an image can be represented. (2) A camera that captures the surface unit.

(3) A computer that controls the projector and evaluates the recording of the camera in such a way that a recording of a movement or a dwelling of an interaction component on or in front of the surface unit as a functionality of a

Input pointer is interpretable.

There is at least one wireless interface between the module box and the surface unit. This wireless interface is in particular a transmission interface for the transmission of data and / or energy.

One embodiment consists in that the interaction component is a hand or a finger of a user or a pointer unit. With his hand, the user can make an entry on the surface unit, onto which a user surface (graphic user interface (GUI)) is projected expediently based on the projector. For this purpose, the user moves his hand over the user interface, a mouse pointer being controlled by the movement of the hand. A predetermined action can be drawn on the user interface, for example, by holding the hand for a predetermined period of time at a specific point on the projection. This lingering action is triggered which is associated with a projected user interface. This corresponds to controlling a user interface using a computer mouse on a conventional personal computer.

Another embodiment is that at least one wireless interface is a sound interface. there the sound interface can preferably be an ultrasound interface. The ultrasound interface expediently has an ultrasound transmitter and an ultrasound receiver, which can be implemented on the surface unit and in the module box.

One embodiment consists of the at least one wireless interface being an optical interface. In particular, the optical interface can have an infrared transmitter and an infrared receiver. The

The infrared transmitter is preferably an infrared light-emitting diode or an infrared laser diode.

It is also an embodiment that the at least one wireless interface is an electromagnetic interface. The transmitter of the electromagnetic interface can be designed as one of the following components

a) Transmitting antenna, b) directional antenna, c) directional coupler, d) dipole antenna.

The receiver of the electromagnetic interface can be designed as one of the following components: a) antenna coil, b) slot antenna, c) dipole antenna.

A further embodiment consists in that the respective transmitter of the at least one wireless interface m of the module box and the respective receiver of the at least one wireless interface m of the surface unit are designed. The respective transmitter of the at least one wireless interface m of the surface unit and the respective receiver of the at least one wireless interface m of the module box can be designed accordingly. Another development is that an input device is provided in the surface unit. The input device can include the following: a) joystick, b) button, c) rocker arm, d) distance-sensitive sensor, e) touch-sensitive sensor, f) pressure-dependent resistance, g) spring plate, h) switch,

I) speech recognition system, j) trackball, k) mouse,

1) touchpad, m) fingertip sensor, n) roller, handwheel.

The possible input devices mentioned above represent only a selection.

An additional embodiment is that the surface unit has a sensor for converting light energy into electrical energy, the sensor supplying the surface unit with current. In particular, the sensor for converting light energy into electrical energy can be a solar cell.

Exemplary embodiments of the invention are illustrated and explained below with the aid of the drawings.

Show it

Fig.l an arrangement for interaction;

2 shows a processor unit; 3 shows an arrangement for interaction, the one

Has surface unit with additional functionalities;

4 shows a surface unit with possible input devices.

Fig.l shows how a virtual touch screen works. First, the surface unit in the form of a passive user interface BOF is presented.

Surface unit is understood in particular to mean the physical expression of the base surface onto which a projection is made. The content of the projection is e.g. the BOF user interface.

The user interface BOF (interaction area, image of a graphical user interface GUI) is mapped to a predeterminable area (surface), here a projection display PD (interaction area). The Pro etionsdisplay PD replaces a conventional screen. The input is made by pointing directly with the interaction component, here a hand H on the

BOF user interface. This means that, for example, the keyboard, mouse, touchscreen or digitizing tablet of conventional systems can be replaced. The recognition of the gestures and the positioning of the hand within the user interface BOF are realized by a video-based system that is able to recognize and track the projection and shape of the human hand, for example, in real time. Furthermore, the projection display PD is illuminated with infrared light in FIG. The infrared light source IRL can advantageously be designed using infrared light-emitting diodes. A camera K, preferably with a special infrared filter IRF, which is particularly sensitive in an infrared spectral range is configured, the projection display PD includes the hand H of the user. With a pro _j ector P, which is controlled by a computer R, the operating surface BOF is imaged on the projection display PD. The user interface BOF can be configured as a graphical user interface (GUI) on a monitor of the computer R. A mouse pointer MZ is moved by the hand H of the user. Instead of the hand H, a pointer unit can also be used as the interaction component.

If the function or functionality associated with the actuation of a field F is to be triggered on the user interface BOF, the hand H is moved to the field F, the mouse pointer MZ follows the hand H. The hand H remains above the for a predeterminable period of time Field F, the function associated with field F is drawn on computer R. In conventional systems, this corresponds to double-clicking the mouse when the mouse pointer is over a virtual switch.

A processor unit PRZE (computer) is shown in FIG. The processor unit PRZE comprises a processor CPU, a memory MEM and an input / output interface IOS, which is used in different ways via an interface IFC: via a graphic interface, an output on a monitor MON, in particular a projector, becomes visible and / or printed on a PRT printer. An entry is made using a mouse MAS or a keyboard TAST. The processor unit PRZE also has a data bus BUS, which ensures the connection of a memory MEM, the processor CPU and the input / output interface IOS. Additional components can also be connected to the data bus BUS, for example additional memory, data memory (hard disk), camera, frame grabber, detection unit, input devices, network connections or scanners. 3 shows an arrangement for interaction, which has a surface unit, which surface unit comprises additional functional acts.

A module box 111 comprises a computer (processor unit, evaluation unit) 112, a camera (Detektionsemheit for shafts, preferably emitted from IRL radiation) 113, a pro _j ector 127, an infrared light 128 (source of waves for non-visible light, preferably infrared light).

The surface unit 121 comprises a multiplicity of components which, in addition to the contactless interaction described above, also enable a touch-sensitive interaction.

In the arrangement of Figure 3, some wireless interfaces are shown. A laser diode 114, which is provided in the module box, radiates specifically onto a photodiode 120, which is arranged in the surface unit 121. Optical energy is thus transmitted from laser diode 114 to the associated sensor (photodiode 120). In the reverse direction, energy or data / signals are transmitted from an optical transmitter 119, for example a light-emitting diode or a laser diode, both of which preferably emit light in the non-visible spectral range, to a photodiode. The energy transmitted, in particular from the module box 111 to the surface unit 121, can be used by converting electrical energy at the receiver to operate an input device or the surface unit itself. Parts of the surface unit can be made interchangeable.

A joystick 118 shows a signaling input device that controls the optical transmitter 119. The signals that are transmitted to the evaluation unit 112 via the interaction of the joystick 118 are emitted via the duplex connection Transmitters 119, 114 and receivers 120, 115. The _Ü transmission takes place without electrical lines, that is certainly invisible against vandalism and for the user.

The surface unit 121 is supplied with energy via a solar cell 122. In addition to the solar cell, energy can be transmitted from the module box 111 to the surface unit 121 in a targeted manner by the laser diode 114 feeding the optical receiver 120, the optical receiver 120 providing the light energy m electrical energy for operation the surface unit converts.

FIG. 3 additionally shows a full-duplex transmitter-receiver connection based on ultrasound. For this purpose, ultrasonic transmitters 123 and 125 or ultrasonic receivers 124 and 126 are provided.

Another wireless interface of an electromagnetic type is shown in the form of the receiving and transmitting coils 130 and 116. A keypad 117 transmits signals which are transmitted via the transmission coil 116 from the surface unit 121 to the module box 111, there the reception coil 130. The computer 112 takes over the coding of the transmitted signals m predetermined actions.

4 shows a surface unit 216 with possible input devices. An area 211 is provided as an information area. Two light-sensitive input devices (photodiodes) are suitable for recording certain ambient lighting (optical button,

Reflex light barrier). Otherwise, m are provided in FIG. 4: a joystick 214, pushbutton or push switch 217, a projection surface 218, a three-dimensional design of an interaction surface in the form of an elevation 219 and a depression 220 and a surface 221, which is shown by the camera 113 m in FIG is recorded. Bibliography :

[1] US 5,528,263

[2] DE 197 08 240

Claims

claims

1. Arrangement for interaction, a) beι a surface unit is provided, which is carried out separately from a module box; b) beι the module box has the following components: (1) a projector, based on which on the

Surface unit an image can be displayed; (2) a camera that picks up the surface unit;

(3) a computer which controls the projector and evaluates recordings of the camera in such a way that a recording of a movement or a dwelling of an interaction component on or in front of the surface unit as a functionality of a

Input pointer is interpretable; c) beι the between the module box and the

Surface unit at least one wireless interface exists.

2. Arrangement according to claim 1, wherein the at least one wireless interface is a sound interface.

3. Arrangement according to claim 2, wherein the sound interface is an ultrasound interface.

4. Arrangement according to claim 3, wherein the ultrasound interface

Has ultrasonic transmitter and an ultrasonic receiver.

5. Arrangement according to claim 1, wherein the at least one wireless interface is an optical interface.

6. An arrangement according to claim 5, wherein the optical interface comprises an infrared transmitter and an infrared receiver.

7. The arrangement according to claim 6, wherein the infrared transmitter is an infrared light-emitting diode or an infrared laser diode.

8. The arrangement according to claim 1, wherein the at least one wireless interface is an electromagnetic interface.

9. Arrangement according to claim 8, wherein the electromagnetic interface as a transmitter has one of the following components: a) transmitting antenna, b) directional antenna, c) directional coupler, d) dipole antenna.

10. The arrangement according to claim 9, wherein the electromagnetic interface as a receiver has one of the following components: a) antenna coil, b) slot antenna, c) dipole antenna.

11. Arrangement according to one of the preceding claims, in which the respective transmitter of the at least one wireless interface m of the module box and the respective receiver of the at least one wireless interface m of the surface unit are designed.

12. Arrangement according to one of the preceding claims, wherein the respective transmitter of the at least one wireless interface m of the surface unit and the respective receiver of the at least one wireless interface m of the module box are designed.

13. Arrangement according to one of the preceding claims, in which an input device is provided in the surface unit.

14. Arrangement according to claim 13, wherein the input device comprises one of the following options: o) joystick, p) button, q) rocker arm, r) distance-sensitive sensor, s) touch-sensitive sensor, t) pressure-dependent resistance, u) spring plate, v) Switch, w) speech recognition system, x) trackball, y) mouse, z) touchpad, aa) Fmgertιp sensor, bb) roller, handwheel.

15. Arrangement according to one of the preceding claims, wherein the surface unit has a sensor for converting light energy m electrical energy, which sensor supplies the surface unit with current.

16. The arrangement according to claim 15, wherein the sensor for converting light energy m electrical energy is a solar cell.

7. Arrangement according to one of the preceding claims, wherein the interaction component comprises one of the following possibilities: a) a hand of a user; b) a finger of a user; your pointer unit.