DE102021118382A1 - New input device - Google Patents

Abstract

Input device (1) for operation with the fingers (2) of an operator, having at least two touch-sensitive and pressure-sensitive screen modules (3), each of which enables precise position detection of a touch by the operator's finger (2) and for displaying graphical interface layouts (4) are set up, wherein the touch-sensitive and pressure-sensitive screen modules (3) are each designed with at least one vibration motor (5), the input device (1) being set up for this purpose when the respective screen module (3) is touched with a finger (2) of the operator to provide haptic feedback

Description

Technical background

The invention relates to a new type of input device which can be used, for example, for a personal computer and which is particularly suitable for replacing the usual, widespread input devices, keyboard and mouse, for merging them into one input device, or for at least one of these devices (keyboard or mouse). replace to improve usability.

Basically, a personal computer (whether desktop device or notebook) is operated today predominantly with a keyboard and mouse, with the mouse often being substituted by a so-called touchpad or a control stick, particularly in the case of notebooks.

With this type of operation, there is often the problem that extensive movement of the operator's hand is required to switch between keyboard and mouse or touchpad.

The keyboard is one of the most universal peripherals for desktop and laptop computers, and yet it is based on the QWERTY system, which dates back to the 1870s. It is arguably the oldest part of the desktop and laptop computers in use today. Keyboard use is ubiquitous in word processing, web browsing, multimedia streaming, and gaming. However, the design of the QWERTY layout is not optimized for the ergonomic requirements of modern demands. From Scripture US 5,689,253A an ergonomic keyboard with orthogonal keys is known. In addition, the operation of the function keys with the thumb is considered in the design. Devices are also known in which keys for switching between different settings (e.g. upper and lower case) are moved to prominent positions in order to ergonomically simplify switching.

However, the static representation of the keys on the keyboard limits the areas of application of the keyboard and their intuitive use. From the WO 2017/019390 A1 a keyboard with a projection system is known. With this projection system, keys are dynamically projected onto the keyboard. Thus, different layouts and intuitive macros are possible. Input devices, including proximity-sensing devices (also commonly referred to as touchpads or touch-sensing devices), are used in a variety of electronic systems. Out US 8,896,539 B2 a multi-touch keyboard is known in which the input depends on the finger position or finger movement. Out DE 11 2009 003 647 T5 a display input device is known in which vibrations between the finger and the touch panel are detected by a vibration detection sensor and operating elements under the finger are displayed enlarged.

So-called virtual keyboards have also been described. State-of-the-art virtual keyboards project onto surfaces and will likely never become the "preferred" keyboard for users. Virtual keyboards have a futuristic look and can be used instead of keyboards for short sessions, but for the “normal” or “heavy” computer user, the virtual keyboard lacks the haptic and tactile and many more features. a maximum of control and precision when entering text is only possible with a physical device due to the tactility.

Some research concepts examine and demonstrate a mechanical keyboard capable of capturing rich and expressive motion gestures performed both on and directly above the device. Another paper examined keyboards with gesture sensors and integrated function keys into gestures of individual keys. However, these gestures are limited on the two-dimensional surface and cannot be operated blindly due to the lack of a tactile surface or haptic feedback.

Mechanical keyboards can give users tactile feedback when the keys on the keyboards are pressed. Users can tell if the button is pressed by the tactile feedback. However, the mechanical keyboards are very bulky and only allow a few forms of interaction (no gestures). At least for this reason, the new electronic handheld devices z. B. provided with touch keyboards, since the touch keyboards are very thin compared to the mechanical keyboards. However, the touch keyboards do not provide tactile feedback for users. Accordingly, in the absence of tactile feedback, users cannot know if they actually pressed the buttons. In order to still give the users feedback, acoustic or optical signals are alternatively generated as feedback. However, many users prefer direct tactile feedback rather than an audible or visual signal. It is also known that humans react more slowly to visual and acoustic stimuli than to tactile stimuli. From the U.S. 2009/167694 A1 a keyboard with vibration feedback is known, with two different vibrating key groups. previous input devices did not combine haptic feedback with visual feedback (screens).

Some input devices are also able to recognize and determine the force exerted on a sensing surface in addition to the position information. From the U.S. 9,335,844 B2 an input device is known which is touch-sensitive and pressure-sensitive. Depending on the pressure, it can function as a touchpad or keypad. However, this input device lacks the tactile and haptic advantages of the classic mechanical keyboard.

With keyboards, the layout (the assignment of the individual keys) can be adjusted via software, but the change in functionality can only be seen on keyboards with screens or on virtual keyboards that are displayed on screens. From the WO 2017/019390 A1 an input device is known in which the displayed layouts can be adjusted visually. The neo community offers various layouts for download that are optimized for specific applications. Many applications allow the user to create time-saving "macros", but most users do not take advantage of this because the barrier to learning how to program the macros is very high. Users often buy more than one keyboard or controller, each made for a specific purpose. Many gaming applications only use about 8 buttons, and the unused buttons become redundant during gaming. Many graphics and imaging users perform thousands of mouse clicks every day to accomplish tasks that could be greatly simplified with a smarter human-computer interface. The keyboard layout can be completely reassigned by software.

The novel input device described here discloses a new solution that allows the user to switch between different operating modes, particularly with regard to the need for movement. In addition to the classic keyboard and the touchscreen, the novel input device described here is a new form of input device that adapts visually, offers local haptic feedback, and enables a new class of operating actions through the combination of touch and pressure controls.

Disclosure of Invention

Described here is an input device for operation with an operator's fingers, having at least two touch-sensitive and pressure-sensitive screen modules, each of which enables precise detection of a touch by the operator's finger and is set up for the display of graphical interface layouts, the touch-sensitive and pressure-sensitive screen modules each having at least a vibration motor, the input device being set up to provide haptic feedback when the respective screen module is touched with a finger of the operator.

The touch-sensitive and pressure-sensitive screen modules of the input device described here have local individual haptic feedback and pressure sensitivity for operation by swiping and typing by touch operation and pressure movements. The input device has a division of the screen modules for separate interaction and for separate tactile and haptic operation and feedback within one device. An essential feature of the input device described here is that at least two, but preferably more and very particularly preferably five, touch-sensitive and pressure-sensitive screen modules are provided, which are each assigned to individual fingers of an operator's hand. Each screen module enables position-accurate detection of a touch and each screen module has its own pressure sensor and vibration motor. It is a finding underlying the input device described here that the assignment of a screen module with its own touch detection, its own pressure detection and its own vibration motor (both independent of the touch detection, pressure detection and the vibration motors of other screen modules of the input device), significant improvements in providing the user experience that is similar to the user experience when operating a regular keyboard. The input device is therefore divided into separate screen modules (each with its own touch recognition, pressure recognition and vibration motors). Each finger is then assigned to an individual screen module. Vibrations can be carried out completely independently of each other for each finger individually and sometimes simultaneously or overlapping in time. This enables the user experience to be significantly improved.

Nevertheless, the input device preferably has a uniform surface for the operator. This uniform surface preferably extends over several screen modules. It is possible that there are (small) interruptions in the surface in a transition area between the screen modules, which for example as a type of (narrow) gap between two adjacent screen modules are executed. However, the operator can swipe his finger across such interruptions and thus switch from one screen module of the input device to another (neighboring/adjacent) screen module of the input device in one swipe movement.

The input device preferably has sensors for using multiple operating actions such as touching, clicking, pressing, swiping, typing and drumming. The vibration motor and the pressure-sensitive screen module have essential elements for enabling these operating actions to be recognized in order to provide the operator with feedback which corresponds to these operating actions or is perceived by the operator as a suitable/comprehensible reaction to the respective operating action. The input device has screen modules for graphically displaying the interface layout. Interface layouts preferably have one or more control elements that can be displayed dynamically on the screen modules. Interface layouts feature controls such as symbols, numbers, letters, indicators, touchpads, sliders, drum pads, and joysticks.

The input device is set up as a touchscreen for the precise detection of touches. For this purpose, the screen module preferably has a screen layer that is touch-capable and particularly preferably multi-touch-capable. The term "touch-enabled" here means that the screen layer can recognize a touch and can also determine where this touch took place. In this way it can be recognized whether an operator is touching a specific position of the screen module with a finger. If necessary, it can also be recognized in this way how the position changes at which the operator's finger touches the screen module (movement in the two-dimensional area on the surface). The term "multi-touch capable" means that the screen layer can recognize multiple touches with multiple fingers at the same time and can also determine where the touches took place. A multi-touch screen layer also enables more complex operations on a single screen module. The screen module (and in particular the screen layer set up in the screen module) is particularly preferably designed for operation by touch gestures. The input device or each individual screen module of the input device has a pressure sensitivity for operation using pressure gestures. The pressure sensitivity is set up in particular in such a way that the strength of the pressure exerted with a finger on the surface of the screen module can be recognized and thus also evaluated. The input device or each individual screen module of the input device thus has sensors for differentiating different pressure movements and pressure levels of the operator's fingers on the touch-sensitive screen modules. The input device or each individual screen module of the input device has a vibration function for operation with dynamic haptic feedback to mimic a realistic click feeling. The input device also features local dynamic vibration feedback for blind typing in augmented reality. The vibration function is realized with at least one vibration motor. The at least one vibration motor is preferably set up in such a way that it can generate a vibration with a movement amplitude in the direction of a touch surface of the screen module (X direction or Y direction).

The display of operating elements or interface layouts on the screen modules and the control for evaluating touches on the screen modules are preferably coordinated with one another. The input device is set up to evaluate touches by the operator as indicated to the operator by the display of the control elements or the interface layout.

The input device can be operated with one or more fingers using various operating actions. An operator action always refers to a specific sequence of touches on the screen modules of the input device, which is recognized by the sensors and evaluated by the software in order to trigger specific control commands. Operating actions take place by touching the screen modules, for example the touch surfaces of the screen modules, and can have different characteristics. Light touches can be made with a pressure of 50 grams. Strong touches can be made with a pressure of 120 grams. Operating actions for operating the input device can, for example, also include movement profiles such as mouse movements, 2- to 4-finger gestures, and pinch-to-zoom or the like. Service actions can be chained together and result in a new service action. E.g.: double touch, double click; Touch or move when released after a light/strong press. The input device can be set up to react in an impact-sensitive manner. In the case of impact sensitivity, not only is the pressure (e.g. 50 grams or 120 grams) that occurs when touched recognized, but also the slope of the pressure increase during the impact movement, e.g. in the unit [grams/second]. Operating actions can also be evaluated differently depending on the length of the operating action, e.g. short vs. long hold.

It is preferred if a touch surface of each touch-sensitive screen module is divided into at least two tactile areas, with the touch surface having an orientation structure that can be felt by the operator's fingers to form the tactile areas.

The orientation structure that can be felt can be felt, in particular, haptically. The tactile orientation structure can be formed, for example, by arranging a concave dent in each of the at least two tactile areas, which offers the fingers of an operator a haptic orientation on the touch surface of the touch-sensitive screen module. The entirety of the dents in each tactile area thus forms the tactile orientation structure here. This is a first possibility of forming the tactile orientation structure. In further embodiment variants, the tactile orientation structure can also be formed, for example, with grooves, elevations or similar structural features of the touch surface.

On conventional keyboards, the keys in different rows of keys are always slightly offset from one another. Historically, this layout came about because it was mechanically advantageous for typewriters. In the input device described here, it is preferred if the individual touch areas on the screen modules are each arranged in rows and columns that are aligned with one another in both directions (X direction and Y direction) and are not offset. The input device therefore preferably has straightened or orthogonal touch area columns. This is primarily for creating a cleaner physical layout and symmetrical movement between the left and right hands.

The touch surface of the input device preferably has elevations between arched indentations of adjacent touch areas. Such elevations then preferably also form the tactile orientation structure. The elevations preferably form boundaries of the respective tactile areas that can be haptically detected by the operator. The dents or elevations are perceptible to the operator when running his finger over them, but do not impede his movement. The dimples are deep enough to guide the operator's fingers when typing blindly and shallow enough to comfortably perform gestures on the multi-touch-enabled surface, with the operator swiping across the boundaries of each tactile area as needed exerts a pressure on the screen module or exerts a touch that can be perceived by the screen module. No pressure is usually required to detect a touch. Rather, the presence of a finger of the operator when touched can preferably be perceived independently of the pressure.

It is also preferred if a first group of touch-sensitive screen modules is provided for operation with the index finger, middle finger, ring finger and little finger, which are arranged in a row next to one another and a further touch-sensitive screen module is arranged below for operation with the thumb.

In addition, it is preferred if the touch-sensitive screen modules of the first group each have at least three rows of touch areas arranged one above the other.

The tactile areas of the screen modules assume the role of the keys of the classic keyboard. Screen modules with up to six key areas combine separate keys on the classic keyboard.

In this context, it is also preferred if the first group comprises exactly four touch-sensitive screen modules, with each of the fingers index finger, middle finger, ring finger and little finger being assigned or assignable to exactly one touch-sensitive screen module, with the touch-sensitive screen modules for the index finger and little finger two each Columns of juxtaposed touch areas and the touch-sensitive screen modules for the ring finger and middle finger each have exactly one column of juxtaposed touch areas.

On the touch-sensitive screen modules, relevant operating elements can be displayed directly under a finger with which these operating elements are to be operated. A resting position of finger positions on the screen modules is preferably provided, which in each case defines a touch area of the fingers. These resting finger positions preferably form a so-called "Homerow" - a series of finger positions for the index finger, middle finger, ring finger and little finger. The size and position of the screen modules and tactile areas relative to each other is preferably defined in such a way that, starting from one of the finger positions in the "Homerow", other tactile areas of the respective screen module are within reach of the fingers and there is good mobility of the fingers to these other tactile areas. The screen modules for the middle finger and ring finger have a width of one touch area because they have less mobility. The screen modules for the index finger and little fingers have a width of two tactile ranges because these two fingers have greater flexibility. In addition to the column of tactile areas with the finger position in the “Homerow”, there is preferably a column of tactile areas further out in the display modules for the index finger and the little finger.

In comparison to the classic keyboard, the input device preferably has a smaller number of keys or key areas. This makes the keys easier to reach when typing blindly. The design and presentation of the control elements is preferably based on the mobility and range of the fingers, also independently of the shape and design of the touch areas. This applies in particular to operating elements that also extend across multiple touch areas and possibly even across multiple screen modules. The input device has a dynamic representation of controls near the finger position of the fingers in the "Homerow" for quick accessibility and operability. The arrangement of the tactile areas in an orthogonal arrangement that is not offset from one another suggests and teaches the operator how to type correctly. The input device has an arrangement of screen modules and tactile areas to suggest the optimal placement of the hand on the input device. The rapid blind operability of an input device with all fingers of an operator's hand essentially depends on the tactile areas around a finger position being easily accessible. The tactile areas assigned to a finger for operation are each combined in a screen module in the input device, which is set up to also offer the operator haptic feedback with the vibration motor.

The other screen module for operation with the thumb is preferably set up to take on the role of the "modifier keys" of the classic keyboard. The further screen module preferably also has tactile areas and preferably exactly four tactile areas, which are arranged in a 2×2 layout. The tactile areas of the further screen module are preferably at least partially set up as “modifier keys”. When using a keyboard, the thumb is usually only used for the space character or "SPACE" and is therefore usually free. By using the thumb (in particular for operating "modifier keys"), it enables, for example, 2-button shortcuts (e.g. consisting of a "modifier key" such as "alt", "shift" or "ctrl" as well as a letter and a number) very quickly, especially without removing your hand from the finger positions in the "Homerow" or adopting special uncomfortable finger positions.

It is also preferred if graphical interface layouts of the first group of touch-sensitive screen modules can be selected by operating actions on the further touch-sensitive screen module with the thumb.

For example, interface layouts can provide operating options for different applications. Optionally, such interface layouts can also be automatically selected on the input device if the operator himself or automatically opens a specific application for which the respective interface layout is intended. The method described here of using the further screen module for operation with the thumb to select the interface layout is an advantageous addition to the automatic selection of the intended interface layout.

It is also preferred if at least one keyboard interface layout and at least one touchpad interface layout can be displayed on the first group of touch-sensitive screen modules, and a change between the at least one keyboard interface layout and the at least one touchpad interface layout by confirming the further touch-sensitive Screen module can be done with the thumb.

The keyboard interface layout makes it possible to take over the usual operating functions of a keyboard. The four screen modules of the first group of the input device have a total of 18 keys. With two input devices used for the left and right hand, a total of 36 keys are available. This is sufficient to provide a keyboard that can be operated very quickly and easily with 8 fingers (+ 2 thumbs).

The touchpad interface layout replaces the usual mouse or touchpad on a notebook/laptop. It is particularly advantageous that the touchpad can be embodied over several screen modules of the input device. When operating the touchpad, the operator can swipe across the border of the screen modules. The controller/software of the input device is preferably set up to compensate for this and to ensure consistent evaluation of the inputs on a number of screen modules.

By combining a keyboard layout and a touchpad interface layout in the input device, a very unproblematic and comprehensive possibility of operation can be created. The input device is (particularly in combination with another mirror-inverted input device) suitable for completely replacing the keyboard and mouse used to operate a PC.

• - ein Layout umfassend mindestens einen Schieberegler;
• - ein Pfeiltastenlayout; und
• - mindestens ein benutzerdefiniertes Interfacelayout.

It is also preferred if at least one special interface layout can also be displayed on the first group of touch-sensitive screen modules and a change to this special interface layout can be made by confirming the other touch-sensitive screen module with your thumb, with the at least one special interface layout provides at least one of the following functions:

• - a layout comprising at least one slider;
• - an arrow key layout; and
• - at least one custom interface layout.

Interface layouts have at least one control element. Interface layout controls are icons, sliders, indicators, touchpads, joysticks, or drum pads. Interface layouts show any arrangement of control elements to be divided into different control areas. In a system made up of two input devices that are mirror-inverted to one another, the left input device is used in embodiment variants to display symbols and function keys, and the mirrored right input device is used to display a touchpad or similar components. Many other arrangements are feasible. Controls may be located on individual touch areas, across touch area boundaries on entire screen modules, across screen module boundaries, or the entire touch surface of the device.

Operating the controls creates customizable haptic feedback. Control elements have an adjustable pressure sensor to differentiate between different impact strengths for typing or drums. Input and output parameters of the control elements can be adjusted by the operator in terms of touch sensitivity, both visually and haptically (e.g. click threshold, strength of vibration feedback). These settings can also be permanently linked to an interface layout and automatically predetermined by the selection of a specific interface layout.

The control elements of such interface layouts are preferably adapted to the grid specified by columns and rows of touch areas. In the case of user-defined interface layouts, the operator preferably has the option of positioning, scaling and configuring control elements on the input device grid. Controls can be moved, edited and deleted by the operator in an interface for modifying user-defined interface layouts, preferably using drag'n'drop technology. When editing the control elements, clicking on the control element opens options for setting the appearance and functionality of the control element. The surface for modifying user-defined interface layouts is particularly preferably offered at least partially on the input device or its screen modules themselves.

Interface layouts can be created and edited by the operator via a client. Interface layouts with already optimized application-specific layouts can also be made available to the operator on a crowd-sourced layout library.

It is also preferred if by exerting a specified pressure on the touch-sensitive screen modules at the screen positions specified by the interface layout, a change can be made to at least one sub-operating menu, in which it is possible to switch between sub-functions for the functionality provided at the screen position, wherein in the Sub-operating menu Sub-functions can be selected by swiping with your finger.

A simple possibility was created here to combine several functions in a touch area or on a button simulated in a touch area. Depending on the strength of the pressure, the user accesses different virtual operating levels of the input device. Depending on which virtual operating level the user goes to, the pressure on the touch area is interpreted differently. For example, a first virtual level, reachable by a slight pressing motion, can be used to output lowercase letters like "a". in the second virtual level below, which can be reached by higher pressure, capital letters such as "A" can then be output. Pressing movements can be used within a control element (e.g. a button of a control element simulated in a touch area) to switch to further sub-control menus (control levels). in sub-operating menus, the operating element can be operated by releasing it or by swiping. By pressing on a letter, you can switch between different variations of a letter and you can choose between different variations of the letter by an additional swiping movement within a sliding menu.

It should be noted here that the selection of a specific virtual level can only be recognized with certainty when it is clear that the operator does not allow the operating pressure to increase any further and that there is no longer the possibility of changing to a lower virtual level. This is achieved in that, depending on the selected virtual level, the selection of the level is recognized either by a high of the pressure movement or by a low of the pressure movement, with a low meaning that the pressure is still increasing and a high meaning that the pressure is already falling again . Basically, for all virtual levels below which there are further virtual levels that can be reached with a higher pressure, the selection of the Final level can only be recognized when the pressure movement is high. An alternative would be to recognize the selection when the operator, under constant pressure, begins to perform further operator actions, such as swiping to select certain variants of a letter. In a lowest virtual level, it is no longer important to only recognize the selection at the top of the pressure movement, but the recognition can already take place at the bottom of the pressure movement, because a further increase in pressure no longer enables a change to another, even deeper virtual level .

• - eine ergonomische transparente Berührungsoberfläche, welche Dellen und Erhebungen bereitstellt, die einem Bediener eine haptische Orientierung bieten,
• - eine Bildschirmschicht,
• - eine multitouch-fähige Oberfläche,
• - eine Platinenschicht zumindest mit einem Vibrationsmotor und einem Drucksensor,
• - ein gemeinsames Gehäuse für mehrere Bildschirmmodule, auf welchem die Berührungsoberfläche, die Bildschirmschicht, die multitouch-fähige Oberfläche und die Platinenschicht mit mindestens einem elastisch verformbaren Abstandhalter beweglich gelagert sind.

In a preferred embodiment variant, the touch-sensitive and pressure-sensitive screen modules of the input device have a layered structure with at least the following layers:

• - an ergonomic transparent touch surface that provides dimples and bumps that provide haptic orientation for an operator,
• - a screen layer,
• - a multi-touch capable surface,
• - a board layer at least with a vibration motor and a pressure sensor,
• - A common housing for several screen modules on which the touch surface, the screen layer, the multi-touch capable surface and the circuit board layer are movably mounted with at least one elastically deformable spacer.

The layered structure described here is a possible, particularly preferred structure to which the input device described here is not limited. It can also be constructed differently in order to fulfill the functional characteristics described. For example, depending on the application and manufacturing options, the order of the layers can be changed.

The haptic vibration motor and the pressure sensor are preferably integrated directly into a circuit board of the circuit board layer. The screen layer, which is essential for the screen module, also serves as a weight which can be set in motion by the vibration motor and preferably oscillates on the elastically deformable spacers. The screen layer itself thus serves as part of the haptic vibration motor. The vibration motor preferably generates a vibration/oscillation in a plane of the various layers (X-direction or Y-direction) and not perpendicular to this plane. An operator then perceives the vibration as pleasant haptic feedback. The deformable spacers can be designed as silicone pads, for example. The deformable spacers are preferably elastic and they also serve in particular to direct the force of a vibration through the layers into the operator's finger and not into the ground underneath the input device.

The individual touch-sensitive screen modules of the input device are preferably installed in a continuous housing. The housing is preferably L-shaped, so that the four screen modules for the index finger, middle finger, ring finger and little finger are arranged in a row next to each other and the other screen module for operation with the thumb is located below the screen module for the index finger.

Preferably, the input device has side magnets that provide releasable couplings to which other input devices of this type can be releasably attached.
Other input device modules can be, for example, input devices for smart buttons, knobs or similar components.
A combined input device is also described here, comprising two input devices according to one of the preceding claims, which are designed to be mirror images of one another and are intended for operation with two hands of an operator.

Two input devices that are mirror-inverted to each other provide the user with operating options that largely correspond to operation with a keyboard, with the input devices offering many other advantageous operating options. The combined input device consists of two input devices. An input device is preferably assigned to each hand of the operator. The operating angle of the input devices to each other can be freely adjusted by the operator to optimize the angle of the wrists. The adjustability of the operating angles prevents injuries such as RSI syndrome. The separability of the combined input device shows increased freedom and flexibility in the choice of posture and additional input devices (e.g. mouse) to improve long-term performance.

• 1: Das beschriebene kombinierte Eingabegerät bestehend aus zwei einzelnen Eingabegeräten für beide Hände eines Bedieners;
• 2: Die Berührungsoberfläche des Eingabegerätes in einer dreidimensionalen schematischen Ansicht;
• 3: Die Berührungsoberfläche des Eingabegerätes in der Profilansicht;
• 4a: Eine Darstellung des Schichtaufbaus eines Bildschirmmoduls des Eingabegeräts in einer Explosionsdarstellung von unten;
• 4b: Eine Darstellung des Schichtaufbaus eines Bildschirmmoduls des Eingabegeräts in ein einer Explosionsdarstellung von oben;
• 5: eine Skizze der Bedienung eines kombinierten Eingabegeräts ergänzt um Erweiterungsbildschirmmodule;
• 6a: ein beschriebenes kombiniertes Eingabegerät mit Tastatur-Interfacelayout;
• 6b: ein beschriebenes kombiniertes Eingabegerät mit Interfacelayout für weitere Symbole und Zahlen;
• 6c: ein beschriebenes kombiniertes Eingabegerät mit Touchpad-Interfacelayout;
• 6d: ein beschriebenes kombiniertes Eingabegerät mit Befehls-Interfacelayout;
• 6e: ein beschriebenes kombiniertes Eingabegerät mit Befehls-Interfacelayout;
• 7: verschiedene Bedienelemente eines Interfacelayouts;
• 8: ein Diagramm eines haptischen Feedbacks bei einem Druck auf das Eingabegerät;
• 9a: Beispielstruktur eines Unter-Bedienmenüs mit dem Symbol-Buchstaben „a“ mit mehreren Ebenen;
• 9b: Beispielstruktur eines Unter-Bedienmenüs mit einem Touchpad zur Bedienung einer Camera-Ansicht;
• 10a: eine erste Variante eines anwendungsspezifischen Interfacelayouts; und
• 10b: eine zweite Variante eines anwendungsspezifischen Interfacelayouts.

The input device is preferably designed with a number of different interfaces, for example: MIDI or MQTT, in order to control loT devices. The various interfaces and their communication protocols can be expanded using adapters. Thus, the input device can be used as a universal control device for various devices (not just computers). The invention and the technical environment are explained in more detail below with reference to the figures. The figures show preferred exemplary embodiments to which the invention is not limited. The figures and in particular the proportions shown in the figures are only schematic. Show it.

• 1 : The combined input device described consisting of two individual input devices for both hands of an operator;
• 2 : The touch surface of the input device in a three-dimensional schematic view;
• 3 : The touch surface of the input device in profile view;
• 4a : An illustration of the layer structure of a screen module of the input device in an exploded view from below;
• 4b : A representation of the layer structure of a screen module of the input device in an exploded view from above;
• 5 : a sketch of the operation of a combined input device supplemented with expansion screen modules;
• 6a : a described combination input device with keyboard interface layout;
• 6b : a described combined input device with an interface layout for additional symbols and numbers;
• 6c : a described combined input device with touchpad interface layout;
• 6d : a described combination input device with command interface layout;
• 6e : a described combination input device with command interface layout;
• 7 : various controls of an interface layout;
• 8th : a diagram of haptic feedback when the input device is pressed;
• 9a : Example structure of a sub-operating menu with the symbol letter "a" with several levels;
• 9b : Example structure of a sub-operating menu with a touchpad for operating a camera view;
• 10a : a first variant of an application-specific interface layout; and
• 10b : a second variant of an application-specific interface layout.

In 1 1 shows the combined input device 23, which consists of two mirrored input devices 1 (one for each hand of an operator). Each input device 1 consists of five screen modules 3, each of which provides a plurality of touch areas 7. Each input device 1 has two screen modules 3 with 3x2 touch areas 7, two screen modules 3 with 3x1 touch areas 7, and another screen module 42 with 2x2 touch areas 7. The screen modules 3 are aligned in such a way that each screen module 3 can be assigned to a finger 2 and can be operated by hand when not in use. Magnets 30 can be arranged in the housing 38 of the input device 1 , which connect the two input devices 1 which are reflected to form the combined input device 23 or to form an expansion screen module 25 . For this purpose, there are contacts 39 on the inside of the input device 1 for charging and for communication between the input devices 1. Other contacts 39 are on the outside of the other screen module 42. On the outside of the input device 1 there is a USB-C port 40 for connection with other devices and for charging. On the screen layer 18 of the screen modules 3, graphic operating elements 41 such as symbols 37, sliders 34, touchpads 35 and joysticks 36 can be displayed. These fill at least one touch area 7, but can also fill several screen modules 3. The arrangement of the screen modules 3 and touch areas 7 are based on the ideal finger position 11 of a resting hand. The finger positions 11 form a so-called "Homerow" for the fingers other than the thumbs. Starting from the finger positions 11 in the "Homerow", each touch area 7 of the input devices 1 can be quickly reached by small movements of the fingers. The further screen module 42 is provided for operation with the thumb and is arranged below the screen modules 3, on whose touch areas 7 the finger positions 11 forming “Homerow” are arranged.

2 shows the ergonomic touch surface 6 of the input device 1, with each touch area 7 being assigned a dent 8. The dents 8 can be localized when typing blindly, since they are separated by an elevation 17. On the indentations 8 for the index finger screen modules 3 there are preferably still bumpy orientation aids 33 for orienting the fingers 2 when typing blindly. These orientation aids 33 enable the operator to quickly find the correct positions of all hands with all fingers on the input devices 1, even without looking.

In 3 the ergonomic touch surface 6 of the input device 1 is shown in profile. The height H between the deep points of the dents 8 for the touch areas 7 and the elevations 17 is between 0.4 and 2 mm. The length L of the dents 8 for the touch areas 7 is 12-19 mm. On the one hand, these dimensions mean that the operator has a very good orientation with his fingers on the touch surface 6 of the input device. On the other hand, these dimensions ensure that a quick change with the finger from one touch area 7 to the adjacent touch areas 7 is possible.

In 4a and 4b the structure of the input device 1 is shown. The input device has a housing 38 into which the respective screen modules 3 are inserted. The structure in the form of layers of a single screen module 1 is shown here in the form of an exploded view. 4a shows a view from below while 4b shows a view from above. On a top side, each screen module 3 has an ergonomic touch surface 6, which has the structure with dents, elevations, etc. described further above. Below this is a multi-touch-capable surface 19, which makes it possible to recognize touches by the operator. Below this is a screen layer 18 which enables the graphical representation of interface layouts. Below the screen layer 18 is a circuit board layer 20 which is movably mounted on the housing 38 with elastically deformable spacers 21 together with all the layers arranged above it (touch surface 6, multi-touch capable surface 19 and screen layer 18). The circuit board layer 20 has a vibration motor 5 and a pressure sensor 24 here. Additional components (not shown separately here) may be present and may be part of circuit board layer 20 or connected to circuit board layer 20 . Due to the mounting of the circuit board layer 20 with the elastically deformable spacers 21, the circuit board layer 20 and the further layers arranged thereon (touch surface 6, multi-touch-capable surface 19 and screen layer 18) are on the one hand downwards (in a z-direction) relative to the housing 38 as also elastically reversibly movable in the plane (in the x-direction and in the y-direction). The possibility of movement in the z-direction primarily serves to allow a movement that allows different pressures to be registered with the pressure sensor 24 . The elastically deformable spacers 21 deform in the z-direction in proportion to the applied force of pressure. The pressure sensor 24 determines the pressure exerted on the circuit board layer by measuring the distance between the housing 38 and the circuit board layer 18. The elastically deformable spacers 21 also serve to do this that the force of a vibration 43 is conducted via the layers into the finger 2 and not into the subsoil under the input device 1.

In 5 shows how the combined input device 23 consisting of two input devices 1 can be combined with expansion screen modules 25 and connected. The two input devices 1, which are designed to be mirror images of one another, together form the combined input device 23. The expansion screen modules 25 can be connected to an input device 1 with detachable couplings 22, such as magnets 30, for example.

the 6a until 6e give various examples of interface layouts 4 that can be displayed with the input devices 1 described in order to provide the operator with operating options.

In 6a the keyboard interface layout 13 is shown. Operating elements 41, such as letters, symbols 37 are shown on the interface layout grid of the input device 1 on the three rows 12 and the six columns 10 per input device 1. Interface layouts 4 such as the keyboard interface layout 13 are displayed on the screen modules 3 and operated. The upper and first group 9 of the screen modules 3 represents actual interface layouts 4. The further screen modules 42 are preferably set up so that you can switch between the interface layouts 4 with your thumb.

In 6b an interface layout 4 of the combined input device 23 is shown for further symbols 37 and numbers. in the other screen module 42 is marked with a thick circle how this interface layout 4 can be selected. Operating elements 41, such as numbers, symbols 37 are shown on the interface layout grid of the input device 1 on the three rows 12 and the six columns 10 per input device 1.

In 6c a touchpad interface layout 14 of the combined input device 23 is shown. in the other screen module 42 is marked with a thick circle how this interface layout 4 can be selected. The touchpad 35 fills the right input device 1 of the first group 9 of the combined input device 23 over all four screen modules 3. The touchpad 35 extends thus also across the boundaries between screen modules 3. A control element 41 is provided on each screen module 3 on the left-hand input device 1 . Here, in combination with the touchpad 35 on the right input device 1, particularly simple operation is made possible. The operating elements 41 here are special operating elements 41 which an operator in combination with a touchpad 35 urgently needs.

In 6d an interface layout 4 of the combined input device 23 for command keys is shown. in the other screen module 42 is marked with a thick circle how this interface layout 4 can be selected. Operating elements 41 are here in each case individual command keys with which hardware connected to the input device 23 can be prompted directly to execute more complex commands. If necessary, such command keys can also be programmed individually.

In 6e an interface layout 4 of the combined input device 23 for PC navigation is shown. In the other screen module 42, a thick circle marks how this interface layout 4 can be selected. Operating elements 41 such as modifier keys 27, indicators 50 and sliders 34 are shown on the interface layout grid of the input device 1 on the three rows 12 and the six columns 10 per input device 1. They can each be assigned special functions of operation.

In 7 various special operating elements 41 such as a slider 34, a touchpad 35, a joystick 36, an indicator 50 and an icon 37 can be seen. Interface layouts 4 consist of combinations of control elements 41.

In 8th is visualized in a diagram how different functions are realized depending on the pressure on an input device or on the touch surface of the input device. An application of this principle is, for example, that a user will recognize a letter as either an uppercase "A" or a lowercase "a" depending on how hard he presses on a control element of the interface layout. A slight pressure movement 31 and a strong pressure movement 32 are plotted here on the force axis over the time axis. The printing movements 31, 32 each take place in the Z-direction. Upon a slight press movement 31, the depressed screen module 3 first generates a vibration 43 at the low end of a gentle press movement 45 to simulate clicking, and at the high end of a gentle press movement 44 it generates another vibration 43 to simulate the click of releasing a key.

In the case of a strong pressing movement 32, the pressed screen module 3 first generates a vibration 43 at the bottom of a light pressing movement 45 and then, upon reaching the bottom of a strong pressing movement 47, another vibration 43 to simulate two-level clicking. When released, the screen module 3 first generates a vibration 43 at the top of a strong pressing movement 46 and then at the top of a light pressing movement 44 to simulate the release on two levels.

The various pressure movements 31, 32 and the associated highs and lows of these pressure movements 44, 45, 46 and 47, which are assigned to specific pressure threshold values, define virtual levels (here a first virtual level 48 and a second virtual level 49) that the user can specifically select and on which the user has certain operating options available.

The input device is set up to recognize the information as to whether the operator wants to select the first virtual level 48 (here the small "a") or the second virtual level 49 below it (here the capital "A") when based on the Form of the temporal pressure movement 31.32 is clearly recognizable whether the operator wanted to select the respective level. If the user wanted to select a level below which further levels can be found (here the level with the small "a"), this is only recognized when the pressure movement 31 returns or the pressure of the operator decreases. This point is in 8th also marked. From this point it is clear that the operator will not go down to a level below. If the user selects the lowest level, the respective selection can already be recognized before the printing movement 32 is reversed. This is marked here by the recognition of the capital "A". Then it is already clear at the point in time at which the operator exceeds the low of the strong pressure movement 47 that the respective level (here the capital “A”) should be selected.

In the 9a and 9b another operating option is shown, which can be used with the in 8th technology shown to switch between different virtual levels 48 and 49 can be implemented.

In 9a is shown as a slight pressure movement 31 in a first virtual level 48 and a strong pressure movement 32 in a second virtual level 49 when pressing an "a" symbol 37 different sub-operating menus 16 for Provide operation with an additional wiping movement 28. If a slight pressure movement 31 is performed with the finger 2 on the touch area 7 with the symbol 37 “a” of a screen module 3, a sub-operating menu 16 with the options “a”, “ä” and “ä” opens. An option can be selected with a swipe movement 28 and operated by releasing it. The submenu is preferably set up in such a way that the pressure movement takes place while maintaining the pressure that the operator has applied in order to get into the respective virtual plane 48, 49. As related to 8th described, the operator arrives in the respective virtual plane 48, 49 when the pressure is applied in the Z direction. The wiping movement then takes place in a plane parallel to the touch surface in the X direction and/or in the Y direction.

In order to generate an "a", the first virtual plane 48 is selected with a slight pressure movement 31 . Without a swiping movement 28, the “a” is then created. In this case, the screen module 3 generates a vibration 43 when it is pressed down at the bottom of a slight pressure movement 45, and the sub-control menu 16 with “a”, “ä” and “a” appears. When released without a wiping movement 28, the screen module 3 generates a second vibration 43 when a slight pressure movement 44 is applied, and an “a” is recognized as a command.

In order to generate an "a", the first virtual plane 48 is selected with a slight pressure movement 31 . A wiping movement 28 is then necessary. The screen module 3 generates a vibration 43 when you press down on the low of a slight pressure movement 45, then a swiping movement 28 in the sub-control menu 16 to the left is performed on the "a" and released at this point, so that when you release the screen module 3 with the high of a slight pressure movement 44 generates a second vibration 43, and "a" is recognized as a command.

A strong pressure movement 32 in the second virtual plane 49 without a wiping movement 28 is required to generate an “A”. In this case, the screen module 3 generates a first vibration 43 when it is pressed down at the bottom of a slight pressure movement 45, and the sub-control menu 16 with “ä”, “ä” and “ä” appears. With further pressing in a strong pressing movement 32, the screen module 3 generates another vibration 43 at the bottom of a strong pressing movement 47 and the sub-control menu 16 with "A", "Ä", and "A" appears. When released without a swiping motion 28, the screen module 3 generates a third vibration 43 at the top of a strong press motion 46 and a fourth vibration 43 at the top of a light press motion 44 and an "A" is recognized as a command.

A strong pressure movement 32 in the second virtual plane 49 with a wiping movement 28 is required to generate an “A”. In this case, the screen module 3 generates a first vibration 43 when it is pressed down at the bottom of a slight pressure movement 45, and the sub-control menu 16 with “ä”, “ä” and “ä” appears. With further pressing in a strong pressing movement 32, the screen module 3 generates another vibration 43 at the bottom of a strong pressing movement 47 and the sub-control menu 16 with "A", "Ä", and "A" appears. A swiping movement 28 in the sub-control menu 16 to the right is then performed on the “A” and released. When released at the position after the swipe motion 28, the screen module 3 generates a third vibration 43 at the top of a strong press motion 46 and a fourth vibration 43 at the top of a light press motion 44 and an "A" is recognized as a command.

In 9b shows how this principle can be used in connection with the touchpad 35, for example for camera operation. A normal touch operation using a wiping movement 28 and a slight pressing movement 31 to switch to a first virtual plane 38 can be used to perform different functions when operating a camera using the touchpad 35 . The camera rotation position can be rotated within an application program 29 with swiping movements 28 over the touchpad 35 operating element 41 without relevant pressure. The screen module 3 does not generate any vibration 43 for click simulation. When performing wiping movements 28 within the slight pressing movement 31 after the first vibration 43 after reaching the bottom of a slight pressing movement 45 (for changing to a first virtual plane 48), the zoom function of the camera position is operated. When the light pressure movement 31 is released, the screen module generates a second vibration 43 at the top of a light pressure movement 44 and the camera rotation position can be operated again with wiping movements 28 .

In 10a a special interface layout 15 for an application program 29 such as "Ableton" is shown. The input device 1 automatically switches to the appropriate special interface layout 15 as soon as the application program 29 is running in the foreground. The special interface layout 15 for the music production application program Ableton contains relevant operating elements 41 such as sliders 34 for volume, a touchpad 35 for filter settings, and various drum pads 26.

In 10b Another special interface layout 15 for an application program 29 such as "Photoshop" is shown. The input device 1 automatically switches the appropriate special interface layout 15 as soon as the application program 29 is running in the foreground. The special interface layout 15 for the photo editing application program Photoshop contains relevant control elements 41 such as slide controls 34 for lighting or a touchpad 35 for the color setting.

Reference List

1

input device

2

finger

3

screen module

4

interface layout

5

vibration motor

6

touch surface

7

tactile range

8th

dent

9

first group

10

Split

11

finger position

12

Row

13

Keyboard interface layout

14

Touchpad interface layout

15

Special interface layout

16

sub-operating menu

17

surveys

18

screen layer

19

multi-touch capable surface

20

board layer

21

deformable spacers

22

detachable couplings

23

combined input device

24

pressure sensor

25

Expansion screen module

26

drum pad

27

modifier key

28

swipe motion

29

application program

30

magnet

31

Slight pressure movement

32

Strong pressure movement

33

guidance

34

slider

35

touchpad

36

joystick

37

symbol

38

Housing

39

contacts

40

USB-C port

41

control element

42

Another screen module

43

vibration

44

High of a light pressing motion

45

Low of a slight pressing motion

46

High of a strong pushing movement

47

Low of a strong pressure movement

48

first virtual level

49

second virtual level

50

indicator

H

height of the dents

L

Longer the dents

QUOTES INCLUDED IN DESCRIPTION

This list of documents cited by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent Literature Cited

• US5689253A [0004]
• WO 2017019390 A1 [0005, 0010]
• US8896539B2 [0005]
• DE 112009003647 T5 [0005]
• US2009167694A1 [0008]
• US 9335844 B2 [0009]

Claims (12)

Input device (1) for operation with the fingers (2) of an operator, having at least two touch-sensitive and pressure-sensitive screen modules (3), each of which enables precise position detection of a touch by the operator's finger (2) and for displaying graphical interface layouts (4) are set up, wherein the touch-sensitive and pressure-sensitive screen modules (3) are each designed with at least one vibration motor (5), the input device (1) being set up for this purpose when the respective screen module (3) is touched with a finger (2) of the operator to provide haptic feedback. input device (1) after claim 1 wherein a touch surface (6) of each touch-sensitive screen module (3) is divided into at least two touch areas (7), the touch surface (6) for forming the touch areas (7) having an orientation structure that can be felt by the fingers of the operator. Input device (1) according to one of the preceding claims, wherein a first group (9) of touch-sensitive screen modules (3) is provided for operation with the index finger, middle finger, ring finger and little finger, which are arranged in a row next to one another and underneath a further touch-sensitive screen module (3, 42) for thumb operation. input device (1) after claim 3 , wherein the touch-sensitive screen modules (3) of the first group (9) each have at least three rows (12) of touch areas (7) arranged one above the other. input device (1) after claim 3 or 4 , the first group (9) comprising exactly four touch-sensitive screen modules (3), each of the fingers (2) index finger, middle finger, ring finger and little finger being assigned exactly one touch-sensitive screen module (3), the touch-sensitive screen modules (3) for Index finger and little finger each have two columns (10) of touch areas (7) arranged next to one another and the touch-sensitive screen modules (3) for the ring finger and middle finger each have exactly one column (10) of touch areas (7) arranged next to one another. Input device (1) according to one of claims 3 until 5 , wherein a selection of graphical interface layouts (4) of the first group (9) of touch-sensitive screen modules (3) can be made by operating actions on the further touch-sensitive screen module (3, 42) with the thumb. input device (1) after claim 6 , wherein at least one keyboard interface layout (13) and at least one touchpad interface layout (14) can be displayed on the first group (9) of touch-sensitive screen modules (3), and a change between the at least one keyboard interface layout (13) and the at least one touchpad interface layout (14) by confirming the further touch-sensitive screen module (3, 42) with the thumb. input device (1) after claim 7 , wherein at least one special interface layout (15) can also be displayed on the first group (9) of touch-sensitive screen modules (3) and a change to this special interface layout (15) by confirming the further touch-sensitive screen module (3, 42) can be done with the thumb, wherein the at least one special interface layout (15) provides at least one of the following functions: - a layout comprising at least one slider; - an arrow key layout; and - at least one custom interface layout. Input device (1) according to one of the preceding claims, wherein by exerting a predetermined pressure on the touch-sensitive screen modules (3) at screen positions predetermined by the interface layout (4), a change to at least one sub-operating menu (16) can take place, in which between Sub-functions can be switched to the functionality provided at the screen position, with sub-functions being able to be selected in the sub-control menu (16) by wiping movements (28) with the finger (2). Input device (1) according to one of the preceding claims, wherein the touch-sensitive and pressure-sensitive screen modules (3) have a layered structure with at least the following layers: - an ergonomic transparent touch surface (6) which provides dents (8) and elevations (17). that offer an operator haptic orientation, - a screen layer (18), - a multi-touch capable surface (19), - a circuit board layer (20) at least with a vibration motor (5) and a pressure sensor (24), and - a common Housing (38) for multiple screen modules (3), on which the touch surface (6), the screen layer (18), the multi touch-enabled surface (19) and the circuit board layer (20) are movably mounted with at least one elastically deformable spacer (21). Input device (1), the individual touch-sensitive screen modules (3) being connected to one another by detachable couplings (22). Combined input device (23) comprising two input devices (1) according to one of the preceding claims, which are designed to be mirror images of one another and are intended for operation with two hands of an operator.

Images