EP3140083A1

EP3140083A1 - Operating device and control system

Info

Publication number: EP3140083A1
Application number: EP15728382.1A
Authority: EP
Inventors: Helmut Bugl; Benjamin HACKL; Semir HAMZIC; Andreas MÜLLER; Thomas Wurm
Original assignee: Keba AG
Current assignee: Keba AG
Priority date: 2014-05-09
Filing date: 2015-05-05
Publication date: 2017-03-15
Also published as: CN106457564B; AT515719A1; JP6629759B2; WO2015168716A1; JP2017525000A; CN106457564A

Abstract

The invention relates to an operating device (2) for producing control commands for the control device (3) of a machine or installation (4), characterized by an evaluation device (13) that is connected to the touch sensor (8) of a touchscreen (5) and that comprises a real-time data-processing device (14) and an output interface (16), wherein the real-time data-processing device (14) of the evaluation device (13) is arranged to take the sensor data from the touch sensor (8) as a basis for generating control commands for the control device (3) and providing said control commands on the output interface (16), the real-time data-processing device (14) of the evaluation device (13) being independent in relation to the data-processing device (11) of a visual display device (10). The invention also relates to a control system and to a method for controlling a machine or installation.

Description

Operating device and control system

The invention relates to an operating device according to the preamble of claim 1, a control system according to the preamble of claim 19, a method for controlling a machine or system, and a method for creating a graphical user interface for the visualization device of an operating device and / or a control system. The problem underlying the invention is that so far the inputs on the touch screen of an HMI device (e.g., control panel or handheld terminal) have not been satisfactorily, i. only limited real-time capable detected and can be forwarded in real time as control commands to a machine control. In the case of operator actions on virtual operating elements or control panels of the touchscreen for triggering movement movements or other actions, delayed reactions of the machine can occur, from which a significantly deteriorated operability and, occasionally, a certain hazard potential, at least with regard to damage to machine components, Tools or workpieces can result. For this reason, additional discrete operating elements (e.g., mechanical keys) whose actuation state is detected and detected directly, i.e., mechanically driven, in addition to the touch screen, have heretofore been provided for directly performing traversing movements of machine axes. not transmitted via the computer unit of the HMI device to the machine control.

The HMI device (control panel or handheld terminal) has a processor and memory that performs software both for displaying information to the user and for detecting inputs by the user and communicating with the controller of a machine via a suitable data interface and exchanges information as well as sends commands to the machine control. With the aid of this software even complex process diagrams, machine graphics, process data, etc. can be graphically presented and animated on a case-by-case basis. The essential parts of this software are usually an operating system (eg Windows or Linux) with integrated support for graphical user interfaces under which then a device-specific basic software for the control panel or the handheld terminal (eg drivers, function libraries) and a machine software. specific application software will be executed. In principle, machine-specific software components for the operation and visualization can also be executed only on the machine control itself and the communication between the HMI device and the machine control can take place in a generally held format, eg in the manner of an HTTP server in the machine control and a browser on the HMI device.

The device-specific basic software is developed and provided by the manufacturer of the HMI device. The machine-specific application software is usually developed and provided by the manufacturer of the machine or system in which the HMI device is integrated, whereby the machine manufacturer uses software tools and libraries of the HMI device manufacturer to create this machine-specific software, in particular the screen masks ,

The use of standard operating systems on such HMI devices facilitates the creation of the machine-specific software and the graphical user interface through the use of numerous available and widely used software tools, standardized programming interfaces, etc. as well as generally integration into the industrial control environment. The use of license-free, open operating system platforms is generally sought because of their wide distribution, the disclosed source code and cost reasons.

However, there is the problem that these operating systems are not or not sufficiently real-time capable and thus a guaranteed and sufficiently short response time to operator actions can not be achieved. This applies in particular to those parts which carry out the recording and evaluation of touch inputs. Also, errors or inconveniences in, for example, the application software may result in certain parts of the operating system or the base software being temporarily or permanently no longer being executed properly or delayed, thereby compromising the reliability of the operating functions.

These difficulties mean that not really time-critical machine settings (parameter settings) can be made or changed via the virtual operating elements on a touch screen, but that for the immediate triggering of process parameters. Movements or other actions, such as when setting up or programming of movements, still additional conventional controls, such as mechanical buttons or membrane keys, are provided adjacent to the touch screen, via which the operation s information detected and transmitted directly and without delay to the control device of the machine can be. This is usually done via a direct coupling of such controls to the machine control.

The object of the invention is to eliminate the disadvantages of the prior art and to provide an operating device or a control system, on the one hand, from which the advantages of using the widely used (open) operating systems and the tools for creating and configuring remain graphical user interfaces (eg their powerful libraries for the graphical output), but still a reliable real-time capable evaluation of touch operations including forwarding to the control device of the machine or system is enabled. The use of additional controls next to the touch screen should be avoided.

This object is achieved with an operating device of the type mentioned at the beginning, by an off-value device which is connected to the touch sensor of the touch screen and which comprises a real-time data processing device and an output interface, wherein the real-time data processing device of the output device is set up to operate in dependence to generate control commands for the control device and to provide them at the output interface, the real-time data processing device of the evaluation device being independent of the data processing device of the visualization device.

The generation and provision of the control commands for the control device at the output interface takes place uninfluenced by the time of the data processing device of the / a visualization device. The time sequence or the speed of the data processing in the data processing device of the value device is independent of the data processing device of the visualization device. The data processing in the data processing device of the value device is therefore independent of a processing process running on the data processing device of the visualization device. The generation of the control commands in the data processing device of the evaluation unit, its provision or transmission to the control device can thus be done by bypassing the data processing device of the visualization device or the interface for the visualization device.

The generation of real-time-critical control commands for the control device takes place according to the invention separately from the visualization of the control elements by a visualization device. The visualization device may be located within the operator device (e.g., in the form of a handheld device) or external to it. The evaluation device or the data processing device of the evaluation device forms, with reference to the visualization device or the data processing device of the visualization device, a bypass path (bypass) between the touch sensor and the control device, so to speak , This bypass path allows the processing of sensor data and the provision of control commands in real time and thus enables real-time control of a machine or plant, bypassing the visualization device or its data processing device. In other words, the evaluation device and the visualization device, or at least their data processing devices, are connected in parallel between the touch sensor and the control device. The data processing means of the evaluator and the visualizer may be separate (i.e., stand-alone) processors, or may be functionally separate (i.e., stand-alone) cores of a processor unit.

In one possible embodiment, a common, real-time-capable bus for the control device, visualization unit and evaluation unit can be provided and, as it were, looped through to the evaluation unit by the visualization unit. It is also conceivable that the visualization device and the evaluation device are structurally combined and have a common real-time capable data bus (or mixed: real-time capable and non-realtime). real-time capable) to the controller have out. Such variants do not change the basic principle that the data from the data processing device of the evaluation device (temporally) uninfluenced by the data processing device of the visualization device reach the control device via the common bus.

In any case, an independent output interface (not shared with the visualization device) of the value-value device and a separate real-time-capable data connection are preferably transmitted to the control device. The visualization device can be used in addition to the visualization function, i. Providing output data / image data for the display (as before) for generating and providing non-real-time control commands or machine or system parameters. The control system according to the invention can therefore, in addition to the real-time capable (bypass) path, which is formed by the evaluation device or its data processing device, comprise a non-real-time-capable path, which is formed by the visualization device. Both paths lead to the control device.

The invention thus provides a real-time-capable operating device and a real-time-capable control system for at least part of the control commands sent to the control device. Under real-time is to be understood in particular that from corresponding (possibly by a touch controller processed) sensor data of the touch sensor within a predetermined period of time, in particular within a guaranteed short period of time, or a predetermined time grid corresponding control commands are provided at the output interface of Aus Werteinrichtung and thus also in real time to the control device can be transmitted.

An operating element in the sense of the present invention is an operating element of the touchscreen. It is thus a touch-sensitive control element, which is visualized on the display of the touch screen and is actuated by touch. The operating element can also be referred to as a control panel which occupies a spatially predetermined area on the touchscreen. The real-time evaluation device preferably has a processor, memory means, an input interface for reading in the sensor data and an output interface for outputting control actions corresponding to operator actions on the touchscreen. Preferably, the off value device is in the communication connection between the touch sensor or a downstream touch controller, which detects the touch operator actions based on the sensor signals and provides raw data to the operator actions, eg coordinate pairs to the detected touch points, and the visualization, for the visualization and non-real-time-capable operation is responsible.

The evaluation device reads the sensor signals of the touch sensor (which were optionally converted by the touch controller into sensor raw data and preferably also sent to the visualization device). At the same time, the visualization device of the evaluation device can provide information about the position and the type as well as possibly about the release state of the control elements or control panels shown on the display (configuration data).

From the sensor data and from configuration data for the operating elements, the value-setting device independently and in real-time determines the operating states of the individual control elements on the touchscreen, generates corresponding control commands (or actuating information) and transmits these via the output interface to the control device without involvement of the visualization device. to the machine or plant control. The software of the evaluation device is real-time capable and comparatively simple, slim and reliable.

In parallel, the visualization device may receive the same sensor data on the various touch events, e.g. from a touch controller or via the evaluation device, and these for non-real-time critical operations, e.g. Parameterization, evaluation.

It is essential that the off value device of the visualization device or - if the visualization device is located outside the control device - of the Interface for a visualization device is functionally separated and is equipped with a standalone processor unit and stand-alone operating software and has its own output interface for the transmission of control commands to the control device of the machine or system.

In an advantageous development, the visualization device can indeed configure the value device via a configuration interface and thus change its mode of operation, e.g. the nature, number and position of virtual controls, but only within a narrow and defined framework, and without compromising the real-time responsiveness of the evaluation device.

Due to the fact that the evaluation device evaluates the touch operator actions autonomously and independently of the visualization device, but in addition does not need to take on additional computationally time-consuming tasks, the evaluation of the operating processes is carried out within a defined narrow time frame, i. in real time, possible. Due to the direct forwarding of the control commands to the control device, bypassing the visualization device, preferably by direct connection via a real-time communication bus, the immediate and instantaneous execution of machines or system functions, in particular traversing movements is possible.

Since the software of the evaluation device is developed and executed independently of the visualization software of the visualization device, it is normally created and carefully checked by the manufacturer of the HMI device and then no longer changed (at least not in the context of the comparatively frequent and varied adaptation of the visualization software a particular machine by the machine manufacturer or an end user), high reliability and safety of the real-time operating functions can be ensured.

Preferably, the evaluation device is connected to the visualization device via a preferably bidirectional data connection.

The visualization device is preferably set up in order to configure the configuration data concerning at least one control element visualized on the display To provide, wherein the configuration data preferably contain information about the position, size, orientation, type, associated machine or system function, the associated machine or system parameters, the release state and / or the current operating state or set value of the at least one control element, and the evaluation device is set up to generate the control commands for the control device as a function of the configuration data.

The visualization device is preferably connected at least indirectly to the touch sensor and to the control device and configured to generate control commands and / or machine or system parameters for the control device as a function of control elements of the touchscreen operated by an operator.

The sensor data connection between the touch sensor and the off-value device preferably comprises a branch leading to the visualization unit or to the interface for the visualization device.

A preferred solution is characterized by a point-to-point connection between the touch controller and the evaluation device, wherein the sensor data (touch data) are forwarded by the evaluation device to the visualization device. In this case, the evaluation device is, as it were, interposed between the touch controller and the visualization device.

In a further embodiment, the operating device is a mobile, preferably portable operating device which can be connected to the control device of a machine or system via a data connection, in particular via a flexible line or a radio link.

Preferably, the operating device comprises at least one actuator for generating a haptic signal for the operator and the evaluation device is connected to the actuator and adapted to actuate the at least one actuator in dependence on the sensor data of the touch sensor. This haptic feedback can vary with the type of control, with the operator, the location, the operating state or the release state of the control. In this way, the operator receives a clear feedback whether an operator action was registered as such, or already in Advance whether a particular virtual control is touched (to find a specific control without visual control).

A preferred embodiment is characterized in that the output interface of the evaluation device, are transmitted via the control commands to the controller, at least one real-time control output or at least one digital or analog control output, preferably the output interface comprises at least two control outputs, each control output a different machines - or plant-related functions is assigned. This allows reliable real-time transmission to the control device. Instead of connecting the evaluation device to the controller via a real-time-capable communication interface, direct digital or analog control outputs on the evaluation device would also be conceivable (i.e., a separate signal line per machine or system function, for example). As a result, a technically particularly simple, fast and interference-free signal connection of an HMI device to one or more control devices or occasionally also directly to actuators or their actuators is made possible.

Preferably, the touch sensor is a multi-touch sensor and the off value device is configured to evaluate sensor data from the multi-touch sensor, wherein preferably the operation of at least two control elements can be evaluated simultaneously. This expands the functionality of the operating device and increases its reliability. A multi-touch sensor is also already important in connection with merely one control element, since, for example, an activation of a control element by means of a two-finger gesture can prevent unwanted false tripping.

A preferred embodiment is characterized in that the evaluation device is adapted to a movement or contact pattern on the touch sensor, which is performed before, during and / or after the actual operation, which is performed by operating a control element, and for the Execution of a control process is absolutely necessary to check and only after a positive check of this motion or touch pattern to provide a control operation corresponding control command at the output interface, preferably the movement or Berüh- is a preliminary gesture for activating the control element for a certain period of time or the simultaneous actuation of another control element.

For activating the controls, special gestures, i. in relation to the actual actuation process necessarily additional introductory, accompanying or final movement or contact pattern be provided on the touch sensor, whose execution is checked by the evaluation and so the risk of unwanted operation in the course of accidental or unintentional contact of the controls is reduced. It is also possible to provide functions and gestures which temporarily block the triggering or actuation of all operating elements, for example in order to be able to temporarily clean the entire touch surface without triggering unwanted operating processes.

A preferred embodiment is characterized in that between the touch sensor and the off value device a touch controller is connected, which is set up to detect the touches on the touch screen or actuation of the control elements of the touch screen and as sensor raw data provide. The sensor raw data may contain, for example, coordinate pairs for describing the position of one or more touch points. In this context, it is mentioned that the sensor raw data are already processed data, which are obtained from the sensor signals.

For the touch controller, normally the entire touch sensor is a unitary area, i. a location-resolving sensor. Whether and where there are control elements (virtual controls), the touch controller does not "know". Such assignment or checking takes place only in the off value device or in the visualization device.

A possible embodiment is characterized in that the evaluation device is designed in such a way or connected to the data stream of the touch controller that the data stream is supplied in the same way to both the value device (13) and the visualization device (10). In this embodiment, the visualization device is also connected to the touch sensor and thus connected to the sensor data stream independently of the evaluation device. In this way, the evaluation device can store the sensor data relating to an actuation of real-time-capable virtual operating Evaluate elements while the visualization device, the sensor data regarding non-real-time relevant inputs, such as swipe gestures to switch between different screens, can evaluate. It may sometimes depend on which of the three units is the master, ie causes the transfer of data. It might be that the out-of-value device reads passively, but it could also passively read the visualization device or even both (if the touch controller is the master and simply sends whatever data is required). Preferably, the touch controller is structurally and functionally integrated in the evaluation device. As a result, the operating device can be dimensioned more easily with regard to the data connections, which also results in cost savings.

A preferred embodiment is characterized in that the touch controller and the evaluation device in addition to the sensor data connection, are transmitted via the sensor raw data to the evaluation, via a communication link, preferably in the manner of an interrupt signal line, interconnected via that the touch controller of the evaluation asynchronously and without delay can signal the presence of a relevant for the evaluation operation. This reduces the response time of the evaluator to a touch event and can facilitate the maintenance of the real-time condition (i.e., a guaranteed response within a defined short period of time).

A preferred embodiment is characterized in that the evaluation device has at least three interfaces, wherein sensor raw data are received via a first interface and forwarded via a second interface to the visualization device or to the interface for the visualization device, wherein preferably the evaluation device on the second interface, the behavior of a touch controller at least partially emulated. The behavior of the 'emulated touch controller' may differ slightly from the actual touch controller that is present at an input interface of the evaluation device. Although this fundamentally allows prefiltering of the sensor data by the off-value device, it is possible that only those sensor data relevant to the visualization device are forwarded to it such does not take place if individual pairs of coordinates are not necessarily clearly assignable to a real-time-capable or a non-real-time-relevant actuation process. The possibility of such an assignment arises only from a contiguous sequence of coordinate pairs that describe, for example, a movement or a gesture with a specific starting or end point. However, this must determine and evaluate both the off-value device and the visualization device separately and therefore each of the two units requires the complete data stream from the touch controller. The reason for the deviant behavior of the emulated touch controller to the actual touch controller may be that there are a variety of different touch controllers on the market with slightly different interfaces and protocols. When using different controllers in different product variants (eg display sizes), only the evaluation device has to be adapted, while in the direction of the visualization device a certain standard touch controller is always emulated and no adjustments are necessary there.

Another reason is that certain settings or parameters of the actual touch controller may only be specified by the value device and may not be re-adjusted by the visualization device (this could impair the reliability of the evaluation device). Such parameterizations by the visualization device must therefore be intercepted by the value device.

The off-value device preferably comprises at least one memory connected to the real-time data processing device. The additionally required (configuration) data for the real-time evaluation can be retrieved directly from the memory. The real-time processing is thereby guaranteed, since the evaluation device is not dependent on the timely transmission of such data from other units. A preferred embodiment is characterized in that configuration data relating to at least one display element can be contained in the memory and / or stored, the configuration data preferably containing information about the position, size, orientation, type, associated machine or system function to- ordered machine or system parameters, the release state and / or the current operating state or set value of the at least one control element included.

As a result, a rapid and above all unambiguous generation of corresponding control commands can take place.

A preferred embodiment is characterized in that in the memory calibration information contained and / or can be stored, with which the coordinate information supplied by the touch controller are corrected, preferably in terms of offset, scaling and / or equalization. A faulty operation is thereby avoided. An adaptation to copy-dependent deviations of the sensor parameters is basically possible (eg, each copy of a touch may behave somewhat differently due to process variations in production), so that the operation of this embodiment is possible independently of such specimen scatters on each control panel. The storage means can thus contain (also in addition): configuration data for the position or position of at least one operating element relative to the dimensions or coordinates of a touchscreen; Configuration data describing the release state (ie whether an actuation can ever lead to an activation of a machine or system function or whether the relevant control element is blocked) by at least one operating element; Configuration data that allows an assignment of at least one operating element to a specific machine or system function or a machine or system parameters; Status information describing the current actuation state or set value of at least one control element. The control elements that can be detected by the evaluation device preferably include digital actuation elements in the manner of touch-sensitive buttons, on-off switches, two-stage or multistage slide switches or rotary switches or other switches or the like. The control elements that can be detected by the evaluation device preferably comprise analogue or quasi-analogue or finely resolving actuating elements in the form of one- or two-dimensional sliders, joysticks or turntables (handwheels), potentiometers, touch pads, each with or without automatic return to one starting position. The object is also achieved with a control system for controlling a machine or plant, in particular a manipulator, a processing device or a production plant, by means of an operating device, comprising: a control device for controlling the machine or system; an operating device connected to the control device, in particular according to one of the preceding claims, for operating the machine or system by an operator, wherein the operating device has a touch screen, which is formed from a display for visualization of control elements and a touch sensor superimposed on the display; a visualization device connected to the display for providing image data for the display of the touchscreen, wherein the visualization device comprises a data processing device; characterized by an evaluation device which is connected to the touch sensor of the touchscreen and which comprises a real-time data processing device and an output interface connected to the control device, wherein the real-time data processing device of the Auswerteinrichtung is adapted to control the sensor data of the touch sensor control commands for the control device and to be transmitted via the output interface to the control device, wherein the real-time data processing device of the value device is independent with respect to the data processing device of the visualization device.

The generation and provision of the control commands for the control device at the output interface takes place temporally uninfluenced by the data processing device of the visualization device. The data processing in the data processing device of the value device is therefore independent of a processing process running on the data processing device of the visualization device. The generation of the control commands in the data processing device of the evaluation unit, its provision or transmission to the control device can thus be carried out bypassing the data processing device of the visualization device. Preferably, the output interface of the evaluation device is connected to the control device via a real-time capable data bus (eg SERCOS, Profinet, EtherCAT, Varan, PowerLink, EtherNet / IP or other Real Time Ethernet connections).

The operating device is preferably a structural unit separate from the control device, in particular a mobile, preferably portable operating device, and the evaluation device is integrated in this structural unit.

In a further embodiment, the visualization device is arranged outside the operating device, wherein preferably the visualization device is integrated in the control device.

The object is also achieved by methods for controlling a machine or plant by means of an operating device and / or by means of a control system according to one of the embodiments described above, wherein the visualization device provides image data for the display of the touch screen so that controls are visualized on the display and wherein the Real-time data processing device of the value device as a function of the sensor data of the touch sensor of the touch screen generates control commands and transmitted to the control device. The generation and transmission of the control commands takes place temporally uninfluenced by the data processing of the visualization device. The time sequence or the speed of the data processing in the data processing device of the evaluation device is independent of the data processing device of the visualization device.

A preferred embodiment is characterized in that configuration data relating to at least one control panel shown or displayable on the display, preferably in the form of a parameter set, are loaded into the evaluation device, the configuration data preferably containing information about the position, size, orientation, type assigned machine or system function, the assigned machine or system parameter, the release state and / or the current actuation state or set value of the at least one operating element, and that the value device generates the control commands for the control device as a function of the configuration data. The configuration data are preferably generated by the visualization device and / or by the control device and transmitted to the evaluation device.

A preferred embodiment is characterized in that the evaluation device receives information from the visualization device, preferably via regular communication or signaling, from which conclusions about the correct or incorrect execution of the visualization software running on the data processing device of the visualization device can be obtained. The evaluation device can thus assume a watchdog function, whereby the generation of false, i. can not be avoided by the operator unintended control commands.

A preferred embodiment is characterized in that the configuration data relating to an operating element, which are transmitted from the visualization device and / or control device to the evaluation device, are / are provided within the evaluation device with a time-limited validity, and that the corresponding operating element is deactivated after the expiration of validity, so that operation of the machine or system is disabled via this control. Thus, the parameterization of the virtual control elements and / or their activation status (ie release for issuing control commands) transmitted by the visualization device to the evaluation device can be provided with a time-limited validity within the evaluation device. After their timing, the activation status is set to "not activated" and thus the command output is disabled via the respective control element.Also, this is a special execution of a watchdog function.A preferred embodiment is characterized in that the visualization device via a Data connection between the evaluation and visualization device reads out the operating states of the monitored by the evaluation controls and with respect to non-real-time critical functions, such as for an optical, acoustic or tactile feedback to the operator, evaluates. Thus, the evaluation of tasks is kept free, which are not subject to the real-time requirement. The real-time reliability is thus increased or easier to implement. A preferred embodiment is characterized in that in the evaluation device after changing an operating state of a control element, a timer is started with a predetermined expiry time and that an error signal is sent to the controller or a control element is set in a deactivated state when the visualization device is not before the expiry of the expiration time with a confirmation information answers, which refers to a completed update of the information displayed on the touch screen, in particular regarding the operating state of the control element. It can be started specifically for each control a separate timer or it can alternatively be provided a common timer for all controls. The advantage of this embodiment is that the evaluation device responds without delay to actuation processes, but also recognizes when the visualization does not respond to such actuation processes within a reasonable time. In this way, a movement can be aborted if the discrepancy between the actual and visually displayed actuation or switching state lasts too long and, for this reason, misunderstandings can occur for the operator.

A preferred embodiment is characterized in that the evaluation device is signal-technically coupled to one or more actuators in such a way that, upon actuation of a control element or already when the touch screen is merely touched, a haptic feedback is generated to the operator.

The positions and the type of operating elements are preferably stored in the evaluation device and preferably correspond with existing haptic markings on the touchscreen. The positions and the type of controls may be fixed in the evaluation device. In this case, the parameterization on the part of the visualization software can be limited to the activation or deactivation of the operating elements (virtual operating elements) and optionally to the presetting of the machine function associated with a specific operating element as a function of the screen mask just shown. A preferred embodiment is characterized in that a deactivation of a monostable operating element, preferably as a virtual pushbutton or virtual joystick with automatic return to an unactuated starting position, regardless of the actual operating state of the control element to the undelayed signaling of an unactuated state by the Visualisierungseinrich- leads to the evaluation device or to a control command corresponding to the unactuated state from the value device to the control device. As a result, faulty control can be avoided. Preferably, an additional data connection is provided between the evaluation device and the visualization device, preferably in the manner of an interrupt line, with which the value device signals an operating procedure of an operator to the visualization device. This information can convert the visualization device into a displayable feedback on the display, in particular a change in the representation of the relevant control element.

A preferred embodiment is characterized in that a real-time bus connection, for example a real-time Ethernet bus connection, is provided for transmitting the real-time control commands, via which the operating states of the control elements are cyclically transmitted in a predetermined time grid to the control device. This also ensures real-time transmission from the evaluator to the controller, i. for machine or system control.

The object is also achieved with a method for creating a graphical user interface for the visualization device of an operating device and / or a control system according to one of the embodiments described above by means of a development environment, wherein the development environment at least one control element is selected from a predetermined set of available controls and this at least one operating element is parameterized with regard to its position, size, orientation, size, type, assigned machine or system function, its assigned machine or system parameter and / or its release state, and wherein from the assignment of the parameters to the at least one Bediendeld configuration data, in particular in the form of a parameter record, are generated for the evaluation. The development environment preferably itself has a graphical user interface with which the various operating elements can be selected, placed and parameterized in a simple and intuitive manner, and their subsequent appearance on the operating device can be visually checked in advance.

The invention also relates to a method for operating a real-time evaluation device, characterized in that configuration data concerning at least one operating element which can be evaluated by the evaluation device is loaded from the visualization device or from the control device into the evaluation device, wherein the configuration data preferably contains information about the position, size, Orientation, type, associated machine or system function, the associated machine or system parameters, the release state and / or the current operating state or set value of the at least one control element included. The configuration data can be transmitted during the initialization phase of the operating device or else during operation for adapting the configuration data to different operating states.

For a better understanding of the invention, this will be explained in more detail with reference to the following figures.

In each case, in a highly simplified, schematic representation: FIG. 1 control system of a machine or system; 2 shows an operating device according to the prior art; Fig. 3 A control system according to the prior art; 4 shows an operating device according to the invention; a control system according to the invention;

6 shows a variant of a control system according to the invention; 7 shows a further variant of a control system according to the invention;

8 shows an evaluation device in detail;

9 shows a memory content of the value means in a schematic representation;

By way of introduction, it should be noted that in the differently described embodiments, the same parts are provided with the same reference numerals or the same component designations, wherein the disclosures contained in the entire description can be mutatis mutandis to the same parts with the same reference numerals or component names. Also, the location information chosen in the description, such as top, bottom, side, etc. related to the immediately described and illustrated figure and these position information in a change in position mutatis mutandis to transfer to the new location.

DESCRIPTION OF THE FIGURES

The embodiments show possible embodiments of the operating device or the control system, it being noted at this point that the invention is not limited to the specifically illustrated embodiments thereof, but also various combinations of the individual embodiments are mutually possible and this variation possibility due to the teaching technical action by objective invention in the skill of working in this technical field expert. Furthermore, individual features or combinations of features from the different exemplary embodiments shown and described can also represent independent, inventive or inventive solutions.

The task underlying the independent inventive solutions can be found in the description. Above all, the individual embodiments shown in the figures can form the subject of independent solutions according to the invention. The relevant objects and solutions according to the invention can be found in the detailed descriptions of these figures. For the sake of the order, it should finally be pointed out that for a better understanding of the construction of the invention, these or their components have been shown partially unevenly and / or enlarged and / or reduced in size.

Figure 1 shows a typical field of application for the invention, an injection molding machine including control and an HMI device in the manner of an integrated into the machine control panel. In the figure 1, the control panel is both integrated into the machine and shown pulled out for clarity. The control of the machine is also shown outside the actual machine construction for better clarity of the signal flow, but in practice is often accommodated directly in or in the machine in a control cabinet. The machine controller controls the flow of energy to the various actuators of the machine in response to a predetermined manufacturing program, signals from the sensors representing the current state of the machine and the manufacturing process being performed, and commands and parameters set by an operator through the machine also be connected to the machine control associated control panel. For this purpose, a machine control typically has at least one CPU, memory means for storing a program and for data, various interfaces for connection to the machine, their actuators and sensors and for connection to input and output devices such as a stationary and / or a mobile control panel or for the connection to a network for remote access to data, programs and functions of the machine.

The control panel may be stationary, ie be integrated into the structure of the machine, or be designed as a mobile handheld terminal, which is technically in operative connection via a flexible cable connection or via a wireless radio link with the machine control. FIG. 1 shows only a stationary operating panel. An operating panel has output means in the form of a screen, preferably a high-resolution color display, via which or which information about the machine, its operating state and the executed manufacturing process can be output to an operator.

Furthermore, a control panel has a number of input means or controls by which the operator can change parameters of the machine, select information about the machine or process for output, switch operating modes, start and stop automatic machining processes, and machine functions, e.g. It is also possible to manually trigger traversing movements directly and without delay.

2 shows an enlarged and exemplary representation of a front view of an operating device 30 in the form of a control panel according to the prior art. It has a first operating area 31 which consists of a touch-sensitive screen (a touch display), i. a structural overlay of a high-resolution color display, and a transparent, touch-sensitive sensor is formed. In this first operating area 31, the output of information to the operator and inputs of the operator, for example, to adapt the information displayed and to adjust the operation and process parameters of the machine.

In this first operating area 31, a large number of virtual operating elements and input fields can be displayed and hidden depending on the situation and used. The inputs and outputs in this first operating area 31 are processed or processed by a visualization device which is signal-connected to the touch display. In particular, information is read out from the machine or system controller by the visualization device and prepared for output to the operator, and conversely, for example, updated parameter values are transmitted to the machine controller.

The visualization device (or visualization computer) typically has processors, memory means and interfaces, as are well known to those skilled in the art. For example, an operating system with appropriate support for graphical user interfaces, such as Windows or Linux, is used on the visualization device Use under which then a machine-specific visualization software gets executed.

The inputs via this first operating area 31 have in common that they are not suitable for triggering or executing machine functions which are directly coupled to an operating procedure, for example the method of machine axes, which is to take place only during the duration of a keystroke. The reason for this is that the conventional operating systems used are not or only to a limited extent real-time capable and in the detection and forwarding of operations on the controls (virtual controls) can always lead to delays (such as read / write to storage media or Network connections, garbage collection, reinitialization of components, etc.). In addition, at least parts of the visualization software are often written by persons to whom the special measures and rules for the creation of real-time software components are only insufficiently known or familiar.

Therefore, in a control unit 30 according to the prior art, a further, second operating area 32 is provided, which has an array of mechanical buttons, switches, turntables, multi-axis joysticks and the like. For example, conventional membrane keys are used for the keys. The signal outputs of these operating elements are not recorded and forwarded via the software of the visualization device, but are conducted directly to the machine control independently of the visualization device. Thus, the operations on these controls are detected and implemented reliably and essentially instantaneously by the machine control. Changes to the state of the machine or process are first redirected to the visualization device by the machine control system and output to the operator via the operating device or HMI.

In this second operating area 32, other simple output devices, such as LEDs or signal lamps can be accommodated, which signal the switching state and / or the release state of a control element or the operating state of a machine component. These can be controlled directly by the machine control or also by the visualization device. The first and second operating areas 31, 32 are structurally separated and do not overlap.

In FIG. 3, the structure of an operating device 30 or operating terminals according to the prior art described in FIG. 2 and the signal-technical linking of the essential components are shown in greatly simplified form.

FIG. 3 shows two input and output components of the operating device 30: a touch screen 5 forming the first operating area 31 comprising a high-resolution, graphics-capable display 6 and a transparent touch sensor 8 arranged above it, and mechanical input elements forming the second operating area 32.

The signals of the mechanical input elements of the second operating area 32 used for immediate machine operation are fed directly to the control device 3 (i.e., machine control) of the machine 4.

The signals of the touch sensor 8 (touch sensor) are detected by a special touch controller 21 and converted into a data sequence (touch raw data), which essentially represent the positions (coordinates) of one or more contact points registered by the touch sensor 8 , Depending on the touch technology used, these touch raw data may also include further information, for example about the actuation pressure or the size of the contact surface. For example, capacitive or resistive sensors as well as sensors based on the piezo-effect or similar principles can be used as the touch sensor. Depending on the touch technology used, it may be suitable for detecting only one or preferably also several simultaneous touch points. The touch controller 21 is usually embodied as an ASIC and is often connected to the visualization device 10 via an RS232, an RS485, an I2C or a USB interface. The touch sensor 8 can have a plurality of discrete sensor fields or sensor zones, each corresponding to a specific operating or input element or having a homogeneous large sensor area with a sufficiently fine spatial resolution. The touch raw data are transmitted via a suitable signal connection, for example via a USB interface, to the visualization device 10 or can be queried by the latter via this interface at regular time intervals. From the raw touch data, the visualization device determines the actuation processes for the various control elements 7a currently shown. Simple coordinate evaluations of touch points can be made or even more complex sequences (eg gestures) and changes can be evaluated and made plausible. Thus, for example, different functions (eg activation / deactivation or incrementing / decrementing) can be triggered at a specific control element 7a depending on the gestural movement that has been carried out, or unwanted intentional actuations can be distinguished (eg touches being hidden by the palm of the hand) or even sporadic actuation signals According to any electromagnetic interference fields are filtered out or ignored. Some of the described measures can be implemented both in the visualization device 10 and in the touch controller 21.

The visualization device 10 transmits via a further interface or signal connection (for example a VGA, DVI or HDMI interface) the image data required for the screen output to the display 6. The display 6 can be any of a number of available technologies. Preferably, the display 6 is a high-resolution, graphics-capable color display with sufficient life and thermal and mechanical load capacity for reliable use in industrial environments.

Via a third interface or data connection, for example an Ethernet bus connection, the visualization device exchanges parameters, operating states and process data, error information and the like with the control device 3 and prepares this information for display on the display 6. Depending on the operating or fault state and also depending on the choice of the operator, data is grouped in a suitable and clear manner in various screens (windows). In the following, the invention will be described in more detail, wherein the preceding remarks on the display 6, the touch controller 21 and the visualization device 10, in particular for their construction, also apply with regard to the subject matter according to the invention, unless otherwise stated below. 4 now shows the front view of an operating device 2 according to an embodiment of the invention. It is characterized by a largely flat front, which is formed by a large parts of the entire front surface engaging, high-resolution (color) display 6 together with a built-lieh superimposed touch sensor 8.

In the upper first input / output area are input / output elements, conventional touch controls 7a, shown, which are provided for displaying and changing machine parameters and process states. In the middle part or the second input / output area an overview picture of the controlled machine or system is shown. It can be used to both represent certain machine states (i.e., the map may be changed depending on the machine state and display the machine state), as well as for the simple, intuitive selection of a machine component for the presentation and modification of detailed data in the first input / output area. In the lower third input / output area of the operating device 2, some control elements 7b for triggering control commands (machine commands) for immediate execution by the machine or system are shown by way of example. These are virtual controls in the form of bistable inputs and switches, monostable buttons and sliders and turntable, with which, for example, positions or speeds are quasi analog predeterminable.

FIGS. 5 to 7 now show various possibilities of implementation and signal-technical integration of a real-time evaluation device 13 according to the invention into the control system 1 of a machine or plant 4. a manipulator, a processing device for machining workpieces and / or a manufacturing s anläge for the production of modules be.

The control system 1 for controlling a machine or installation 4 comprises a control device 3 for controlling the machine or installation 4 and an operating device 2 connected to the control device 3 for operating the machine or installation 4 by an operator. The operating device 2 has a touchscreen 5, which is formed from a display 6 for the visualization of operating elements 7a, 7b and a touch sensor 8 superimposed on the display 6. In the embodiment of FIG. 5, the operating device comprises 2 a visualization device 10 connected to the display 6 via a data connection 29 with a data processing device 11 for providing image data for the display 6 of the touch screen 5.

The control system 1 comprises an evaluation device 13 whose input interface 26 is connected to the touch sensor 8 of the touch screen 5 (here via a preceding touch controller 21) via a sensor data link 18. The evaluation device 13 comprises a real-time data processing device 14 (CPU) and an output interface 16 connected to the control device 3. The value-value device 13 contains software with which the real-time data processing device 14 is capable of depending on the sensor data of the touch sensor 8 to generate control commands for the control device 3 and provide via the output interface 16 or to the control device 3 to transmit. The real-time data processing device 14 of the evaluation device 13 is independent in relation to the data processing device 11 of the visualization device 10. That the data processing devices 11, 14 are separate data processing devices. Since the value-value device 13 is connected in parallel to the visualization device 10, the transmission of the control commands to the control device 3 takes place without the visualization device 10, i. via an independent signal path or an independent data channel to the control device 3.

At this point, the difference between the controls 7a and 7b is highlighted again. The operating elements 7b are associated with machine functions which are controlled via the value-value device 13 in real time. The operating elements 7a are associated with machine functions or machine parameters which do not require real-time control and are controlled or transmitted via the visualization device.

5, the data stream of the touch raw data generated by the touch controller 21 is split and transmitted equally to the visualization device 10 as well as to the evaluation device 13. The visualization device evaluates the touch inputs. lent to the non-real-time relevant operations, such as the user desired change to another screen or the display and change a machine or process parameter or other functions such as saving or loading whole sets of parameters or the output of operating or service instructions. This includes other functionalities that are well-known in the context of graphical user interfaces on computer-based control and data processing systems.

The evaluation device 13 or its data processing device 14 forms between the touch sensor 8 of the touch screen 5 and the control device 3 of the machine or system 4 a bypass path to the visualization device 10 and their data processing device 11 for operations via the touch sensor 8. The evaluation device 13 continuously analyzes the same data stream on sensor data with regard to such operating patterns which are suitable for actuating the operating elements 7b parameterized in their memory 15 (FIG. 8). The evaluation device 13 can adapt the status information about the actuation states of the operating elements 7b, i. into converted control commands, and this converted status information within a defined time frame or a predetermined time frame, i. in real time, via an output interface 16 and a real-time data link 22 of the controller 3 available. The transmission of the control commands from the off value device 13 to the control device 3 preferably takes place cyclically and in each case completely so that a current image of the actuation states of all operating elements is present in the control device 3 at all times.

A preferably bidirectional data connection 17 (in particular as a parameter interface) between the real-time evaluation device 13 and the visualization device 10 enables the parameterization of the control elements 7b monitored by the evaluation device 13 by the visualization device 10. In the simplest case, this can be simple release information be, with the only determines whether and which of the controls 7b are unlocked for an operator action. In this simple case, the type, the position and the orientation of the operating elements 7b can be stored permanently in at least one memory 15 of the value device 13 (FIGS. 8 and 9).

The nature of the respective operating elements 7b, ie the virtual control elements, their position on the screen, their orientation (orientation) and their initial operating state can but also parameterizable, in particular by the visualization device 10 via the data connection 17 can be specified. Such a parameterization of the evaluation device 13 can take place once during the initialization phase of the operating device 2 but also several times during ongoing operation, for example, to adapt the type and number of operating elements 7b to different operating and operating situations.

An additional data connection 27 can be provided between the value-value device 13 and the visualization device 10, preferably in the manner of an interrupt line, with which the evaluation device 13 signals an operating procedure of an operator to the visualization device 10.

The touch controller 21 and the evaluation device 13 can, in addition to the sensor data connection 18, are transmitted via the raw sensor data to the value-value device 13 via a communication connection 23, preferably in the manner of an interrupt signal line. Via this communication connection, the touch controller 21 of the value-value device 13 can signal in advance asynchronously and without delay the presence of an operating process relevant for the evaluation.

The operating device 2 may be a structurally independent operating device 24, in particular in the form of an HMI (human-machine-interface) panel, a handheld terminal, a handheld programmer, a TPU (teach pendant unit) and the like .. Alternatively, it would also be conceivable that the operating device 2 together with the control device 3 forms a structural unit. The operating elements 7a and 7b shown in FIGS. 4 to 7 represent virtual input elements which are visually variable via the display 6 and represent the use of structurally independent and additional mechanical operating elements for delivering real-time machine commands and movement commands necessary. The elimination of additional mechanical input elements results in significant simplifications in terms of production engineering, for example in terms of sealing against the ingress of dirt and liquids. The elimination of mechanically moving components reduces wear-related failures. Since the appearance, the number, position and function of the operating elements 7a, 7b are determined by the values in the evaluation device 13 and in the visualization device. 10 software executed or their parameterization is determined, the number and variant diversity of the remaining hardware components is reduced. The adaptation to machine-specific requirements is also simplified for series with smaller quantities. Thus, the same operating device can be used for a number of different machines with different functional scope, and the display and selection of the operating elements can be adapted by software to the respective machine or system 4.

The touch sensor 8 used is preferably a high-resolution, capacitive multi-touch sensor. This sensor technology enables the detection of simultaneous multiple touches through a front, in particular a glass surface, through. Such a front forms a mechanically and chemically particularly resistant, scratch-resistant and durable front surface for an operating device 24 in an industrial environment.

Due to the lack of printed key labels, these can not be tapped over time and fade. A clear marking is made possible by the display on the underlying display 6. Similarly, the representation of directly assigned machine or switching states, for example, by a corresponding color coding on the display 6. At the front special special tactile markings 25 may be attached, which allow a user, the individual controls 7a, 7b also feel without direct visual control or to find a very specific control element 7a, 7b. This is advantageous, in particular, for those operating elements 7b which are provided for triggering immediate actions, in particular traversing movements, in accordance with the present invention, since the operator's gaze is often directed to the relevant machine component in order to control a specific position and this visually to monitor.

Figures 5 to 7 show next to the display 6 for the output of information to the operator and the full-surface touch sensor 8 for the input or the detection of operator actions even a superimposed front 9, for example in the form of a transparent (glass) plate , The front 9 has haptic markings 25. These markings 25 make it easier to find an operator on the operating device 24 during an operation. envorganges in which, for example, machine axes moved manually controlled, in particular positioned, and while the view is directed to the respective machine component and not on the control unit 24. The position of a control element 7b is easy to feel, so that the operator can locate a specific control element 7b without looking at the touchscreen 5. On a completely flat control surface, however, the operating elements 7b are initially only visually perceptible, so that this potential disadvantage is preferably compensated for by haptically perceivable markings 25 applied to the front 9. FIG. 6 shows a variant in which the data of the touch controller 21 are initially guided solely to the value device 13 and are evaluated there as previously described. The control actions on the operating elements 7b detected by the evaluation device 13 are coded into corresponding control commands and sent directly to the control device 3.

Via a further interface of the evaluation device 13 and a data connection 17 to the visualization device 10, data are forwarded from the value device 13 to the visualization computer 10, essentially corresponding to the touch raw data, so that the latter also receives the touch inputs with respect to the non-realtime evaluate and implement critical operating procedures via the graphical user interface. Such functions include, for example, switching between different screens or selecting particular groups of status values or parameters for display on the display 6 (screen) or the like. For this purpose, the evaluation device 13 can emulate the interface behavior of a touch controller and send the data stream supplied by the actual touch controller 21 substantially unchanged. The advantage of this method is that in the visualization device, a standard driver for the evaluation of the touch data can be used. However, it is also possible to provide a proprietary communication format between the visualization device 10 and the evaluation device 13, which permits more extensive functionalities, eg in connection with the parameterization of the value-value device 13 and various safety functions. An essential advantage of this configuration according to FIG. 6 is that the evaluation device 13 has exclusive access to the touch controller 21 and can therefore execute the access to the touch data with regard to a reliable manner that conforms to the real-time requirements. Disturbances in the data stream due to any effects on the part of the visualization device can not occur. Compliance with real-time conditions is not endangered.

FIG. 7 shows a further advantageous implementation in which the visualization device 10 is not arranged in the operating device 24 but outside. In this case, the operating device 2 or the operating device 24 comprises an interface 12 for a visualization device 10. The interface 12 may be part of a plug connection or a component of a radio-based interface. The visualization device can be integrated in the control device or combined with it structurally. The connection of the operating device 24 to the control device 3 via a data connection 19, which may be formed wired or wireless-based. This comprises: a data connection 22 between evaluation device 13 and control device 3, preferably in the form of a real-time bus, e.g. Real-time Ethernet, for the transmission of real-time control commands; another data link 17, e.g. in the manner of a USB interface or I2C interface for the transmission of the sensor data from the evaluation device 13 to the visualization device 10 and for the transmission of configuration data from the visualization device 10 to the evaluation device 13; and a further data connection 29 for transmitting the display output data from the visualization device 10 to the display 6. The display output data can be designed, for example, according to the VGA, DVI or HDMI standard. An advantage of this configuration is that the operating device 24 itself is completely independent of the computing power required for visualization, which can vary greatly depending on the application complexity, operating system and programming platform. As a result, the variety of variants and thus the manufacturing and warehousing costs can be further reduced.

The omission of the fixed labeled and arranged mechanical keys (from the prior art) and the replacement by the configurable and real-time capable virtual controls or controls 7 leads in connection with the arrangement of FIG. 7 to a control unit 24, which in practice for a very broad spectrum of applications and because individual design requirements can be used. This results in very cost-effective solutions.

FIG. 7 also shows a variant in which the touch controller 21 (from FIGS. 5 and 6), which converts the signals of the touch sensor 8 into a corresponding data stream, is already structurally integrated into the evaluation device 13 or the value device 13 already takes over the function of the touch controller 21 with. Thus, the evaluation device 13 additional and direct sensor information for the evaluation are available, which allow a more reliable and faster response. In particular, in the case of a multi-touch-capable evaluation, the direct integration can enable a faster acquisition and reaction to the inputs.

As a further variant, FIG. 7 shows the activation of at least one actuator 20 of the operating device 2 by the evaluation device 13. The actuator 20 is mechanically coupled to the front 9 and can couple mechanical oscillations or impulses into it.

These can be perceived when touched by an operator. In this way, the operator may also be given tactile feedback on the response to an operator action. Various parameters of the tactile feedback may be variable, such as the intensity or frequency of pulses, and may also be parameterizable depending on the application (e.g., in the design of the graphical user interface).

In all three variants according to FIGS. 5 to 7, changes to the operating states of the operating elements 7b can also be fed directly to the visualization device 10 via the data connection 17 between the evaluation device 13 and the visualization device 10 and used there to adapt the display output.

In the embodiments according to the prior art of FIG. 3, however, the visualization device 10 receives updated information about inputs or changed operating states via the mechanical input elements only after access to the control device 3, since this is signal-technically coupled only to the control device 3.

In a further embodiment of the invention, via the data connection 22 for the transmission of the real-time control commands to the control device 3, conversely, data from the control device 3 are transmitted to the output unit 13. With such data, for example, the behavior or the stored actuation state of a control element can be influenced. The invention is not limited to the use of a particular touch technology. In particular, various common types and technologies, in particular based on capacitive, resistive or piezoelectric technologies, come into question as a touch sensor. 8 shows a simplified and exemplary construction of an evaluation device 13. This includes a real-time data processing device 14 in the form of a processor unit (CPU) and at least one memory 15, which includes a random access memory (RAM) in which a software for execution by the Data processing device 14 and required for the execution of data are stored. In addition, a non-volatile memory 15 (ROM) may be provided, in which a program code and data are stored, which should be available again after the power supply is switched off again. This may be read only memory (ROM), but it may also be nonvolatile writable memory (NVRAM - Non Volatile RAM). The non-volatile memory 15 may also contain only a simple bootloader program which, after the supply voltage has been switched on, makes it possible to load the actual program code of the evaluation device 13 together with data via the data connection 17 by the visualization device 10.

Furthermore, the real-time evaluation device 13 has a first (USB) interface 26 for the connection to the touch controller 21 or directly to the touch sensor 8 and a second (USB) interface 28 for the connection to the visualization device 10. However, these interfaces can also be designed according to other interface standards such as I2C or according to a proprietary standard. Another (RT Ethernet) interface - the output interface 16 - is provided for the real-time transmission of the control commands to the control device 3. Internally, the individual components and interfaces of the off-value device preferably communicate via a common bus connection. The software of the evaluator 13 is normally simple and without much overhead. In particular, it does not have a complex operating system and is preferably not based on a graphical user interface. The clearly defined purpose of the soft- ware, the clearly limited scope of functions and the absence of complex operating system functionalities enable a high level of functional reliability, short program execution times and a predictable and guaranteeable temporal behavior (= real-time behavior). The evaluation device 13 can be structurally independent as well as designed to be combined with other control components, in particular with the visualization device 10 or the control device 3. It is only important that the evaluation device 13 has an independent data processing device 14, which differs from the data processing device 11 of the visualization device 10 or is functionally separated from it. Basically, the two data processing devices 11 and 14 can also be two independently executing cores of a common CPU.

Preferably, the visualization device 10 is configured to provide the evaluation device 13 with configuration data relating to at least one control element 7b visualized on the display 6. Alternatively or additionally, such configuration data may be stored in the memory 15 of the value device 13. The configuration data preferably contain information about the position, size, orientation, type, the permissible or associated operating gestures for an actuation and their parameterisable characteristics, parameters for any active tactile feedback, assigned machine or system function, the assigned machine or system parameters , the release state and / or the current actuation state or set value of the at least one control element. The evaluation device 13 or its software is set up to generate the control commands for the control device 3 as a function of the configuration data.

FIG. 9 schematically shows an extract of the memory 15 (RAM) of a real-time evaluation device 13 according to the invention. In a part of the memory 15 (program memory) rather) the program code provided for execution by the data processing device 14 (CPU) is stored. Another part contains general data required to execute the program code or incurred during its execution. In a third part of the memory 15, the parameters and data for the individual control elements 7 that can be detected by the value device 13 are stored (configuration data or "parameterized operating elements").

For each control element at least one identifier is deposited in the manner of an identifier, which allows the assignment of an operator action performed on the respective control element to a specific machine or system function or a parameter. Furthermore, data can be stored which determine the type and the basic properties of the respective control element. In other parameters, the position, the size and the orientation with respect to the display or with respect to the touch sensor can be specified, if these properties are made changeable. Via an enable status, the function of a virtual control element can be temporarily blocked and released again (enable state). In principle, this mechanism also makes it possible to define several control elements superimposed on the touchscreen, one of which is then activated depending on the operating or operating state of the machine / system while the others are locked. Corresponding to this, the visual representation on the display can be adapted.

Finally, the data for an operating element may also have its current or last operating status, e.g. On / Off state, map in a state value.

As already explained in detail initially, the invention relates not only to the operating device and the control system but also to methods for controlling a machine or system which have already been described in detail above. Also, the invention relates to a method for creating a graphical user interface for the visualization device 10 of an operating device 2 and / or a control system 1 by means of a development environment Reference designation Control system 27 Data connection

Control device 28 Interface

Control device 29 Data connection

Machine or system 30 HMI device according to the state of the touchscreen technology

Display 31 First operating area a Operating element 32 Second operating area b Control element that is linked to the real-time function

touch sensor

front

0 visualization device

1 data processing device

2 interface for visualization device 10

3 evaluation device

4 real-time data processing device

5 memory

6 Output interface

7 data connection

8 sensor data connection

9 data connection

0 actuator

1 touch controller

2 real-time data connection

3 communication connection

4 structural unit

5 haptic markings

6 input interface

Claims

Patent claims

1. operating device (2) for generating control commands for the control device (3) of a machine or system (4), wherein the operating device (2) comprises:

a touchscreen (5) which is formed from a display (6) for the visualization of operating elements (7b) and a touch sensor (8) superimposed on the display; a visualization device (10) connected to the display (6), which comprises a data processing device (11), or an interface (12) connected to the display (6) for such a visualization device (10) for providing image data for the display (6 ) of the touchscreen (5);

characterized by an evaluation means (13) connected to the touch sensor (8) of the touch screen (5) and comprising a real time data processing means (14) and an output interface (16), the real time data processing means (14) being the off Valve means (13) is arranged to generate in dependence of the sensor data of the touch sensor (8) control commands for the control device (3) and to provide at the output interface (16), wherein the real-time data processing means (14) of the Auswerteinrichtung (13) in Reference to the data processing device (11) of the / a visualization device (10) is independent.

2. Operating device according to claim 1, characterized in that the evaluation device (13) with the visualization device (10) via a preferably bidirectional data connection (17) is connected.

3. Operating device according to claim 1 or 2, characterized in that the visualization device (10) is adapted to provide the evaluation device (13) configuration data concerning at least one on the display (6) visualized control element (7b), wherein the configuration data preferably information about the position, size, orientation, type, associated machine or system function, the associated machine or system parameters, the release state and / or the current operating state or set value of the at least one control element (7b) included, and the value means (13) is arranged to generate the control commands for the control device (3) as a function of the configuration data.

4. Operating device according to one of the preceding claims, characterized in that the visualization device (10) is connected at least indirectly with the touch sensor (8) and with the control device (3) and for generating control commands and / or machine or system parameters for the Control device (3) is set up as a function of operating elements (7a) of the touchscreen (5) operated by an operator.

5. Operating device according to one of the preceding claims, characterized in that the sensor data connection (18) between the touch sensor (8) and the off value means (13) comprises a branch, the visualization device (10) or the interface (12) for a visualization device (10) leads.

6. Operating device according to one of the preceding claims, characterized in that the operating device (2) is a mobile, preferably portable operating device with the control device (3) of a machine or system (4) via a data connection (19), in particular via a flexible line or a radio link, is connectable.

7. Operating device according to one of the preceding claims, characterized in that the operating device (2) comprises at least one actuator (20) for generating a haptic signal for the operator and that the evaluation device (13) connected to the actuator (20) and adapted thereto is to actuate the at least one actuator (20) depending on the sensor data of the touch sensor (8).

8. Operating device according to one of the preceding claims, characterized in that the output interface (16) of the evaluation device (13), are transmitted via the control commands to the control device (3) comprises at least one real-time control output or at least one digital or analog control output, wherein preferably the output interface (16) comprises at least two control outputs, wherein each control output is assigned a different machine or plant-related function.

9. Operating device according to one of the preceding claims, characterized in that the touch sensor (8) is a multi-touch sensor and the Auswerteeinrich- Device (13) is adapted to evaluate sensor data from the multi-touch sensor, wherein preferably the operation of at least two control elements (7b) can be evaluated simultaneously.

10. Operating device according to one of the preceding claims, characterized in that the value means (13) is adapted to a movement or contact pattern on the touch sensor (8), before, during and / or after the actual operation, the by operation of a control element (7b) is carried out, executed and for the execution of a control process is absolutely necessary to check and only after positive verification of this motion or touch pattern to provide a control operation corresponding control command at the output interface (16), preferably the Motion or touch pattern is an introductory gesture for activating the operating element (7b) for a certain period of time or the simultaneous actuation of a further operating element (7b).

11. Operating device according to one of the preceding claims, characterized in that between the touch sensor (8) and the evaluation device (13), a touch controller (21) is connected, which is adapted to the operation of the operating elements (7a, 7b) of Touchscreens (5) to capture and provide as sensor raw data.

12. Operating device according to claim 11, characterized in that the evaluation device (13) is designed such or connected to the data stream of the touch controller (21) that the data stream both the evaluation device (13) and the visualization device (10). is supplied in the same way.

13. Operating device according to claim 11 or 12, characterized in that the touch controller (21) is structurally and functionally integrated in the evaluation device (13).

14. Operating device according to one of claims 11 to 13, characterized marked, that the touch controller (21) and the off value means (13) in addition to the sensor data connection (18), via the sensor raw data to the Aus Werteinrichtung ( 13), are connected to one another via a communication connection (23), preferably in the manner of an interrupt signal line, via which the touch controller (21) of the ejector teeinrichtung (13) asynchronously and without delay, the presence of a relevant for the evaluation operation signal.

15. Operating device according to one of the preceding claims, characterized in that the value-out device (13) has at least three interfaces, wherein

Sensor raw data are received via a first interface and forwarded via a second interface to the visualization device (10) or to the interface (12) for a visualization device (10), preferably the evaluation device (13) at the second interface, the behavior of a Touch controller (21) at least partially emulated.

16. Operating device according to one of the preceding claims, characterized in that the value-out device (13) comprises at least one memory (15) connected to the real-time data processing device (14).

17. Operating device according to claim 16, characterized in that in the memory (15) configuration data concerning at least one display (6) representable control (7b) contain and / or can be stored, the configuration data preferably information about the position, size, orientation , Type, associated machine or system function, the associated machine or system parameters, the release state and / or the current operating state or set value of the at least one control element (7b) included.

18. Operating device according to claim 16 or 17, characterized in that in the memory (15) contain calibration information and / or can be stored, with which the coordinate information supplied by the touch controller are corrected, preferably in terms of offset, scaling and / or equalization ,

19. Control system (1) for controlling a machine or plant (4), in particular a manipulator, a processing device or a production plant, by means of an operating device (2), comprising:

a control device (3) for controlling the machine or system (4); an operating device (2) connected to the control device (3), in particular according to one of the preceding claims, for operating the machine or system (4) by an operator, the operating device (2) having a touch screen (5) consisting of a display (6) is formed for the visualization of operating elements (7b) and a touch sensor (8) superimposed on the display (6);

a visualization device (10) connected to the display (6) for providing image data for the display (6) of the touch screen (5), the visualization device (10) comprising a data processing device (11);

characterized by an evaluation means (13) connected to the touch sensor (8) of the touch screen (5) and comprising a real time data processing means (14) and an output interface (16) connected to the control means (3), the real time ones Data processing device (14) of the evaluation device (13) is arranged to generate control commands for the control device (3) in dependence on the sensor data of the touch sensor (8) and to transmit them to the control device (3) via the output interface (16), wherein the Real-time data processing device (14) of the evaluation device (13) with respect to the data processing device (11) of the visualization device (10) is independent.

20. Control system according to claim 19, characterized in that the Ausga- trimming parts (16) of the evaluation device (13) to the control device (3) via a real-time data bus (22) is connected.

21. Control system according to claim 19 or 20, characterized in that the operating device (2) is one of the control device (3) separate structural unit (24), in particular a mobile, preferably portable control unit, and that the evaluation device ( 13) is integrated in this structural unit (24).

22. Control system according to claim 21, characterized in that the visualization device (10) outside the operating device (24) is arranged, wherein preferably the visualization device (10) in the control device (3) is integrated.

23. Method for controlling a machine or installation (4) by means of an operating device (2) according to one of claims 1 to 18 and / or by means of a control system (1) according to one of claims 19 to 22, wherein the visualization device (10) provides image data for the display (6) of the touchscreen (5) so that operating elements (7b) are visualized on the display (6) and wherein the real-time data processing device (14) The value device (13) generates control commands as a function of the sensor data of the touch sensor (8) of the touch screen (5) and transmits them to the control device (3).

24. Method according to claim 23, characterized in that configuration data regarding at least one operating element (7b) shown or displayable on the display (6), preferably in the form of a parameter set, are loaded into the value means, the configuration data preferably containing information about the position, Size, orientation, type, assigned machine or system function, the assigned Maschinenbzw. Plant parameters, the release state and / or the current operating state or set value of the at least one control element (7b) included, and that the evaluation device (13) generates the control commands for the control device (3) in dependence of the configuration gurationsdaten.

25. The method according to claim 24, characterized in that the configuration data from the visualization device (10) and / or generated by the control device (3) and transmitted to the evaluation device (13).

26. The method according to any one of claims 23 to 25, characterized in that the evaluation device (13), preferably via regular communication or signaling, receives information from the visualization device (10), from which conclusions about the correct or incorrect execution of the on the Data processing device (11) of the visualization device (10) current visualization software can be obtained.

27. The method according to any one of claims 23 to 26, characterized in that the one control element (7b) related configuration data, which from the Visualisierungsein- direction (10) and / or control device (3) to the evaluation device (13) are transmitted within the evaluation device (13) are or are provided with a time-limited validity, and that the corresponding operating element (7b) after the temporal Expiry of validity is disabled so that operation of the machine or system (4) via this control (7b) is locked.

28. The method according to any one of claims 23 to 27, characterized in that the visualization device (10) via a data connection (17) between the evaluation device (13) and visualization device (10) the actuation states of the evaluation device (13) monitored controls (7b) and evaluates with respect to non-real-time critical functions, such as for an optical, acoustic or tactile feedback to the operator.

29. The method according to any one of claims 23 to 28, characterized in that in the evaluation device (13) after changing an operating state of a control element (7b), a timer is started with a predetermined Abiaufzeit and that an error signal to the control device (3) is sent or an operating element (7b) is put into a deactivated state if the visualization device (10) does not respond with an acknowledgment information prior to the end of the expiration time, based on an update of the information displayed on the touch screen (5), in particular with regard to the actuation state of the operating element (7b) relates.

30. The method according to any one of claims 23 to 29, characterized in that the evaluation device (13) with one or more actuators (20) is technically coupled in such a manner that upon actuation of a control element (7b) or already at mere touch of the touch screen (5 ) a haptic feedback is generated to the operator.

31. The method according to any one of claims 23 to 30, characterized in that the positions and the type of operating elements (7b) are deposited in the off value device (13) and preferably correspond with existing haptic markings on the touch screen (5).

32. Method according to claim 23, characterized in that deactivation of a monostable operating element (7b), preferably in the form of a pushbutton or joystick with automatic return to an unactuated starting position, by the visualization device (10) independently of the actual actuation state of the Operating element (7b) leads to an instantaneous signaling of an unactuated state to the evaluation device (13) or to a control command corresponding to the unactuated state from the value device (13) to the control device (3).

33. The method according to any one of claims 23 to 32, characterized in that between the off value means (13) and the visualization device (10) an additional data connection (27) is provided, preferably in the manner of an interrupt line with which the evaluation device (13) signals an operation of an operator to the visualization device (10).

34. The method according to any one of claims 23 to 33, characterized in that for the transmission of real-time control commands a real-time data connection (22), for example, a real-time Ethernet bus connection is provided, via which the actuation states of the operating elements (7b) or the actuation states of the operating elements (7b) associated control commands are transmitted cyclically in a predetermined time grid to the control device (3).

35. A method for creating a graphical user interface for the visualization device (10) of an operating device (2) according to one of claims 1 to 18 and / or a control system (1) according to one of claims 19 to 22 by means of a development environment, wherein with the development environment at least one operating element (7b) is selected from a predefined set of available operating elements (7b) and this at least one operating element (7b) with regard to its position, size, orientation, size, type, associated machine or system function, its associated machine or Parameter parameters and / or its release state is parameterized, and wherein the assignment of the parameters to the at least one operating element (7b) configuration data, in particular in the form of a parameter data set, for the evaluation device (13) are generated.