EP3646313B1 - Apparatus and method for displaying image data on a secured display - Google Patents

Apparatus and method for displaying image data on a secured display Download PDF

Info

Publication number
EP3646313B1
EP3646313B1 EP18753075.3A EP18753075A EP3646313B1 EP 3646313 B1 EP3646313 B1 EP 3646313B1 EP 18753075 A EP18753075 A EP 18753075A EP 3646313 B1 EP3646313 B1 EP 3646313B1
Authority
EP
European Patent Office
Prior art keywords
image data
display
changed
facility
target value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP18753075.3A
Other languages
German (de)
French (fr)
Other versions
EP3646313A1 (en
Inventor
Torsten DRAFFEHN
Wolfgang Ebeling
Jörg Steinbach
Volker SWATON
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Siemens Mobility GmbH
Original Assignee
Siemens Mobility GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Siemens Mobility GmbH filed Critical Siemens Mobility GmbH
Publication of EP3646313A1 publication Critical patent/EP3646313A1/en
Application granted granted Critical
Publication of EP3646313B1 publication Critical patent/EP3646313B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/003—Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto
    • G09G5/005—Adapting incoming signals to the display format of the display terminal
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2330/00—Aspects of power supply; Aspects of display protection and defect management
    • G09G2330/02—Details of power systems and of start or stop of display operation
    • G09G2330/027—Arrangements or methods related to powering off a display
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2330/00—Aspects of power supply; Aspects of display protection and defect management
    • G09G2330/12—Test circuits or failure detection circuits included in a display system, as permanent part thereof
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/06—Colour space transformation
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2358/00—Arrangements for display data security

Definitions

  • So-called secure ads are used when the information displayed is security-relevant.
  • Such safe displays must meet a Safety Integrity Level SILO up to SIL4 and are used, for example, in rail vehicles or control rooms in a railway system.
  • a method for securely displaying image data on a display device either the displayed image information is checked, for example using a camera, or the image data sent to the display device is compared with predetermined image data.
  • a procedure is, for example, from EP 0 645 711 A1 known.
  • Display devices such as TFT monitors are used to create secure displays.
  • TFT monitors are used to create secure displays.
  • the color values can lead to the so-called white point being at a slightly different color location for each display device. If several such display devices are installed next to each other, the differences in the color representation are noticeable. If these display devices have not been calibrated in the factory, the image data can also be adapted to the specific display device. However, this can lead to problems if the changed image data is to be compared with the specified image data to check the secure display. The individual color setting would indicate an error.
  • the invention is therefore based on the object of providing a method and a device for outputting image data for a reliable display, with which individual color correction is possible.
  • the object is achieved according to the invention by a method for outputting image data for a secure display according to claim 1, in which the image data are generated for a display device, in which the generated image data are changed depending on a characteristic of the display device, in particular on a color correction offset value, and in which representative correction data are determined for a difference between generated and changed image data.
  • the solution according to the invention has the advantage that the correction data is representative of the change in the image data and can be taken into account when checking the image data later. This means that color matching is possible for each individual display device, such as a TFT monitor, and security-relevant checking of the displayed image data is still possible.
  • the image data for the display device are first generated in the usual way.
  • the generated image data are then modified in such a way that a color correction for the specific display device takes place.
  • This color correction is individual for each device due to the differences between different display devices.
  • the correction data is determined according to the invention.
  • This correction data is representative of the change in the image data and documents a difference between the generated and the changed image data.
  • the change in the image data can be taken into account during the later review of the image data, which is necessary according to SILO to SIL4.
  • the changed image data can be calculated back using the correction data.
  • the changed image data are recalculated using the correction data in such a way that they are the same as the generated image data, and the recalculated image data are compared with the expected target value image data.
  • changed target value image data are calculated from the expected target value image data and the correction data and these are compared with the changed image data.
  • individual target values i.e. expected target value image data
  • these expected target value image data are safety-relevant, for example SIL2, they would of course have to be created in a correspondingly secure manner and the expected target value image data would have to be validated.
  • These individual changed target value image data for each display device could, for example, be stored in the display device itself.
  • the generation of the changed target value image data could also take place in a secure part of the display device, taking the correction data into account.
  • At least one test value for the recalculated image data or the changed target value image data can be determined and, for comparison, the test value of the recalculated image data or the changed target value image data can be determined with at least one test value for the expected target value image data are compared.
  • Using the test values has the advantage that checking the image data is simplified, computing power can be saved and the result of the check is available more quickly. For example, checksums or checksums can be calculated as test values.
  • a reaction that is perceptible to an operator can be triggered, in particular a power supply to the display device can be switched off if an inadmissible result was determined during the comparison.
  • the image data or test values can only be compared with respect to predetermined display areas of the display device.
  • the correction data can be checked to see whether it lies within a predetermined allowable range.
  • the correction data can be determined pixel by pixel and/or color by color. This ensures that correction data is available even for the smallest units to be displayed.
  • the image data contains values for each pixel, whereby these values consist of at least three color components. The desired color is mixed together from the three color components.
  • the device according to the invention can be designed to carry out the method according to one of the aforementioned embodiments.
  • the invention also relates to a rail vehicle with at least one display device designed for safe display.
  • the rail vehicle has a device according to the invention according to one of the previously mentioned embodiments.
  • the invention can also be used, for example, in nuclear power plants or aircraft.
  • the invention also relates to a device for the secure display of image data with a display device, with a device for outputting image data and with a comparison device for checking the image data.
  • the device for outputting image data is designed according to one of the previously mentioned embodiments.
  • the device for outputting image data and the comparison device for checking the image data can be designed to be functionally independent of one another.
  • the single figure shows a schematic representation of a device according to the invention for the secure display of image data in an exemplary embodiment.
  • the device 1 for the secure display of image data which is used, for example, in a rail vehicle according to the invention (not shown), comprises a display device 2, a device 3 for outputting image data and a comparison device 4.
  • the display device 2 is, for example, a commercially available TFT monitor on which image data is displayed.
  • a speedometer or something similar, for example, is displayed on the display device 2.
  • the display device 2 is supplied with energy via a power supply 5.
  • the device 3 for outputting image data has a device 6 for generating image data for the display device 2.
  • the device 6 is, for example, a non-secure computer or calculator with a corresponding graphics card with graphics output.
  • the device 3 for outputting image data includes a processing module 7, which is arranged between the device 6 for generating image data and the display device 2 and is connected to both.
  • the processing module 7 includes a switch 8, a deserializer 9, a computing device 10 and two serializers 11, 12.
  • the device 6 for generating image data provides the generated image data 13, a signal selection 14 and a color correction offset value 15 on the output side.
  • the image data 13 generated is, for example, an LVDS 18-bit signal with three lanes for color mixing and a clock signal that specifies the clock for image construction.
  • the color correction offset value 15 indicates by how much the so-called white point of the individual display device 2 deviates from a theoretically optimal white point due to manufacturing reasons.
  • the generated image data 13, the signal selection 14 and the color correction offset value 15 are transmitted to the processing module 7.
  • the switch 8 connects the generated image data 13, the signal selection 14 and, if necessary, a signal 16 from an external interface 17.
  • the switch 8 can be controlled with the signal selection 14, for example to be able to switch to an external signal from the external interface 17.
  • the generated image data 13 is forwarded from the switch 8 to the computing device 10 via the deserializer 9.
  • the image data 13 are not yet changed by the switch 8 or the deserializer 9.
  • the computing device 10 changes the generated image data 13 to changed image data 18, taking into account the color correction offset value 15.
  • the computing device 10 determines correction data 19, which represents a difference between the changed image data 18 and the generated image data 13 are representative.
  • the changed image data 18 and the correction data 19 are passed via the serializers 11, 12 and then output by the processing module 7 to the display device 2 or to the comparison device 4.
  • the comparison device 4 includes a monitoring device 20 and also a processing module 21.
  • the processing module 21 has two deserializers 22, 23, an adding device 24 and a computing device 25.
  • the correction data 19 are transmitted directly to the comparison device 4 from the device 3 for outputting image data.
  • the changed image data 18 and also a clock signal 27 are transmitted from the device 3 to the display device 2.
  • the changed image data 18 and the clock signal 27 for the comparison device 4 are tapped at a point 26 shown as an example.
  • the changed image data 18 and the correction data 19 are transmitted to the adding device 24 via the deserializers 22, 23.
  • the adding device 24 recalculates the changed image data 18 using the correction data 19 in such a way that it is again equal to the generated image data 13.
  • This recalculated image data 28 is passed on to the computing device 25.
  • the monitoring device 20 passes section coordinates 29 to the computing device 25.
  • the section coordinates 29 indicate the predetermined display areas of the display device 2 that are safety-relevant and therefore have to be checked.
  • the term adding device 24 is only to be seen as an example here. In the adding device 24, other arithmetic operations than additions can also be carried out, for example if the data was previously obtained by subtraction. In any case, the adding device 24 reverses the arithmetic operation so that the recalculated image data 28 can again be compared with the generated image data 13.
  • test values are calculated for the recalculated image data. This is done to make checking the image data easier.
  • the test values for the recalculated image data 28 are only calculated in the display areas defined by the section coordinates 29.
  • the test values are, for example, checksums or checksums.
  • the test values 30 are finally transmitted from the computing device 25 to the monitoring device 20.
  • the test values 30 are compared with predetermined and stored target value test values (not shown). These target value test values were determined from expected target value image data (not shown). Only if the test values 30 are the same as the setpoint test values in the monitoring device 20 is the display on the display device 2 safe and operation can be continued.
  • the monitoring device 20 If the monitoring device 20 detects a deviation between the test value 30 and the setpoint test value, the monitoring device 20 sends a corresponding switch-off signal 31 to a switch-off device 32.
  • the switch-off device 32 is, for example, a corresponding one Switch that interrupts the power supply 5 of the display device 2.
  • a white balance can be carried out individually for the display device 2. In this way, individual manufacturing-related inaccuracies of the display device 2 can be compensated. Despite the individually modified image data 18, these can be checked using the correction data 19 in order to meet the requirements for a safe display in a safety-relevant system.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Hardware Design (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Controls And Circuits For Display Device (AREA)

Description

Sogenannte sichere Anzeigen werden verwendet, wenn die angezeigte Information sicherheitsrelevant ist. Solche sichere Anzeigen müssen einen Safety Integrity Level SILO bis SIL4 erfüllen und werden beispielsweise in Schienenfahrzeugen oder Leitständen einer eisenbahntechnischen Anlage eingesetzt. Der Einsatz in anderen sicherheitsrelevanten Anlagen, wie beispielsweise Atomkraftwerken, ist selbstverständlich ebenfalls möglich.So-called secure ads are used when the information displayed is security-relevant. Such safe displays must meet a Safety Integrity Level SILO up to SIL4 and are used, for example, in rail vehicles or control rooms in a railway system. Use in other safety-relevant systems, such as nuclear power plants, is of course also possible.

Bei einem Verfahren zur sicheren Anzeige von Bilddaten auf einer Anzeigeeinrichtung werden entweder die angezeigten Bildinformationen überprüft, beispielsweise mittels einer Kamera, oder die zur Anzeigeeinrichtung ausgesendeten Bilddaten werden mit vorgegebenen Bilddaten verglichen. Ein solches Verfahren ist beispielsweise aus der EP 0 645 711 A1 bekannt.In a method for securely displaying image data on a display device, either the displayed image information is checked, for example using a camera, or the image data sent to the display device is compared with predetermined image data. Such a procedure is, for example, from EP 0 645 711 A1 known.

Ein Verfahren zur Wiederherstellung von vollständigen Bilddaten ist in der US 2013/0308858 A1 beschrieben.A procedure for restoring complete image data is in the US 2013/0308858 A1 described.

Zur Realisierung von sicheren Anzeigen werden beispielsweise Anzeigegeräte, wie TFT-Monitore, eingesetzt. Bei der Produktion von Standard-TFT-Modulen kann es zu Schwankungen hinsichtlich der Farbwerte kommen. Dieses kann dazu führen, dass der sogenannte Weißpunkt bei jeder Anzeigeeinrichtung an einem etwas unterschiedlichen Farbort liegen kann. Werden mehrere solcher Anzeigeeinrichtungen nebeneinander aufgebaut, fallen die Unterschiede in der Farbdarstellung auf. Wenn diese Anzeigegeräte nicht im Werk kalibriert worden sind, kann auch eine entsprechende Anpassung der Bilddaten auf die spezielle Anzeigeeinrichtung stattfinden. Allerdings kann dies zu Problemen führen, wenn die veränderten Bilddaten für die Überprüfung der sicheren Anzeige mit den vorgegebenen Bilddaten verglichen werden sollen. Durch die individuelle Farbeinstellung würde ein Fehler angezeigt werden.Display devices such as TFT monitors are used to create secure displays. During the production of standard TFT modules, there may be variations in the color values. This can lead to the so-called white point being at a slightly different color location for each display device. If several such display devices are installed next to each other, the differences in the color representation are noticeable. If these display devices have not been calibrated in the factory, the image data can also be adapted to the specific display device. However, this can lead to problems if the changed image data is to be compared with the specified image data to check the secure display. The individual color setting would indicate an error.

Der Erfindung liegt somit die Aufgabe zugrunde, ein Verfahren und eine Vorrichtung zur Ausgabe von Bilddaten für eine sichere Anzeige bereitzustellen, mit denen eine individuelle Farbkorrektur möglich ist.The invention is therefore based on the object of providing a method and a device for outputting image data for a reliable display, with which individual color correction is possible.

Die Aufgabe wird erfindungsgemäß gelöst durch ein Verfahren zur Ausgabe von Bilddaten für eine sichere Anzeige gemäß Patentanspruch 1, bei dem die Bilddaten für eine Anzeigeeinrichtung erzeugt werden, bei dem die erzeugten Bilddaten in Abhängigkeit von einer Kenngröße der Anzeigeeinrichtung, insbesondere von einem Farbkorrekturoffsetwert, verändert werden und bei dem für einen Unterschied zwischen erzeugten und veränderten Bilddaten repräsentative Korrekturdaten ermittelt werden.The object is achieved according to the invention by a method for outputting image data for a secure display according to claim 1, in which the image data are generated for a display device, in which the generated image data are changed depending on a characteristic of the display device, in particular on a color correction offset value, and in which representative correction data are determined for a difference between generated and changed image data.

Ferner wird die Aufgabe gelöst durch eine Vorrichtung zur Ausgabe von Bilddaten für eine sichere Anzeige gemäß Patentanspruch 7, mit wenigstens einer Einrichtung zur Erzeugung von Bilddaten für eine Anzeigeeinrichtung und mit wenigstens einer Recheneinrichtung, die zum Verändern der erzeugten Bilddaten in Abhängigkeit von einer Kenngröße der Anzeigeeinrichtung, insbesondere von einem Farbkorrekturoffsetwert, und zum Ermitteln von für einen Unterschied zwischen erzeugten und veränderten Bilddaten repräsentativen Korrekturdaten ausgebildet ist.Furthermore, the object is achieved by a device for outputting image data for a secure display according to claim 7, with at least one device for generating image data for a display device and with at least one computing device which is designed to change the generated image data depending on a characteristic of the display device, in particular on a color correction offset value, and to determine correction data representative of a difference between generated and changed image data.

Die erfindungsgemäße Lösung hat den Vorteil, dass die Korrekturdaten für die Veränderung der Bilddaten repräsentativ sind und bei einer späteren Überprüfung der Bilddaten berücksichtigt werden können. So ist ein Farbabgleich für jedes individuelle Anzeigegerät, wie beispielsweise einen TFT-Monitor, möglich und trotzdem die sicherheitsrelevante Überprüfung der angezeigten Bilddaten.The solution according to the invention has the advantage that the correction data is representative of the change in the image data and can be taken into account when checking the image data later. This means that color matching is possible for each individual display device, such as a TFT monitor, and security-relevant checking of the displayed image data is still possible.

Bei dem erfindungsgemäßen Verfahren werden zunächst die Bilddaten für die Anzeigeeinrichtung in gewohnter Weise erzeugt. Anschließend werden diese erzeugten Bilddaten so verändert, dass eine Farbkorrektur für die spezielle Anzeigeeinrichtung stattfindet. Diese Farbkorrektur ist bedingt durch die Unterschiede zwischen verschiedenen Anzeigeeinrichtungen für jedes Gerät individuell. Um diese individuelle Veränderung der Bilddaten quasi zu dokumentieren, werden die Korrekturdaten erfindungsgemäß ermittelt. Diese Korrekturdaten sind für die Veränderung der Bilddaten repräsentativ und dokumentieren einen Unterschied zwischen den erzeugten und den veränderten Bilddaten. Mit Hilfe der Korrekturdaten kann bei der späteren Überprüfung der Bilddaten, die gemäß SILO bis SIL4 nötig ist, die Veränderung der Bilddaten berücksichtigt werden. Beispielsweise können die veränderten Bilddaten mittels der Korrekturdaten zurückgerechnet werden.In the method according to the invention, the image data for the display device are first generated in the usual way. The generated image data are then modified in such a way that a color correction for the specific display device takes place. This color correction is individual for each device due to the differences between different display devices. In order to document this individual change in the image data, the correction data is determined according to the invention. This correction data is representative of the change in the image data and documents a difference between the generated and the changed image data. With the help of the correction data, the change in the image data can be taken into account during the later review of the image data, which is necessary according to SILO to SIL4. For example, the changed image data can be calculated back using the correction data.

Die erfindungsgemäße Lösung kann durch vorteilhafte Ausgestaltungen weiterentwickelt werden, wie sie im Folgenden beschrieben sind.The solution according to the invention can be further developed through advantageous embodiments, as described below.

Erfindungsgemäß werden die veränderten Bilddaten mittels der Korrekturdaten derart zurückgerechnet, dass sie mit den erzeugten Bilddaten gleich sind, und die zurückgerechneten Bilddaten mit erwarteten Sollwert-Bilddaten verglichen. Dies hat den Vorteil, dass die zurückgerechneten Bilddaten wie gewohnt überprüft werden können. Alternativ oder zusätzlich werden erfindungsgemäß aus den erwarteten Sollwert-Bilddaten und den Korrekturdaten veränderte Sollwert-Bilddaten errechnet und diese mit den veränderten Bilddaten verglichen. Für die letztgenannte Ausführungsform müssten für jede Anzeigeeinrichtung individuelle Sollwerte, also erwartete Sollwert-Bilddaten, errechnet werden. Da diese erwarteten Sollwert-Bilddaten sicherheitsrelevant sind, beispielsweise SIL2, müsste die Erstellung selbstverständlich auf entsprechend sichere Weise geschehen und die erwarteten Sollwert-Bilddaten validiert werden. Diese individuellen veränderten Sollwert-Bilddaten für jede Anzeigeeinrichtung könnten beispielsweise in der Anzeigeeinrichtung selber hinterlegt werden. Die Generierung der veränderten Sollwert-Bilddaten könnte auch in einem sicheren Teil der Anzeigeeinrichtung unter Berücksichtigung der Korrekturdaten geschehen.According to the invention, the changed image data are recalculated using the correction data in such a way that they are the same as the generated image data, and the recalculated image data are compared with the expected target value image data. This has the advantage that the recalculated image data can be checked as usual. Alternatively or additionally, according to the invention, changed target value image data are calculated from the expected target value image data and the correction data and these are compared with the changed image data. For the latter embodiment, individual target values, i.e. expected target value image data, would have to be calculated for each display device. Since these expected target value image data are safety-relevant, for example SIL2, they would of course have to be created in a correspondingly secure manner and the expected target value image data would have to be validated. These individual changed target value image data for each display device could, for example, be stored in the display device itself. The generation of the changed target value image data could also take place in a secure part of the display device, taking the correction data into account.

Um die Überprüfung der Bilddaten für ein erfindungsgemäßes Verfahren für eine sichere Anzeige zu vereinfachen, kann wenigstens ein Prüfwert für die zurückgerechneten Bilddaten oder die veränderten Sollwert-Bilddaten ermittelt und zum Vergleich der Prüfwert der zurückgerechneten Bilddaten oder der veränderten Sollwert-Bilddaten mit wenigstens einem Prüfwert für die erwarteten Sollwert-Bilddaten verglichen werden. Die Verwendung der Prüfwerte hat den Vorteil, dass die Überprüfung der Bilddaten vereinfacht wird, Rechenleistung eingespart werden kann und das Ergebnis der Überprüfung schneller vorliegt. Als Prüfwert können beispielsweise Prüfsummen oder Checksummen berechnet werden.In order to simplify the checking of the image data for a method according to the invention for a reliable display, at least one test value for the recalculated image data or the changed target value image data can be determined and, for comparison, the test value of the recalculated image data or the changed target value image data can be determined with at least one test value for the expected target value image data are compared. Using the test values has the advantage that checking the image data is simplified, computing power can be saved and the result of the check is available more quickly. For example, checksums or checksums can be calculated as test values.

Ferner kann eine für einen Bediener wahrnehmbare Reaktion ausgelöst wird, insbesondere eine Energieversorgung der Anzeigeeinrichtung abgeschaltet werden, wenn beim Vergleichen ein unzulässiges Ergebnis ermittelt wurde. Dies hat den Vorteil, dass im Fehlerfall eine sofortige Reaktion ausgelöst wird, die beispielsweise für den Triebfahrzeugführer oder einen anderen Bediener ersichtlich ist. Selbstverständlich kann im Falle eines unzulässigen Ergebnisses auch jede andere wahrnehmbare Konsequenz ausgelöst werden.Furthermore, a reaction that is perceptible to an operator can be triggered, in particular a power supply to the display device can be switched off if an inadmissible result was determined during the comparison. This has the advantage that in the event of an error, an immediate reaction is triggered, which is visible to the driver or another operator, for example. Of course, in the event of an unacceptable result, any other noticeable consequence can also be triggered.

Um die Überprüfung der Bilddaten noch mehr zu vereinfachen, können die Bilddaten oder Prüfwerte nur bezüglich vorbestimmter Anzeigebereiche der Anzeigeeinrichtung verglichen werden.In order to simplify the checking of the image data even more, the image data or test values can only be compared with respect to predetermined display areas of the display device.

Um sicherzustellen, dass durch die Farbkorrektur die eigentliche Pixelfarbe nicht zu stark verändert wird, können die Korrekturdaten überprüft werden, ob sie in einem vorbestimmten zulässigen Bereich liegen.To ensure that the color correction does not change the actual pixel color too much, the correction data can be checked to see whether it lies within a predetermined allowable range.

Ferner können die Korrekturdaten pixelweise und/oder farbweise ermittelt werden. So ist sichergestellt, dass auch für die kleinsten anzuzeigenden Einheiten Korrekturdaten vorliegen. Bei üblichen Anzeigeeinrichtungen, wie beispielsweise TFT-Modulen und TFT-Displays, enthalten die Bilddaten Werte für jedes Pixel, wobei diese Werte aus wenigstens drei Farbkomponenten bestehen. Aus den drei Farbkomponenten wird die gewünschte Farbe zusammengemischt.Furthermore, the correction data can be determined pixel by pixel and/or color by color. This ensures that correction data is available even for the smallest units to be displayed. In conventional display devices, such as TFT modules and TFT displays, the image data contains values for each pixel, whereby these values consist of at least three color components. The desired color is mixed together from the three color components.

In einer vorteilhaften Ausgestaltung der erfindungsgemäßen Vorrichtung kann diese zur Durchführung des Verfahrens nach einer der zuvor genannten Ausführungsformen ausgebildet sein.In an advantageous embodiment of the device according to the invention, it can be designed to carry out the method according to one of the aforementioned embodiments.

Ferner betrifft die Erfindung auch ein Schienenfahrzeug mit wenigstens einer zur sicheren Anzeige ausgebildeten Anzeigeeinrichtung. Um bei den eingesetzten Anzeigeeinrichtungen eine Farbkorrektur durchführen zu können, ist erfindungsgemäß vorgesehen, dass das Schienenfahrzeug eine erfindungsgemäße Vorrichtung nach einer der zuvor genannten Ausführungsformen aufweist. Alternativ kann die Erfindung auch beispielsweise in Kernkraftwerken oder Flugzeugen eingesetzt werden.Furthermore, the invention also relates to a rail vehicle with at least one display device designed for safe display. In order to be able to carry out color correction in the display devices used, it is provided according to the invention that the rail vehicle has a device according to the invention according to one of the previously mentioned embodiments. Alternatively, the invention can also be used, for example, in nuclear power plants or aircraft.

Schließlich betrifft die Erfindung auch eine Einrichtung zur sicheren Anzeige von Bilddaten mit einer Anzeigeeinrichtung, mit einer Vorrichtung zur Ausgabe von Bilddaten und mit einer Vergleichseinrichtung zur Überprüfung der Bilddaten. Erfindungsgemäß ist die Vorrichtung zur Ausgabe von Bilddaten nach einer der zuvor genannten Ausführungsformen ausgebildet. Die Vorrichtung zur Ausgabe von Bilddaten und die Vergleichseinrichtung zur Überprüfung der Bilddaten können funktional unabhängig voneinander ausgestaltet sein.Finally, the invention also relates to a device for the secure display of image data with a display device, with a device for outputting image data and with a comparison device for checking the image data. According to the invention, the device for outputting image data is designed according to one of the previously mentioned embodiments. The device for outputting image data and the comparison device for checking the image data can be designed to be functionally independent of one another.

Im Folgenden wird die Erfindung mit Bezug auf die beispielhafte Ausführungsform in der beigefügten Figur beschrieben.In the following, the invention is described with reference to the exemplary embodiment in the attached figure.

Die einzige Figur zeigt eine schematische Darstellung einer erfindungsgemäßen Einrichtung zur sicheren Anzeige von Bilddaten in einer beispielhaften Ausführungsform.The single figure shows a schematic representation of a device according to the invention for the secure display of image data in an exemplary embodiment.

Die Einrichtung 1 zur sicheren Anzeige von Bilddaten, die beispielsweise in einem erfindungsgemäßen Schienenfahrzeug (nicht dargestellt) eingesetzt ist, umfasst eine Anzeigeeinrichtung 2, eine Vorrichtung 3 zur Ausgabe von Bilddaten und eine Vergleichseinrichtung 4.The device 1 for the secure display of image data, which is used, for example, in a rail vehicle according to the invention (not shown), comprises a display device 2, a device 3 for outputting image data and a comparison device 4.

Die Anzeigeeinrichtung 2 ist beispielsweise ein handelsüblicher TFT-Monitor, auf dem Bilddaten dargestellt werden. Beim Einsatz in dem erfindungsgemäßen Schienenfahrzeug wird auf der Anzeigeeinrichtung 2 beispielsweise ein Tacho oder ähnliches dargestellt. Die Anzeigeeinrichtung 2 wird über eine Stromversorgung 5 mit Energie versorgt.The display device 2 is, for example, a commercially available TFT monitor on which image data is displayed. When used in the rail vehicle according to the invention, a speedometer or something similar, for example, is displayed on the display device 2. The display device 2 is supplied with energy via a power supply 5.

Die Vorrichtung 3 zur Ausgabe von Bilddaten weist eine Einrichtung 6 zur Erzeugung von Bilddaten für die Anzeigeeinrichtung 2 auf. Die Einrichtung 6 ist beispielsweise ein nicht sicherer Computer oder Rechner mit entsprechender Graphikkarte mit Graphikausgabe. Weiterhin umfasst die Vorrichtung 3 zur Ausgabe von Bilddaten ein Verarbeitungsmodul 7, das zwischen der Einrichtung 6 zur Erzeugung von Bilddaten und der Anzeigeeinrichtung 2 angeordnet ist und mit beiden verbunden ist. Das Verarbeitungsmodul 7 umfasst einen Switch 8, einen Deserialisierer 9, eine Recheneinrichtung 10 und zwei Serialisierer 11, 12.The device 3 for outputting image data has a device 6 for generating image data for the display device 2. The device 6 is, for example, a non-secure computer or calculator with a corresponding graphics card with graphics output. Furthermore, the device 3 for outputting image data includes a processing module 7, which is arranged between the device 6 for generating image data and the display device 2 and is connected to both. The processing module 7 includes a switch 8, a deserializer 9, a computing device 10 and two serializers 11, 12.

Die Einrichtung 6 zur Erzeugung von Bilddaten stellt ausgangsseitig die erzeugten Bilddaten 13, eine Signalauswahl 14 und einen Farbkorrekturoffsetwert 15 bereit. Die erzeugten Bilddaten 13 sind beispielsweise ein LVDS-18 Bit Signal mit drei Spuren (Lanes) für die Farbmischung sowie ein Clock-Signal, das den Takt angibt für den Bildaufbau.The device 6 for generating image data provides the generated image data 13, a signal selection 14 and a color correction offset value 15 on the output side. The image data 13 generated is, for example, an LVDS 18-bit signal with three lanes for color mixing and a clock signal that specifies the clock for image construction.

Der Farbkorrekturoffsetwert 15 gibt an, um wieviel der sogenannte Weißpunkt der individuellen Anzeigeeinrichtung 2 fertigungsbedingt von einem theoretisch optimalen Weißpunkt abweicht. Die erzeugten Bilddaten 13, die Signalauswahl 14 und der Farbkorrekturoffsetwert 15 werden an das Verarbeitungsmodul 7 übermittelt.The color correction offset value 15 indicates by how much the so-called white point of the individual display device 2 deviates from a theoretically optimal white point due to manufacturing reasons. The generated image data 13, the signal selection 14 and the color correction offset value 15 are transmitted to the processing module 7.

In dem Verarbeitungsmodul 7 verbindet der Switch 8 die erzeugten Bilddaten 13, die Signalauswahl 14 und ggf. ein Signal 16 von einer externen Schnittstelle 17.In the processing module 7, the switch 8 connects the generated image data 13, the signal selection 14 and, if necessary, a signal 16 from an external interface 17.

Mit der Signalauswahl 14 kann der Switch 8 gesteuert werden, um beispielsweise auf ein externes Signal von der externen Schnittstelle 17 umschalten zu können.The switch 8 can be controlled with the signal selection 14, for example to be able to switch to an external signal from the external interface 17.

Von dem Switch 8 werden ausgangsseitig die erzeugten Bilddaten 13 über den Deserialisierer 9 an die Recheneinrichtung 10 weitergeleitet. Die Bilddaten 13 werden durch den Switch 8 oder den Deserialisier 9 noch nicht verändert Die Recheneinrichtung 10 verändert die erzeugten Bilddaten 13 unter Berücksichtigung des Farbkorrekturoffsetwerts 15 zu veränderten Bilddaten 18. Weiterhin ermittelt die Recheneinrichtung 10 Korrekturdaten 19, die für einen Unterschied zwischen den veränderten Bilddaten 18 und den erzeugten Bilddaten 13 repräsentativ sind. Die veränderten Bilddaten 18 und die Korrekturdaten 19 werden über die Serialisierer 11, 12 geleitet und anschließend von dem Verarbeitungsmodul 7 ausgegeben zur Anzeigeeinrichtung 2 bzw. zur Vergleichseinrichtung 4.On the output side, the generated image data 13 is forwarded from the switch 8 to the computing device 10 via the deserializer 9. The image data 13 are not yet changed by the switch 8 or the deserializer 9. The computing device 10 changes the generated image data 13 to changed image data 18, taking into account the color correction offset value 15. Furthermore, the computing device 10 determines correction data 19, which represents a difference between the changed image data 18 and the generated image data 13 are representative. The changed image data 18 and the correction data 19 are passed via the serializers 11, 12 and then output by the processing module 7 to the display device 2 or to the comparison device 4.

Die Vergleichseinrichtung 4 umfasst eine Überwachungseinrichtung 20 und ebenfalls ein Verarbeitungsmodul 21. Das Verarbeitungsmodul 21 weist zwei Deserialisierer 22, 23, eine Addiereinrichtung 24 und eine Recheneinrichtung 25 auf.The comparison device 4 includes a monitoring device 20 and also a processing module 21. The processing module 21 has two deserializers 22, 23, an adding device 24 and a computing device 25.

Die Korrekturdaten 19 werden von der Vorrichtung 3 zur Ausgabe von Bilddaten direkt an die Vergleichseinrichtung 4 übermittelt. Die veränderten Bilddaten 18 und auch ein Taktsignal 27 werden von der Vorrichtung 3 an die Anzeigeeinrichtung 2 übermittelt. An einem beispielhaft dargestellten Punkt 26 werden die veränderten Bilddaten 18 und das Taktsignal 27 für die Vergleichseinrichtung 4 abgegriffen. In dem Verarbeitungsmodul 21 werden die veränderten Bilddaten 18 und die Korrekturdaten 19 über die Deserialisierer 22, 23 an die Addiereinrichtung 24 übermittelt.The correction data 19 are transmitted directly to the comparison device 4 from the device 3 for outputting image data. The changed image data 18 and also a clock signal 27 are transmitted from the device 3 to the display device 2. The changed image data 18 and the clock signal 27 for the comparison device 4 are tapped at a point 26 shown as an example. In the processing module 21, the changed image data 18 and the correction data 19 are transmitted to the adding device 24 via the deserializers 22, 23.

Die Addiereinrichtung 24 rechnet die veränderten Bilddaten 18 mittels der Korrekturdaten 19 derart zurück, dass diese mit den erzeugten Bilddaten 13 wieder gleich sind. Diese zurückgerechneten Bilddaten 28 werden an die Recheneinrichtung 25 weitergegeben. Von der Überwachungseinrichtung 20 werden Ausschnittskoordinaten 29 an die Recheneinrichtung 25 weitergegeben. Die Ausschnittskoordinaten 29 geben die vorbestimmten Anzeigebereiche der Anzeigeeinrichtung 2 an, die sicherheitsrelevant sind und die somit überprüft werden müssen. Die Bezeichnung Addiereinrichtung 24 ist hier nur beispielhaft zu sehen. In der Addiereinrichtung 24 können auch andere Rechenoperationen als Additionen durchgeführt werden, beispielsweise wenn die Daten zuvor durch Subtraktion gewonnen wurden. In jedem Fall macht die Addiereinrichtung 24 die Rechenoperation rückgängig, so dass die zurückgerechneten Bilddaten 28 wieder mit den erzeugten Bilddaten 13 vergleichbar sind.The adding device 24 recalculates the changed image data 18 using the correction data 19 in such a way that it is again equal to the generated image data 13. This recalculated image data 28 is passed on to the computing device 25. The monitoring device 20 passes section coordinates 29 to the computing device 25. The section coordinates 29 indicate the predetermined display areas of the display device 2 that are safety-relevant and therefore have to be checked. The term adding device 24 is only to be seen as an example here. In the adding device 24, other arithmetic operations than additions can also be carried out, for example if the data was previously obtained by subtraction. In any case, the adding device 24 reverses the arithmetic operation so that the recalculated image data 28 can again be compared with the generated image data 13.

In der Recheneinrichtung 25 werden für die zurückgerechneten Bilddaten Prüfwerte berechnet. Dies geschieht, um die Überprüfung der Bilddaten zu vereinfachen. Um die Berechnungen weiter zu vereinfachen, werden die Prüfwerte für die zurückgerechneten Bilddaten 28 lediglich in den durch die Ausschnittskoordinaten 29 definierten Anzeigebereichen berechnet. Die Prüfwerte sind beispielsweise Prüfsummen oder Checksummen. Die Prüfwerte 30 werden von der Recheneinrichtung 25 schließlich an die Überwachungseinrichtung 20 übermittelt. In der Überwachungseinrichtung 20 werden die Prüfwerte 30 mit vorgegebenen und abgespeicherten Sollwert-Prüfwerten (nicht dargestellt) verglichen. Diese Sollwert-Prüfwerte sind aus erwarteten Sollwert-Bilddaten (nicht dargestellt) ermittelt worden. Nur wenn die Prüfwerte 30 mit den Sollwert-Prüfwerten in der Überwachungseinrichtung 20 gleich sind, ist die Anzeige auf der Anzeigeeinrichtung 2 sicher und der Betrieb kann fortgesetzt werden. Sollte von der Überwachungseinrichtung 20 eine Abweichung zwischen Prüfwert 30 und Sollwert-Prüfwert festgestellt werden, gibt die Überwachungseinrichtung 20 ein entsprechendes Abschaltsignal 31 an eine Abschaltvorrichtung 32. Die Abschalteinrichtung 32 ist beispielsweise ein entsprechender Schalter, der die Stromversorgung 5 der Anzeigeeinrichtung 2 unterbricht.In the computing device 25, test values are calculated for the recalculated image data. This is done to make checking the image data easier. In order to further simplify the calculations, the test values for the recalculated image data 28 are only calculated in the display areas defined by the section coordinates 29. The test values are, for example, checksums or checksums. The test values 30 are finally transmitted from the computing device 25 to the monitoring device 20. In the monitoring device 20, the test values 30 are compared with predetermined and stored target value test values (not shown). These target value test values were determined from expected target value image data (not shown). Only if the test values 30 are the same as the setpoint test values in the monitoring device 20 is the display on the display device 2 safe and operation can be continued. If the monitoring device 20 detects a deviation between the test value 30 and the setpoint test value, the monitoring device 20 sends a corresponding switch-off signal 31 to a switch-off device 32. The switch-off device 32 is, for example, a corresponding one Switch that interrupts the power supply 5 of the display device 2.

Mit Hilfe der Erfindung, in der in der Figur beispielhaft dargestellten Ausführungsform kann ein Weißabgleich für die Anzeigeeinrichtung 2 individuell ausgeführt werden. So können individuelle fertigungsbedingte Ungenauigkeiten der Anzeigeeinrichtung 2 ausgeglichen werden. Trotz der individuell veränderten Bilddaten 18 können diese mit Hilfe der Korrekturdaten 19 überprüft werden, um die Voraussetzungen für eine sichere Anzeige in einer sicherheitsrelevanten Anlage zu erfüllen. With the aid of the invention, in the embodiment shown as an example in the figure, a white balance can be carried out individually for the display device 2. In this way, individual manufacturing-related inaccuracies of the display device 2 can be compensated. Despite the individually modified image data 18, these can be checked using the correction data 19 in order to meet the requirements for a safe display in a safety-relevant system.

Claims (10)

  1. Method for outputting image data for a secure display,
    in which the image data (13) is generated for a display facility (2),
    in which the generated image data (13) is changed as a function of a colour correction offset value (15) as a characteristic variable (15) of the display facility (2) and in which correction data (19) is determined which is representative of a difference between generated and changed image data (13, 18), wherein
    the changed image data (18) is calculated back by means of the correction data (19) such that it can be compared again with the generated image data (13) and the image data (28)
    calculated back is compared with predetermined expected target value image data, and/or is calculated from the predetermined expected target value image data and this is compared with the changed image data (28).
  2. Method according to claim 1,
    characterised in that
    at least one test value (30) for the image data (28) calculated back or the changed target value image data is determined and for comparison the test value of the image data calculated back or the changed target value image data is compared with at least one test value for the expected target value image data.
  3. Method according to one of the claims mentioned above,
    characterised in that
    a reaction which can be perceived by the user is triggered, in particular a power supply (5) of the display facility (2) is switched off if a result which is not permissible is determined as part of the comparison.
  4. Method according to one of the claims mentioned above,
    characterised in that
    only the image data or test values relating to predetermined display regions of the display facility are compared.
  5. Method according to one of the claims mentioned above,
    characterised in that
    the correction data (19) is checked as to whether it lies in a predetermined permissible region.
  6. Method according to one of the claims mentioned above,
    characterised in that
    the correction data (19) is determined on a pixel-by-pixel basis and/or a colour basis.
  7. Apparatus (13) for outputting image data for a secure display,
    with at least one facility (6) for generating image data for a display facility (2),
    with at least one computing facility (10) which is designed to change the generated image data as a function of a colour correction offset value (15) as a characteristic variable (15) of the display facility and to determine correction data (19) which is representative of a difference between generated and changed image data,
    wherein the changed image data (18) is calculated back by means of the correction data (19) such that it can be compared again with the generated image data (13), and the image data (28) calculated back is compared with predetermined expected target value image data, and/or changed target value image data is calculated from the predetermined expected target value image data and the correction data and this is compared with the changed image data (28).
  8. Apparatus according to claim 7,
    characterised in that
    the apparatus is designed to carry out the method according to one of claims 1 to 6.
  9. Rail vehicle with at least one display facility (2) designed for secure display,
    characterised in that
    the rail vehicle has an apparatus (3) according to one of claims 7 or 8.
  10. Facility (1) for secure display of image data, with a display facility (2),
    with an apparatus (3) for outputting image data and with a comparison facility (4) for checking the image data,
    characterised in that
    the apparatus (3) for outputting image data is designed according to one of claims 7 or 8.
EP18753075.3A 2017-08-24 2018-07-25 Apparatus and method for displaying image data on a secured display Active EP3646313B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102017214814.4A DE102017214814A1 (en) 2017-08-24 2017-08-24 Method and device for outputting image data for a secure display
PCT/EP2018/070150 WO2019038009A1 (en) 2017-08-24 2018-07-25 Method and device for outputting image data for secure display

Publications (2)

Publication Number Publication Date
EP3646313A1 EP3646313A1 (en) 2020-05-06
EP3646313B1 true EP3646313B1 (en) 2024-04-03

Family

ID=63168366

Family Applications (1)

Application Number Title Priority Date Filing Date
EP18753075.3A Active EP3646313B1 (en) 2017-08-24 2018-07-25 Apparatus and method for displaying image data on a secured display

Country Status (3)

Country Link
EP (1) EP3646313B1 (en)
DE (1) DE102017214814A1 (en)
WO (1) WO2019038009A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4432273A1 (en) * 2023-03-16 2024-09-18 Siemens Mobility GmbH Method for the computer-aided operation of a secure optical display

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE4332143A1 (en) 1993-09-17 1995-03-23 Siemens Ag Process for operating a visual display device and devices for carrying out the process
KR20090091837A (en) * 2008-02-26 2009-08-31 명관 이 System and method to monitor a wheel of train
EP2671374B1 (en) * 2011-01-31 2015-07-22 Dolby Laboratories Licensing Corporation Systems and methods for restoring color and non-color related integrity in an image
JP2016042954A (en) * 2014-08-21 2016-04-04 キヤノン株式会社 Display control device, display system, and display control method

Also Published As

Publication number Publication date
EP3646313A1 (en) 2020-05-06
WO2019038009A1 (en) 2019-02-28
DE102017214814A1 (en) 2019-02-28

Similar Documents

Publication Publication Date Title
DE102015002973B4 (en) Method for the joint representation of safety-critical and non-safety-critical information and display device
EP2198421A1 (en) Test device, display device and method for testing the validity of display signals
EP2833349B1 (en) Method and device for representing a safety-relevant state
EP2595016A1 (en) Method, computer program, computer-readable medium and computing unit for operating field devices
EP1989470A1 (en) Safety concept for a transmission actuator device
DE102017218643A1 (en) Function module, control unit for an operation assistance system and working device
DE102007029321B4 (en) Method for operating a field device in a user-friendly mode
EP3646313B1 (en) Apparatus and method for displaying image data on a secured display
WO2013164181A1 (en) Method for depicting safety-critical data via a display unit; display unit
EP3074293B1 (en) Method for revealing errors in a signal-box computer system, and signal-box computer system
DE102019205237A1 (en) Displaying ASIL-D information using a less secure device
EP1416283B1 (en) Device for displaying vehicle speed
EP0465793A2 (en) Multiprocessor system for control and diagnostic device in a motor vehicle
DE102006057708A1 (en) System architecture for securing updation of firmware, has processor, system bus for data transfer to and from processor and local memory unit for storing firmware
WO2014048641A1 (en) User interface and method for error diagnosis of an industrial plant
DE102010026392A1 (en) Procedure for the safe parameterization of a safety device
EP2941738A1 (en) Method for revealing errors in a signal box computer system, and signal box computer system
EP1702827A1 (en) Control panel
DE102008012953A1 (en) Method for checking control and display system of rail vehicle, involves determining system information over system of rail vehicle or configuration information over configuration of rail vehicle with starting of rail vehicle
EP3317856B1 (en) Method for checking the accuracy of a representation of image data on a display means and display device
EP0977100A1 (en) Safety device for a programmable logic controller
WO2001055852A1 (en) Method and device for control of a display device for a railway control system
DE102012212780A1 (en) Operating device for inputting operating commands for controlling a technical system
DE102008038618A1 (en) Failure condition indicating method for track-bound vehicle, involves activating lights by output devices during reception of control signal in cab or activating failure indicator lamp during reception of control signal
DD262930A5 (en) Method and arrangement for entering information into fail-safe computer systems

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20200128

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20211216

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20231123

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 502018014383

Country of ref document: DE

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20240403

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240403

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240403

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240803

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240403

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240403

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240403

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240919

Year of fee payment: 7

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240704

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240805

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240403

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240403

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: AT

Payment date: 20240606

Year of fee payment: 7

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240403

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240403

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240805

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240403

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240703

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240403

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240803

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240403

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240704

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240403

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240403

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240403

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240403

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240703