EP2289056B1 - Portable device with warning system and method - Google Patents

Portable device with warning system and method Download PDF

Info

Publication number
EP2289056B1
EP2289056B1 EP08874596A EP08874596A EP2289056B1 EP 2289056 B1 EP2289056 B1 EP 2289056B1 EP 08874596 A EP08874596 A EP 08874596A EP 08874596 A EP08874596 A EP 08874596A EP 2289056 B1 EP2289056 B1 EP 2289056B1
Authority
EP
European Patent Office
Prior art keywords
detection system
approach
portable device
movement
output
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Not-in-force
Application number
EP08874596A
Other languages
German (de)
French (fr)
Other versions
EP2289056A1 (en
Inventor
Frank Fischer
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Robert Bosch GmbH
Original Assignee
Robert Bosch GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Robert Bosch GmbH filed Critical Robert Bosch GmbH
Publication of EP2289056A1 publication Critical patent/EP2289056A1/en
Application granted granted Critical
Publication of EP2289056B1 publication Critical patent/EP2289056B1/en
Not-in-force legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/005—Traffic control systems for road vehicles including pedestrian guidance indicator
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • G08G1/166—Anti-collision systems for active traffic, e.g. moving vehicles, pedestrians, bikes

Definitions

  • the invention relates to a portable device for presenting information according to claim 1 and a method for outputting information according to claim 10.
  • the object of the invention is to provide an improved portable device for presenting information and an improved method for outputting information.
  • the object of the invention is achieved by the portable device according to claim 1 and by the method according to claim 10.
  • An advantage of the invention is that the user of the portable device is warned when approaching an object. This is achieved by providing a detection system which detects an approach of the portable device to an object and outputs a signal. In this way, safety in using the portable device is increased.
  • the portable device may be, for example, a mobile phone, a navigation device, a video device or an MP3 player or a PDA.
  • a fixed distance to an object or a collision course with respect to the object can be detected and used for the output of the warning signal.
  • an inertial sensor system can be used. In this way, known systems for a precise detection of the movement and / or the direction of movement can be used.
  • the detection system has an ultrasound or infrared or light or radar unit with which a distance to an object can be measured.
  • the detection system has a camera and an image evaluation unit with which an approach to an object can be detected.
  • a camera that is already integrated, for example, in a mobile phone, in addition to detecting an approach to an object can be used.
  • the detection system comprises a digital map and a location system.
  • the digital map With the help of the digital map, the position of objects can be detected and the relative position of the portable device to the objects can be determined. Even with this system, an approach to an object can be determined simply and precisely.
  • the detection system comprises a step measuring system, with which steps of the user of the portable device and thus a movement of the portable device can be determined in a simple manner
  • the device serves to assist a visually impaired person, so that the warning of objects additionally improves the safety of the movement of the visually impaired person.
  • FIG. 1 shows in a schematic representation of a portable device 1.
  • the portable device is used to display and output of optical and / or acoustic information.
  • the portable device may be in the form of, for example, a mobile phone, an MP3 player, a personal digital assistant (PDA), a navigation device, a video device or a device for assisting a visually impaired person.
  • the portable device 1 has, depending on the embodiment, the corresponding electrical and mechanical components of the embodiment.
  • the portable device 1 may include a display 2, a speaker 3, a data and program memory 4, a detection system 5, a keyboard 6, a camera 7, an inertial sensor system 8, a distance measuring system 9, a speaker 10 and / or a vibration generator 11 exhibit.
  • video information for example a computer game
  • an SMS text for example a computer game
  • navigation information for example a photograph
  • acoustic information for example music or a voice output, can be output with the aid of the loudspeaker 10.
  • the detection system 5 serves to detect an approach of the portable device 1 to an object, for example a building or a lamppost or a road or a vehicle.
  • the approach may be, for example, that a predetermined distance between an object and the portable device is exceeded. This can be checked, for example, with the aid of a distance measuring system 9.
  • the distance measuring system may for example be in the form of an ultrasound, an infrared, a light, or a radar system, with which the distance between the portable device 1 and an object is detected. Recognizes the detection system 5 that the distance between the portable device 1 and the object is smaller than a predetermined distance, a warning signal, such as an acoustic and / or an optical and / or a vibration signal by means of a vibration generator 11 is output.
  • optical proximity switches can be used.
  • the detection system 5 has a camera 7 with which an image of the surroundings is taken. Furthermore, in this embodiment, the detection system 5 is connected to an image evaluation unit, which can detect an approach to an object, for example a building, a street, a lamppost, a tree, another person or a car, based on the image taken with the camera 7. For this purpose, the image evaluation unit can detect an object independently and also determine an approximation due to the change in the size of the object. Under approach is understood both a predetermined distance and a given collision course on an object. If the detection system 5 detects that there is a collision course with respect to an object or falls below a predetermined distance from the object, then an acoustic, an optical or a vibration signal is output. A collision course is present when the portable device 1 moves in the direction of an object and falls below a predetermined minimum distance.
  • an image evaluation unit can detect an approach to an object, for example a building, a street, a lamppost, a tree, another person or a car, based on the
  • an inertial sensor system 8 can be used to determine the movement and the direction of movement of the portable device 1.
  • the inertial sensor system 8 can detect a moving direction and a moving speed or determine whether the portable device 1 is used during walking (step detection).
  • the detection system 5 comprises a location system such as a GPS, with which the position of the portable device and a direction of movement and movement of the portable device 1 can be detected.
  • a digital map can be stored in the data and program memory 4, which is used to correct the location and / or to determine positions of objects. For example, positions of streets and buildings are stored in the digital map, so that the approach to a street or the Approaching a building based on the data of the digital map can be detected and a warning signal can alert to attention.
  • the detection system may include a step detection system that detects movement of the portable device 1 based on the steps of the user. Often the reminder function is only interesting while running.
  • the signals and measured values of the various embodiments of the detection system 5 can be combined with one another in the form of subsystems or in the form of overall systems in order to determine an approach to an object.
  • data on objects of the digital map can be used to enhance the approximation using image recognition.
  • the data of the location determination unit may be used to improve the position of the portable device 1 in consideration of the signals of the inertial sensor system 8.
  • the data of the digital map can be combined with the signals of the distance measuring system 9 in order to obtain a more accurate statement about an approach to an object.
  • the described portable device is particularly suitable for additionally providing a warning signal to people with visual impairments when they approach an object, in particular approaching at a certain distance or are in collision course with the object.
  • FIG. 2 shows a situation in which a person 12 carries a portable device 1, wherein the portable device 1 outputs audible and / or visual information.
  • the portable device 1 detects an approach to an object 13, which is formed in the form of a lamppost. If the portable device recognizes, with the aid of the detection system 5, that the person 12 is approaching the object 13 closer than a predetermined minimum distance, for example 1 meter, then the portable device 1 becomes an acoustic and / or an optical and / or a vibration signal output.
  • the detection system can also determine a collision course in the direction of the object and output, for example, falls below a minimum distance an audible, visual or vibrating warning signal. In this way, the person 12 is warned of an undesirable collision with the object 13. This increases safety for the operator of the portable device.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Traffic Control Systems (AREA)
  • Emergency Alarm Devices (AREA)
  • Telephone Function (AREA)
  • Navigation (AREA)
  • Telephonic Communication Services (AREA)

Abstract

A portable device for displaying information, in particular a mobile telephone, having a display for outputting image or text information, a detection system, and an output unit. The detection system is designed to detect an approach of the portable device to an object. The output unit may output a warning signal in the event of a predefined approach to the object, in particular a defined distance.

Description

Die Erfindung betrifft eine tragbare Vorrichtung zum Darstellen von Informationen gemäß Anspruch 1 und ein Verfahren zur Ausgabe von Informationen gemäß Anspruch 10.The invention relates to a portable device for presenting information according to claim 1 and a method for outputting information according to claim 10.

Stand der TechnikState of the art

In den letzten Jahren sind verschiedene Zusatzfunktionen wie z. B. SMS, Spiele-Anwendungen, Fußgängernavigation, Fotographie, Videodarstellung und Wegstreckenmessung für tragbare Vorrichtungen wie z. B. Mobiltelefone und andere tragbare Vorrichtungen wie z. B. MP3-Player, PDA, Navigationsgeräte entwickelt worden. Derartige Vorrichtungen sind bekannt z.B. aus der US 2006/0261 978 A1 sowie der EP 1 754 954 A1 .In recent years, various additional functions such. As SMS, games applications, pedestrian navigation, photography, video presentation and distance measurement for portable devices such. As mobile phones and other portable devices such. As MP3 player, PDA, navigation devices have been developed. Such devices are known, for example from the US 2006/0261 978 A1 as well as the EP 1 754 954 A1 ,

Eine sichtbare Folge ist die allgegenwärtige Präsenz und der Einsatz von tragbaren Vorrichtungen bei Fußgängern. Durch Ablenkung aufgrund der Konzentration auf die mobile Anwendung sinkt die Aufmerksamkeit der Fußgänger für die Sicherheit im Straßenverkehr, wodurch es immer wieder zu Unfällen kommt.One visible consequence is the ubiquitous presence and use of portable devices in pedestrians. Distracted by the concentration on the mobile application, pedestrian attention is reduced to road safety, resulting in accidents.

Offenbarung der ErfindungDisclosure of the invention

Die Aufgabe der Erfindung besteht darin, eine verbesserte tragbare Vorrichtung zum Darstellen von Informationen und ein verbessertes Verfahren zur Ausgabe von Informationen bereitzustellen.The object of the invention is to provide an improved portable device for presenting information and an improved method for outputting information.

Die Aufgabe der Erfindung wird durch die tragbare Vorrichtung gemäß Patentanspruch 1 und durch das Verfahren gemäß Patentanspruch 10 gelöst.The object of the invention is achieved by the portable device according to claim 1 and by the method according to claim 10.

Ein Vorteil der Erfindung besteht darin, dass der Nutzer der tragbaren Vorrichtung bei Annäherung an ein Objekt gewarnt wird. Dies wird dadurch erreicht, dass ein Detektionssystem vorgesehen ist, die ein Annähern der tragbaren Vorrichtung an ein Objekt erkennt und ein Signal ausgibt. Auf diese Weise wird die Sicherheit bei der Benutzung der tragbaren Vorrichtung erhöht.An advantage of the invention is that the user of the portable device is warned when approaching an object. This is achieved by providing a detection system which detects an approach of the portable device to an object and outputs a signal. In this way, safety in using the portable device is increased.

Weitere vorteilhafte Ausführungsformen der Erfindung sind in den abhängigen Ansprüchen angegeben.Further advantageous embodiments of the invention are specified in the dependent claims.

Die tragbare Vorrichtung kann beispielsweise ein Mobiltelefon, ein Navigationsgerät, ein Videogerät oder ein MP3-Spieler oder ein PDA sein.The portable device may be, for example, a mobile phone, a navigation device, a video device or an MP3 player or a PDA.

Abhängig von der gewählten Ausführungsform kann beispielsweise ein festgelegter Abstand zu einem Objekt oder ein Kollisionskurs in Bezug auf das Objekt erfasst und für die Ausgabe des Warnsignals verwendet werden.Depending on the chosen embodiment, for example, a fixed distance to an object or a collision course with respect to the object can be detected and used for the output of the warning signal.

Zur Erkennung der Bewegung kann ein Inertialsensorsystem verwendet werden. Auf diese Weise können bekannte Systeme für eine präzise Erkennung der Bewegung und/oder der Bewegungsrichtung eingesetzt werden.To detect the movement, an inertial sensor system can be used. In this way, known systems for a precise detection of the movement and / or the direction of movement can be used.

In einer weiteren Ausführungsform weist das Detektionssystem eine Ultraschall- oder Infrarot- oder Licht- oder Radareinheit auf, mit der ein Abstand zu einem Objekt gemessen werden kann.In a further embodiment, the detection system has an ultrasound or infrared or light or radar unit with which a distance to an object can be measured.

In einer weiteren Ausführungsform weist das Detektionssystem eine Kamera und eine Bildauswertungseinheit auf, mit der eine Annäherung an ein Objekt erfasst werden kann. Auf diese Weise kann eine Kamera, die beispielsweise bei einem Mobiltelefon schon integriert ist, zusätzlich zur Erkennung einer Annäherung an ein Objekt verwendet werden. Somit treten keine größeren Zusatzkosten auf.In a further embodiment, the detection system has a camera and an image evaluation unit with which an approach to an object can be detected. In this way, a camera that is already integrated, for example, in a mobile phone, in addition to detecting an approach to an object can be used. Thus, there are no major additional costs.

In einer weiteren Ausführungsform weist das Detektionssystem eine digitale Karte und ein Ortsbestimmungssystem auf. Mit Hilfe der digitalen Karte kann die Position von Objekten erfasst und die Relativposition der tragbaren Vorrichtung zu den Objekten ermittelt werden. Auch mit diesem System kann eine Annäherung an ein Objekt einfach und präzise ermittelt werden.In another embodiment, the detection system comprises a digital map and a location system. With the help of the digital map, the position of objects can be detected and the relative position of the portable device to the objects can be determined. Even with this system, an approach to an object can be determined simply and precisely.

Erfindungsgemäß weist das Detektionssystem ein Schrittmesssystem auf, mit dem Schritte des Benutzers der tragbaren Vorrichtung und damit eine Bewegung der tragbaren Vorrichtung auf einfache Weise ermittelt werden konnenAccording to the invention, the detection system comprises a step measuring system, with which steps of the user of the portable device and thus a movement of the portable device can be determined in a simple manner

In einer weiteren Ausführungsform dient die Vorrichtung zur Unterstützung eines sehbehinderten Menschen, so dass die Warnung vor Objekten zusätzlich die Sicherheit für die Bewegung des sehbehinderten Menschen verbessert.In a further embodiment, the device serves to assist a visually impaired person, so that the warning of objects additionally improves the safety of the movement of the visually impaired person.

Kurze Beschreibung der Zeichnungen

  • Figur 1 zeigt in einer schematischen Darstellung eine tragbare Vorrichtung zur Ausgabe von optischen und/oder akustischen Informationen, und
  • Figur 2 zeigt einen Fußgänger, der die tragbare Vorrichtung benutzt.
Brief description of the drawings
  • FIG. 1 shows a schematic representation of a portable device for outputting optical and / or acoustic information, and
  • FIG. 2 shows a pedestrian using the portable device.

Ausführungsform(en) der ErfindungEmbodiment (s) of the invention

Figur 1 zeigt in einer schematischen Darstellung eine tragbare Vorrichtung 1. Die tragbare Vorrichtung dient zur Darstellung und Ausgabe von optischen und/oder akustischen Informationen. Die tragbare Vorrichtung kann beispielsweise in Form eines Mobiltelefons, eines MP3-Spielers, eines PDA (personal digital assistant), eines Navigationsgerätes, eines Videogerätes oder einer Vorrichtung zur Unterstützung eines sehbehinderten Menschen ausgebildet sein. Die tragbare Vorrichtung 1 weist abhängig von der Ausbildungsform die entsprechenden elektrischen und mechanischen Bauteile der Ausbildungsform auf. Beispielsweise kann die tragbare Vorrichtung 1 eine Anzeige 2, einen Lautsprecher 3, einen Daten- und Programmspeicher 4, ein Detektionssystem 5, eine Tastatur 6, eine Kamera 7, ein Inertialsensorsystem 8, ein Abstandsmesssystem 9, einen Lautsprecher 10 und/oder einen Vibrationsgenerator 11 aufweisen. Mit Hilfe der Anzeige 2 kann eine Videoinformation, beispielsweise ein Computerspiel, ein SMS-Text, eine Navigationsinformation, ein Foto, ein Video oder eine Wegstreckenmessinformation ausgegeben werden. Zudem kann mit Hilfe des Lautsprechers 10 eine akustische Information, beispielsweise Musik oder eine Sprachausgabe ausgegeben werden. FIG. 1 shows in a schematic representation of a portable device 1. The portable device is used to display and output of optical and / or acoustic information. The portable device may be in the form of, for example, a mobile phone, an MP3 player, a personal digital assistant (PDA), a navigation device, a video device or a device for assisting a visually impaired person. The portable device 1 has, depending on the embodiment, the corresponding electrical and mechanical components of the embodiment. For example, the portable device 1 may include a display 2, a speaker 3, a data and program memory 4, a detection system 5, a keyboard 6, a camera 7, an inertial sensor system 8, a distance measuring system 9, a speaker 10 and / or a vibration generator 11 exhibit. With the aid of the display 2, video information, for example a computer game, an SMS text, navigation information, a photograph, a video or a distance measurement information can be output. In addition, acoustic information, for example music or a voice output, can be output with the aid of the loudspeaker 10.

Das Detektionssystem 5 dient dazu, eine Annäherung der tragbaren Vorrichtung 1 zu einem Objekt, beispielsweise einem Gebäude oder einem Laternenpfahl oder einer Straße oder einem Fahrzeug zu erfassen. Die Annäherung kann beispielsweise darin bestehen, dass ein vorgegebener Abstand zwischen einem Objekt und der tragbaren Vorrichtung unterschritten wird. Dies kann beispielsweise mit Hilfe eines Abstandsmesssystems 9 überprüft werden. Das Abstandsmesssystem kann beispielsweise in Form eines Ultraschall-, eines Infrarot-, eines Licht-, oder eines Radarsystems ausgebildet sein, mit dem der Abstand zwischen der tragbaren Vorrichtung 1 und einem Objekt erfasst wird. Erkennt das Detektionssystem 5, dass der Abstand zwischen der tragbaren Vorrichtung 1 und dem Objekt kleiner als ein vorgegebener Abstand ist, so wird ein Warnsignal, beispielsweise ein akustisches und/oder ein optisches und/oder ein Vibrationssignal mit Hilfe eines Vibrationsgenerators 11 ausgegeben.The detection system 5 serves to detect an approach of the portable device 1 to an object, for example a building or a lamppost or a road or a vehicle. The approach may be, for example, that a predetermined distance between an object and the portable device is exceeded. This can be checked, for example, with the aid of a distance measuring system 9. The distance measuring system may for example be in the form of an ultrasound, an infrared, a light, or a radar system, with which the distance between the portable device 1 and an object is detected. Recognizes the detection system 5 that the distance between the portable device 1 and the object is smaller than a predetermined distance, a warning signal, such as an acoustic and / or an optical and / or a vibration signal by means of a vibration generator 11 is output.

Es ist nicht zwingend nötig, den Abstand exakt zu bestimmen. Ferner kann es ausreichend sein, dass eine kritische Umgebungssituation erkannt wird. Hierfür können z. B. optische Näherungsschalter eingesetzt werden.It is not absolutely necessary to determine the distance exactly. Furthermore, it may be sufficient that a critical environmental situation is detected. For this purpose, for. B. optical proximity switches can be used.

In einer weiteren Ausführungsform verfügt das Detektionssystem 5 über eine Kamera 7, mit der ein Bild der Umgebung aufgenommen wird. Weiterhin ist das Detektionssystem 5 in dieser Ausführungsform mit einer Bildauswerteeinheit verbunden, die aufgrund des mit der Kamera 7 aufgenommenen Bildes eine Annäherung an ein Objekt, beispielsweise ein Gebäude, eine Straße, einen Laternenmasten, einen Baum, eine andere Person oder ein Auto erkennen kann. Dazu kann die Bildauswerteeinheit ein Objekt selbständig erfassen und aufgrund der Änderung der Größe des Objektes auch eine Annäherung feststellen. Unter Annäherung wird sowohl ein vorgegebener Abstand als auch ein vorgegebener Kollisionskurs auf ein Objekt verstanden. Erkennt das Detektionssystem 5, dass ein Kollisionskurs in Bezug auf ein Objekt besteht oder ein vorgegebener Abstand zum Objekt unterschritten wird, so wird ein akustisches, ein optisches oder ein Vibrationssignal ausgegeben. Ein Kollisionskurs liegt dann vor, wenn sich die tragbare Vorrichtung 1 in Richtung auf ein Objekt bewegt und ein vorgegebener Mindestabstand unterschritten wird.In a further embodiment, the detection system 5 has a camera 7 with which an image of the surroundings is taken. Furthermore, in this embodiment, the detection system 5 is connected to an image evaluation unit, which can detect an approach to an object, for example a building, a street, a lamppost, a tree, another person or a car, based on the image taken with the camera 7. For this purpose, the image evaluation unit can detect an object independently and also determine an approximation due to the change in the size of the object. Under approach is understood both a predetermined distance and a given collision course on an object. If the detection system 5 detects that there is a collision course with respect to an object or falls below a predetermined distance from the object, then an acoustic, an optical or a vibration signal is output. A collision course is present when the portable device 1 moves in the direction of an object and falls below a predetermined minimum distance.

Beispielsweise kann zur Ermittlung der Bewegung und der Bewegungsrichtung der tragbaren Vorrichtung 1 ein Inertialsensorsystem 8 verwendet werden. Das Inertialsensorsystem 8 kann eine Bewegungsrichtung und eine Bewegungsgeschwindigkeit erfassen oder feststellen, ob die tragbare Vorrichtung 1 während des Gehens verwendet wird (Schritterkennung).For example, an inertial sensor system 8 can be used to determine the movement and the direction of movement of the portable device 1. The inertial sensor system 8 can detect a moving direction and a moving speed or determine whether the portable device 1 is used during walking (step detection).

In einer weiteren Ausführungsform weist das Detektionssystem 5 ein Ortsbestimmungssystem wie beispielsweise ein GPS auf, mit dem die Position der tragbaren Vorrichtung und eine Bewegungsrichtung und eine Bewegung der tragbaren Vorrichtung 1 erfasst werden kann. Zudem kann eine digitale Karte im Daten- und Programmspeicher 4 abgelegt sein, die zur Korrektur der Ortsbestimmung und/oder zur Festlegung von Positionen von Objekten verwendet wird. Beispielsweise sind in der digitalen Karte Positionen von Straßen und Gebäuden abgespeichert, so dass die Annäherung an eine Straße oder die Annäherung an ein Gebäude aufgrund der Daten der digitalen Karte erfasst werden kann und ein Erinnerungssignal zur Aufmerksamkeit warnen kann.In a further embodiment, the detection system 5 comprises a location system such as a GPS, with which the position of the portable device and a direction of movement and movement of the portable device 1 can be detected. In addition, a digital map can be stored in the data and program memory 4, which is used to correct the location and / or to determine positions of objects. For example, positions of streets and buildings are stored in the digital map, so that the approach to a street or the Approaching a building based on the data of the digital map can be detected and a warning signal can alert to attention.

In einer weiteren Ausführungsform kann das Detektionssystem ein Schritterfassungssystem aufweisen, das eine Bewegung der tragbaren Vorrichtung 1 anhand der Schritte des Benutzers erkennt. Häufig ist die Erinnerungsfunktion nur während des Laufens interessant.In another embodiment, the detection system may include a step detection system that detects movement of the portable device 1 based on the steps of the user. Often the reminder function is only interesting while running.

Die Signale und Messwerte der verschiedenen Ausführungsformen des Detektionssystems 5 können miteinander in Form von Teilsystemen oder in Form von Gesamtsystemen kombiniert werden, um eine Annäherung an ein Objekt zu ermitteln.The signals and measured values of the various embodiments of the detection system 5 can be combined with one another in the form of subsystems or in the form of overall systems in order to determine an approach to an object.

Beispielsweise können Daten über Objekte der digitalen Karte benutzt werden, um die Annäherung mit Hilfe der Bilderkennung zu verbessern. Zudem können die Daten der Ortsbestimmungseinheit dazu verwendet werden, um die Position der tragbaren Vorrichtung 1 unter Berücksichtigung der Signale des Inertialsensorsystems 8 zu verbessern. Weiterhin können die Daten der digitalen Karte mit den Signalen des Abstandsmesssystems 9 kombiniert werden, um eine genauere Aussage über eine Annäherung an ein Objekt zu erhalten.For example, data on objects of the digital map can be used to enhance the approximation using image recognition. In addition, the data of the location determination unit may be used to improve the position of the portable device 1 in consideration of the signals of the inertial sensor system 8. Furthermore, the data of the digital map can be combined with the signals of the distance measuring system 9 in order to obtain a more accurate statement about an approach to an object.

Die beschriebene tragbare Vorrichtung eignet sich insbesondere dazu, um Personen mit Sehbehinderungen zusätzlich mit einem Warnsignal zu versorgen, wenn sich diese einem Objekt annähern, insbesondere unter einem bestimmten Abstand annähern oder sich auf einem Kollisionskurs mit dem Objekt befinden.The described portable device is particularly suitable for additionally providing a warning signal to people with visual impairments when they approach an object, in particular approaching at a certain distance or are in collision course with the object.

Figur 2 zeigt eine Situation, bei der eine Person 12 eine tragbare Vorrichtung 1 trägt, wobei die tragbare Vorrichtung 1 akustische und/oder optische Informationen ausgibt. Zudem ermittelt die tragbare Vorrichtung 1 eine Annäherung an ein Objekt 13, das in Form eines Laternenmastes ausgebildet ist. Erkennt die tragbare Vorrichtung mit Hilfe des Detektionssystems 5, dass sich die Person 12 näher als einen vorgegebenen Mindestabstand, beispielsweise 1 Meter, an das Objekt 13 nähert, so wird von der tragbaren Vorrichtung 1 ein akustisches und/oder ein optisches und/oder ein Vibrationssignal ausgegeben. Außerdem kann das Detektionssystem auch einen Kollisionskurs in Richtung auf das Objekt ermitteln und beispielsweise bei Unterschreiten eines Mindestabstandes ein akustisches, optisches oder vibrierendes Warnsignal ausgeben. Auf diese Weise wird die Person 12 vor einem unerwünschten Zusammenstoßen mit dem Objekt 13 gewarnt. Dadurch wird die Sicherheit für die Bedienperson der tragbaren Vorrichtung erhöht. FIG. 2 shows a situation in which a person 12 carries a portable device 1, wherein the portable device 1 outputs audible and / or visual information. In addition, the portable device 1 detects an approach to an object 13, which is formed in the form of a lamppost. If the portable device recognizes, with the aid of the detection system 5, that the person 12 is approaching the object 13 closer than a predetermined minimum distance, for example 1 meter, then the portable device 1 becomes an acoustic and / or an optical and / or a vibration signal output. In addition, the detection system can also determine a collision course in the direction of the object and output, for example, falls below a minimum distance an audible, visual or vibrating warning signal. In this way, the person 12 is warned of an undesirable collision with the object 13. This increases safety for the operator of the portable device.

Claims (10)

  1. Portable apparatus (1) for presenting information, particularly a mobile telephone, having a display for the output of picture or text information, having a detection system (5), having an output unit (2, 3, 11), wherein the detection system (5) is designed to recognize an approach by the portable apparatus (1) to an object (13), wherein a prescribed approach to the object (13), particularly a stipulated distance, can prompt the output unit (2, 3, 11) to be used to output a signal, characterized in that the detection system has a pedometer system which can be used to ascertain a movement by an operator.
  2. Apparatus according to Claim 1, wherein the portable apparatus is a mobile telephone or a navigation appliance or a video appliance or an MP3 player or a PDA.
  3. Apparatus according to one of Claims 1 and 2, wherein the detection system (5) is designed to recognize a collision course between the portable apparatus and the object as a prescribed approach.
  4. Apparatus according to one of Claims 1 to 3, wherein the detection system (5) has an inertial sensor system (8) in order to sense a movement and/or a direction of movement of the apparatus.
  5. Apparatus according to one of Claims 1 to 4, wherein the detection system (5) has an ultrasound or infrared or light or radar unit (9) which can be used to measure a distance to the object (13).
  6. Apparatus according to one of Claims 1 to 5, wherein the detection system (5) has a camera (7) and an image evaluation unit (14) in order to sense an approach to the object.
  7. Apparatus according to one of Claims 1 to 6, wherein the detection system (5) has a digital map and a position-finding system, particularly a GPS system, wherein the digital map and the position-finding system can be used to ascertain an approach to an object (13) or to a location contained in a map.
  8. Apparatus according to one of Claims 1 to 6, wherein a memory (4) having a digital map with local positions of objects (13) is provided.
  9. Apparatus according to one of Claims 1 to 8, wherein the apparatus is designed to assist a visually impaired human being.
  10. Method for outputting an audible and/or visual piece of information using a portable apparatus (1) having a pedometer system for ascertaining the movement of an operator, particularly for outputting a picture or a text, wherein the portable apparatus (1) monitors whether the portable apparatus (1) approaches an object (13), and wherein a stipulated approach to the object (13) prompts a signal to be output by the portable apparatus (1).
EP08874596A 2008-06-10 2008-12-02 Portable device with warning system and method Not-in-force EP2289056B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102008002322A DE102008002322A1 (en) 2008-06-10 2008-06-10 Portable device with warning system and method
PCT/EP2008/066608 WO2009149770A1 (en) 2008-06-10 2008-12-02 Portable device with warning system and method

Publications (2)

Publication Number Publication Date
EP2289056A1 EP2289056A1 (en) 2011-03-02
EP2289056B1 true EP2289056B1 (en) 2011-09-07

Family

ID=40317016

Family Applications (1)

Application Number Title Priority Date Filing Date
EP08874596A Not-in-force EP2289056B1 (en) 2008-06-10 2008-12-02 Portable device with warning system and method

Country Status (8)

Country Link
US (1) US20110143816A1 (en)
EP (1) EP2289056B1 (en)
JP (1) JP2011526432A (en)
KR (1) KR20110027684A (en)
CN (1) CN102057411A (en)
AT (1) ATE523866T1 (en)
DE (1) DE102008002322A1 (en)
WO (1) WO2009149770A1 (en)

Families Citing this family (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102010056538A1 (en) * 2010-12-29 2012-07-05 Jörg Dauner Active signals processing method for electric cars, involves sending wireless signal by vehicle, where wireless signal is processed as warning signal for endangered person, and processing vehicle-side radiated wireless signals
US8620239B2 (en) * 2011-01-14 2013-12-31 Qualcomm Incorporated Dynamic DC-offset determination for proximity sensing
JP5691568B2 (en) * 2011-01-28 2015-04-01 ソニー株式会社 Information processing apparatus, notification method, and program
JP2012155655A (en) * 2011-01-28 2012-08-16 Sony Corp Information processing device, notification method, and program
US8760275B2 (en) * 2011-04-15 2014-06-24 Avaya Inc. Obstacle warning system and method
WO2013067539A1 (en) 2011-11-04 2013-05-10 Massachusetts Eye & Ear Infirmary Adaptive visual assistive device
CN103247144B (en) * 2012-02-07 2015-07-22 宇龙计算机通信科技(深圳)有限公司 Traffic safety prompting method and mobile terminal
TW201413662A (en) * 2012-09-19 2014-04-01 Elitegroup Computer Sys Co Ltd Portable mobile device
JP2014194363A (en) 2013-03-28 2014-10-09 Fujitsu Ltd Guidance device, guidance method, and program
US20150193660A1 (en) * 2014-01-04 2015-07-09 Eric Schimpff Collision avoidance system for mobile devices
WO2015116126A1 (en) * 2014-01-31 2015-08-06 Hewlett-Packard Development Company, L.P. Notifying users of mobile devices
CN104933825B (en) 2014-03-18 2018-03-13 华为技术有限公司 The alarm method and device of a kind of mobile terminal
JP6337646B2 (en) * 2014-06-26 2018-06-06 株式会社Ｊｖｃケンウッド In-vehicle video system, video transfer system, video transfer method, and video transfer program
SG10201406053QA (en) * 2014-09-25 2016-04-28 Huawei Internat Pte Ltd Method and system for accident avoidance
US10036809B2 (en) 2014-09-26 2018-07-31 The Regents Of The University Of Michigan Real-time warning for distracted pedestrians with smartphones
CN104376683A (en) * 2014-12-02 2015-02-25 上海斐讯数据通信技术有限公司 Alarm method and alarm device on basis of position detection
CN105578820A (en) * 2015-07-25 2016-05-11 宇龙计算机通信科技(深圳)有限公司 Mobile terminal rear cover and mobile terminal
US10726281B2 (en) * 2015-07-29 2020-07-28 Invensense, Inc. Method and apparatus for user and moving vehicle detection
JP6262800B2 (en) * 2016-04-27 2018-01-17 京セラ株式会社 Portable electronic device, portable electronic device control method, and portable electronic device control program
CN107784830A (en) * 2016-08-30 2018-03-09 林皓 A kind of traffic monitoring system and its monitoring and managing method
CN106448147B (en) * 2016-11-29 2023-03-03 厦门精图信息技术有限公司 Pedestrian anti-collision intelligent early warning system based on mobile terminal
CN106686290B (en) * 2016-12-30 2018-12-04 维沃移动通信有限公司 A kind of alarm method and mobile terminal based on rotating camera
US10663298B2 (en) * 2017-06-25 2020-05-26 Invensense, Inc. Method and apparatus for characterizing platform motion
CN108765934B (en) * 2018-04-12 2020-08-18 杭州后博科技有限公司 System and method for reminding pedestrians to pass through aisle intersection
US10660560B2 (en) * 2018-08-27 2020-05-26 International Business Machiness Corporation Predictive fall prevention using corrective sensory stimulation
KR102645503B1 (en) * 2020-11-06 2024-03-11 카페24 주식회사 Artificial intelligence-based walking guidance device and method for departure prevention
DE102021201310A1 (en) 2021-02-11 2022-08-11 Robert Bosch Gesellschaft mit beschränkter Haftung Method for providing an alarm system function in a mobile device and mobile device, in particular a mobile phone or tablet computer

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3170151A (en) * 1962-04-20 1965-02-16 Roth Alexander Optical walking aid for the blind
US6502032B1 (en) * 2001-06-25 2002-12-31 The United States Of America As Represented By The Secretary Of The Air Force GPS urban navigation system for the blind
US20040183674A1 (en) * 2003-01-31 2004-09-23 Ruvarac Thomas C. Apparatus, system and method for monitoring a location of a portable device
JP2005004542A (en) * 2003-06-12 2005-01-06 Mitsubishi Electric Corp Mobile communication device, server device, walk guidance device, method of mobile communication, method of communication, method of guiding walk, program, and computer readable recording medium for recording this program
US7546204B2 (en) * 2004-05-12 2009-06-09 Takashi Yoshimine Information processor, portable apparatus and information processing method
JP4646697B2 (en) * 2005-05-18 2011-03-09 富士通株式会社 Portable device
KR100630156B1 (en) * 2005-09-16 2006-10-02 삼성전자주식회사 Appratus and method for detecting step in personal navigation terminal
CN2921858Y (en) * 2006-05-24 2007-07-11 乐金电子(中国)研究开发中心有限公司 Blind guiding mobile telephone
CN101132578A (en) * 2007-07-25 2008-02-27 庆邦电子（深圳）有限公司 GPS locating and navigation method and mobile phone with the same function

Also Published As

Publication number Publication date
ATE523866T1 (en) 2011-09-15
KR20110027684A (en) 2011-03-16
EP2289056A1 (en) 2011-03-02
WO2009149770A1 (en) 2009-12-17
CN102057411A (en) 2011-05-11
DE102008002322A1 (en) 2009-12-17
JP2011526432A (en) 2011-10-06
US20110143816A1 (en) 2011-06-16

Similar Documents

Publication Publication Date Title
EP2289056B1 (en) Portable device with warning system and method
US8605948B2 (en) Driver assistance system for monitoring driving safety and corresponding method for detecting and evaluating a vehicle movement
DE102007062680B4 (en) Traffic congestion meter, traffic congestion indicator and program
CN107533803B (en) Block control device
JP5499277B2 (en) Dangerous driving prevention awareness judgment system and dangerous driving prevention awareness judgment method
CN111152788B (en) Method and system for preventing mistaken stepping of accelerator pedal of vehicle
JP2006209455A (en) Apparatus, system and method for diagnosing vehicle drive
DE102010060225A1 (en) Vehicle information offering device
SE539157C2 (en) Identification of safety risks in a vehicle to notify fellow road users
JP2006209455A5 (en)
DE112009004342T5 (en) Mobile object position detection device
DE102018202394A1 (en) Method for operating a sound output device of a motor vehicle, control device, and motor vehicle
DE102017009594A1 (en) Method for detecting a water depth
JP2009301267A (en) Driving support device
van der Horst et al. The Peripheral Detection Task (PDT): On-line measurement of driver cognitive workload and selective attention
DE102005061701A1 (en) A method of remembering a route guidance for a destination route
JP2008090683A (en) Onboard navigation device
JP2010039686A (en) Voice output device
EP2022456A2 (en) Portable navigation device
DE102005012131B3 (en) Method of determining the upper speed limit for a vehicle over a given route where a maximum speed limit is given
WO2014198548A1 (en) Method and system for the detection of one or more persons by a vehicle
US20170011129A1 (en) In-Vehicle Device, Information System, and Output Control Method
WO2018024389A1 (en) Method, device and system for warning about a wrong-way-driving situation for a vehicle
KR20120007232A (en) System for providing customized driving attention information
DE102011120501B4 (en) Mobile communication device with a means for detecting and / or detecting movement data and associated method

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20110110

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA MK RS

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

DAX Request for extension of the european patent (deleted)
GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 502008004832

Country of ref document: DE

Effective date: 20111103

REG Reference to a national code

Ref country code: NL

Ref legal event code: VDEP

Effective date: 20110907

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110907

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110907

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110907

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111207

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110907

LTIE Lt: invalidation of european patent or patent extension

Effective date: 20110907

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110907

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110907

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110907

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111208

REG Reference to a national code

Ref country code: IE

Ref legal event code: FD4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110907

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110907

Ref country code: IE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110907

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120107

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110907

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110907

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110907

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110907

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120109

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110907

BERE Be: lapsed

Owner name: ROBERT BOSCH G.M.B.H.

Effective date: 20111231

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110907

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20111231

26N No opposition filed

Effective date: 20120611

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 502008004832

Country of ref document: DE

Effective date: 20120611

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20111231

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110907

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111218

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20111202

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111207

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20121202

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110907

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20121231

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110907

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20121231

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20121202

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 523866

Country of ref document: AT

Kind code of ref document: T

Effective date: 20131202

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20131202

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 8

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 9

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 10

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20181218

Year of fee payment: 11

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20190221

Year of fee payment: 11

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 502008004832

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200701

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191231