EP2851331B1 - Method for controlling a picking vehicle - Google Patents

Method for controlling a picking vehicle Download PDF

Info

Publication number
EP2851331B1
EP2851331B1 EP14183114.9A EP14183114A EP2851331B1 EP 2851331 B1 EP2851331 B1 EP 2851331B1 EP 14183114 A EP14183114 A EP 14183114A EP 2851331 B1 EP2851331 B1 EP 2851331B1
Authority
EP
European Patent Office
Prior art keywords
person
truck
optical sensor
industrial truck
monitoring
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP14183114.9A
Other languages
German (de)
French (fr)
Other versions
EP2851331A1 (en
Inventor
Tino Krüger-Basjmeleh
Volker Viereck
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
STILL GmbH
Original Assignee
STILL GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by STILL GmbH filed Critical STILL GmbH
Priority to PL14183114T priority Critical patent/PL2851331T3/en
Publication of EP2851331A1 publication Critical patent/EP2851331A1/en
Application granted granted Critical
Publication of EP2851331B1 publication Critical patent/EP2851331B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B66HOISTING; LIFTING; HAULING
    • B66FHOISTING, LIFTING, HAULING OR PUSHING, NOT OTHERWISE PROVIDED FOR, e.g. DEVICES WHICH APPLY A LIFTING OR PUSHING FORCE DIRECTLY TO THE SURFACE OF A LOAD
    • B66F9/00Devices for lifting or lowering bulky or heavy goods for loading or unloading purposes
    • B66F9/06Devices for lifting or lowering bulky or heavy goods for loading or unloading purposes movable, with their loads, on wheels or the like, e.g. fork-lift trucks
    • B66F9/075Constructional features or details
    • B66F9/07581Remote controls
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B66HOISTING; LIFTING; HAULING
    • B66FHOISTING, LIFTING, HAULING OR PUSHING, NOT OTHERWISE PROVIDED FOR, e.g. DEVICES WHICH APPLY A LIFTING OR PUSHING FORCE DIRECTLY TO THE SURFACE OF A LOAD
    • B66F9/00Devices for lifting or lowering bulky or heavy goods for loading or unloading purposes
    • B66F9/06Devices for lifting or lowering bulky or heavy goods for loading or unloading purposes movable, with their loads, on wheels or the like, e.g. fork-lift trucks
    • B66F9/063Automatically guided

Definitions

  • the invention relates to a method for controlling a picking truck.
  • the invention relates to a method for controlling an industrial truck, especially in picking, wherein the truck has an optical sensor with a monitoring area, which is connected to a control device, and the control device by evaluating the data of the optical sensor, the position of a person within of the monitoring area is determined and tracked.
  • the pedestrian pallet truck For picking, the compilation of goods deliveries in a warehouse, very often pedestrian pallet trucks are used as a form of material handling equipment.
  • the pedestrian pallet truck is moved with a charge carrier on load forks, such as a pallet or a wire mesh basket, through the rack aisles of a warehouse in which the goods to be picked or picked are stored.
  • load forks such as a pallet or a wire mesh basket
  • the order picker or the person removing the goods picks up the goods located on the shelf, places them on or in the load carrier of the drawbar lift truck and then has to drive to the next picking location within the goods warehouse or the rack aisle of the goods warehouse for the next goods.
  • the order picker moves to a goods delivery with the pedestrian pallet truck.
  • a solution with a wireless glove is known, which attracts the picking person and through which the control of the truck is made possible. It can be integrated as a command button in the glove individual functions such as "forward", "back”, etc., which can be pressed by the picking person or the order picker. After picking up a particular product at a pickup location, thus after picking, for example, the "before" command key on the glove can be pressed by the order picker and the truck continues to move in the aisle as long as the order picker presses the command key on the glove holds. Due to the integration of the radio command transmitter in the glove, it does not have to be worn as a separate object by the order picker and does not disturb the latter in its freedom of movement.
  • Another known method is the control by voice commands, in which, however, additional location information or information about the position is required to automatically lead the truck between the receiving locations of the goods of the picking process.
  • the truck is automatically guided to the next location via a goods management systems and a voice input from the order picker, when the picking is completed at a recording location.
  • the Goods removal manually.
  • a disadvantage of this prior art is that background noise can interfere. Also, the additional location information must be available.
  • partially automated systems based on commands via voice input have the disadvantage that they have to be laboriously installed. For example, installations in warehouses are required for navigation and communication, a goods management system must be connected and servers must be made available in order to achieve the functionality of such a system. However, this suffers from the flexibility of the logistics processes, as rapid changes are only possible to a limited extent. Also, the economic and time effort is often spared. In all the systems described above, there is also a fundamental need for the employees performing the order picking to be trained and instructed in the system.
  • US 2002/190849 A1 discloses a method according to the preamble of claim 1.
  • the present invention has for its object to provide a method for controlling an industrial truck, in particular an industrial truck for picking, available, which avoids the aforementioned disadvantages and with which the truck in a simple manner with a person together when loading goods can be moved.
  • the object is achieved in that in a method for controlling an industrial truck, in particular during order picking, wherein the truck has an optical sensor with a monitoring area which is connected to a control device, and the control device by evaluating the data of the optical sensor Position of a person within the surveillance area determines and tracks, within the surveillance area is defined at least one monitoring limit and is detected by the control device when the person's position has exceeded the monitoring limit, and the truck is driven by the control device via a traction drive of the truck until the position of the person is back on the original side of the monitoring limit ,
  • the monitoring boundary provides a kind of virtual or invisible boundary established within the area of the environment monitored by the optical sensor, the surveillance area. Whenever a person exceeds this limit, the truck automatically tracks. This can be compared in the figurative sense as two magnets that are held together with the same polarity and thus the one is always pushed on too close approach of the other.
  • the optical sensor can be any type of sensor from a laser scanner, which detects the environment alone in one plane, up to a camera. It can be a capture only one-dimensional act up to a three-dimensional detection of pixels, in which the distance to the optical sensor is detected.
  • the data points corresponding to a person within the surveillance area are detected, and as these points move beyond the monitoring boundary, the truck is tracked by the control device, which may be part of a vehicle control, for example, or a separate computer, specifically for this purpose.
  • the truck is moved and with this together the monitoring area including the monitoring limit until it is moved back over the position of the person and the person is back on the original side of the monitoring boundary.
  • the inventive method allows a particularly advantageous semi-automated picking.
  • An employee, the picker has no time to track or move the truck.
  • the truck so to speak, automatically follows its running movements while, for example, it goes along a series of containers with the goods to be picked or a rack aisle and removes these goods.
  • the employee can remove the goods completely freely and place them on, for example, a pallet or a container on the fork of the truck. Whenever all the goods have been picked in one area and the employee continues to move along the rack aisle, the truck follows him automatically.
  • Advantageous for this is no fixed installation in For example, a warehouse needed and no additional device that would have to be carried by the employee. In particular, however, no elaborate training or briefing is required because the operation by the picking employee is ultimately only to pick the goods as usual and continue each.
  • Industrial trucks with the method according to the invention can be used as a single or in a variety of vehicles in a warehouse without major preparations must be made.
  • the optical sensor can detect distances to objects in the surveillance area.
  • the optical sensor can detect the monitoring area in three dimensions.
  • the optical sensor detects not only distances in a plane but also with a certain height extent and thus the objects three-dimensionally, for example a time of flight camera compared to a simple laser scanner, the detection and the tracking or "tracking" of a person can continue be improved.
  • changes in the position of the person are tracked and the changes in the values are only judged as correct with continuous changes.
  • the data of the position of the person are differentiated from data of further moving objects or persons in the case of an at least partial optical overlap by recognizing the image data with constant change as the position of the tracking person.
  • a second person runs in the optical axis between the optical sensor and the tracked person and partially covers them, it can thus be recognized by the control device "which" person is the right one to carry on.
  • the pixels of the second person have opposite the pixels of the first person an instant jump in the distance.
  • forecast values are determined for the future position of the person within which the position should be located.
  • a prognosis value for temporally subsequent positions of the person to be tracked can be created.
  • the person should therefore be in a certain position or within a certain range of positions after the previous movements up to the time of determining this forecast value for the future.
  • the originally tracked person is detected by means of a search within the prognosis values.
  • a search using the forecast values makes it possible to track the person again capture. This applies at least, as long as the coverage does not last too long.
  • the position of a person to be tracked in a learning phase is detected by a movement of the person.
  • the person to be tracked can be identified by the control device.
  • a short phase of the detection of a targeted movement of the serving or commissioning person which can be triggered for example by a special circuit or command input in the truck.
  • the optical sensor can be a camera and the position of a person to be tracked can be detected by means of image processing methods.
  • the monitoring boundary on one side or on both sides of the industrial truck separates a front area of the monitoring area from a rear area with respect to a movement direction of the industrial truck.
  • an employee is usually taken from a shelf or containers in an area on the side of the truck and the goods are loaded onto the truck.
  • the truck is a pallet truck with a corresponding container or a pallet on the forks, which is moved forward with rearward oriented fork with a drawbar.
  • Such a truck is then moved by the inventive method in an optimal way with the picking staff when the monitoring limit is positioned substantially transverse to the direction of travel in the front region of the vehicle.
  • the monitoring limit starting from the truck perpendicular to the direction of movement of the truck to a maximum of 45 ° to Movement direction arranged.
  • the optical sensor can advantageously be arranged and yet the monitoring limit be located at the front of the industrial truck.
  • the truck can be moved with the steering set in the straight-ahead position.
  • both the data of the optical sensor can be used, and also or alternatively data of a personal protection device, as far as such is provided and installed in the truck.
  • personal protection devices are present in autonomous vehicles and monitor a nearby area for collisions with obstacles or people. These are usually based on monitoring at a low altitude above the ground, for example by laser scanners, which scan surfaces in accordance with standards at a height of approx. 150 mm above the ground.
  • edge and the orientation of one or both sides of a rack aisle are detected and at a distance from the edge or both edges a target track is set.
  • the shelf columns are detected as components of the shelves of the rack aisle. From the positions of the components of the shelf can be determined at a distance from a target track on which the truck should move.
  • This setpoint track can also be placed in the middle of a rack aisle in double-sided shelves or, depending on the wishes of the serving person On one side of the rack aisle, so that the most ergonomic work possible. It is also possible that such a specification of a target track, an alignment of the truck takes place and it is not necessary at the beginning of order picking in a rack aisle, that the truck is aligned exactly by hand. Rather, the truck must only be driven into the rack aisle and aligns with the first movement movements to this rack aisle.
  • the end of a rack aisle is detected and terminated on reaching the end of the rack aisle the process.
  • the truck can then stop automatically or follow the succession to the person.
  • the data obtained by the optical sensor in particular when it comes to three-dimensional data, in addition to the tracking of the person and the determination of a desired track also be used in general for the vehicle guidance, for example in the sense of an autonomously driving vehicle.
  • the truck can be designed so that after completion of a picking on request or by entering a command the truck automatically moves to a goods delivery and drops the container or pallet. This leads to a further time saving, since the commissioning person does not have to control the truck there. This is optimally supplemented if it is provided that the truck also drives autonomously on request to a first working position of a picking person.
  • the object is also achieved by an industrial truck with an optical sensor having a monitoring area, which is connected to a control device, which performs a method described above.
  • the truck has the advantages already described.
  • the Fig. 1 1 shows schematically in plan view an industrial truck 1, in which the method according to the invention is used, with a picking person 2.
  • An observation area 7 is detected within the rack aisle 3 by an optical sensor 5, which in the present example is designed as a camera 6 with an all-round pick-up area and the ability to detect clearances.
  • the data of the optical sensor 5 are processed and the position of the person to be tracked 2 detected.
  • a monitoring limit 8 is stored in the control device, which includes a front area 9 of the monitoring area 7 from a rear area 10 of the monitoring area 7 separates. While the truck is 1, the picking person 2 removes goods from the load rack 4 and places them on a transport pallet 11.
  • the Fig. 2 schematically shows in plan the truck 1 with the transport pallet 11 and the person 2 of Fig. 1 in a driving movement of the truck 1 in the direction of movement shown by the arrow.
  • the position of the person 2 is now in the front area 9 of the monitoring area 7. Therefore, the control device controls a traction drive of the truck 1 so long and moves the truck 1 in the direction of movement until the person 2 again behind the monitoring limit 8 in the Rear area 10 of the monitoring area 7 is located.
  • the edge 12 of the load rack 4 is determined from the data of the optical sensor 5 and set at a distance from this edge 12 a target track on which the truck 1 is moved by the control device.
  • the Fig. 3 shows the truck 1 of Figure. 1 in a side view with the load shelf 4 of the rack aisle 3 and additional goods containers 13 from which goods can also be removed during order picking.
  • the truck 1 with the optical sensor 5, which is designed as a camera 6, has the monitoring area 7.
  • the camera 6 can record, for example as a time of flight camera (ToF) via runtime measurements, the distance of the pixels to the camera 6.
  • the optical sensor 5 can also be designed as a laser scanner 14, which scans in a plane 15 and thereby detects the distances to target objects. This corresponds to a two-dimensional detection.

Landscapes

  • Engineering & Computer Science (AREA)
  • Transportation (AREA)
  • Structural Engineering (AREA)
  • Civil Engineering (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Geology (AREA)
  • Mechanical Engineering (AREA)
  • Warehouses Or Storage Devices (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)

Description

Die Erfindung betrifft ein Verfahren zur Steuerung eines Kommissionierflurförderzeugs. Insbesondere betrifft die Erfindung ein Verfahren zur Steuerung eines Flurförderzeugs, vor allem bei der Kommissionierung, wobei das Flurförderzeug einen optischen Sensor mit einem Überwachungsbereich aufweist, der mit einer Steuerungsvorrichtung verbunden ist, und die Steuerungsvorrichtung durch Auswertung der Daten des optischen Sensors die Position einer Person innerhalb des Überwachungsbereichs bestimmt und nachführt.The invention relates to a method for controlling a picking truck. In particular, the invention relates to a method for controlling an industrial truck, especially in picking, wherein the truck has an optical sensor with a monitoring area, which is connected to a control device, and the control device by evaluating the data of the optical sensor, the position of a person within of the monitoring area is determined and tracked.

Zum Kommissionieren, dem Zusammenstellen von Warenlieferungen in einem Warenlager, werden sehr häufig Deichselhubwagen als eine Form von Flurförderzeugen eingesetzt. Dabei wird der Deichselhubwagen mit einem auf Lastgabeln befindlichen Ladungsträger, etwa einer Palette oder einem Drahtgitterkorb, durch die Regalgassen eines Warenlagers bewegt, in denen die zu kommissionierenden bzw. aufzunehmenden Waren gelagert sind. Der Kommissionierer bzw. die ausführende Person entnimmt dabei je nach Auftrag die im Regal befindlichen Waren, legt diese auf bzw. in den Ladungsträger des Deichselhubwagens und muss sodann zum nächsten Aufnahmeort innerhalb des Warenlagers bzw. der Regalgasse des Warenlagers für die nächste Ware fahren. Wenn der jeweilige Auftrag zur Kommissionierung vollständig abgearbeitet ist, fährt der Kommissionierer mit dem Deichselhubwagen zu einer Warenabgabe. Die Fahrstrecken von einem Aufnahmeort zum nächsten sind dabei in der großen Mehrzahl kurze bis sehr kurze gerade Strecken innerhalb einer Regalgasse bzw. an einem Regal entlang in dem Warenlager. In der Summe entsteht jedoch durch die Vielzahl der erforderlichen Fahrbewegungen des Flurförderzeugs, die von dem Kommissionierer vorgenommen werden müssen, ein erheblicher Zeitbedarf, da der Kommissionierer zu einer Steuerdeichsel des Gabelhubwagens gehen muss, um die entsprechenden Fahrschalter zu betätigen und das Fahrzeug bei der Fahrbewegung zu steuern, und im Anschluss wieder zurück an die Position zum Aufnehmen der Waren.For picking, the compilation of goods deliveries in a warehouse, very often pedestrian pallet trucks are used as a form of material handling equipment. In this case, the pedestrian pallet truck is moved with a charge carrier on load forks, such as a pallet or a wire mesh basket, through the rack aisles of a warehouse in which the goods to be picked or picked are stored. Depending on the order, the order picker or the person removing the goods picks up the goods located on the shelf, places them on or in the load carrier of the drawbar lift truck and then has to drive to the next picking location within the goods warehouse or the rack aisle of the goods warehouse for the next goods. When the respective picking order has been completely processed, the order picker moves to a goods delivery with the pedestrian pallet truck. The routes from one location to the next are in the great majority short to very short straight distances within a rack aisle or along a shelf in the warehouse. In sum, however, due to the large number of required travel movements of the truck, which must be made by the order picker, a considerable amount of time, since the order picker has to go to a control handlebar of the pallet truck to operate the corresponding driving switch and the vehicle during the driving movement control, and then back to the position for picking up the goods.

Es sind daher Verfahren und Flurförderzeuge für eine teilautomatisierte Kommissionierung bekannt. Diese teilautomatisierte Kommissionierung erfolgt im Grundsatz ähnlich, nur muss der Kommissionierer das Flurförderzeug zumindest auf den geraden Fahrstrecken entlang eines Regals nicht mehr selbst führen, sondern das Flurförderzeug verfährt automatisch. Die Steuerung des Flurförderzeugs bei der Fahrt zur nächsten Aufnahmestation kann dabei beispielsweise durch eine Funkfernsteuerung, so etwa einen Funkhandschuh mit einer Funkfernsteuerungsfunktion, oder durch eine Steuerung des Flurförderzeugs über Sprachbefehle erfolgen.There are therefore known methods and trucks for semi-automated order picking. This semi-automated picking takes place in the Principle similar, only the order picker, the truck must at least on the straight routes along a shelf no longer lead itself, but the truck moves automatically. The control of the truck while driving to the next receiving station can be done for example by a radio remote control, such as a wireless glove with a radio remote control function, or by controlling the truck via voice commands.

Beispielsweise ist eine Lösung mit einem Funkhandschuh bekannt, den die kommissionierende Person anzieht und durch den die Steuerung des Flurförderzeugs ermöglicht wird. Es können dabei Einzelfunktionen wie "vor", "zurück" usw. als Befehlstaster in den Handschuh integriert sein, die von der kommissionierenden Person, bzw. dem Kommissionierer, gedrückt werden können. Nach dem Aufnehmen einer bestimmten Ware an einem Aufnahmeort, somit nach erfolgter Kommissionierung, kann beispielsweise die "vor"-Befehlstaste an dem Handschuh von dem Kommissionierer gedrückt werden und das Flurförderzeug bewegt sich in dem Regalgang weiter, solange der Kommissionierer die Befehlstaste an dem Handschuh gedrückt hält. Durch die Integration des Funkbefehlsgebers in den Handschuh muss dieser nicht als eigener Gegenstand von dem Kommissionierer getragen werden und stört diesen nicht in seiner Bewegungsfreiheit.For example, a solution with a wireless glove is known, which attracts the picking person and through which the control of the truck is made possible. It can be integrated as a command button in the glove individual functions such as "forward", "back", etc., which can be pressed by the picking person or the order picker. After picking up a particular product at a pickup location, thus after picking, for example, the "before" command key on the glove can be pressed by the order picker and the truck continues to move in the aisle as long as the order picker presses the command key on the glove holds. Due to the integration of the radio command transmitter in the glove, it does not have to be worn as a separate object by the order picker and does not disturb the latter in its freedom of movement.

Nachteilig an diesem Stand der Technik ist jedoch, dass die Nutzung eines solchen Handschuhs durch eine Mehrzahl von Personen zu Akzeptanzproblemen führen kann. Auch muss jedes Mal eine Bedienung mit der anderen Hand erfolgen. Die Funkhandschuhe haben den weiteren Nachteil, dass diese durch die tägliche Handhabung verschleißen und somit regelmäßig ersetzt werden müssen. Dadurch werden zusätzliche, laufende Kosten verursacht.A disadvantage of this prior art, however, is that the use of such a glove by a plurality of persons can lead to acceptance problems. Also every time an operation with the other hand must be done. The wireless gloves have the further disadvantage that they wear out through daily handling and thus need to be replaced regularly. This causes additional, ongoing costs.

Ein weiteres bekanntes Verfahren ist die Steuerung durch Sprachbefehle, bei der jedoch zusätzlich eine Ortsinformation bzw. Information über die Position benötigt wird, um das Flurförderzeug zwischen den Aufnahmeorten der Waren des Kommissioniervorgangs automatisiert zu führen. Dabei wird über ein Warenmanagementsystemen und eine Spracheingabe des Kommissionierers das Flurförderzeug automatisiert zum nächsten Aufnahmeort geführt, wenn die Kommissionierung an einem Aufnahmeort beendet ist. Auch dabei erfolgt die Warenentnahme manuell.Another known method is the control by voice commands, in which, however, additional location information or information about the position is required to automatically lead the truck between the receiving locations of the goods of the picking process. In this case, the truck is automatically guided to the next location via a goods management systems and a voice input from the order picker, when the picking is completed at a recording location. Also here is the Goods removal manually.

Nachteilig an diesem Stand der Technik ist, dass Hintergrundgeräusche stören können. Auch muss die zusätzliche Ortsinformation zur Verfügung stehen. Ganz allgemein haben teilautomatisierten Systeme auf Basis von Befehlen über Spracheingabe den Nachteil, dass diese aufwändig installiert werden müssen. Beispielsweise sind Einbauten in Lagerhallen zur Navigation und Kommunikation erforderlich, muss ein Warenmanagementsystem angebunden und müssen Server zur Verfügung gestellt werden, um die Funktionsfähigkeit eines solchen Systems zu erreichen. Hierunter leidet jedoch die Flexibilität der Logistikprozesse, da schnelle Änderungen nur bedingt möglich sind. Auch wird der wirtschaftliche und zeitliche Aufwand oftmals gescheut. Bei allen zuvor beschriebenen Systemen besteht auch grundsätzlich die Notwendigkeit, dass die die Kommissionierung durchführenden Mitarbeiter eingearbeitet und in das System eingewiesen werden müssen.A disadvantage of this prior art is that background noise can interfere. Also, the additional location information must be available. In general, partially automated systems based on commands via voice input have the disadvantage that they have to be laboriously installed. For example, installations in warehouses are required for navigation and communication, a goods management system must be connected and servers must be made available in order to achieve the functionality of such a system. However, this suffers from the flexibility of the logistics processes, as rapid changes are only possible to a limited extent. Also, the economic and time effort is often spared. In all the systems described above, there is also a fundamental need for the employees performing the order picking to be trained and instructed in the system.

Aus der EP 2 533 119 A1 ist eine Vorrichtung zur Funkfernsteuerung eines Flurförderzeugs mit einem Funkhandschuh bekannt, das zur Kommissionierung eingesetzt werden kann.From the EP 2 533 119 A1 a device for radio remote control of a truck with a wireless glove is known, which can be used for picking.

US 2002/190849 A1 offenbart ein Verfahren gemäß dem Oberbegriff des Anspruchs 1. US 2002/190849 A1 discloses a method according to the preamble of claim 1.

Der vorliegenden Erfindung liegt die Aufgabe zugrunde, ein Verfahren zur Steuerung eines Flurförderzeugs, insbesondere eines Flurförderzeugs zur Kommissionierung, zur Verfügung zu stellen, das die zuvor genannten Nachteile vermeidet und mit dem das Flurförderzeug auf einfache Art und Weise mit einer Person zusammen beim Aufladen von Waren mitbewegt werden kann.The present invention has for its object to provide a method for controlling an industrial truck, in particular an industrial truck for picking, available, which avoids the aforementioned disadvantages and with which the truck in a simple manner with a person together when loading goods can be moved.

Diese Aufgabe wird durch ein Verfahren mit den Merkmalen des unabhängigen Patentanspruchs 1 sowie durch einen Flurförderzeug mit den Merkmalen 17 gelöst. Vorteilhaft der Weiterbildungen der Erfindung sind in den Unteransprüchen angegeben.This object is achieved by a method having the features of independent claim 1 and by a truck with the features 17. Advantageously, the developments of the invention are specified in the subclaims.

Die Aufgabe wird erfindungsgemäß dadurch gelöst, dass bei einem Verfahren zur Steuerung eines Flurförderzeugs, insbesondere bei der Kommissionierung, wobei das Flurförderzeug einen optischen Sensor mit einem Überwachungsbereich aufweist, der mit einer Steuerungsvorrichtung verbunden ist, und die Steuerungsvorrichtung durch Auswertung der Daten des optischen Sensors die Position einer Person innerhalb des Überwachungsbereichs bestimmt und nachführt, innerhalb des Überwachungsbereichs mindestens eine Überwachungsgrenze definiert ist und von der Steuerungsvorrichtung erfasst wird, wenn die Position der Person die Überwachungsgrenze überschritten hat, sowie von der Steuerungsvorrichtung über einen Fahrantrieb des Flurförderzeugs das Flurförderzeug solange bewegt wird, bis die Position der Person wieder auf der ursprünglichen Seite der Überwachungsgrenze ist.The object is achieved in that in a method for controlling an industrial truck, in particular during order picking, wherein the truck has an optical sensor with a monitoring area which is connected to a control device, and the control device by evaluating the data of the optical sensor Position of a person within the surveillance area determines and tracks, within the surveillance area is defined at least one monitoring limit and is detected by the control device when the person's position has exceeded the monitoring limit, and the truck is driven by the control device via a traction drive of the truck until the position of the person is back on the original side of the monitoring limit ,

Die Überwachungsgrenze stellt eine Art virtuelle bzw. nicht sichtbare Grenze da, die innerhalb des von dem optischen Sensor überwachten Bereichs der Umgebung, des Überwachungsbereichs, eingerichtet ist. Immer wenn eine Person diese Grenze überschreitet, wird das Flurförderzeug automatisch nachgeführt. Dies kann im übertragenen Sinne verglichen werden wie zwei Magneten, die polungsgleich aneinander gehalten werden und dadurch der eine immer bei zu weiter Annäherung des anderen weitergeschoben wird. Dabei kann der optische Sensor jegliche Art von Sensor sein von einem Laserscanner, der allein in einer Ebene die Umgebung erfasst, bis hin zu einer Kamera. Es kann sich um eine Erfassung nur eindimensional handeln bis hin zu einer dreidimensionalen Erfassung von Bildpunkten, bei denen auch der Abstand zu dem optischen Sensor erfasst wird. Bei dem Verfahren werden die Datenpunkte, die einer Person innerhalb des Überwachungsbereiches entsprechen, erfasst und bei einer Bewegung dieser Punkte über die Überwachungsgrenze hinweg wird das Flurförderzeug von der Steuerungsvorrichtung nachgeführt, die zum Beispiel Teil einer Fahrzeugsteuerung sein kann, aber auch ein getrennter Rechner, eigens für diesen Zweck. Dabei wird das Flurförderzeug bewegt und mit diesem zusammen der Überwachungsbereich einschließlich der Überwachungsgrenze, bis diese wieder über die Position der Person hinweg bewegt wird und die Person sich wieder auf der ursprünglichen Seite der Überwachungsgrenze befindet. Das erfindungsgemäße Verfahren ermöglicht eine besonders vorteilhafte teilautomatisierte Kommissionierung. Ein Mitarbeiter, der Kommissionierer, hat keinerlei zeitlichen Aufwand für das Nachführen oder Bewegen des Flurförderzeugs. Das Flurförderzeug folgt sozusagen automatisch seinen Laufbewegungen, während er beispielsweise an einer Reihe von Behältern mit den zu kommissionierenden Waren oder einer Regalgasse entlang geht und diese Waren entnimmt. Der Mitarbeiter kann völlig frei die Waren entnehmen und auf beispielsweise eine Palette oder in einen Behälter auf Lastgabel des Flurförderzeugs legen. Immer, wenn alle Waren in einem Bereich kommissioniert sind und der Mitarbeiter sich weiter entlang der Regalgasse bewegt, so folgt ihm das Flurförderzeug automatisch. Vorteilhaft ist hierfür keinerlei feste Installation in beispielsweise einem Warenlager nötig und keinerlei zusätzliche Vorrichtung, die von dem Mitarbeiter mitgeführt werden müsste. Es ist insbesondere jedoch auch keinerlei aufwändige Einarbeitung oder Einweisung erforderlich, da die Bedienung durch den kommissionierenden Mitarbeiter letztlich darin besteht, nur wie üblich die Waren zu kommissionieren und jeweils weiterzugehen. Flurförderzeuge mit dem erfindungsgemäßen Verfahren können als einzelne oder in einer Vielzahl von Fahrzeugen in einem Warenlager eingesetzt werden, ohne dass größere Vorbereitungen getroffen werden müssen.The monitoring boundary provides a kind of virtual or invisible boundary established within the area of the environment monitored by the optical sensor, the surveillance area. Whenever a person exceeds this limit, the truck automatically tracks. This can be compared in the figurative sense as two magnets that are held together with the same polarity and thus the one is always pushed on too close approach of the other. In this case, the optical sensor can be any type of sensor from a laser scanner, which detects the environment alone in one plane, up to a camera. It can be a capture only one-dimensional act up to a three-dimensional detection of pixels, in which the distance to the optical sensor is detected. In the method, the data points corresponding to a person within the surveillance area are detected, and as these points move beyond the monitoring boundary, the truck is tracked by the control device, which may be part of a vehicle control, for example, or a separate computer, specifically for this purpose. In this case, the truck is moved and with this together the monitoring area including the monitoring limit until it is moved back over the position of the person and the person is back on the original side of the monitoring boundary. The inventive method allows a particularly advantageous semi-automated picking. An employee, the picker, has no time to track or move the truck. The truck, so to speak, automatically follows its running movements while, for example, it goes along a series of containers with the goods to be picked or a rack aisle and removes these goods. The employee can remove the goods completely freely and place them on, for example, a pallet or a container on the fork of the truck. Whenever all the goods have been picked in one area and the employee continues to move along the rack aisle, the truck follows him automatically. Advantageous for this is no fixed installation in For example, a warehouse needed and no additional device that would have to be carried by the employee. In particular, however, no elaborate training or briefing is required because the operation by the picking employee is ultimately only to pick the goods as usual and continue each. Industrial trucks with the method according to the invention can be used as a single or in a variety of vehicles in a warehouse without major preparations must be made.

Vorteilhaft kann der optische Sensor Abstände zu Objekten in dem Überwachungsbereich erfassen.Advantageously, the optical sensor can detect distances to objects in the surveillance area.

Durch eine Erfassung der Abstände zu den Objekten in dem Überwachungsbereich wird das Nachverfolgen einer Person und insbesondere das Unterscheiden von sich bewegenden Gegenständen wie etwa Personen gegenüber feststehenden Gegenständen stark vereinfacht. Es kann somit nicht nur die Bewegung in einer Bildebene sondern auch in zwei Dimensionen einschließlich einer Bewegung aus dieser Bildebene heraus beurteilt werden. Dabei wird eine Beurteilung der Bewegung hinsichtlich beispielsweise einer kontinuierlichen Bewegung und der Bewegungsgeschwindigkeit möglich, die etwa bei einer sich zu Fuß bewegenden Person sich innerhalb gewisser Rahmenwerte halten muss.By detecting the distances to the objects in the surveillance area, tracing a person, and in particular distinguishing moving objects such as persons from fixed objects, is greatly simplified. Thus, not only the movement in one image plane but also in two dimensions including a movement out of this image plane can be judged. In this case, an assessment of the movement with respect to, for example, a continuous movement and the movement speed is possible, which, for example, must be kept within certain frame values in the case of a person walking on foot.

In einer weiteren Ausgestaltung des Verfahrens kann der optische Sensor den Überwachungsbereich dreidimensional erfassen.In a further embodiment of the method, the optical sensor can detect the monitoring area in three dimensions.

Wenn der optische Sensor nicht nur Abstände in einer Ebene sondern auch mit einer gewissen Höhenausdehnung und somit die Gegenstände dreidimensional erfasst, etwa beispielsweise eine Time of Flight Kamera gegenüber einem einfachen Laserscanner, kann dadurch die Erfassung und das Nachverfolgen bzw. "Tracking" einer Person weiter verbessert werden.If the optical sensor detects not only distances in a plane but also with a certain height extent and thus the objects three-dimensionally, for example a time of flight camera compared to a simple laser scanner, the detection and the tracking or "tracking" of a person can continue be improved.

In günstiger Ausführungsform werden Änderungen der Position der Person verfolgt und die Änderungen der Werte nur bei stetigen Änderungen als richtig bewertet.In a favorable embodiment, changes in the position of the person are tracked and the changes in the values are only judged as correct with continuous changes.

Dadurch können Fehler vermieden werden. Das sich eine Person kontinuierlich in dem umgebenden Raum bewegen muss, lassen plötzliche, zum Beispiel im mathematischen Sinne unstetige Sprünge in der Position auf einen Fehler schließen. Es ist denkbar, in einem solchen Fall aus Sicherheitsgründen das Verfahren zu beenden oder das Flurförderzeug anzuhaltenAs a result, errors can be avoided. The fact that a person has to move continuously in the surrounding space makes sudden jumps in position, for example in the mathematical sense, inaccurate. It is conceivable in such a case for safety reasons to end the process or to stop the truck

In einer weiteren Ausgestaltung des Verfahrens werden die Daten der Position der Person von Daten weiterer sich bewegende Objekte bzw. Personen im Falle einer zumindest teilweisen optischen Überdeckung unterschieden, indem die Bilddaten mit stetiger Änderung als Position der nachgeführten Person erkannt werden.In a further refinement of the method, the data of the position of the person are differentiated from data of further moving objects or persons in the case of an at least partial optical overlap by recognizing the image data with constant change as the position of the tracking person.

Wenn eine zweite Person in die optische Achse zwischen dem optischen Sensor und der nachgeführten Person läuft und diese teilweise überdeckt, kann somit durch die Steuerungsvorrichtung erkannt werden, "welche" Person die richtige und weiter nachzuführen ist. Die Bildpunkte der zweiten Person weisen gegenüber den Bildpunkten der ersten Person einen unvermittelten Sprung im Abstand auf.If a second person runs in the optical axis between the optical sensor and the tracked person and partially covers them, it can thus be recognized by the control device "which" person is the right one to carry on. The pixels of the second person have opposite the pixels of the first person an instant jump in the distance.

Vorteilhaft werden für die zukünftige Position der Person Prognosewerte bestimmt, innerhalb denen sich die Position befinden sollte.Advantageously, forecast values are determined for the future position of the person within which the position should be located.

Aus dem Verlauf der bisherigen Bewegung einer Person bzw. der Bildpunkte der Darstellung dieser Person sowie den für die Bewegung einer Person im Raum möglichen Werten kann ein Prognosewerte für zeitlich nachfolgende Positionen der nachzuführenden Person erstellt werden. Die Person sollte sich also nach den bisherigen Bewegungen bis zum Zeitpunkt der Bestimmung dieses Prognosewertes für die Zukunft an einer bestimmten Position bzw. innerhalb eines bestimmten Bereiches von Positionen befinden.From the course of the previous movement of a person or the pixels of the representation of this person as well as the values that are possible for the movement of a person in space, a prognosis value for temporally subsequent positions of the person to be tracked can be created. The person should therefore be in a certain position or within a certain range of positions after the previous movements up to the time of determining this forecast value for the future.

In einer Weiterbildung des Verfahrens wird bei einer optischen Überdeckung der nachgeführten Person durch weitere Objekte bzw. Personen die ursprünglich nachgeführte Person anhand einer Suche innerhalb der Prognosewerte erfasst.In a further development of the method, if the tracking person is optically covered by further objects or persons, the originally tracked person is detected by means of a search within the prognosis values.

Wenn beispielsweise eine zweite Person so in der optischen Achse vor die nachgeführte Person tritt, dass diese zumindest kurzzeitig vollständig überdeckt ist, so ermöglicht eine Suche mithilfe der Prognosewerte es, die nachzuführen Person wieder zu erfassen. Dies gilt zumindest, soweit die Überdeckung nicht zu lange andauert.For example, if a second person occurs in the optical axis in front of the tracked person, that it is completely covered at least for a short time, then a search using the forecast values makes it possible to track the person again capture. This applies at least, as long as the coverage does not last too long.

Vorteilhaft wird die Position einer nachzuführenden Person in einer Lernphase durch eine Bewegung der Person erkannt.Advantageously, the position of a person to be tracked in a learning phase is detected by a movement of the person.

Dadurch kann zu Beginn des Arbeitens mit dem Flurförderzeug auf einfache Art und Weise die nachzuführende Person durch die Steuerungsvorrichtung identifiziert werden. Nach dem Start erfolgt eine kurze Phase der Erfassung einer gezielten Bewegung der bedienenden bzw. kommissionierenden Person, die beispielsweise durch eine spezielle Schaltung oder Befehlseingabe bei dem Flurförderzeug ausgelöst werden kann.As a result, at the beginning of working with the truck in a simple manner, the person to be tracked can be identified by the control device. After the start, a short phase of the detection of a targeted movement of the serving or commissioning person, which can be triggered for example by a special circuit or command input in the truck.

Der optische Sensor kann eine Kamera sein und die Position einer nachzuführenden Person mittels Methoden der Bildverarbeitung erkannt werden.The optical sensor can be a camera and the position of a person to be tracked can be detected by means of image processing methods.

Für die Erkennung von Personen im dreidimensionalen-Bereich bestehen zahlreiche, am Markt zur Verfügung stehende Lösungen, die hier eingesetzt werden können.There are many solutions available on the market for detecting people in the three-dimensional area that can be used here.

In einer günstigen Ausgestaltung des Verfahrens trennt die Überwachungsgrenze auf einer Seite oder beidseitig des Flurförderzeugs einen in Bezug auf eine Bewegungsrichtung des Flurförderzeugs vorderen Bereich des Überwachungsbereichs von einem hinteren Bereich.In a favorable embodiment of the method, the monitoring boundary on one side or on both sides of the industrial truck separates a front area of the monitoring area from a rear area with respect to a movement direction of the industrial truck.

Beim Kommissionieren wird im Regelfall ein Mitarbeiter in einem Bereich seitlich des Flurförderzeugs aus einem Regal oder Behältern die Waren entnehmen und diese auf das Flurförderzeug verladen. Häufig handelt es sich bei dem Flurförderzeug dabei um einen Gabelhubwagen mit einem entsprechenden Behälter oder einer Palette auf den Lastgabeln, der mit nach hinten orientierter Lastgabel mit einer Deichsel voran bewegt wird. Ein solches Flurförderzeug wird dann durch das erfindungsgemäße Verfahren auf optimale Weise mit dem kommissionierenden Mitarbeiter mitbewegt wenn die Überwachungsgrenze im Wesentlichen quer zur Fahrtrichtung im vorderen Bereich des Fahrzeugs positioniert ist.During picking, an employee is usually taken from a shelf or containers in an area on the side of the truck and the goods are loaded onto the truck. Often, the truck is a pallet truck with a corresponding container or a pallet on the forks, which is moved forward with rearward oriented fork with a drawbar. Such a truck is then moved by the inventive method in an optimal way with the picking staff when the monitoring limit is positioned substantially transverse to the direction of travel in the front region of the vehicle.

Vorteilhaft ist die Überwachungsgrenze ausgehend von dem Flurförderzeug senkrecht zur Bewegungsrichtung des Flurförderzeugs bis maximal unter 45° zur Bewegungsrichtung angeordnet.Advantageously, the monitoring limit, starting from the truck perpendicular to the direction of movement of the truck to a maximum of 45 ° to Movement direction arranged.

Indem die Überwachungsgrenze von dem Flurförderzeug aus betrachtet seitlich schräg nach vorne in Bezug auf eine Bewegungsrichtung orientiert ist, kann der optische Sensor vorteilhaft angeordnet werden und dennoch die Überwachungsgrenze an der Vorderseite des Flurförderzeugs sich befinden.Since the monitoring boundary, viewed from the industrial truck, is oriented laterally obliquely forward with respect to a direction of movement, the optical sensor can advantageously be arranged and yet the monitoring limit be located at the front of the industrial truck.

Das Flurförderzeug kann mit in Geradeausstellung festgelegter Lenkung bewegt werden.The truck can be moved with the steering set in the straight-ahead position.

In vielen Warenlagern mit industriellen Hallenböden ist eine solche Spursteuerung bei Regalgassen oftmals ausreichend und keine geregelte Steuerung erforderlich.In many warehouses with industrial hall floors such track control is often sufficient for rack aisles and no controlled control is required.

Es kann mithilfe der Daten des optischen Sensors und/oder den Daten einer Personenschutzvorrichtung eine Spurerkennung durchgeführt werden.It can be carried out using the data of the optical sensor and / or the data of a personal protection device, a lane detection.

Dies ermöglicht eine gezielte Steuerung der Lenkung des Flurförderzeugs. Dabei können sowohl die Daten des optischen Sensors herangezogen werden, als auch daneben oder alternativ Daten einer Personenschutzvorrichtung, soweit eine solche bei dem Flurförderzeug vorgesehen und eingebaut ist. Solche Personenschutzvorrichtungen sind bei autonom fahrenden Fahrzeugen vorhanden und überwachen einen Nahbereich auf Kollisionen mit Hindernissen oder Personen. Diese basieren zumeist auf einer Überwachung in einer geringen Höhe über dem Boden, beispielsweise durch Laserscanner, die entsprechend von Normen in ca. 150 mm Höhe über dem Boden flächig scannen.This allows targeted control of the steering of the truck. In this case, both the data of the optical sensor can be used, and also or alternatively data of a personal protection device, as far as such is provided and installed in the truck. Such personal protection devices are present in autonomous vehicles and monitor a nearby area for collisions with obstacles or people. These are usually based on monitoring at a low altitude above the ground, for example by laser scanners, which scan surfaces in accordance with standards at a height of approx. 150 mm above the ground.

In einer vorteilhaften Weiterbildung werden der Rand und die Orientierung einer oder beider Seiten einer Regalgasse erfasst und in einem Abstand von dem Rand oder beiden Rändern wird eine Sollspur festgelegt.In an advantageous development of the edge and the orientation of one or both sides of a rack aisle are detected and at a distance from the edge or both edges a target track is set.

Es werden beispielsweise die Regalsäulen als Komponenten der Regale der Regalgasse erfasst. Aus den Positionen der Komponenten des Regals kann in einem Abstand hiervon eine Sollspur bestimmt werden, auf der das Flurförderzeug sich bewegen soll. Diese Sollspur kann auch bei beidseitigen Regalen innerhalb einer Regalgasse in die Mitte gelegt werden oder je nach Wunsch der bedienenden Person auf eine Seite der Regalgasse, so dass ein möglichst ergonomisches Arbeiten möglich ist. Es ist auch möglich, dass durch eine solche Vorgabe einer Sollspur ein Ausrichten des Flurförderzeugs erfolgt und es beim Beginn des Kommissionierens in einer Regalgasse nicht erforderlich ist, dass das Flurförderzeug exakt von Hand ausgerichtet wird. Vielmehr muss das Flurförderzeug nur in die Regalgasse gefahren werden und richtet sich bei den ersten Fahrbewegungen zu dieser Regalgasse aus.For example, the shelf columns are detected as components of the shelves of the rack aisle. From the positions of the components of the shelf can be determined at a distance from a target track on which the truck should move. This setpoint track can also be placed in the middle of a rack aisle in double-sided shelves or, depending on the wishes of the serving person On one side of the rack aisle, so that the most ergonomic work possible. It is also possible that such a specification of a target track, an alignment of the truck takes place and it is not necessary at the beginning of order picking in a rack aisle, that the truck is aligned exactly by hand. Rather, the truck must only be driven into the rack aisle and aligns with the first movement movements to this rack aisle.

In einer vorteilhaften Ausgestaltung des Verfahrens wird das Ende einer Regalgasse erkannt und bei Erreichen des Endes der Regalgasse das Verfahren beendet.In an advantageous embodiment of the method, the end of a rack aisle is detected and terminated on reaching the end of the rack aisle the process.

Das Flurförderzeug kann dann automatisch stehen bleiben bzw. das Nachfolgen zu der Person beendet werden.The truck can then stop automatically or follow the succession to the person.

Es können aus den Daten Hindernisse erkannt werden und die Fahrzeugsteuerung kann bei der Fahrbewegung diesen ausweichen.It can be detected from the data obstacles and the vehicle control can avoid this during the driving movement.

Die durch den optischen Sensor gewonnenen Daten, insbesondere wenn es sich um dreidimensionale Daten handelt, können neben der Nachführung der Person sowie der Bestimmung einer Sollspur auch allgemein für die Fahrzeugführung verwendet werden, beispielsweise im Sinne eines autonom fahrenden Fahrzeugs. So kann bei entsprechender Auslegung der Steuerungsvorrichtung das Flurförderzeug so ausgeführt sein, dass nach Beendigung einer Kommissionierung auf Anforderung hin oder Eingabe eines Befehls das Flurförderzeug automatisch zu einer Warenabgabe fährt und den Behälter oder die Palette absetzt. Dies führt zu einer weiteren Zeitersparnis, da die kommissionierende Person das Flurförderzeug nicht dorthin steuern muss. Dies wird optimal ergänzt, wenn vorgesehen ist, dass das Flurförderzeug auf Anforderung hin auch autonom bis zu einer ersten Arbeitsposition einer kommissionierenden Person fährt.The data obtained by the optical sensor, in particular when it comes to three-dimensional data, in addition to the tracking of the person and the determination of a desired track also be used in general for the vehicle guidance, for example in the sense of an autonomously driving vehicle. Thus, with appropriate design of the control device, the truck can be designed so that after completion of a picking on request or by entering a command the truck automatically moves to a goods delivery and drops the container or pallet. This leads to a further time saving, since the commissioning person does not have to control the truck there. This is optimally supplemented if it is provided that the truck also drives autonomously on request to a first working position of a picking person.

Weiterhin ist es auch denkbar, die Daten für eine Befehlssteuerung des Flurförderzeugs durch Gesten bzw. Bewegungen der nachgeführten Person zu verwenden. Dadurch können beispielsweise weitere Befehle, die nicht die Fahrbewegung betreffen, wie etwa eine Hubbewegung etc., gesteuert werden. Schließlich können diese Daten auch dafür eingesetzt werden, die Höhe einer Beladung auf einer Palette auf der Lastgabel zu erfassen und zu überwachen.Furthermore, it is also conceivable to use the data for a command control of the truck by gestures or movements of the tracked person. As a result, for example, further commands that do not affect the travel movement, such as a lifting movement, etc., are controlled. Finally, these data can also be used to detect and monitor the height of a load on a pallet on the forklift.

Die Aufgabe wird auch gelöst durch Flurförderzeug mit einem optischen Sensor mit einem Überwachungsbereich, der mit einer Steuerungsvorrichtung verbunden ist, die ein zuvor beschriebenes Verfahren durchführt.The object is also achieved by an industrial truck with an optical sensor having a monitoring area, which is connected to a control device, which performs a method described above.

Das Flurförderzeug weist die bereits geschilderten Vorteile auf.The truck has the advantages already described.

Weitere Vorteile und Einzelheiten der Erfindung werden anhand der in den schematischen Figuren dargestellten Ausführungsbeispiele näher erläutert. Hierbei zeigt,

Fig. 1
schematisch in Aufsicht ein Flurförderzeug, bei dem das erfindungsgemäße Verfahren zur Anwendung kommt, mit einer kommissionierenden Person,
Fig. 2
schematisch in Aufsicht das Flurförderzeug und die Person der Fig. 1 bei einer Fahrbewegung des Flurförderzeugs und
Fig. 3
das Flurförderzeug der Figur. 1 in einer seitlichen Ansicht.
Further advantages and details of the invention will be explained in more detail with reference to the embodiments illustrated in the schematic figures. This shows
Fig. 1
schematically in plan view of an industrial truck, in which the inventive method is used, with a commissioning person,
Fig. 2
schematic in plan the truck and the person of Fig. 1 at a driving movement of the truck and
Fig. 3
the truck of the Figure. 1 in a side view.

Die Fig. 1 zeigt schematisch in Aufsicht ein Flurförderzeug 1, bei dem das erfindungsgemäße Verfahren zur Anwendung kommt, mit einer kommissionierenden Person 2. Das Flurförderzeug 1, hier in der Ausführungsform als Gabelhubwagen mit in Bezug auf eine Bewegungsrichtung nach hinten gerichteten Gabeln, befindet sich in einer Regalgasse 3 mit einem seitlichen Lastregal 4, wobei ein auf der anderen Seite befindliches Lastregal nicht dargestellt ist. Durch einen optischen Sensor 5, der im vorliegenden Beispiel als Kamera 6 mit einem Rundumaufnahmebereich und der Fähigkeit, Abstände zu erfassen, ausgeführt ist, wird innerhalb der Regalgasse 3 ein Überwachungsbereich 7 erfasst. In einer ebenfalls nicht dargestellten Steuerung werden die Daten des optischen Sensors 5 verarbeitet und die Position der nachzuführenden Person 2 erfasst. Dabei ist in der Steuerungsvorrichtung eine Überwachungsgrenze 8 hinterlegt, die einen vorderen Bereich 9 des Überwachungsbereichs 7 von einem hinteren Bereich 10 des Überwachungsbereichs 7 trennt. Während das Flurförderzeug 1 steht, entnimmt die kommissionierende Person 2 aus dem Lastregal 4 Waren und legt diese auf eine Transportpalette 11.The Fig. 1 1 shows schematically in plan view an industrial truck 1, in which the method according to the invention is used, with a picking person 2. The industrial truck 1, here in the embodiment as a pallet truck with forwards directed forks in relation to a direction of movement, is located in a rack aisle 3 with a lateral load shelf 4, wherein a load shelf located on the other side is not shown. An observation area 7 is detected within the rack aisle 3 by an optical sensor 5, which in the present example is designed as a camera 6 with an all-round pick-up area and the ability to detect clearances. In a control, also not shown, the data of the optical sensor 5 are processed and the position of the person to be tracked 2 detected. In this case, a monitoring limit 8 is stored in the control device, which includes a front area 9 of the monitoring area 7 from a rear area 10 of the monitoring area 7 separates. While the truck is 1, the picking person 2 removes goods from the load rack 4 and places them on a transport pallet 11.

Die Fig. 2 zeigt schematisch in Aufsicht das Flurförderzeug 1 mit der Transportpalette 11 und die Person 2 der Fig. 1 bei einer Fahrbewegung des Flurförderzeugs 1 in der durch den Pfeil dargestellten Bewegungsrichtung. Das Flurförderzeug 1 in der Regalgasse 3 mit dem seitlichen Lastregal 4 erfasst durch den optischen Sensor 5 in Form der Kamera 6 den Überwachungsbereich 7, der durch die Überwachungsgrenze 8 in einen vorderen Bereich 9 des Überwachungsbereichs 7 sowie einen hinteren Bereich 10 des Überwachungsbereichs 7 getrennt ist. Die Position der Person 2 befindet sich nun in dem vorderen Bereich 9 des Überwachungsbereichs 7. Daher steuert die Steuerungsvorrichtung einen Fahrantrieb des Flurförderzeugs 1 so lange an und verfährt das Flurförderzeug 1 in der Bewegungsrichtung, bis die Person 2 sich wieder hinter der Überwachungsgrenze 8 in dem hinteren Bereich 10 des Überwachungsbereichs 7 befindet. Dabei wird aus den Daten des optischen Sensors 5 der Rand 12 des Lastregals 4 bestimmt und in einem Abstand zu diesem Rand 12 eine Sollspur festgelegt, auf der das Flurförderzeug 1 von der Steuerungsvorrichtung verfahren wird.The Fig. 2 schematically shows in plan the truck 1 with the transport pallet 11 and the person 2 of Fig. 1 in a driving movement of the truck 1 in the direction of movement shown by the arrow. The truck 1 in the aisle 3 with the lateral load shelf 4 detected by the optical sensor 5 in the form of the camera 6, the monitoring area 7, which is separated by the monitoring boundary 8 in a front portion 9 of the monitoring area 7 and a rear portion 10 of the monitoring area 7 , The position of the person 2 is now in the front area 9 of the monitoring area 7. Therefore, the control device controls a traction drive of the truck 1 so long and moves the truck 1 in the direction of movement until the person 2 again behind the monitoring limit 8 in the Rear area 10 of the monitoring area 7 is located. In this case, the edge 12 of the load rack 4 is determined from the data of the optical sensor 5 and set at a distance from this edge 12 a target track on which the truck 1 is moved by the control device.

Vorteilhaft ist keinerlei Einweisung der kommissionierenden Person 2 erforderlich und folgt das Flurförderzeug 1 der Person 2 ohne dass Zeit erfordernde Bedienhandlungen der Person 2 erforderlich wären.Advantageously, no briefing of the picking person 2 is required and follows the truck 1 of the person 2 without time-requiring operator actions of the person 2 would be required.

Die Fig. 3 zeigt das Flurförderzeug 1 der Figur. 1 in einer seitlichen Ansicht mit dem Lastregal 4 der Regalgasse 3 sowie zusätzlichen Warenbehältern 13, aus denen beim Kommissionieren ebenfalls Waren entnommen werden können. Das Flurförderzeug 1 mit dem optischen Sensor 5, der als Kamera 6 ausgeführt ist, weist den Überwachungsbereich 7 auf. In dieser Ausführungsform erfolgt eine dreidimensionale Überwachung, da die Kamera 6 beispielsweise als Time of Flight Kamera (ToF) über Laufzeitmessungen den Abstand der Bildpunkte zu der Kamera 6 erfassen kann. In einer alternativen Ausführungsform, die gleichzeitig in die Fig. 3 eingezeichnet ist, kann der optische Sensor 5 auch als Laserscanner 14 ausgeführt sein, der in einer Ebene 15 scannt und dabei die Abstände zu Zielobjekten erfasst. Dies entspricht einer zweidimensionalen Erfassung.The Fig. 3 shows the truck 1 of Figure. 1 in a side view with the load shelf 4 of the rack aisle 3 and additional goods containers 13 from which goods can also be removed during order picking. The truck 1 with the optical sensor 5, which is designed as a camera 6, has the monitoring area 7. In this embodiment, there is a three-dimensional monitoring, since the camera 6 can record, for example as a time of flight camera (ToF) via runtime measurements, the distance of the pixels to the camera 6. In an alternative embodiment, simultaneously in the Fig. 3 is drawn, the optical sensor 5 can also be designed as a laser scanner 14, which scans in a plane 15 and thereby detects the distances to target objects. This corresponds to a two-dimensional detection.

Claims (17)

  1. Method for controlling an industrial truck (1), in particular during picking operations, wherein the industrial truck (1) has an optical sensor (5) which has a monitoring range (7) and is connected to a control device, and the control device determines and tracks the position of a person (2) within the monitoring range (7) by evaluating the data of the optical sensor (5),
    characterized
    in that within the monitoring range (7) at least one monitoring limit (8) is defined and detected via the control device if the position of the person (2) has exceeded the monitoring limit (8), and the industrial truck (1) is moved by the control device by means of a traction drive of the industrial truck (1) until the position of the person (2) is again on the original side of the monitoring limit (8).
  2. Method according to Claim 1,
    characterized
    in that the optical sensor (5) can detect distances from objects in the monitoring range (7).
  3. Method according to Claim 2,
    characterized
    in that the optical sensor (5) can detect the monitoring range (7) in a three-dimensional fashion.
  4. Method according to one of Claims 2 to 3,
    characterized
    in that changes in the position of the person (2) can be tracked, and the changes of the values can be evaluated as correct only in the case of continuous changes.
  5. Method according to Claim 4,
    characterized
    in that the data of the position of the person (2) can be differentiated from data of further moving objects or persons in the case of at least partial visual overlap in that the image data with continuous change are detected as the position of the tracked person (2).
  6. Method according to Claim 4 or 5,
    characterized
    in that prediction values within which the position should be located are determined for the future position of the person (2).
  7. Method according to Claim 6,
    characterized
    in that in the case of visual overlap of the tracked person (2) by other objects or persons, the originally tracked person (2) is determined on the basis of a search within the prediction values.
  8. The method according to one of Claims 1 to 7,
    characterized
    in that the position of a person (2) who is to be tracked is detected in a learning phase by a movement of the person.
  9. Method according to one of Claims 1 to 8,
    characterized
    in that the optical sensor (5) is a camera (6), and the position of a person (2) who is to be tracked is detected by means of image processing methods.
  10. Method according to one of Claims 1 to 9,
    characterized
    in that the monitoring limit (8) divides, on one or both sides of the industrial truck, a region (9) of the monitoring region (7), which is the front one in the direction of movement of the industrial truck, from a rear region (10).
  11. Method according to Claim 10,
    characterized
    in that the monitoring limit (8) is arranged starting from the industrial truck (1) perpendicularly with respect to the direction of movement of the industrial truck (1) as far as a maximum of less than 45° with respect to the direction of movement.
  12. Method according to one of Claims 1 to 11,
    characterized
    in that the industrial truck (1) is moved with the steering fixed in the straight-ahead position.
  13. Method according to one of Claims 1 to 11,
    characterized
    in that lane detection is carried out for a setpoint lane of the industrial truck (1) to be followed, using the data of the optical sensor and/or the data of a person protection device.
  14. Method according to Claim 13,
    characterized
    in that the edge (12) and the orientation of one side or of both sides of a rack aisle (3) are sensed, and a setpoint lane is defined at a distance from the edge (12) or both edges.
  15. Method according to Claim 14,
    characterized
    in that the end of a rack aisle (3) is detected, and when the end of the rack aisle (3) is reached the method is ended.
  16. Method according to one of Claims 13 to 15,
    characterized
    in that obstacles are detected on the basis of the data, and the vehicle controller avoids said obstacles during the driving movement.
  17. Industrial truck having an optical sensor (5) with the monitoring range (7), wherein the optical sensor (5) is connected to a control device which carries out a method according to one of Claims 1 to 16.
EP14183114.9A 2013-09-21 2014-09-02 Method for controlling a picking vehicle Active EP2851331B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PL14183114T PL2851331T3 (en) 2013-09-21 2014-09-02 Method for controlling a picking vehicle

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
DE102013110456.8A DE102013110456A1 (en) 2013-09-21 2013-09-21 Method for controlling a picking truck

Publications (2)

Publication Number Publication Date
EP2851331A1 EP2851331A1 (en) 2015-03-25
EP2851331B1 true EP2851331B1 (en) 2016-02-24

Family

ID=51429156

Family Applications (1)

Application Number Title Priority Date Filing Date
EP14183114.9A Active EP2851331B1 (en) 2013-09-21 2014-09-02 Method for controlling a picking vehicle

Country Status (3)

Country Link
EP (1) EP2851331B1 (en)
DE (1) DE102013110456A1 (en)
PL (1) PL2851331T3 (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3287412A1 (en) * 2016-08-24 2018-02-28 Jungheinrich Aktiengesellschaft Industrial truck and method for controlling an industrial truck
DE102016123542A1 (en) 2016-12-06 2018-06-07 Jungheinrich Aktiengesellschaft Method for automatic alignment of a truck in a warehouse and system of an industrial truck and a warehouse
DE102016123541A1 (en) 2016-12-06 2018-06-07 Jungheinrich Aktiengesellschaft Method for automatic alignment of a truck in a warehouse and system of an industrial truck and a warehouse
EP3511288A1 (en) 2018-01-10 2019-07-17 STILL GmbH Method for controlling a minor function of an industrial truck by means of a macro program and corresponding industrial truck
EP3511885A1 (en) 2018-01-10 2019-07-17 STILL GmbH Method and system for bidirectional communication in intralogistics by mobile voice input and output device
EP3511287A1 (en) 2018-01-10 2019-07-17 STILL GmbH Method for exchanging information between industrial trucks and intralogistics system with corresponding industrial trucks
EP3511884A1 (en) 2018-01-10 2019-07-17 STILL GmbH Method and system for bidirectional communication in intralogistics using smart device
US10482760B2 (en) 2017-12-01 2019-11-19 Jungheinrich Aktiengesellschaft Method for linking a second remote control unit to a first remote control unit
DE102018104998B4 (en) 2018-03-05 2023-09-21 Jungheinrich Ag Method for setting up a monitoring zone in a goods logistics facility and location system for a goods logistics facility

Families Citing this family (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6669948B2 (en) 2014-09-08 2020-03-18 日本電産株式会社 Moving object control device and moving object
DE102015111699A1 (en) * 2015-07-17 2017-01-19 Still Gmbh Control method for a picking truck
DE102015111611A1 (en) * 2015-07-17 2017-01-19 Still Gmbh Method for driving control of a picking truck
DE102015111697A1 (en) * 2015-07-17 2017-01-19 Still Gmbh Method for controlling a picking truck
EP3118152B1 (en) 2015-07-17 2018-12-05 STILL GmbH Method for controlling the speed of a picking industrial truck
DE102015113445A1 (en) 2015-07-17 2017-01-19 Still Gmbh Method for speed control of a picking truck
DE102015216482A1 (en) * 2015-08-28 2017-03-02 Robert Bosch Gmbh Control device and method for controlling a material handling vehicle, operating device and method for operating a material handling vehicle and picking system
DE102015119736A1 (en) 2015-11-16 2017-05-18 Linde Material Handling Gmbh Control method for industrial trucks
DE102015223329A1 (en) * 2015-11-25 2017-06-01 Zf Friedrichshafen Ag Method for controlling a material handling vehicle, evaluation device for a material handling vehicle, industrial truck
DE102016104088A1 (en) * 2016-03-07 2017-09-07 Still Gmbh Industrial truck with assistance device
EP3216746B1 (en) * 2016-03-07 2022-11-23 STILL GmbH Industrial truck with ambient sensors
DE102016104089A1 (en) * 2016-03-07 2017-09-07 Still Gmbh Method for controlling an industrial truck during picking
EP3216745B1 (en) * 2016-03-07 2018-04-11 STILL GmbH Control method for industrial truck
WO2018039559A2 (en) 2016-08-26 2018-03-01 Crown Equipment Corporation Materials handling vehicle obstacle scanning tools
DE102017200737A1 (en) 2017-01-18 2018-07-19 Robert Bosch Gmbh Manually movable and motorized transport vehicle
DE102017120998A1 (en) 2017-09-12 2019-03-14 Still Gmbh Method and device for calibrating an optical sensor on an industrial truck
DE102017120996A1 (en) 2017-09-12 2019-03-14 Still Gmbh Method and device for collision avoidance during operation of an industrial truck
DE102018109301A1 (en) 2018-01-10 2019-07-11 Still Gmbh Method and system for bidirectional communication in intralogistics using mobile speech input and output devices
DE102018109300A1 (en) 2018-01-10 2019-07-11 Still Gmbh Method for controlling a small functionality of a truck by means of a macro program and corresponding truck
DE102018109299A1 (en) 2018-01-10 2019-07-11 Still Gmbh Method for exchanging information between industrial trucks and intralogistic system with corresponding industrial trucks
DE102018109297A1 (en) 2018-01-10 2019-07-11 Still Gmbh Method and system for bidirectional communication in intralogistics via smart device
DE102018109298A1 (en) 2018-01-10 2019-07-11 Still Gmbh Method and system for detecting persons and / or vehicles when picking by means of radio location
CN110945450B (en) * 2018-10-10 2022-04-05 灵动科技(北京)有限公司 Human-computer interaction automatic guidance vehicle
CN113256195B (en) * 2020-02-10 2024-09-20 北京京东振世信息技术有限公司 Goods picking implementation method and device
DE102020215149A1 (en) 2020-12-01 2022-06-02 Robert Bosch Gesellschaft mit beschränkter Haftung Method for operating a vehicle in a rack aisle depending on route information

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002132347A (en) * 2000-10-23 2002-05-10 Shinko Electric Co Ltd Automatic conveyance system
US6784800B2 (en) * 2001-06-19 2004-08-31 Signal Tech Industrial vehicle safety system
US7017689B2 (en) * 2004-05-06 2006-03-28 Crown Equipment Corporation Electrical steering assist for material handling vehicles
US8195366B2 (en) * 2007-09-13 2012-06-05 The Raymond Corporation Control system for a pallet truck
EP2533119B1 (en) 2009-07-02 2017-02-08 Crown Equipment Corporation Apparatus for remotely controlling a materials handling vehicle

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102016115703A1 (en) 2016-08-24 2018-03-01 Jungheinrich Aktiengesellschaft Industrial truck and method for controlling an industrial truck
EP3287412A1 (en) * 2016-08-24 2018-02-28 Jungheinrich Aktiengesellschaft Industrial truck and method for controlling an industrial truck
US10556785B2 (en) 2016-08-24 2020-02-11 Jungheinrich Aktiengesellschaft Industrial truck and method for controlling an industrial truck
DE102016123542A1 (en) 2016-12-06 2018-06-07 Jungheinrich Aktiengesellschaft Method for automatic alignment of a truck in a warehouse and system of an industrial truck and a warehouse
DE102016123541A1 (en) 2016-12-06 2018-06-07 Jungheinrich Aktiengesellschaft Method for automatic alignment of a truck in a warehouse and system of an industrial truck and a warehouse
EP3333115A1 (en) * 2016-12-06 2018-06-13 Jungheinrich Aktiengesellschaft Method for automatic alignment of an industrial truck in a warehouse and system consisting of an industrial truck and a warehouse
US10289118B2 (en) 2016-12-06 2019-05-14 Jungheinrich Aktiengesellschaft Method for automatically aligning an industrial truck in a warehouse, and system comprising an industrial truck and a warehouse
US10482760B2 (en) 2017-12-01 2019-11-19 Jungheinrich Aktiengesellschaft Method for linking a second remote control unit to a first remote control unit
EP3511288A1 (en) 2018-01-10 2019-07-17 STILL GmbH Method for controlling a minor function of an industrial truck by means of a macro program and corresponding industrial truck
EP3511884A1 (en) 2018-01-10 2019-07-17 STILL GmbH Method and system for bidirectional communication in intralogistics using smart device
EP3511287A1 (en) 2018-01-10 2019-07-17 STILL GmbH Method for exchanging information between industrial trucks and intralogistics system with corresponding industrial trucks
EP3511885A1 (en) 2018-01-10 2019-07-17 STILL GmbH Method and system for bidirectional communication in intralogistics by mobile voice input and output device
DE102018104998B4 (en) 2018-03-05 2023-09-21 Jungheinrich Ag Method for setting up a monitoring zone in a goods logistics facility and location system for a goods logistics facility

Also Published As

Publication number Publication date
PL2851331T3 (en) 2016-07-29
DE102013110456A1 (en) 2015-03-26
EP2851331A1 (en) 2015-03-25

Similar Documents

Publication Publication Date Title
EP2851331B1 (en) Method for controlling a picking vehicle
EP2468678B1 (en) Industrial truck with a sensor for detecting the surroundings and method for operating such an industrial truck
EP3587220B1 (en) Vehicle with shunting system
EP2653429B1 (en) Control method for an industrial truck and industrial truck
EP3323770A1 (en) Method for providing the positions of storage areas in a warehouse and an industrial truck
EP3118152B1 (en) Method for controlling the speed of a picking industrial truck
EP3369696B1 (en) Industrial truck with sensor concept and industrial truck
DE202020103181U1 (en) An autonomous forklift
EP1641704A1 (en) Movable sensor device on the loading means of a forklift
EP2653430B1 (en) Control method for unloading of an industrial truck and industrial truck
EP3118706B1 (en) Method for controlling a picking vehicle
EP3333116B1 (en) Method for automatic alignment of an industrial truck in a warehouse and system consisting of an industrial truck and a warehouse
EP3453672A1 (en) Method and device for collision avoidance during the operation of an industrial truck
EP2987760B1 (en) Industrial truck
DE102015113445A1 (en) Method for speed control of a picking truck
DE202019102253U1 (en) Device for verifying storage goods positions in intralogistic storage systems
DE102019203200A1 (en) Control system for a driverless transport vehicle
EP3511287A1 (en) Method for exchanging information between industrial trucks and intralogistics system with corresponding industrial trucks
DE102021107530B3 (en) Method for automatically controlling an industrial truck and industrial truck for a method for automatically controlling and system for an automatically controlled industrial truck
DE102020215149A1 (en) Method for operating a vehicle in a rack aisle depending on route information
EP3170773A1 (en) Method for the manual picking of a plurality of items in a warehouse
EP3118707B1 (en) Method for controlling the locomotion of a picking truck
EP3168183B1 (en) Control method for an industrial truck and corresponding industrial truck
DE102016001839B3 (en) Driverless transport system
DE102013112016A1 (en) Industrial truck with a lifting height assistance system

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20140902

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

R17P Request for examination filed (corrected)

Effective date: 20150325

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

RIC1 Information provided on ipc code assigned before grant

Ipc: B66F 9/06 20060101ALI20150728BHEP

Ipc: B66F 9/075 20060101AFI20150728BHEP

INTG Intention to grant announced

Effective date: 20150902

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 776585

Country of ref document: AT

Kind code of ref document: T

Effective date: 20160315

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 502014000392

Country of ref document: DE

REG Reference to a national code

Ref country code: SE

Ref legal event code: TRGR

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20160224

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160224

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160224

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160524

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160525

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160224

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160224

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160624

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160224

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160224

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160224

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 3

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160224

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160224

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 502014000392

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160224

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160224

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160224

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 502014000392

Country of ref document: DE

Representative=s name: PATENTSHIP PATENTANWALTSGESELLSCHAFT MBH, DE

26N No opposition filed

Effective date: 20161125

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 502014000392

Country of ref document: DE

Representative=s name: PATENTSHIP PATENTANWALTSGESELLSCHAFT MBH, DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160224

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160524

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160930

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160224

REG Reference to a national code

Ref country code: FR

Ref legal event code: RU

Effective date: 20170406

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160902

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160902

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 4

REG Reference to a national code

Ref country code: FR

Ref legal event code: D7

Effective date: 20170828

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20160930

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20140902

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160224

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160224

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160224

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160224

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170930

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170930

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 5

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160224

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160224

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 776585

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190902

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190902

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230518

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: CZ

Payment date: 20230818

Year of fee payment: 10

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: SE

Payment date: 20230921

Year of fee payment: 10

Ref country code: PL

Payment date: 20230818

Year of fee payment: 10

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20230929

Year of fee payment: 10

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240919

Year of fee payment: 11

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20240923

Year of fee payment: 11

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20240924

Year of fee payment: 11