EP2990532B1 - Selbstfahrende baumaschine und verfahren zur visualisierung des bearbeitungsumfeldes einer sich im gelände bewegenden baumaschine - Google Patents

Selbstfahrende baumaschine und verfahren zur visualisierung des bearbeitungsumfeldes einer sich im gelände bewegenden baumaschine Download PDF

Info

Publication number
EP2990532B1
EP2990532B1 EP15180819.3A EP15180819A EP2990532B1 EP 2990532 B1 EP2990532 B1 EP 2990532B1 EP 15180819 A EP15180819 A EP 15180819A EP 2990532 B1 EP2990532 B1 EP 2990532B1
Authority
EP
European Patent Office
Prior art keywords
construction machine
project
orientation
terrain
coordinate system
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP15180819.3A
Other languages
German (de)
English (en)
French (fr)
Other versions
EP2990532A1 (de
Inventor
Matthias Fritz
Cyrus Barimani
Günter HÄHN
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wirtgen GmbH
Original Assignee
Wirtgen GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wirtgen GmbH filed Critical Wirtgen GmbH
Publication of EP2990532A1 publication Critical patent/EP2990532A1/de
Application granted granted Critical
Publication of EP2990532B1 publication Critical patent/EP2990532B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • E—FIXED CONSTRUCTIONS
    • E01—CONSTRUCTION OF ROADS, RAILWAYS, OR BRIDGES
    • E01C—CONSTRUCTION OF, OR SURFACES FOR, ROADS, SPORTS GROUNDS, OR THE LIKE; MACHINES OR AUXILIARY TOOLS FOR CONSTRUCTION OR REPAIR
    • E01C23/00—Auxiliary devices or arrangements for constructing, repairing, reconditioning, or taking-up road or like surfaces
    • E01C23/01—Devices or auxiliary means for setting-out or checking the configuration of new surfacing, e.g. templates, screed or reference line supports; Applications of apparatus for measuring, indicating, or recording the surface configuration of existing surfacing, e.g. profilographs
    • E—FIXED CONSTRUCTIONS
    • E01—CONSTRUCTION OF ROADS, RAILWAYS, OR BRIDGES
    • E01C—CONSTRUCTION OF, OR SURFACES FOR, ROADS, SPORTS GROUNDS, OR THE LIKE; MACHINES OR AUXILIARY TOOLS FOR CONSTRUCTION OR REPAIR
    • E01C19/00—Machines, tools or auxiliary devices for preparing or distributing paving materials, for working the placed materials, or for forming, consolidating, or finishing the paving
    • E01C19/48—Machines, tools or auxiliary devices for preparing or distributing paving materials, for working the placed materials, or for forming, consolidating, or finishing the paving for laying-down the materials and consolidating them, or finishing the surface, e.g. slip forms therefor, forming kerbs or gutters in a continuous operation in situ
    • E01C19/4886—Machines, tools or auxiliary devices for preparing or distributing paving materials, for working the placed materials, or for forming, consolidating, or finishing the paving for laying-down the materials and consolidating them, or finishing the surface, e.g. slip forms therefor, forming kerbs or gutters in a continuous operation in situ for forming in a continuous operation kerbs, gutters, berms, safety kerbs, median barriers or like structures in situ, e.g. by slip-forming, by extrusion

Definitions

  • the invention relates to a self-propelled construction machine, in particular a road milling machine or a slipform paver, which has a chassis, the front and rear wheels or drives in the working direction, carried by the chassis machine frame, a drive device for driving the front and / or rear wheels or drives and a steering device for directing the front and / or rear wheels or drives, so that the construction machine can perform translational and / or rotational movements in the field.
  • the invention relates to a method for visualizing the processing environment of a terrain machine moving in the field, in particular a road milling machine or a slipform paver.
  • the slipform pavers have a device for forming flowable material, in particular concrete, which is also referred to as concrete hollow. With the concrete trough structures of different education, such as baffles or gutters can be produced.
  • the working device has a milling drum equipped with milling tools, with the material from the road surface can be milled in a given working width.
  • the EP 2 336 424 A2 describes a self-propelled construction machine having data describing a target curve descriptive data in an independent of the position and orientation of the construction machine reference system and a control unit which is designed such that a reference point on the construction machine starting from a predetermined starting point at which the construction machine has a predetermined position and orientation in the field, moves on the target curve.
  • a method for controlling a construction machine in which the data describing a setpoint curve in an independent of the position and orientation of the construction machine reference system by means of a measuring device (rover) determined in the field and read into a working memory of the construction machine.
  • the known method allows the control of the construction machine without major surveying effort with high accuracy.
  • the problem arises that objects already present in the terrain, for example water drains, hydrants or manhole covers, must be taken into account.
  • the building should not be located on a water drain, or the area of the terrain, such as a fire hydrant or manhole cover, should not be altered.
  • the milling drum of a road milling machine for example, when driving over a hydrant, taking into account a safety distance within a predetermined distance, which is dependent on the dimensions of the hydrant, must be raised from a predetermined position with respect to the surface to be machined.
  • the operator can not see the exact position of the hydrant at the level of the milling drum in practice, however, because the milling drum is located below the control station. Therefore, the position of a hydrant in the field is in practice marked with lateral lines that are recognizable to the operator or to another person.
  • the marking of existing objects in the field proves to be disadvantageous in practice.
  • the marking of the objects requires an additional step.
  • the lines are not or difficult to reach in the dark detect. Incidentally, the marking of the objects in the rain is not readily possible. Because of the inaccuracies, it is therefore necessary to choose a relatively large safety margin, which requires major reworking.
  • slipform paver In a slipform paver, the same problem arises when a building is to be erected, which should not be on but adjacent to existing objects. For example, if the structure is to extend along a curb, water courses adjacent to the curb can not be detected by the operator if the water drains are immediately in front of or adjacent to the machine.
  • a slipform paver adds to the difficulty that no short-term corrections to the trajectory are possible if it is determined shortly before the water intake that the planned trajectory is on this.
  • an automatic control of the construction machine also taking into account objects existing in the terrain, it is fundamentally possible to determine the shape and position of the objects in the terrain. If the shape and position of the objects are known, an intervention in the machine control can also be carried out automatically, for example, when driving over the object, the milling drum of a road milling machine can be raised automatically.
  • this requires an exact determination of the shape and position of the object, for example the hydrant with respect to the coordinate system in which the construction machine is to move. Otherwise the hydrant or the construction machine may be damaged.
  • the invention has for its object to provide a self-propelled construction machine, in particular a road milling machine or a slipform paver, which simplifies the consideration of off-site objects in the control of the construction machine for the construction of a building or the change of the terrain in practice.
  • Another object of the invention is to provide a method with which the consideration of existing in the field objects can be simplified.
  • the construction machine according to the invention is a self-propelled construction machine, which has a working device for the construction of structures on the site, for example a device for forming concrete, or a device for changing the terrain, for example a milling drum.
  • a working device for the construction of structures on the site for example a device for forming concrete, or a device for changing the terrain, for example a milling drum.
  • the construction machine can be, for example, a road milling machine or a slipform paver. It can also be a paver with the same problem of considering objects already in the terrain.
  • the construction machine has an image recording or recording unit for recording an image of the terrain, which is in a dependent on the position and orientation of the construction machine in the field coordinate system, and a display unit for displaying the image of the area.
  • the image section should be selected so that all areas relevant to the control of the construction machine are detected, wherein the image section may also include areas that are not visible to the operator from the control station.
  • the image recording unit may include one or more camera systems. If the image recording unit has a plurality of camera systems, the image section can be composed of a plurality of images, each recorded with a camera system. Each camera system can also be assigned a separate image section.
  • the camera system may include one camera or two cameras (stereo camera system). If a three-dimensional scene is imaged onto the two-dimensional image plane of the camera when shooting with a camera, a clear correlation between the coordinates of an object, the coordinates of the image of the object on the image plane and the focal length of the camera results. However, the depth information is lost due to the two-dimensional image.
  • the camera system has only one camera, since in practice, the curvature of the terrain surface in the captured by the camera image detail can be neglected. Moreover, only two-dimensional scenes are relevant to the invention, i. H. the outlines of objects in a plane (terrain surface). However, the invention is not limited thereto.
  • the at least one camera system of the image recording unit can also be a stereo camera system comprising two cameras, which are arranged parallel to the axis at a predetermined horizontal distance, in accordance with the known methods the disparity to gain the depth information.
  • the invention requires a device for providing project data which describes the shape and position of at least one project in a coordinate system independent of the position and orientation of the construction machine.
  • a project is understood to mean all work to be carried out with the construction machine, which forms the basis for the control of the construction machine, whereby the project is determined by which works (form) are carried out at a specific location (location).
  • the project may involve the construction of a building or the alteration of the site.
  • the project data may be the data describing the shape and location of a building to be erected in the field.
  • the project data can be, for example, the shape and position of a baffle to be constructed descriptive data or in the known road milling machines to be processed in the area or not to be processed surface descriptive data.
  • the project data represent parameters for the control of the construction machine, which include, for example, the feed rate and inclination of the concrete trough of a slipform paver or the milling depth of a milling machine.
  • parameters for the control of the construction machine include, for example, the feed rate and inclination of the concrete trough of a slipform paver or the milling depth of a milling machine.
  • the construction machine has a data processing unit which is configured such that the image section of the terrain displayed on the display unit the part of the project lying in the image section is superimposed so that at least a part of the project is displayed in the image section.
  • the display unit thus shows not only the real image detail, but also a virtual image of the project, so that the perception of the machine operator is extended. Consequently, the operator can recognize on the display unit whether the project underlying the control matches reality.
  • the machine operator can intervene in the machine control in advance. Alternatively, an automatic intervention in the machine control can be made.
  • This error may, for example, be that the terrain object (s) that reflect reality have not been or have not been detected correctly for the control of the construction machine.
  • the machine operator can recognize when the surface to be machined, for example the surface to be milled with a road milling machine, lies on a hydrant or the structure to be erected with a slipform paver, for example a guide wall, should run over a water inlet.
  • a preferred embodiment of the invention provides that the construction machine has a device for determining position / orientation data describing the position and orientation of the construction machine in a coordinate system independent of the construction machine.
  • the project data are determined in a coordinate system independent of the position and orientation of the construction machine, which does not change with the movement of the construction machine in the field.
  • the means for determining the position / orientation data describing the position and orientation of the construction machine preferably comprises a global navigation satellite system (GNSS) comprising first and second GNSS receivers for decoding GNSS signals of the Global Navigation Satellite System (GNSS) and Correction signals of a reference station for determining the position and orientation of the construction machine may have, wherein the first and second GNSS receiver on the construction machine in different positions are arranged. With the first and second GNSS receiver, the measurement accuracy can be increased.
  • GNSS global navigation satellite system
  • the position and orientation of the construction machine can also be determined with a satellite-independent system, for example with a tachymeter.
  • a further preferred embodiment provides that the project data describing the shape and position of the at least one project in the coordinate system independent of the position and orientation of the construction machine in dependence on the known position and orientation of the construction machine in the non-construction machine coordinate system in be transformed depending on the position and orientation of the construction machine coordinate system.
  • the project data available in the fixed coordinate system can then be superimposed on the image section in real time, so that the project is always visible in correct alignment with the real image, which can constantly change with the movement of the construction machine.
  • the project data comprise at least one outline of the project descriptive data, wherein the data processing unit is configured such that in the image section of the terrain, the at least one outline of the project is displayed.
  • the outline and the shape of the project in the image section are sufficiently marked. If the project is, for example, a building, the building can also be highlighted by a colored underlay or a hatching or can be represented by it alone.
  • the data processing unit is configured to determine object data describing the shape and location of at least one real object in the image area of the terrain, the project data then being compared to the object data.
  • object data is understood to be all data describing the shape and position of the objects present in the terrain and recorded with the image recording unit, which objects are represented as real objects in the image detail.
  • the object data can describe, for example, the position and shape of a structure, for example a hydrant or a water inlet in the area, which should not be covered or damaged during the construction of a building or the change of the terrain.
  • the comparison of the project data with the object data allows beyond the extension of the perception of the machine operator, a computer-aided monitoring of the control of the construction machine, where it can be determined that the determined project data does not match the object data (reality).
  • the known mathematical algorithms can be used, for example, to be able to determine whether the building is actually next to the water inlet.
  • a particularly simple evaluation of the data provides for determining the distance between at least one reference point related to the outline of the project and at least one reference point related to the outline of the object.
  • the reference point can lie even on the outline, for example on a circle or circular arc, or next to the outline, for example, lie on the center of a circle.
  • the determined distance is preferably compared with a predetermined limit value. If the distance between reference points lying on the contour lines is smaller than a predetermined limit, it can be concluded that a minimum distance is not maintained. This minimum distance can be visualized on the display unit. Another possibility is to base the areas enclosed by the contours on the evaluation. It is also possible to determine whether the outline of the project defined around the object, taking into account a predetermined minimum distance, intersects with the outline of the object. In the case that the project line intersects the object line, it can be concluded that the project line does not enclose the object line, ie the project and the object do not match, but at least partially overlap.
  • the construction machine preferably has an alarm unit which gives an optical and / or audible and / or tactile alarm when the data processing unit has determined that the project and the object do not match, for example the project line and object line intersect and / or or the determined distance between the outlines of project and object is less than a predetermined limit. It is also possible to generate a control signal for intervention in the machine control.
  • the construction machine has an interface for reading in the project data and a memory unit for storing the read-in project data. This makes it possible to determine the project data required for controlling the construction machine in advance.
  • the project data are preferably determined in the field with a preferably satellite-based measuring device (rover).
  • FIGS. 1A and 1B show in side view and plan view as an example of a self-propelled construction machine a slipform paver.
  • slipform paver is in the EP 1 103 659 B1 described in detail. Since slipform pavers as such belong to the prior art, only the essential components of the construction machine of the invention will be described here.
  • the slipform paver 1 has a machine frame 2, which is supported by a chassis 3.
  • the chassis 3 has two front and two rear crawler tracks 4A, 4B, which are attached to front and rear lifting columns 5A, 5B.
  • the working direction (Direction of travel) of the slipform paver is marked with an arrow A. But it can also be provided only a front or rear drive.
  • the track drives 4A, 4B and lifting columns 5A, 5B constitute the drive means of the slipform paver for performing translational and / or rotational movements of the construction machine on the field.
  • the track drives 4A, 4B and lifting columns 5A, 5B constitute the drive means of the slipform paver for performing translational and / or rotational movements of the construction machine on the field.
  • the machine frame 2 By raising and lowering the lifting columns 5A, 5B, the machine frame 2 can be moved with respect to the ground in the height and inclination.
  • the chain drives 4A, 4B the slipform paver can be moved back and forth.
  • the construction machine has three translatory and three rotational degrees of freedom.
  • the slipform paver 1 has an only hinted illustrated device 6 for forming concrete, which is referred to below as a concrete trough.
  • the concrete trough 6 represents the working device of the slipform paver for the construction of a building structure with a predetermined shape on the site.
  • FIGS. 2A and 2B show as a further example of a self-propelled construction machine, a road milling machine in the side view, wherein the same reference numerals are used for the corresponding parts.
  • the road milling machine 1 has a machine frame 2, which is supported by a chassis 3.
  • the chassis 3 again has front and rear crawler tracks 4A, 4B mounted on front and rear lifting columns 5A, 5B. But it can also be provided only a front or rear drive.
  • the road milling machine has a working device for changing the terrain. This is a milling device 6 with a equipped with milling tools milling drum, which is not recognizable in the figures.
  • the milled material is transported away with a conveyor F.
  • Fig. 3 The road surface to be worked with a road milling machine is in Fig. 3 shown.
  • the project is to milled the pavement of the road.
  • objects O on the road such as manhole cover in the middle of the road surface and water inlets at the Side of the road surface.
  • Fig. 3 shows two manhole covers 9, 10 and a water inlet 11, which are run over by the road milling machine when milling the road surface.
  • the representation in Fig. 3 but does not correspond to the field of vision of the machine operator.
  • the objects O on the road can not be seen by the operator on the platform of the construction machine, since they are located directly in front of the construction machine or below the machine.
  • the machine operator can not recognize the manhole cover, in particular, when the milling drum is only a short distance in front of the manhole cover, ie exactly at the time when the machine operator has to lift the milling drum. However, this area can not be monitored with a camera because of the flying milling material in the milling drum housing.
  • the position and shape of the circular manhole covers 9, 10 are clearly described by three lying on the circumference reference points O 11 , O 12 , O 13 and O 21 , O 22 , O 23 .
  • the position and shape of the rectangular water inlets are described by four reference points O 31 , O 32 , O 33 , O 34 located at the corners of the water inlet.
  • the project is described by previously created project data, which are read in via a suitable interface 12A in a working memory 12 of the construction machine ( Fig. 8 ).
  • the project data contains the coordinates of reference points characteristic of the project, which are detected in a coordinate system (X, Y, Z) independent of the position and orientation of the construction machine.
  • the reference points lie on the contours 13, 14, 15, which enclose the contours 16, 17, 18 of the objects O at a predetermined minimum distance ⁇ .
  • the objects O are circular Manhole covers 9, 10 and rectangular water inlets 11 are, the outlines describing the project are also circles and rectangles.
  • the circular outlines 13, 14 of the project are uniquely identified by the coordinates of three reference points P 11 , P 12 , P 13 and P 21 , P 22 , P 23 and the rectangular outlines 15 of the project by the coordinates of four reference points P 31 , P 32 , P 33 , P 34 in the independent of the movement of the construction machine coordinate system (X, Y, Z) described.
  • the project data comprises the coordinates of the reference points of the project in the fixed coordinate system independent of the movement of the construction machine (X, Y, Z). They mark the area to be cut, which lies outside the outlines 13, 14, 15 of the project. The area not to be processed is the area within the outlines 13, 14, 15 of the project, in which the objects O lie. This clearly determines the project.
  • the project data can be determined as follows.
  • the fixed coordinate system (X, Y, Z) is preferably the coordinate system of a global navigation satellite system (GNSS), so that the reference points of the objects can be easily detected with a measuring device (rover).
  • GNSS global navigation satellite system
  • the project data can be stored in an external storage unit, for example a USB stick, and read into the internal storage unit 12 of the construction machine via the interface 12A. With this data, the construction machine can then be controlled. When the road milling machine reaches a surface that is not to be machined, the milling drum is automatically raised with respect to the ground. As soon as the road milling machine has traveled over the area not to be worked, the milling drum is lowered again. This avoids that the manhole cover 9, 10 or water inlet 11 or the construction machine is damaged. The raising and lowering of the milling drum can also be done with a manual intervention in the machine control, with the machine operator, the time at which the intervention is to be signaled.
  • an external storage unit for example a USB stick
  • the road milling machine has an image recording unit 19, which has a camera system 19A arranged on the machine frame 2, with which an image section 20A of the terrain to be processed, ie the road surface with the manhole covers and water inlets, is taken.
  • the camera system 19A detects an area not visible to the operator by the operator.
  • the image section 20A is displayed on a display unit 20, for example an LC display.
  • Fig. 4 shows the display of the display unit 20. While the road milling machine moves in the field, constantly changing the image shown in the image section 20A, so that the operator can recognize that he moves with the road milling machine on a manhole cover 9, 10 or water inlet 11.
  • the road milling machine has a data processing unit 21, with which the project data available is processed.
  • the data processing unit 21 is configured such that the image section 20A of the terrain displayed on the display unit 20 is superimposed on the project lying in the image section.
  • the outlines 16, 17, 18 of the project which identify the surface to be worked or the surface not to be processed, are displayed in the image section 20A as they correspond to the previously determined project data. The operator can thus immediately recognize on the display unit 20 if the project data should not correspond to reality, ie the outlines 16, 17, 18 of the project do not concentrically surround the outlines 13, 14, 15 of the objects O in the predetermined minimum distance ⁇ should. If the manhole cover and water inlets within the displayed outlines On the other hand, the control of the road milling machine can take place without any further intervention in the machine control.
  • the image section 20A is assigned a coordinate system (x, y, z) which depends on the movement of the construction machine in the terrain and which is in Fig. 3 is shown.
  • the position (origin) and orientation of this coordinate system corresponds to the location and viewing angle of the camera 19A on the construction machine.
  • the position and shape of the objects O are also described by corresponding coordinates.
  • the coordinate system (x, y, z) dependent on the movement of the construction machine in the field can be a three-dimensional or two-dimensional coordinate system.
  • Fig. 3 the general case of a coordinate system with an x-axis, y-axis and z-axis is shown.
  • a two-dimensional coordinate system With a negligible curvature of the terrain surface and the consideration of only two-dimensional objects but a two-dimensional coordinate system is sufficient. However, this presupposes that the x / y plane of the coordinate system is parallel to the terrain surface assumed to be assumed, which is assumed below.
  • the camera system may be a stereo camera system or a camera system with only one camera. With negligible curvature of the terrain surface and / or the consideration of only two-dimensional objects but a camera system with only one camera is sufficient. If the camera system is a stereo camera system, three-dimensional images can also be displayed on the display unit 20 using the known methods.
  • the construction machine For determining the position and orientation of the construction machine and thus also the position and orientation (viewing angle) of the camera system 19A in the coordinate system (X, Y, Z) independent of the position and orientation of the construction machine, the construction machine has a device 22 which Provides position / orientation data of the construction machine ( Fig. 8 ).
  • This device may comprise a first GNSS receiver 22A and second GNSS receiver 22B arranged on the construction machine at different positions S1, S2.
  • Fig. 1B shows the position S1 and S2 of the two GNSS receivers 22A and 22B on the slipform paver.
  • the first and second GNSS receivers 22A, 22B decode the GNSS signals of the global navigation satellite system (GNSS) and correction signals of a reference station for determining the position and orientation of the construction machine.
  • GNSS global navigation satellite system
  • Such systems which enable a highly accurate determination of the position / orientation data, belong to the state of the art.
  • an electronic compass K may be provided to detect the orientation of the construction machine.
  • Fig. 2B shows the position S1 of the first GNSS receiver 22A and the position S2 of the compass K on the road milling machine.
  • the compass can also be dispensed with if the orientation of the construction machine is calculated.
  • the orientation can be calculated by determining the position of a reference point of the construction machine at successive times and determining the direction of the movement from the change in position.
  • the accuracy can be additionally increased by including the steering angle in the calculation.
  • the data processing unit 21 receives the current position / orientation data, which is continuously provided by the device 22 for determining the position and orientation of the construction machine, and transforms the shape and position of the project in the coordinate system independent of the position and orientation of the construction machine (FIG. X, Y, Z) depending on the position and orientation of the construction machine in the non-construction machine coordinate system in the dependent on the position and orientation of the construction machine machine coordinate system (x, y, z). This data transformation takes place in real time. After the coordinates of the reference points characterizing the outlines of the project in the machine coordinate system are known, the outlines 16, 17, 18 of the project are displayed in the image section 20A ( Fig. 4 ). The operations of the data processing unit required to generate the contours belong to the state of the art.
  • the data processing unit 21 can comprise an image processing unit which can automatically recognize whether the real objects O match the virtual objects, ie the real outline 13, 14, 15 of an object O (hydrant or water inlet) shown in the image section, actually within the associated virtual one Outline 16, 17, 18 of the project lies.
  • the data processing unit 21 is configured such that the shape and position of the real object O (hydrant or water inlet) recorded by the camera system 19A is determined in the image section 20A.
  • the data processing unit 21 can make use of the known methods for image recognition.
  • the shape and position of the real object in the image section are described by object data.
  • the circular outline of the manhole cover 9 is described by the three contoured reference points P 11 , P 12 , P 13 ( Fig. 3 ).
  • the object data is compared in the data processing unit 21 with the project data to determine whether the real objects match the virtual objects.
  • the data processing unit checks whether the outline 13 of the real object, for example the manhole cover 9, lies within the outline 16 of the project. For this purpose, the data processing unit 21 checks whether the two contour lines 13, 16 intersect. If the contours 13, 16 do not intersect, it is concluded that the object data corresponds to reality. Otherwise, an erroneous determination of the object data is concluded.
  • Fig. 5A shows the case that the object data match the project data, ie the outlines 13, 16 have no intersection while Fig. 5B the case shows that the object data do not match the project data, ie the outlines 13, 16 intersect at two points P s .
  • the data processing unit 21 can still determine whether a minimum distance ⁇ is maintained.
  • the data processing unit determines two reference points P A1 and P A2 , which are assigned to the outline 13 of the object or the outline 16 of the project.
  • reference points P A1, P A2 points are determined lying on the circular contour lines 13, 16 especially close to each other ( Fig. 5A ).
  • the data processing unit 21 determines the distance a between the lines lying on the outline of reference points P A1, P A2 and compares the distance with a predetermined limit value. If the distance between the points is smaller than the predetermined limit value, it is concluded that the outline 13 of the object lies within the project, since the outlines 13, 16 do not intersect.
  • the reference points can also be the midpoints or line or area centers of the circular outlines. In an exact alignment taking into account the predetermined minimum distance ⁇ the outlines 13, 16 have a common center or line or centroid, ie the distance between the centers should be as small as possible.
  • the above embodiment is only to be understood as an embodiment for comparing the project data and object data with each other.
  • the data can also be evaluated with all other known algorithms to infer whether the real objects match the virtual objects.
  • the construction machine has an alarm unit 23 which gives an optical and / or audible and / or tactile alarm when the data processing unit 21 has determined that the two contour lines 13, 16 do not match and / or the distance a is smaller than a predetermined one Limit is ( Fig. 8 ).
  • the operator can be pointed to an erroneous determination of the object data also by color shading of certain surfaces, hatching or by markings.
  • the distance a can also be displayed.
  • FIGS. 6 and 7 described a further embodiment of the invention, which differs from the previous embodiment in that the project does not change the terrain with a road milling machine ( Fig. 2 ), but the erection of a structure with a slipform paver ( Fig. 1 ).
  • the slipform paver like the road milling machine, has an image recording unit 12 and a data processing unit 21 and a device 12 for providing the project data (FIG. Fig. 8 ).
  • the corresponding parts are provided with the same reference numerals.
  • the slipform paver project is a traffic island bounded laterally by a curb 25 of concrete.
  • the curb 25 has, for example, a straight section 25A, followed by a semicircular section 25B.
  • the curb 25 should be adjacent to a rectangular water inlet 26, which requires precise control of the slipform paver.
  • the project data again contains the coordinates of characteristic reference points for the project, which are detected in a coordinate system (X, Y, Z) independent of the position and orientation of the construction machine.
  • the project data describe the shape and position of the curb 25.
  • the shape and position of the straight portion 25A can be described, for example, by two reference points P 1 , P 2 and P 3 , P 4 at the beginning and end of the inner and outer Outline 27, 28 of the curb 25 are.
  • the semi-circular portion 25B may be described, for example, by three reference points P 2 , P 5 , P 6 and P 4 , P 7 , P 8 lying on the inner and outer contour lines 27, 28, respectively.
  • the previously determined project data relating to the GNSS system independent of the position and orientation are read into the working memory 12 of the slipform paver via the interface 12A.
  • the control unit of the slipform paver is configured such that the slipform paver moves on a path corresponding to the course of the curb 25 to be erected.
  • FIGS. 6 and 7 show the image section 20A taken by the camera system 19A of the image recording unit 19 and displayed on the display unit 20, in which the terrain lying in working direction A in front of the slipform paver and a part of the slipform paver with the concrete recess 6 can be seen.
  • the terrain position and orientation determining means 22 of the slipform paver continuously calculates the current position / orientation data, and the data processing unit 21 obtains the project data which is in the GNSS system (X, Y, P) independent of the position and orientation of the slipform paver.
  • Z) are transformed into the machine coordinate system (x, y, z) which depends on the position and orientation of the slipform paver and which corresponds to the viewing angle of the camera system.
  • FIGS. 6 and 7 show a possibility of displaying the curb 25 in the image section 20A by the contour lines 27, 28, which show the machine operator the course of the curb, which is made with the slipform paver, if the stored project data of the control are based.
  • contour lines 27, 28 show the machine operator the course of the curb, which is made with the slipform paver, if the stored project data of the control are based.
  • color backgrounds, hatchings, auxiliary lines or markings can also be generated by the data processing unit 21 and displayed on the display unit 20.
  • the machine operator can check the correct course of the curb 25 in the image section 20A. He can see in advance whether the curb 25, for example, runs next to the water inlet 26.
  • Fig. 6 shows the case of a correct course of the curb 25 close to, ie at a predetermined minimum distance from the water inlet 26, while Fig. 7 the case shows that the curb 25 extends over the water inlet 26.
  • the Alarm unit 23 an alarm signal, so that the operator can make an intervention in the machine control.
  • the data processing unit 21 with an image recognition determines the coordinates of reference points O 1 , O 2 , O 3 , O 4 of the rectangular water inlet 26 in the machine coordinate system (x, y, z) corresponding to the camera image. Since the standardized shape and size of the water inlet 26 is known, for example, the coordinates of the corner points of the water inlet can be determined with an image recognition without much computational effort. These coordinates then provide the object data that is compared to the project data to determine if the design is consistent with reality. For this purpose, it can be checked with the data processing unit 21, for example, whether the contours of the curb and water inlet intersect, and / or with the data processing unit, for example, the distance between the contours can be calculated, as described with reference to the other embodiment.

Landscapes

  • Engineering & Computer Science (AREA)
  • Architecture (AREA)
  • Civil Engineering (AREA)
  • Structural Engineering (AREA)
  • Road Paving Machines (AREA)
  • Component Parts Of Construction Machinery (AREA)
  • Road Repair (AREA)
EP15180819.3A 2014-08-28 2015-08-12 Selbstfahrende baumaschine und verfahren zur visualisierung des bearbeitungsumfeldes einer sich im gelände bewegenden baumaschine Active EP2990532B1 (de)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
DE102014012836.9A DE102014012836B4 (de) 2014-08-28 2014-08-28 Selbstfahrende Baumaschine und Verfahren zur Visualisierung des Bearbeitungsumfeldes einer sich im Gelände bewegenden Baumaschine

Publications (2)

Publication Number Publication Date
EP2990532A1 EP2990532A1 (de) 2016-03-02
EP2990532B1 true EP2990532B1 (de) 2017-02-15

Family

ID=53871917

Family Applications (1)

Application Number Title Priority Date Filing Date
EP15180819.3A Active EP2990532B1 (de) 2014-08-28 2015-08-12 Selbstfahrende baumaschine und verfahren zur visualisierung des bearbeitungsumfeldes einer sich im gelände bewegenden baumaschine

Country Status (4)

Country Link
US (1) US9719217B2 (zh)
EP (1) EP2990532B1 (zh)
CN (2) CN205024576U (zh)
DE (1) DE102014012836B4 (zh)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102014012836B4 (de) * 2014-08-28 2018-09-13 Wirtgen Gmbh Selbstfahrende Baumaschine und Verfahren zur Visualisierung des Bearbeitungsumfeldes einer sich im Gelände bewegenden Baumaschine
EP3276079B1 (en) 2016-07-26 2021-07-14 Caterpillar Paving Products Inc. Control system for a road paver
DE102018119962A1 (de) * 2018-08-16 2020-02-20 Wirtgen Gmbh Selbstfahrende Baumaschine und Verfahren zum Steuern einer selbstfahrenden Baumaschine
US11913188B2 (en) * 2021-02-11 2024-02-27 R.H. Borden And Company, Llc Automated tool access to a manhole through automated manhole cover removal using self-driving vehicle
CN113126560B (zh) * 2021-03-18 2024-07-12 浙江美通筑路机械股份有限公司 一种带机器视觉的井盖铣刨控制系统及控制方法
CN114263085B (zh) * 2022-01-13 2024-07-05 武汉英途工程智能设备有限公司 一种摊铺机智能摊铺控制系统
CN114737456B (zh) * 2022-05-06 2023-08-01 江苏徐工工程机械研究院有限公司 铣刨转子和铣刨机
CN118563626B (zh) * 2024-07-30 2024-10-11 四川顶圣工程项目管理有限公司 一种公路工程施工路面质量精细化管理系统

Family Cites Families (76)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1992003701A1 (en) 1990-08-17 1992-03-05 Spatial Positioning Systems, Inc. Spatial positioning system
FR2683336B1 (fr) 1991-11-06 1996-10-31 Laserdot Dispositif de guidage asservi sur faisceau laser pour une machine de travaux publics.
DE59205359D1 (de) * 1992-04-21 1996-03-28 Pietzsch Ibp Gmbh Einrichtung zum Führen von Fahrzeugen
ZA948824B (en) 1993-12-08 1995-07-11 Caterpillar Inc Method and apparatus for operating geography altering machinery relative to a work site
US5519620A (en) 1994-02-18 1996-05-21 Trimble Navigation Limited Centimeter accurate global positioning system receiver for on-the-fly real-time kinematic measurement and control
US5838277A (en) 1994-05-20 1998-11-17 Trimble Navigation Limited GPS-based controller module
US5549412A (en) 1995-05-24 1996-08-27 Blaw-Knox Construction Equipment Corporation Position referencing, measuring and paving method and apparatus for a profiler and paver
US5612864A (en) 1995-06-20 1997-03-18 Caterpillar Inc. Apparatus and method for determining the position of a work implement
KR100256620B1 (ko) 1995-10-30 2000-05-15 모리 하루오 네비게이션장치
US6113309A (en) 1996-08-20 2000-09-05 Hollon; Edmund D. Uniform compaction of asphalt concrete
US6287048B1 (en) 1996-08-20 2001-09-11 Edmund D. Hollon Uniform compaction of asphalt concrete
DE19647150C2 (de) 1996-11-14 2001-02-01 Moba Mobile Automation Gmbh Vorrichtung und Verfahren zum Steuern der Einbauhöhe eines Straßenfertigers
US6047227A (en) 1996-11-19 2000-04-04 Caterpillar Inc. Method and apparatus for operating geography altering machinery relative to a work site
US5929807A (en) 1997-03-07 1999-07-27 Trimble Navigation Limited Method and apparatus for precision location of GPS survey tilt pole
DE19756676C1 (de) 1997-12-19 1999-06-02 Wirtgen Gmbh Verfahren und Vorrichtung zum Abfräsen von Verkehrsflächen
GB2333862B (en) 1998-02-02 2002-01-09 Caterpillar Paving Prod Method and apparatus for controllably avoiding an obstruction to a cold planer
US6140957A (en) 1998-03-12 2000-10-31 Trimble Navigation Limited Method and apparatus for navigation guidance
US6088644A (en) 1998-08-12 2000-07-11 Caterpillar Inc. Method and apparatus for determining a path to be traversed by a mobile machine
US7399139B2 (en) 1998-10-27 2008-07-15 Somero Enterprises, Inc. Apparatus and method for three-dimensional contouring
US8478492B2 (en) * 1998-11-27 2013-07-02 Caterpillar Trimble Control Technologies, Inc. Method and system for performing non-contact based determination of the position of an implement
US6074693A (en) 1999-02-22 2000-06-13 Trimble Navigation Limited Global positioning system controlled paint sprayer
US6425186B1 (en) 1999-03-12 2002-07-30 Michael L. Oliver Apparatus and method of surveying
US6191732B1 (en) 1999-05-25 2001-02-20 Carlson Software Real-time surveying/earth moving system
DE29918747U1 (de) 1999-10-25 2000-02-24 MOBA - Mobile Automation GmbH, 65604 Elz Vorrichtung zum Steuern eines Straßenfertigers
DE19957048C1 (de) 1999-11-26 2001-08-09 Wirtgen Gmbh Gleitschalungsfertiger
CN1134571C (zh) * 1999-12-02 2004-01-14 何永辉 铺路机路面平直度测量控制系统
ATE279584T1 (de) 2000-01-19 2004-10-15 Joseph Voegele Ag Verfahren zum steuern einer baumaschine bzw. eines strassenfertigers und strassenfertiger
US7183942B2 (en) * 2000-01-26 2007-02-27 Origin Technologies Limited Speed trap detection and warning system
DE60140440D1 (de) 2000-05-05 2009-12-24 Robert A Hasara Lasergesteuerte baumaschine
DE10060903C2 (de) 2000-12-07 2002-10-31 Moba Mobile Automation Gmbh Laser-Höhenregeleinrichtung für eine Baumaschine
US6655465B2 (en) 2001-03-16 2003-12-02 David S. Carlson Blade control apparatuses and methods for an earth-moving machine
AUPR396501A0 (en) 2001-03-26 2001-04-26 Edgeroi Pty Ltd Ground marking apparatus
US6769836B2 (en) 2002-04-11 2004-08-03 Enviro-Pave, Inc. Hot-in-place asphalt recycling machine and process
EP2575271B1 (en) 2002-09-23 2014-09-10 Topcon GPS LLC Position Estimation Using a Network of Global-Positioning Receivers
US8271194B2 (en) 2004-03-19 2012-09-18 Hemisphere Gps Llc Method and system using GNSS phase measurements for relative positioning
US6907336B2 (en) 2003-03-31 2005-06-14 Deere & Company Method and system for efficiently traversing an area with a work vehicle
DE10317160A1 (de) 2003-04-14 2004-11-18 Wacker Construction Equipment Ag System und Verfahren zur automatisierten Bodenverdichtung
US7443167B2 (en) 2003-08-28 2008-10-28 Science Applications International Corporation Interleaved magnetometry and pulsed electromagnetic detection of underground objects
US7002513B2 (en) 2004-03-26 2006-02-21 Topcon Gps, Llc Estimation and resolution of carrier wave ambiguities in a position navigation system
US7256710B2 (en) * 2004-06-30 2007-08-14 The Boeing Company Methods and systems for graphically displaying sources for and natures of aircraft flight control instructions
DE102004040136B4 (de) 2004-08-19 2008-05-08 Abg Allgemeine Baumaschinen-Gesellschaft Mbh Gerät zum Abfräsen von Verkehrsflächen
US7437221B2 (en) * 2004-12-16 2008-10-14 Raytheon Company Interactive device for legacy cockpit environments
EP1672122A1 (de) 2004-12-17 2006-06-21 Leica Geosystems AG Verfahren und Vorrichtung vom Kontrollieren einer Strassenbearbeitungsmaschine
US7363154B2 (en) 2005-10-12 2008-04-22 Trimble Navigation Limited Method and system for determining the path of a mobile machine
US7856302B2 (en) 2005-12-23 2010-12-21 Caterpillar Inc Work machine with transition region control system
US20070179702A1 (en) * 2006-01-27 2007-08-02 Garmin Ltd., A Cayman Islands Corporation Combined receiver and power adapter
FR2898196B1 (fr) * 2006-03-01 2008-04-25 Eurocopter France Procede et dispositif de positionnement hybride
DE102006020293B4 (de) 2006-04-27 2013-07-11 Wirtgen Gmbh Straßenbaumaschine, Nivelliereinrichtung sowie Verfahren zum Regeln der Frästiefe oder Fräsneigung bei einer Straßenbaumaschine
DE102006024123B4 (de) 2006-05-22 2010-02-25 Wirtgen Gmbh Selbstfahrende Baumaschine, sowie Verfahren zum Bearbeiten von Bodenoberflächen
US7617061B2 (en) 2006-11-03 2009-11-10 Topcon Positioning Systems, Inc. Method and apparatus for accurately determining height coordinates in a satellite/laser positioning system
DE102006062129B4 (de) 2006-12-22 2010-08-05 Wirtgen Gmbh Straßenbaumaschine sowie Verfahren zur Messung der Frästiefe
US8068962B2 (en) 2007-04-05 2011-11-29 Power Curbers, Inc. 3D control system for construction machines
US7717521B2 (en) 2007-07-09 2010-05-18 Hall David R Metal detector for an asphalt milling machine
JP4954291B2 (ja) * 2007-11-12 2012-06-13 三菱電機株式会社 デジタル放送受信機
US8018376B2 (en) 2008-04-08 2011-09-13 Hemisphere Gps Llc GNSS-based mobile communication system and method
US7946787B2 (en) 2008-06-27 2011-05-24 Caterpillar Inc. Paving system and method
US8401744B2 (en) 2008-07-22 2013-03-19 Trimble Navigation Limited System and method for configuring a guidance controller
US20120001638A1 (en) 2010-06-30 2012-01-05 Hall David R Assembly and Method for Identifying a Ferrous Material
US8174437B2 (en) 2009-07-29 2012-05-08 Hemisphere Gps Llc System and method for augmenting DGNSS with internally-generated differential correction
CN101718072B (zh) * 2009-12-04 2011-06-08 中国人民解放军国防科学技术大学 基于摄像测量的摊铺机自动找平系统及方法
DE102009059106A1 (de) 2009-12-18 2011-06-22 Wirtgen GmbH, 53578 Selbstfahrende Baumaschine und Verfahren zur Steuerung einer selbstfahrenden Baumaschine
PL2366830T3 (pl) 2010-03-18 2016-11-30 System i sposób dla nanoszenia nawierzchni drogowej
DE102010014695A1 (de) 2010-04-12 2011-10-13 Dynapac Gmbh Verfahren zum Abfräsen von Oberflächenbelägen, vorzugsweise Straßenbelägen
US8314608B2 (en) 2010-06-30 2012-11-20 Hall David R Method of determining distance to a ferrous material
KR101169985B1 (ko) * 2010-08-16 2012-08-01 (주)우리로드솔루션 도로표시 제거장치
DE102010048185B4 (de) * 2010-10-13 2021-10-28 Wirtgen Gmbh Selbstfahrende Baumaschine
US8498788B2 (en) 2010-10-26 2013-07-30 Deere & Company Method and system for determining a planned path of a vehicle
CN201865031U (zh) * 2010-11-25 2011-06-15 湖南高翔重工科技有限公司 多用途筑路车
US8794867B2 (en) 2011-05-26 2014-08-05 Trimble Navigation Limited Asphalt milling machine control and method
US8738304B2 (en) * 2011-08-02 2014-05-27 David R. Hall System for acquiring data from a component
DE102013006464B4 (de) 2012-04-16 2020-03-19 Bomag Gmbh Baumaschine mit einer Manövriereinrichtung, Verfahren zur Erleichterung des Manövrierens einer Baumaschine und Manövriereinrichtung für eine Baumaschine
US8989968B2 (en) * 2012-10-12 2015-03-24 Wirtgen Gmbh Self-propelled civil engineering machine system with field rover
US9096977B2 (en) 2013-05-23 2015-08-04 Wirtgen Gmbh Milling machine with location indicator system
CN103345225A (zh) * 2013-07-01 2013-10-09 广东惠利普路桥信息工程有限公司 摊铺过程中的远程布氏密实度图像分析与测量系统
SE537279C2 (sv) * 2013-07-12 2015-03-24 BAE Systems Hägglunds AB System och förfarande för behandling av taktisk informationhos stridsfordon
DE102014012836B4 (de) * 2014-08-28 2018-09-13 Wirtgen Gmbh Selbstfahrende Baumaschine und Verfahren zur Visualisierung des Bearbeitungsumfeldes einer sich im Gelände bewegenden Baumaschine

Also Published As

Publication number Publication date
EP2990532A1 (de) 2016-03-02
US9719217B2 (en) 2017-08-01
DE102014012836A1 (de) 2016-03-03
DE102014012836B4 (de) 2018-09-13
US20160060825A1 (en) 2016-03-03
CN105386397B (zh) 2017-09-12
CN205024576U (zh) 2016-02-10
CN105386397A (zh) 2016-03-09

Similar Documents

Publication Publication Date Title
EP2990532B1 (de) Selbstfahrende baumaschine und verfahren zur visualisierung des bearbeitungsumfeldes einer sich im gelände bewegenden baumaschine
EP2990533B1 (de) Selbstfahrende baumaschine und verfahren zum steuern einer selbstfahrenden baumaschine
EP2990534B1 (de) Selbstfahrende baumaschine und verfahren zur steuerung einer selbstfahrenden baumaschine
EP3502821B1 (de) Selbstfahrende baumaschine und verfahren zum steuern einer selbstfahrenden baumaschine
DE102017221134A1 (de) Verfahren und Vorrichtung zum Betreiben eines mobilen Systems
DE102018207706A1 (de) Umgebungsabtastung unter Verwendung einer Arbeitsmaschine mit einer Kamera
EP3162964B1 (de) Vorrichtung und verfahren zur bereitstellung einer fahrzeugumgebungsansicht für ein fahrzeug
DE102009059106A1 (de) Selbstfahrende Baumaschine und Verfahren zur Steuerung einer selbstfahrenden Baumaschine
EP2568310A2 (de) Verfahren, System sowie Vorrichtung zur Lokalisation eines Fahrzeugs relativ zu einem vordefinierten Bezugssystem
DE102019112413A1 (de) Verfahren und vorrichtung zur multi-sensor-datenfusion für automatisierte und autonome fahrzeuge
EP1673589A1 (de) Verfahren und vorrichtung zur bestimmung der aktualposition eines geodätischen instrumentes
EP2963631B1 (de) Verfahren zum ermitteln eines parkplatzes aus einer anzahl von messpunkten
DE102014205640B4 (de) Vermessung mittels mobilem Gerät
DE102020206190A1 (de) System und verfahren für die konfiguration einer arbeitsstätten warnzone.
DE102007060056A1 (de) Ansteuersystem und Ansteuerverfahren für ein Roboterfahrzeug sowie Roboterfahrzeug
EP2983010B1 (de) Verfahren zur korrektur in einem speicher eines arbeitsrechners gespeicherter positionsdaten
EP2660501B1 (de) Verfahren zur Visualisierung der Position eines Fahrzeugs zur Befahrung eines Kanals
DE102019129831A1 (de) Automatisches kupplungssystem mit auswahl des betreffenden anhängers aus mehreren identifizierten anhängern
DE102018113015A1 (de) Autonome, mobile Arbeitmaschine
EP3435030A1 (de) Verfahren zur erstellung eines 3d-modells von einem objekt
DE102019205484A1 (de) Verfahren und Vorrichtung zum Erstellen eines Höhenprofils für einen von einem Fahrzeug zu befahrenden Bereich
DE102022115469A1 (de) System und verfahren zur markierung einer begrenzung während der definition eines autonomen arbeitsbereichs
DE102020006851A1 (de) Vorrichtung zur stückweisen Höhenmessung
DE102019202299B4 (de) Verfahren zur Online-Kalibrierung und Kalibriereinrichtung
EP2837909A2 (de) Verfahren und Anordnung zur Kartierung eines landwirtschaftlichen Feldes

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

17P Request for examination filed

Effective date: 20160426

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20160902

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 867972

Country of ref document: AT

Kind code of ref document: T

Effective date: 20170315

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 502015000587

Country of ref document: DE

REG Reference to a national code

Ref country code: SE

Ref legal event code: TRGR

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20170215

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170516

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170515

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170215

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170215

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170215

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 3

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170215

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170515

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170615

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170215

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170215

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170215

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170215

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170215

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170215

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170215

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 502015000587

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170215

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170215

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170215

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20171116

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170215

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170215

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20170831

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170812

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170812

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 4

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170831

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170215

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180831

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180831

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20150812

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170215

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170215

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170215

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170215

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170615

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 867972

Country of ref document: AT

Kind code of ref document: T

Effective date: 20200812

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200812

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230525

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20230831

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: SE

Payment date: 20230823

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240819

Year of fee payment: 10

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20240822

Year of fee payment: 10

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20240821

Year of fee payment: 10